JP2017118319A - Video retrieval system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately extract only a scene that is important for a supervising job, from among video images such as monitor video images that are recorded for a long time.SOLUTION: A video retrieval system analyzes video images and extracts a video target scene that becomes an extraction target. The video retrieval system comprises a retrieval start point determination part 106 and a work video phase identification part 108. Based on work information 112 inputted from the outside, the retrieval start point determination part 106 and the work video phase identification part 108 sets a start point video frame to be a retrieval start point of a video image to be the extraction target from among video images captured by a video imaging device 101 that images equipment of a work target. With the set start point video frame defined as a reference, video images captured by the video imaging device 101 are retrieved and the video target scene is identified.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a video search system, and more particularly to a technique effective for searching important scenes in video shot by a surveillance camera or the like.

  With the widespread use of imaging devices that can be connected to a network such as a Web camera, it has become possible to construct a large-scale monitoring system over a wide area using an IP (Internet Protocol) network or the like.

  In such a monitoring system, a large amount of monitoring images can be acquired from cameras installed at a plurality of locations, and multi-faceted monitoring is possible. However, there is a limit to the amount of video that can be visually confirmed by a single observer.

  For this reason, simply increasing the number of cameras causes a problem such as an oversight of an important scene by a supervisor. Therefore, in order to realize efficient wide area monitoring, a technique for confirming a large amount of video in a short time is necessary.

  As this type of video confirmation technique, there is a technique for browsing a monitoring video for a long time in a short time by browsing a portion with little change in the monitoring video, that is, a low-importance portion at high speed (for example, see Patent Document 1). ).

  In this technique, for example, a part with a large change in video and a part with a small change are distinguished, a part with a small change is played back at high speed, and a part with a large change is played back at a low speed or a normal speed.

  As a technique for efficiently confirming work over a long period of time as in a construction site, for example, Patent Document 2 is known. This patent document 2 is for taking an image only at a time set by a timer means.

  Furthermore, the above-mentioned Patent Document 2 discloses a technique that allows a long-term work progress to be confirmed in a short time by photographing only highly important images. In this method, the movement of a person or an object is detected by an optical system sensor, and an image is taken only when the movement is detected.

JP 2007-306322 A JP 2002-24330 A

  In the techniques of Patent Document 1 and Patent Document 2 described above, it is determined whether or not the scene is important using only the amount of change in the image and the movement of the moving object at the work site. Therefore, depending on the environment used, there is a possibility that a scene with low importance is erroneously determined to be important. Or, conversely, a scene with high importance may be erroneously determined to be low in importance.

  For example, a supervisor who monitors work at a plant facility, construction site, etc., is proceeding according to a plan prepared in advance by the site worker, or reports errors or false reports described in the work report submitted by the worker. There are cases where surveillance video is used to check whether there is any.

  In such an application, it is necessary to determine only an active scene among the various scenes shown in the monitoring video as an important scene. However, the above-described technology reacts to movements other than work.

  Therefore, for example, a scene that is less important for the supervisor, such as a scene in which an operator walks toward a rest area, may be erroneously determined as an important scene. In order to further improve the efficiency of monitoring work, a scene extraction technique that considers information other than movement is required.

  An object of the present invention is to provide a technique capable of accurately extracting only a scene important for a supervisory work from a video recorded for a long time such as a monitoring video.

  The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.

  Of the inventions disclosed in the present application, the outline of typical ones will be briefly described as follows.

  That is, a typical video search system has a video search specifying unit. The video search specifying unit sets and sets a start video frame that is a search start point of a video to be extracted from a video shot by a video shooting device that shoots a target device based on search information input from the outside. A video target scene is specified by searching for a video shot by the video shooting device using the starting video frame as a starting point.

  The video search specifying unit includes a search start point determining unit and a video section specifying unit. Based on the search information, the search start point determination unit generates search start point data including a video shooting apparatus that has shot the target device and the work date and time of the device. Based on the search start data generated by the search start point determination unit, the video section specifying unit sets a start video frame that is a search start point of the video to be extracted from the video shot by the video shooting device that captures the work target device. Then, starting from the set start video frame, the video taken by the video shooting device is analyzed and searched to specify the video target scene.

  In particular, the search start data generated by the search start point determination unit has the work date and time performed on the device, and the video section specifying unit sets a frame image of the date and time that matches the work date and time as the start video frame.

  Among the inventions disclosed in the present application, effects obtained by typical ones will be briefly described as follows.

  (1) A scene necessary for supervision work can be extracted with high accuracy in a short time.

  (2) According to the above (1), the efficiency of the supervisory work can be improved.

2 is an explanatory diagram illustrating an example of a configuration in a video search system according to Embodiment 1. FIG. It is explanatory drawing which shows the other structural example in the video | video search system of FIG. It is a flowchart which shows an example of operation | movement in the video | video search system of FIG. It is explanatory drawing of the work information stored in the work information input device which the video search system of FIG. 2 has. It is explanatory drawing which shows an example of a data structure in the apparatus imaging | photography data stored in the memory | storage device which the video search system of FIG. 2 has. It is explanatory drawing which shows an example of a structure of the search origin data produced in the search origin determination process which is a process of step S302 of FIG. It is explanatory drawing which shows an example in the structure of the video data accumulate | stored in the video storage part which the video search system of FIG. 2 has. It is explanatory drawing which shows the outline | summary of the work video area identification process by the work video area identification part which the video search system of FIG. 2 has. It is a flowchart which shows an example of the work video area specific process by the work video area specific | specification part which the video search system of FIG. 2 has. It is explanatory drawing which shows an example of the display content displayed on a display apparatus by the process of step S304 of FIG. FIG. 10 is an explanatory diagram illustrating an example of a configuration in a video search system according to a second embodiment. It is explanatory drawing which shows an example of a structure in the work list which the work information input part which the video search system of FIG. 11 has acquires. It is explanatory drawing which shows an example of the display content displayed on a display apparatus when the difference by the work content evaluation part which the video search system of FIG. 11 has is large. 10 is an explanatory diagram illustrating an example of a configuration of a video search system according to Embodiment 3. FIG. It is explanatory drawing which shows an example in the structure of the video data accumulate | stored in the video storage part which the video search system of FIG. 14 has.

  In the following embodiments, when it is necessary for the sake of convenience, the description will be divided into a plurality of sections or embodiments. However, unless otherwise specified, they are not irrelevant to each other. There are some or all of the modifications, details, supplementary explanations, and the like.

  Further, in the following embodiments, when referring to the number of elements (including the number, numerical value, quantity, range, etc.), especially when clearly indicated and when clearly limited to a specific number in principle, etc. Except, it is not limited to the specific number, and may be more or less than the specific number.

  Further, in the following embodiments, the constituent elements (including element steps and the like) are not necessarily indispensable unless otherwise specified and apparently essential in principle. Needless to say.

  Similarly, in the following embodiments, when referring to the shape, positional relationship, etc. of components, etc., the shape of the component is substantially the case unless it is clearly specified and the case where it is clearly not apparent in principle. And the like are included. The same applies to the above numerical values and ranges.

  In all the drawings for explaining the embodiments, the same members are denoted by the same reference symbols in principle, and the repeated explanation thereof is omitted.

(Embodiment 1)
<Overview>
The representative video search system of the embodiment, when the work information 112 is input, searches only a video related to the work information 112 from among a large number of videos shot by the video shooting device 101 in the facility. In this system, the searched video is linked to the work information 112 and presented on the display device 111.

<Example of video search system configuration>
Hereinafter, embodiments will be described in detail.

  FIG. 1 is an explanatory diagram showing an example of the configuration of the video search system according to the first embodiment.

  As shown in FIG. 1, the video search system has a configuration including a video shooting device 101, a work information input device 104, a display device 111, and a video search device 100. The video imaging device 101 is, for example, a network camera or a digital camera, and is provided at various locations in the facility, for example, and captures video in the facility.

  The work information input device 104 holds work information 112 shown in FIG. The work information input device 104 may be, for example, an external storage memory such as a USB (Universal Serial Bus) memory, or may be an input device such as a keyboard or a mouse. The work information 112 is information such as the content and time of work performed in a facility, for example. The display device 111 is a liquid crystal display, for example, and presents a processing result.

  The video search device 100 includes a video input unit 102, a video storage unit 103, a work information input unit 105, a search start point determination unit 106, a device shooting data storage unit 107, a work video section specifying unit 108, a work time zone information storage unit 109, And a display content generation unit 110.

  The video input unit 102 receives video data from the video shooting device 101 and converts it into a frame image, in other words, a still image. Then, the camera ID 113, which is a number that uniquely identifies the video photographing apparatus 101 shown in FIG. 5, is associated with the video photographing date 701 shown in FIG.

  The work information input unit 105 receives the work information 112 from the work information input device 104, selects only information necessary for the subsequent processing, and outputs it to the search start point determination unit 106. Based on the data input from the work information input unit 105, the search start point determination unit 106 obtains the time when the worker is highly likely to appear and the camera ID 113 showing the work.

  The work video section specifying unit 108 analyzes the video of the video storage unit 103 and specifies the work start time and end time. The search start point determination unit 106 and the work video section specifying unit 108 constitute a video search specifying unit. Then, the specified time zone is output to the work time zone information storage unit 109. The display content generation unit 110 generates an output screen in which the work information 112 and the work time period video are associated with each other.

<Application example of video search system>
Next, an application example of the video search system in FIG. 1 will be described.

  FIG. 2 is an explanatory diagram showing another configuration example of the video search system of FIG. FIG. 2 shows an example in which the video search device 100 included in the video search system of FIG. 1 is configured by applying a personal computer which is a computer. The video imaging device 101, the work information input device 104, and the display device 111 in FIG. 2 are the same as those in FIG.

  In FIG. 2, the video search apparatus 100 is a computer such as a general personal computer as described above, and includes a processor 251, a main memory 252, a video input interface (I / F) 253, a document input interface 255, an output interface 256, And a storage device 257. These are configured to be connected to each other by a bus or the like.

  The processor 251 executes a program stored in the main memory 252. The main memory 252 includes a semiconductor memory exemplified by a flash memory, for example, and stores a program executed by the processor 251 and data referred to by the processor 251.

  Specifically, at least a part of the program and data stored in the storage device 257 is copied to the main memory 252 as necessary. The video input interface 253 is an interface for acquiring video, and is connected to the video shooting device 101.

  The document input interface 255 is an interface for acquiring work information, and is connected to the work information input device 104. The output interface 256 is an interface that outputs a screen to be displayed on the display device 111.

  The storage device 257 is a non-volatile storage device such as a hard disk device (HDD) or a flash memory. The storage device 257 in FIG. 2 has a program format that executes at least the functions of the video input unit 102, the work information input unit 105, the search start point determination unit 106, the work video section identification unit 108, and the display content generation unit 110 in FIG. Software is stored.

  The storage device 257 stores storage areas for storing the data stored in the video storage unit 103, the data stored in the work time zone information storage unit 109, and the data stored in the device shooting data storage unit 107, respectively. Respectively. In particular, the device photographing data storage unit 107 in the storage device 257 stores device photographing data 115 shown in FIG.

  Programs that execute the functions of the video input unit 102, the work information input unit 105, the search start point determination unit 106, the work video section specification unit 108, and the display content generation unit 110 are programs executed by the processor 251. In the following description, processing executed by each of these functions is executed by the processor 251.

<Operation example by video search system>
Next, the operation of the video search system in FIG. 2 will be described.

  FIG. 3 is a flowchart showing an example of operation in the video search system of FIG. In FIG. 3, for example, a USB memory is assumed as the work information input device 104.

  First, when the work supervisor or field worker connects the work information input device 104 to the video search device 100, work information input processing by the work information input unit 105 is executed (step S301).

  In the process of step S301, data used in the process of step S302, which is the next step, is acquired from the work information 112 stored in the work information input device 104.

<Example of work information>
FIG. 4 is an explanatory diagram of work information stored in the work information input device 104 included in the video search system of FIG. FIG. 4A shows an example of work information, and FIG. 4B shows a configuration example of the work list 114 acquired by the work information input unit 105.

  The work information 112 as search information is a document describing work contents and work man-hours such as a work plan, a work manual, or a work report, and is work report data including electronic data, for example.

  This work information 112 is created in accordance with a predetermined format, and includes item number 201, device ID 202, device name 203, work time 204, and result 401 as shown in FIG. 4A. There is a table.

  In the upper part of the table, a work day 205 and a worker name 206 are shown. In addition, the number of workers may be described.

  The device ID 202 is information that uniquely identifies a work target device. The device name 203 indicates the name of the device to be worked. The work time 204 is a time when the work is performed. A result 401 is a photograph taken at the time of work. For example, in the case of an operation of reading a meter of a certain device, photograph data of the read meter is attached.

  In the work information input process in step S301, a work list 114 is generated. The work list 114 includes an item number 201, a device ID 202, and a work date / time 207, and is acquired from the work information illustrated in FIG. Here, the work date and time 207 to be acquired are the work date 205 and the work time 204 in FIG. Records are added to the work list 114 by the number of works.

  As for the work date and time, as described above, the work time 204 and the work date 205 described in the document of the work information 112 may be adopted as they are, or a photograph is attached to the result 401 in the work information 112. In such a case, data on the photographing date and time of the photograph may be acquired.

  In this case, the shooting date and time of the photograph is acquired from an Exif (Exchangeable image file format) file that is shooting information added by the digital camera. As a result, the correct work date and time can be obtained even if the worker incorrectly enters the work date 205 and the work time 204, omission of entry, or when the work information 112 does not have a column for describing the work time. Can do.

  Subsequently, a search start point determination process by the search start point determination unit 106 is executed (step S302). In the process of step S302, search starting point data 610 shown in FIG. 6 is generated based on the work list 114 and the device photographing data 115 created in the process of step S301. The search start data 610 includes a video photographing apparatus 101 that is highly likely to be viewed by a worker, a list of work dates, and the like.

  Here, the device shooting data 115 is data created by a supervisor or worker visually confirming a monitoring image before the operation of the image search device 100, and a list of devices in the facility and image shooting capable of monitoring the device. The correspondence relationship with the apparatus 101 is shown.

<Configuration example of device shooting data>
FIG. 5 is an explanatory diagram showing an example of a data configuration in the device shooting data 115 stored in the storage device 257 included in the video search system of FIG.

  The data structure of the device shooting data 115 includes a device ID 202, a camera ID 113, and position coordinates 501. When there are a plurality of video photographing apparatuses 101 that can shoot for one work target device, one video photographing device 101 at a position closer to the work target device is selected and the camera of the selected video photographing device 101 is selected. ID 113 is stored. Alternatively, a plurality of camera IDs 113 may be stored.

  In addition, when the area where the device appears in the monitoring image can be specified, the position coordinate 501 is additionally written in addition to the camera ID 113. The position coordinate 501 indicates the position of the monitoring device on the image plane displayed on the display device 111.

  Thereby, since the processing range of the person detection performed in the search process which is the process of step S303 in FIG. 3 described later can be narrowed, the processing time in the search process can be shortened.

  In addition, when the video photographing apparatus 101 having the PTZ (pan, tilt, zoom) function is used, there is a case where the work target device cannot be photographed depending on the setting value of the PTZ function. Therefore, a PTZ value that can shoot the work target device may be confirmed in advance, and the value may be included in the device shooting data 115.

  The device shooting data 115 may be generated once when the video search device 100 is used for the first time, and is data that does not need to be updated unless the installation positions of the video shooting device 101 and the device fluctuate.

<Configuration example of search origin data>
FIG. 6 is an explanatory diagram showing an example of the configuration of the search start point data 610 created by the search start point determination process that is the process of step S302 of FIG.

  As described above, the search starting point data 610 is composed of the item number 601, the work target device ID 602, the work date and time 604, and the camera ID 605 of the video photographing apparatus 101 that can photograph the work scenery.

  Among these, the device ID 602 and the work date / time 604 are obtained by acquiring the device ID 202 and the work date / time 207 from the work list 114 of FIG. 4B. The camera ID 605 is obtained by referring to the device shooting data 115 and obtaining a camera ID corresponding to the device ID 602.

  Through the above-described operation, a list of the times when the worker is highly likely to appear and the video photographing apparatus 101 is generated.

  Subsequently, in FIG. 3, a work video section specifying process is performed by the work video section specifying unit 108 (step S <b> 303). This work video section specifying process is a section specifying process for specifying a video in which a work scene is shown from videos stored in the video storage unit 103. At that time, the scene identification accuracy is improved by using the camera ID 602 and the work date and time 604 obtained in the process of step S302.

<Example configuration of video data>
FIG. 7 is an explanatory diagram showing an example of the configuration of the video data stored in the video storage unit 103 included in the video search system of FIG.

  As shown in FIG. 7, the configuration of the video data stored in the video storage unit 103 is such that the frame images acquired by the video input unit 102 are organized and stored for each video shooting device 101 and for each shooting date. . The frame image 702 is provided with shooting time data, so that an image at a specific time can be easily extracted.

<Overview of work video section identification processing>
Next, an outline of the work video section specifying process will be described with reference to FIG.

  FIG. 8 is an explanatory diagram showing an outline of work video section specifying processing by the work video section specifying unit 108 included in the video search system of FIG.

  A rectangle 801 illustrated in FIG. 8 schematically represents a continuous frame image by the frame image 702 of the video data stored in the video storage unit 103. Among these, a square indicated by hatching indicates an image in which a worker is shown, and a square indicated by white outline indicates an image in which no worker is shown.

  An image 805, which is one of the squares indicated by hatching, is a starting image taken at the time of the work date 604 stored in the search starting data 610 shown in FIG. In the work video section specifying process, a frame 702 taken at the same time zone as the image 805 that is a starting image is analyzed to obtain a work section 803 that is a section in which a worker is shown.

  As a result, by searching only the video near the frame image of the image 805 that is the starting image, it is possible to prevent a scene unrelated to work from being erroneously extracted and to perform high-accuracy video search. it can.

  As shown in FIGS. 8A and 8B, the positional relationship between the image 805 and the work section 803 has two cases. First, as shown in FIG. 8A, the work section 803 includes an image 805, that is, a start frame that becomes a start video frame. The second case is a case where the starting frame is not included in the work section 803 as shown in FIG.

  In the case shown in FIG. 8A, the work section 803 can be specified by searching only a narrow range near the image 805 of the starting image. On the other hand, in the case shown in FIG. 8B, there are sections in which workers are shown before and after the starting image 805, that is, sections that are candidates for the work sections.

  In these cases, a section closer to the image 805 that is the starting frame is selected, and this is set as a work section 803. For example, in the case shown in FIG. 8B, there are sections in which workers are shown in the work section candidate 1 and the work section candidate 2, but the work section closer to the image 805 is the work section. Therefore, the work section candidate 2 is selected as a work section 803.

<Example of work video section identification processing>
Next, the work video section specifying process will be described in detail with reference to FIG.

  FIG. 9 is a flowchart showing an example of work video section specifying processing by the work video section specifying unit 108 included in the video search system of FIG.

First, the starting frame F ₀ is determined (step S1021). As described above, the start point frame F _0, of the frame image 702 stored in the video storage unit 103, frame tied to the camera ID602 and the working time 604, i.e., to set the image.

Then, the person detection process is executed on the image of the starting frame F ₀ (step S1022). This person detection process can use an existing person detection process. For example, the difference between the pixel values of the background image captured in the absence of a person in advance and the starting frame image is taken, and a person is shown when the area where the difference is greater than or equal to a predetermined threshold value. Can be used. Or the object detection technique based on the collation result with a some template image etc. which are described in Unexamined-Japanese-Patent No. 2013-92955 can be utilized.

At this process, if it is determined that the person is reflected in the origin frame F _0, the origin frame F ₀ is determined to be within the work section 803 (step S1023). This can be regarded as, for example, the case of FIG. Thereafter, a process for searching for frames at both ends of the work section is executed (step S1010).

On the other hand, if it is determined that the person is not reflected (step S1023), the origin frame F ₀ is considered to be in a position other than work section 803. This can be regarded as the case of FIG. In that case, first, an approximate position of the work section 803 is obtained (S1011), and then both ends of the work section 803 are obtained (S1012).

  Next, processing for searching for frames at both ends of the work section, which is processing in step S1010, will be described.

First, the process starts by moving the target frame F _i subject to image analysis to F ₀ −1 which is one frame before. Against subsequent image of the frame of interest F _i subjected to a person detection process.

When a person is detected, an image one frame before is further acquired to perform person detection. This process is repeated until no person is detected in the image of the frame of interest. A frame in which no person is detected is set as a start frame F _s of the work section 803. This start frame F _s becomes a work start frame 804 showing a work start video for the device in FIG.

Subsequently, an end frame F _e of the work section 803 is obtained. The end frame F _e is a work end frame 806 that indicates a frame in which work on the device in FIG. 8 is finished. In this process, as in the case of calculating the start frame F _s , images are analyzed in order from the starting frame F _0, it is determined whether or not a person is shown, and the first frame in which no person is shown is determined as a work section 803. _Is the end frame F _e of.

  As described above, the start and end frames of the work section 803 can be specified.

  Next, the process of step S1011 in FIG. 9 will be described.

As described above, this is processing in step S1023 of FIG. 9 when it is determined that the case of FIG. 8B, that is, the starting frame F ₀ is at a position other than the work section 803.

First, locate the first frame a person is detected while moving the focus frame F _i before one frame, the detected frame and end frame F _e of the working section candidate 1. Then, the first frame in which the person is detected is searched while moving the attention frame F _i backward by one frame at a time, and this is set as the start frame F _s of the work section candidate 2.

  Thereafter, the distance from the image 805 which is the starting frame to the work section candidate 1 and the work section candidate 2 shown in FIG. 8B, in other words, the number of image frames is calculated, and a section having a small number of frames is set as the work section 803. . In this case, not the number of image frames but the time from the image 805 which is the starting frame to the work section candidate 1 and the work section candidate 2 shown in FIG.

If true work section 803 was in the opposite direction, as a starting point the end frame F _e of the working section candidates 1 obtained in the above, performs person detection processing while moving before the target frame by one frame. A frame in which a person can no longer be detected is set as a start frame F _s of the work section 803.

On the other hand, when it is determined that the work section 803 is in the forward direction, the person detection process is performed while starting the start frame F _s of the work section candidate 2 and moving the target frame backward by one frame. A frame in which a person cannot be detected is defined as an end frame F _e of the work section 803. As described above, the start and end frames of the work section can be specified.

  Now, the description returns to the flowchart of FIG.

  When the work video section identification process, which is the process of step S303, is finished, the display content generation unit 110 generates the display content (step S304) and outputs it to the display device 111. This is a process of associating the work information 112 acquired in the process of step S301 with the video during the work searched in the process of step S303.

<Display example of display device>
FIG. 10 is an explanatory diagram showing an example of display contents displayed on the display device by the process of step S304 of FIG.

  As shown in FIG. 10, the data generated by the processing in step S304 includes an item number 201, a device ID 202, a device name 203, an inspection date 901, and an image 902. For the item number 201, the device ID 202, and the device name 203, the data described in the work information 112 shown in FIG.

  Although the information described in the work information 112 can be used as it is for the inspection date 901, as shown in FIG. 8B, the time described in the work information 112 and the person in the monitoring video are displayed. When a difference occurs in the detected time, the time obtained from the monitoring video may be written together.

  As a result, the supervisor can notice that there is a difference between the contents of the work information 112 and the monitoring video, and the work can be preferentially viewed. As a result, it is possible to quickly take subsequent measures such as investigating the cause of the difference.

  As described above, it is possible to accurately extract scenes necessary for the supervision work from a large number of monitoring videos in a short time. As a result, the efficiency of supervisory work by supervisors and the like can be improved.

(Embodiment 2)
<Overview>
In the second embodiment, a video search system that evaluates the difference between the content of the work information 112 and the work content of the monitoring video and outputs the evaluation result to the supervisor screen will be described.

<Example of video search system configuration>
FIG. 11 is an explanatory diagram showing an example of the configuration of the video search system according to the second embodiment.

  The video search system of FIG. 11 is different from the video search system of FIG. 1 of the first embodiment in that a work content evaluation unit 1101 is newly provided. Other configurations are the same as those of the first embodiment shown in FIG.

  FIG. 12 is an explanatory diagram showing an example of a configuration in the work list 114 acquired by the work information input unit 105 included in the video search system of FIG.

<Example of work list>
The work list 114 shown in FIG. 12 has a configuration in which the number of workers 1201 indicating the number of workers performing the work is added to the contents of the work list 114 shown in FIG.

  The work video section specifying unit 108 detects a person in the frame image 702 in the video storage unit 103 in addition to the work video section specifying process for specifying the start time and end time of the work described in the first embodiment. It has a function to execute processing. In this person detection process, it is possible to obtain the number of persons shown in the image.

  In the process of specifying the work video section, since the start and end frames of each work are obtained from the video as described in the first embodiment, the work start time, the end time, and the work It is also possible to calculate the time required.

<Example of processing by the work content evaluation unit>
The work content evaluation unit 1101 describes the number of workers, the work start time, the work end time, and the time required for the work obtained as a result of the video analysis by the work video section specifying unit 108, and the work list 114 shown in FIG. Compare with information.

  As a result of the comparison, a delay of more than a set time, for example, the start of work was delayed by more than 1 hour, or the work that was originally planned to be performed by two people, only one worker was shown in the monitoring video, etc. When the difference is large, the result is output as an alert to the display content generation unit 110 and displayed on the display device 111.

<Display example of display device>
FIG. 13 is an explanatory diagram illustrating an example of display content displayed on the display device 111 when the difference by the work content evaluation unit 1101 in FIG. 11 is large.

  In the display example of FIG. 13, in addition to the display example in FIG. 10 of the first embodiment, the number of workers 903 which is an item indicating the number of workers is newly added. The number of workers 903 includes the number of workers obtained as a result of video analysis when the number of workers obtained as a result of video analysis and the number of workers in the work list 114 in FIG. 12 are compared. And a message indicating that the number of workers is different.

  For example, when the number of workers shown in the work list 114 in FIG. 12 is two and the number of workers obtained as a result of video analysis is one, the number of workers 903 in FIG. "One person (with differences)" is displayed.

  In addition, the display content generation unit 110 may generate display content by performing processing such as changing the color or enlarging only the part where there is a difference so that the part where the difference occurs is particularly noticeable.

  In this way, the difference between the work content described in the work information 112 of FIG. 4A and the work content obtained by analyzing the monitoring video is evaluated, and the result is presented to the supervisor, thereby monitoring It is possible to make it easier to understand events that should be confirmed by a person. As a result, the supervisory monitoring efficiency can be improved.

In the first embodiment, in the search start point determination process in step S302 in FIG. 3, when the start frame F ₀ is determined in the process in step S1022 in FIG. An example of using it as it is is shown.

  However, when the same worker works on the same day, if the work performed earlier is delayed, the next work may also be delayed. Therefore, as a result of evaluating the difference in the work content evaluation unit 1101, if it is determined that the difference in the end time of any work is large, the possibility that the work data in the search starting data 610 in FIG. The data of the work date and time 604 of the high work is updated, and the search start point determination process in step S302 is executed again.

  As described above, it is possible to search for a more accurate scene by updating the contents of the work information 112 in FIG. 4A using the result of analyzing the video monitoring.

(Embodiment 3)
<Example of video search system configuration>
FIG. 14 is an explanatory diagram showing an example of the configuration of the video search system according to the third embodiment.

  The third embodiment has a configuration in which a person detection unit 1031 is newly provided in the configuration of FIG. 1 in the first embodiment. Other than the person detection unit 1031 is the same as in FIG.

<Processing example of person detection unit>
In FIG. 14, the frame image acquired by the video input unit 102 is output to the person detection unit 1031. The person detection unit 1031 performs the process of step S1022 in FIG. 9 of the first embodiment, that is, the person detection process for all input images, and associates the frame image 702 with the person detection result. The image is output to the video storage unit 103. Data in which the frame image 702 is associated with the person detection result is person detection data.

<Example configuration of video data>
FIG. 15 is an explanatory diagram showing an example of the configuration of video data stored in the video storage unit 103 included in the video search system of FIG.

  The configuration of the video data stored in the video storage unit 103 is newly added with a person detection result 1401 indicating the result of person detection by the person detection unit 1031 in addition to the contents shown in FIG. 7 of the first embodiment. ing.

  The person detection result 1401 may store, for example, the number of persons detected in each frame (image), or indicate whether or not a person exists as shown in FIG. You may make it do.

  Also in the third embodiment, the work video section specifying unit 108 specifies a section in which a person is shown according to the same flow shown in FIG. At this time, for the person detection process in steps S1022, S1010, and S1011 of FIG. 9, it is only necessary to refer to the person detection result 1401 stored in the video storage unit 103.

  By performing person detection in advance in the flow for storing video, the work video section specifying process shown in FIG. 3, that is, the image analysis in the process of step S303 in FIG. 3, can be omitted.

  As a result, a higher-speed video search can be realized, and the efficiency of supervisory monitoring work can be further improved.

  As mentioned above, the invention made by the present inventor has been specifically described based on the embodiment. However, the present invention is not limited to the embodiment, and various modifications can be made without departing from the scope of the invention. Needless to say.

  In addition, this invention is not limited to above-described embodiment, Various modifications are included. For example, the above-described embodiment has been described in detail for easy understanding of the present invention, and is not necessarily limited to one having all the configurations described.

  Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. . In addition, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

DESCRIPTION OF SYMBOLS 100 Image | video search device 101 Image | video imaging device 102 Image | video input part 103 Image | video accumulation | storage part 104 Work information input device 105 Work information input part 106 Search origin determination part 107 Device imaging | photography data storage part 108 Work image | video area specification part 109 Work time zone information storage part 110 Display Content Generation Unit 1101 Work Content Evaluation Unit 111 Display Device 112 Work Information 114 Work List 115 Device Shooting Data 251 Processor 252 Main Memory 253 Video Input Interface 255 Document Input Interface 255 Document Input Interface 256 Output Interface 257 Storage Device

Claims (11)

A video search system that analyzes video and extracts video target scenes to be extracted,
Based on the search information input from the outside, a starting video frame is set as a search starting point for the video to be extracted from the video shot by the video shooting device that captures the work target device, and the set starting video frame is set as the starting point. As a video search system, the video search system includes a video search specifying unit that searches for video shot by the video shooting device and specifies the video target scene. The video search system according to claim 1,
The video search specifying unit
Based on the search information, a video shooting device that images the device to be worked and a search start point determination unit that generates search start point data including the work date and time of the device;
Based on the search start point data generated by the search start point determination unit, the start point video frame that is the search start point of the video to be extracted is set and set from the video shot by the video shooting device that captures the work target device. Analyzing and searching the video captured by the video imaging device starting from the origin video frame, and identifying the video target scene,
A video search system. The video search system according to claim 2, wherein
The search starting point data generated by the search starting point determining unit has a work date and time performed on the device,
The video section specifying unit sets a frame image having a date and time that matches the work date and time as the starting video frame. The video search system according to claim 3, wherein
The search starting point determination unit generates a shooting date and time attached to the photo data included in the search information as a work date and time of the search starting point data,
The video search system, wherein the photo data is photo data taken when work on the device is completed. The video search system according to any one of claims 2 to 4,
The video section specifying unit determines whether or not a person is shown in the starting video frame, and when the person is shown in the starting video frame, the video target scene to be extracted from the starting video frame as a starting point Perform the section identification process to identify
The section specifying process searches for a first frame image in which a person is not shown in a frame image before the start video frame and sets it as a work start frame, and a person in a frame image after the start video frame A video search system that searches for a first frame image that is not shown and sets it as a work end frame, and uses the set frame images from the work start frame to the work end frame as the video target scene to be extracted. The video search system according to claim 5, wherein
When the video section specifying unit determines that a person is not shown in the start video frame, the first video frame in which the person is shown in the frame image before the start video frame and the start video frame 2nd frame images in which a person is reflected in each of the subsequent frame images are searched, and either the first image frame or the second image frame that is closer to the origin video frame is the origin video. A video search system that sets as a frame, executes the section specifying process, and specifies the video target scene to be extracted. The video search system according to claim 5 or 6,
A person detection unit for detecting a person in a frame image captured by the video imaging device;
A video storage unit for storing person detection data in which the frame image and a person detection result executed by the person detection unit are linked;
Have
The video search system, wherein the video section specifying unit refers to the person detection data stored in the video storage unit to determine whether or not a person is shown in the video target scene. The video search system according to any one of claims 5 to 7,
A work content evaluation unit for evaluating the work content for the device;
The work content evaluation unit compares the date and time of the work start frame identified by the video section identification unit with the work date and time performed on the device included in the search information, and alerts when there is a time difference greater than a set value. Video search system to output. The video search system according to any one of claims 5 to 7,
A work content evaluation unit for evaluating the work content for the device;
The video section specifying unit detects the number of persons shown in the video target scene,
The work content evaluation unit obtains the number of workers working on the device from the search information, and compares the obtained number of workers with the number of persons shown in the video target scene detected by the video section specifying unit. A video search system that outputs an alert when the number of people does not match. The video search system according to claim 9, wherein
A person detection unit that detects a person in the frame image captured by the video imaging apparatus and detects the number of persons in the frame image;
A video storage unit for storing person detection data in which the frame image and the number of persons executed by the person detection unit are associated;
Have
The video search system, wherein the video section specifying unit refers to the person detection data stored in the video storage unit and determines the number of persons appearing in the video target scene. The video search system according to any one of claims 1 to 10,
The video search system, wherein the search information is work report data for reporting a work result for the device.

Images