JP2020024669A - Detection device and control method thereof - Google Patents

Abstract

To make it possible to distinguish and detect an object according to a state of the object.SOLUTION: A detection device comprises: detecting means for detecting a plurality of parts of an object included in a frame image of a moving image to detect the object included in a video image; intrusion determining means for determining that the object has invaded a preset region with respect to the frame image; and deciding means for deciding to notify a determination result depending on whether a predetermined part is included in the detected part.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a technology for detecting an object included in an image.

  A system that detects a person entering a facility by installing a surveillance camera and analyzing images obtained by the surveillance camera for the purpose of ensuring the safety of facilities such as condominiums, buildings, school buildings, station buildings, and airports. Has been introduced. Patent Literature 1 discloses a technique for detecting a human body by cascade-connecting strong classifiers for detecting parts in detecting a human body region in an image. Patent Document 2 discloses an intruder monitoring device that determines whether a person has entered a prohibited area on an image.

US Patent Application Publication No. 2007/237378 JP-A-9-50585

  By the way, in the above-mentioned conventional technology, the detection is performed without distinguishing the state of the human body included in the video. Therefore, when an image as shown in FIG. 1 is acquired by the camera, a human body (driver 102) riding on the vehicle and a human body (pedestrian 103) walking on the road are detected. Will be. For example, it is determined that a human body has been detected in the dashed rectangle 104 and the dashed rectangle 105, and tracking is started. Then, for example, when a tracking human body passes through the intrusion detection line 110, a warning is output that an intruder has been detected. That is, it is not possible to cope with a use case in which entry of a vehicle is desired to be excluded from the detection target (= a human body in the vehicle is desired to be excluded from the detection target).

  The present invention has been made in view of such a problem, and it is an object of the present invention to provide a technique that allows an object to be distinguished and detected according to the state of the object.

  In order to solve the above-mentioned problems, a detection device according to the present invention has the following configuration. That is, the detection device includes: a detection unit configured to detect a plurality of portions of an object included in a frame image of a moving image; and an intrusion determination unit configured to determine that the object has entered a predetermined region of the frame image. And determining means for determining whether to notify the determination result by the intrusion determining means depending on whether a predetermined part is included in the part detected by the detecting means. Alternatively, the detection device may include a detection unit that detects a plurality of parts of the object included in the frame image of the moving image, and whether the object is a pedestrian depending on whether the part detected by the detection unit includes a predetermined part. Determining means for determining whether the vehicle is a passenger.

  ADVANTAGE OF THE INVENTION According to this invention, the technique which makes an object distinguishable and detectable according to the state of an object can be provided.

It is a figure explaining the example of intrusion detection. FIG. 3 is a diagram illustrating the principle of intrusion detection in the first embodiment. FIG. 4 is a diagram illustrating a human body state determination logic based on detection of a plurality of parts. FIG. 2 is a diagram exemplarily illustrating a hardware configuration of a detection system. It is a figure which shows the functional structure of a detection apparatus exemplarily. It is a figure showing the example of the screen which sets up the condition of intrusion detection. It is a flowchart of intrusion detection processing in the first embodiment. It is a flow chart of intrusion detection processing in a 2nd embodiment.

  Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. The following embodiments are merely examples, and are not intended to limit the scope of the present invention.

(1st Embodiment)
As a first embodiment of a detection device according to the present invention, a detection system that includes a camera that captures an image of a predetermined monitoring area and a detection device that detects an object from a moving image acquired by the camera will be described below as an example. Will be described. In the following description, a human body is assumed as an object to be detected, but another object may be used.

<System configuration>
FIG. 4 is a diagram exemplarily illustrating a hardware configuration of the detection system. As described above, the detection system includes the detection device 400 and the camera 450. Note that here, the configuration in which the detection device 400 receives an image from one camera 450 via the network 430 is shown, but the detection device 400 may be configured to receive images from a plurality of cameras. Alternatively, a configuration may be adopted in which a camera is mounted on the detection device 400 itself, and an image is directly acquired without passing through the network 430.

  The detection device 400 includes a CPU 401, a ROM 402, a RAM 403, an external storage I / F 404, an input / output I / F 405, and a network I / F 406. The ROM 402 stores programs executed by the CPU 401 and various setting data. The CPU 401 reads the program stored in the ROM 402 into the RAM 403 and executes the program, thereby realizing each functional unit described later with reference to FIG.

  The external storage I / F 404 is an interface for connecting an external storage device 407 such as a hard disk drive (HDD). The input / output I / F 405 is an interface for connecting to an input device such as a keyboard 421, a mouse 422, a touch panel (not shown), and an output device such as a display 423. The network I / F 406 is an interface for connecting to the network 430 and communicating with an external device such as the camera 450. The detection device 400 can be realized by, for example, a general-purpose PC (personal computer), smartphone, or tablet, and does not depend on a specific device form.

  The camera 450 includes a CPU 451, a ROM 452, a RAM 453, an imaging unit 454, and a network I / F 455. The ROM 452 stores programs executed by the CPU 451 and various setting data. The CPU 451 reads the program stored in the ROM 452 into the RAM 453 and executes the program, thereby realizing an imaging process by the imaging unit 454 and a communication process via the network I / F 455. The imaging unit 454 includes an imaging optical system and an imaging device such as a CCD or a CMOS, and generates a moving image (a plurality of frame images) by imaging. A configuration in which compression encoding and the like are also performed may be performed.

  FIG. 5 is a diagram exemplarily showing a functional configuration of the detection device. The detection device 400 includes, as functional units, a communication unit 501, an image acquisition unit 502, a preprocessing unit 503, a human body detection unit 510, a human body tracking unit 504, a state determination unit 505, an intrusion determination unit 506, a result generation unit 508, and an area setting. Unit 507. The human body detection unit 510 includes a plurality of detection units (a head detection unit 511, a face detection unit 512, an upper body detection unit 513, and a whole body detection unit 514) each detecting a different target part, and a result integration unit 515. Including.

  The communication unit 501 performs communication with an external device via the network 530. For example, a compression-encoded moving image is received from the camera 450. The image acquiring unit 502 decodes the acquired moving image and stores the decoded moving image in the external storage device 407 as, for example, a plurality of frame images. The preprocessing unit 503 corrects the frame image so that the feature amount can be easily extracted.

  The human body detection unit 510 detects a human body included in the frame image. More specifically, the head detection unit 511, the face detection unit 512, the upper body detection unit 513, and the whole body detection unit 514 each execute a detection process, and the result integration unit 515 integrates the detection results to detect a human body. Details of the detection processing will be described later with reference to FIGS.

  The human body tracking unit 504 associates the human body detected by the human body detection unit 510 among a plurality of frame images, and tracks the human body. The state determination unit 505 determines the state of the human body detected by the human body detection unit 510. Here, a driver or a pedestrian is identified. The area setting unit 507 sets an image area that is a predetermined area for performing intrusion detection determination. Here, as shown in FIG. 2, it is assumed that a boundary line is set as an intrusion detection line in a straight line (segment) shape with respect to an imaging region indicated by a moving image. An intrusion detection line may be set. Further, it may be set as an intrusion detection area of an arbitrary shape. When setting as an area, for example, intrusion detection is determined on the condition that a target human body exists inside the area for a predetermined time or longer.

  The intrusion determination unit 506 determines the intrusion of a human body to be detected according to the setting by the area setting unit 507. The result generation unit 508 generates information of the detection result by the intrusion determination unit 506, and issues an alarm notification. For example, a warning is displayed on the display unit 522, or a warning is notified to an external device (not shown) via the network 530.

<Principle of intrusion detection>
FIG. 2 is a diagram illustrating the principle of intrusion detection in the first embodiment. In FIG. 2, as in FIG. 1, an image including a human body (driver 202) riding on a vehicle 201 and a human body (pedestrian 203) walking on a road is acquired by a camera. The state is shown.

  An intrusion detection line 210 is set for an image, and the detection device performs intrusion detection determination when a person passes through the intrusion detection line 210. Further, the part detection information 204 and 205 exemplarily show information indicating detection results of four parts detected by four discriminators included in the detection device.

  In the first embodiment, the detection device determines the state of the human bodies 202 and 203 that have passed the intrusion detection line 210 based on the part detection information 204 and 205. Specifically, it is determined whether the human body is a driver or a pedestrian. Then, it is determined that the driver is not an intruder with respect to the human body 202, and no alarm notification is made. On the other hand, a human body 203 (a human body that is not a driver), which is a pedestrian, is determined to be an intruder and an alarm is issued. Here, a human body riding in the vehicle is referred to as a "driver" for convenience, but the present invention can be similarly applied to other vehicle occupants (a human body in a passenger seat or a rear seat).

  FIG. 3 is a diagram illustrating a human body state determination logic based on detection of a plurality of parts. FIG. 3A shows a flow of the human body state determination process, and FIG. 3B shows a table showing a determination result based on a combination pattern of the part detection results.

  The discriminators 302, 304, 306, and 308 each perform a detection process on the frame image corresponding to a different part (body region) of the human body. The discriminators 302, 304, 306, and 308 correspond to the head detection unit 511, the face detection unit 512, the upper body detection unit 513, and the whole body detection unit 514 in FIG. In other words, in the first embodiment, “face 301”, “head 303”, “upper body 305”, and “whole body 307” are assumed as “parts”.

  Each discriminator (strong discriminator) is configured by, for example, cascade connection of a plurality of weak discriminators. The weak classifier detects a pattern of an image feature such as an edge or a color. Note that the optimal detection pattern of the image feature may be obtained by machine learning.

  In the human body detection process 309, a human body is detected based on the result of the part detection by each of the classifiers (strong classifiers). The human body detection 309 corresponds to the result integration unit 515 in FIG. For integration of the detection results, for example, an evaluation function based on a weighted sum of each part can be used.

  In the state determination process 310, the detected state of the human body is determined based on the result of the part detection by each of the classifiers (strong classifiers). Here, based on the combination pattern of the part detection results, the detected human body is determined to be “driver”, “pedestrian”, or “unknown” indicating that it is other than these.

  For example, as shown in FIG. 3B, when at least one of the face and the head is detected in the frame image and no other part is detected, the human body is determined to be a driver. When the whole body is detected in the frame image, the human body is determined to be a pedestrian. That is, the case where the whole body is detected in the frame image is a case where the whole human body including the lower body of the human body which is unlikely to be detected in the case of a vehicle occupant such as a driver is detected, Person. On the other hand, when the whole body is not detected in the frame image, it means that the entire human body including the lower body has not been detected, and therefore, it is determined that the driver is a vehicle occupant. Note that, in the table of FIG. 3B, “〇” indicates that detection has been performed, and “×” indicates that detection has not been performed. Further, "-" indicates that detection is not performed (not used for state determination). Note that the table in FIG. 3B is merely an example, and other combinations of human body parts may be used, or another state may be determined. Further, for example, the plurality of detection units included in the human body detection unit 510 may be configured to include two or more detectors among the above-described four. For example, it is configured to include two parts, a head detection part 511 and a whole body detection part 514, and determines the state based on the detection results by the two detectors. Further, in the present embodiment, the configuration including the above-described four detectors is employed, but a configuration further including a detector that detects the lower body of a human body may be employed. In the case of a vehicle occupant such as a driver, the possibility that the lower body of the human body is detected in the frame image is low. Therefore, when the lower body is detected by the detector, the pedestrian is determined. On the other hand, if the lower body is not detected by the detector, it is determined that the driver is a vehicle occupant. Examples of the detector that detects the lower body include a detector that detects the entire lower body from the waist, a detector that detects a part below the ankle, and a detector that detects the legs.

  FIG. 6 is a diagram illustrating an example of a screen for setting conditions for intrusion detection. Here, it is assumed that the setting of the detection target, the detection area, and the detection determination point is received via the dialog 600 displayed on the display 423. Of course, it may be configured to receive other conditions. Here, in the dialog 600, a radio button 601 for selecting a detection target, a setting button 602 for setting an area, and a display 603 for displaying a detection determination point are arranged. Further, an OK button 604 for confirming the setting and a cancel button 605 for canceling the setting are further arranged.

  The radio button 601 for selecting a detection target receives one selection from a plurality of options by a click operation with the mouse 422 in limiting the intrusion detection target. For example, “exclude a driver” excludes a human body determined as a “driver” from intrusion detection targets. That is, a human body determined to be “pedestrian” or “unknown” is an intrusion detection target. “Detect only pedestrians” excludes the human body determined as “unknown” in addition to the human body determined as “driver” from the intrusion detection targets.

  The setting button 602 accepts setting of an area for determining intrusion detection. For example, when the setting button 602 is pressed by a click operation with the mouse 422, the scene shown in FIG. 2 is displayed, and a user interface (UI) for receiving a change of the intrusion detection line 210 is displayed. The UI corresponds to the area setting unit 507 in FIG. As described above, in addition to being set as an intrusion detection line (line), it is also possible to set as an intrusion detection area (area). The display 603 of the detection determination point is a display indicating the detection point in the area for determining the intrusion detection. For example, the feet, the center, the right, the left, and the like are displayed as the detection points.

<Operation of the device>
FIG. 7 is a flowchart of the intrusion detection processing in the first embodiment. As described above, each functional unit illustrated in FIG. 5 is realized by the CPU 401 reading the program stored in the ROM 402 into the RAM 403 and executing the program.

  In step S701, the image acquisition unit 502 acquires a moving image from the camera 450 via the network 530. Then, the obtained moving image is decoded and stored in the external storage device 407 as, for example, a plurality of frame images.

  In step S702, the preprocessing unit 503 corrects the frame image so that the feature amount can be easily extracted, and the human body detection unit 510 detects a human body included in the frame image. Here, as described above, each of the four detection units 511 to 514 performs a process of detecting each of the four parts of the human body (head, face, upper body, and whole body). Then, the result integrating unit 515 integrates the detection results of the four detection units to detect a human body.

  In S703, the human body tracking unit 504 tracks the human body detected by the human body detection unit 510 over a plurality of frame images. For example, the detection result of the human body in the preceding (= past) frame image is linked and held as a history. In the first embodiment, all the human bodies detected by the human body detection unit 510 are tracked.

  In step S704, the intrusion determination unit 506 determines the intrusion of a human body to be detected according to the setting of the area setting unit 507. For example, it is determined that the movement vector of the human body crosses the intrusion detection line 210 in a predetermined direction.

  In S705, the state determination unit 505 determines the state of the human body crossing the intrusion detection line 210. Specifically, it is determined whether it is a driver, a pedestrian, or unknown by the method described with reference to FIG. If the driver is a driver, the process proceeds to S707. If the driver is not a driver (ie, a pedestrian or unknown), the process proceeds to S706. In S706, if it is a pedestrian, the process proceeds to S708, and if it is not a pedestrian (that is, if it is unknown), the process proceeds to S709.

  In step S707, the intrusion determination unit 506 records that the target human body is a normal entrant. On the other hand, in S708, the intrusion determination unit 506 records that the target human body is an abnormally entrant (pedestrian). In step S709, the intrusion determination unit 506 records that the target human body is an abnormally entered person (unknown). Then, the intrusion determination unit 506 issues an intrusion detection event after recording the abnormal entrance person in S708 or S709.

  In step S711, the intrusion determination unit 506 determines whether the intrusion detection determination in step S704 has been performed for all the human bodies detected in step S702 and tracked in step S703. If there is a human body for which no determination has been made, the process returns to S704, and the determination is made for the remaining human body. If the determination has been completed for all human bodies, the process proceeds to S712.

  In step S712, when the intrusion detection event is issued, the result generation unit 508 issues an alarm notification to make the user recognize that the abnormal entrance has been detected. For example, a warning is displayed on the display unit 522, or a warning is notified to an external device (not shown) via the network 530.

  In S713, the detection device 400 makes an end determination. For example, when there is a termination instruction from the user, the processing is terminated. If there is no instruction, the process returns to S701 to continue the process.

  As described above, according to the first embodiment, a human body is detected based on the detection results of the plurality of detectors, and the state of the human body (driver, pedestrian, unknown) is determined. Then, issuance of an intrusion detection event is controlled based on the determined state. For example, when the human body is determined to be a driver, the issuance of an intrusion detection event is suppressed. Thereby, for example, it is possible to suppress a warning notification about the intrusion of the vehicle (the intrusion of the human body in the vehicle). That is, it is possible to cope with a use case in which entry of a vehicle is desired to be excluded from detection targets.

  In the above description, a human body is assumed as the detection target object, but various objects can be designated as the detection target object. That is, any object that can take a plurality of states based on the detection results of the plurality of classifiers may be used.

(2nd Embodiment)
In the second embodiment, a mode in which the number of human bodies targeted for tracking and intrusion detection is reduced to reduce the processing load will be described. Specifically, an embodiment will be described in which a state determination is performed on a detected human body, and a human body (pedestrian and unknown) excluding a human body (driver) in a specific state is tracked to perform intrusion detection determination. The principle of intrusion detection, the state determination logic, and the hardware configuration are the same as those in the first embodiment (FIGS. 2 to 4), and thus description thereof is omitted.

  The functional configuration of the second embodiment is almost the same as that of the first embodiment (FIG. 5), but the processing order is different from that of the first embodiment. Specifically, in the second embodiment, the human body tracking unit 504 and the state determination unit 505 in FIG. 5 are switched, and the human body tracking is performed after performing the state determination. Hereinafter, parts different from the first embodiment will be mainly described.

<Operation of the device>
FIG. 8 is a flowchart of the intrusion detection processing in the second embodiment. As in the first embodiment, each functional unit illustrated in FIG. 5 is realized by the CPU 401 reading a program stored in the ROM 402 into the RAM 403 and executing the program.

  Steps S801 and S802 are the same as steps S701 and S702 of the first embodiment. In S803, the state determination unit 505 determines the state of the human body detected in S802. Specifically, it is determined whether it is a driver, a pedestrian, or unknown by the method described with reference to FIG. If the driver is a driver, the process proceeds to S805. If the driver is not a driver (ie, a pedestrian or unknown), the process proceeds to S804. In step S804, if the user is a pedestrian, the process proceeds to step S806. If the user is not a pedestrian (ie, unknown), the process proceeds to step S807.

  In step S805, the state determination unit 505 determines that the target human body is a normal candidate for the entrance. That is, even if the human body passes through the intrusion detection line 210, it is recorded as a normal entrant. On the other hand, in S806, the intrusion determination unit 506 determines that the target human body is an abnormal entry candidate (pedestrian). In step S807, the intrusion determination unit 506 determines that the target human body is an abnormal entry candidate (unknown). Then, in step S808, the intrusion determination unit 506 sets the human body recorded as the abnormal entry candidate in step S806 or S807 as a tracking target.

  In step S809, the intrusion determination unit 506 determines whether the state determination in steps S803 to S808 has been performed for all the human bodies detected in step S802. If there is a human body for which no determination has been made, the process returns to S803, and the determination is made for the remaining human body. If the determination has been completed for all the human bodies, the process proceeds to S810.

  In S810, the human body tracking unit 504 tracks the human body set as the tracking target in S808 over a plurality of frame images. Then, in S811, the intrusion determination unit 506 determines the intrusion of a human body to be detected according to the setting by the area setting unit 507, and in S812, the intrusion determination unit 506 issues an intrusion detection event. That is, this is because all the human bodies set as tracking targets in S808 are human bodies recorded as abnormal entrance persons (abnormal entrance candidate candidates) when passing through the intrusion detection line 210.

  In step S813, the intrusion determination unit 506 determines whether the state determination in step S811 has been performed for all the human bodies set as tracking targets in step S808. If there is a human body for which no determination has been made, the process returns to S811, and the determination is made for the remaining human body. If the determination has been completed for all the human bodies set as tracking targets in S808, the process proceeds to S814. Steps S814 and S815 are the same as steps S712 and S713 of the first embodiment.

  As described above, according to the second embodiment, a target human body for tracking and intrusion detection is determined based on the detected result of the detected human body state (driver, pedestrian, unknown). That is, in the second embodiment, it is possible to reduce the number of human bodies for performing tracking and intrusion detection determination as compared to the first embodiment. Therefore, the processing load can be reduced. In addition, based on the determination result in FIG. 3B, the result generation unit 508 may count the human body determined to be the target of intrusion detection, generate the count result, and output the result.

(Modification)
In the above-described embodiment, the state (driver, walking, unknown) of the human body is determined by integrating the detection results obtained by the plurality of detectors (determiners) that detect different parts of the human body. On the other hand, one or more detectors for detecting each part of the human body and one or more detectors for detecting each part of the vehicle are provided, and the detection results of these detectors are integrated to determine the state of the human body. May be.

  For example, a horizontally long moving body region included in a moving image is determined to be a vehicle. Then, for example, when the human body passes through the intrusion detection line 210 and the human body region is located in the middle (forward direction) of the vehicle region, the human body is determined to be the driver. That is, the determination of the state of the human body may be performed by integrating the detection results of objects other than the human body.

(Other Examples)
The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can be realized by a circuit (for example, an ASIC) that realizes one or more functions.

  400 detection device; 401 CPU; 402 ROM; 403 RAM; 430 network; 450 camera; 501 communication unit; 502 image acquisition unit; 510 human body detection unit; 504 human body tracking unit; 505 state determination unit; 506 intrusion determination unit; Generator

Claims (11)

Detecting means for detecting a plurality of parts of the object included in the frame image of the moving image;
Intrusion determination means for determining that the object has entered a region set in advance with respect to the frame image,
Determining means for determining whether to notify a determination result by the intrusion determining means, depending on whether a predetermined part is included in the part detected by the detecting means,
A detection device comprising: The detection device according to claim 1, further comprising a notification unit configured to notify the determination result. Further comprising a setting means for receiving the setting of the area,
The detection device according to claim 1, wherein the setting unit receives the setting of the area as a straight line, a curve, or a combination thereof indicating a boundary line of the area. 4. The method according to claim 1, wherein the determining unit determines to notify a determination result by the intrusion determining unit when a part detected by the detecting unit includes a predetermined part. 5. The detecting device according to the above. The detection device according to any one of claims 1 to 4, wherein the object is a human body, and the predetermined part is a whole body of the human body or a lower body of the human body. 2. The method according to claim 1, wherein the determining unit determines that the determination result is notified to the user to recognize that the pedestrian has entered when a predetermined part is included in the part detected by the detecting unit. The detection device according to any one of claims 1 to 5. The detection device according to claim 1, wherein the detection unit detects a plurality of parts of the object using a plurality of detectors. Detecting means for detecting a plurality of parts of the object included in the frame image of the moving image;
A determination unit that determines whether the object is a pedestrian or a vehicle occupant, by including a predetermined region in the region detected by the detection unit;
A detection device comprising: A detection step of detecting a plurality of parts of the object included in the frame image of the video,
An intrusion determination step of determining that the object has invaded a preset region with respect to the frame image;
A determination step of determining whether to notify a determination result in the intrusion determination step, depending on whether a predetermined part is included in the part detected in the detection step,
A method for controlling a detection device, comprising: A detection step of detecting a plurality of parts of the object included in the frame image of the moving image;
A determination step of determining whether the object is a pedestrian or a vehicle occupant by including a predetermined part in the part detected in the detection step,
A method for controlling a detection device, comprising:   A program for causing a computer to function as each unit of the detection device according to claim 1.