JP2020021110A - Warning system, warning control device, and warning method - Google Patents

Abstract

To quickly and efficiently notify a driver of a warning due to an intrusion of an object into a test course while preventing a decline of a test efficiency.SOLUTION: A warning control device 40 detects an object using a plurality of input images (S3), acquires a movement history information of the object (S4), and then determines the type of the object (S5). The device outputs the type when the type of the object can be identified, and when the type cannot be identified, it determines whether or not to issue a warning depending upon whether or not the movement history of the object falls under a predetermined warning-exclusion condition (S6). When the movement history of the object falls under the predetermined warning-exclusion condition, it does not issue a warning, and the movement history of the object does not fall under the predetermined warning-exclusion condition, it issues the warning.SELECTED DRAWING: Figure 1

Description

  An object of the present invention is to promptly and accurately and efficiently notify a driver of an alarm when a wildlife or an object such as a suspicious person enters a test course, and to suppress a decrease in test efficiency due to a false notification to the driver. The present invention relates to an alarm system, an alarm control device, and an alarm method that can perform an alarm.

  2. Description of the Related Art Conventionally, in a vehicle course, various tests such as a high-speed running test of the vehicle have to be performed in a strictly confidential environment. Such test courses are provided with outer fences to prevent the invasion of wild animals (hereinafter simply referred to as "animals"). In some cases, animals may enter the test course beyond this outer fence. For this reason, in the test course, a plurality of surveillance cameras for monitoring an object entering the test course are provided, and an observer detects an abnormality while visually recognizing a video imaged by the surveillance camera. .

  However, if the image of the surveillance camera is checked visually by the observer, human labor becomes excessive and monitoring may be missed. For example, if the observer overlooks the fact that the animal has crossed the outer fence and invaded the travel path on which the test vehicle travels, there is a possibility that the animal will collide with the test vehicle traveling on the travel path. For this reason, it is conceivable that the animal is automatically recognized using a conventional image processing technique such as template matching and the test is stopped when the animal enters the test course. It is also conceivable that the animal is learned using the conventional technology relating to deep learning (see Patent Literature 1 and Patent Literature 2) and whether or not the input image includes the animal is determined.

JP 2015-210824 A JP 2017-187954 A

  However, in order to recognize the presence of an animal by the image processing technique such as the template matching described above, the form of the animal must be included in the input image in a distinguishable manner. Similarly, when determining the presence or absence of an animal using deep learning, the form of the animal must be included in the input image in a distinguishable manner.

  However, in many cases, an actual image captured by a surveillance camera installed on a test course does not include the form of an animal in a distinguishable manner. This is because the test course has a large area, the number of surveillance cameras that can be installed is limited to a certain number, and it is necessary to take an image of an animal located at a position distant from the surveillance camera. As a result, the number of pixels related to the animal included in the input image decreases, and a situation occurs in which the animal cannot be detected even if template matching or deep learning is used.

  Therefore, if the input image does not sufficiently include the animal's appearance, give priority to avoiding the collision between the animal and the test vehicle, and stop the test even if it is unclear whether the animal is an animal. Although it is inevitable, for example, if the test vehicle or the observer is erroneously recognized as an animal and the test is frequently stopped, the test efficiency will be significantly reduced. Although the focal length of the surveillance camera can be controlled, if the focus of the surveillance camera is zoomed in and focused on a specific object, the field of view of the surveillance camera is limited by that much, and other It is not desirable to perform a process that relies on zooming in the focal point because there is a possibility that detection of an animal located at a location may be missed.

  From these facts, it is an important issue how to notify a driver of a warning accompanying an object entering a test course quickly and efficiently while preventing a decrease in test efficiency. Note that such a problem also occurs when a suspicious person attempts to enter the test course. This is because a suspicious person trying to obtain information on the test vehicle of the new vehicle type may get over the outer fence and enter the test course.

  The present invention has been made to solve the above-mentioned problems (problems) of the prior art, and it is possible to prevent a decrease in test efficiency and to promptly and efficiently give a driver an alarm accompanying entry of an object into a test course. It is an object of the present invention to provide an alarm system, an alarm control device, and an alarm method that can provide an informed notification.

  In order to solve the above-described problems, the present invention provides a plurality of imaging devices, an alarm lamp that notifies a user of an alarm, and an alarm control device communicably connected to the plurality of imaging devices and the alarm lamp. An alarm system comprising: a detection unit that detects an object included in an image captured by each imaging device; and a movement history of the object by tracking the object detected by the detection unit. A movement history information acquisition unit that acquires information, a type determination unit that determines the type of the object detected by the detection unit, and a warning light that is used based on the determination result by the type determination unit and the travel history information. And an alarm determining unit for determining whether or not to issue an alarm.

  Also, the present invention is characterized in that, in the above invention, the detection unit detects an object included in an image captured by each imaging device based on a movement vector of a pixel between a plurality of images.

  Further, according to the present invention, in the above invention, the movement history information acquisition unit specifies a rectangular area including the object based on a movement vector of a pixel forming the object detected by the detection unit and a predetermined likelihood. Then, information relating to the movement history of the specified rectangular area is obtained as movement history information.

  Also, in the present invention according to the above invention, the type determination unit inputs an input image including a partial image of a rectangular area in the image to a multilayer neural network, and outputs a probability for each type output from the multilayer neural network. The type of the object detected by the detection unit is determined based on

  Further, the present invention is characterized in that, in the above invention, the multilayer neural network performs supervised learning using a large number of input images including at least an animal, a suspicious person, a guard, and a vehicle.

  Further, according to the present invention, in the above invention, the multilayer neural network outputs a probability that an object included in the input image is an animal, a suspicious person, a guard, or a vehicle.

  Further, according to the present invention, in the above invention, the warning determination unit turns off the warning light when at least the type detection unit determines that the object detected by the detection unit is a vehicle or a supervisor. It is characterized in that it is determined not to use the used alarm.

  Further, in the present invention according to the above invention, the alarm determination unit uses the warning light when the type determination unit determines that the object detected by the detection unit is an animal or a suspicious person. It is determined that an alarm is issued.

  Further, in the above invention, in the above invention, when the type of the object detected by the detection unit is not specified by the type determination unit, the movement determination information of the object includes a predetermined alarm exclusion. If the condition is satisfied, it is determined that the warning using the warning light is not to be performed. If the movement history information of the object does not correspond to the predetermined alarm exclusion condition, it is determined that the warning using the warning light is to be performed. It is characterized by doing.

  In addition, the present invention provides a detection unit that detects an object included in an image captured by a predetermined imaging device, and a movement history that tracks an object detected by the detection unit and obtains movement history information of the object. An information acquisition unit, a type determination unit that determines the type of the object detected by the detection unit, and whether to perform an alarm using a warning light based on the determination result by the type determination unit and the movement history information. And an alarm determination unit for determining

  According to another aspect of the present invention, there is provided an alarm system including: a plurality of imaging devices; an alarm lamp that notifies a user of an alarm; and an alarm control device that is communicably connected to the plurality of imaging devices and the alarm lamp. A method, wherein the alarm control device detects an object included in an image captured by each imaging device, and obtains movement history information of the object by tracking the object detected in the detection process. Moving history information acquiring step, a type determining step of determining the type of the object detected by the detecting step, and a warning using the warning light based on the determination result by the type determining step and the moving history information. And an alarm determining step of determining whether or not to perform.

  ADVANTAGE OF THE INVENTION According to this invention, a warning accompanying the invasion of an object to a test course can be promptly and efficiently notified to a driver while preventing a decrease in test efficiency. In particular, even when the number of pixels relating to the object included in the input image is small, an alarm accompanying the intrusion of the object into the test course can be issued without unnecessary stoppage of the test.

FIG. 1 is an explanatory diagram for describing an outline of the system configuration and operation of the alarm system according to the present embodiment. FIG. 2 is an explanatory diagram for explaining the concept of the alarm system according to the present embodiment. FIG. 3 is a functional block diagram showing a configuration of the alarm control device shown in FIG. FIG. 4 is a diagram illustrating an example of a type determination using a CNN by the type determination unit illustrated in FIG. 3. FIG. 5 is a flowchart showing a processing procedure of the alarm control device shown in FIG. FIG. 6 is a flowchart showing a processing procedure relating to the object detection and the acquisition of the movement history information shown in step S102 of FIG. FIG. 7 is a flowchart showing a procedure of the alarm determination process shown in step S104 of FIG. FIG. 8 is a flowchart illustrating a procedure for generating learning data used when performing supervised learning of the CNN. FIG. 9 is a diagram illustrating an example of a hardware configuration of the alarm control device illustrated in FIG. 1.

  Hereinafter, an alarm system, an alarm control device, and an alarm method according to the present embodiment will be described with reference to the accompanying drawings. In the present embodiment, a case where the present invention is applied to a test course for performing a running test of a vehicle will be described. Hereinafter, a case where the object A1 is reflected in the input image A will be described.

<Overview of system configuration and operation>
First, an overview of the system configuration and operation of the alarm system according to the present embodiment will be described. FIG. 1 is an explanatory diagram for explaining an outline of the system configuration and operation of the alarm system according to the present embodiment. As shown in FIG. 1, the alarm system includes a monitoring camera 10, an alarm light 20, a speaker 30, and an alarm control device 40.

  The monitoring camera 10 is an imaging device that captures an image of the object A1 that enters the test course T from outside the test course T, and is disposed at regular intervals (for example, every several hundred meters) around the outer fence of the test course T. Have been. The monitoring camera 10 transmits the input image A to the alarm control device 40 via an IP network such as a LAN (Local Area Network). Although the case where the input image A is transmitted from the monitoring camera 10 to the alarm control device 40 is shown here, a moving image can also be transmitted from the monitoring camera 10 to the alarm control device 40. When transmitting a moving image, the alarm control device 40 performs a process of cutting out the input image A for each frame from the moving image.

  The warning light 20 is a display device for visually notifying a driver who drives the test vehicle and a supervisor who monitors the test course T of the occurrence of an abnormality in the test course T. The warning lights 20 are arranged at regular intervals (for example, every several hundred meters) so that the driver of the test vehicle running on the test course T can visually recognize the warning lights, and the display is controlled by the warning control device 40. Here, it is assumed that the warning light 20 is displayed in green while the running test of the test vehicle is being performed, and the warning light 20 is flashed in red if an event to stop the running test occurs. For this reason, for example, if the driver of the test vehicle visually recognizes the flashing of the red warning light 20 while traveling on the road, it is immediately determined that the test is to be stopped and the test vehicle is moved to a predetermined position.

  The speaker 30 is a notification device for acoustically notifying a driver driving the test vehicle, a supervisor monitoring the test course T, and the like of a test situation on the test course T. When an event to stop the running test occurs, a predetermined alarm sound is output according to the instruction of the alarm control device 40. For this reason, for example, if the driver of the test vehicle confirms the warning sound while traveling on the traveling road, it is immediately determined that the test has been stopped and the vehicle moves to a predetermined position.

  When it is determined that the object A1 such as an animal or a suspicious person has entered the test course T, the alarm control device 40 controls the display of the alarm light 20 and the output control of the alarm sound from the speaker 30 to the driver of the test vehicle. It is a device that gives an alarm. Further, when the alarm control device 40 cannot determine the type of the object A1 that has entered the test course T and determines that the object A1 is not likely to be a legitimate object such as a vehicle or a guard, Also, a warning is issued to the driver of the test vehicle by controlling the display of the warning light 20 and controlling the output of a warning sound from the speaker 30. The detailed description of the alarm control device 40 will be described later.

  Here, when the object A1 is imaged by the monitoring camera 10 (S1), the monitoring camera 10 sequentially transmits the captured input image A to the alarm control device 40 (S2). The alarm control device 40 detects the object A1 using the plurality of input images A (S3), acquires movement history information of the object A1 (S4), and then classifies the object A1 (animal, monitor, suspicious). Person or vehicle) is determined (S5). For this type determination, deep learning using a CNN (Convolutional Neural Network) is used. If the type of the object A1 can be identified, the type is output. If the type cannot be identified, “UNKNOWN (unknown)” indicating that the type of the object A1 cannot be identified is output. Thereafter, it is determined whether or not to give an alarm based on the result of the type determination (S6). Specifically, when the type of the object A1 is determined to be “animal” or “suspicious person”, it is determined that a warning is issued, and the type of the object A1 is determined to be “vehicle” or “monitoring person”. In this case, it is determined that no alarm is issued.

  When the type of the object A1 is determined to be “UNKNOWN”, it is determined whether or not the movement history of the object A1 satisfies a predetermined alarm exclusion condition. The alarm exclusion condition refers to a condition in which the possibility that the object A1 collides with the test vehicle is low and no alarm should be issued. Examples of the alarm exclusion condition include, for example, “when the object A1 is stopped at a predetermined monitoring position” and “when the object A1 is traveling around the traveling road at a constant speed along the traveling road”. No. This is because, although the type of the object A1 determined by the deep learning is unknown, the possibility that the object A1 collides with the test vehicle is low. If the movement history of the object A1 satisfies the predetermined alarm exclusion condition, no alarm is issued, and if the movement history of the object A1 does not satisfy the predetermined exclusion condition, an alarm is issued. This is because if the alarm is issued even when the possibility that the object A1 collides with the test vehicle is low, the test efficiency is reduced. If it is determined that an alarm should be issued, an alarm instruction is issued to the alarm lamp 20 and the speaker 30 (S7). The alarm lamp 20 is turned on in red, and the speaker 30 outputs an alarm sound.

<Concept of alarm system>
Next, the concept of the alarm system according to the present embodiment will be described. FIG. 2 is an explanatory diagram for explaining the concept of the alarm system according to the present embodiment. Here, a situation where the object A1 is reflected in the input image A captured by the monitoring camera 10 is shown.

  As described in the background art, in order to accurately detect an object A1 such as an animal or a suspicious person, it is desired that the form of the object A1 be included in the input image A to the extent that it can be distinguished. However, since the area of the test course is very large and it is not practical to install the monitoring camera 10 at any location around the outer fence of the test course, the monitoring cameras 10 are mutually fixed at a certain distance (for example, several hundred meters). ) It must be installed at a remote location. As a result, the number of pixels of the object A1 included in the input image A decreases, and a situation occurs in which an animal cannot be detected even if template matching or deep learning is used. Therefore, a situation may occur in which the object A1 collides with the test vehicle traveling on the traveling path. On the other hand, if the test is stopped in spite of the fact that the object A1 is a test vehicle or a supervisor, the test efficiency will be reduced.

  Therefore, in the alarm system according to the present embodiment, even if the number of pixels related to the object A1 in the input image A is relatively small, the type of the object A1 (animal, monitor, suspicious person, vehicle Make an effort to judge). This is because even if the number of pixels related to the object A1 is small, the feature amount of the object A1 may become apparent. Specifically, the type of the object A1 is determined as much as possible using deep learning using the CNN.

  Then, when the type of the object A1 can be specified from the output result of the CNN, it is checked whether the type of the object A1 is a vehicle, a supervisor, an animal, or a suspicious individual. As a result, if the type of the object A1 is a vehicle or a monitor, the test is continued and no alarm is issued, and if the type of the object A1 is an animal or a suspicious person, an alarm is issued to stop the test. .

  On the other hand, in some cases, the type of the object A1 cannot be specified from the output result of the CNN. This is because the type determination by the CNN has a limit because the number of pixels related to the object A1 is too small. In such a case, if the movement history of the object A1 satisfies the alarm exclusion condition, the test is continued and no alarm is issued. This is because if the alarm is issued even when the possibility that the object A1 collides with the test vehicle is low, the test efficiency is reduced. On the other hand, if the movement history of the object A1 does not correspond to the alarm exclusion condition, an alarm to stop the test is issued.

  Thus, even when the object A1 cannot be photographed at an appropriate distance by the monitoring camera 10 and the number of pixels relating to the object A1 included in the input image A is small, the test course T It is possible to promptly and efficiently notify the driver of an alarm accompanying the entry of the object A1 into the vehicle.

<Configuration of alarm control device 40>
Next, the configuration of the alarm control device 40 shown in FIG. 1 will be described. FIG. 3 is a functional block diagram showing the configuration of the alarm control device 40 shown in FIG. As shown in FIG. 1, the alarm control device 40 includes an input unit 41, a display unit 42, a communication interface unit (hereinafter, simply referred to as a “communication I / F unit”) 43, a storage unit 44, and a control unit 45. .

  The input unit 41 is an input device such as a keyboard or a mouse, and the display unit 42 is a display device such as a liquid crystal panel or a display device. The communication I / F unit 43 is an interface unit for communicating with the monitoring camera 10, the alarm lamp 20, and the speaker 30 via a LAN or the like.

  The storage unit 44 is a storage device such as a nonvolatile memory or a hard disk device, and stores the learning data 44a, the movement history information 44b, and the like. The learning data 44a is image data for learning that is used when the CNN described later performs supervised learning. The movement history information 44b is information indicating the movement history of the object A1.

  The control unit 45 is a control unit that performs overall control of the alarm control device 40, and includes a detection unit 45a, a movement history information acquisition unit 45b, a type determination unit 45c, an alarm determination unit 45d, and an alarm instruction unit 45e. Have. Although a detailed description will be given later, the CPU (Central Processing Unit) of the alarm control device 40 sends the detection unit 45a, the movement history information acquisition unit 45b, the type determination unit 45c, the alarm determination unit 45d, and the alarm instruction unit 45e to each other. A process corresponding to each functional unit is formed by loading an alarm control program including a corresponding routine from a nonvolatile memory or the like onto a RAM (Random Access Memory) as a main storage device and executing the program.

  The detection unit 45a is a processing unit that detects an object A1 included in the input image A captured by the monitoring camera 10 based on a movement vector of pixels between images of a plurality of continuous frames (input image A). Specifically, if there is a moving pixel in a plurality of input images A, the object A1 is detected based on the moving pixel. Note that the pixel movement vector is a vector (for example, an optical flow) between pixels indicating the same location of the object A1.

  The movement history information acquisition unit 45b specifies a rectangular area including the object A1 based on a movement vector of a pixel forming the object A1 detected by the detection unit 45a and a predetermined likelihood, and specifies a movement history of the specified rectangular area. Is a processing unit that acquires the information according to (1) as movement history information. Specifically, an optical flow and a likelihood are calculated for each pixel forming the input image A, a past object extraction result is moved based on the optical flow to extract a moving pixel, and a rectangular area as a particle is extracted. Arranged on the input image A, the rectangular area is moved based on the optical flow at the object extraction result position in the rectangular area, and the likelihood calculated for each rectangular area based on the object extraction result inside and outside the rectangular area is used for the relocation. The object A1 is detected using the sampled particle filter, and the resampling is performed using the likelihood calculated for each rectangular region based on the color histogram of the position of the object extraction result inside and outside the rectangular region. The object A1 is tracked. The specific processing is similar to that of Japanese Patent Application No. 2018-046582. When the rectangular area including the object A1 is specified in this way, for example, the movement history of the center point of the rectangular area (transition of the coordinate position of the center point) is acquired as the movement history information.

  The type determination unit 45c is a processing unit that determines the type of the object A1 detected by the detection unit, and performs the type determination using a multilayer convolutional neural network (CNN). As the CNN, “caffe”, “TensorFlow”, “Chainer”, “CNTK”, or the like provided as an open source product can be used.

  Here, the type determination using the CNN will be further described. FIG. 4 is a diagram illustrating an example of the type determination using the CNN by the type determination unit 45c illustrated in FIG. Here, it is assumed that the supervised learning by the CNN 50 has already been completed. If the input image A including the object A1 is input to the CNN 50 shown in FIG. 4, the CNN 50 has a probability P1 that the object A1 is an animal, a probability P2 that the object A1 is a suspicious person, and a monitor P And the probability P4 that the object A1 is a vehicle is output. Note that P1, P2, P3, and P4 are values between 0 and 1, and the threshold value is “0.8”, and it is assumed that a plurality of probabilities do not simultaneously exceed the threshold value.

  If the probability P1, P2, P3 or P4 is equal to or greater than "0.8", the corresponding type is specified. If the probability P1 output from the CNN 50 is “0.8” or more, the object A1 is specified as an animal. If the probability P2 output from the CNN 50 is “0.8” or more, the object A1 is determined. Is identified as a suspicious individual, and if the probability P3 output from the CNN 50 is “0.8” or more, the object A1 is identified as a monitor, and the probability P4 output from the CNN 50 is “0. If it is 8 or more, the object A1 is specified as a vehicle.

  On the other hand, if all of the probabilities P1, P2, P3, and P4 are less than “0.8”, the type is determined to be indefinite (UNKNOWN). As described above, when the number of pixels of the object A1 included in the input image A is extremely small and an information amount (feature amount) enough to identify the object A1 cannot be obtained, the object A1 is used even if the CNN 50 is used. Cannot be specified, and the type is determined.

  When the type determination unit 45c determines that the object A1 is an animal or a suspicious person, the warning determination unit 45d determines to perform a warning using the warning light 20 and the speaker 30. In addition, when the type determination unit 45c determines that the object A1 is a vehicle or a monitor, it determines that the warning using the warning light 20 and the speaker 30 is not performed. If the type of the object A1 is determined to be “UNKNOWN”, it is determined that no alarm is to be issued if the movement history of the object A1 falls under the alarm exclusion condition. If not, it is determined that an alarm is issued.

  The warning instruction unit 45e is a processing unit that issues a warning instruction to the warning lamp 20 and the speaker 30 when the warning determination unit 45d determines that a warning is to be performed. The warning light 20 stores in advance a display mode of the warning light in association with the data pattern of the warning instruction, and performs display in a display mode corresponding to the data pattern included in the warning instruction of the warning instruction section 45e. Similarly, a pattern of an output sound is stored in the speaker 30 in advance in association with the data pattern of the warning instruction, and outputs an output sound corresponding to the data pattern included in the warning instruction of the warning instruction unit 45e. Note that the speaker 30 can also output an output sound included in a warning instruction output from the warning instruction unit 45e.

<Processing procedure of alarm control device 40>
Next, a processing procedure of the alarm control device 40 shown in FIG. 3 will be described. FIG. 5 is a flowchart showing a processing procedure of the alarm control device 40 shown in FIG. As shown in the drawing, the alarm control device 40 receives a plurality of input images A from the monitoring camera 10 (Step S101). The plurality of input images A are images of frames that are continuous in time series. Note that a moving image may be input from the monitoring camera 10 and an input image A corresponding to each frame may be cut out from the moving image.

  Thereafter, the alarm control device 40 performs object detection and acquisition of movement history information using the plurality of input images A (step S102), and determines the type of the object A1 in the input image A (step S103). In the type determination, as described above, deep learning using the CNN is used. If the type can be specified by the type determination, the type determination result is output. From the CNN, the probability P1 that the type of the object A1 is “animal”, the probability P2 that the type of the object A1 is “suspicious person”, the probability P3 that the type of the object A1 is “monitoring person”, and the type of the object A1 are Since the probability P4 of “vehicle” is output, if any one of the probabilities is equal to or greater than the threshold value “0.8”, the type having the probability is specified as the type of the object A1.

  Thereafter, the alarm control device 40 makes an alarm determination as to whether or not an alarm should be issued (step S104). Specifically, if it is determined that the type of the object A1 is “vehicle” or “monitoring person”, it is determined that no alarm is to be issued, and that the type of the object A1 is “animal” or “suspicious person”. If specified, it is determined that an alarm is issued. If it is determined that the type of the object A1 is “UNKNOWN”, it is determined that no alarm is issued if the movement history of the object A1 satisfies the alarm exclusion condition. For example, when the object A1 is moving at a constant speed along the traveling path as if it were a vehicle, and when the object A1 is stopped at a predetermined monitoring position as if it is a supervisor It is determined that no alarm is issued. If the alarm exclusion condition does not apply, it is determined that an alarm is to be issued.

  When it is determined that a warning should be issued (Step S105; Yes), a warning instruction is issued from the warning instruction unit 45e to the warning light 20 and the speaker 30 (Step S106). The warning light 20 that has received the warning instruction causes the warning light to blink red, and a predetermined output sound is output from the speaker 30. Thus, the driver driving the test vehicle can recognize that the test has been stopped. If it is determined that the alarm should not be issued (Step S105; No), no alarm instruction is issued.

  Next, a procedure related to object detection and acquisition of movement history information shown in step S102 of FIG. 5 will be described. FIG. 6 is a flowchart showing a procedure relating to object detection and acquisition of movement history information shown in step S102 of FIG.

  As shown in FIG. 6, the alarm control device 40 calculates an optical flow and a likelihood for each pixel of the input image A (step S201), and extracts a moving pixel based on the optical flow (step S202). Thereafter, a rectangle as a particle is arranged on the input image A (step S203), and the rectangle is moved based on the optical flow (step S204). Thereafter, the history of the movement vector at the center of the rectangle is stored in the storage unit 44 as the movement history information 44b (step S205). Thereby, the partial image of the rectangular area is input to the CNN as the input image A, and the type determination can be efficiently performed, and the movement history information 44b can be obtained.

  Next, the processing procedure of the alarm determination shown in step S104 of FIG. 5 will be described. FIG. 7 is a flowchart showing the alarm determination procedure shown in step S104 of FIG. As shown in the drawing, when the type of the object A1 is specified by the type determination unit 45c (step S301; Yes), it is determined whether the type of the object A1 is “animal” or “suspicious person”. A determination is made (step S302).

  As a result, when the type of the object A1 is “animal” or “suspicious person” (Step S302; Yes), it is determined that the test should be stopped, that is, an alarm should be issued (Step S303). On the other hand, when the type of the object A1 is not “animal” or “suspicious person” (Step S302; No), it is determined that the test is continued, that is, no alarm is issued (Step S304). If the test is stopped until the object A1 is a "vehicle" or a "monitoring person", the same test must be repeated again, and the test efficiency is reduced.

  On the other hand, when the type of the object A1 is not specified by the type determination unit 45c (Step S301; No), it is determined whether or not the movement history of the object A1 satisfies the alarm exclusion condition (Step S305). For example, when the object A1 is stopped at a predetermined monitoring position or when the object A1 is traveling around the traveling road at a constant speed along the traveling road, it is determined that no alarm is issued. As a result, when the movement history of the object A1 satisfies the alarm exclusion condition (step S306; Yes), it is determined that the test is continued, that is, no alarm is issued (step S307), whereas the movement history of the object A1 is alarmed. When the condition does not correspond to the exclusion condition (Step S306; No), it is determined that the test is stopped, that is, an alarm is issued (Step S308).

  Next, a generation procedure of the learning data 44a used when the CNN performs the supervised learning will be described. FIG. 8 is an explanatory diagram for explaining a generation procedure of the learning data 44a used when the CNN performs the supervised learning. Here, the image of the object A1 included in the input image A is reduced so that the type of the object A1 can be determined as much as possible even when the number of pixels of the object A1 included in the input image A is small. In this state, learning data 44a used for supervised learning is generated.

  As shown in FIG. 8, a composite image of a background image and an animal is generated (step S401), and this composite image is used as a part of the learning data 44a. Similarly, a composite image of the background image and the suspicious individual is generated (step S402), and this composite image is used as a part of the learning data 44a. In addition, a composite image of the background image and the observer is generated (step S403), and this composite image is used as a part of the learning data 44a. Further, a combined image of the background image and the vehicle is generated (step S404), and this combined image is used as a part of the learning data 44a.

  Thereafter, the image of each object A1 is reduced at a predetermined magnification (step S405). If the reduction ratio is not equal to or smaller than a predetermined threshold (step S406; No), the process proceeds to step S401 to perform the same processing. repeat. Then, when the reduction ratio becomes equal to or less than the predetermined threshold value (step S406; Yes), the process ends. By performing such a series of processes, even when the object A1 included in the input image A is small, it is possible to determine the type of the object A1.

  As described above, in the present embodiment, when the object A1 is imaged by the surveillance camera 10, the input image A imaged by the surveillance camera 10 is sequentially transmitted to the alarm control device 40, and the alarm control device 40 Detects the object A1 using a plurality of input images A, obtains the movement history information of the object A1, determines the type (animal, monitor, suspicious person or vehicle) of the object A1, If the type of A1 is not specified, it is determined whether or not the movement history of the object A1 satisfies the alarm exclusion condition. If it is determined that an alarm should be issued, the alarm lamp 20 and the speaker 30 are alerted. Since the warning light 20 is illuminated in red and the warning sound is output from the speaker 30 by giving an instruction, it is possible to prevent the test efficiency from lowering and to prevent the object A1 from entering the test course T while preventing the test efficiency from decreasing. It can be rapidly and efficiently notifying an alarm to the driver.

<Relationship with hardware>
Next, the correspondence between the alarm control device 40 according to the present embodiment and the main hardware configuration of the computer will be described. FIG. 9 is a diagram illustrating an example of a hardware configuration.

  A general computer has a configuration in which a CPU 61, a ROM (Read Only Memory) 62, a RAM 63, a nonvolatile memory 64, and the like are connected by a bus. A hard disk device may be provided instead of the nonvolatile memory 64. For convenience of explanation, only a basic hardware configuration is shown.

  Here, the ROM 62 or the non-volatile memory 64 stores a program or the like necessary for starting an operating system (hereinafter, simply referred to as “OS”). The OS program is read from the memory 64 and executed.

  On the other hand, various application programs executed on the OS are stored in the non-volatile memory 64. When the CPU 61 executes the application program while using the RAM 63 as a main memory, a process corresponding to the application is executed. You.

  The alarm processing program of the alarm control device 40 according to the present embodiment is also stored in the non-volatile memory 64 or the like, like the other application programs, and the CPU 61 loads and executes the alarm control program. Become. The alarm control device 40 according to the embodiment includes routines corresponding to the detection unit 45a, the movement history information acquisition unit 45b, the type determination unit 45c, the alarm determination unit 45d, and the alarm instruction unit 45e illustrated in FIG. The alarm control program is stored in the nonvolatile memory 64 or the like. When the alarm control program is loaded and executed by the CPU 61, processes corresponding to the detection unit 45a, the movement history information acquisition unit 45b, the type determination unit 45c, the alarm determination unit 45d, and the alarm instruction unit 45e are generated. Note that the learning data 44a, the setting data, and the like are stored in the nonvolatile memory 64 in advance.

  Further, in the above-described embodiment, the case where the operation is executed on a stand-alone device has been described. However, the present invention is not limited to this, and may be applied to a case where the operation is realized in a cloud edge server device. it can. Also, the present invention can be applied to a case where distributed processing is performed by a plurality of computers.

  Further, in the above-described embodiment, the case where an object such as an animal or an intruder is set as the alarm target is described. However, the present invention is not limited to this, and entry into the test course is not permitted. The present invention can be applied to a case where various objects such as a suspicious object and a drone are set as alarm targets.

  In addition, each configuration illustrated in the above embodiment is a schematic function, and does not necessarily need to be physically configured as illustrated. In other words, the form of distribution / integration of each device is not limited to the one shown in the drawing, and all or a part thereof is functionally or physically distributed / integrated in arbitrary units according to various loads and usage conditions. Can be configured.

  INDUSTRIAL APPLICABILITY The alarm system, the alarm control device, and the alarm method according to the present invention are useful for quickly and efficiently notifying a driver of an alarm accompanying entry of an object to a test course while preventing a decrease in test efficiency.

A input image A1 object 10 surveillance camera 20 indicator light 30 speaker 40 alarm control device 41 input unit 42 display unit 43 communication I / F unit 44 storage unit 44a learning data 44b movement history information 45 control unit 45a detection unit 45b acquisition of movement history information Unit 45c type judgment unit 45d alarm judgment unit 45e alarm processing unit 50 CNN
61 CPU
62 ROM
63 RAM
64 nonvolatile memory 65 bus

Claims (11)

A plurality of imaging devices, an alarm light that notifies a user of an alarm, and an alarm system including an alarm control device communicably connected to the plurality of imaging devices and the alarm light,
The alarm control device,
A detection unit that detects an object included in an image captured by each imaging device;
A movement history information acquisition unit that acquires the movement history information of the object by tracking the object detected by the detection unit;
A type determination unit that determines the type of the object detected by the detection unit,
An alarm system comprising: an alarm determination unit that determines whether to perform an alarm using the alarm light based on a determination result by the type determination unit and the movement history information. The detection unit,
The alarm system according to claim 1, wherein an object included in an image captured by each imaging device is detected based on a movement vector of a pixel between a plurality of images. The movement history information acquisition unit,
A rectangular area including the object is specified based on a movement vector of a pixel forming the object detected by the detection unit and a predetermined likelihood, and information on a movement history of the specified rectangular area is acquired as movement history information. The alarm system according to claim 1 or 2, wherein: The type determination unit,
An input image including a partial image of a rectangular area in the image is input to a multilayer neural network, and a type of an object detected by the detection unit is determined based on a probability for each type output from the multilayer neural network. The alarm system according to claim 3, wherein: The multilayer neural network includes:
5. The alarm system according to claim 4, wherein supervised learning is performed using a large number of input images including at least an animal, a suspicious person, a guard, and a vehicle. The multilayer neural network includes:
The alarm system according to claim 4, wherein the probability that the object included in the input image is an animal, a suspicious person, a guard, or a vehicle is output. The alarm determination unit,
5. The method according to claim 4, wherein at least when the type determination unit determines that the object detected by the detection unit is a vehicle or a supervisor, it is determined that the warning using the warning light is not performed. The alarm system according to 1. The alarm determination unit,
The method according to claim 4, wherein when the type detection unit determines that the object detected by the detection unit is an animal or a suspicious person, it is determined that a warning using the warning light is to be performed. Alarm system. The alarm determination unit,
When the type of the object detected by the detection unit is not specified by the type determination unit, if the movement history information of the object satisfies a predetermined alarm exclusion condition, the alarm using the alarm light is not performed. 5. The alarm system according to claim 4, wherein if the movement history information of the object does not correspond to a predetermined alarm exclusion condition, it is determined that an alarm using the alarm lamp is to be performed. A detection unit that detects an object included in an image captured by a predetermined imaging device;
A movement history information acquisition unit that acquires the movement history information of the object by tracking the object detected by the detection unit;
A type determination unit that determines the type of the object detected by the detection unit,
An alarm control device, comprising: an alarm determination unit that determines whether to perform an alarm using an alarm lamp based on a determination result by the type determination unit and the movement history information. A plurality of imaging devices, an alarm light that notifies a user of an alarm, and an alarm method in an alarm system including an alarm control device communicably connected to the plurality of imaging devices and the alarm light,
A detection step in which the alarm control device detects an object included in an image captured by each imaging device,
A movement history information acquisition step of tracking an object detected by the detection step and acquiring movement history information of the object;
A type determination step of determining the type of the object detected by the detection step,
An alarm determination step of determining whether to perform an alarm using the alarm light based on a determination result in the type determination step and the movement history information.

Images