JP7004786B1 - Detection device and detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect an intruder in an exclusion zone while suppressing the occurrence of erroneous detection. A detection device is a detection device that detects an object that has invaded the inside of a predetermined exclusion zone, and has an acquisition unit that acquires an image generated by imaging a space including the exclusion zone. , Judgment from the captured image whether or not the specific portion that identifies the rectangular region corresponding to the object and all of the plurality of points of interest in the rectangular region are included in the corresponding region corresponding to the exclusion zone in the captured image. It has a unit and an output unit that outputs an alarm when it is determined that all of the plurality of points of interest are included in the corresponding area. [Selection diagram] Fig. 1

Description

The present invention relates to a detection device and a detection method.

Conventionally, it has been performed to detect an intruder into a predetermined restricted area by analyzing an image captured by an image pickup device such as a surveillance camera. For example, in Patent Document 1,
An intruder detection device that detects an intruder existing in an exclusion zone based on a camera image captured by a surveillance camera is described.

The intruder detection device described in Patent Document 1 divides a camera image into a plurality of rectangular small areas, and detects a small area in which a moving object exists based on a time change of a feature amount calculated for each small area. do. Then, the intruder detection device detects a moving object as an intruder when at least a part of the detected small area is located inside the intrusion prohibited area preset in the camera image.

Japanese Unexamined Patent Publication No. 2007-23319

The intruder detection device described in Patent Document 1 detects an intruder based on the positional relationship between a small area including a moving object and an exclusion zone. Therefore, even if the moving object is not located inside the exclusion zone, if a part of the small area is located inside the exclusion zone, the moving object will be erroneously detected as an intruder. there were. Therefore, it has been required to reduce erroneous detection in such an intruder detection device.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a detection device and a detection method capable of suppressing erroneous detection when detecting an intruder in an exclusion zone. do.

The detection device according to the present invention is a detection device that detects an object that has invaded the inside of a predetermined no-entry area, and has an acquisition unit that acquires an image generated by imaging a space including the no-entry area. , Judgment from the captured image whether or not the specific portion that identifies the rectangular region corresponding to the object and all of the plurality of points of interest in the rectangular region are included in the corresponding region corresponding to the inaccessible prohibited area in the captured image. It is characterized by having a unit and an output unit that outputs an alarm when it is determined that all of a plurality of points of interest are included in the corresponding area.

Further, in the detection device according to the present invention, it is preferable that a plurality of points of interest are both end points of the lower side of the rectangular region.

Further, in the detection device according to the present invention, it is preferable that a plurality of points of interest are selected according to the operation of the user.

Further, in the detection device according to the present invention, the acquisition unit acquires a plurality of captured images having different imaging times, and the specific unit identifies a rectangular region based on the difference information between the plurality of captured images. Is preferable.

The detection method according to the present invention is a detection method executed by a detection device that detects an object that has invaded the inside of a predetermined exclusion zone, and is an image captured by imaging a space including the exclusion zone. Is obtained, a rectangular area corresponding to the object is specified from the captured image, and it is determined whether or not all of the plurality of points of interest in the rectangular area are included in the corresponding area corresponding to the exclusion zone in the captured image. It is characterized by including outputting an alarm when it is determined that all of the plurality of points of interest are included in the corresponding area.

The detection device and the detection method according to the present invention make it possible to suppress erroneous detection when detecting an intruder in an exclusion zone.

It is a figure which shows an example of the schematic structure of the detection system 1. It is a figure which shows an example of the schematic structure of the detection device 2. It is a figure which shows an example of the data structure of the captured image table T1. It is a flow chart which shows an example of the flow of a detection process. (A) is a diagram showing an example of a background image, and (b) is a diagram showing an example of a captured image.

Hereinafter, various embodiments of the present invention will be described with reference to the drawings. It should be noted that the technical scope of the present invention is not limited to these embodiments but extends to the inventions described in the claims and their equivalents.

FIG. 1 is a diagram showing an example of a schematic configuration of the detection system 1 according to the present invention. The detection system 1 is a system for detecting an object that has invaded the inside of a predetermined exclusion zone in a facility. The detection system 1 includes an image pickup device 2, a detection device 3, and a user terminal 4.

The image pickup device 2 is a device that captures an image of a space including an exclusion zone and generates an captured image, and is, for example, a surveillance camera. The image pickup apparatus 2 generates an image to be captured by taking an image of a space including an exclusion zone at predetermined time intervals (for example, 1/30 second). In the example shown in FIG. 1, the captured image includes an exclusion zone A set in the site of the facility surrounded by the fence F and a road R outside the exclusion zone A. The image pickup device 2 sequentially transmits the generated captured image to the detection device 3.

The detection device 3 is a device that detects an intruder based on an captured image. When an intruder is detected, the detection device 3 outputs an alarm to the user terminal 4. The user terminal 4 is an information processing terminal such as a PC (Personal Computer), a tablet terminal, or a multifunctional mobile phone (smartphone) used by a user who monitors a facility. The user terminal 4 displays the alarm output from the detection device 3 or outputs it as voice.

FIG. 2 is a diagram showing an example of a schematic configuration of the detection device 3. The detection device 3 has a storage unit 31, a video receiving unit 32, a video transmitting unit 33, a communication unit 34, and a processing unit 35.

The storage unit 31 is configured to store programs and data, and includes, for example, a semiconductor memory. The storage unit 31 stores an operating system program, a driver program, an application program, data, and the like used for processing by the processing unit 35. The program is installed in the storage unit 31 from a computer-readable and non-temporary portable storage medium such as a CD (Compact Disc) -ROM (Read Only Memory) using a setup program or the like.

The image receiving unit 32 is configured for the detection device 3 to receive image data from the image pickup device 2, and includes an interface circuit. The interface circuit included in the video receiving unit 32 is, for example, a circuit of a video transmission interface such as HD-TVI (High Definition Transport Video Interface). The image receiving unit 32 supplies the data received from the image pickup device 2 to the processing unit 35.

The video transmission unit 33 is configured for the detection device 3 to transmit image data to the user terminal 4, and includes an interface circuit. The interface circuit included in the video transmission unit 33 is a circuit of a video transmission interface such as NTSC (National Television System Committee). The video transmission unit 33 transmits the data supplied from the processing unit 35 to the user terminal 4.

The communication unit 34 is configured for the detection device 3 to communicate with the user terminal 4, and includes a communication interface circuit. The communication interface circuit included in the communication unit 34 is a communication interface circuit such as a wired LAN (Local Area Network) or a wireless LAN. The communication unit 34 transmits the data supplied from the processing unit 35 to the user terminal 4, and supplies the data received from the user terminal 4 to the processing unit 35.

When the image pickup device 2 is a so-called network camera, the communication unit 34 may also have the function of the video reception unit 32. Further, the communication unit 34 may also serve as the function of the video transmission unit 33.

The processing unit 35 is a device that comprehensively controls the operation of the detection device 3, and includes one or a plurality of processors and peripheral circuits thereof. The processing unit 35 includes, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The processing unit 35 may include a DSP (Digital Signal Processor), an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), and the like. The processing unit 35 controls the operation of each unit of the detection device 3 based on the program stored in the storage unit 31, and executes various processes.

The processing unit 35 includes an acquisition unit 351, a specific unit 352, a determination unit 353, and an output unit 354 as its functional blocks. Each of these parts is a functional module realized by a program executed by the processing unit 35. Each of these parts may be mounted on the detection device 3 as firmware.

FIG. 3 is a diagram showing an example of the data structure of the captured image table T1 stored in the storage unit 31. The captured image table T1 stores information of an image ID, an image file, a captured time, a background image, and a rectangular area in association with each other.

The image ID is identification information of the captured image transmitted from the imaging device 2. The image file is identification information of image data at the time of imaging, and includes a file name, a file path, and the like. The imaging time is information indicating the time when the captured image was generated. The background image is information indicating whether or not the captured image is used as a background image that does not include an object such as a person, an animal, or an automobile in the detection process described later. The information of the rectangular region is the information indicating the region corresponding to the object in the captured image, which is specified in the detection process described later. In the example shown in FIG. 3, the information in the rectangular region is the coordinates of the pixels corresponding to the four vertices of the rectangular region among the pixels constituting the captured image.

Of the data in the captured image table T1, the data excluding the background image is generated by the detection device 3 in the detection process described later. The background image data is set by the user who confirmed the captured image.

FIG. 4 is a flow chart showing an example of the flow of the detection process executed by the detection device 3. The detection process is executed for each captured image each time the captured image is transmitted from the image pickup device 2 to the detection device 3. The detection process is realized by the processing unit 35 that executes the program stored in the storage unit 31 in cooperation with each configuration of the detection device 3.

First, the acquisition unit 351 acquires an captured image generated by imaging a space including an exclusion zone (S11). The acquisition unit 351 receives the image transmitted from the image pickup device 2 via the image reception unit 32, and acquires a frame image included in the received image as an image to be captured. Further, the acquisition unit 351 acquires the imaging time of each frame image from the received video. The acquisition unit 351 associates the acquired image with the image ID and the image capture time and stores the image in the image table T1.

Subsequently, the specifying unit 352 specifies a rectangular region corresponding to the object from the captured image (S12). The rectangular area corresponding to the object is a rectangular area circumscribing the object in the captured image. The specifying unit 352 specifies a rectangular region, for example, based on the difference information between a plurality of captured images having different imaging times.

In this case, the specific unit 352 refers to the captured image table T1 and acquires a background image which is an captured image generated by imaging a predetermined space at a time when there is no object. The specific unit 352 calculates the difference information indicating the difference in the luminance value between each pixel of the background image and each pixel of the acquired captured image. Based on the difference information, the specific unit 352 extracts pixels whose absolute value of the difference is equal to or greater than a predetermined value from each pixel of the captured image. When a pixel is extracted, the specifying unit 352 specifies a rectangular region circumscribing the extracted pixel as a rectangular region corresponding to the object. If the pixels are not extracted, the specific unit 352 does not specify the rectangular area corresponding to the object.

When the rectangular region is specified, the specific unit 352 stores the information of the specified rectangular region in the captured image table T1 in association with the captured image.

The specific unit 352 may classify the extracted pixels into a plurality of groups based on the mutual positional relationship, and set a plurality of rectangular areas circumscribing the pixels belonging to each group. Pixel classification is performed, for example, by using hierarchical clustering and merging these groups when the shortest distance between pixels belonging to different groups is less than a predetermined distance. As a result, even when a plurality of objects are included in the captured image, each object is appropriately detected.

FIG. 5A is a diagram showing an example of a background image B not including an object image, and FIG. 5B is a diagram showing an example of a captured image P including an object. In the example shown in FIG. 5B, the captured image P includes a person S1 located inside the exclusion zone A and a vehicle S2 traveling on a road R outside the exclusion zone A. The specifying unit 352 identifies the rectangular region K1 corresponding to the person S1 and the rectangular region K2 corresponding to the automobile S2 based on the difference information between the background image B and the captured image P.

Returning to FIG. 4, when the rectangular area is specified (S13-Yes), the determination unit 353 determines whether or not all of the plurality of points of interest in the rectangular area are included in the corresponding area corresponding to the exclusion zone in the captured image. (S14). The plurality of points of interest are, for example, both end points of the lower edge of the rectangular area. In this case, the determination unit 353 determines whether both of the lower end points are included in the corresponding area, or whether at least one of the lower end points is not included in the corresponding area.

As shown in FIG. 5B, both end points C1 and C2 of the lower side of the rectangular region K1 corresponding to the person S1 are included in the corresponding region (shown by shaded shades) corresponding to the exclusion zone A. Therefore, the determination unit 353 determines that all of the plurality of points of interest in the rectangular region K1 are included in the corresponding region. Further, of the end points C3 and C4 on the lower side of the rectangular region K2 corresponding to the automobile S2, the right end point C4 is included in the corresponding area, but the left end point C3 is not included in the corresponding area. Therefore, the determination unit 353 determines that at least one of the plurality of points of interest in the rectangular area K2 is not included in the corresponding area.

Returning to FIG. 4, when it is determined that all of the plurality of points of interest are included in the corresponding area (S15-Yes), the output unit 354 outputs an alarm (S16). For example, the output unit 354 outputs an alarm by transmitting voice information for outputting a predetermined alarm sound to the user terminal 4 via the communication unit 34. The output unit 354 may output an alarm by transmitting display data of a message indicating that the object has entered the restricted area to the user terminal 4 via the communication unit 34. When it is determined that at least one of the plurality of points of interest is not included in the corresponding area (S15-No), the output unit 354 does not output an alarm.

Subsequently, the output unit 354 outputs the captured image on which the rectangular region is superimposed (S17), and ends the detection process. The output unit 354 refers to the information of the rectangular region of the captured image table T1 and generates the display data of the captured image on which the rectangular region is superimposed as shown in FIG. 5 (b). The output unit 354 transmits the generated display data to the user terminal 4 via the video transmission unit 33. The output unit 354 transmits the generated display data as video data in which the captured image on which the rectangular region is superimposed is used as the frame image. The user terminal 4 stores and displays the received video data.

When the rectangular region corresponding to the object is not specified in S13 (S13-No), the output unit 354 outputs the captured image (S18) and ends the detection process. The output unit 354 transmits video data using the captured image as a frame image to the user terminal 4 via the video transmission unit 33. The user terminal 4 stores and displays the received video data.

As described above, the detection device 3 outputs an alarm when all of the plurality of points of interest in the rectangular region corresponding to the object in the captured image are included in the corresponding region corresponding to the exclusion zone. This makes it possible for the detection device 3 to suppress erroneous detection when detecting an intruder in the exclusion zone.

That is, as shown in FIG. 5B, even when the object is located outside the restricted area, a part of the rectangular area corresponding to the object is included in the corresponding area corresponding to the restricted area. The object may be falsely detected as an intruder. On the other hand, since the detection device 3 detects the object as an intruder only when all of the plurality of points of interest in the rectangular region are included in the corresponding region, the occurrence of erroneous detection can be suppressed.

Further, the detection device 3 uses both end points of the lower side of the rectangular region as a plurality of points of interest. This makes it possible for the detection device 3 to detect an intruder in the exclusion zone while further suppressing the occurrence of erroneous detection. For example, as shown in FIG. 5B, although the automobile S2 is located outside the exclusion zone A, the center point C5 of the lower side of the rectangular region K2 corresponding to the automobile S2.
Is included in the corresponding area of the exclusion zone A. Therefore, if only the center point of the lower side of the rectangular region is used as the point of interest, erroneous detection will occur. On the other hand, since the left end points C3 of the lower end points C3 and C4 of the rectangular area K2 are not included in the corresponding area, erroneous detection occurs when the lower end points of the rectangular area are used as points of interest. Does not occur.

That is, in most cases where both end points of the lower side of the rectangular area are included in the corresponding area, the lower end of the object (or the foot when the object is a person) is located inside the exclusion zone. Therefore, by detecting the object as an intruder when both the end points of the lower end of the rectangular area are included in the corresponding area, the detection device 3 can detect the intruder with high accuracy.

Further, the detection device 3 specifies a rectangular region based on the difference information between the background image captured in advance and the acquired captured image. This allows the detection device 3 to detect an intruder with a small computational load.

That is, the calculation of the difference information between the background image and the captured image is performed by an addition / subtraction operation. Further, the process of extracting the pixels whose absolute value of the difference is equal to or higher than the predetermined value from each pixel of the captured image is performed by a comparison operation. Further, since the process of setting the rectangular area circumscribing the extracted pixels is the process of calculating the maximum and minimum values of the pixel coordinate values (X coordinate and Y coordinate) of the extracted pixels, comparison is performed. It is done by calculation. Since the rectangular region is specified only by these operations with a small calculation load, the calculation load of the detection device 3 can be suppressed.

In the above description, the specific unit 352 specifies the rectangular region based on the difference information between the background image and the captured image, but the present invention is not limited to such an example. For example, the specifying unit 352 may specify a rectangular region based on the difference information between the captured image and the comparison target image which is the captured image acquired immediately before the captured image. Further, the specifying unit 352 may specify the rectangular area without using the difference information. For example, the specifying unit 352 may specify a rectangular area using a trained model trained to specify a rectangular area corresponding to an object such as a person or an animal when an image is input. The trained model is, for example, R-CNN (Region with Convolutional Neural Network) or SSD (Single Shot Detector).

In the above description, it is assumed that the plurality of points of interest are both end points of the lower side of the rectangular area, but the present invention is not limited to such an example. The plurality of points of interest may be the vertices of a rectangular area. Further, the plurality of points of interest may be a combination of both end points of the lower side of the rectangular area and the center point, or may be a combination of the end points of the rectangular area and the center of gravity point of the rectangular area. The plurality of points of interest may be a combination of the center point of the lower side of the rectangular area and the center of gravity point of the rectangular area. Further, the point of interest may be any point on or inside the rectangular region. For example, the point of interest may be a point separated by a predetermined distance from the center of gravity of the rectangular region to either the top, bottom, left, or right. The predetermined distance may be a distance determined according to the size of the rectangular area (for example, a distance corresponding to 5% of the width or height of the rectangular area).

Further, the points of interest may be selected from points on and inside the rectangular area according to the user's operation. In this case, the user operates the user terminal 4 in advance to select a point of interest. For example, the user terminal 4 presents a plurality of options regarding a point of interest to the user and accepts the selection from the user. The options are, for example, the center point of the lower side of the rectangular area, the center point of the rectangular area, the end points of the lower side of the rectangular area, and the vertices of the rectangular area. The user terminal 4 generates selection information indicating the options selected by the user and transmits the selection information to the detection device 3. The detection device 3 stores the received selection information in the storage unit 31.

In S14 of the detection process, the determination unit 363 sets the points of interest based on the selection information stored in the storage unit 31, and determines whether or not all of the points of interest are included in the corresponding area. This allows the detection device 3 to allow the user to select an appropriate detection accuracy according to the degree of erroneous detection. The plurality of options may include an option in which the number of points of interest is one (in the above example, the option of the center point of the lower side of the rectangular area and the option of the center of gravity point of the rectangular area).

Further, the point of interest may be automatically selected by the detection device 3. In this case, when the determination unit 363 determines in S14 of the detection process that all of the points of interest are included in the corresponding area, the determination unit 363 stores the determination time at which the determination is made in the storage unit 31. When the determination time is stored at a predetermined frequency (for example, several tens of times a day) or more, the determination unit 363 determines that erroneous detection occurs frequently, and the selection information stored in the storage unit 31. Change the point of interest by changing. In this case, the determination unit 363 changes the point of interest according to, for example, a predetermined priority. This makes it possible for the detection device 3 to suppress erroneous detection without waiting for an operation by the user.

In the above description, when the rectangular region corresponding to the object is specified, the detection device 3 outputs the captured image in which the rectangular region is superimposed, but the present invention is not limited to such an example. When it is determined in the detection process that at least one of the plurality of points of interest is not included in the corresponding area (S15-No), the output unit 354 outputs an captured image in which the rectangular area is not superimposed. Good (S18). As a result, the user terminal 4 displays the rectangular area only when the object is located inside the exclusion zone, so that the detection device 3 visually recognizes that the intruder has been detected. Make it easy to do. Further, the output unit 354 may output only an alarm without executing S17 and S18.

In the above description, the detection device 3 is supposed to output the captured image in which the rectangular region is superimposed when the rectangular region is specified, but the present invention is not limited to such an example. The detection device 3 may determine whether or not to output the captured image in which the rectangular region is superimposed according to the user's selection. In this case, the user operates the user terminal 4 in advance to select whether or not to display the rectangular area. The user terminal 4 generates selection information indicating the options selected by the user and transmits the selection information to the detection device 3. The detection device 3 stores the received selection information in the storage unit 31. In S17, the output unit 354 outputs a captured image in which the rectangular area is superimposed when the selection information indicates that the rectangular area is displayed, and when the selection information indicates that the rectangular area is displayed, the rectangular area is displayed. Outputs a captured image in which is not superimposed. This makes it possible for the detection device 3 to prevent the visibility of the image from being lowered due to the rectangular area. Further, the detection device 3 may output both the captured image in which the rectangular region is superimposed and the captured image in which the rectangular region is not superimposed. In this case, the user selects which captured image is to be displayed according to the operation of the user terminal 4.

In the above description, the detection system 1 includes the detection device 3 and the user terminal 4, but the functions of the detection device 3 and the user terminal 4 may be realized by one device. For example, the function of the detection device 3 may be realized by the user terminal 4. Further, only a part of the functions of the detection device 3 may be realized by the user terminal 4.

It will be appreciated by those skilled in the art that various changes, substitutions and modifications can be made to this without departing from the spirit and scope of the invention. For example, the processes of the above-mentioned parts may be executed in different orders as appropriate within the scope of the present invention. Further, the above-described embodiments and modifications may be carried out in appropriate combinations within the scope of the present invention.

3 Detection device 31 Storage unit 32 Video reception unit 33 Video transmission unit 34 Communication unit 35 Processing unit 351 Acquisition unit 352 Specific unit 353 Judgment unit 354 Output unit

Claims (4)

It is a detection device that detects an object that has invaded the inside of a predetermined exclusion zone.
An acquisition unit that acquires a captured image generated by imaging a space including the exclusion zone, and
From the captured image, a specific portion that identifies a rectangular region corresponding to the object, and a specific portion.
A determination unit for determining whether or not all of the plurality of points of interest, which are both end points of the lower side of the rectangular region, are included in the corresponding region corresponding to the exclusion zone in the captured image.
A detection device including an output unit that outputs an alarm when it is determined that all of the plurality of points of interest are included in the corresponding area. The plurality of points of interest are selected from points on and inside the rectangular area according to the user's operation.
The detection device according to claim 1 . The acquisition unit acquires a plurality of captured images having different imaging times, and obtains them.
The specific unit identifies the rectangular region based on the difference information between the plurality of captured images.
The detection device according to claim 1 or 2 . A detection method performed by a detection device that detects an object that has invaded the inside of a predetermined exclusion zone.
The captured image generated by imaging the space including the exclusion zone is acquired.
From the captured image, a rectangular area corresponding to the object is identified.
It is determined whether or not all of the plurality of points of interest, which are both end points of the lower side of the rectangular region, are included in the corresponding region corresponding to the exclusion zone in the captured image.
A detection method comprising outputting an alarm when it is determined that all of the plurality of points of interest are included in the corresponding area.

Images