JPWO2015087730A1 - Monitoring system - Google Patents

Abstract

A monitoring system is disclosed which has an improved defect that the situation can be confirmed only for the cameras displayed on the monitor when an intruder enters the field of view of a plurality of cameras at the same time. The system includes a video acquisition unit that acquires video captured by a plurality of cameras, an object detection unit that detects an object from the video acquired by the video acquisition unit, a video display unit, and a video displayed on the video display unit. Video display control means for displaying. The video display control means has functions of a video reception unit, an alarm reception unit, a map display unit, a buzzer control unit, a video switching unit, a coordinate conversion unit, a map creation unit, a position determination unit, an object superposition unit, and a trajectory superposition unit. The video display control means includes a first output obtained by superimposing a camera arrangement image and a perspective three-dimensional object image on a planar map image, an image representing an intruder or an intruder when an alarm is received by an alarm receiver, and an intrusion locus. And a second output in which the arrangement image of the camera is superimposed on the planar map image.

Description

  The present invention relates to a monitoring system.

As a background art, a conventional monitoring system having an intruder automatic detection function will be described with reference to FIGS.
FIG. 1 shows a system configuration diagram of a conventional monitoring system.
The system in FIG. 1 monitors the site with five cameras, and includes a camera 101, an object detection unit 102, a video distribution unit 103, a LAN_SW (Local Area Network SWitch) 104, and a monitor display PC (Personal Computer). 105.
An image captured by the camera 101 is input to the object detection unit 102.
The object detection unit 102 performs an intruder / intruder detection process on the input video, superimposes various information (lines, characters, markers, etc.) on the input video, and outputs a detection result video.
The video distribution unit 103 compresses the detection result video output from the object detection unit 102 and distributes the video through the LAN. The video distributed by the video distribution unit 103 is received and decoded by the monitor display PC 105 and displayed on the monitor unit 210 connected to the monitor display PC 105.

FIG. 4 is a block diagram of the object detection unit 102.
An object detection unit 102 receives an image signal from a camera 101 connected by a coaxial cable or the like, performs an A / D conversion (Analog / Digital Conversion), and the like, and inputs an image input I / F (InterFace). 401, an image memory 402 for temporarily storing images, a work memory unit 403 for temporarily storing data for image processing, a CPU (Central Processing Unit) 204 for executing a program, and a video stored in the image memory 402 Is an image output I / F 405 for D / A conversion and output to the outside, a program memory 406, a data bus 407, a LAN_I / F 408 for alarm notification to the monitor display PC 105 and parameter setting from the monitor display PC 105 It is configured.

An intruder / intruder detection software 409 is stored in the program memory 406.
The intruder / intruder detection software 409 captures the video 101 of the camera with the image input I / F 401 and uses the image memory 402 and the work memory 403 to perform inter-frame difference processing represented by a general motion detector, threshold value, and the like. Intruder / intruder detection is performed by processing and object determination (area / width / height determination).
The intruder / intruder detection software 409 outputs an image stored in the image memory 402 (an image in the middle of processing (difference image or binary image), a processing result image obtained by superimposing a processing result on an input image, or the like). It has a function of displaying on a monitor or the like using the I / F 405. Further, the detection result, that is, the intruder / intruder detection start / end signal is notified to the monitor display PC 105 by the LAN_I / F 201.

FIG. 2 is a functional block diagram of the monitor display PC 105, and FIG. 3 is a diagram in which the monitor video generated by the monitor display PC 105 is displayed on the monitor.
In FIG. 2, the monitor display PC 105 receives an alarm notification from the object detection unit 102, transmits / receives parameters of the object detection unit 102, LAN_I / F 201 used to receive distribution video from the video distribution unit 103, and calculates and stores data. Memory 202 for performing, HDD 23 storing map data, CPU 204 for executing a program, input I / F 205 for operating a monitor display PC with a pointing device 212 or a keyboard, a program memory 206, a buzzer 211 Buzzer output I / F 207 for controlling (ON / OFF), monitor output I / F 208 for D / A converting the generated display map image 306 and display camera video 307 and outputting to the monitor, data bus 209, Pointing device 212, monitor unit 2 0-1～210-2, and a buzzer 211.

The program memory 206 stores monitor display software. The monitor display software includes a video reception unit 206-1, an alarm reception unit 206-2, a map display unit 206-3, a buzzer control unit 206-4, and video switching. Part 206-5.
The video receiving unit 206-1 receives the video distributed from the video distribution unit 103 using the LAN_I / F 201, decodes the received video, holds it in a memory, and stores video data before decoding as necessary. Is also recorded on the HDD. Here, the video of the camera stored in the memory or the HDD may be all cameras or only the camera displayed in the display camera video 307.
The alarm receiving unit 206-2 receives the intruder / intruder detection start / end signal output from the object detection unit 102, and based on the received signal state, the alarm state (previous and current) of each camera. Intruder or presence / absence of intruder) is recorded in the memory 202. For management purposes, each alarm is assigned a serial number (received alarm ID) in the order of reception, and a notification type, a camera ID of the notification source, time, position information, and the like are recorded.
The stored intruder or intruder detection start / end signal is generated as a map display screen generation on the map display unit 206-3, a buzzer control on the buzzer control unit 206-4, or a camera image on the video switching unit 206-5. It is used for the selection control of the screen to be displayed.

Operations of the map display unit 206-3, the buzzer control unit 206-4, and the video switching unit 206-5 will be described with reference to FIG.
The map data 303 stored in the HDD 23 is read and stored as a map image (1) on the memory 202. For the map image (1), the number of cameras / display position information and the button name / display position information stored in the HDD 23 are read, and the map display switch button 301, camera icon 302, alarm are stored in the map image (1). A confirmation button 304 and a buzzer stop button 305 are added and saved on the memory 202 as a map image (2).
Based on the alarm status of each camera stored in the memory 202 by the alarm receiver 206-2 (previous and current intruder / intruder present / not present), a map display switching button 301, an alarm confirmation button 304, It has a function of changing the color of the buzzer stop button 305.
In this known example, the color of the map display switching button 301 is “green” when there is no intruder / intruder, and the color of the map display switching button 301 is “red” when there is an intruder / intruder. The map display switching button 301 in the place corresponding to the image 303 (1F to 3F and the outer periphery) sets the color to “white”.
When there is no intruder or intruder, the color of the camera icon 302 is “blue”, and when there is an intruder or intruder, the color of the camera icon 302 is “red”, and the display camera image displayed on the monitor unit 210 The camera corresponding to 307 sets the color of the camera icon 302 to “orange”.
The alarm confirmation button 304 and the buzzer stop button 305 are normally set to “light yellow”, and when the state changes from “no alarm” to “alarm present”, the color of the alarm confirmation button 304 and the buzzer stop button 305 is “red”. To "".

Even when there is an alarm, when the monitor operates the pointing device 212 and selects the alarm confirmation button 304 or the buzzer stop button 305, the normal “light yellow” is set. Thereby, a display map image 306 to be displayed on the monitor unit 210 is generated. Note that the pointing device 212 only needs to allow the user to arbitrarily specify buttons on the screen, and generally a mouse or a touch panel is often used.
In the buzzer control unit 206-4, the alarm state of each camera stored in the memory 202 by the alarm receiving unit 206-2 (previous and present / no intruder / intruder) and the buzzer stop button 305 are displayed. The state is determined, and the buzzer 211 is controlled to be sounded and stopped.
Once in the alarm state of each camera: No intruder / intruder, and present: Intruder / intruder, and buzzer stop button 305 is in normal state (button color is yellow) Control is performed so that the buzzer 211 sounds in response to the buzzer output I / F 207. When the buzzer stop operation (operates the pointing device 212. When the buzzer stop button 305 selects the alarm state (button color is red)), the buzzer output I / F 207 stops the buzzer 211 from sounding. Control.

The video switching unit 206-5 switches the display camera video 307 to the camera selected by the monitor when the monitor's display camera video 307 is switched (operating the pointing device 212 and the camera icon 302 is selected).
When all video images are stored in the memory or HDD 23 by the video receiver 206-1, the display camera video 307 stored in the memory or HDD 23 may be read, and only the camera of the display camera video 307 is stored in the memory. Alternatively, when the image is stored in the HDD 23, the display camera image 307 is read after the switching of the camera received by the image receiving unit 206-1 and the image reception.
The video switching unit 206-5 is in a state where an alarm has not occurred (the alarm confirmation button 304 is in a normal state (button color is yellow)) and a new alarm has occurred (in the alarm state of each camera, one time before : No intruder or intruder and currently: intruder or camera with intruder) is also controlled to switch the display camera image 307, and when the alarm confirmation button 304 is pressed, other cameras When an alarm is generated in the above case, control is also performed to switch the display camera image 307 to the image of the camera in which the alarm has occurred.

  Further, in the monitoring field, for example, in camera monitoring in a parking lot or the like, a blind spot may occur in a camera image due to an obstacle with only one camera. In addition, when a plurality of cameras are used to cover the blind spot, a monitor for displaying monitoring images as many as the number of cameras is required even though there is only one space to be monitored. In general, when monitoring images from a plurality of cameras, the image of each camera may be displayed in each area obtained by dividing one monitor screen into 16 parts, or may be displayed sequentially by time division, and may be monitored by one monitor. However, even if the number of monitors is reduced, the number of information to be monitored increases by the number of divided screens, so that there is a burden on the observer who looks at the monitor as there are multiple monitors.

JP 2005-117580 A JP 2008-5450 A JP 2000-33279 A JP 2011-151594 A JP 2006-148221 A

There are four problems with the background art described above.
The first problem is that when a plurality of intruders / intruders enter the fields of view of a plurality of cameras at the same time, the situation can be confirmed only for the cameras displayed on the monitor. In order to check other cameras, it is necessary to operate the surveillance staff. Depending on the time lag until the surveillance staff operates, there are no intruders or intruders on the camera when they are confirmed. Will occur.
The second problem is that when there are overlapping parts of the field of view on the camera, even a single intruder can generate multiple alarms. It is easy to make a mistake, or alarm confirmation is required for all cameras.
The third problem is that in a medium-scale / large-scale monitoring system, it takes effort to create a monitoring camera installation location map.
The fourth problem is that when a single space is monitored by a plurality of cameras, the number of monitors is the same as the number of cameras, although the overlapping area is monitored.

  The monitoring system according to the present invention includes a video acquisition unit that acquires video captured by a plurality of cameras, an object detection unit that detects an object from the video acquired by the video acquisition unit, a video display unit, and the video display unit. A video display control means for displaying video on the video display control means, the video display control means comprising: a video reception unit, an alarm reception unit, a map display unit, a buzzer control unit, a video switching unit, a coordinate conversion unit, and a map creation unit; The video display control means has a first output obtained by superimposing a camera arrangement image and a perspective three-dimensional object image on a planar map image, and an alarm receiving unit. A second output in which an image representing an intruder or an intruder, an intrusion locus, and a camera arrangement image are superimposed on a planar map image when an alarm is received is provided.

  The monitoring system of the present invention is the above-described monitoring system, wherein the video display control means further includes a CG selection unit, and the video display control means adds the camera arrangement image and the perspective three-dimensional object image to the planar map image. A first output superimposed, and a second output in which a CG image representing an intruder or an intruder, an intrusion locus, and a camera arrangement image are superimposed on a planar map image when an alarm is received by the alarm receiver. It is characterized by.

Furthermore, the monitoring system of the present invention is a monitoring system having at least two cameras and an image processing device that performs image processing on the camera image, and the image processing device includes a plurality of marker processing units, an overhead coordinate conversion unit, and an overhead view. It has an image creation unit and an overhead view image synthesis unit, the camera captures the same marker, the marker has front, back, left and right identifiable patterns, the marker processing unit detects the marker from the camera image, and the overhead view coordinate conversion unit Performs a predetermined rotation on the camera image based on the marker,
The overhead image creation unit that creates the overhead image from the camera image calculates the camera parameter of the camera height and the angle with the ground where the camera is installed from the length and inclination of two or more sides of the marker, and calculates the calculated camera parameter Is converted into an image from directly above as seen with a virtual overhead camera, and if it is an object of the same size, an overhead image is created so that it will be the same size no matter where it is placed on the screen, The image combining unit combines a plurality of overhead images into one image.

According to the present invention, since a plurality of cameras can be monitored with a small number of monitors without switching operation, it is possible to provide a monitor display that reduces the burden on the supervisor.
Also, by converting the camera image into a bird's-eye view image, the entire space to be monitored can be grasped with one monitoring image.

The block diagram of the monitoring system which concerns on the prior art and the Example of this invention. The block diagram explaining the function of PC105 for monitor display of the conventional monitoring system. The figure explaining the monitor display of the conventional monitoring system. The block diagram of the object detection part 102 of the conventional monitoring system. The functional block diagram of the object detection part 2 of Example 1 of this invention. FIG. 3 is a functional block diagram of a monitor display PC 5 according to the first embodiment. The monitor screen figure of the bird's-eye view display by PC5 for monitor display. The monitor screen figure of the bird's-eye view display by PC5 for monitor displays. The monitor screen figure of the bird's-eye view display by PC5 for monitor display provided with the face recognition function. The monitor screen figure of PC5 for monitor display which displays an object by CG. The figure explaining the virtual overhead view image in the monitoring system of Example 2 of this invention. The figure for demonstrating the relationship between the height of a camera, and distance. The figure which looked at FIG. 12 from right above. The figure for demonstrating the relationship between an image pick-up element and an image distance. In the monitoring system of Example 2, it is a figure which shows the state which put the marker on the road. The block diagram of the marker process part 2200 of the monitoring system of Example 2. FIG. FIG. 6 is a block diagram of an image processing apparatus 2000 of the monitoring system according to the second embodiment. The figure explaining arrangement | positioning which image | photographs a marker with a some camera, and produces | generates an overhead image. The block diagram of the bird's-eye view image synthetic | combination part 2005 of the monitoring system of Example 2. FIG. The figure which shows the other design of the marker used with the monitoring system of Example 2. FIG.

  The monitoring system of each embodiment of the present invention maps a camera video and displays it on a screen so that a monitoring area such as a building is seen through a bird's-eye view. With the mashup method, an online map or the like provided by a third party can be used as a map image of the monitoring area. In addition, the detected intruder image is combined and displayed in a size enlarged in accordance with the detection position. From the image of the intruder, individuals are identified by video content analysis (VCA) technology, and the intruder is located and tracked each time it appears on the camera.

  The monitoring system according to the first embodiment includes five cameras 1, five object detection units 2, five video distribution units 3, LAN_SW 4, and a monitor display PC 5. That is, the configuration at the system level is the same as that of the conventional system shown in FIG.

(Function of the object detection unit)
FIG. 5 is a functional block diagram of the object detection unit 2 of the monitoring system according to the first embodiment. The object detection unit 2 includes an image input I / F 21, an image memory 22, a work memory unit 23, a CPU 204, an image output I / F 25, a program memory 26, a buzzer output I / F 27, a LAN_I / F 28, and a data bus 29. ing. The object detection unit 2 is similar in configuration to the conventional object detection unit 102 in hardware, but the program stored in the program memory 26 is significantly different from the conventional one.
The program memory 26 includes an image capture unit 31, a background creation unit 32, a difference processing unit 33, a binarization processing unit 34, a labeling unit 35, a tracking processing unit 36, a determination processing unit 37, an object type determination unit 38, and a display The function module includes an object region extraction unit 39, a world coordinate calculation unit 40, a display trajectory creation unit 41, and a display data creation unit 42.

The image capturing unit 31 captures the video of the camera 1 using the image input I / F 21 to convert the input image into, for example, an input image composed of luminance signals of 256 gradations from 0 to 255. Save in the memory 22.
The background creation unit 32 generates a background image without an object by, for example, a weighted average while capturing a certain number of input images stored in the image memory 22 and stores the background image in the image memory 22. This intruder / intruder detection is based on the assumption that the object exists for a short time.

The difference processing unit 33 calculates a difference for each pixel between the background image and the input image stored in the image memory 22 and stores the difference image in the image memory 22.
Note that the difference for each pixel of the background image and the input image is an absolute value difference (| input image−background image |) for extracting a changed portion, or a difference (input image−) for extracting a portion that is brighter than the background. Background image) and a difference (background image-input image) for extracting a portion darkened with respect to the background, which is selected depending on the application.

The binarization processing unit 34 performs threshold processing on the difference image stored in the image memory 22, and, for example, if the gradation of the luminance value is 256 levels, a pixel whose difference value is less than the predetermined threshold is determined as luminance. A binary image having a luminance value of 255 for a pixel having a value of 0 and a threshold value or more is generated and stored in the image memory 22.
The labeling unit 35 numbers the white pixels obtained in the binary image for each block, calculates the number of objects, stores the labeling image in the image memory 22, and applies a feature amount to the labeling image. Extraction processing is performed, and information on the area, width, height, and foot position of each object is obtained.

  In the tracking processing unit 36, the information on the foot position obtained by the labeling processing unit 35 is compared between the past and the present, and the object having the closest foot position is correlated as the same object, thereby tracking the object. Appearance time, stay time, trajectory information, and elapsed time after disappearance (0 if no disappearance, trajectory information is deleted when the disappearance time exceeds a certain value) are calculated for each object.

The determination processing unit 37 uses the foot position information, area, width, and height obtained by the labeling processing unit 35 and the stay time and trajectory information for each object calculated by the tracking processing unit 36 to set detection conditions set in advance. If the detection conditions are met, it is determined that there is an intruder / intruder, the intruder / intruder flag, and the intruder / intruder foot position information (screen The X, Y position), width, and height are stored in the work memory 23.
The detection condition set in advance by the determination processing unit 37 is detected when a virtual area is set on the screen and an object enters the virtual area or when an object exists in the virtual area for the duration of stay or longer. . In addition, a virtual line is set on the boundary line, that is, the screen, and detection is performed when the virtual line crosses the virtual line in the designated direction. Furthermore, the size of the object may be taken into consideration.

  The object type determination unit 38 generates object type information necessary for selecting the object image 1001 created by CG on the monitor display PC 5. The object image 1001 may be an icon or illustration with good visibility, details of which will be described later. As the object type, a person, a car, a person on a two-wheeled vehicle, and other objects are used.

Note that the determination processing unit 37 calculates an object type for an object determined to be an intruder or an intruder.
As a method for determining the object type, there are the following classification methods.
Classification based on size determination is based on the width (pixel) and height (pixel) of the object, the actual object width based on the camera installation conditions (camera height, depression angle, focal length, CCD size) and the foot position of the object. (M) and height (m) are calculated and classified into four types: human, person riding a motorcycle (bicycle, motorcycle), car, and others.
The classification based on speed is based on the size determination, and the latest and oldest trajectory information (pixels) is determined for the object that is determined to be a person or a person riding a motorcycle. , The depression angle, the focal length, and the CCD size are converted into the actual distance (m), and the speed is calculated from the time difference between the latest and oldest trajectory information, thereby classifying the person and the person on the motorcycle.

As a method of improving the classification accuracy determined by the classification based on the size determination and the classification based on the speed, there is a method of checking the similarity of the contour lines of the object.
First, the outline of the object is extracted by edge extraction processing such as a Laplacian filter around the position where the object exists, and the extracted outline of the object, a human outline model prepared in advance, By determining which contour model or contour model of a person riding a motorcycle has the most similarity, it is classified into a person, a car, or a person riding a motorcycle, and there is no similarity in any model Objects are classified as other.
Here, in order to see the similarity, the contour model of the person prepared in advance, the contour model of the car, and the contour model of the person riding the two-wheeled vehicle are determined by the position of the object (distance (m) from the camera). Enlarge or reduce to eliminate the effect of object size change. In addition, by calculating the similarity for all the range around the position where the object exists, the binary image obtained by the binarization processing unit 34, the position, width, and the like of the object obtained by the labeling processing step 35 are obtained. Even when the height is detected to be smaller than the actual object, the classification accuracy can be improved.

The display object region extraction unit 39 extracts an object image 803 to be displayed on the monitor display PC 5. The object image 803 will be described later with reference to FIG.
In general, when the binary image obtained by the binarization processing unit 34 or the position, width, and height of the object obtained by the labeling processing unit 35 is used, it is affected by the contrast of the shadow / object. There is a problem that a part of the object is missing or too large, and the accuracy is low.
Therefore, the foot position, width, and height of the obtained object are expanded according to the object type obtained by the object type determination unit 38 (for example, in the case of a person, the foot position, width, and height are increased to 50 cm of the object width). The height of the object is expanded by 1.7 m), and edge extraction is performed with a Laplacian filter or the like within the expanded range, and the range according to the object type + specified range (centered around the portion where the edges are most concentrated) For example, by cutting out a range of 5%), cutting out the range of the object.
Further, the influence of the shadow is obtained by determining the correlation between the background image stored in the image memory 22 and the input image, and determining that the portion with high correlation is other than the object and the portion with low correlation is the object. It is possible to reduce the influence of shadows by performing the above-described processing on a given range.

The world coordinate calculation unit 40 converts the foot position of the object calculated by the display object region extraction unit 39 into a distance from the camera using camera installation conditions (camera height, depression angle, focal length, CCD size). By converting and adding the preset camera position information (relative camera position (unit is meter) when the upper left point of the map in the site is (0,0)) The position information of is calculated.
The display trajectory creation unit 41 converts the trajectory information (pixels) obtained by the tracking processing unit 36 into a distance from the camera using the camera installation conditions in the same manner as the world coordinate calculation unit 40, and in advance. The actual position information on the site is calculated by adding the set camera position.
Here, the tracking processing unit 36 is used for the trajectory, but the accuracy can be improved by performing the processing of the display object region extraction unit 39 before the tracking processing unit 36.

  The display data creation unit 42 includes the object type information obtained by the object type determination unit 38, the object image 803 obtained by the display object region extraction unit 39, and the actual site of the object obtained by the world coordinate calculation unit 40. The location information above and the actual site trajectory information obtained by the display trajectory creation unit 41 are collected as a single data, for example, a status message or a file, and the collected data is sent to the monitor display PC 105 via the LAN_I / F 201. Sent to. Since the object image 803 can occur irregularly, it is difficult to apply inter-frame predictive coding, and the object image 803 is compressed using JPEG or intra-predictive coding, which has the effect of suppressing display delay. There is also.

(Monitor display PC functions)
FIG. 6 shows a functional block diagram of the monitor display PC 5 in the first embodiment.
The monitor display PC is a means for controlling the display of the camera video. The PC 5 in this example uses the function of deleting the duplicate object based on the position information of the object, the face extraction and the face authentication function, and the duplicate object. And a function of superimposing the object image (face) 1201 on the monitor display image 801.

First, a function for deleting an overlapping object based on the position information of the object will be described with reference to FIGS. 6, 8, and 9. FIG.
In the monitor display PC 5 shown in FIG. 6, the configuration according to reference numerals 21 to 29 is the same as 201 to 209 in the conventional monitor display PC 105 except for 26, and thus the description thereof is omitted. The program memory 26 is different from the conventional one in that it includes a coordinate conversion unit 51, a map creation unit 52, a position determination unit 53, an object superimposing unit 54, and a trajectory superimposing unit 55.

  The coordinate conversion unit 51 is a commercial map stored in the HDD 23, an aerial photograph, a satellite photograph, or an online map provided by Google Inc. Is converted into an image (bird's-eye display / overhead display) from a predetermined viewpoint in the perspective projection method of FIG. The bird's-eye view display is a display as if the ground surface is viewed obliquely downward as shown in FIG. 9, and is obtained by coordinate-transforming the original map image using a perspective transformation generally used in CG or the like. It is done. Let this map image 801 be the lowest layer (layer 0). A building floor diagram (bird's-eye view) 802 in which a skeleton of the building is drawn with a wire frame model is superimposed on the perspective view and viewpoint corresponding to the map image 801. That is, the building floor view (bird's eye view) 802 is drawn on layer 1 above layer 0. Initially, the skeleton information of the building is generated as a polygon from the outline of the building in the map image 801 and is pushed upward by a given number of floors and generated. On the other hand, the bird's-eye view display is a display as if the ground surface was looked down from directly above as shown in FIG. 8, d, an orthographic projection method, or the like is used.

  The map creation unit 52 includes camera position information (screen) on the actual site stored in the HDD 23 on an image obtained by superimposing a building floor diagram (bird's eye view) 802 on the map image 801 created by the coordinate conversion unit 51. The camera icon 302 is superimposed on the position designated by the camera position (m) when the upper left is (0, 0), and the alarm confirmation button 304 and the buzzer stop button are placed at the designated position on the screen. 305 is superimposed. That is, the camera icon 302 and the like are drawn on the layer 2.

  The position determination unit 53 determines the overlap of objects based on the actual position / trajectory information on the site transmitted from each of the object detection units 2. The actual position / trajectory information on the site obtained from the plurality of object detection units 2 is compared and collated, and when the error is within a specified value (for example, 5%), it is determined that they are the same object. If it is determined that the objects are the same, the position information of the object image to be displayed with the highest resolution (number of pixels) remains, and the position information with the lower resolution is deleted.

Based on an instruction from the position determination unit 53, the object superimposing unit 54 transmits the object region image received from the object detection unit 2 as the object image 803 (803-1, 803-2) from the object detection unit 2. Superimpose the actual location information on the site with 0 transmittance. That is, drawing is performed on the uppermost layer 4. At this time, the position of the foot of the person indicated by the object image 803 corresponds to the actual position, but the size is not changed. In other words, since the object image 803 is configured with the number of pixels when the image is taken by the camera, the object is displayed as an image that is different (large) from the scale on the actual map, and provides an object image that is easy for the observer to view. be able to.
However, when the size of the object image 803 received from the camera 1 is equal to or smaller than the specified value, the object superimposing unit 54 performs a complementing process such as an enlargement process + bicubic complementation to reduce the image quality and is easy to see. You may enlarge it. In addition, when superimposing the object image 803 detected in the building, a balloon figure is drawn for each floor of the building floor diagram (bird's eye view), and photographed with a camera installed on each floor in the balloon. The object image 803 may be displayed side by side.
The trajectory superimposing unit 55 draws the trajectory 804 on the layer 3 based on the actual trajectory information received from the object detection unit 2 based on the instruction from the position determination unit 53 and superimposes it on the map image 801.

  FIG. 8 is an example of a bird's eye view display of a monitor screen by the monitor display PC 5. The two monitor units 210-1 and 210-2 connected to the monitor display PC 5 are installed side by side so that the map image 801 is continuously displayed on the entire screen so as to straddle the monitor unit. . In the bird's-eye view, since the display height is compressed and a horizontally long display screen is obtained, such a multi-display configuration is preferable. The bird's-eye / overhead switching button 808 switches between bird's-eye and bird's-eye, and since FIG. 8 is in a bird's-eye view, an icon indicating the bird's-eye view of the switching destination is displayed. When the pointing device 212 is a mouse, the direction of the line of sight can be changed by dragging while pressing the right button on the map image 801, and the map image 801 can be slid by dragging while pressing the left button, that is, the point where the line of sight intersects the ground surface. Can be changed. The map image 801 is displayed so that the point is the center.

  FIG. 9 is an example of a bird's-eye view display of the monitor screen by the monitor display PC 5. The two monitor units 210-1 and 210-2 connected to the monitor display PC 5 are installed side by side so that the map image 801 is continuously displayed on the entire screen so as to straddle the monitor unit. The bird's-eye / overhead switching button 808 switches between bird's-eye and bird's-eye, and since FIG. 8 is in a bird's-eye view, an icon indicating the bird's-eye view of the switching destination is displayed. When the pointing device 212 is a mouse, the direction of the line of sight can be changed by dragging while pressing the right button on the map image 801, and the map image 801 can be slid by dragging while pressing the left button, that is, the point where the line of sight intersects the ground surface. Can be changed. The map image 801 is displayed so that the point is the center.

  Next, with reference to FIGS. 6 and 9, a function for deleting an overlapping object using the face extraction and face authentication functions and a function for superimposing an object image (face) 1201 on the monitor display image 801 will be described. This function is mainly realized by the face authentication unit 56 and the face superimposing unit 57 in the monitor display PC 5 of FIG.

The face authentication unit 56 detects a human face area in the video using the face detection technique for the video received from the video distribution unit 103 and stored in the memory 22 or the like, and the image feature amount of the area is detected. And is recorded in the HDD 23 together with face image data cut out from the area, time, position information, and the like. A well-known Joint Haar-linke feature can be used as a discriminator used for face detection or a feature amount to be extracted. In addition, when detecting not only a person's face but a whole body image, CoHOG (Co-occurrence Histograms of Oriented Gradients) etc. can be utilized. When it is difficult to detect a face or a human figure in real time from the entire video, priority is given to detection inside the object image 803, and processing is performed by thinning out the frame if there is remaining power. As the face position information, the position information of the corresponding object image 803 can be used as it is.
Then, using the faces just detected one by one as a key, search for a face having a similar feature amount from the other faces detected now and the face images recorded in the HDD 23 in the past. Collate. When it is determined that they are the same, an object ID, trajectory information (the past position information of the person recorded in the HDD 23), and the like are returned. Here, the same object ID is assigned to the same person.

The face authentication unit 56 has the highest resolution or the score (face-likeness or detection reliability) for the person who is determined to be the same at almost the same time (and the same place). Only the highest image is shown and notified to the position determination unit 53 and the face superimposing unit 57, and redundant alarms such as the ID of the camera that captured the low-resolution person and the position information of the person are displayed. Information that can be associated is notified to the buzzer control unit 206-4 that manages the alarm state. This eliminates redundant alarms. Receiving the notification, the position determining unit 53 controls the object superimposing unit 54 so that the locus by the face authentication is displayed with priority.
The face authentication unit 56 also estimates that the same person has moved (that is, one intrusion event) to the person who is determined to be the same even if the time and place are slightly different, and the older one Overwrite alarm with new alarm. The overwriting mentioned here includes various methods that can integrate these alarms, for example, using the same ID as the old alarm reception ID for the new alarm. The HDD 23 holds a trajectory after disappearance for a longer time than the tracking processing unit, and the trajectories that have been interrupted can be connected.
The face authentication unit 56 stores one or a predetermined number of images having the highest resolution and score for the persons determined to be the same, and stores them in the HDD 23 together with the integrated trajectory information by adding new trajectory information. For a person who is not recorded (that is, for the first time), the current face image, object ID, position information from the object detection unit 2 and trajectory information (if any) are stored in the HDD 23. The HDD 23 may store a face of a person (for example, an employee) that does not need to be tracked or displayed on the monitor screen as a white list. In this case, the face authentication unit 56 does nothing even if it matches the person in the white list.

The face superimposing unit 57 superimposes the object image (face) 1201 on the map image 801 using the face image obtained by the face authenticating unit 56 and the actual location information in the site, and the trajectory information recorded in the HDD 23. Is used to superimpose the locus 804-1 on the map image 801. This superposition is maintained at the last detected position for a predetermined time after the face is not detected unless the user performs a special operation. If tracking of the tracking processing unit 36 continues even if the face cannot be detected, the updated object image 804 can be displayed at the latest position.
When a face is detected in a building or the density of faces detected in a certain area is high, a balloon is created for each floor or area instead of superimposing the object image (face) 1201 on the correct position. The object image (face) 1201 may be displayed side by side in the balloon, and the locus may be displayed only when a specific object image (face) 1201 is selected.
By performing the processing described above, the monitor display of FIG. 9 can be realized. To eliminate duplication, there is a method of combining with a physical security system using RFID (Radio Frequency IDentification) or the like in addition to face authentication.

  Next, a monitor display PC 105 having a function of displaying an object image in CG will be described with reference to FIG. 1, FIG. 6, FIG. 7, and FIG. This function is mainly realized by the CG selection unit 58 of FIG. The other configuration of the monitor display PC 105 in FIG. 6 is the same as that described above, and thus the description thereof is omitted.

  The CG selection unit 58 refers to the object type information calculated by the object type determination unit 38 of the object detection unit 2, reads an object image (CG) 1001 corresponding to the object type from the HDD 23, and reads a map image 801 or a building floor diagram. (Bird's-eye view) Superimposed display is performed by drawing on a layer above 802. That is, in FIG. 7, instead of the actual object image 803, a prescribed icon is displayed. In this way, it is not necessary to perform the video decoding process in the video receiving unit 26-1 only to obtain the object image 803, and the object image (CG) is used when the number of object images 803 is large or unclear. 1001 may be superior in visibility and the amount of information that can be grasped. For example, the color or the like can be changed according to the degree of suspiciousness. If the object image (CG) 1001 is a color that is not often used in the map image 801, it can be visually conspicuous, and the number of objects can be grasped from the area of the color.

  However, when the object image is set to CG, it is tempting to see the video of the actual camera. Therefore, when the object image (CG) 1001 is selected by the pointing device 212, the video switching unit 26-5 displays the image of FIG. The display is switched from the display as shown in FIG. That is, in FIG. 10, the monitor unit 210-1 has a map image 801, a building floor diagram (bird's-eye view) 802, a camera icon 302, an object image (CG) 1001, and a trajectory 804 only in a portion where an object exists. The selected camera image is displayed on the monitor unit 210-2 without any coordinate conversion. Further, a display switching button 1101 is displayed on the monitor unit 210-1. When the user presses the display switching button 1101, the video switching unit 26-5 returns the display mode to the linked display as shown in FIG.

  As described above, since the method of displaying an object image in CG can be superimposed on the building floor diagram (bird's eye view) for the number of people, it is useful in a monitoring place where there are many buildings. It is. By displaying the object image in CG, even if the face is shown large, it can be easily provided as an evidence image to the outside in a privacy-protected form. In addition, even when only a part of an object is visible due to crowds of people, it is possible to provide an easy-to-view video.

  The monitoring system according to the second embodiment includes an image processing apparatus 2000 that calibrates a camera by imaging a marker (marker) provided in a monitoring area with a plurality of cameras, and other configurations are implemented. This is the same as the monitoring system of Example 1.

  FIG. 11 is a diagram illustrating an overhead image generally obtained by a surveillance camera. In order to capture an image over a wide range, when installing the camera 141, it is common to install the camera 141 so that the depression angle is shallow (slightly downward from the horizontal) and the depth direction of the road 143 can be seen. However, in the visual field 142 that can be seen by this camera, the distance on the screen appears to decrease as it goes in the depth direction (upward on the monitor screen when the camera image 144 is viewed on the monitor). In other words, the scale on the screen is different between the case where the car 140 is traveling in front and the case where it is traveling in the back, so if there is no other information to know the distance, the distance from the camera to the car is grasped. It becomes difficult.

Therefore, the distance is easily grasped by converting to a virtual overhead image 155 as shown in FIG.
The image captured by the camera 141 is converted into an image as if captured by the camera from directly above by image processing. Here, the virtual camera located directly above is referred to as an overhead camera 145. The bird's-eye view 146 photographed by this bird's-eye camera 145 has a constant scale at any position as if it was cut out in a rectangular shape on the map, making it easy to grasp the position of the car 140 traveling on the road 143. Hereinafter, an image in the overhead view field 146 photographed by the virtual overhead camera 145 is referred to as an overhead image 148.

Next, camera installation conditions (hereinafter referred to as camera parameters) necessary for creating the overhead image 148 from the camera image 144 will be conceptually described with reference to FIGS.
FIG. 12 shows a state in which the camera 141 is installed on the camera column 166 at a height H as viewed from the side. At this time, the camera 141 is installed downward toward the road 143, and the optical axis passing through the center of the camera 141 captures the center of a car separated by a distance X at an angle θ. At this time, the relationship of the following formula 1 holds for H, X, and θ.

X = H × tan (θ) (1)
However, it is assumed that the camera column 166 and the road 143 form a right angle. When this equation is modified, the following equation 2 is obtained.
θ = atan (X / H) Equation 2
Equation 2 indicates that if the distance X to the object reflected at the center of the screen of the camera 141 and the installation height H of the camera are known, the angle θ at which the camera is facing can be found.

Further, FIG. 13 shows a state in which the camera 141 and its subject are viewed vertically downward from above.
As an example, it is assumed that the subject a 173 and the subject b 174 are located at distances Xa and Xb from the principal point of the lens 172, respectively, and the subject a 173 is closer to the subject b 174. The optical axis of the camera intersects the center of both subjects at right angles. It is assumed that when the distance from the end of the subject a 173 to the optical axis is the length of Ya, the light from that end passes through the lens 172 mounted on the camera 141 and reaches the end of the image sensor 171. The distance (image height) from the end of the image sensor 171 to the optical axis is assumed to be Za. Light rays that reach the image sensor 171 from the end of the subject a 173 through the principal point of the lens 172 make an angle of φa with respect to the optical axis. The distance Yb from the optical axis related to the subject b 174 and the image sensor 171 and the angle φb from the optical axis at the focal position are the same as the subject a 173.
Further, when the size of the subject a 173 and the size of the subject b 174 is the same, the distance from the optical axis to the end of the subject surface has the relationship of Equation 3.

Ya = Yb Equation 3
Although the two subjects have the same size, but have different distances from the camera 141, the angle θb of the light that enters the lens 172 from the end of the subject b174, which has the relationship shown in Expression 4, is smaller than the angle θa of the subject a173. .
θa> θb Equation 4

FIG. 14 is a diagram for explaining the relationship between the image sensor and the focal length.
In FIG. 14, the camera 141 captures the subject a173 and the subject b174, and φa and φb in the figure are the same as the angles described in FIG. However, since the path of light after passing through the lens 172 is shown, the angles φa and φb are shown upside down. In general, the distance on the optical axis from the focal point 183 in the lens 172 to the image sensor 171 is defined as a focal length f182. The light emitted from the end of the subject a 173 reaches a position on the surface of the image sensor 171 at a distance Za from the optical axis. Similarly, light emitted from the end of the subject b174 reaches a position away from the optical axis by a distance Zb. In this example, the light that exits the end of the subject a9 reaches the end of the image sensor 7. When the image sensor 7, the subject a 173, and the subject b 174 are perpendicular to the optical axis and the optical axis is the center of the subject, the distance Za and the image sensor size 181 are in the relationship of Equation 5.

Image sensor size = Za x 2 Equation 5
Looking at FIG. 14 for the subject b174, the light exiting the end of the subject b174 reaches the image sensor 171 at an angle φb. The light that reaches the image sensor 171 from the relationship of Equation 4 is at a distance Zb from the optical axis, and is in the relationship of Equation 6.
Za> Zb Expression 6
The subject a 173 and the subject b 174 have the same size as shown in Equation 3, but the subject b 174 is smaller on the image sensor 171 as shown in Equation 6. This is because the distance between the two subjects is different, and this is why the farther away the object looks, the smaller.

Here, FIG. 13 and FIG.
Xa / Ya = f / Za Equation 7
Xb / Yb = f / Zb Equation 8
When formulas 7 and 8 are transformed into formulas for obtaining distances Xa and Xb, formulas 9 and 10 are obtained.
Xa = Ya × f / Za Equation 9
Xb = Yb × f / Zb Expression 10
Here, f is an image distance (distance from the principal point of the lens to the image plane), which is given by the lens 172 to be used, and Za and Zb are values that can be measured on the photographed image. And Yb, the distances Xa and Xb to the subject can be obtained.
Therefore, if the subject is on the optical axis when viewed from the side, the distance to the subject is obtained, and a camera that captures a subject whose size is known by substituting the distance into X in Equation 2 The angle θ of 141 is obtained.

In the second embodiment, an angle θ or the like, which is a camera parameter of the camera 141, is obtained by shooting a subject whose size is known on the ground or floor of the monitoring area. Hereinafter, the subject is referred to as a marker.
FIG. 15 is a diagram illustrating a state where the marker 192 is placed on the road 143. The marker 192 is a square or a rectangle whose size is known, and the direction can be recognized from the pattern on the surface. For ease of explanation, the case where the side of the marker is orthogonal to the optical axis is illustrated.

In FIG. 15, a camera image 191 shows a road 143 and a marker 192. Since the upper direction of the monitor screen 191 corresponds to the depth, the upper side of the marker 192 appears shorter than the lower side. This corresponds to the relationship of Equation 6. When the length of the lower half of the marker 192 on the image plane is Za and the length of the upper half is Zb, the distances Xa and Xb from the camera 141 to the upper side and the lower side are obtained by Expressions 9 and 10. I can do it. If the installation height H is given, the angle θ of the camera 141 can be approximately calculated by Expression 2 where X is the average value of the distances Xa and Xb.
To obtain camera parameters under more general conditions, the methods described in Patent Documents 4 and 5 may be used.

FIG. 16 shows a configuration of a marker processing unit 2002 that calculates camera parameters in the second embodiment.
A marker 192 is shown in the image 2201 from the input camera. In order to find the marker 192, the straight line detection unit 2202 obtains a straight line component in the image using Hough transform or the like. Since the marker 192 has a predetermined size, the straight line is picked up by narrowing down to a straight line having a length about the size of the marker shown in the image. The obtained straight line coordinates are output to the subsequent marker position detection unit 2203.
The marker position detection 2203 picks up a closed straight line portion that is a square (which appears to be deformed into a trapezoid because the depth is reduced) among the obtained straight lines. Then, the coordinate position of the camera image at the center position of the marker 14 is stored.
Further, the marker inclination calculation unit 2204 in the subsequent stage can determine the distance from the camera of each of the upper side and the lower side of the marker 192 from the ratio of the length of the upper side and the lower side of the marker 192 that appears to be deformed into a trapezoid. There is no problem even if this straight line appears to be inclined, and the center position of the straight line may be used as the starting point of the distance.
The camera parameter calculation unit 2205 obtains an angle θ of the camera based on the distance from the camera on the upper side and the lower side of the marker 192 and outputs it as information for use in overhead coordinate conversion. The obtained camera parameters can be notified to the world coordinate calculation unit 40 by bidirectional communication with the object detection unit 2.

Once the camera parameters are determined, the camera image 191 can be converted into an overhead image 193. The upper side and the lower side of the square marker 192 are originally the same length, and appear in the same length in the overhead image 191. In the camera image (a), the object was shrunk as it went back, but by converting it to a bird's-eye view image, the shrinkage to the depth disappears and the distance in the depth direction becomes easy to understand.
For example, a 1 m object shown in the foreground and a 1 m object shown in the back are displayed in the same size in the overhead image (b). That is, in order to create a bird's-eye view image, an object that is originally the same size may be projected on the monitor screen 191 so as to have the same size.

FIG. 18 is a diagram for explaining the connection of images obtained by photographing the same area with a plurality of cameras.
The camera A visual field 2104 captured by the camera A 2103 has an obstacle 2105, and a camera A blind spot 2106 is generated. In order to take an image of the blind spot that cannot be seen, the camera B 2101 is installed and the camera B visual field 2102 covers the area of the camera A blind spot 2106.
When the camera is installed as shown in FIG. 18, the marker 2107 is placed at a position where the camera A visual field 2104 and the camera B visual field 2102 overlap. Based on the marker 2107, the image of each camera is converted into an overhead image. After converting to a bird's-eye view image, the images from each camera are combined into a single image. Since the pattern of the marker 2107 can be seen, it is synthesized after affine transformation (resizing and rotating) so that the marker 2107 is exactly overlapped. Since the synthesized overhead image is displayed on the monitor 210, the monitor can monitor a plurality of camera images as one image.

FIG. 17 shows a block diagram of an image processing apparatus 2000 that performs the above-described image conversion and composition.
Since the image processing apparatus 2000 receives inputs from a plurality of cameras, the same processing is provided in parallel inside, but the processing common to each camera is representative of the processing of the camera image 2001-1 input from the camera number 1. To explain.
The camera image 2001-1 is input to the marker processing unit 2002-1 described with reference to FIG. 16, camera parameters are calculated from the marker image information, and are output to the downstream overhead coordinate conversion unit 2003-1.

The overhead coordinate conversion unit 2003-1 is an image from directly above as seen by a virtual overhead camera based on the camera parameter (camera angle) calculated by the marker processing unit 2002-1 on the input camera image 2001-1. Therefore, which pixel is associated with which part of the overhead image is calculated and output to the overhead image creation unit 2004-1.
The overhead image creation unit 2004-1 arranges and outputs each pixel to the coordinates after the overhead image conversion calculated by the overhead coordinate conversion unit 2003-1. The image taken by the camera has shrunk as it goes on the monitor screen 2006. However, by converting to overhead coordinates, an object of the same size will have the same size no matter where it is placed on the monitor screen 2006. .
The bird's-eye view image combining unit 2005 combines the bird's-eye view images from a plurality of cameras into a single image and outputs it to the monitor 2006.

FIG. 19 shows a block diagram of the overhead view image synthesis unit 2005.
The input image is an image converted to a bird's-eye view, and the bird's-eye view image is input by the number of cameras. A case where n cameras of 1, 2,..., N are connected will be described as an example.
The bird's-eye view image synthesis unit 2205 adjusts the position of one of the n bird's-eye-view images as a reference bird's-eye image, with the other bird's-eye view images aligned with the reference bird's-eye image. In this example, the bird's-eye view image created from the image of the first camera will be referred to as a reference bird's-eye view image, and the operations other than the reference bird's-eye image will be described using the image of the second camera as an example. There are various means for positioning the marker 192. Here, an example is described in which the marker position is matched by rotating the overhead image until it matches the reference overhead image. To do. As other implementation means, any means may be used as long as it is a pattern matching process used in image processing, such as performing alignment by paying attention to a straight line portion.

The overhead image 2401-2 created from the second camera image is input to the overhead image movement rotation unit 2402-2. The image is moved so that the position of the marker 192 is centered, and the overhead image is rotated at a small angle, for example, at an angle of 1 degree, and is output to the pattern matching unit 2403-2. The center coordinates of the marker necessary for the rotation process have already been obtained by the marker position detection unit 2203 in FIG.
The pattern matching unit 2403-2 compares the overhead image and the first camera, which is the reference overhead image, and the marker image, or compares them in pixel units.
At this time, since the brightness may be different for each camera, a certain degree of brightness variation is allowed. For example, if the brightness of a pixel is 256 gradations, there is a means of comparing with 64 gradations of 1/4. Alternatively, brightness normalization processing may be performed so that the brightness of white and black becomes the same value in both.

The pattern matching result of the pattern matching unit 2403-2 is returned to the preceding overhead image movement rotation unit 2402-2. If the pattern matching results match, the rotated overhead image is output to the overhead image overlay unit 2404 at the subsequent stage.
The bird's-eye view image moving / rotating unit 2402-2 further rotates the bird's-eye view image by a small angle (for example, 1 degree) when the pattern matching unit 2403-2 returns a result that the pattern matching results do not match. The loop that returns this result continues until the pattern matching results match or rotates 360 degrees.
In this way, the first reference bird's-eye view image and the 2nd to n-th bird's-eye view images are all input to the bird's-eye view image superimposing unit 2404, superimposed on one image at the marker position, and output as the bird's-eye view synthesized image 2405. . Information indicating the relative position between the cameras obtained by the pattern matching unit 2403 can be notified to the world coordinate calculation unit 40.

An example of the design of the marker 192 is shown in (a), (b), and (c) of FIG.
The outer periphery of the marker 192 is bordered by a black line, and a white band is formed on the inner side of the marker 192. Further, the marker 192 has a pattern that can be recognized even when the marker 192 rotates. The reason why the inside of the black frame is white is to make it easy to detect the straight line between the upper side and the lower side. The color of the marker 192 does not have to be black and white, but it is easier to simplify the image processing when performing image processing with only the brightness component, and the black and white pattern is handled because it is easy to distinguish even if the lighting is colored. Cheap. In addition, a rectangle may be used as long as each side of the marker 192 intersects at a right angle, but a square is easy to handle in order to perform rotation processing.

As described above, in the second embodiment, the marker 192 reflected in the camera is searched from the image to obtain the camera parameter, and the angle for converting the camera parameter into the overhead image is obtained. A single bird's-eye view image is created by combining the images into one sheet based on the markers 192 of each bird's-eye view image converted according to the obtained angle, and outputting it to the monitor 2006.
Note that the image processing apparatus 2000 can be realized by the monitor display PC 5 or the CPU of the object detection unit 2 without separately providing hardware.
The image input to the apparatus may not be a live video, but may be image data stored in a recording medium such as a hard disk or image data input via a network. The image data may be image data that has undergone image compression processing. In this case, the processing described above may be performed after the image expansion processing. Further, the output destination of the overhead image does not have to be a monitor, but may be a recording medium such as a hard disk or a network.
In addition, images captured by a plurality of cameras may be displayed not only in a bird's-eye view but also in a bird's-eye view as viewed obliquely from above with a camera. A bird's eye view display can be easily handled by simply changing the numerical value of the tilt used when converting the coordinates into an image viewed from directly above. Once all the camera images are converted into a bird's-eye view display and combined into a single image, coordinate conversion processing may be performed on the one bird's-eye view image to convert it into a bird's-eye view display. As a means for creating a bird's-eye view image that displays each floor of a building, this marker may be used or a plurality of camera images may be combined. Furthermore, an intruder or an intruder, its trajectory, and the result of face authentication may be combined with an overhead image or a bird's-eye image obtained by combining images from a plurality of cameras using this marker.

  The present invention can be applied to CCTV (Closed-Circuit TeleVision) and other devices, systems, management software, etc. that display remote images for monitoring purposes, and is also provided as a service for general households and individuals for business purposes. Can be done.

1: camera, 2: object detection unit, 3: video distribution unit, 4: LAN_SW, 5: PC for monitor display,
21: Image input I / F, 22: Image memory, 23: Work memory, 24: CPU, 25: Image output I / F, 26: Program memory, 26-1: Video receiver, 26-2: Alarm receiver 26-3: Map display section, 26-4: Buzzer control section, 26-5: Video switching section, 27: Buzzer output I / F, 28: Monitor output I / F, 29: Data bus,
31: Image capturing unit, 32: Background creation unit, 33: Difference processing unit, 34: Binarization processing unit, 35: Labeling unit, 36: Tracking processing unit, 37: Determination processing unit, 38: Object type determination unit 39: Display object region extraction unit, 40: World coordinate calculation unit, 41: Display trajectory creation unit, 42: Display data creation unit,
51: coordinate conversion unit, 52: map creation unit, 53: position determination unit, 54: object image superimposition unit, 55: locus superimposition unit, 56: face authentication unit, 57: face superimposition unit, 58: CG selection unit,
212: Pointing device, 210-1, 210-2: Monitor unit, 211: Buzzer,
304: Alarm confirmation button, 305: Buzzer stop button,
401: Image input I / F, 402: Image memory, 403: Work memory, 404: CPU, 405: Image output I / F, 408: LAN I / F,
801: Map image, 802: Building floor plan (bird's-eye view), 803-1, 803-2: Object image, 804-1, 804-2: Trajectory, 1001-1: Object image (CG), 1101: Display switch button, 1201: Object Image (face),
2000: Image processing device, 2001-1 to 2001-n: Camera image, 2002-1 to 2002-n: Marker processing unit, 2003-1 to 2003-n: Overhead coordinate conversion unit, 2004-1 to 2004-n: Overhead image creation unit, 2005: Overhead image synthesis unit, 2006: Monitor unit, 2202: Straight line detection unit, 2203: Marker position detection unit, 2204: Marker inclination calculation unit, 2205: Camera parameter calculation unit

Claims (3)

Video acquisition means for acquiring videos taken by a plurality of cameras, object detection means for detecting an object from the acquired videos, video display means, and video display control means for displaying video on the video display means In the monitoring system,
The video display control means has functions of a video reception unit, an alarm reception unit, a map display unit, a buzzer control unit, a video switching unit, a coordinate conversion unit, a map creation unit, a position determination unit, an object superimposition unit, and a trajectory superposition unit. And
The video display control means includes a first output obtained by superimposing the arrangement image of the camera and a perspective three-dimensional object image on a planar map image, and an image representing an intruder or an intruder when an alarm is received by the alarm receiver. And a second output in which the intrusion locus and the arrangement image of the camera are superimposed on a planar map image. The monitoring system according to claim 1,
The video display control means further includes a CG selection unit,
The video display control means includes a first output obtained by superimposing a camera arrangement image and a perspective three-dimensional object image on a planar map image, and a CG representing an intruder or an intruder when an alarm is received by the alarm receiver. A monitoring system comprising a second output in which an image, an intrusion locus, and an arrangement image of the camera are superimposed on a planar map image. The monitoring system according to claim 1, further comprising an image processing apparatus having a plurality of marker processing units, an overhead coordinate conversion unit, an overhead image creation unit, and an overhead image synthesis unit,
The camera takes the same marker,
The marker has an identifiable pattern of front, rear, left and right,
The marker processing unit detects a marker from a camera image,
The overhead view coordinate conversion unit performs a predetermined rotation on the camera image based on the marker,
A bird's-eye view image creation unit that creates a bird's-eye view image from a camera image calculates camera parameters of the height of the camera and the angle with the ground on which the camera is installed from the length and inclination of two or more sides of the marker. Based on the parameters, it is converted to an image from directly above as seen with a virtual overhead camera, and if it is an object of the same size, create an overhead image so that it will be the same size no matter where it is placed on the screen, or Create a bird's-eye view as seen from one diagonally upper point from multiple camera images,
The monitoring system, wherein the overhead image synthesis unit synthesizes a plurality of overhead images or bird's-eye images into one image.

Images