KR100927554B1 - Night video surveillance system and method based on day and night video composition - Google Patents

Abstract

PURPOSE: A night video monitoring system based on day and night video composition and a method thereof are provided to increase understandings about a monitoring area and easily recognize whether an accident occurs, an accident occurring location and an accident situation. CONSTITUTION: A night video monitoring system based on day and night video composition comprises the following steps. A motion mask image is obtained from an inputted video frame(S104). A generated daytime background image and the inputted video frame are synthesized(S105). The synthesized image is an image which replaces pixels with no motion in the input video frame with pixels of a daytime background static image.

Description

Night video surveillance system and method based on day and night video synthesis {NIGHT VIDEO SURVEILLANCE SYSTEM AND METHOD BASED ON DAY AND NIGHT VIDEO COMPOSITION}

The present invention relates to a night video surveillance system and method based on day and night video synthesis, and more specifically, to the night surveillance by properly synthesizing the day and the image acquired at night through a fixed video camera in the same monitoring area A system and method for providing a more effective image.

When monitoring the outdoor surveillance area by using CCTV at night, it is difficult to recognize because the terrain / features of the surveillance area are very dark in the image if there is not enough lighting for the entire surveillance area. Therefore, if the surveillance personnel are monitoring several places at the same time through multiple cameras, it is difficult to recognize which place they are looking at by looking at the screen. In addition, even if an intruder or the like appears in the surveillance area, the terrain and features corresponding to the intruder and the background appear dark on the screen, so it is easy to miss it unless the screen is concentrated. Even if it is recognized that an event has occurred, there is a problem that it is difficult to recognize at which position within the surveillance area in relation to which terrain / feature.

If a lighting device cannot be installed in the surveillance area, an infrared or laser emitter is usually installed next to the camera for night surveillance. When looking at an image acquired in such a situation, it usually appears relatively bright only at the center portion of the image, and it appears darker as it leaves the center of the image, and also comes out as a black and white image rather than color. Therefore, in the case of using a transmitter, it helps to detect moving objects and the like, but it is difficult to solve the above problems.

In some cases, a thermal imaging camera may be used instead of a combination of a conventional CCTV camera and a floodlight. When a thermal camera is used, even if there is no lighting in the surveillance area, it is possible to recognize the terrain / feature to some extent through the screen, and in particular, it is easier to detect when a heating element such as a person moves. However, thermal cameras are very expensive equipment, and there is a problem that the representation of terrain / feature is still inferior.

Meanwhile, as the prior art, Korean Patent No. 10-0829215 discloses an invention entitled "Synthetic Image System for Hidden Object Detection", which is obtained from a general CCTV camera and a thermal camera for the same subject. The present invention relates to a technique of synthesizing an image obtained from a camera and obtaining a new image.

However, since the prior art is based on the premise that two or more cameras having different characteristics are used, it is not applicable to a night surveillance system using a single CCTV camera, and a specific synthesis method between images is not disclosed.

The present invention is to solve the above-mentioned problems, when performing the video surveillance for the place where the lighting is lacking at night in the real-time night video image obtained using only a single CCTV camera without using a special camera camera Night video surveillance based on day and night image synthesis, which replaces the background with pre-acquired background images to enhance the understanding of the surveillance area and easily recognize the occurrence and location and progress of the incident. Provides a system and method.

In order to achieve the above object, a night image surveillance method based on day and night image synthesis, the method comprising the steps of: (a) generating a daytime background still image from the video image data for the surveillance area collected from the video camera for a predetermined period of the day, (b) receiving a video frame for the surveillance area during night surveillance, (c) acquiring a motion mask image from the input video frame, and (d) based on the obtained motion mask image, And synthesizing the generated weekly background image with the input video frame, wherein the synthesized image is an image in which no motionless pixels in the input video frame are replaced with pixels of the weekly background still image. Night vision video surveillance method.

In order to achieve the above object, in the night video surveillance system based on day and night video synthesis, an image acquisition unit for obtaining a real-time video image of the surveillance area from the video camera, from the video camera at night surveillance of the surveillance area An image synthesizer for synthesizing the acquired real-time video image with the pre-obtained weekly background image to generate a composite video image from the daytime still image and the respective video frames of the real-time image as well as the real-time video image for the surveillance area. An image storage unit for storing the obtained motion mask image, an image reproducing unit for regenerating a composite video image using the stored real-time video image, a weekly background still image, and a motion mask image, and an image display for outputting the composite video image on a screen Including but not limited to The sex video image provides a nighttime video surveillance system based on day and night image synthesis, wherein the motionless pixels in the input real-time video image are replaced with the pixels of the daytime background image.

According to the night video surveillance system and method according to the present invention, when performing a video surveillance for a place where lighting is insufficient at night by using a CCTV camera, the monitor through the composite video night and day, which of the monitoring screen currently being viewed It's effective to know immediately if you are pointing to a place.

In addition, according to the night vision monitoring system and method according to the present invention, it is possible to easily determine whether the occurrence of the event in the surveillance area and the location of the event in connection with which terrain / feature in the surveillance area.

Hereinafter, a night vision monitoring system and method based on day and night image synthesis according to an embodiment of the present invention will be described with reference to FIGS. 1 to 4. In the business card describing the present embodiments, redundant additional descriptions may be omitted.

1 is a block diagram of a video surveillance system 100 according to an embodiment of the present invention.

Referring to FIG. 1, the video surveillance system 100 according to an embodiment of the present invention includes a video camera 110 and an image processing device 120.

The video camera 110 may be a general fixed camera or a PTZ camera capable of rotating / zoom. The video camera 110 may be an analog video signal such as NTSC / PAL or an Ethernet / USB / IEEE1394 depending on a method of outputting a video signal. It may be a camera that outputs a digital video signal through a digital interface.

The image processing apparatus 120 is a device that receives and processes video data from the video camera 110. The image processing apparatus 120 is functionally provided with an image obtaining unit 121, an image synthesizing unit 122, an image displaying unit 123, and an image storing unit 124. ), An image reproducing unit 125, an image transmitting unit 126, a user interface 127, and a system control unit 128.

The image acquisition unit 121 receives a video signal from the video camera 110 to obtain a series of digital video frame data. When the video camera 110 is an analog camera, a frame capture function for converting an analog video signal into digital video data is included. In the case of a digital camera, the video camera 110 is encoded in a format such as MPEG4 / H.264. And decode the input video data.

The image synthesizing unit 122 synthesizes a real-time video image obtained from the video camera 110 with a background image of the surveillance region acquired during the day at night during the night surveillance of the surveillance region, which is useful for night surveillance. Create a video image.

The image display unit 123 outputs a real-time video image obtained from the video camera 110, a composite video image synthesized through the image synthesizing unit 123, and a video image reproduced through the image reproducing unit 125 to the screen. do.

The image storage unit 124 encodes video image data obtained through the image acquisition unit 121 and stores the encoded video image data in a storage device such as a hard disk drive (HDD). In addition, the image storage unit 124 stores the weekly background image and the motion mask image, which are necessary elements for reproducing the composite video image, when the image is reproduced.

The image reproducing unit 125 decodes the video image data of the user-specified location stored in the image storing unit 124 and transmits the decoded image to the image display unit 123. In addition, the image reproducing unit 125 regenerates the synthesized video image by using the weekly background image and the motion mask image stored in the image storing unit 124 when necessary.

The image transmitter 126 encodes the video image data and transmits the encoded image data to an external system or a terminal through various wired / wireless communication means.

The user interface 127 receives various commands for controlling the image processing apparatus 120 from the user, or outputs a processing result of the image processing apparatus 120.

The system controller 128 performs organic connection and control between each module in the system.

2 is a flowchart illustrating a method for synthesizing day and night images performed by the image synthesizer 122 of the image processing apparatus 120.

It is assumed that the video camera 110 is fixed while steps S101 to S105 are performed. Thus, when the video camera 110 is a PTZ camera, the same pan / tilt angle and zoom magnification are maintained while steps S101 to S105 are performed.

The image processing apparatus 120 performs steps S101 to S102 during the day.

In operation S101, the image processing apparatus 120 collects video image data for a surveillance region for a predetermined time.

In operation S102, the image processing apparatus 120 processes the video image data collected in operation S101 to generate a weekly background still image including only the background area of the surveillance region from which moving objects are removed.

The image processing apparatus 120 repeatedly processes steps S103 to S105 in real time at night.

In operation S103, the image processing apparatus 120 receives a video frame for the surveillance area from the video camera 110.

In operation S104, the image processing apparatus 120 obtains a motion mask image from the input video frame. The motion mask image is an image in which moving pixels in an input video frame are represented by one or more values, and pixels which are not represented by a value of zero. Here, the moving pixels are pixels belonging to an area mainly occupied by the moving object in the image.

In operation S105, the image processing apparatus 120 synthesizes the input video frame and the weekly background still image using the motion mask image. The synthesized image is an image in which no motion pixels in the input video frame, that is, pixels corresponding to a background, are replaced with pixels of a weekly background still image. Through the synthesized image, the monitor can more clearly grasp where the movement of the object is occurring.

3 is a flowchart illustrating a method of generating a weekly background still image corresponding to steps S101 to S102 of FIG. 2 in more detail.

In operation S201, the image processing apparatus 120 receives a video frame from the video camera 110.

에 대응하는 밝기 히스토그램을

라고 하고,

의

번째 빈을

라고 하고, 픽셀 밝기의 최대값을

라고 하고, 입력 비디오 프레임에서 픽셀

의 밝기 값을

라고 하면, 밝기 히스토그램

는 하기 수학식 1과 같이 업데이트된다.In operation S202, the image processing apparatus 120 updates a brightness histogram for each pixel of the input video frame. The brightness histogram is a one-dimensional array consisting of N bins. pixel

Corresponding to the brightness histogram

Say,

of

Th bin

Called the maximum pixel brightness

Say, the pixel in the input video frame

Brightness value

Speaking of brightness histogram

Is updated as in Equation 1 below.

In this case, it is assumed that the value of each bin of the brightness histogram is initially initialized to zero.

In operation S203, the image processing apparatus 120 checks whether a specified time has elapsed. If the specified time has not elapsed, steps S201 to S202 are repeated. If the specified time has elapsed, the process proceeds to step S204.

의 최대값을 갖는 빈의 인덱스 값을

라고 하고, 주간 배경 정지 영상에서 픽셀

의 밝기 값을

라고 하면, 주간 배 경 정지 영상은 하기 수학식 2와 같이 주어진다.In operation S204, the image processing apparatus 120 generates a weekly background still image by finding the maximum value of the brightness histogram for each pixel. The value of each bin of the brightness histogram accumulated for a certain time indicates the frequency of appearance of pixels having a brightness value corresponding to the bin. When shooting a certain area with moving cameras for a certain period of time with the camera fixed, if the shooting time is sufficient and the time the moving object stays in a specific place is not long for the shooting time, the shooting time at most pixel positions The most frequently exposed frequency is the pixels corresponding to the background. Therefore, after a certain shooting time has elapsed, the brightness of a pixel corresponding to the bin having the maximum value of the brightness histogram of each pixel is very likely to be the pixel value of the background at that position. Brightness histogram

The index value of the bean with the maximum value of

In the weekly background still image,

Brightness value

In this case, the weekly background still image is given by Equation 2 below.

FIG. 4 is a flowchart illustrating a motion mask image acquisition method corresponding to steps S103 to S104 of FIG. 2 in more detail.

In operation S301, the image processing apparatus 120 receives a video frame from the video camera 110.

의 밝기 분포는 그것의 평균값

과 표준편차

에 의해 표현할 수 있다. 입력 비디오 프레임의 픽셀

의 밝기 값을

라고 하면,

과

는 하기의 수학식 3과 같이 업데이트된다.In operation S302, the image processing apparatus 120 updates a background model by using the input video frame. The background model is a model of a distribution of brightness values of pixels corresponding to the background using a specific function. If the brightness of a pixel corresponding to the background is always constant even after elapse of time, the background model does not need to be updated. However, in most outdoor environments, the brightness value of the background pixel changes due to a change in lighting or the like over time. Therefore, periodic background model needs to be updated. If you model the distribution of the brightness values of the background pixels as a Gaussian distribution, the background pixels

The brightness distribution of is its average value

And standard deviation

Can be represented by Pixels of the input video frame

Brightness value

Speaking of

and

Is updated as in Equation 3 below.

는 사용자가 지정한 학습 상수(Learning Constant)이며,

의 값을 갖는다.In Equation 3

Is a user-specified learning constant.

Has the value of.

의 밝기 값을

라고 하면,

는 하기 수학식 4와 같이 주어진다.In operation S303, the image processing apparatus 120 obtains an initial motion mask image based on a difference between the input video frame and the background model. The motion mask image is an image indicating an area where a motion occurs. Pixel of initial motion mask image

Brightness value

Speaking of

Is given by Equation 4 below.

는 사용자가 지정한 문턱치(Threshold Value)이다.In Equation 4

Is a user-specified threshold value.

에서 입력 비디오 프레임

와 배경 영상

사이의 텍스처의 유사성

은 하기 수학식 5에 의해 주어진다.In operation S304, the image processing apparatus 120 finds and removes noise pixels generated in a shadow or a sudden change in illumination from the initial motion mask image. By removing these noise pixels, the motion mask image more accurately represents only the motion regions generated in the moving objects. When removing a noise pixel, unlike a moving object, a motion region generated by a shadow or a sudden change in illumination utilizes the fact that an input video frame and a background image have similar texture components in the region. Specific pixels

Input video frame

And background image

Similarity of textures between

Is given by Equation 5 below.

은 픽셀

를 중심으로 의 윈도우 내에 들어오는 픽셀들의 집합을 의미한다. 텍스처 유사성

가 일정값보다 크면 픽셀

를 노이즈 픽셀로 간주하고

으로 설정한다.In Equation 5

Silver pixels

It means the set of pixels that come into the window of. Texture similarity

Is greater than a value, pixels

Is considered a noise pixel,

Set to.

In operation S305, the image processing apparatus 120 may apply a binary morphological operation to the motion mask image to obtain a final motion mask image. The morphology operation is applied to the motion mask image to remove motion mask regions that are very small in area, and merge adjacently adjacent motion mask regions to produce a cleaner motion mask image.

, 상기 주간 배경 정지 영상

, 상기 모션 마스크 영상

이 주어졌을 때, 도 2의 단계(S105)에서 합성 영상

는 하기 수학식 6과 같이 구할 수 있다.Input video frames

Weekly background still images

The motion mask image

Is given, the composite image in step S105 of FIG.

Can be obtained as in Equation 6 below.

는, 모션 영역에서는 입력 비디오 프레임의 픽셀 값들이, 모션 영역 이외의 영역에서는 주간 배경 정지 영상의 픽셀 값들이 채워짐에 의해 생성된 영상이 된다. 그러나 상기 모션 마스크 영상을 그대로 사용할 경우에 합성 결과를 보면, 배경 영역과 모션 영역 사이의 경계선 부근에서 부드럽게 이어지지 않는 단점이 있다. 이러한 단점을 해결하기 위해 모션 마스크 영상에 가우시안 필터(Gaussian Filter) 등과 같은 저주파 통과 필터를 적용하여 모션 마스크 영상을 블러링(Blurring)하여 사용한다. 블러링된 모션 마스크 영상은 모션 마스크 영역의 가장자리에서

의 값이 255에서 0으로 부드럽게 변하게 된다. 따라서 블러링된 모션 마스크 영상을 상기 수학식 6에 적용하여 만들 어진 합성 영상은 모션 영역의 가장자리가 배경 영역과 부드럽게 이어진다.Composite image generated by the equation (6)

Is an image generated by filling pixel values of an input video frame in a motion region and pixel values of a weekly background still image in an area other than the motion region. However, in the case of using the motion mask image as it is, the synthesis result shows a disadvantage that it does not smoothly run near the boundary line between the background region and the motion region. In order to solve this disadvantage, a low-pass filter such as a Gaussian filter is applied to the motion mask image to blur the motion mask image. The blurred motion mask image is taken from the edge of the motion mask area.

The value of is smoothly changed from 255 to 0. Therefore, in the composite image created by applying the blurred motion mask image to Equation 6, the edge of the motion region is smoothly connected to the background region.

In FIG. 1, the image storage unit 1024 of the image processing apparatus 120 may directly store the real-time synthesized image obtained in step S105 of FIG. 2 to reproduce the synthesized video image in step S102. A method of storing the obtained weekly background still image and the motion mask image obtained in step S104 is taken. That is, one weekly background still image is stored for one fixed video camera, and an input video frame and a motion mask image corresponding thereto are sequentially stored in pairs. When reproducing the composite video image, the image reproducing unit 1025 retrieves the stored input video frame, the motion mask image, and the weekly background still image, and generates the synthesized image by using Equation 6. The reason for storing the motion mask image instead of the direct storage of the synthesized image is that the compression efficiency is much better than that of the general image when compressed and stored due to the simple structure of the motion mask image.

In the above, the present invention has been described in detail with reference to the accompanying drawings and embodiments, the accompanying drawings and the above embodiments are described by way of example to help those of ordinary skill in the art to understand the present invention. It is. Accordingly, the above embodiments are to be considered as illustrative and not restrictive, and the scope of the invention should be construed in accordance with the invention set forth in the appended claims, the scope of which is to be understood by those of ordinary skill in the art. Include various changes, alternatives, and equivalents by the party.

1 is a block diagram of a video surveillance system according to an embodiment of the present invention.

2 is a flowchart illustrating a method of synthesizing day and night images according to an embodiment of the present invention.

3 is a flowchart illustrating a method of generating a weekly background still image according to an embodiment of the present invention.

4 is a flowchart illustrating a method of obtaining a motion mask image according to an embodiment of the present invention.

Claims (7)

In the night video surveillance method based on day and night video synthesis, (a) generating a weekly background still image from the video image data for the surveillance area collected from the video camera for a predetermined period of day; (b) receiving a video frame for the surveillance area during night surveillance; (c) obtaining a motion mask image from the input video frame; and (d) synthesizing the generated weekly background image with the input video frame based on the obtained motion mask image, The synthesized image is a night image monitoring method, wherein the motion-free pixels in the input video frame is replaced with the pixels of the weekly background still image. The method of claim 1, The daytime background still image is a night image monitoring method, characterized in that made only of the background area of the monitoring area from which moving objects are removed. The method of claim 1, The motion mask image is an image in which moving pixels in the input video frame are displayed with a value greater than or equal to 1, and pixels in motion are displayed as a value of 0, and the moving pixels are moved objects in the input video frame. The night video surveillance method, characterized in that the pixels belonging to the area occupied. The method of claim 1, The step (a) of generating the weekly background still image may include: (a1) receiving a video image of the surveillance area from the video camera during a predetermined period of day; (a2) initially initializing a brightness histogram corresponding to each pixel for each video frame of the input video image, and thereafter updating the brightness histogram using the brightness value of the pixel; (a3) completing the brightness histogram for each pixel by repeating step (a2) of updating the brightness histogram for each video frame received during the predetermined period; and (a4) The brightness value of each pixel of the daytime background still image is a bin having the maximum value of the completed brightness histogram corresponding to the pixel in step (a3), and the bin having the maximum value. And employing a brightness value indicated by). The method of claim 1, (C) acquiring the motion mask image may include: (c1) generating a background model modeling a distribution of brightness values of each pixel corresponding to a background of the input video frame; (c2) acquiring an initial motion mask image based on a difference between the input video frame and the background model; (c3) removing noise pixels from the initial motion mask image; and and (c4) acquiring a final motion mask image by applying a morphology operation to the initial motion mask image from which the noise pixel is removed. The method of claim 5, wherein And the final motion mask image is blurred by applying a low pass filter to synthesize the generated daytime background image and the input video frame. In the night video surveillance system based on day and night video synthesis, Image acquisition unit for obtaining a real-time video image for the surveillance area from the video camera, An image synthesizer configured to synthesize a real-time video image obtained from the video camera with a pre-obtained daytime background image during the night-time monitoring of the surveillance area to generate a composite video image; An image storage unit for storing a live background image and a motion mask image obtained from each video frame of the real-time image as well as a real-time video image of the surveillance region; An image reproducing unit reproducing a composite video image using the stored real-time video image, a weekly background still image, and a motion mask image; An image display unit for outputting the composite video image on the screen, The composite video image is a night video surveillance system based on day and night image synthesis, characterized in that the image is replaced with the pixels of the daytime still image in the input real-time video image.

Images