KR101038650B1 - Adaptive modeling method for background image, detecting method and system for illegal-stopping and parking vehicle using it - Google Patents

Abstract

PURPOSE: An illegally parked car controlling system using an adaptive background image modeling method is provided to perform an exact modeling function through a quick removal of an effect of an object which stayed within a monitoring region. CONSTITUTION: An unmanned illegal parking management system extracts a foreground image by an object from a video frame(S140). The system senses the movement or the pause of the foreground region(S150). If the fore ground region is not extracted or the extracted foreground region is in moving, a Gaussian model by pixel about the entire region of the background is updated. If the extracted foreground region is moved after a predetermined time, the Gaussian model about the stopped area is initialized(S160).

Description

ADAPTIVE MODELING METHOD FOR BACKGROUND IMAGE, DETECTING METHOD AND SYSTEM FOR ILLEGAL-STOPPING AND PARKING VEHICLE USING IT}

The present invention relates to an adaptive background image modeling method, an illegal parking unmanned enforcement method using the same, and an illegal parking unmanned enforcement system. More particularly, the present invention relates to a situation change of a moving object in modeling a background image with an adaptive Gaussian mixture model. Therefore, by updating the background image globally or locally updating the background image, the background image is accurately updated by reducing the influence of the moving object on the background image, and the moving object according to the change of the situation obtained in the process of updating the background image. The present invention relates to an adaptive background image modeling method for detecting illegal parking and detecting illegal parking, an unmanned parking control method using the same, and an illegal parking control system.

A method of monitoring a moving object by photographing a specified area using a camera, and extracting the foreground area according to the appearance of the moving object by subtracting the current image from the background image of the designated area, and extracting the foreground area without missing in this process Inevitably, accurate background image acquisition is inevitably required. In addition, it is important to accurately update the background image in real time by adapting to a change in the background that changes every frame.

As a method of acquiring a background image, there is an adaptive Gaussian Mixture model method that can cope with a change in brightness such as shadows and highlights. The adaptive Gaussian mixture model models a pixel value of each pixel by successively input image frames as a Gaussian distribution, and determines whether an object appears according to the mean and covariance of the modeled Gaussian distribution, wherein the current frame The degree of deviation from the mean value of the Gaussian distribution is determined based on the covariance value to detect whether the pixel is caused by an object or a pixel due to a change in the background. The adaptive Gaussian mixture model then updates the mean and covariance of the Gaussian distribution with each frame input. Accordingly, the adaptive Gaussian mixture model method appropriately copes with the change in the brightness of the background and can detect the appearance of an object probabilisticly and accurately.

However, this adaptive Gaussian mixture model effectively adapts to the passage of a moving object because the moving object passes through the specified area because the influence of the pixel value by the moving object is small and its effect disappears after the moving object disappears. For example, when a moving object stops in a designated area for a long time, the pixel value caused by the moving object greatly affects the Gaussian distribution, and thus it is not accurately modeled as a background image. In addition, when the object is moved again, the Gaussian distribution of the pixel corresponding to the stop position is greatly influenced by the pixel value of the object, so that the object can be modeled as an original background only after the object is updated for a long time. . Therefore, modeling a background image with an adaptive Gaussian mixture model requires a method of rapidly updating the real background even if the object is stopped and moved for a long time in the surveillance area.

On the other hand, the illegal parking control is to detect and control a vehicle that has stopped more than a specified time by photographing the illegal parking control area on the road, and even if many vehicles enter the enforcement area, the error of the background image by the vehicle should be minimized. In particular, since the vehicle parked in the intermittent area is detected, the effect of the afterimage caused by the parked vehicle should be quickly removed from the background image. In addition, it is preferable to detect the parked vehicle using information obtained in the process of updating the background image.

In addition, even if illegal parking stops are assigned a plurality of control areas separated from each other on the road, it is preferable to solve the cost difficulties by monitoring with one camera and to detect the foreground image quickly by processing only the control area. .

Accordingly, an object of the present invention is to provide an adaptive background image modeling that can accurately model a background image by quickly removing a residual image of a moving object after stopping and modeling a background image according to a moving situation of a moving object in a region of interest to be monitored. To provide a way.

Another object of the present invention is to accurately and quickly detect a parked vehicle based on a background image corresponding to a moving situation of a vehicle, and to utilize information obtained during a modeling process of a background image. It is to provide an illegal parking control method and an illegal parking control system that can monitor the enforcement area.

In order to achieve the above object, the present invention stores and manages the position and time history of the foreground area by the object in the history management module 140 to update or globally update the background image of the area photographed by the camera. In the adaptive background image modeling method of modeling while the camera is initialized, the Gaussian model is initialized by calculating the average and covariance according to the Gaussian distribution for each pixel in the initial continuous frame of the image taken by the camera. Step S130; A foreground region extraction step S140 of extracting a foreground region by an object from an image frame after the initialization step S130 based on the pixel-specific Gaussian model; A situation recognition step (S150) of managing the location and time history of the extracted foreground area by the history management module 140 and detecting movement or stop of the foreground area based on the history; If the foreground area is not extracted and the extracted foreground area is moving, the Gaussian model for each pixel of the background is updated, and if the extracted foreground area stops for more than a preset time and then moves, And a background update step (S160) of initializing the Gaussian model with the current pixel value.

The background updating step (S160) is characterized in that when the extracted foreground region stops for more than a preset time, the Gaussian model for the pixels of the stopped foreground region is initialized to the pixel value of the currently extracted foreground region. .

The present invention provides an illegal parking control method, comprising: a photographing step of photographing an illegal parking control area using a camera in a continuous frame (S10); On the basis of the Gaussian model initialization step (S130) and the Gaussian model for each pixel, which calculates the average and covariance according to the Gaussian distribution for each pixel in the initial continuous frame of the image taken by the camera. Foreground region extraction step (S140) for extracting the foreground area by the object in the image frame after the initialization step (S130), the history and location history of the extracted foreground area is managed by the history management module 140, the foreground based on the history In the situation recognition step (S150) of detecting the movement or stop of the region, when the foreground region is not extracted, and when the extracted foreground region is moving, the Gaussian model for each pixel of the background is updated and the extracted foreground region is previewed. If the user stops after the set time and moves, the Gaussian model for the stopped area is changed to the current pixel value. Initializing the background update step (S160), including the contextual modeling step (S100); An intermittent image acquisition step (S200) of sequentially obtaining at least one foreground region corresponding to the stationary object in the context-aware modeling step (S100), sequentially determining the extracted foreground region and sequentially obtaining the interrupted image including the license plate region; Vehicle number recognition step of recognizing the vehicle number in the interrupted image (S300); An intermittent information generation step (S400) of generating intermittent information by linking the intermittent situation information including the photographing date and time, the photographing position or the vehicle number with the intermittent image; It is characterized in that it comprises a; and whether or not to determine whether to crack down in accordance with the regulation time of the parking stop by accumulating the time course of the crackdown information having the same crackdown situation information (S500).

The present invention includes an on-site system (200) for detecting crackdown information on a vehicle in an illegal parking control crackdown area, and a center system (300) for determining illegal parking based on crackdown information in an illegal parking control system. Is configured, the field system 200, the camera 210 to shoot in a continuous frame under the control of the camera control unit 211; A schedule storage unit 220 for storing mapping information on the location and distance of the area of interest AOI designated as an illegal parking control area; An initialization module 110 for initializing the adaptive Gaussian mixture model for each pixel for the background image by calculating an average and a covariance according to the Gaussian distribution for each pixel in an initial continuous frame of the image obtained by photographing; A foreground region extraction module 120 for extracting a foreground region OOI by an object from an image frame after initialization based on the pixel-specific adaptive Gaussian mixture model; A history management module 140 for managing the location and time history of the extracted foreground area; And update the pixel-specific Gaussian model for the entire background of the background when the foreground area is not extracted and when the extracted foreground area is moving, and stops the extracted foreground area after moving for more than a preset time. A background update module 150 for initializing an average value of the adaptive Gaussian mixture model with respect to the current pixel value; After receiving the mapping information of the foreground area that has been stopped for more than a preset time, the image is enlarged to obtain a crackdown image. Intermittent information acquisition unit 240 for generating intermittent information in association with the connection; Communication unit 270 for transmitting the generated intermittent information to the center system 300; Characterized in that configured to include.

Therefore, the present invention made as described above has the advantage of accurately detecting the object in the region of interest because it is possible to accurately model the background image by quickly removing the influence of the object stopped in the region of interest.

In addition, the present invention has the advantage that the object can be detected while the signal is processed quickly because only the region where the foreground region exists in the pixel-specific Gaussian model even if the frequency of occurrence of the stationary object increases.

In addition, since the present invention detects the parking vehicle with information obtained in the modeling process of the background image in controlling the illegal parking vehicle, the vehicle can be quickly processed without a separate target selection process for crackdown, and is designated as an illegal parking enforcement area. In addition to improving the speed of vehicle detection by modeling as a background image, it also has a cost advantage by monitoring multiple areas with a single camera.

1 is a flow chart of an adaptive background image modeling method according to an embodiment of the present invention.
2 is a block diagram of a context-aware modeling unit 100 for implementing an adaptive background image modeling method according to an embodiment of the present invention.
Figure 3 is a block diagram of an illegal parking unmanned enforcement system according to an embodiment of the present invention.
Figure 4 is a block diagram of the enforcement information acquisition unit 240 mounted on the field system 200 and the sub-site system 200A in the illegal parking unmanned enforcement system according to an embodiment of the present invention.
5 is a diagram showing an example of applying an illegal parking unmanned enforcement system according to an embodiment of the present invention at a crossroad.
Figure 6 is an exemplary view of the image taken in the on-site system 200 or sub-site system 200A in the illegal parking unmanned enforcement system according to an embodiment of the present invention.
7 is a flow chart of the illegal parking control method according to an embodiment of the present invention.
FIG. 8 is an image diagram illustrating a search detection mode that is an illegal parking control mode in an illegal parking control system according to an embodiment of the present invention. FIG.
9 is an operation flowchart of a search detection mode that is an illegal parking control mode in the illegal parking control system according to an embodiment of the present invention.
10 is a flowchart illustrating an operation of a user designated mode that is an illegal parking control mode in the illegal parking control system according to an embodiment of the present invention.
FIG. 11 is a view showing an image shaping process by a lens to illustrate focus correction according to embodiments of the present invention. FIG.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, it is to be noted that the same reference numerals as shown in the configuration or action, the same reference numerals are used as much as possible when indicating the same configuration or action in the other drawings. In addition, in describing the present invention below, when it is determined that a detailed description of a related known function or known configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

Prior to describing an embodiment of the present invention, an adaptive Gaussian Mixture Model will be described.

The adaptive Gaussian mixture model models the distribution of pixel values as Gaussian probability distributions for each pixel in a frame of a continuously captured image, wherein the average and covariance parameters of the Gaussian model for each pixel are calculated every frame. Update every time. Here, although one Gaussian distribution may be used for each pixel, K Gaussian distributions, which are usually designated as 3 to 5, are used to approximate each pixel value representing a recently observed distribution. This will be described in detail.

First, the statistical information on the pixel value of the pixel {x, y} at a certain time t is assumed to be [Equation 1] below.

[Equation 1]

는 K 가우시안 모델이며, 시간 t에서

의 값을 갖는 입력영상의 화소에 대한 확률은 아래의 [수학식 2]로 추정된다.Here, I is an input image,

Is the K Gaussian model and at time t

The probability of the pixel of the input image having the value of is estimated by Equation 2 below.

[Equation 2]

와

로 표시하였으며,

는 가우시안 확률밀도함수(Gaussian probability density function)으로서 아래의 [수학식 3]으로 구한다.Where K is the number of Gaussian models for each pixel, and the mean and covariance matrix of the i th Gaussian mixture model at time t

Wow

,

Is a Gaussian probability density function obtained from Equation 3 below.

&Quot; (3) "

here, Is the weight of the i th Gaussian model at time t, and is obtained from Equation 4 below.

&Quot; (4) "

는 학습율로서 이 값이 크면 기존에 구축된 모델이 새로 입력되는 영상에 의해서 빠르게 변경되고 이 값이 작으면 천천히 변경되므로, 사용자가 적절한 값으로 지정한다. 또한,

는 화소값이 가우시안 모델에 정합되면 1, 그렇지 아니한 다른 가우시안 모델들은 0의 값을 갖는다.here,

Is a learning rate. If this value is large, the existing model is quickly changed by the newly input image. If this value is small, it is changed slowly. Also,

Is 1 if the pixel value matches the Gaussian model, and other Gaussian models otherwise have a value of zero.

)과 공분산(

)은 아래의 [수학식 4]로 구한다. And mean (

) And covariance (

) Is obtained from Equation 4 below.

[Equation 5]

는 단위 행렬 I를 이용하여

와 같은 형태를 갖는다. 그리고, K개의 가우시안 분포들은

의 값들을 통하여 정렬되며, 첫 번째 분포 B가 배경 모델로 사용된다. 이때의 B는 아래의 [수학식 5]로 결정된다.On the other hand, the equation of [Equation 5] is made under the premise that the RGB pixel values (red, green and blue pixels) have independent variances from each other.

Using the identity matrix I

Has the same form as And the K Gaussian distributions

Sorted by values, the first distribution B is used as the background model. B at this time is determined by Equation 5 below.

&Quot; (6) "

Here, the threshold T (threshold) is the minimum of all weights obtained from the background model.

)도 학습시키면서, 각 가우시안 분포의 평균 및 공분산을 학습하므로, 단일 가우시안 분포로 모델링할 때보다 감시대상 영역의 실제 배경 영상 변화에 유연하게 대처한다.The adaptive Gaussian mixture model described above may determine whether the current pixel is the background or the foreground area pixel due to the appearance of the moving object according to how similar the background pixel corresponding to the current pixel is by a statistical method. To be able. Accordingly, if the adaptive Gaussian mixture model is used as the background image, the object can be detected by adapting to the brightness change well while being less influenced by noise, and the effect of the moving object is temporary even when the moving object passes through the monitored area. As a result, the effect remains on the background image but the effect is less. Moreover, after the moving object passes, the effect remaining by the actual background image gradually disappears. In addition, the weights for the K Gaussian distributions (

), While learning the mean and covariance of each Gaussian distribution, it responds more flexibly to changes in the actual background image of the monitored region than when modeling with a single Gaussian distribution.

However, the adaptive Gaussian mixture model increases the effect when the moving object stops in the monitored area for more than a certain time. Moreover, even if the stopped object is moved again and moves out of the monitored area, the adaptive Gaussian mixture model is applied to the pixel where the object stops. Since the remaining influence is gradually reduced, there is a problem in that it takes a long time to update the real background image disappeared. This phenomenon makes it difficult to detect later appearing objects.

The adaptive background image modeling method according to the present invention is an invention devised to solve the above-mentioned problems, and it is possible to quickly model a background image modeled on an actual background even if an object appears and passes or stops and disappears in a region of interest. Adapt. This will be described in detail with reference to FIGS. 1 and 2.

1 is a flowchart of an adaptive background image modeling method according to an exemplary embodiment of the present invention, and FIG. 2 is a block diagram of a context-aware modeling unit 100 for implementing an adaptive background image modeling method according to an exemplary embodiment of the present invention. It is also.

Referring to FIG. 2, the situational awareness modeling unit 100 receives a continuous image frame photographed by a camera, obtains a background image using a Gaussian model, and for this purpose, a Gaussian distribution for each pixel with a predetermined number of initial frames. An initialization module 110 for initializing the Gaussian model by calculating the mean and the covariance according to; A foreground region extraction module 120 for extracting a foreground region generated by an object by using the Gaussian model as a background image in an input image frame after initializing the Gaussian model; A module 130 for recognizing a situation in which the foreground region moves or stops when the foreground region is extracted; A background update module 150 for initializing the Gaussian model and updating the Gaussian model according to a recognized situation, and extracting the foreground region from the foreground region extraction module 120 using the Gaussian model as a background image; And a history management module 140 for managing the history of the extracted location of the foreground area, the history of temporal movements and stops of the foreground area so as to be utilized when the situation recognition module 130 recognizes the situation. .

In this case, the background update module 150 includes: a global background update module 151 for updating a Gaussian model for the pixels of the entire background image; And a local background update module 152 for updating only the Gaussian model of the pixel corresponding to the extracted foreground region. That is, according to the present invention, the entire background image is updated according to the recognized situation, or only the extracted foreground area is updated.

Next, an adaptive background image modeling method according to an embodiment of the present invention will be described with reference to FIG. 1.

First, in a state in which an image captured by the camera is input as a continuous frame (S110), it is determined whether to initialize the Gaussian mixed model (S120). In general, when a background image is modeled using an adaptive Gaussian model, a Gaussian mixture model for each pixel is trained using 15 to 30 frames in the initial input image. That is, when the situation-aware modeling unit 100 configured as shown in FIG. 2 is mounted on the image processing apparatus, the background image to be initially modeled as a Gaussian model is learned, and after the initial learning, the image to receive the trained Gaussian model. To upgrade.

In this case, when it is determined to initialize, the initialization module 110 trains the Gaussian mixture model to the background update model 150 until a predetermined number of frames are received from the initial input image frame (S130). When initialized as described above, the weight, average, and covariance for each pixel of the Gaussian mixture model are calculated according to the actual background image, thereby initializing the Gaussian mixture model for each pixel.

After learning the Gaussian mixture model with the initial image frame, the foreground region extraction module 120 extracts the foreground region by the object from the input image frame based on the Gaussian mixture model learned as the background image (S140). The foreground area extraction process applied to the embodiment of the present invention is as follows. The foreground region appearing in the image by the object appearing in the region captured by the camera is extracted based on the Gaussian mixture model (S141), and the extracted foreground region includes shadows, thereby removing the shadows (S142). After blocking each pixel of the foreground area from which the shadow is removed according to the correlation (S143), the part determined to be noise (noise) is removed (S144), and merging between adjacent blobs and blobs by motion are removed. In operation S145, a minimum enclosing rectangle (MER) including an object boundary is extracted to obtain an object of interest (OOI) (S146). As described above, the process of acquiring the foreground area based on the background image is not limited to the method described in the embodiment of the present invention.

After performing the process of extracting the image of the foreground area as described above (S140), the situation recognition module 130 recognizes whether the foreground area is extracted and whether the extracted foreground area moves or stops ( S150, selectively activates the global background update module 151 and the local background update module 152 of the background update module 150 according to the recognized situation (S160).

Here, the situation awareness module 130 transmits and manages the temporal and spatial history of the extracted foreground area OOI to the history management module 140, and based on the history of the foreground area managed by the history management module 140. The temporal and spatial changes of the foreground area OOI may be recognized in frames continuously input. That is, the situation awareness module 130 checks whether the foreground area OOI is extracted (S151), and when the foreground area OOI is not extracted, the Gaussian mixture model for all pixels as the currently input image frame. When the foreground area OOI is extracted (S161) and the foreground area OOI is extracted, the history according to the temporal and spatial change of the foreground area OOI is managed (S152). On the other hand, the step (S151) of checking whether the foreground area is extracted, even if the foreground area is extracted, it is preferable to determine whether the extracted foreground area is of interest and adopt it as the foreground area only if it is of interest. For example, when detecting a vehicle, the extracted foreground area should be at least a certain size to determine the foreground area of the vehicle.

Next, the situation awareness module 130 checks whether the foreground area OOI is stopped based on the history management (S153). Here, when a plurality of foreground areas OOI are detected, each of the foreground areas OOI is checked for stopping, and when the following steps are executed, they are separated and processed separately.

At this time, the situation awareness module 130 starts counting the stopped time for the foreground area (OOI) determined to be stopped in the step (S153) of checking whether the foreground area is stopped, and the stopped time is a preset time. It is checked whether or not the stop is abnormal (S154).

)으로 하는 것을 의미한다. 아울러, 초기화할 경우에 가우시안 모델의 공분산은 초기화 전에 사용하였던 공분산(

)을 그대로 사용할 수 있으며, 공분산(

)을 그대로 사용하는 것은 과거 이력에 의해서 공분산(

)이 결정되는 것인데 현재 화소값으로 평균값(

)을 중심으로 공분산을 다시 학습 또는 업데이트하기엔 시간이 경과하기 때문이다. 여기서, 정지한 전경영역의 화소만 현재 촬영된 전경영역의 화소값으로 초기화하므로, 초기화한 전경영역 위치의 화소에 대해 이전에 업데이트하였던 가우시안 분포에 대한 정보는 삭제되므로, 이후에, 정지한 전경영역이 가우시안 모델의 배경영상으로 된다. 이에 따라, 초기화 후에는 정지한 전경영역을 제외하고 새로 진입하는 물체만 전경영역으로 추출되며, 그만큼 물체에 따른 전경영역의 추출 시간을 단축할 수 있다. 한편, 정지한 전경영역이 가우시안 모델의 배경영상으로 된 상태이므로 상기한 전경영역 추출단계(S140)에서 추출되지 않지만, 정지한 전경영역에 대한 이력이 이력관리모듈(140)에 의해서 관리되므로 정지한 전경영역에 대해서는 상기한 전경영역 추출단계(S140)에서 감지되는 것이 아니라 전경영역(OOI)의 이력관리 단계(S152)에서 감지된다.In addition, the situation awareness module 130 may perform the extraction of the Gaussian model for the pixels in the foreground area OOI that are currently stopped when the foreground area OOI enters the shooting area and stops for more than a predetermined time. Initialize to a value (S163). In this case, the initialization is performed by selecting only pixels belonging to the stationary foreground area among all pixels, and locally updating the background. The pixel value of the currently extracted foreground area is averaged as a parameter of the Gaussian model (

Means). In addition, when initializing, the covariance of the Gaussian model is used.

) Can be used as is, and covariance (

) As it is, the covariance (

) Is determined, and the average value (

This is because it takes time to relearn or update the covariance. Here, since only the pixels of the stopped foreground region are initialized to the pixel values of the currently photographed foreground region, the information on the Gaussian distribution previously updated for the pixel of the initialized foreground region position is deleted. It becomes the background image of this Gaussian model. Accordingly, after initialization, only the newly entering object is extracted to the foreground area except for the stopped foreground area, and thus the extraction time of the foreground area according to the object can be shortened. On the other hand, because the stopped foreground area is a background image of the Gaussian model, it is not extracted in the foreground area extraction step (S140), but because the history of the stopped foreground area is managed by the history management module 140, The foreground area is not detected in the foreground area extraction step (S140) but is detected in the history management step (S152) of the foreground area (OOI).

In addition, if the situation awareness module 130 stops the foreground area OOI but does not reach the preset time, the controller 130 proceeds to step S110 of receiving an input of the next frame, and stops the foreground area OOI in the next frame. If it stops continuously, increase the time counter. Accordingly, when the global area OOI stops within a preset time, the next frame is processed without updating the Gaussian model, thereby eliminating the effect of the afterimage caused by the foreground area OOI.

The situation awareness module 130 checks whether the foreground area OOI is stopped or not in the step S153, with respect to the foreground area OOI which is determined to be moving without stopping, and the foreground area OOI is the photographing area of the camera. In step S155, it is checked whether the vehicle stops after the preset time or moves.

Then, the situation awareness module 130 globally updates the Gaussian mixture model for all pixels with the currently input image frame, if the foreground area OI has not stopped for more than a preset time (S161). The Gaussian mixture model for the extracted foreground region OOI pixel is also updated, but the influence of the current value of the moving foreground region OOI on the Gaussian mixture model is small. In this case, since it is determined that the foreground area moving after stopping within a preset time also moves, the Gaussian mixture model is updated globally. In addition, if the user enters the shooting area and appears as the foreground area (OOI) in the image frame, it is a new foreground area (OOI). Although not shown in FIG. 1, if a new foreground area is registered in the history management of the foreground area, the process may be performed to update the global background update step (S161).

On the other hand, when the situation awareness model 130 confirms that the foreground area OOI has been stopped for more than a preset time and started to move, the current foreground area OOI leaves and the foreground area OOI previously stopped. Since the pixels of the position become the background image, the pixels of the position where the foreground area OI was previously stopped are initialized with the current frame pixel values (S162). In this case, the initialization is to locally initialize the Gaussian model to the current frame with respect to the position where the stopped foreground area (OOI) leaves, which means that the pixel value of the current frame is the average value of the Gaussian model. Here too, the previously used covariance value can be used.

According to the present invention, in order to limit a specific range of an area photographed by a camera and to detect only moving objects within a limited specific range, an area corresponding to a specific range is previously designated on an image frame, and an area specified among all regions of the image frame is specified. Only the background image may be modeled by a pixel-specific Gaussian model. This is because there is a case where it is not necessary to generate a background image for an area outside of the interest, thereby improving the processing speed. In this case, the initialization background update (S130) and the global background update (S161) are performed only for the designated area, and in the foreground area extraction process (S140), only the foreground area corresponding to the moving object is extracted only in the designated area. . For example, as described with reference to FIG. 6 below, an area of interest (AOI) is designated in the captured image B, and the moving object A is only within the designated area of interest AOI. ) Is to extract the foreground area (OOI) that is optimally included.

As described above, in the adaptive background image modeling method according to an embodiment of the present invention, the background image is modeled by using an adaptive Gaussian Mixture Model, but the change of the situation according to the moving form of the moving object is detected. This is an invention that quickly updates the Gaussian model by adapting to the change of situation, and when the object is not detected and the moving object is detected, the background image is updated globally, and the moving object is stopped for more than a preset time or Otherwise, in the case of stopping and moving for more than a preset time, only the region of the stopped position is updated locally. Accordingly, the present invention can quickly remove the effect of the pixel value of the object remaining in the Gaussian model as an afterimage according to the stationary object, and accurately model the background image with the pixel value corresponding to the actual background. The object detection error that may have occurred can also be reduced. In addition, since the background image is updated locally, the time required for obtaining the background image may be shortened.

In addition, the situation-aware modeling unit 100 of FIG. 2 for implementing the adaptive background image modeling method according to an embodiment of the present invention may be mounted and used in an apparatus for detecting an object by image processing, as described below. In particular, it can be usefully used to crack down on illegal parking using the situation information obtained in the situation recognition step (S150) by the situation recognition module 130. Here, the setting time for determining whether the object is stopped may be appropriately set by the user according to the applicable field. For example, the setting time may be changed according to the driving type of the vehicle on the road when applied to the illegal parking control described below. However, it is appropriate to set it to about 30 seconds.

3 is a block diagram of an illegal parking control system according to an embodiment of the present invention, Figure 4 is a block diagram of the enforcement information acquisition unit 240 mounted on the field system 200 and the sub-site system (200A). 5 is a diagram illustrating an example of applying an illegal parking control system according to an embodiment of the present invention to a range.

And, Figure 6 is an exemplary view of the image taken in the field system 200 or sub-site system 200A in the illegal parking unmanned enforcement system according to an embodiment of the present invention.

3 and 4, the illegal parking control system according to an embodiment of the present invention, the center system (based on the enforcement information obtained from the on-site system 200 and sub-site system 200A installed on the road) In step 300, the government decides whether to crack down and performs administrative procedures to crack down on illegal parking. Here, according to an embodiment of the present invention, the field system 200 collects illegal parking information through short-range communication with the sub-site system 200A and collects the illegal parking information collected by itself, and collects the illegal parking information. Send to 300.

At this time, the field system 200 and the sub-site system 200A are configured to include the situation-aware modeling unit 100 of FIG.

In detail, the field system 200 and the sub field system 200A may include a camera 210, a camera controller 211, a schedule storage unit 220, a field environment sensing unit 230, and a situational awareness modeling unit 100. ), The intermittent information obtaining unit 240, the broadcasting unit 250, the power supply unit 260 and the communication unit 270.

Here, looking at the structural difference between the field system 200 and the sub-site system 200A, the communication unit 270 of the sub-site system 200A uses a short range communication unit 271 for communication with the field system 200. The communication unit 270 with the field system 200 includes a short range communication unit 271 for communication with the sub-site system 200A, but also includes a remote communication unit 272 for communication with the center system 300. . The short-range communication method between the field system 200 and the sub-site system 200A preferably includes short-range wireless communication in consideration of installation cost and ease of installation, and between the field system 200 and the center system 300. The telecommunication method is preferably made of wired communication using a dedicated line in consideration of the amount and speed of information to be transmitted.

In addition, the sub-site system 200A includes a power generation unit 261 that generates power on its own, and supplies the power to the power supply unit 260, but the field system 200 supplies the power generation unit 261 to the power generation unit 261. Since it is not provided, it is connected to the power supply unit 260 to an external power line to receive power from the outside. The power generation unit 261 may be configured as a photovoltaic device using a solar cell, a wind turbine using a wind power, or a device in parallel with the photovoltaic device and the wind power generator, and thus, connected to an external power line. The sub-site system 200A can be easily installed without laying the wires. On the other hand, the power supply unit 260 provided in the field system 200 and the sub-site system 200A supplies electricity at a voltage required for internal operation.

Hereinafter, the camera 210, the camera controller 211, the schedule storage unit 220, the field environment sensing unit 230, and the situational awareness modeling unit included in the field system 200 and the sub-site system 200A in common. A description will be given of the control unit 100, the intermittent information acquisition unit 240 and the broadcasting unit 250, and only the field system 200 will be described.

The camera 210 photographs a road and provides an image in a continuous frame, and the camera controller 211 not only adjusts the shutter speed, auto focus or white balance of the camera 210 but also PTZ (pan, tilt, zoom to adjust the imaging direction and focus. That is, as can be seen with reference to FIG. 5, the road on which the field system 200 is installed can be installed on not only four streets but also a two-way one-way lane of interest, which is an administratively designated illegal parking control area (AOI: Area). of interest) is distributed, so in order to capture AOIs separated from each other by one camera, the camera should be rotated to periodically change the imaging direction. To photograph the license plate to photograph and recognize the license plate, it is necessary to control the PTZ (pan, tilt, zoom) of the camera 210.

The schedule storage unit 220 is based on the location information of the AOIs that are separated from each other and the mapping information for the distance information for each location in each of the AOIs that are extended as shown in FIG. 5. Schedule information including direction adjustment information for sequentially photographing the plurality of ROIs and focusing information for photographing each point of the ROI with the same size is stored. On the other hand, such schedule information is also used when manually photographing a specific point in accordance with the instructions of the center system 300.

The field environment sensing unit 230 senses environmental factors surrounding the quality of the image captured by the camera 210 and includes an illumination sensor. This is because the image captured by the camera 210 is adjusted to the shutter speed, auto focus and white balance according to the ambient illumination to obtain a more desired level of image and to accurately focus. In addition, the field system 200 controls the camera controller 211 according to the value sensed by the field environment sensing unit 230.

As described above with reference to FIGS. 1 and 2, the situation awareness modeling unit 100 models a background image from an image received from the camera 210, and the foreground area OI detected by the situation awareness module 130. The history information of the) to be utilized in the intermittent information acquisition unit 240. That is, the situation-aware modeling unit 100 transmits mapping information of the foreground area OI that has been stopped for a predetermined time or more to the enforcement information acquisition unit 240.

The intermittent information acquisition unit 240 controls the camera controller 211 according to the mapping information of the foreground image (OOI) received from the situation-aware modeling unit 100 to enlarge and photograph the foreground image (OOI) part. In addition, intermittent information is generated. Specifically, referring to FIG. 4, the enforcement information obtaining unit 240 controls the camera control unit 211 according to mapping information and adjusts the PTZ of the camera 210 to detect the vehicle in the foreground image OOI. Intermittent image acquisition module 241 for photographing the license plate portion of the camera 210, the vehicle number recognition module 242 for recognizing the vehicle number using the image captured by the intermittent image acquisition module 241, and the situation And a watermarking module 243 for generating intermittent information obtained by watermarking the information on the intermittent image. Here, the intermittent situation information watermarked on the intermittent image includes a photographing date and time, photographing position, and mapping information. That is, according to the present invention, since watermarking is not carried out by adopting the conventional method of describing the intermittent situation information in the file name of the intermittent video, it is possible not only to maintain security but also to restrict the limited number of bytes in the file name. Can be displayed in detail, and furthermore, even when transmitting the file of the intermittent video, which is watermarked to the intermittent situation information, to the center system 300, there is no need to manage the intermittent situation information separately, and the intermittent situation information may be lost. none. The watermarking technique can be adopted by selecting any one of a variety of well-known methods, but obtained by performing inverse transformation after inserting a watermark in a frequency domain by performing DCT transformation, FFT transformation or wavelet transformation on an intermittent image. By using the method, it is desirable to prevent the original of the interrupted image from being damaged.

On the other hand, the intermittent situation information is the management of the intermittent image for each site when operating the vehicle number, the file index information for managing the intermittent image, a plurality of on-site systems or a plurality of sub-site systems in connection with one center system For management information, may include. At this time, even if the vehicle number is operated to include the vehicle number in the enforcement situation information, even if the vehicle number recognition module 242 of the enforcement information acquisition unit 240 fails to recognize the vehicle number, even if only the remaining information except the vehicle number It is preferable to generate the interruption information by including the information and watermarking the interrupted image. This is because the center system 300 may crack down and crack down on crackdown information that does not recognize the vehicle number.

The intermittent image acquisition module 241 acquires the intermittent image by adjusting the PTZ of the camera 210, so that the intermittent image can be obtained with a uniform resolution.

The broadcasting unit 250 performs genealogy broadcasting according to the instructions of the center system 300. That is, the center system 300 receives intermittent information from the field system 200 and transmits the broadcast signal necessary for genealogy broadcasting to the center system 300 when the vehicle parked for more than a predetermined time is detected and transmits the genealogy broadcast. Let's do it.

The on-site system 200 configured as described above is configured to transmit the intermittent information obtained by watermarking the intermittent situation information to the intermittent video to the center system 300, and to generate the intermittent information in the same manner as the on-site system 200. It also serves as a relay that receives the intermittent information from the field system 200A by short-range wireless communication and transmits it to the center system 300.

Then, the center system 300 collects the enforcement information received from the on-site system 200 and the sub-site system 200A and selects a target for stopping vehicle parking. For this purpose, the center system 300 extracts the intermittent situation information from the received intermittent information, classifies the intermittent information according to the extracted intermittent situation information, and records and controls the database. It is configured to include an enforcement decision server 320 for determining whether to determine the vehicle to be subject to enforcement, and vehicle inquiry server 330 for performing administrative procedures such as vehicle inquiry on the vehicle determined to be subject to enforcement.

In this case, the control video management server 310 is classified into a database by separating the information continuously received in time according to the enforcement situation information, and also, with the enforcement information that correctly recognized the vehicle number and the enforcement information that does not recognize the vehicle number By indexing the database so that the index information can be assigned, even if the vehicle number is unrecognized, the parking stop is controlled by the video.

The crackdown confirmation server 320 collects crackdown information classified according to the crackdown situation information, and checks whether the same vehicle parked or stopped more than the prescribed time, and if the vehicle parked more than the prescribed time from the crackdown information, the The vehicle is determined to be parked. Then, the enforcement information determined as the parked vehicle is transmitted to the vehicle inquiry server 330 to perform an administrative vehicle inquiry for the illegally parked vehicle. On the other hand, when the control server 320 is found to be a vehicle parked illegally, or a vehicle detected as having stopped at a predetermined time even though it has not reached the prescribed time, a site corresponding to the broadcast signal for the passage is detected. The system 200 is transmitted to the system 200 to perform the genealogy broadcast through the broadcasting unit 250. At this time, the field system 200 transmits the received broadcast signal to the sub field system 200A corresponding to short-range wireless communication when the received broadcast signal is to be transmitted to the sub field system 200A.

7 is a flow chart of the illegal parking control method according to an embodiment of the present invention, it is made using the above-described illegal parking unmanned enforcement system.

On-site system 200 is taken in a continuous frame while adjusting the settings of the camera to adapt to the site environment, and, according to the mapping information and schedule information stored in the schedule storage unit 220, PTZ (pan, tilt, Zoom) (S10). That is, the field system 200 adjusts the camera speed by adjusting shutter speed, auto focus, and white balance, which are set values of the camera controller 211, according to the sensing value detected by the field environment sensing unit 230. In operation 210, a clear image is obtained, and a region of interest AIO to be photographed is captured according to schedule information based on mapping information.

The field system 200 sequentially processes the image obtained as a continuous frame in the photographing process as a situational recognition modeling step S100, an intermittent image acquisition step S200, a vehicle number recognition step S300, and an interruption information generation step S400. The control unit transmits the control information to the center system 300 by watermarking the control situation information on the control image. In addition, the center system 300 performs the step of determining whether to crack down based on the crackdown information (S500).

The situation-aware modeling step (S100) is performed by the adaptive background image modeling method described above with reference to FIG. 1 and, accordingly, adaptively updates the Gaussian mixture model according to the movement type of the vehicle A of interest. By modeling the background image, the vehicle A may be detected to stop for more than a predetermined time in the situation recognition step (S150) by the situation recognition module 130. 5 and 6, the area of interest AOI, which is an illegal parking control area, is long on the road, so when turning the camera 210 and capturing each area of interest AOI, one area of interest ( The vehicle detected by the AOI may be formed in plural. Accordingly, the situational awareness modeling step S100 may be performed in one area of interest AIO when the object of interest (OOI), which is the foreground area including the vehicle A, is detected in the foreground area extraction step S140. When detecting a plurality of regions of interest OOI stopped for a predetermined time or more, a sequence number i is determined to the regions of interest OOI and the number of detected regions of interest OOI is set to the maximum value MAX. The license plate area is sequentially obtained in the subsequent intermittent image acquisition step (S300). Meanwhile, as shown in FIG. 6, the image B obtained by sequentially photographing the region of interest AOI based on the mapping information of the plurality of regions of interest AOI is actually the region of interest AOI which is to control illegal parking. Since other areas are also photographed, only the pixels of the region of interest AOI are modeled as a Gaussian mixture model based on the mapping information of the region of interest AIOI, thereby reducing the signal processing time by not modeling an unnecessary region. It is preferable. Here, the predetermined time for determining whether the region of interest OI is stopped has a value smaller than a prescribed time for determining whether or not to illegally park.

The intermittent image acquisition step (S200) is a step of capturing an image of the vehicle license plate area by the intermittent information acquisition unit 240, and mapping to the region of interest (OOI) detected in the situation recognition modeling step (S100). The license plate image is obtained by adjusting the PTZ of the camera 210 according to the mapping information. At this time, if the detected number of interested areas (OOIs) has been determined as a sequence number, the sequence number (i) is increased by one in association with the vehicle number recognition step (S300) and the crackdown information generation step (S400) described later (S600). Lastly, until the number (MAX) is reached (S210) to sequentially perform the license plate image acquisition step. For example, a process of obtaining a license plate image in an i th region of interest (OOI) is as follows. First, photographing is performed to obtain the license plate area by adjusting the photographing coordinates of the camera according to the mapping information of the i th region of interest (OOI), and checking focus is corrected (S231) until the focus is corrected. S232) to shoot, thereby obtaining a clear image. Next, in order to check whether the license plate area is obtained (S240), in this case, since the license plate generally prints the vehicle number on the standardized frame, it is possible to confirm whether the license plate area is obtained using the image of the standardized license plate frame. . In this case, if the license plate area is not detected, the vehicle number may not be recognized in the next step, so that the number of interests (OO) of the next number (i + 1) is photographed by raising the order (i) by one (S600). Then, when the license plate area is detected, the photographing area is precisely adjusted to the area where the license plate area is detected (S250) while checking whether the focus is corrected (S251), and correcting the focus until the focus is accurately corrected (S252). And shoot, thereby obtaining a clear image. At this time, the photographing is a photographing performed to recognize the vehicle number from the license plate area. On the other hand, the image obtained in the photographing (S230) for obtaining the license plate area may be treated as an intermittent image, but the captured image may be treated as an intermittent image in order to recognize a vehicle number for photographing the license plate more clearly.

In the vehicle number recognition step (S300), the vehicle number is recognized from the license plate image obtained as described above. The technology for recognizing a vehicle number is a technique known in the art to which the present invention pertains, and thus detailed description thereof will be omitted.

The intermittent information generating step (S400), the vehicle number obtained in the vehicle number recognition step (S300), the photographing position determined by the mapping coordinates according to the mapping information, the installation location of the field system 200 (or the site) The intermittent situation information including at least one of the identifiers of the system) is collected, and the intermittent intermittent situation information is watermarked to the intermittent image obtained in the intermittent image acquisition step S300 to generate the intermittent information. Then, the control unit returns to the control image acquisition step S200 in order to transmit the generated control information to the center system 300 and to generate control information for the next interested region OI.

On the other hand, by using the image taken in the sub-site system (200A) by performing the situational awareness modeling step (S100), the enforcement image acquisition step (S200), vehicle number recognition step (S300) and the enforcement information generation step (S400) When generating, to receive the generated intermittent information in the short-range wireless communication and transmits to the center system 300.

The determination whether the enforcement step (S500), the enforcement information received from the on-site system 200 and the sub-site system (200A) is collected, the enforcement situation information is extracted from the enforcement information and classified according to the enforcement situation information and databased, In addition, by managing the temporal history of the crackdown information, if the cumulative amount of temporal cumulative amount of the same crackdown situation reaches the prescribed time, it is determined as the final crackdown target, and the administrative process is performed through the inquiry of the crackdown information. On the other hand, the intermittent determination server 320 is detected as a vehicle parked illegally, or if the vehicle is detected as a vehicle parked by a predetermined time that does not reach the prescribed time, but the predetermined time, the broadcast signal for the passage corresponding to By transmitting to the field system 200, the genealogy broadcast through the broadcast unit 250 is made.

8 is an image diagram illustrating a search detection mode that is an added illegal parking control mode in the illegal parking control system according to an embodiment of the present invention, and FIG. 9 is an operation flowchart of the added search detection mode.

The search detection mode to be described with reference to FIGS. 8 and 9 does not model a background image, but instead is an operation mode for controlling illegal parking according to a history of a recognized vehicle number and a detected position by recognizing a vehicle number.

Referring to FIG. 8, an area of interest (AOI) designated as an illegal parking control area is divided into a plurality of areas, and each divided area is treated as a sector of interest (SOI). . At this time, the sectors of interest SOI cover the entire area of the region of interest AOI. In addition, adjacent sectors of interest (SOI) are partially overlapped. This is because when the adjacent sectors of interest (SOI) are contiguous with each other without partially overlapping, it is difficult to detect the license plate area placed in the contiguous part when detecting the license plate area.

In order to have the search detection mode, the field system 200 stores mapping information corresponding to coordinates of sectors of interest (SOI) and a photographing schedule in the schedule storage unit 220. In addition, the intermittent information acquisition unit 240 of the field system 200 receives an instruction to operate in the search detection mode from the center system 300, while adjusting the PTZ of the camera to make the area of interest (SOI) an image area to be photographed. It is configured to take the sectors of interest (SOI) in order by taking a sequence number (i), and to recognize the vehicle number in the image taken for each sector of interest (SOI) and to make a DB of the time history of the recognized vehicle number more than the prescribed time If it is determined that the vehicle is stopped, the control unit generates the control information by watermarking the control situation information on the control image and transmits the control system to the center system 300. An operation sequence of the search detection mode will be described with reference to FIG. 9 as follows.

The search detection mode may include an adaptive camera environment setting step (S10 ′) of controlling the camera controller 210 according to the sensing value of the field environment sensing unit 230 to optimize the image to be taken by the camera 210 according to the environment; An initialization step S20 of setting the order i to 1 to sequentially photograph the sectors of interest SOI according to the order i; An intermittent image acquisition step (S30) of repeatedly acquiring an intermittent image by repeatedly photographing each interested sector SOI by the number MAX of the sectors of interest SOI according to order i; Vehicle number recognition step of recognizing the vehicle number in the interrupted image (S40); Vehicle number history management step of managing the history of the recognized vehicle number (S50); A control information generation step (S50) of generating control information for the control image determined that the vehicle is parked or stopped according to the history of the vehicle number and transmitting the generated control information to the center system 300 (S50); The control system determines whether or not illegal parking is performed according to the control information in the center system 300 and performs an administrative procedure.

In this case, the step S30 of obtaining the interrupted image may be performed by mapping the coordinates of the sector area SOI in place of the region of interest OI. Otherwise, the step S200 may be described with reference to FIG. 7. Since it is the same as, it will be briefly described here. In the intermittent image acquisition step (S30), it is determined whether the sequence number (i) is greater than MAX (S31) and if it is less than or equal to MAX, the PTZ of the camera is adjusted according to the mapping information on the sector area (SOI) of the i-th sequence. (S32), photographing to obtain the position of the license plate area (S33), but if the focus is not in focus after the focus is corrected again (S34, S35), and checks whether the license plate area is detected in the captured image ( S36). At this time, if the license plate area is not detected, the sequence number (i) is increased by one step (S80) and the sector area (SOI) of the next sequence number (i + 1) is photographed, and when the license plate area is detected, the position of the license plate area is detected. By adjusting the PTZ of the camera to take a picture to recognize the vehicle number (S37), if the image is out of focus at this time to correct the focus and shoot again (S38, S39). The intermittent image to be obtained in the intermittent image acquisition step (S30) may be an image obtained from the image capture (S33) for acquiring the position of the license plate area, or an image obtained from the image capture (S37) to recognize the vehicle number.

On the other hand, the vehicle number history management step (S50), and manages the DB of the history of the enforcement status information of the control image, the vehicle number is recognized. Then, the control information generation step (S50) checks the history of the control situation information, if there is a vehicle number parked for more than a specified time to generate the control information consisting of the control image and the control situation information to the center system (S60) Then, the process proceeds to the step S200 of capturing the captured image of the sector area SOI of the next sequence i + 1. On the other hand, the step of determining whether the crackdown (S70) receives the crackdown information only for the vehicle parked over the prescribed time, and manages the crackdown information as a DB, only determine whether the crackdown is determined, not contrast with the prescribed time You may also do it. When the sequence number i is greater than MAX in starting the intermittent image acquisition step S30, the process returns to the adaptive camera configuration step S10 ', initializes the sequence number i, and sequentially orders the sector areas SOI in sequence. Shoot with.

10 is a flowchart illustrating an operation of a user designated mode that is an illegal parking control mode in the illegal parking control system according to an embodiment of the present invention.

The user designation mode shown in FIG. 10 is an operation mode for designating a photographing position in the center system 300 to limit the field system 200 to a designated photographing position to control illegal parking.

The user designation mode may include a shooting position designation step (S1) of designating a shooting position in the center system 300; PTZ adjustment step (S2) of adjusting the PTZ of the camera in the field system 200 to fit the designated shooting position; An adaptive camera environment setting step (S3) of adjusting the shutter speed, the auto focus or the white balance of the camera according to the sensing value of the field environment sensing unit 230; Shooting step (S4) to shoot with the camera, but not in focus (S5) to correct the focus (S5 ') to obtain a sharp intermittent image by shooting again (S4); The vehicle number recognition step (S6), the vehicle number history management step (S7), the crackdown information generation step (S8), and the crackdown determination step (S9) are performed in the same manner as the operation described with reference to FIG. Therefore, duplicate description is omitted. However, the field system 200 when operating in the user-specified mode fixes the position to be photographed at the designated place, and returns to the adaptive camera environment setting step S3 in the intermittent information generating step S8.

On the other hand, in the shooting position designation step (S1), it is possible to specify the PTZ control value of the camera, or may be specified according to the mapping information obtained by mapping the screen obtained by the camera and the scene on the road as coordinates.

The above-described search detection mode and user designation mode are modes that can be selectively operated according to the situation of the site to control illegal parking. For example, in a situation where it is difficult to generate a background image, it operates in a search detection mode when a specific illegal parking control area is quickly monitored, and operates in a user designated mode when intensively monitoring only a specific point.

FIG. 11 is a view showing an image shaping process by a lens in order to explain focus correction in embodiments of the present invention.

7, the focus correction (S232, S252, S35, S39, S5 ') shown in Figs. 7, 9 and 10 is a step of correcting the focus when the focus is not focused by checking the captured image, This focus correction will be described in detail with reference to FIG. 11.

The lens will be in focus according to the position where the object or image is formed, and the lens of the lens will be mathematically a point or spherical surface at a certain distance from the camera while in focus. In addition, a depth of field (DOF) is a distance from a close to a far distance from a camera in an area in which a person is focused on the image. Here, 'depth of field (DOF)' means that the focus is well focused back and forth from the focused distance, whereas 'low depth of field (DOF)' means focusing even a little from the focused distance. This means that it doesn't fit well. In other words, the depth of field (DOF) refers to the front and rear distance of the focus body in focus.

Referring to FIG. 11, the position s of the CCD image in which the image is neatly formed is called a depth of focus, and the position z of the object in which the image is neatly formed is called a subject depth DOF. At this time, the depth of field is determined according to the position of the blur circle / circle of confusion.

At this time, the distance c between the image and the confusion circle is calculated by the following Equation 7 as a ratio with the lens aperture diameter a.

[Equation 7]

을 이용하여 렌즈의 초점 거리(f)를 구할 수 있고, 또한, 피사계 심도(DOF)도 아래의 [수학식 8]로 구할 수 있다.Where the law of the lens

The focal length f of the lens can be obtained using, and the depth of field can also be obtained from Equation 8 below.

[Equation 8]

In addition, the focal length f and the zoom information may be obtained from the camera, and the subject depth DOF may be calculated according to the size of the license plate of the extracted vehicle.

Accordingly, the present invention will obtain a clear image if the camera is in focus according to the relationship between the focal length f, the zoom and the depth of field (DOF), otherwise obtain a blurred image. As we can see, we use this association to correct the focus. That is, the present invention determines whether the focus is formed according to the edge of the photographed image, and corrects the zoom of the camera. In other words, it determines whether the focus is achieved as the edge size of each pixel to correct the focus.

는 아래의 [수학식 9]로 구할 수 있고, 여기서,

는 각각 3x3 소벨(sobel) 연산자를 나타내며 입력 영상의 각 화소와 콘볼루션(convolution)을 통해서 에지의 크기가 계산된다.Specifically, the edge size of each pixel with respect to the input image I (x, y)

Can be obtained from Equation 9 below, where

Each represents a 3x3 sobel operator, and the size of the edge is calculated through convolution with each pixel of the input image.

[Equation 9]

The presence or absence of focus is calculated by S (x, y) as shown in [Equation 9], so that S (x, y) greater than a certain threshold (T: Threshold) can be used. Obtain as

[Equation 10]

Therefore, if the TN value is larger than the preset reference value, the in-focus value is smaller. If the TN value is smaller than the preset reference value, the focus is adjusted according to a comparison result between the TN value and the preset reference value.

Although illustrated and described in the specific embodiments to illustrate the technical spirit of the present invention, the present invention is not limited to the same configuration and operation as the specific embodiment as described above, within the limits that various modifications do not depart from the scope of the invention It can be carried out in. Therefore, such modifications should also be regarded as belonging to the scope of the present invention, and the scope of the present invention should be determined by the claims below.

AOI: area of interest OOI: foreground area (area of interest)
SOI: sector area A: object (object of interest)
100: contextual modeling unit 110: initialization module
120: foreground area extraction module 130: context awareness module
140: history management module 150: background update module
151: global background update module 152: local background update module
200: field system 200A: sub field system
210: camera 211: camera control unit
220: schedule storage unit 230: field environment sensing unit
240: intermittent information acquisition unit 250: broadcasting unit
260: power supply unit 170: communication unit
171: telecommunications unit 172: short-range communication unit
300: center system 210: intermittent video management server
220: intermittent confirmation server 230: vehicle query server

Claims (15)

In the adaptive background image modeling method for storing the location and time history of the foreground area by the object in the history management module 140 to model the background image of the area photographed by the camera while updating or globally updating the foreground area. In
A Gaussian model initialization step (S130) of initializing a Gaussian model for each pixel for the background image by calculating an average and a covariance according to the Gaussian distribution for each pixel in the initial continuous frame of the image photographed by the camera;
A foreground region extraction step S140 of extracting a foreground region by an object from an image frame after the initialization step S130 based on the pixel-specific Gaussian model;
A situation recognition step (S150) of managing the location and time history of the extracted foreground area by the history management module 140 and detecting movement or stop of the foreground area based on the history;
If the foreground area is not extracted and the extracted foreground area is moving, the Gaussian model for each pixel of the background is updated, and if the extracted foreground area stops for more than a preset time and then moves, A background update step (S160) of initializing a Gaussian model for the current pixel value;
Adaptive background image modeling method comprising a.
The method of claim 1,
The background update step (S160),
An adaptive background image modeling method comprising initializing a Gaussian model for pixels of a stopped foreground area to a pixel value of a currently extracted foreground area when the extracted foreground area stops for a predetermined time or more.
3. The method according to claim 1 or 2,
Gaussian model by pixel
An adaptive background image modeling method which is limited to a predetermined ROI in an image frame and extracts only a foreground region generated within a limited ROI.
3. The method according to claim 1 or 2,
The situation recognition step (S150),
As the Gaussian model is updated for the pixels in the foreground area that has been stopped for a predetermined time or more, when the foreground area for the stationary object is not extracted in the foreground area extraction step (S140), it is managed by the history management module 140. Adaptive background image modeling method characterized by detecting the foreground area for the object by the history of the foreground area.
3. The method according to claim 1 or 2,
Initialization of the Gaussian model for the pixel in the background update step (S160),
Adaptive background image modeling method characterized in that the pixel value corresponding to the current frame is the average value of the Gaussian model, and the covariance is the covariance value before initialization.
Store and manage the location and time history of the foreground area by the vehicle in the history management module 140 in the image taken by the camera, update the background image according to the history of the foreground area, and use the history information of the history management module 140. In the illegal parking control method for cracking down illegally parked vehicles,
A photographing step of photographing the illegal parking control area using a camera in a continuous frame (S10);
A Gaussian model initialization step (S130) of initializing a Gaussian model for each pixel for the background image by calculating an average and a covariance according to the Gaussian distribution for each pixel in the initial continuous frame of the image photographed by the camera; A foreground region extraction step S140 of extracting a foreground region by an object from an image frame after the initialization step S130 based on the pixel-specific Gaussian model; A situation recognition step (S150) of managing the location and time history of the extracted foreground area by the history management module 140 and detecting movement or stop of the foreground area based on the history; If the foreground area is not extracted and the extracted foreground area is moving, the Gaussian model for each pixel of the background is updated, and if the extracted foreground area stops for more than a preset time and then moves, A context-sensitive modeling step (S100) including a background update step (S160) of initializing a Gaussian model for the current pixel value;
An intermittent image acquisition step (S200) of sequentially obtaining at least one foreground region corresponding to the stationary object in the context-aware modeling step (S100), sequentially determining the extracted foreground region and sequentially obtaining the interrupted image including the license plate region;
Vehicle number recognition step of recognizing the vehicle number in the interrupted image (S300);
An intermittent information generation step (S400) of generating intermittent information by linking the intermittent situation information including the photographing date and time, the photographing position or the vehicle number with the intermittent image;
A crackdown determination step (S500) of determining whether to crackdown in accordance with the regulation for stopping parking by accumulating the temporal passage of the crackdown information having the same crackdown situation information;
Illegal parking control method using adaptive background image modeling, characterized in that it comprises a.
The method of claim 6,
The situational awareness modeling step (S100),
Illegal using adaptive background image modeling, which designates only the illegal parking control area as the region of interest in the captured image, and models the Gaussian model only for the region of interest, and extracts only the foreground region generated in the region of interest. Unmanned parking control method.
The method of claim 7, wherein
The control information generation step (S400),
After focusing to the foreground area and zooming in, it detects the license plate position, and focuses on the detected license plate position to obtain an image of the license plate area.
At the time of focusing, the unscaled parking is unmanned using adaptive background image modeling, which calculates the edge size of each pixel in the captured image and adjusts the camera focus according to the result of comparing the edge size with a predetermined reference value. How to crack down.
On-site system 200 for detecting the crackdown information for vehicles in the illegal parking control area, and the center system 300 for determining illegal parking based on the crackdown information,
The field system 200,
A camera 210 photographing a continuous frame according to the control of the camera controller 211;
A schedule storage unit 220 for storing mapping information on the location and distance of the area of interest AOI designated as an illegal parking control area;
An initialization module 110 for initializing the adaptive Gaussian mixture model for each pixel for the background image by calculating an average and a covariance according to the Gaussian distribution for each pixel in an initial continuous frame of the image obtained by photographing; A foreground region extraction module 120 for extracting a foreground region OOI by an object from an image frame after initialization based on the pixel-specific adaptive Gaussian mixture model; A history management module 140 for managing the location and time history of the extracted foreground area; And update the pixel-specific Gaussian model for the entire background of the background when the foreground area is not extracted and when the extracted foreground area is moving, and stops the extracted foreground area after moving for more than a preset time. A background update module 150 for initializing an average value of the adaptive Gaussian mixture model with respect to the current pixel value;
After receiving the mapping information of the foreground area that has been stopped for more than a preset time, the image is enlarged to obtain a crackdown image, and after recognizing the vehicle number from the crackdown image, the crackdown situation information including the shooting date, shooting location, or vehicle number Intermittent information acquisition unit 240 for generating intermittent information in association with the connection;
Communication unit 270 for transmitting the generated intermittent information to the center system 300;
Illegal parking unmanned enforcement system, characterized in that it comprises a.
The method of claim 9,
The background update module 150,
When the extracted foreground area stops for more than a preset time, illegal parking control is initialized by initializing the average value of the adaptive Gaussian mixture model for the pixels of the stopped foreground area to the pixel value of the currently extracted foreground area. system.
The method of claim 10,
The field system 200,
Based on the mapping information, the schedule information for sequentially photographing a plurality of ROIs is stored in the schedule storage unit 220, so that a plurality of parking stop areas designated in the field are photographed by turning one camera 210. When the parking vehicle is detected by the situation-aware modeling unit 100, the parking vehicle is photographed at a uniform resolution by focus adjustment based on mapping information of the ROI corresponding to the situation.
The background update module 150,
Illegal parking control system, characterized in that it specifies an illegal Gaussian interception area only as a region of interest in the captured image, modeling it as an adaptive Gaussian mixture model only for the region of interest, and extracting only the foreground region generated in the region of interest. .
The method according to any one of claims 9 to 11,
The intermittent information acquisition unit 240,
Watermarking the intermittent situation information to the intermittent video to be transmitted to the center system 300,
The center system 300,
Illegal parking control system, characterized in that to extract the watermarked shooting situation information from the control image and classify the control image according to the control situation information.
The method according to any one of claims 9 to 11,
The field system 200,
The on-site environment sensing unit 230 for detecting illuminance and illegal parking control system, characterized in that for controlling the shutter speed, autofocus or white balance of the camera 210 according to the illuminance.
The method according to any one of claims 9 to 11,
The field system 200,
The region of interest is partitioned into a plurality of sectors of interest, and adjacent sectors of interest are partially overlapped, and the mapping information for each of the partitions of interest is partitioned and stored in the schedule storage unit 220.
By repeatedly recording the sectors of interest and recognizing the vehicle number, it checks the history of the recognized vehicle number and generates the control image which is linked with the control situation information to the center system which is detected over the prescribed time and transmits it to the center system. Unattended parking control system, characterized in that the operation detection mode is provided with the detection information acquisition unit 240.
The method according to any one of claims 9 to 11,
The field system 200,
The center system 300 is configured in the same manner as the field system 200 and connected to the sub-site system 200A installed in a nearby site by short-range wireless communication, and receives the intermittent information obtained from the sub-site system 200A. To,
The sub field system 200A,
Illegal parking control system, characterized in that the solar cell or wind power generator is provided to cover the power.

Images