KR101154350B1 - Multiple Images Processing system using Detection of Objects and segmenting - recognizing of High- resolution Image and there of. - Google Patents

Abstract

PURPOSE: A multiple images processing system and a method thereof are provided to monitor an object from a high resolution camera unit. CONSTITUTION: A high-resolution camera unit(100) photographs an image having a wide angle of a downtown area. A detected object division and recognition unit(200) generates an image pyramid from the photographed image. The detected object division and recognition unit generates a background model based on an LBP(Local Binary Pattern) histogram by using the generated image pyramid. The recognition unit divides the detected object into division window.

Description

Multiple Images Processing system using Detection of Objects and segmenting-recognizing of High-resolution Image and there of.}

The present invention relates to a multi-image processing system and method using object detection and segmentation recognition of high-resolution images, and more particularly, from one high-resolution camera unit having an angle of view capable of monitoring multiple lanes on a road and wide-area monitoring of an urban area. Generates an image pyramid by acquiring the captured image, generates a background model based on the LBP histogram using the generated image pyramid, and divides the detection object in the generated background model based on the size of the transmitted image. It detects the object from the blob of the divided detection object and divides it into the size of the window, and detects the object information of the high resolution image so that the recognized object information and the high resolution image information corresponding to the object area can be sent to the image management means for storage management. And Image Processing System and Method Using Segmentation Recognition will be.

Conventional anti-theft vehicle number recognition system or vehicle speed measurement system is applying a variety of devices or methods for detecting the vehicle.

For example, roof coil, vehicle detector, Auto IRIS lens, over 1 million pixel digital CCD camera, strobe lighting device, image frame grab board, vehicle identification key controller, etc. were applied.

Recently, an image detection method has been applied, and this method drives light emission in synchronization with a surveillance camera for photographing a surveillance area, a number recognition camera for photographing a vehicle detected from the surveillance video of a surveillance camera, and a surveillance camera and a number recognition camera. Infrared LED lighting device is installed.

In this way, the video sensing device using the camera reads the surveillance video of the surveillance camera to check the appearance of the vehicle and controls the driving of the number recognition camera and the infrared LED light so that the vehicle can be photographed by the number recognition camera while tracking the vehicle in the surveillance image. And a vehicle number recognition main controller that recognizes the vehicle number in the captured image frame, and the vehicle number recognition main controller divides the surveillance image into a read area and a non-read area to monitor only the read area to read the appearance of the vehicle.

This method does not require constant resetting of the detection area, is easy to maintain and semipermanent.

In addition, the possibility of disconnection due to road construction is excluded, and there is an advantage that little maintenance costs are incurred.

On the other hand, there is a vehicle number recognition-based vehicle detection method that recognizes the license plate by taking a still image at regular intervals from the video stream.

A real-time traffic situation detection system disclosed in Korean Patent Publication No. 2011-0099992 (September 9, 2011) discloses an object having a detected movement (for example, an object appearing on a road includes a vehicle, a person, an animal, and a fire). In order to recognize the image, the image analyzer applies the 'Haar-based object detection' method, which is a very effective method of detecting the target by learning the target using the Ad boost method.

However, this method has a problem in that the load of the system for continuously performing the identification of the vehicle number and the vehicle missed during the identification of the vehicle number occur.

In addition, it has an angle of view that does not cover multiple lanes, and the monitoring angle of view in the situation room is small, inevitably had disadvantages of control.

On the other hand, the object boundary estimation method of the video disclosed in the Republic of Korea Patent Publication No. 2009-0084460 (2009.08.05) is a method of modeling the background as the most representative method to distinguish the object and the background in the video.

In the above method, when a fixed camera is used, a pixel which does not change for a predetermined time is set as a background.

A technology that distinguishes objects and backgrounds by recognizing pixels whose pixel values rapidly change on the basis of a specified background.As the background may change in some cases, it can reduce the sensitivity to changes by updating the background at specific times. Will be provided.

Specifically, since the front and back frames of the background are similar to the background modeling method to distinguish the object from the background, this method can effectively model the background.

In addition, the modeled background can be used to quickly classify objects, which recognizes all changes as objects instead of the background.

Background modeling using the Gaussian mixture model or Kernel density estimation, such as the typical method, is to update the background to appropriately respond to instantaneous or continuous changes.

However, the above-described techniques have only one camera unit in one lane, and thus have an angle of view that cannot cover a wide range of multiple lanes. Accordingly, the situation room monitors multiple cameras and still has disadvantages of inefficient control. I had to have.

Republic of Korea Patent Publication No. 2011-0099992 (2011.09.09) Republic of Korea Patent Publication No. 2009-0084460 (2009.08.05)

The present invention has been made to solve the above-described problems, and generates an image pyramid by obtaining an image taken from one high-resolution camera unit having an angle of view capable of monitoring multiple lanes on a road and wide area monitoring of an urban area. Creates a background model based on the LBP histogram using the image pyramid, divides the detection object in the generated background model into the size of the split window based on the size of the transmitted image, and blobs of the divided detection object An object of the present invention is to provide a multi-image processing system and method using object detection and segmentation recognition of a high resolution image, which transmits and manages recognized object information and high resolution image information to an image management means.

In order to achieve the above object, a multi-image processing system using object detection and segmentation recognition of a high-resolution image according to the present invention includes a high-resolution camera unit for capturing an image having a wide field of view of a road area or a city area including multiple lanes and shoulders. Wow; Generates an image pyramid by acquiring the captured image from the high resolution camera unit, generates a background model based on the LBP histogram using the generated image pyramid, and detects the detected object in the generated background model of the transmission image. A detection object division recognizing means for dividing the image into a size of a partition window based on the size, and recognizing the object from the blobs of the divided detection objects and transmitting the recognized object information and the high resolution image information to the image management means; And image management means for outputting or managing object information and high resolution image information transmitted from the detection object division recognizing means on a display screen.

Detection object segmentation means of the multi-image processing system using the object detection and segmentation recognition of the high-resolution image according to the present invention is an image providing unit for receiving the high-resolution image information captured from the high-resolution camera unit to provide to the image pyramid generation unit, the image providing unit A pyramid image generation unit generating a pyramid image by performing Gaussian pyramid sampling on an image provided by the Pyramid image generation unit and a background model based on an LBP histogram for the pyramid image provided by the pyramid image generation unit. And a background model generator for generating the object model and providing the object model to the object recognizer by dividing the detection object in the background model provided by the background model generator into a partition window size based on the size of the transmission image. , The partitioned object provided by the object partitioner Recognizing a human body or a vehicle from a blob, an object recognition unit for estimating a key in the case of a human body and a character of a license plate in the case of a vehicle and providing the information to the information transmitting unit, and object recognition information provided by the object recognition unit And an information transmitter for transmitting the captured high resolution image information to the image management means.

As described above, the multi-image processing system and method using the object detection and segment recognition of the high resolution image of the present invention,

Image pyramid is generated by acquiring images captured from one high resolution camera unit having multiple angles of view and wide angle monitoring of urban areas on the road.Based on LBP histogram, the generated image pyramid is used. The background model is created, the detection object in the generated background model is divided into the size of the split window based on the size of the transmitted image, the object is recognized from the blobs of the divided detection object, By transmitting the corresponding high-resolution image information to the image management means for storage management, it is possible to provide an effect of reducing the installation cost and the efficiency to monitor the object in multiple lanes with one high-resolution camera unit instead of a plurality.

1 is an overall configuration diagram of a multiple image processing system using object detection and segmentation recognition of a high resolution image according to an embodiment of the present invention.
2 is a block diagram of detection object recognition means of a multi-image processing system using object detection and segmentation recognition of a high resolution image according to an embodiment of the present invention.
3 is a block diagram of a background model generator of a multi-image processing system using object detection and segmentation recognition of a high resolution image according to an exemplary embodiment of the present invention.
4 is a screen illustrating an example of generating a bounding box after removing noise of a multi-image processing system using object detection and segmentation recognition of a high resolution image according to an exemplary embodiment of the present invention.
5 is an exemplary diagram illustrating a LBP calculation process of a multi-image processing system using object detection and segmentation recognition of a high resolution image according to an embodiment of the present invention.
FIG. 6 is an exemplary diagram illustrating a partition window when a center of a detection object of a multi-image processing system using object detection and segmentation recognition of a high resolution image is included in a stable partition area according to an embodiment of the present invention; FIG.
FIG. 7 is an exemplary diagram illustrating a split window when a center of a sensing object of a multi-image processing system using object detection and segmentation recognition of a high resolution image is out of a stable segmentation area according to an embodiment of the present invention; FIG.
8 is a flowchart of a multi-image processing method using object detection and segmentation recognition of a high resolution image according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of a multi-image processing system and method using object detection and segmentation recognition of a high resolution image will be described in detail with reference to the accompanying drawings.

1 is an overall configuration diagram of a multi-image processing system using object detection and segmentation recognition of a high resolution image according to an embodiment of the present invention.

As shown in FIG. 1, the multi-image processing system using the object detection and the segmentation recognition of the high resolution image of the present invention,

A high resolution camera unit 100 for capturing an image having a wide angle of view of a road area or a downtown area including multiple lanes and shoulders;

Generates an image pyramid by acquiring the captured image from the high resolution camera unit, generates a background model based on the LBP histogram using the generated image pyramid, and detects the detected object in the generated background model of the transmission image. A detection object division recognizing means (200) for dividing the image into a size of a partition window based on the size, and recognizing the object from the blobs of the divided detection objects and transmitting the recognized object information and the high resolution image information to the image management means;

And image management means 300 for outputting or managing the object information and the high resolution image information transmitted from the detection object division recognizing means on a display screen.

The high resolution camera unit 100 is preferably applied to an ultra high resolution digital camera of 5M or more, and when using this, it is possible to obtain an ultra high resolution image of 2448 x 2048 or more.

Since the ultra-high resolution image has a wide angle of view, it has a wide angle of view of a road area or a downtown area including multiple lanes and shoulders.

However, since a technique for efficiently classifying objects is required to process them efficiently, the object segmentation method of the present invention is to divide the size of the divided window based on the size of the transmission image.

The detection object dividing recognition unit 200 generates an image pyramid by obtaining an image captured from a high resolution camera unit, and generates a background model based on a histogram of an LBP using the generated image pyramid. The detection object in the generated background model is divided into the size of the split window based on the size of the transmission image, and the object information is recognized by recognizing the object from the blobs of the divided detection object and the high resolution image information corresponding to the object area. Is sent to the image management means.

The image management means 300 outputs or manages object information and high resolution image information transmitted from the detection object division recognizing means on the display screen.

That is, it is possible to monitor the multiple lanes photographed by one camera, and the monitoring angle becomes wider, thereby reducing the screen to be monitored.

In addition, the image management means 300,

And a motion information database for storing and managing vehicle number information and motion object detection information.

That is, the partition window (bounding box processed image) including the detection object partitioned by the detection object partition recognition means of the present invention and the vehicle number information recognized on the partition window is stored and managed in the motion information database, When saving, it can be saved including ID or location information of the high resolution camera that captured the image, and shooting time information.

The moving object information refers to a split window of a moving vehicle and a split window of a moving human body.

Since only the rectangular box processed by the bounding box is stored and managed as described above, the moving object information is retrieved from 1 pm to 2 pm on November 30, 2011, for example, in a control center having image management means installed as needed. By extracting only the bounding boxed rectangular box obtained in the corresponding time zone, efficient search is possible.

In addition, since information on the region where the motion occurred within the ultra-high resolution is also stored, it is also possible to search for a specific area designated by an administrator in the wide-area surveillance image.

Conventionally, since the background image itself including the detection object information is searched, a high-quality hardware device must be used for the search, thereby increasing the installation cost and checking only the object where the event is generated. This required considerable retrieval time, which forced the administrator to overwork.

However, in the case of the present invention, only the detection object processed by the rectangular box except the background image is searched as described above, so that a high-quality hardware device is not required for the search, and a synergistic effect is provided to shorten the search time and minimize the administrator's work fatigue. Will occur.

2 is a block diagram of detection object recognition means of a multi-image processing system using object detection and segmentation recognition of a high resolution image according to an exemplary embodiment of the present invention.

As shown in FIG. 2, the detection object dividing recognition means 200 is:

An image providing unit 210 receiving high resolution image information photographed from the high resolution camera unit and providing the image information to the image pyramid generation unit;

A pyramid image generator 220 for generating a pyramid image by performing Gaussian pyramid sampling on an image provided by an image provider and providing the background model generator;

A background model generator 230 for generating a background model based on an LBP histogram for the pyramid image provided by the pyramid image generator and providing the background model to an object divider;

An object splitter 240 for dividing the detection object in the background model provided by the background model generator into a split window size based on the size of the transmission image, and providing the detected object to the object recognizer;

Recognizing a human body or a vehicle from a blob of the divided object provided by the object splitter, an object recognition unit for estimating a key in the case of a human body and recognizing a character of a license plate in the case of a vehicle and providing the information to the information transmitting unit ( 250),

And an information transmitter 260 for transmitting the object recognition information and the high resolution image information captured by the object recognition unit to the image management means.

The image providing unit 210 receives the high resolution image information photographed from the high resolution camera unit and provides the image pyramid generation unit.

In this case, the provided image is an ultra high resolution image image of 2448 x 2048 or more.

The pyramid image generator 220 generates a pyramid image by performing Gaussian pyramid sampling on the image provided by the image provider and provides the generated background model to the background model generator.

In other words, the Gaussian pyramid sampling is a repetitive filtering and subsampling of the original image to obtain successive images with reduced resolution.

This speeds up the computational speed and feature point detection through the image processing on the reduced image, so that ultra-high resolution video images of 2448 x 2048 or higher can be processed at high speed.

The background model generator 230 generates a background model based on an LBP histogram for the pyramid image provided by the pyramid image generator and provides the object model to the object divider.

That is, in order to generate a solid background model, the background model is trained based on the LBP histogram (local binary pattern histogram), which speeds up the performance of object recognition and provides a very stable recognition rate.

Since the generation process is processed through the module configured in FIG. 3, it will be described in detail below.

The object dividing unit 240 divides the detection object in the background model provided by the background model generating unit into a split window size based on the size of the transmission image and provides the object recognition unit.

Specifically, when using an ultra high resolution digital camera of 5M or more, an ultra high resolution image of 2448 x 2048 or more can be obtained. Since the ultra high resolution image has a wide angle of view, it is possible to acquire images including two-lane roads and shoulders.

However, since the step of dividing the object is required for efficient remote transmission, the object is divided by the object dividing unit.

That is, the object segmentation is divided into the size of the partition window based on the size of the transmission image.

In this case, when the center of the detected object 20 is located in the internal stable partition area 10 of the original high resolution image, the center of the detection object in the background model and the center of the transmission reference image window as shown in FIG. If the center of the detection object is located in an area outside the internal stable partition, otherwise, the boundary of the internal stable partition is divided to be the center of the partition window 30 as shown in FIG. 7.

The segmentation of the transmission image is set by transmitting a segmentation window having a constant size based on the center of the detection object.

In this case, the transmission image having a constant size is generated only when the center setting of the detection object is included in the stable partition area and when it is not included.

This is because the image with the margin in the blob in which the motion is generated can be used as the image data including the more reliable detection object than in storing the image corresponding to the motion area blob.

The center of the detected object is calculated by the processing of the object divider, and the size of the detected object in the background model is determined by using a connected component technique, which creates a bounding box of the detected object as shown in FIG. After that, the center of the index will be calculated.

The object recognition unit 250 recognizes a human body or a vehicle from a blob of the divided object provided by the object divider, but estimates the height in the case of the human body and transmits the information by recognizing the character of the license plate in the case of the vehicle. Will be provided as wealth.

The technology for determining the license plate area and position of the image, the technology for recognizing the license plate, and the technology for estimating the height of the human body are well known to those skilled in the art, and thus detailed descriptions thereof will be omitted.

Thereafter, the information transmitter 260 transmits the object recognition information and the captured high resolution image information provided by the object recognition unit to the image management means.

3 is a block diagram of a background model generator of a multi-image processing system using object detection and segmentation recognition of a high resolution image according to an exemplary embodiment of the present invention.

As shown in Figure 3, the background model generation unit 230,

A block histogram calculation module 231 for calculating a block LBP histogram for each frame of the pyramid image;

A model histogram comparison module 232 for comparing a block LBP histogram calculated by the block histogram calculation module with a model histogram,

A background histogram setting module 233 for setting a background histogram among model histograms using the persistence of the histogram represented by the similarity between the frames;

The background histogram set by the background histogram setting module is characterized in that it comprises a background model selection module 234 for selecting as a background model when it is determined to exceed the predetermined threshold value.

The process will be described in detail with reference to FIG. 5, and assume a case of a 3 × 3 mask.

The learning process for creating a background model based on the histogram of LBP features is as follows.

The block histogram calculation module 231 calculates a block LBP histogram for each frame of the pyramid image (using binarization by a center value).

That is, a block LBP histogram x _t is calculated for each frame-input image.

In this case, the model histogram comparison module 232 compares the block histogram (LBP) histogram calculated by the block histogram calculation module with the model histogram.

In other words, the LBP histogram x _t calculated by the block histogram calculation module is compared with respect to the K model histograms {x ₁ , x ₂ , ..., x _t }.

In this case, the comparison measure uses a histogram intersection method.

Thereafter, the background histogram setting module 233 sets the background histogram among the model histograms using the persistence of the histogram indicated by the similarity between the frames.

In other words, it is to select which model histogram is best suited for the background. For this purpose, the histogram persistence which is shown due to the similarity between frames is used.

Specifically, k ^th Since the persistence of the model histogram is directly related to the weights w _{k , t} , the model histograms are sorted in descending order by weight so that the most likely background histogram is at the top of the sort list.

The background model selection module 234 determines whether the background histogram set by the background histogram setting module exceeds a preset threshold and selects it as a background model.

That is, the B model histograms located at the top of the sort list as the last step of the update procedure are selected to be the background model when the following Equation 1 condition is satisfied.

Here, T _B is a threshold value designated by a user.

w _{1 , t} + w _{2 , t} + ... + w _{B , t} > T _B 0 ≤ T _B ≤ 1 (Equation 1)

The w _{1 , t,} w _{2 , t,} w _{B , t} means weights given to the bars of the histogram.

At this time, the VIEW B-detection of background histogram is selected in the previous frame with respect to the new block histogram _{{x 1, x 2, ...} , x B} x t If no match is found, the block belongs to the foreground, otherwise the block is determined as the background.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

8 is a flowchart of a multi-image processing method using object detection and segmentation recognition of a high resolution image according to an embodiment of the present invention.

As shown in FIG. 8, in the multi-image processing method using object detection and segmentation recognition of a high resolution image,

Receiving the high resolution image information captured by the high resolution camera unit and providing the high resolution image unit to the image pyramid generator 600;

Generating a pyramid image by performing Gaussian pyramid sampling on the provided image, and providing the generated pyramid image to the background model generator (610);

Generating a background model based on an LBP histogram with respect to the provided pyramid image, and providing the generated background model to the object dividing unit (620);

Dividing the detection object in the provided background model into a split window size based on the size of the transmission image, and providing the split partition window to the object recognition unit (630);

Recognizing a human body or a vehicle from a blob of the provided divided object, estimating a key in the case of a human body and recognizing a character in a license plate in the case of a vehicle and providing the information to the information transmitting unit (640);

And transmitting the provided object recognition information and the captured high resolution image information to the image management means (650).

That is, the image providing unit 210 receives the high resolution image information photographed from the high resolution camera unit and provides the image pyramid generation unit (step 600).

In this case, the pyramid image is generated by performing Gaussian pyramid sampling on the image acquired by the pyramid image generator 220, and provides the generated pyramid image to the background model generator (step 610).

Thereafter, the background model is generated based on the LBP histogram with respect to the pyramid image acquired by the background model generator 230, and the generated background model is provided to the object splitter (step 620).

That is, the learning process for generating the background model based on the histogram of the LBP characteristic is performed, and thus, the adaptive background generation is possible.

Thereafter, the detection object in the background model obtained by the object dividing unit 240 is divided into a split window size based on the size of the transmission image and provided to the object recognizing unit (step 630).

Through the object segmentation process, a super resolution image of 2448 x 2048 or more can be obtained when a 5M class or higher resolution digital camera is used.

That is, since the ultra-high resolution image has a wide angle of view, it is possible to acquire an image including a two-lane road and a shoulder.

Thereafter, the human body or the vehicle is recognized from the blobs of the divided objects obtained by the object recognition unit 250, in the case of the human body, the key is estimated, and in the case of the vehicle, the character of the license plate is provided to the information transmitting unit. (Step 640)

Thereafter, the object recognition information and the captured high resolution image information acquired by the information transmitting unit 260 are transmitted to the image managing means (step 650).

Accordingly, the image management means 300 configured at a remote location outputs or stores and manages the received object recognition information and the captured high resolution image information on the display screen.

For example, if the background model is sent to the image management means once every minute and the recognition information of the detection object is sent out, an ultra-high resolution (2448 x 2048 or more) image of 5M or more can be obtained.

This will cover all lane surveillance.

The transmitted information is represented by mapping the start point (top left) and the end point (bottom right) of the detection object to the high resolution image by referring to the object recognition information and the position coordinate value of the detection object for the image divided into the high resolution image. Is to be done.

In addition, since the object detection information is stored and used in the motion information database together with the vehicle recognition information, more efficient video surveillance can be provided.

Although the present invention has been described with reference to the embodiments shown in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

100: high resolution camera unit
200: detection object detection means
300: image management means

Claims (7)

delete delete delete In a multi-image processing system using object detection and segmentation recognition of high resolution image,
A high resolution camera unit 100 for capturing an image having a wide angle of view of a road area or a downtown area including multiple lanes and shoulders;
Generates an image pyramid by acquiring the captured image from the high resolution camera unit, generates a background model based on the LBP histogram using the generated image pyramid, and detects the detected object in the generated background model of the transmission image. A detection object division recognizing means (200) for dividing the image into a size of a partition window based on the size, and recognizing the object from the blobs of the divided detection objects and transmitting the recognized object information and the high resolution image information to the image management means;
And image management means 300 for outputting or managing the object information and the high resolution image information transmitted from the detection object division recognizing means on a display screen.
The detection object division recognition means 200,
An image providing unit 210 receiving high resolution image information photographed from the high resolution camera unit and providing the image information to the image pyramid generation unit;
A pyramid image generator 220 for generating a pyramid image by performing Gaussian pyramid sampling on an image provided by an image provider and providing the background model generator;
A background model generator 230 for generating a background model based on an LBP histogram for the pyramid image provided by the pyramid image generator and providing the background model to an object divider;
An object splitter 240 for dividing the detection object in the background model provided by the background model generator into a split window size based on the size of the transmission image, and providing the detected object to the object recognizer;
Recognizing a human body or a vehicle from a blob of the divided object provided by the object splitter, an object recognition unit for estimating a key in the case of a human body and recognizing a character of a license plate in the case of a vehicle and providing the information to the information transmitting unit ( 250),
And an information transmitter 260 for transmitting the object recognition information and the high resolution image information photographed by the object recognition unit to the image management means.
The object dividing unit 240,
If the center of the detected object is located within the stable segmentation area of the high resolution image, the region where the center of the detected object is aligned with the center of the transmission reference image window in the background model is divided, and the background is outside the stable segmentation area of the high resolution image. When the center of the detection object in the model is located, the multi-image processing system using the object detection and segmentation recognition of the high resolution image, characterized by dividing the boundary of the stable partition to be the center of the partition window.
delete delete delete

Images