KR20110110526A - Cluster analysis method using differential image and system thereof - Google Patents

Abstract

Cluster analysis method using the difference image of the present invention, (a) receiving image information from the camera; (b) the cluster analysis system calculating a difference image based on a difference between a previous frame and a current frame from the image information; (c) binarizing and distilling the difference image, and determining a position of the moving object by determining a region of motion from the binarization processing and the distillation filtering information; (d) generating a cluster using a clustering technique from valid pixels corresponding to the determined at least one moving object position; (e) dividing the image information into a grid having a predetermined number of divisions, and counting the number of clusters for each divided region; and providing a cluster analysis method using a difference image.

Description

Cluster analysis method using system and differential image

The present invention relates to a cluster analysis method, and more particularly, to a method of detecting a movement of a moving object using a difference image according to a change of an image, and analyzing a cluster using the clustering technique from the moving objects.

In the hardware system configuration of moving object tracking, the detection of moving objects requires a system having a simple algorithm at low cost. In order to overcome the problems of the conventional method, an omnidirectional moving object is automatically detected in a continuous image obtained from a moving camera that causes background motion by using a method integrating a cumulative difference image and a motion estimation technique. An algorithm has been proposed. The cumulative difference image is an image that detects motion of an intermediate frame using three consecutive frames. The cumulative difference image is used to find a candidate region in which moving objects exist by removing a background having a uniform texture from the continuous images. The motion estimation technique is used to remove the background by using the characteristics of the fast change of the background in the continuous image acquired by the mobile camera.

On the other hand, the conventional network-based intelligent mobile robot, which is used for a variety of services, such a robot can acquire the image itself by using a mounted camera by using a low-cost embedded processor, but obtained image processing It was difficult to have a function of controlling the position by identifying the movement of the processed image. As a result, most network-based mobile robots have passively adopted moving objects, that is, human approach and direct control.

Clustering is used to classify data of similar characteristics. Among the various clustering methods, in particular, the method of Single Linkage, which performs clustering only by the distance between pixels, and the method of Ward, all pairs of each pixel in the cluster by Ward's method. There is a method of clustering pixels whose minimum sum of squares (SSW) between them is the minimum value.

Therefore, the clustering method to analyze the movement of the moving object from the continuously acquired images to obtain clustered cluster information, and to build a server system for cluster analysis and the position of the mobile robot in conjunction with the mobile robot based on this method There is a need to provide a system for control.

The present invention was devised in view of the above-mentioned necessity, obtains a difference image from successively acquired images, detects the movement of a moving object, and analyzes the difference image to generate a cluster using a clustering technique from effective pixels of the difference image. An object of the present invention is to provide a cluster analysis method and system.

In addition, another object of the present invention is to provide a mobile robot and a mobile robot integrated cluster analysis system for receiving a cluster information analyzed in conjunction with a cluster analysis system using a difference image and moving the position according to the movement of the moving object.

According to an aspect of the present invention, there is provided a cluster analysis method using a difference image, the method comprising: receiving image information from a camera; (b) the cluster analysis system calculating a difference image based on a difference between a previous frame and a current frame from the image information; (c) binarizing and distilling the difference image, and determining a position of the moving object by determining a region of motion from the binarization processing and the distillation filtering information; (d) generating a cluster using a clustering technique from valid pixels corresponding to the determined at least one moving object position; (e) dividing the image information into a grid having a preset number of divisions, and counting the number of clusters for each divided region.

According to an aspect of the present invention, there is provided a cluster analysis system using a difference image, including: a camera unit obtaining image information; An input image acquisition unit for decoding the image information from the camera unit into a format reproducible; A difference image calculation unit configured to calculate a difference image based on a difference between a previous frame and a current frame of the image information; A binarization and noise processor configured to binarize the difference image and filter noises; A cluster calculation unit for generating a cluster from an area of the effective pixel determined as a moving object from the binarization processing and miscellaneous filtered information; And a cluster number counter that counts the calculated number of clusters for each area of a preset grid.

Cluster analysis method and system using the difference image according to the present invention, by analyzing the motion of the moving object using the difference image from the obtained continuous image and clustering, providing clustered cluster information for each region of the image information accurately and efficiently It works.

In addition, by using the analyzed cluster information to determine the number of positions to move, according to the determination result by the mobile robot that can move to the location, there is an effect that can be utilized in various service areas.

1 is a flowchart illustrating a cluster analysis method using difference images according to an embodiment of the present invention.
2 is a diagram illustrating a difference image calculated by a background image, image information, and a difference between two images.
3 is a diagram illustrating a process of dividing a grid area based on image information of a camera.
4A is a diagram illustrating a process applied to a cluster by setting weights according to perspectives of image information and a concept of a set center point.
4B is a diagram illustrating an embodiment of a place where a center point of a cluster for each grid area is applied.
5 is a view showing a result of counting the number of clusters for each grid area.
6 is a diagram illustrating a part of a program screen prepared for parameter setting and cluster analysis.
7 is a diagram illustrating an XML source screen containing cluster information.
8 is a block diagram of a cluster analysis system according to an embodiment of the present invention.
9 is a block diagram of a cluster analysis system according to another exemplary embodiment of the present invention.
10 is a block diagram of a cluster analysis system according to another embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in different forms, only the embodiments to make the disclosure of the present invention complete, and having ordinary skill in the art to which the present invention pertains. It is provided to fully inform the scope of the invention, and the invention is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

1 is a flowchart illustrating a cluster analysis method using difference images according to an embodiment of the present invention.

First, image information is acquired from a camera (S100).

Also, video information obtained from a camera is basically decoded according to a video standard such as MPEG4 or H.264.

The obtained image information is provided to a cluster analysis program prepared in a cluster analysis system in advance to analyze the cluster based on the image information, and it is determined whether to set parameters (S102).

These parameters are variables necessary for cluster analysis from image information. Basically, there are grid partition setting value, binarization threshold value, ghosting processing threshold value, clustering threshold value, cluster center point setting value, and weight based on perspective. A detailed description of the influence of the parameter setting will be described later.

Subsequently, in step S102, when there is a setting request according to whether to set the parameters, the number of parameters is received and stored from the outside (S104).

If there is no setting request, a default value of preset parameters is set by default.

When the setting is completed, it is determined whether to request a cluster analysis on the program (S106).

A procedure for generating a cluster is performed based on image information of consecutive frames obtained from a camera by a cluster analysis request. First, a background image necessary for calculating a difference image containing information necessary for cluster generation is generated. (S108).

The background image is generated based on image information of continuously accumulated frames, and is preferably generated by Equation 1 below.

Here, BI is a background image, CI is a current image, n is a current cumulative frame number, k is a cumulative target frame number, and x and y are pixel coordinate numbers.

The background image calculated as in Equation 1 becomes an average value of the image information of the accumulated frames, and as a result, the pixels of the moving object that are changed from the plurality of frames become faint. In contrast, the fixed objects remain valid and the background image remains effective. Will be.

Now, the difference image DI (x, y), which is a difference between the currently input image information CI (x, y) and the background image BI (x, y) generated by the calculation of Equation 2, is calculated. It becomes (S110).

The calculated difference image is binarized by a calculation according to Equation 3 below.

는 이진화 처리된 차영상이고, F(x,y,j)와 F(x,y,k)는 각각 배경영상과 입력되는 영상정보가 되며,

는 이진화 임계값이다.

Are binarized difference images, F (x, y, j) and F (x, y, k) are background images and input image information, respectively.

Is the binarization threshold.

In the binarization process, miscellaneous noise occurs according to the magnitude of the binarization threshold, and an expansion operation is performed to filter out the miscellaneous noise (S112).

FIG. 2 illustrates an embodiment of the difference image D, which is calculated by the background image B, the currently input image information I, and the difference between the two images, and then binarized, and filtered by the expansion operation. Doing. As shown in FIG. 2, the difference image D has information about an effective pixel (white area) for determining a moving object.

Now, a clustering process is performed to generate a cluster from valid pixels (S114). The clustering technique used in the embodiment of the present invention is performed based on the ART2 algorithm, and is configured to generate clusters by connecting components between pixels at the closest distance by using Manhattan distance.

When the cluster is generated through the clustering process, the number of clusters is counted for each partition region of the preset grid (S116).

According to the counting result, the cluster can determine the largest number of regions for each region, and this region can be determined as the region where the moving object is the most dense.

FIG. 5 is a diagram showing a result of counting the number of clusters for each grid area.

6 is a diagram illustrating a part of a program screen prepared for parameter setting and cluster analysis. As shown in the part of the parameter setting of FIG. 6 (the red border area of FIG. 6), the above-described parameters are set. In the embodiment of the present invention, the program development environment used Visual Studio .NET 2005 C ++. In addition, the Open CV library is referred to to acquire and reproduce video information from a camera in real time. A SDK for a network camera and a decoder for MPEG4 / H.264 are also provided.

In more detail about the parameters, the x, y-axis grid number setting value of the x and y axis coordinates determines the number of screens to be divided in the x and y axis directions.

The binarization threshold may indicate more areas as effective pixels (white areas in FIG. 2) when the value is smaller, but blemishes frequently occur, and when the value is larger, blemishes may be less. As a result, information about many effective pixel areas may be lost.

In addition, the noise threshold is regarded as a small cluster of small size if the value is small, the larger size of the cluster may be considered as a small volume.

The clustering threshold clusters only the contours of the effective pixels at a distance smaller than the threshold value when performing clustering, and the larger the value, the larger the area becomes as a cluster.

As shown in FIG. 3, the grid area is divided based on the image information. The grid area is divided according to the setting value of the grid partition area. When the grid area is divided by m in the x-axis direction and n in the y-axis direction, You can divide by m * n (horizontal * vertical). In particular, in this case, it is not necessary to perform cluster analysis on the area where the actual mobile robot cannot move based on the floor surface, and therefore, it is preferable to set the invalid area (disable area of FIG. 6) by external input among the divided whole areas. Do.

4A is a diagram illustrating a process applied to a cluster by setting weights according to perspectives of image information, and a concept of a set center point. FIG. 4B is a view illustrating an embodiment of a place where a center point of a cluster for each grid area is applied. to be.

As shown in FIG. 4A, the weight according to the perspective is set to reflect the difference in the cluster since a distance difference occurs between clusters corresponding to different coordinate values on the Y axis. Further, in the embodiment of the present invention, the shape of the cluster is configured to be an ellipse shape so that the y axis radius is larger than the x axis radius at the center of the cluster. This is configured to generate a cluster to determine a more accurate position of the moving object by reflecting the shape of the person corresponding to the moving object. Now, the radius of the elliptic cluster is small or large according to the increase or decrease of the weight.

The center point setting is a value that sets the center point of the cluster, which is required for counting the number of clusters generated later based on this center point, and as shown in FIG. 4A to count the correct number, It is preferable to select the lower end of the as a center point. An example of such a set center point is shown in FIG. 4B.

On the other hand, it is possible to receive the cluster information generated by the request from the outside (preferably, a mobile robot) from the outside (S118, S120, S122).

The external mobile robot determines the number of positions to move based on the received cluster information, and moves to the corresponding position according to the determination result (S124, S126).

The cluster information includes at least information on the region having the largest number of clusters for each region of the grid according to the counting result. In addition, the cluster information may include information on the number of clusters of all regions, that is, regions having at least one cluster number, and the number of clusters of corresponding regions.

In addition to the information on the number of clusters for the currently analyzed image information, the regions having at least one cluster number may be calculated by calculating the number of clusters for each grid region based on the image information of frames accumulated in a certain section at the request of the mobile robot. It may also include the number of clusters per region.

That is, when the cluster information generation request is made from outside, what information (information on the cluster area / number of the current input image or the cluster area / number of the image of a frame accumulated for a certain period) is included depending on the situation. The decision is made, and a request is made to the cluster analysis system. The cluster analysis system generates cluster information including information on the number of clusters for each grid area calculated on request and transmits the information to the outside.

The cluster information generated in the cluster analysis system is preferably converted into an XML (extensible markup language) format that can be used in a wired / wireless communication network.

7 shows an XML source screen containing cluster information.

Referring to FIG. 7, cluster information (response part S of FIG. 7) may include coordinate values (position parts P1 and P2 of FIG. 7) for a grid area having at least one cluster number and the number of clusters of the corresponding area. (Value portion, V1, and V2 of FIG. 7) is basically included, and additionally, mode information (mode portion, R1) included in external request information (R portion of FIG. 7) and obtained from a plurality of cameras. It also includes identification information (camid portion, R2) of the camera 101 or 102 that transmitted the image information. Furthermore, depending on the request information from the outside, the generated cluster information F may not include any coordinate values or the number of clusters.

The request information R includes the mode information R2. In the embodiment of the present invention, the request information R has four mode values (0, 1, 2, 3). For example, mode 0 is a cluster information request including both a coordinate value and cluster number of an area having at least one cluster based on the current input image information, and mode 1 is based on the current input image information (the most recent frame image information). This is a request for cluster information including the coordinates and the number of clusters in the region with the largest number of clusters.

In addition, mode 2 is a request for cluster information including both the coordinate value and the number of clusters of a region having at least one cluster based on image information of frames accumulated in a certain section, and mode 3 is based on image information of frames accumulated in a certain section. It may be a request for cluster information including coordinates and the number of clusters in the region with the largest number of clusters.

In addition, mode 0 and mode 1 will be described in more detail with reference to FIG. 5. If the current input image information (the image information of the most recent frame) is the same as that of FIG. 5, in the case of mode 0, cluster information is generated. The grid region (coordinate value, A2, A4, B2, B4, C4, D4, E4) of at least one number includes the cluster values (4 (A2 region), 2 (A4 region), 2 (B2 area), 2 (B4 area), 4 (C4 area), 3 (D4 area), and 1 (E4 area) In addition, in mode 1, the cluster information generated includes the largest number of clusters. Only the grid area (A2 in Fig. 5) and the number of clusters (4 (A2 area)) of the area are included.

8 is a block diagram of a cluster analysis system using difference images according to an embodiment of the present invention.

Referring to FIG. 8, the cluster analysis system using the difference image of the present invention may include a plurality of cameras 100, a wired / wireless communication network 200, a cluster analysis system 300, and a mobile robot 400.

The plurality of cameras 100 are respectively installed in easy-to-shoot locations to obtain image information.

The camera is a network camera capable of transmitting the captured image information to the cluster analysis system 300 through a wired or wireless communication network, and may also be an IP camera.

The wired / wireless communication network 200 is based on a conventional wired / wireless internet system, and descriptions that do not correspond to the object of the present invention will be omitted.

The cluster analysis system 300 receives image information from a plurality of cameras 100 and generates a background image based on at least one accumulated frame of the image information, and generates a difference image by the difference between the image information and the background image. Create In addition, through the binarization processing and the noise filtering of the difference image, the position of the moving object is determined, a cluster is generated from the effective pixels of the determined position using a clustering technique, and the number of clusters is counted for each region of a preset grid. Furthermore, by receiving a request from a mobile robot connected to the wired / wireless communication network 200, the cluster information of the XML format including at least information on an area including the largest number of clusters in the grid area is generated. The generated cluster information is transmitted to the mobile robot.

The mobile robot 400 requests cluster information generation from the cluster analysis system 300 through the wired / wireless communication network 200 and receives cluster information generated according to the request.

In this case, the requested information requests to generate cluster information including the number of clusters for each grid region of the accumulated frames based on the image information corresponding to the most recent frame or the accumulated frames of a predetermined period when generating cluster information.

9 is a block diagram of a cluster analysis system according to another embodiment, and includes a mobile robot 400 in which a wired / wireless communication network 200, a cluster analysis system 300, and a camera module 410 are embedded.

The wired / wireless communication network 200 and the cluster analysis system 300 perform the same functions as the functions described above with reference to FIG. 8.

The mobile robot 400 has a camera module 410 for acquiring image information, and transmits the acquired image information to the cluster analysis system 300 through the wired / wireless communication network 200. Furthermore, in response to the cluster information generation request, the cluster analysis system 300 determines the number of positions to be moved by receiving the corresponding cluster information, and as a result, moves to the corresponding position.

10 is a block diagram of a cluster analysis system according to another exemplary embodiment. Referring to FIG. 10, the cluster analysis system 300 includes a camera unit 301, an input image acquisition unit 310, and a background image. Generation unit 320, difference image calculation unit 330, binarization and noise processing unit 340, cluster calculation unit 350, parameter setting unit 360, cluster number counter 370, cluster information generation unit 380 The camera and the mobile robot integrated system including a movement control unit 390.

The camera unit 301 obtains image information by external shooting.

The input image acquisition unit 310 receives the image information from the camera unit 301 and decodes it into a format that can be reproduced.

The background image generator 320 generates the average number of at least one accumulated frame as a background image based on the image information of the input image acquisition unit 310. The set value (section) of frames accumulated in advance may be input from the outside.

The difference image calculation unit 330 calculates a difference image based on a difference between the image information and the background image.

The binarization and noise processing unit 340 binarizes the calculated difference image and filters the noise.

The cluster calculation unit 350 generates a cluster using a clustering technique based on the ART2 algorithm in the region of the effective pixel determined as the moving object from the binarization processing and the noise filtering information (differential image).

The parameter setting unit 360 sets a grid in which an area is divided into m * n on a plane based on a frame of image information, but an invalid area for a non-movable area of a moving object based on a floor in multiple areas of the divided grid. Set to. In addition, a binarization threshold value for determining an effective pixel area for determining a position of the moving object, a clustering threshold value for determining an area of a cluster, and a buzzing threshold number for determining a bloc filtering area by dilation calculation are set. In addition, in order to calculate the number of clusters by the effective area of the grid, the lower end of the cluster is set as the center point, and the weight according to the perspective for each coordinate on the Y axis of the grid is set based on the frame of the image information.

The cluster number counter 370 counts the number of previously calculated clusters for each region of a preset grid.

The cluster information generation unit 380 generates cluster information including at least information about the grid area of the largest number of moving objects according to the counting result.

The movement controller 390 determines the number of positions to move based on the generated cluster information, and controls to move to the corresponding position.

Although embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

100: camera 101: camera 1
102: camera n 200: wired and wireless communication network
300: cluster analysis system 301: camera unit
310: input image acquisition unit 320: background image generation unit
330: difference image calculation unit 340: binarization and noise processing unit
350: cluster calculation unit 360: parameter setting unit
370: cluster number counter 380: cluster information generation unit
390: movement control unit 400: mobile robot
410: camera module

Claims (13)

(a) receiving image information from a camera;
(b) the cluster analysis system receiving the image information and calculating a difference image based on a difference between a previous frame and a current frame;
(c) binarizing and distilling the difference image, and determining a position of the moving object by determining a region of motion from the binarization processing and the distillation filtering information;
(d) generating a cluster using a clustering technique from valid pixels corresponding to the determined at least one moving object position; And
(e) dividing the image information into a grid having a preset number of divisions, and counting the number of clusters for each divided region;
Cluster analysis method using a difference image comprising a. The method according to claim 1, wherein step (b)
(b-1) determining, by the cluster analysis system, whether to generate a background image;
(b-2) generating a background image based on the image information of at least one frame according to the determination result; And
(b-3) calculating a difference image based on a difference between the background image and currently input image information;
Cluster analysis method using a difference image comprising a. The method according to claim 2, wherein step (b-2)
(b-2-a) setting a cumulative number of frames for accumulating a predetermined period from the image information; And
(b-2-b) generating a background image by calculating the average number of accumulated sections by accumulating the image information in units of frames based on the set number of frames;
Cluster analysis method using a difference image comprising a. The method according to claim 1, before step (b)
(a-1) A grid in which an area is divided into m * n on a plane is set on the basis of the frame of the image information, and an invalid area for a non-movable area of a moving object based on a bottom surface in a plurality of areas of the divided grids. Setting to;
(a-2) setting a binarization threshold value for determining an effective pixel area for determining a position of a moving object, a clustering threshold value for determining an area of a cluster, and a noise threshold number for determining a noise filtering area by dilation; ;
(a-3) setting a lower end of the cluster as a center point to calculate the number of clusters for each area of the set grid; And
(a-4) setting weights according to perspectives of coordinates on the Y-axis of the set grid based on the frames of the image information;
Cluster analysis method using a difference image further comprising. The method according to claim 1,
(f) receiving a request from the outside, the cluster analysis system generating cluster information including at least information on the largest number of grid areas in the cluster according to the counting result; And
(g) transmitting the cluster information to the requested outside;
Cluster analysis method using a difference image further comprising. The method of claim 5, wherein step (f)
(f-1) in response to the cluster information request from the outside, the cluster analysis system comprising: receiving image information of a most recent frame or a frame accumulated in a predetermined period;
(f-2) the cluster analysis system, analyzing the cluster based on image information corresponding to a most recent frame or a accumulated frame of a predetermined period according to the setting; And
(f-3) requesting generation of cluster information including the number of clusters of grid regions of frames accumulated from the outside;
Cluster analysis method using a difference image comprising a. A camera unit obtaining image information;
An input image acquisition unit for decoding the image information from the camera unit into a format reproducible;
A difference image calculation unit configured to calculate a difference image based on a difference between a previous frame and a current frame of the image information;
A binarization and noise processor configured to binarize the difference image and filter noises;
A cluster calculation unit for generating a cluster from an area of the effective pixel determined as a moving object from the binarization processing and miscellaneous filtered information; And
A cluster number counter that counts the calculated number of clusters for each region of a preset grid;
Cluster analysis system using a difference image comprising a. The system of claim 7, wherein the system is
A background image generating unit generating an average number of at least one accumulated frame as a background image based on the image information;
A cluster information generation unit configured to generate cluster information including at least cluster information about the largest number of grid areas and cluster values of the corresponding area according to a counting result of the cluster number counter; And
A grid in which an area is divided into m * n on a plane is set on the basis of the frame of the image information, and in a plurality of areas of the divided grid, a non-movable area of the movable object is set as an invalid area based on a floor surface, Set a binarization threshold value for determining an effective pixel area for determining a location of a pixel, a clustering threshold value for determining an area of a cluster, and a noise threshold number for determining a noise filtering area by an expansion calculation, and an effective area of the set grid And a parameter setting unit for setting a lower end of the cluster as a center point to calculate the number of star clusters, and setting weights according to perspectives on the Y-axis of the set grid area based on the frame of the image information. ,
The difference image calculation unit is a cluster analysis system using a difference image to calculate a difference image by the difference between the generated background image and the image information. The system of claim 8, wherein the system is
Analyzing clusters from the difference image calculated based on the image information of the accumulated frames of a certain section and counting the number of clusters for each predetermined grid area of the corresponding frame, and accumulating the most recent frame or the accumulated frame of a certain section according to an external request. And cluster information including at least one cluster area from the image information of the cluster information, and cluster information including some of information on cluster values of the corresponding areas, and transmitting the cluster information to the outside. The method according to claim 7,
And a mobile robot capable of moving to a corresponding position according to a determination result by determining the number of positions to be moved by receiving the cluster information by request through a wired / wireless communication network. The method of claim 10, wherein the mobile robot
And a camera unit to acquire the image information and to transmit the obtained image information to the system. The system of claim 7, wherein the system is
The cluster analysis system using the difference image which is a mobile robot integrated system which can determine the number of positions to move based on the cluster information and move to a corresponding position according to the determination result. The method of claim 7, wherein the camera unit,
Cluster analysis system using a difference image that is provided outside the system and the network camera or IP camera capable of wired and wireless communication with the system.

Images