TWI463416B - A foreground separation method - Google Patents

Description

A method of foreground separation

The present invention is related to a foreground separation method, and more specifically, the present invention relates to a foreground separation method for identifying a foreground image with respect to a target of a specified target.

Nowadays, the development of science and technology is changing with each passing day. Whether it is in the speed of hardware computing or the development of software technology, it has amazing results and continues to progress. The related research on image processing and computer vision has been in recent years. It is growing rapidly and is therefore widely used. In addition, the dynamic analysis and tracking of images has gradually matured. In terms of security monitoring, in addition to the basic functions of basic surveillance and video archiving, it is also necessary to process the continuous images captured by the photographic lens through the related technologies of computer vision and image processing. Analysis to provide detection, identification, tracking, and anomaly warning for specific areas. In the application of the technology, it is necessary to cooperate with effective data processing methods to achieve better results. Among them, how to effectively separate the foreground image and the back view image (or background) from multiple images in multiple images is an important step in monitoring the operation of the system.

In order to effectively distinguish and capture the background and foreground from the image, most of the monitoring systems today use the background subtraction method, the time difference subtraction method and the Gaussian mixture background model. However, both the background subtraction method and the time difference subtraction method need to have a clean background. In other words, if the rear view of the image is too complicated, the scene created or captured will be fragmented and unusable.

To this end, the Gaussian mixture model can overcome the problem of poor quality in the foreground. However, the Gaussian mixture model also has its shortcomings. When the object to be reconnaissed stays on the screen for too long, the Gaussian mixture model will regard the image of the object as part of the background, and the longer the object stays, the longer it will be. The ability of the Gaussian mixture model to identify the target will be quickly degraded. The phenomenon of the opening will directly affect the accuracy of the foreground detection, and will also affect the correctness of the follow-up.

In order to overcome the problem of opening up, the current industry has used a time control variable to extend the blending time to reduce the influence of the static state of the target on its ability to recognize. However, if the object's static time is long, the foreground will still merge into the background and become part of the background. Furthermore, Li et al., in the 2007 issue of Journal of Computer Applications, No. 27, No. 8, pages 2014 to 2017, discloses a method called "Improvement on adaptive mixture Gaussian background model," Method to improve the problem of opening up. However, please refer to FIG. 1. FIG. 1 depicts the effect of using Li to perform foreground separation. Figure 1 is divided into four sections, T1, T2, T3 and T4, which respectively reveal the images of Li's method in the 139th frame, the 209th frame, the 274th frame and the 308th frame, and the corresponding images. Prospect detection results. It can be clearly seen from the figure that the recognition ability of the foreground will be greatly degraded over time, and by T4 it is impossible to extract the foreground of the subject from the image.

Based on this, how to develop a method that can effectively overcome the problem of the foreground object in the traditional Gaussian mixture model and to integrate the background problem in the traditional Gaussian mixture model while separating the foreground and the background is an urgent problem to be solved in the technical field.

In view of the above, one aspect of the present invention is to provide a foreground separation method, and more particularly, to a foreground separation method for identifying a foreground image with respect to a target of a specified target. The invention establishes a reference back-view image as a reference for image processing, and then replaces the corresponding portion of the image to be separated according to the foreground path, and then performs a difference to find a foreground corresponding to one of the specified targets. Imagery, to achieve the benefits of the future.

Specifically, the foreground separation method of the present invention includes steps S1 to S5. Step S1 is to sequentially prepare a first image, a second image, a third image, and a fourth image. Step S2 is to establish a reference background image by using the first image, the second image, and the third image. Step S3 is to establish a foreground path by using the first image, the second image, and the third image. Step S4 is to generate a standard background image by using the fourth image, the reference background image, and the foreground path. Step S5 is to compare the fourth image by using the standard background image to obtain a foreground image corresponding to one of the specified targets.

In the actual application, the step S2 of the upper step further includes a plurality of sub-steps for respectively separating a corresponding static area from the first image, the second image, and the third image, and collecting each static area. Generate a reference background image.

The step S3 of the upper step further includes a sub-step of respectively capturing a corresponding foreground image by using a Gaussian mixture model for the first image, the second image, and the third image; And the second image and the position of the third image foreground image, estimating a predicted position of the front image of the fourth image by a second predetermined method; and utilizing the foreground image corresponding to the first image, the second image, and the third image The image and the predicted position are taken to form a joint to form a foreground path.

In addition, the second predetermined method of the upper finger refers to substituting the positions of the first image, the second image, and the third image into a Kalman filter for calculation.

Furthermore, the step S4 of the upper step further includes a sub-step of respectively capturing a corresponding back-end image from the fourth image and replacing the fourth image by using a region of the reference background image corresponding to the foreground path. Relative to the area of the foreground path to produce a standard background image.

In summary, the present invention discloses a method for effectively overcoming the background problem when the foreground object stays too long in the traditional Gaussian mixture model while separating the foreground and the background. Furthermore, the present invention further improves the conventional Gaussian mixture model, which is immersed in the background due to the long time in which the foreground stays in place, and allows the foreground to be generated after entering the warning zone. Problems that cannot be detected further enhance the accuracy of the conventional monitoring system.

Unless otherwise defined, all technical and scientific terms used in the specification have the same meaning meaning In addition, the present description is only one of the many example methods of the present invention. In the actual use of the present invention, any method or means similar or equivalent to the method and apparatus described in the present specification may be used. It. Furthermore, one or more of the numbers mentioned in the specification include the number itself. In addition, if the specification refers to the electrical connection or coupling between a certain type A and a certain type B, it means that the transmission behavior of energy and data or signals of a certain type A and a certain type of line is not limited by the actual connection. Therefore, all wireless network and optical signal transmission behaviors fall within its scope. Furthermore, the word "this" in the specification is synonymous with the term "the invention".

It should be understood that the present disclosure discloses certain methods and processes for performing the disclosed functions, and is not limited to the order described in the specification, unless the specification is explicitly excluded, otherwise the steps of the steps and processes are used. Freely adjusted by the requirements of the person. Furthermore, the ratios between the various elements in the drawings in the present specification have been adjusted to maintain the simplicity of the drawings, and therefore, the corresponding size and position of the respective elements in the drawing are as described in the specification. The shapes are for reference only, and the arrangement of the features such as the size, position, and shape of each component can be freely changed depending on the requirements of the user without departing from the inventive concept of the present invention. Further, since the properties of the respective elements of the present invention are considered to be similar to each other, the descriptions and reference numerals between the respective elements apply to each other.

In order to make the invention more apparent, the following detailed description of the invention and the examples thereof are included to provide a better understanding of the invention. The present invention discloses a foreground separation method. Briefly, the present invention replaces the corresponding portion of the image to be separated according to the foreground path by establishing a reference background image as a reference for image processing, and then A difference is made to find a foreground image corresponding to one of the specified targets to achieve the effect of the foreground.

In addition, the present invention can be roughly scored into four parts: "establishing a reference back view image", "establishing a foreground path", "establishing a standard back view image", and "identifying a target foreground image". The department will explain separately.

Referring to FIG. 2, FIG. 2 is a flow chart showing the steps of a preferred embodiment of the foreground separation method of the present invention. As can be seen from FIG. 2, the foreground separation method 1 of the present invention includes steps S1 to S5. First, the first operation part is "establishing a reference background image", which mainly includes step S1 and step S2. Step S1 is to sequentially prepare the first image G1, the second image G2, the third image G3, and the fourth image G4. The term "preparation" refers to the use of an image capture device, import from a database, or other similar means.

The first image G1, the second image G2, the third image G3, and the fourth image G4 respectively represent one of a plurality of frames captured at each time. Furthermore, the shooting time of the second image G2 is compared. The first image G1 is late, and so on. It should be noted that the shooting interval between the images on the top is not limited. In order to keep the description concise, this embodiment only uses four images to explain the essence of the present invention. However, in practical applications, the number of images may be thousands to tens of thousands.

In addition, in order to further describe the nature of each image, please refer to FIG. 3A to FIG. 3E, and FIG. 3A and FIG. 3E respectively depict the first image G1 in the preferred embodiment of the present invention. A schematic diagram of the first image G1 after view image B, the first image G1 foreground image F, the static region of the first image G1, and the dynamic region A of the first image G1. As can be seen from FIG. 3A to FIG. 3E, the front view and the rear view of the first image G1 are represented by the first foreground image F and the first back view image B, respectively. The definition of the foreground and the background is a matter of fact, so it is omitted here. In the present embodiment, the foreground image F of the present invention is composed of the shape of a black-and-white map, and the shape of the white map corresponds to the shape of the designated target 10 to represent the position of the index of 10.

In addition, the first image G1 of the present invention also defines a static area S and a dynamic area A. The static area S refers to a relatively stable area in the image. More specifically, please refer to FIG. 3D and FIG. 3E. If the system detects a moving target 10 in the image, the position of the moving target 10 will be defined as a dynamic area A. . At the same time, the part of the specified target 10 other than the dynamic area A will be defined as a static area. In addition, in the present embodiment, the block of the corresponding shape mentioned above is slightly larger than the designated object 10. Furthermore, it is considered that the properties of the second image G2 and the third image G3 are similar to those of the first image G1, and thus will not be described herein.

After each image is prepared, step S2 is performed. Referring to FIG. 2 again, step S2 is to establish a reference background image GR1 by using the first image G1, the second image G2, and the third image G3. To be more specific, please refer to Figure 4, which depicts the way the reference background image GR1 is created. As can be seen from FIG. 4, the reference background image GR1 is formed by mutually compensating and combining the static regions of the first image G1, the second image G2, and the third image G3. It should be noted that in actual applications, the image will be up to thousands of images. At this time, the reference background image GR1 will be continuously updated by the static area in each image with time. More specifically, the system can update the area of the image without moving objects to the reference background image GR1 in a few seconds, and does not update when there is continuous and dramatic change on the screen. Action, in this way, the present invention can avoid the subsequent error caused by the specified target 10 in the image being accidentally stored in the reference background image GR1. In addition, if the picture is completely still, the update of the reference background image GR1 is stopped as appropriate to maintain its correctness.

While performing the first operational part of the present invention, the Gaussian mixture model should be continuously used to monitor the foreground image F of the first image G1, the second image G2, and the third image G3, and the foreground matching and the foreground related information are updated. . If the foreground image F corresponding to the specified target 10 is continuously matched multiple times during the object tracking, it means that the foreground image F has appeared on the screen for a certain period of time, and the foreground image F has the possibility of being dissolved in the background and the background. . Accordingly, the second operational portion of the present invention will be activated.

The second operational part of the present invention is "establishing a foreground path", which mainly includes step S3. Step S3 is to establish a foreground path 30 by using the first image G1, the second image G2, and the third image G3.

Specifically, the operation mode of the foreground path 30 is as follows. First, the first image G1, the second image G2, and the third image G3 are respectively calculated by a Gaussian mixture model to obtain a plurality of corresponding foreground images F. Next, the start position and the end position of each foreground image F are collected. Furthermore, the Kalman filter is used to estimate the position of each foreground image F, the moving rate, the coordinate change mode or other parameters, and the foreground image F of the fourth image G4 is estimated by a conventional image processing method. The location that should exist. The position of the foreground image F of the fourth image G4 calculated by the upper open program is defined as one predicted position 20 as depicted in FIG. Figure 5 depicts a schematic diagram of predicted position 20 in a preferred embodiment of the present invention.

After estimating the predicted position 20 where the foreground image F should exist, the foreground image F corresponding to the first image G1, the second image G2, and the third image G3 is respectively combined with the predicted position 20 of the upper image, that is, A front view path 30 as shown in FIG. 6 can be obtained. FIG. 6 depicts a schematic diagram of the foreground path 30 in the preferred embodiment of the present invention.

After obtaining the foreground path 30, the third part of "Building a standard background image" will be performed. Referring to FIG. 7, FIG. 7 is a schematic diagram showing the formation of a standard back scene image 2 of a preferred embodiment of the present invention. The replacement image of the third operational part mainly includes step S4. Step S4 is to generate a standard background image GR2 by using the fourth image G4, the reference background image GR1, and the foreground path 30.

More specifically, the operation of step S4 is as follows. First, a corresponding back scene image B is captured from the fourth image G4 in a Gaussian mixture model. It should be noted that the fourth image G4 is defined as the foreground of the designated target 10 in the figure, and there is no image captured by the system. The reason why it cannot be retrieved may be that the designated target 10 is partially integrated into the background image B because it is stationary for a long time. At this time, the foreground image of the designated target has been dissolved into the fourth image G4 to a certain extent, and the effect of the knowledge is affected.

Then, the reference rear view image GR1 is used to replace the corresponding area on the fourth image G4 by the area corresponding to the foreground path 30. This will eliminate the possibility that the fourth image G4 may include the foreground image F of the specified target 10. Rear view image GR2. Since the potential area of the designated standard back scene image GR2 may have been replaced by the reference back scene image GR1, the standard back scene image GR2 will not have the image of the specified target 10 or its corresponding foreground.

After obtaining the standard background image GR2, the fourth "identification target image" is performed. The fourth program includes step S5. The step S5 is to compare the fourth image G4 by using the standard background image GR2 to obtain a foreground image GF corresponding to one of the specified targets. The term "comparison" above refers to differential operations or other image processing methods used to compare patterns. Considering that the standard background image GR2 will not contain the foreground image of the specified target, the foreground image GF of the specified target 10 can be clearly obtained by making a difference to the two patterns.

Finally, to illustrate the effects of the present invention, reference is made to Figure 8, which depicts a comparison of effects with prior art in a preferred embodiment of the present invention. As can be seen from the figure, in the prior art at the 308th grid, the foreground is completely dissolved in the background and the specified target 10 cannot be distinguished. In contrast, the present invention remains clearly visible at the 308th grid, until the 438th grid, the designated target 10 remains clear and unaffected. In summary, the present invention discloses a method for effectively overcoming the background object in the traditional Gaussian mixture model while separating the foreground and the background, and letting the foreground not enter the warning zone. Staying and creating undetectable problems enhances the accuracy of the conventional monitoring system.

The features and spirit of the present invention will be more apparent from the detailed description of the preferred embodiments. On the contrary, the intention is to cover various modifications and equivalents within the scope of the invention as claimed. Therefore, the scope of the patented scope of the invention should be construed in the broadest

1. . . Foreground separation method

10. . . Specified target

20. . . Forecast position

30. . . Foreground path

A. . . Dynamic area

B. . . Rear view image

F. . . Foreground image

G1. . . First image

G2. . . Second image

G3. . . Third image

G4. . . Fourth image

GR1. . . Reference background image

GR2. . . Standard rear view image

GF. . . Target image

S. . . Static area

S1~S5. . . Step flow

T1~T4. . . Interval

Figure 1 depicts the effect of foreground separation using prior art techniques.

Figure 2 is a flow chart showing the steps of a preferred embodiment of the foreground separation method of the present invention.

Figure 3A is a schematic illustration of a first image in a preferred embodiment of the present invention.

Figure 3B depicts a first image rear view image in a preferred embodiment of the present invention.

Figure 3C depicts a first image foreground image in a preferred embodiment of the present invention.

Figure 3D depicts the dynamic region of the first image in a preferred embodiment of the present invention.

Figure 3E depicts a static region of a first image in a preferred embodiment of the present invention.

Figure 4 is a diagram showing the manner in which the reference rear view image is created in the preferred embodiment of the present invention.

Figure 5 is a schematic diagram showing the predicted position in a preferred embodiment of the present invention.

Figure 6 is a schematic diagram showing the foreground path in a preferred embodiment of the present invention.

Figure 7 is a block diagram showing the formation of a standard back scene image in a preferred embodiment of the present invention.

Figure 8 is a graph comparing the effects of the prior art with the prior art in a preferred embodiment of the present invention.

[Main component symbol description]

1. . . Foreground separation method

S1~S5. . . Step flow

Claims (5)

A foreground separation method for identifying a foreground image with respect to a target of a specified target, the foreground separation method comprising the following steps: Step S1: sequentially preparing a first image, a second image, and a third An image and a fourth image; Step S2: using the first image, the second image, and the third image to create a reference background image; Step S3: using the first image, the second image, and the third The image is used to establish a foreground path; step S4: using the fourth image, the reference background image and the foreground path to generate a standard background image; and step S5: using the standard background image to perform the fourth image Comparing, in order to obtain a foreground image corresponding to the target of the specified target; wherein, step S3 further comprises the following sub-steps: respectively, the first image, the second image, and the third image are respectively specified Taking a corresponding foreground image from the ground; according to the positions of the foreground images relative to the first image, the second image, and the third image, in a second predetermined method Estimating a predicted position of one of the foreground images of the fourth image; and taking a combination of the foreground image corresponding to the first image, the second image, and the third image, and the predicted position to form the foreground path. The method of separating the foreground according to claim 1, wherein the step S2 further comprises the following substeps: separating the corresponding static from the first image, the second image and the third image respectively a region; and merging the static regions to generate the reference background image. The method for separating a foreground according to claim 1, wherein the specified method refers to performing a calculation using a Gaussian mixture model. The method for separating a foreground according to claim 1, wherein the second predetermined method refers to substituting the positions of the first image, the second image, and the third image into a Kalman filter for calculation. The method for separating a foreground according to claim 1, wherein the step S4 further comprises the following substeps: capturing a corresponding background image from the fourth image; and using the reference background image to correspond to the The area of the foreground path replaces the area of the background image relative to the foreground path of the fourth image to generate the standard background image.