JP2019531552A - Image processing method and apparatus - Google Patents

Abstract

An image processing method and apparatus are provided. The image processing method calculates the sensitivity of the background model based on the sharpness of the detection area of the input image, and when the sharpness is within the first predetermined range, the sensitivity has a negative correlation with the sharpness. Yes; and when each pixel in the input image is processed and one pixel is processed, the one pixel and a background sample corresponding to the one pixel in the background model Calculating the distance between each sample point of the collection and if the distance between the one pixel and at least a first predetermined number of sample points is less than or equal to a predetermined first threshold, A background pixel in the input image is determined, and the first threshold is determined based on the sensitivity, and the sensitivity has a negative linear correlation with the first threshold. The image processor defines the sensitivity for the background model in the VIBE algorithm and adjusts the sensitivity based on the sharpness of the image, thereby improving the accuracy of foreground detection and causing the image to be blurred Therefore, the problem that a complete foreground image cannot be extracted can be avoided.

Description

  The present invention relates to the field of image processing technology, and more particularly to an image processing method and apparatus.

  In the video surveillance field, most intelligent image analysis systems depend on a specific scene, i.e. only in the case of a specific scene can meet performance needs. However, in actual application, scenes are complex and diverse, especially for outdoor scenes. Also, since the background model in the image is susceptible to the influence of many factors, such as noise, background interference, and illumination changes, it is difficult to construct a timely and accurate background model. In recent years, the construction of background models and the detection of foreground images has become one of the research difficulties and attention topics in the video surveillance field at home and abroad.

  Currently, several commonly used pixel-level foreground image detection methods, such as Frame differencing, Mixed of Gaussian Model, Single Gaussian Model, codebook (CodeBook) algorithm.

  When detecting by adopting the frame difference method, the background and foreground are distinguished by judging whether the difference is larger than the threshold using the time difference based on the pixel between two adjacent frames of the image sequence. Although the algorithm is simple to implement and insensitive to lighting changes, it cannot handle complex scenes.

  When performing detection using a single Gaussian model and a mixed Gaussian model, a corresponding Gaussian distribution model is constructed for each pixel point in the image, and the value obtained by the model is greater than a threshold value. Judgment can distinguish the background from the foreground, but single Gaussian models have relatively low extraction accuracy when there is noise interference in the scene. The amount is relatively large and sensitive to illumination changes.

  When detection is performed using the CodeBook algorithm, one CodeBook structure is generated for each pixel in the current image, and each CodeBook structure consists of a plurality of codewords (CodeWord). Can check each CodeWord in the corresponding background model CodeBook and distinguish the background from the foreground based on whether there is a CodeWord that can satisfy a given condition on the pixel, but such an algorithm is , May consume a large amount of memory.

  The introduction of the background art described above is intended to clearly and completely explain the technical solution of the present invention and to make it easier for those skilled in the art to understand it. These technical solutions are described in part of the background art of the present invention and should not be construed as well known to those skilled in the art.

  All of the existing detection methods described above rely on single pixel analysis, ignoring the relationship between pixels. At present, a commonly used background model construction and foreground image extraction method further includes a Visual Background Extractor (VIBE) algorithm, which initializes the background model using a single frame image, and 1 For one pixel point, the pixel value of the pixel point in the adjacent region is randomly selected as the background model sample value by using the close pixel value space distribution characteristic of the adjacent pixel, and the algorithm and other The main difference from the existing algorithm is in the background model update policy, i.e., randomly select a sample of pixels that need to be replaced, and randomly select pixels in the neighboring region to update the background model. The algorithm is fast to compute, has a small amount of computation, and has some robustness against noise. There is, in real-time scene, for example, if the rain, the change of scene, such as thick fog or haze occurs, the image becomes blurred, at this time, it is impossible to extract the complete foreground image.

  Embodiments of the present invention provide an image processing method and apparatus, define sensitivity for a background model in the VIBE algorithm, and adjust the sensitivity based on image sharpness, The accuracy of foreground detection can be improved, and the problem that a complete foreground image cannot be extracted due to unclear images can be avoided.

  The above object of the embodiment of the present invention is realized by the following technical solution.

According to a first aspect of an embodiment of the present invention, an image processing apparatus is provided, the apparatus comprising:
A first calculation unit for calculating a sensitivity of a background model based on a sharpness of a detection area of an input image, wherein the sensitivity is negatively correlated with the sharpness when the sharpness is within a first predetermined range; With the first calculation unit involved,
A first processing unit that performs processing on each pixel in the input image and detects a foreground image of the input image. When performing processing on one pixel, A distance between each sample point of the background sample collection corresponding to the one pixel in the background model is calculated, and a distance between the one pixel and at least a first predetermined number of sample points is a predetermined value. A first processing unit comprising determining if the one pixel is a background pixel in the input image if less than or equal to a first threshold, otherwise determining a foreground pixel;
Alternatively, a first processing unit that performs processing on each pixel in the input image and updates the background model. When processing is performed on one pixel, the one pixel and the background A distance between each sample point of the background sample collection corresponding to the one pixel in the model is calculated, and a distance between the one pixel and at least a first predetermined number of sample points is a predetermined first number. A first processing unit that updates the background model by replacing the one pixel with one sample point if less than one threshold,
The first threshold value is determined based on the sensitivity, and the sensitivity has a negative linear correlation with the first threshold value.

According to a second aspect of an embodiment of the present invention, an image processing method is provided, the method comprising:
Calculating the sensitivity of the background model based on the sharpness of the detection area of the input image, and when the sharpness is within a first predetermined range, the sensitivity is negatively correlated with the sharpness; and the input Processing is performed on each pixel in the image to detect the foreground image of the input image, and when processing is performed on one pixel, the one pixel and the one in the background model When the distance between each sample point in the background sample collection corresponding to the pixel is calculated and the distance between the one pixel and at least the first predetermined number of sample points is equal to or less than a predetermined first threshold value Determining the one pixel as a background pixel in the input image, otherwise determining as a foreground pixel;
Alternatively, processing is performed on each pixel in the input image to update the background model, and when processing is performed on one pixel, the one pixel and the one in the background model The distance between each sample point in the background sample collection corresponding to one pixel is calculated, and the distance between the one pixel and at least a first predetermined number of sample points is equal to or less than a predetermined first threshold value. Including replacing the one pixel with one sample point and updating the background model, wherein the first threshold is determined based on the sensitivity, and the sensitivity is the first There is a negative linear correlation with the threshold.

  The beneficial effect of the embodiment of the present invention is that the image processing method and apparatus of this embodiment define the sensitivity for the background model in the VIBE algorithm and adjust the sensitivity based on the sharpness of the image. As a result, the accuracy of foreground detection can be improved, and the problem that a complete foreground image cannot be extracted due to unclear images can be avoided.

  DETAILED DESCRIPTION Reference will now be made to the following description and drawings to disclose specific embodiments of the invention in detail and to illustrate aspects in which the principles of the invention can be employed. In addition, embodiment of this invention is not limited by these on the range. Within the scope of the appended claims, embodiments of the invention may include various changes, modifications, and alternatives.

  Also, the features described and / or illustrated for one implementation may be used in one or more other embodiments in the same or similar manner and combined with features in other embodiments or other implementations. Features in the form can also be replaced.

  It should be noted that terms such as “comprising / having” as used herein refer to the presence of a feature, element, step, or assembly, but one or more other features, elements, steps, Or it does not exclude the presence or addition of assemblies.

Elements and features described in one drawing or embodiment of the invention may be combined with elements and features shown in one or more other drawings or embodiments. Also, in the drawings, like numerals indicate corresponding parts in the several drawings and are also used to indicate corresponding parts used in several embodiments.
3 is a flowchart of an image processing method according to the first embodiment. 6 is a flowchart of an image processing method according to the second embodiment. 10 is a flowchart of an image processing method in Embodiment 3. FIG. 6 is a diagram showing an image used in Example 3. FIG. 6 is a diagram showing an image used in Example 3. FIG. 6 is a diagram showing an image used in Example 3. FIG. 6 is a diagram showing an image used in Example 3. FIG. 10 is a diagram showing a foreground image detected in the third embodiment. FIG. 10 is a diagram showing a foreground image detected in the third embodiment. FIG. 10 is a diagram showing a foreground image detected in the third embodiment. 10 is a flowchart of processing in step 305 in the second embodiment. 10 is a flowchart of an image processing method in Embodiment 3. 10 is a flowchart of processing of Step 705 in Embodiment 3. FIG. 6 is a configuration diagram of an image processing apparatus in Embodiment 4. FIG. 6 is a configuration diagram of an image processing apparatus in Embodiment 4. FIG. 6 is a hardware configuration diagram of an image processing apparatus according to a fourth embodiment. FIG. 6 is a configuration diagram of an image processing apparatus in Embodiment 4. FIG. 6 is a hardware configuration diagram of an image processing apparatus according to a fourth embodiment.

  The foregoing and other features of the present invention will become apparent upon reference to the accompanying drawings and the following description. It should be noted that while the specification and drawings disclose specific embodiments of the invention, they show only some of the embodiments in which the principles of the invention may be employed and should be understood that It is not limited to the embodiments described, i.e. the invention includes all modifications, variations and alternatives within the scope of the appended claims.

  Embodiments of the present invention will be described below with reference to the drawings.

  The first embodiment provides an image processing method, and FIG. 1 is a flowchart of the image processing method. As shown in FIG. 1, the method includes the following steps.

Step 101: Calculate the sensitivity of the background model based on the sharpness of the detection area of the input image, and when the sharpness is within the first predetermined range, the sensitivity is negatively correlated with the sharpness;
Step 102: Perform processing for each pixel in the input image, detect a foreground image of the input image, and when processing one pixel, of the one pixel and the background model A distance between each sample point of the background sample collection corresponding to the one pixel is calculated, and a distance between the one pixel and at least a first predetermined number of sample points is a predetermined first. If it is less than or equal to a threshold, the one pixel is determined as a background pixel in the input image, otherwise it is determined as a foreground pixel, wherein the first threshold is determined based on the sensitivity. The sensitivity has a negative linear correlation with the first threshold.

In the present embodiment, a background image is composed of pixels of an object that is stationary or very slow in the image, and a foreground image is composed of pixels of a moving object. Detection of the foreground image corresponds to one classification problem, that is, to determine whether each pixel in the image belongs to a background pixel or a foreground pixel. In the VIBE model of this embodiment, the background model includes a background sample collection corresponding to each pixel, and the background model can be initialized using a single frame image. Specifically, by using the close pixel value space distribution characteristics of adjacent pixels for one pixel, pixel values of pixel points in the adjacent region are randomly selected as a background sample collection, and then each pixel By comparing the value with its corresponding background sample collection, it is determined whether it belongs to the background pixel. The number of sample points in the background sample collection is a predetermined quantity (N). For example, v (x) represents the pixel value of pixel x; M (x) = {V1, V2,..., VN} is a background sample collection corresponding to pixel x; S _R (v (x)) is # [{S _R (v (x)) ∩ {V1, V2,..., VN}}] is the first predetermined quantity If it is #min or more, the pixel x belongs to the background pixel.

  In the prior art, the first threshold R usually takes a default value of 20. Therefore, when the image becomes unclear, if the first threshold value R is kept unchanged, a complete foreground image may not be extracted.

  In this example, steps 101-102 described above define the sensitivity for the background model in the VIBE algorithm, adjust the sensitivity based on the sharpness of the image, and the background model based on the sensitivity value. The first threshold value R is adjusted, thereby improving the accuracy of foreground detection and avoiding the problem that a complete foreground image cannot be extracted because the image is unclear.

  In step 101, when the sharpness is within the first predetermined range, the sensitivity has a negative correlation with the sharpness, that is, the higher the sharpness of the detection area of the input image, the lower the sensitivity, Conversely, the lower the clearness of the detection area of the input image, the higher the sensitivity. Among them, when the sharpness is larger than the fifth threshold d, it indicates that the image is very clear. At this time, the sensitivity may be set to the lowest setting value, and the sharpness is higher than the sixth threshold c. Is small, it means that the image is very clear. At this time, the sensitivity may be set to the highest setting value, and the sharpness is within the first predetermined range ([c, d]). In addition, the sensitivity has a negative correlation with the sharpness.

  For example, 0 to 100 (%) is used to represent a numerical range of sensitivity, and the magnitude of the numerical value can indicate the level of sensitivity, for example, 0 represents low sensitivity, and 100 (%) represents sensitivity. Is high.

である。そのうち、Cは、該鮮明さを表し、Sは、該感度を示し、該第一所定範囲は、[c、d]である。なお、該感度と該鮮明さとの負の相関関係は、他の負の相関関数であっても良く、本実施例は、これに限定されない。 The negative correlation between the sensitivity and the sharpness is

It is. Among them, C represents the sharpness, S represents the sensitivity, and the first predetermined range is [c, d]. The negative correlation between the sensitivity and the sharpness may be another negative correlation function, and the present embodiment is not limited to this.

  In step 102, processing is performed on each pixel in the input image to determine whether each pixel belongs to the foreground pixel or the background pixel, and an image composed of pixels determined to belong to the foreground pixel is determined as the foreground image. And confirm. The first threshold in the background model may be determined according to the sensitivity, and the sensitivity is negatively correlated with the first threshold, i.e., the higher the sensitivity, the smaller the first threshold, The lower the sensitivity, the larger the first threshold value. Therefore, the clearer the input image, the smaller the first threshold value. Conversely, the clearer the input image, the larger the first threshold value.

である。そのうち、Rは、該第一閾値を示し、Sは、該感度を表し、該第一閾値の数値範囲は、[a、b]であ。例えば、第一閾値Rの数値範囲が[5，35]の場合、第一閾値Rと感度Sとの負の線形相関関係は、

である。なお、ここで、R及びSの数値範囲を例示的に説明したが、本実施例は、これに限定されない。 In step 102, the sensitivity has a negative linear correlation with the first threshold.For example, when the numerical range of sensitivity is expressed using 0 to 100 (%), the first threshold and the negative linear correlation of the sensitivity. Relationship

It is. Among them, R represents the first threshold, S represents the sensitivity, and the numerical range of the first threshold is [a, b]. For example, when the numerical range of the first threshold R is [5, 35], the negative linear correlation between the first threshold R and the sensitivity S is

It is. In addition, although the numerical range of R and S was demonstrated here exemplarily, a present Example is not limited to this.

  In this embodiment, the method may further include the following steps.

  Step 100 (optional): Calculate the sharpness of the detection area of the input image.

  Among them, the sharpness can be expressed by an average value of gradient magnitudes of pixels in the detection region. For example, the ratio between the sum of the gradient magnitudes of the pixels in the detection area of the input image and the number of pixels in the detection area of the input image can be calculated, and the ratio can be defined as the sharpness.

For example, in step 100, the sharpness can be calculated by the following equation (1).

  Among them, w represents the width of the detection area of the input image, h represents the height of the detection area of the input image, pixel_num represents the number of pixel points in the detection area of the input image, and I Is a pixel value, and i and j indicate the horizontal and vertical coordinates of the pixel point. When the sharpness is calculated by the above method, if the sharpness is greater than 25, the image is very clear, and if the sharpness is less than 5, the image is very blurry. At this time, the sensitivity can be determined based on the sharpness by the following method, that is, when the numerical value range of sensitivity is expressed by 0 to 100 (%), when the sharpness is larger than 25, When the sensitivity is set to 0 and the sharpness is less than 5, the sensitivity is set to 100, and when the sharpness is within the range [5, 25], the sensitivity and the sharpness are negative. Linear correlation, i.e., S = -5C + 125. This is merely an example, and the present embodiment is not limited to this.

  In this embodiment, in order to avoid a redundant calculation amount, a region of interest (ROI) in the input image is selected in step 100, and the region of interest can be set as the detection region. For example, ROI is expressed in binary, and among these, the pixel value of the pixel in the ROI is 1, and the pixel values of the other pixels are 0. This is merely an example, and the present embodiment is not limited to this. For example, the ROI may select a road area or the like, and the ROI may be selected in advance, or may be selected every time an input image is processed. Does not limit this.

  In this embodiment, the scene change usually does not occur suddenly, that is, the sharpness of the adjacent image frames is close, so that the sharpness of the image is regenerated every predetermined time in order to avoid redundant calculations. The sensitivity of the background model can be updated according to the calculated and updated clarity. Therefore, in this embodiment, the method may further include the following (not shown).

  The input image is selected from at least one frame (image) sequence, and one frame image is selected as the input image every second predetermined number of frames.

  In this embodiment, the method may further include updating the background model (not shown) due to lighting changes, background image changes, and the like.

  In the present embodiment, the background model is updated by updating the sample points of the background sample collection in the background model and / or the first threshold value based on the sensitivity of the background model obtained in step 101 described above. Including updates.

  Among them, the update of the sample points in the background sample collection in the background model may adopt the background model update policy of the VIBE algorithm in the prior art, for example, no memory update policy, temporal sampling update policy, spatial adjacency For example, an area update policy. In step 102, when it is determined that the one pixel is a background pixel, the background model is updated by randomly replacing the pixel with one sample point of the corresponding background sample collection. be able to. Note that the present embodiment is not limited to this. For example, the background model is updated by performing processing on an image other than the one selected as the input image in the image sequence using the conventional technique. You may do it. Specifically, the prior art can be referred to.

  Therefore, after the input image is processed, the background model can be updated, and the foreground image can be extracted based on the updated background model for the next one frame waiting to be processed.

  In this embodiment, the input image can be obtained by a conventional method. For example, the input image may be the current frame in the surveillance video. The monitoring video can be acquired by a camera provided above an area where monitoring is required.

  According to the above embodiment, the sensitivity is defined for the background model in the VIBE algorithm, and the sensitivity is adjusted based on the sharpness of the image, thereby improving the accuracy of foreground detection and blurring the image. This prevents the problem that a complete foreground image cannot be extracted due to the above.

  The second embodiment provides an image processing method, and the difference between this method and the first embodiment is that in this embodiment, each pixel in the input image is processed and the background model is updated. There is. Note that duplicate descriptions that are the same as the contents of the first embodiment are omitted. FIG. 2 is a flowchart of the image processing method. As shown in FIG. 2, the method includes the following steps.

Step 201: Calculate the sensitivity of the background model based on the sharpness of the detection area of the input image, and when the sharpness is within the first predetermined range, the sensitivity is negatively correlated with the sharpness;
Step 202: Perform processing on each pixel in the input image, update the background model, and when processing one pixel, of the one pixel and the background model, A distance between each sample point in the background sample collection corresponding to one pixel is calculated, and a distance between the one pixel and at least a first predetermined number of sample points is equal to or less than a predetermined first threshold value. In some cases, replacing the one pixel with one sample point and updating the background model, wherein the first threshold is determined based on the sensitivity, and the sensitivity is equal to the first threshold. There is a negative linear correlation.

  In the present embodiment, the specific implementation method of step 201 can refer to step 101 in the first embodiment, and detailed description thereof is omitted here.

  In the present embodiment, the first threshold value determination method in step 202 can refer to the first embodiment, and detailed description thereof is omitted here.

  In this embodiment, the method may further include the following steps.

  Step 200 (optional): Calculate the sharpness of the detection area of the input image. The implementation method is the same as step 100 in the first embodiment, and detailed description thereof is omitted here.

  Hereinafter, differences from the first embodiment in step 202 will be described.

In step 202, processing is performed on each pixel in the input image, the background model is updated, and each pixel value is compared with the corresponding background sample collection so that it belongs to the background pixel. Determine whether. The number of sample points in the background sample collection is a predetermined quantity N. For example, v (x) indicates the pixel value of the pixel x; M (x) = {V1, V2,..., VN} is the corresponding background sample collection of the pixel x; S _R (v (x)) is , X, and # [{S _R (v (x)) 2 {V1, V2,..., VN}}] is the first predetermined quantity #min. If so, the background x is updated by replacing the pixel x with one sample point, and the background x is replaced by randomly replacing the pixel x with one sample point of the corresponding background sample collection. The model can be updated and the first threshold can be updated.

  In this embodiment, the method may further include the following steps.

  Step 203: A foreground image is extracted from the input image based on the updated background model.

  Of these, the step 203 can be realized using conventional techniques, but the present embodiment is not limited to this.

  In this embodiment, the scene change usually does not occur suddenly, that is, the sharpness of the adjacent image frames is close, so that the sharpness of the image is regenerated every predetermined time to avoid redundant computation. The sensitivity of the background model can be updated based on the calculated and updated clarity. Therefore, in this embodiment, the method may further include the following (not shown).

  The input image is selected from at least one frame (image) sequence, and one frame image is selected as the input image every second predetermined number of frames.

  Among them, for the image waiting to be processed in the next one frame, the background model is updated again using the background model updated in step 202, and the next one frame is updated based on the updated background model. A foreground image is extracted from images waiting for processing.

  According to the above embodiment, the sensitivity is defined for the background model in the VIBE algorithm, and the sensitivity is adjusted based on the sharpness of the image, thereby improving the accuracy of foreground detection and blurring the image. This prevents the problem that a complete foreground image cannot be extracted due to the above.

  The third embodiment provides an image processing method, and FIG. 3 is a flowchart of the image processing method. For surveillance video (image sequence), as shown in FIG. 3, the method includes the following steps:

Step 301: Confirm the current image;
Among them, the current image is the i-th frame in the image sequence.

Step 302: Determine the detection area;
Of these, an area composed of all the pixels of the current image may be set as a detection area, and a region of interest (ROI) set by a user may be set as the detection area.

  In step 302, it can be further determined whether the region of interest needs to be changed. If necessary, the region of interest is determined again as a detection region, and step 303 is performed. Otherwise, step 303 is performed. Do it directly.

Step 303: Calculate the sharpness of the detection area;
Among them, the specific calculation method of the sharpness can refer to step 100, and detailed description thereof is omitted here.

  4A to 4D are diagrams showing images with different sharpness. As shown in FIGS. 4A to 4D, the sharpnesses of FIGS. 4A, 4B, 4C, and 4D calculated by the equation (1) in Example 1 are 20.03, 20.857, 8.06, and 12.895, respectively, that is, clear. The higher the value, the clearer the image. Conversely, the smaller the value, the unclear image. 4C and 4D are relatively low in sharpness because they are affected by weather and lighting.

Step 304: Calculate the sensitivity of the background model based on the sharpness;
The specific calculation method can refer to step 101, and the detailed description thereof is omitted here.

Step 305: process each pixel in the input image to detect a foreground image in the input image;
FIG. 6 can be referred to for a specific implementation method of step 305, and the description thereof is omitted here.

  5A to 5C are diagrams showing foreground images detected according to different sensitivities. As shown in FIGS. 5A to 5C, FIG. 5A is an input image, and FIG. 5B is detection of a foreground image when the conventional technology R = 20 (corresponding to a sensitivity of 50%) is adopted. FIG. 5C shows the result of detecting the foreground image when the sensitivity is adjusted based on the sharpness (for example, the sensitivity is 80 (%) and R = 11). Therefore, as is clear, the accuracy of the foreground image extracted by the method in this embodiment is higher.

  Step 306: Update the background model.

  The specific updating method can refer to the first embodiment, and a detailed description thereof will be omitted here.

  Step 307: It is determined whether the image sequence has passed the image of the second predetermined quantity j frames. If “Yes”, the process returns to Step 301, i = i + j, and if “No”, Step 306 , The background model is updated using conventional technology for images other than those selected as the input image in the image sequence, and it is determined whether the image sequence is completed. Ends.

  FIG. 6 is a flowchart of the method of step 305. As shown in FIG. 6, the method includes the following steps.

Step 601: Select one pixel from the detection area of the input image;
Among them, the one pixel may be selected in order from the left to the right of the pixel and from the top to the bottom.

  For example, if the height and width of the detection area are H and W, respectively, that is, there are a total of H × W pixels, and m is set as an index of pixels, when m is 0, it is located in the upper left corner Represents a pixel, and represents a pixel located in the lower right corner when m = H × W.

  Step 602: calculating a distance between the one pixel and each sample point of the background sample collection corresponding to the one pixel in the background model, and the one pixel and at least a first predetermined quantity If the distance to the sample points is less than or equal to a predetermined first threshold, the one pixel is determined as a background pixel in the input image, otherwise it is determined as a foreground pixel, and each pixel is foreground Whether it belongs to a pixel or a background pixel is determined, and an image composed of pixels determined to belong to a foreground pixel is determined as a foreground image. The first threshold value is determined based on the sensitivity, and the sensitivity has a negative linear correlation with the first threshold value.

  For a specific implementation method of step 602, reference can be made to step 102, and a detailed description thereof will be omitted here.

  Step 603: It is determined whether or not Step 602 has been performed for all the pixel points in the detection region. When the determination result is “Yes”, the process ends, and when “No”, the process returns to Step 601; m = m + 1.

  FIG. 7 is a flowchart of the image processing method. For surveillance video (image sequence), as shown in FIG. 7, the method includes the following steps.

Step 701: Confirm the current image;
Step 702: Determine the detection area;
Step 703: calculating the sharpness of the detection area;
Step 704: Calculate the sensitivity of the background model based on the sharpness;
Among them, the specific implementation method of steps 701 to 704 is the same as that of steps 601 to 604, and detailed description thereof is omitted here.

Step 705: process each pixel in the input image and update the background model;
Step 706: Extract a foreground image from the input image based on the updated background model;
The specific extraction method can refer to Example 2, and the description thereof is omitted here.

  Step 707: It is determined whether the image sequence has passed the image of the second predetermined quantity j frames. If “Yes”, the process returns to Step 701, i = i + j, and “No” (not shown) ), The background model is updated using conventional techniques for images other than those selected as the input image in the image sequence, and it is determined whether the image sequence is completed. , The process ends.

  FIG. 8 is a flowchart of the method of step 706, and as shown in FIG. 8, the method includes the following steps.

Step 801: Select one pixel from the detection area of the input image;
Among them, the one pixel may be selected in order from the left to the right of the pixel and from the top to the bottom.

  For example, if the height and width of the detection area are H and W, respectively, that is, there are a total of H × W pixels, and m is set as an index of pixels, when m is 0, it is located in the upper left corner Represents a pixel, and represents a pixel located in the lower right corner when m = H × W.

  Step 802: calculating a distance between the one pixel and each sample point of the background sample collection corresponding to the one pixel in the background model, and at least the first predetermined quantity When the distance from the sample points is equal to or less than a predetermined first threshold, the background model is updated by replacing the one pixel with one sample point.

  For a specific implementation method of Step 802, Step 202 can be referred to, and detailed description thereof is omitted here.

  Step 803: It is determined whether or not Step 802 has been performed for all the pixel points in the detection region. If the determination result is “Yes”, the process ends. If “No”, the process returns to Step 801, and m = m + 1.

  According to the above embodiment, the sensitivity is defined for the background model in the VIBE algorithm, and the sensitivity is adjusted based on the sharpness of the image, thereby improving the accuracy of foreground detection and blurring the image. This prevents the problem that a complete foreground image cannot be extracted due to the above.

  The fourth embodiment further provides an image processing apparatus, and the principle by which the apparatus solves the problem is similar to the method in the first to third embodiments. The implementation of the method in 3 can be referred to, and redundant explanations with the same contents are omitted.

  FIG. 9 is a configuration diagram of the image processing apparatus in the fourth embodiment. As shown in FIG. 9, the image processing apparatus 900 includes the following.

First calculation unit 901: Calculates the sensitivity of the background model based on the sharpness of the detection area of the input image, and when the sharpness is within the first predetermined range, the sensitivity is negatively correlated with the sharpness There is;
First processing unit 902: processes each pixel in the input image, detects a foreground image of the input image, and when processing one pixel, A distance between each sample point of the background sample collection corresponding to the one pixel in the background model is calculated, and a distance between the one pixel and at least a first predetermined number of sample points is predetermined. The first pixel is determined to be a background pixel in the input image, otherwise it is determined to be a foreground pixel; or processing is performed for each pixel in the input image. , When the background model is updated and processing is performed on one pixel, the one pixel and each sample point of the background sample collection corresponding to the one pixel in the background model, Calculate the distance between the one pixel and less If the distance from the first predetermined number of sample points is equal to or less than a predetermined first threshold value, the background model is updated by replacing the one pixel with one sample point, and the first threshold value is Based on the sensitivity, the sensitivity is negatively correlated with the first threshold.

  According to the above embodiment, the sensitivity is defined for the background model in the VIBE algorithm, and the sensitivity is adjusted based on the sharpness of the image, thereby improving the accuracy of foreground detection and blurring the image. This prevents the problem that a complete foreground image cannot be extracted due to the above.

  FIG. 10 is a configuration diagram of the image processing apparatus in the fourth embodiment. As shown in FIG. 10, the image processing apparatus 1000 includes the following.

First calculation unit 1001: Calculates the sensitivity of the background model based on the sharpness of the detection area of the input image, and when the sharpness is within the first predetermined range, the sensitivity is negatively correlated with the sharpness There is;
First processing unit 1002: Processes each pixel in the input image, detects a foreground image of the input image, and when processing one pixel, A distance between each sample point of the background sample collection corresponding to the one pixel in the background model is calculated, and a distance between the one pixel and at least a first predetermined number of sample points is predetermined. Is determined as a background pixel in the input image, otherwise it is determined as a foreground pixel, of which the first threshold is determined based on the sensitivity. The sensitivity has a negative linear correlation with the first threshold.

  Among them, the specific implementation method of the first calculation unit 1001 and the first processing unit 1002 can refer to steps 101 to 102 in the first embodiment, and detailed description thereof is omitted here.

である。そのうち、Rは、該第一閾値を示し、Sは、該感度を表し、該第一閾値の数値範囲は、[a、b]である。 In this example, the negative linear correlation between the first threshold and the sensitivity is

It is. Among them, R represents the first threshold, S represents the sensitivity, and the numerical range of the first threshold is [a, b].

  In this embodiment, the apparatus 1000 may further include a second calculation unit 1003, which calculates the sharpness of the detection area of the input image and averages the gradient magnitude of the pixels in the detection area. Let the value be the sharpness.

In this embodiment, the second calculation unit 1003 calculates the ratio between the sum of the gradient magnitudes of the pixels in the detection area of the input image and the number of pixels in the detection area of the input image, and the ratio Or the second calculation unit 1003 calculates the sharpness by the following equation.

  Of these, w represents the width of the detection area of the input image, h represents the height of the detection area of the input image, pixel_num represents the number of pixels in the detection area of the input image, and I represents Represents a pixel value, i and j indicate the horizontal and vertical coordinates of the pixel point.

  In this embodiment, scene changes usually do not occur suddenly, i.e., the sharpness of adjacent image frames is so close that the apparatus 1000 further includes: May be included.

  First selection unit 1004: selects the input image from at least one frame (image) sequence, of which the first selection unit 1004 selects one frame image every second predetermined number of frames. Choose as.

  In this embodiment, in order to avoid a redundant calculation amount, the apparatus may further include the following.

  Second selection unit 1005: A region of interest in the input image is selected, and the region of interest is set as the detection region.

  In this embodiment, the apparatus 1000 may further include an update unit 1006, which updates the background model, of which the update unit 1006 is based on the processing result of the first processing unit 1002. The model can be updated and / or the first threshold can be updated based on the processing result of the first processing unit 1002, the specific implementation manner can refer to Example 1, Here, detailed description thereof is omitted.

  FIG. 11 is a hardware configuration diagram of the image processing apparatus according to the embodiment of the present invention. As shown in FIG. 11, the image processing apparatus 1100 may include one interface (not shown), a central processing unit (CPU) 1120, a storage unit 1110, and a transceiver 1140. The storage unit 1110 includes a central processing unit. Connected to device 1120. Among them, the storage device 1110 can store various types of data, can further store a program for image processing, and executes the program under the control of the central processing unit 1120 to execute various types of predetermined data. And a predetermined condition can be stored.

  In one implementation, the functions of the image processing device 1100 can be integrated into the central processing unit 1120. Among them, the central processing unit 1120 may be configured as follows, that is, the sensitivity of the background model is calculated based on the sharpness of the detection area of the input image, and the sharpness is within the first predetermined range. Sometimes, the sensitivity has a negative correlation with the sharpness, and each pixel in the input image is processed to detect the foreground image of the input image, of which one pixel is processed. When doing, calculate the distance between the one pixel and each sample point of the background sample collection corresponding to the one pixel in the background model, and at least a first predetermined If the distance to the number of sample points is less than or equal to a predetermined first threshold, the one pixel is determined as a background pixel in the input image; otherwise, it is determined as a foreground pixel, of which the first A threshold is determined based on the sensitivity, and the sensitivity is determined by the first threshold. There is a negative linear correlation with the value.

  Among them, the central processing unit 1120 may be configured as follows, that is, the input image is selected from at least one frame (image) sequence, and the first selection unit includes a second predetermined unit. One frame image is selected as the input image every other number of frames.

  Among them, the central processing unit 1120 may be configured as follows, that is, calculate the sharpness of the detection area of the input image, and calculate the average value of the gradient magnitudes of the pixels in the detection area. Say it.

  Among them, the central processing unit 1120 may be configured as follows, that is, the sum of the gradient magnitudes of the pixels in the detection region of the input image, and the number of pixels in the detection region of the input image. And the ratio is defined as the sharpness.

Among them, the central processing unit 1120 may be configured as follows, that is, the sharpness is calculated by the following equation.

  Among them, w represents the width of the detection area of the input image, h represents the height of the detection area of the input image, pixel_num represents the number of pixel points in the detection area of the input image, and I Represents a pixel value, and i and j represent the horizontal and vertical coordinates of the pixel point.

  Among them, the central processing unit 1120 may be configured as follows, that is, select a region of interest in the input image and set the region of interest as the detection region.

である。そのうち、Rは、該第一閾値を示し、Sは、該感度を表し、該第一閾値の数値範囲は、[a、b]である。 The negative linear correlation between the first threshold and the sensitivity is

It is. Among them, R represents the first threshold, S represents the sensitivity, and the numerical range of the first threshold is [a, b].

  Among them, the central processing unit 1120 may be configured as follows: update the background model, update the background model based on the processing result for the input image, and / or The first threshold value is updated based on the processing result for the input image.

  The specific implementation method of the central processing unit 1120 can refer to the first embodiment, and detailed description thereof is omitted here.

  In another implementation, the image processing apparatus 1100 described above is configured on a chip (not shown) connected to the central processing unit 1120, and implements the functions of the image processing apparatus 1100 under the control of the central processing unit 1120. May be.

  Note that the image processing apparatus 1100 does not necessarily include all the components illustrated in FIG. Further, the image processing apparatus 1100 may further include parts not shown in FIG. 11, and for this, reference can be made to the prior art.

  FIG. 12 is a configuration diagram of an image processing apparatus according to the fourth embodiment. As shown in FIG. 12, the image processing apparatus 1200 includes a first calculation unit 1201, which calculates the sensitivity of the background model based on the sharpness of the detection area of the input image, and the sharpness is a first predetermined range. The sensitivity is negatively correlated with the sharpness and further includes a first processing unit 1202, which processes each pixel in the input image, and , And when processing is performed for one pixel, the distance between one pixel and each sample point of the background sample collection corresponding to the one pixel in the background model is calculated. If the distance between the one pixel and at least the first predetermined number of sample points is equal to or smaller than a predetermined first threshold, the background model is updated by replacing the one pixel with one sample point. , Of which the first threshold is based on the sensitivity. Be determined Te, sensitive degree, negative linear correlation with said first threshold value.

  Among them, the specific implementation method of the first calculation unit 1201 and the first processing unit 1202 can refer to steps 201 to 202 in the second embodiment, and detailed description thereof is omitted here.

である。そのうち、Rは、該第一閾値を示し、Sは、該感度を表し、該第一閾値の数値範囲は、[a、b]である。 In this example, the negative linear correlation between the first threshold and the sensitivity is

It is. Among them, R represents the first threshold, S represents the sensitivity, and the numerical range of the first threshold is [a, b].

  In this embodiment, the apparatus 1200 may further include a second calculation unit 1203, which calculates the sharpness of the detection area of the input image and averages the gradient magnitude of the pixels in the detection area. Is defined as the sharpness.

In this embodiment, the second calculation unit 1203 calculates the ratio between the sum of the gradient magnitudes of the pixels in the detection region of the input image and the number of pixels in the detection region of the input image, and the ratio Or the second calculation unit 1203 calculates the sharpness by the following equation.

  Among them, w represents the width of the detection area of the input image, h represents the height of the detection area of the input image, pixel_num represents the number of pixels in the detection area of the input image, and I represents Represents a pixel value, i and j indicate the horizontal and vertical coordinates of the pixel point.

  In this embodiment, scene changes usually do not occur suddenly, ie, the sharpness of adjacent image frames is close, so to avoid redundant computations, the apparatus 1200 further includes: May be included.

  First selection unit 1204: Selects the input image from at least one frame (image) sequence, of which the first selection unit 1204 uses one frame image every second predetermined number of frames as the input image. select.

  In the present embodiment, in order to avoid a redundant calculation amount, the apparatus 1200 may further include the following.

  Second selection unit 1205: A region of interest in the input image is selected, and the region of interest is set as the detection region.

  In this embodiment, the apparatus 1200 further includes an extraction unit 1206, which extracts a foreground image from the input image based on the updated background model. The specific implementation method can refer to the second embodiment, and detailed description thereof is omitted here.

  FIG. 13 is a hardware configuration diagram of the image processing apparatus according to the embodiment of the present invention. As shown in FIG. 13, the image processing apparatus 1300 includes one interface (not shown), a central processing unit (CPU) 1320, a storage unit 1310, and a transceiver 1340. The storage unit 1310 is connected to the central processing unit 1320. Connected. Among them, the storage device 1310 can store various types of data, can further store a program for image processing, and executes the program under the control of the central processing unit 1320. And a predetermined condition can be stored.

  In one implementation, the functionality of the image processing device 1300 can be integrated into the central processing unit 1320. Among them, the central processing unit 1320 may be configured as follows, that is, the sensitivity of the background model is calculated based on the sharpness of the detection area of the input image, and the sharpness is within the first predetermined range. When the sensitivity has a negative correlation with the sharpness, processing is performed on each pixel in the input image, the background model is updated, and processing is performed on one of the pixels Calculating a distance between the one pixel and each sample point of the background sample collection corresponding to the one pixel in the background model, and at least a first predetermined quantity If the distance to the sample point is less than or equal to a predetermined first threshold value, the background model is updated by replacing the one pixel with one sample point, and the first threshold value is determined based on the sensitivity. The sensitivity has a negative linear correlation with the first threshold. is there.

  Among them, the central processing unit 1320 may be configured as follows, that is, the input image is selected from at least one frame (image) sequence, and the first selection unit includes a second predetermined unit. One frame image is selected as the input image every other number of frames.

  Among them, the central processing unit 1320 may be configured as follows, that is, calculate the sharpness of the detection area of the input image, and calculate the average value of the gradient magnitude of the pixels in the detection area as the sharpness. To do.

  Among them, the central processing unit 1320 may be configured as follows, that is, the sum of the gradient magnitudes of the pixels in the detection region of the input image, and the number of pixels in the detection region of the input image. And the ratio is defined as the sharpness.

Among them, the central processing unit 1320 may be configured as follows, that is, calculate the sharpness by the following equation.

  Among them, w represents the width of the detection area of the input image, h represents the height of the detection area of the input image, pixel_num represents the number of pixel points in the detection area of the input image, and I Represents a pixel value, and i and j represent the horizontal and vertical coordinates of the pixel point.

  Among them, the central processing unit 1320 may be configured as follows, that is, a region of interest in the input image is selected and the region of interest is generally used as the detection region.

である。そのうち、Rは、該第一閾値を示し、Sは、該感度を表し、該第一閾値の数値範囲は、[a、b]である。 The negative linear correlation between the first threshold and the sensitivity is

It is. Among them, R represents the first threshold, S represents the sensitivity, and the numerical range of the first threshold is [a, b].

  Among them, the central processing unit 1320 may be configured as follows, that is, extract the foreground image from the input image based on the updated background model.

  The specific implementation method of the central processing unit 1320 can refer to the second embodiment, and detailed description thereof is omitted here.

  In another implementation, the above-described image processing apparatus 1300 is configured as a chip (not shown) connected to the central processing unit 1320, and realizes the functions of the image processing apparatus 1300 under the control of the central processing unit 1320. You can also

  Note that the image processing apparatus 1300 does not necessarily include all the components illustrated in FIG. Further, the image processing apparatus 1300 may further include parts not shown in FIG. 13, and the prior art can be referred to for this.

  According to the above embodiment, the sensitivity is defined for the background model in the VIBE algorithm, and the sensitivity is adjusted based on the sharpness of the image, thereby improving the accuracy of foreground detection and blurring the image. This prevents the problem that a complete foreground image cannot be extracted due to the above.

  Embodiments of the present invention further provide a computer readable program, of which when executing the program in an image processing apparatus, the program is stored in the image processing apparatus in the embodiment 1, 2 or The image processing method in 3 is executed.

  The embodiment of the present invention further provides a storage medium storing a computer readable program, wherein the computer readable program is stored in the image processing apparatus in the image processing apparatus according to the first, second, or third embodiment. Let it run.

  The above user apparatus and method may be realized by software or hardware, or may be realized by a combination of hardware and software. The present invention further relates to the following computer-readable program, that is, when the program is executed by a logic component, causes the logic component to realize the above-described user device or structural component, or The logic component is made to implement the various methods or steps described above. The logic component may be, for example, an FPGA (Field Programmable Gate Array), a microprocessor, or a processor used in a computer. The present invention further relates to a storage medium storing the above-mentioned program, for example, a hard disk, a magnetic disk, an optical hard disk, a DVD, a flash memory, and the like.

  Further, one or more combinations of functional blocks described in the drawings and / or one or more combinations of functional blocks include a general-purpose processor, a digital signal processor ( Realized as a DSP), dedicated integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic component, discrete gate or transistor logic component, discrete hardware assembly or any other suitable combination Also good. In addition, one or more combinations of functional blocks and / or one or more combinations of functional blocks described in the drawings may further include combinations of computing devices such as combinations of DSPs and microprocessors, It may be configured as a processor, one or more microprocessors connected by communication with a DSP, or any other such configuration.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to such embodiment, As long as it does not remove | deviate from the meaning of this invention, all the changes with respect to this invention belong to the technical scope of this invention.

Claims (18)

An image processing apparatus,
A first calculation unit for calculating a sensitivity of a background model based on a sharpness of a detection area of an input image, wherein the sensitivity is negatively correlated with the sharpness when the sharpness is within a first predetermined range; A first calculation unit having a relationship; and a first processing unit that performs processing on each pixel in the input image and detects a foreground image of the input image, and performs processing on one pixel And calculating a distance between the one pixel and each sample point of the background sample collection corresponding to the one pixel in the background model, and at least a first predetermined quantity of the one pixel. A first processing unit that determines the one pixel as a background pixel in the input image if the distance to the sample point is equal to or less than a predetermined first threshold; otherwise, determines the foreground pixel; or the input image Each inside A first processing unit that performs processing on an element and updates the background model. When performing processing on one pixel, the one pixel and the one pixel in the background model Calculating a distance between each sample point of the corresponding background sample collection, and when a distance between the one pixel and at least a first predetermined number of sample points is equal to or less than a predetermined first threshold, A first processing unit that updates the background model by replacing one pixel with one sample point;
The apparatus wherein the first threshold is determined based on the sensitivity, the sensitivity having a negative linear correlation with the first threshold. The apparatus of claim 1, wherein
A first selection unit for selecting the input image from at least one frame sequence;
The first selection unit selects one frame as the input image every second predetermined number of frames. The apparatus of claim 1, wherein
The apparatus further includes a second calculation unit that calculates a sharpness of a detection area of the input image and sets an average value of gradient magnitudes of pixels in the detection area as the sharpness. The apparatus according to claim 3, wherein
The second calculation unit calculates a ratio between the sum of the gradient magnitudes of the pixels in the detection area of the input image and the number of pixels in the detection area of the input image, and sets the ratio as the sharpness. The apparatus further comprising a second calculation unit. The apparatus according to claim 3, wherein
The second arithmetic unit is

Calculating the sharpness based on
Here, w represents the width of the detection area of the input image, h represents the height of the detection area of the input image, pixel_num represents the number of pixels in the detection area of the input image, and I represents Indicates the pixel value, i and j indicate the abscissa of the pixel point. The apparatus of claim 1, wherein
The apparatus further includes a second selection unit that selects a region of interest in the input image and uses the region of interest as the detection region. The apparatus of claim 1, wherein
The negative linear correlation between the first threshold and the sensitivity is

And
Here, R represents the first threshold, S represents the sensitivity, and the numerical range of the first threshold is [a, b]. The apparatus of claim 1, wherein
When the first processing unit detects a foreground image of the input image, the apparatus further comprises an update unit for updating the background model;
The apparatus further includes an extraction unit that extracts a foreground image from the input image based on the updated background model when the first processing unit updates the background model. The apparatus according to claim 8, wherein
The update unit updates the background model based on the processing result of the first processing unit and / or updates the first threshold value based on the processing result of the first processing unit. An image processing method comprising:
Calculating the sensitivity of the background model based on the sharpness of the detection area of the input image, and when the sharpness is within a first predetermined range, the sensitivity has a negative correlation with the sharpness; and When each pixel in the input image is processed, a foreground image of the input image is detected, and one pixel is processed, the one pixel and the one pixel in the background model When the distance between each sample point of the background sample collection corresponding to is calculated and the distance between the one pixel and at least the first predetermined number of sample points is equal to or less than a predetermined first threshold value, Determining the one pixel as a background pixel in the input image, otherwise determining as a foreground pixel;
Alternatively, when processing is performed on each pixel in the input image, the background model is updated, and processing is performed on one pixel, the one pixel and the one pixel in the background model are processed. When the distance between each sample point of the background sample collection corresponding to is calculated and the distance between the one pixel and at least the first predetermined number of sample points is equal to or less than a predetermined first threshold value, Updating the background model by replacing the one pixel with one sample point;
The method, wherein the first threshold is determined based on the sensitivity, and the sensitivity has a negative linear correlation with the first threshold. A method according to claim 10, comprising
Selecting the input image from at least one frame sequence, further comprising selecting one frame as the input image every second predetermined number of frames. A method according to claim 10, comprising
The method further includes calculating a sharpness of a detection area of the input image and setting an average value of gradient magnitudes of pixels in the detection area as the sharpness. A method according to claim 12, comprising
A method of calculating a ratio between a sum of gradient magnitudes of pixels in the detection area of the input image and the number of pixels in the detection area of the input image, and setting the ratio as the sharpness. A method according to claim 12, comprising

Calculating the sharpness based on
Here, w represents the width of the detection area of the input image, h represents the height of the detection area of the input image, pixel_num represents the number of pixels in the detection area of the input image, and I represents The pixel value, i and j indicate the abscissa of the pixel point. A method according to claim 10, comprising
The method further comprises selecting a region of interest in the input image and making the region of interest the detection region. A method according to claim 10, comprising
The negative linear correlation between the first threshold and the sensitivity is

And
Here, R represents the first threshold, S represents the sensitivity, and the numerical range of the first threshold is [a, b]. A method according to claim 10, comprising
When processing each pixel in the input image and detecting a foreground image of the input image, the method further comprises updating the background model;
When processing each pixel in the input image and updating the background model, the method further includes extracting a foreground image from the input image based on the updated background model. . The method of claim 17, comprising
Updating the background model based on a processing result for the input image and / or updating the first threshold based on the processing result for the input image;

Images