WO2015129250A1

WO2015129250A1 - Image processing device, and image processing method

Info

Publication number: WO2015129250A1
Application number: PCT/JP2015/000897
Authority: WO
Inventors: 松山　好幸
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2014-02-25
Filing date: 2015-02-24
Publication date: 2015-09-03
Also published as: JP2015180048A

Abstract

　This image processing device is provided with: an image input unit for inputting an image; a median processing unit for applying median filtering to the input image inputted by the image input unit; a movement area detection unit for detecting an area of the input image in which there is movement; an input operation unit for accepting the input of a snowfall correction parameter for the input image; a designated size setting unit for setting the designated size of the movement area in correspondence to the snow correction parameter; a movement area size detection unit for detecting a movement area greater than or equal to the size set by the designated size setting unit; and an image generation unit for generating a snowfall-corrected image by using, as the image of the movement area, the image to which median filtering was applied when the size of the movement area is less than the designated size, and by using, as the image of the movement area, the image inputted by the image input unit when the size of the movement area is greater than or equal to the designated size.

Description

Image processing apparatus and image processing method

The present invention relates to an image processing apparatus and an image processing method for removing snow noise that appears in an image captured during snowfall (hereinafter referred to as “snow correction”).

Conventionally, as a technique for removing snow falling on an image, an image processing device that removes snow by performing median filter processing on pixels at the same coordinates of each frame image for a plurality of time-series frame images Is known (see, for example, Patent Document 1).

This image processing apparatus stores images of a plurality of frames (the number of acquired frames is 2k + 1, k is a positive integer) captured by a fixed surveillance camera, and the pixel value is a luminance value for each pixel at the same coordinate. Are arranged in descending order, and the (k + 1) th luminance value is set as the output luminance at the current time.

Therefore, according to this image processing apparatus, even if a snowflake is captured at a pixel of interest in a frame image at a certain point in time, in the past frame image or the future frame image, there is almost no chance of snow. By adopting the brightness value of the intermediate frame image, it is possible to remove snowflakes in the image.

Japanese Patent No. 3439669

However, if median filter processing is simply performed on time-series frame images, it is possible to remove snowflakes in the image, but there are relatively large moving objects in the image such as the monitoring object. In this case, since the moving area in the image becomes large, there is a problem that a remarkable afterimage is generated in the image subjected to the median filter processing.

Also, when the surveillance camera moves such as pan / tilt, the image moves relatively, and the moving area in the image becomes large, so that a remarkable afterimage is generated in the median filtered image. There is.

In order to solve the above-described conventional problems, the present invention removes snowfall noise and corrects an afterimage generated on a relatively large moving object such as a monitoring target when performing snowfall correction by performing a median filter process on the image. An object of the present invention is to provide an image processing apparatus and an image processing method for reducing the image processing apparatus.

The present invention includes an image input unit that inputs an image, a median processing unit that performs median filter processing on the input image input by the image input unit, and a moving region that detects a moving region in which the input image moves. A detection unit, an input operation unit that receives an input of a snow correction parameter or a rain correction parameter for the input image, a specified size setting unit that sets a specified size of the moving region corresponding to the snow correction parameter or the rain correction parameter, A moving area size detecting unit for detecting a moving area larger than the specified size set by the specified size setting unit; and if the size of the moving area is less than the specified size, the median filter is used as an image of the moving area. When the processed image is used and the size is not less than the specified size, the image is input as the moving area image. Using the input image by parts, comprising an image generator for generating a snowfall corrected image or rainfall corrected image, and an image processing apparatus.

The present invention is an image processing method of an image processing apparatus that performs snowfall correction or rain correction, the step of inputting an image, the step of performing median filter processing on the input image that has been input, and the movement of the input image Detecting a region; receiving a snow correction parameter or a rain correction parameter for the input image; setting a designated size of the moving region corresponding to the snow correction parameter or the rain correction parameter; A moving area size detection unit that detects a moving area that is larger than the size, and if the size of the moving area is less than the specified size, use the image that has been subjected to the median filter processing as the moving area image; If the size is greater than or equal to the size, using the input image as the moving region image, a snowfall correction image is displayed. Or it has a step of generating a rain corrected image, and an image processing method.

According to the present invention, it is possible to remove snowfall noise and to reduce afterimages that occur on relatively large moving objects such as monitoring objects.

FIG. 1 is a block diagram showing the configuration of the image processing apparatus according to this embodiment. FIG. 2A is a diagram for explaining an operation of detecting the size of the moving area in the moving area size detection unit. FIG. 2B is a diagram for explaining a motion region size detection operation in the motion region size detection unit. FIG. 2C is a diagram for explaining a motion region size detection operation in the motion region size detection unit. FIG. 3 is a flowchart for explaining the snowfall correction image generation processing procedure. FIG. 4 is a diagram illustrating an example of a UI screen of the screen display unit when the removal intensity is set to the minimum value. FIG. 5 is a diagram illustrating an example of a UI screen of the screen display unit when the removal intensity is set to an intermediate value. FIG. 6 is a diagram illustrating an example of a UI screen of the screen display unit when the removal intensity is set to the maximum value. FIG. 7 is a diagram illustrating a display example of a moving image immediately before reproduction, in which the moving image viewer display area and the image processing menu display area are displayed on the same screen. FIG. 8 is a diagram illustrating a display example of a moving image immediately after reproduction, in which the moving image viewer display area and the image processing menu display area are displayed on the same screen. FIG. 9 is a diagram illustrating a display example of a moving image after the snowfall and rain removal are performed, in which the moving image viewer display area and the image processing menu display area are displayed on the same screen. FIG. 10 is a diagram showing an example in which the display area of the moving image viewer and the display area of the image processing menu are displayed on the same screen and the details of the snowfall / falling removal menu are shown. FIG. 11 is a diagram showing an example in which the display area of the moving image viewer and the display area of the image processing menu are displayed on the same screen and the details of the snowfall / falling removal menu are shown. FIG. 12 is a diagram illustrating an example in which the display area of the moving image viewer and the display area of the image processing menu are displayed on the same screen, and the details of the snowfall / rain removal menu are shown. FIG. 13 shows a moving image viewer display area and each image processing menu display area on the same screen, and a moving image table after each image processing of spatial gradation correction, multiple image composition NR, and snow and rain removal. It is a figure which shows an example.

Embodiments of an image processing apparatus and an image processing method according to the present invention (hereinafter referred to as “the present embodiment”) will be described with reference to the drawings. The image processing apparatus of this embodiment is applied to an image processing apparatus that processes video output from a camera, video, recorder, or the like.

FIG. 1 is a block diagram showing the configuration of the image processing apparatus 1 in the present embodiment. The image processing apparatus 1 includes a processing control unit 11, a video input unit 12, a median processing unit 13, a moving region detecting unit 14, a moving region size detecting unit 15, a snowfall correction image generating unit 16, and a screen display unit. 17, an input image storage unit 21, a median image storage unit 22, and a moving area processing image storage unit 23.

The video input unit 12 as an example of the image input unit inputs video output from a camera, video, recorder, or the like, and temporarily stores the image data in the input image storage unit 21. The median processing unit 13 executes median filter processing in the time axis direction using a plurality of images (input images) stored in the input image storage unit 21 to generate a median image. In the median processing, the median processing unit 13 calculates, for example, a luminance value for each pixel having the same coordinates with respect to an image of a plurality of frames accumulated in time series (here, the number of frames is 2k + 1, k is a positive integer). Then, the pixel values are arranged in descending order of the luminance value, and the k + 1th luminance value is adopted as the output luminance.

The moving area detecting unit 14 detects moving moving areas including snow and moving objects in units of pixels using the image stored in the input image accumulating unit 21. For detecting the moving area, a method of calculating a difference between pixel values of the previous frame and the current frame (interframe difference method), a difference between pixel values of the background image and the current frame (background difference method), or the like is used.

The moving region size detection unit 15 as an example of the designated size setting unit performs processing for detecting the moving region detected by the moving region detection unit 14 using a contraction / expansion process to be described later. Then, an image (moving region processed image) including the moving region obtained by this detection processing is generated.

The input image storage unit 21 is a memory that stores images (input images) input from the video input unit 12 for several frames.

The median image storage unit 22 is a memory for storing the image (median image) generated by the median processing unit 13. This median image includes the luminance value for each pixel calculated by the median processing unit 13.

The moving area processed image accumulating unit 23 is a memory for accumulating images (moving area processed images) processed by the moving area size detecting unit 15.

The snowfall correction image generation unit 16 uses the input image, the median image, and the moving region processing image stored in the input image storage unit 21, the median image storage unit 22, and the moving region processing image storage unit 23, respectively, to detect snow and the others. The moving object is separated and determined, and an image from which snow has been removed (snowfall correction image) is generated and output.

The screen display unit 17 is a user interface (UI: UI) that specifies a parameter value (removal intensity) corresponding to the size of the moving region used by the moving region size detecting unit 15 and the display unit that displays the input image and the snowfall correction image. Includes an input operation unit as User Interface). For example, the screen display unit 17 is configured using a touch panel.

The processing control unit 11 controls the operation of each unit of the image processing apparatus 1 described above, acquires a parameter value (removal intensity) from the UI of the screen display unit 17, and transfers the set value to the moving region size detection unit 15. Do.

FIG. 2A to FIG. 2C are diagrams for explaining the motion region size detection operation in the motion region size detection unit 15. In the figure, white pixels 41 are pixels (moving region pixels) included in the moving region, and black pixels 42 are pixels not included in the moving region. Shrinkage / expansion processing is used to detect the size of the moving region. In the contraction / expansion process, the white pixel 41 is contracted in the horizontal direction and the vertical direction by the specified contraction size m (pixel) with respect to the moving area image A, respectively. Then, the logical sum of the two images B and C contracted in the horizontal direction and the vertical direction is obtained. Thereafter, the white pixels of the image D obtained by the logical sum are expanded in the horizontal direction and the vertical direction by the same size as the contraction size m (pixels).

When this processing is performed, a moving area having a pixel size (S pixel) of 2m + 1 in the vertical direction or the horizontal direction (one direction) with respect to the specified contraction size m is detected as a moving area that is equal to or larger than the specified size.

Specifically, the case where the shrinkage size m is 2 is shown. FIG. 2A shows a case where the moving area pixels are 3 × 3 pixels. An image B and an image C that are contracted by two pixels in the horizontal direction and the vertical direction are generated for the image A including a 3 × 3 pixel moving area. In the image B and the image C contracted by two pixels, the white pixel 41 disappears.

Image D, which is the logical sum of image B and image C, does not include white pixels. Therefore, even if the process of expanding the image D by two pixels is performed, the image E after the expansion process does not include white pixels representing the moving area. That is, the 3 × 3 moving area pixels do not include 5 pixels that are 2m (m = 2) +1 in one direction and are not detected.

FIG. 2B shows a case where the moving area pixels are 5 × 5 pixels. Similarly, an image B and an image C that are contracted by two pixels in the horizontal direction and the vertical direction are generated for the image A including the moving region of 5 × 5 pixels. In the image B contracted by two pixels in the horizontal direction, five pixels in one vertical column located at the center are white pixels 41. Further, in the image C contracted by two pixels in the vertical direction, five pixels in one horizontal row located at the center are white pixels 41.

In the image D, which is the logical sum of the images B and C, the white pixel 41 is the portion of the vertical and horizontal rows of ten characters located in the center. In the image E expanded by two pixels in the horizontal direction and the vertical direction, the range of white pixels is expanded and detected. That is, the 5 × 5 moving region pixels subjected to the contraction / expansion processing include five pixels that are 2m (m = 2) +1 in one direction, and are thus detected.

FIG. 2C shows a case where the moving region pixels are 5 × 3 pixels. Similarly, an image B and an image C that are contracted by two pixels in the horizontal direction and the vertical direction are generated for the image A including the moving region of 5 × 3 pixels. In the image B contracted by two pixels in the horizontal direction, three pixels in one vertical column located at the center are white pixels. In the image C contracted by two pixels in the vertical direction, white pixels disappear.

In the image D that is the logical sum of the image B and the image C, the vertical column located in the center is a white pixel. In the image E expanded by 2 pixels in the horizontal direction and the vertical direction, the range of white pixels extends to 5 × 7 pixels and is detected. That is, the 5 × 3 moving region pixels subjected to the contraction / expansion processing include five pixels that are 2m (m = 2) +1 in one direction, and are thus detected. As described above, a moving area pixel having a long shape in one direction is easily detected, and moving area pixels having a size less than a specified size (2m + 1) disappear in both the horizontal direction and the vertical direction. Since the snow particles are generally of a constant size regardless of the direction, the image processing apparatus 1 performs the process of combining the shrinking process and the expanding process in the horizontal direction and the vertical direction, so that snow and other moving objects are obtained. It is possible to improve the separation performance.

Next, the operation of the image processing apparatus 1 of the present embodiment will be described with reference to FIG. FIG. 3 is a flowchart for explaining the snowfall correction image generation processing procedure. The snowfall correction image generation process shown in FIG. 3 is executed on a pixel-by-pixel basis for the input image after the moving area size detection process is executed and the moving area image is stored in the moving area processed image storage unit 23.

The snowfall correction image generation unit 16 determines whether or not the pixel to be determined is a moving region pixel (S1). If the pixel is a moving region pixel, the snowfall correction image generation unit 16 determines whether or not the luminance value of the pixel to be determined has increased (S2). The determination of the increase in the luminance value is performed by comparing the luminance values of the previous frame and the current frame.

When the luminance value is increased, the snowfall correction image generation unit 16 determines that the size of the moving region pixel including the pixel to be determined corresponds to the parameter value indicated by the UI of the screen display unit 17. It is determined whether or not the number of pixels is greater than or equal to S detected by the size detector 15 (S3). Here, the S pixel is a specified value, and is represented by 2m + 1 as the contraction size m as described above. The determination in S3 refers to the pixel value of the moving area processed image of the image E described with reference to FIG.

When the size of the moving region pixel is S pixels or more (the pixel value of the image E in FIG. 2 is white), the snowfall correction image generation unit 16 determines that the moving object is a moving object, and determines the pixel value of the input image as the output image. Used as a pixel value (S4). If the luminance value of the pixel to be determined has not increased in step S2, the snowfall correction image generation unit 16 determines that the luminance value is not high, such as white snow, and is input in step S4. The pixel value of the image is used as the pixel value of the output image.

On the other hand, when the size of the moving region pixel is less than S pixels (the pixel value of the image E in FIG. 2 is black), the snowfall correction image generation unit 16 determines that it is snow and uses the pixel value of the median image as the output image. Are used (S5).

On the other hand, if the pixel to be determined in step S1 is not a moving region pixel, the snowfall correction image generation unit 16 determines that the pixel is a background and uses the pixel value of the median image as the pixel value of the output image (S6). ). Here, an input image may be used instead of the median image, but using the median image has an effect of removing noise.

After step S4, S5 or S6, the snowfall correction image generation unit 16 determines whether or not the determination has been completed for all the pixels in the input image (S7). If all the pixels have not been determined, the snowfall correction image generation unit 16 returns to the process of step S1.

On the other hand, when the determination of all pixels is completed, the snowfall correction image generation unit 16 generates a snowfall correction image based on the determination result of all pixels and displays it on the screen display unit 17 (S8). Thereafter, the snowfall correction image generation unit 16 ends this process.

FIG. 4 is a diagram showing a UI screen of the screen display unit 17 when the removal intensity that is the parameter value is set to the minimum value. The UI screen of the screen display unit 17 includes a window 31 in which an input video is displayed, a window 32 in which a corrected video is displayed, a radio button 33 for selecting whether or not to perform an afterimage reduction process, and a slide bar for adjusting the removal intensity. 35 and a numerical value display section 37 are displayed.

When the radio button 33 is selected as “with afterimage reduction processing”, the afterimage reduction processing described in the present embodiment is performed, whereas when “without afterimage reduction processing” is selected, the afterimage reduction processing is performed. Not done.

The slide bar 35 is input in the horizontal direction by the user's finger or the like, and indicates the removal intensity in units of pixels. In FIG. 4, the removal intensity is represented by 5 bits (0 to 32) in pixel units. Further, the removal intensity is a parameter value for performing snowfall correction, and corresponds to the S pixel that is the specified value represented by 2m + 1 described above. Therefore, when the snowflake has a moving area size less than S pixels corresponding to the removal intensity indicated by the slide bar 35, the snowflake is removed.

For example, when the removal intensity is a small set value, large snowflakes are not removed, but the afterimage (multiplexing or disappearance) of the moving object is reduced. On the other hand, when the removal intensity is a large set value, small to large snowflakes are removed, but the afterimage (multiplexing or disappearance) of the moving object increases. In this way, the user can adjust with the slide bar 35 whether to prioritize the extinction of snow or the generation of no afterimage. Further, when erasing the snow, the user can adjust the slide bar 35 even when the large snowflake in the foreground is impossible but many small snowflakes in the back are desired to be erased.

In FIG. 4, the slide bar 35 input by the user is at the left end, and the removal strength is 0. In this case, the corrected video (snow-corrected image) projected on the window 32 is substantially the same as the input video (input image) projected on the window 31, and the snow 51 is not removed at all.

FIG. 5 is a diagram showing a UI screen of the screen display unit 17 when the removal intensity is set to an intermediate value. In FIG. 5, the slide bar 35 input by the user is in the middle, and the removal strength is a value of 16. In this case, compared with the input video, the small snow 51a on the back side is removed from the corrected video (snowfall correction image) displayed on the window 32, but the large snow 51b on the near side is not removed.

FIG. 6 is a diagram showing a UI screen of the screen display unit 17 when the removal intensity is set to the maximum value. In FIG. 6, the slide bar 35 that is input by the user is at the right end, and the removal strength is a value 32. In this case, as compared with the input video, snow has almost completely disappeared in the corrected video (snowfall correction image) displayed in the window 32.

As described above, the image processing apparatus 1 according to the present embodiment uses an input image as an image of a moving area that is equal to or larger than a specified size in the input image when the image is subjected to median filter processing and snowfall correction is performed. Since the median image is used for the image of the moving area that is smaller than the designated size, afterimages that occur in a moving area that is larger than the designated size, such as a monitoring object, can be reduced. In addition, the image processing apparatus 1 can remove snow particle noise less than a specified size from the image.

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

In the above-described embodiment, the image processing apparatus 1 performs image processing on the snow particles in the input image, but is not limited to snow particles, and may be rain particles, for example. In other words, the image processing apparatus 1 can generate an image after snowfall correction processing (snowfall correction image), but can also generate an image after rainfall correction processing (rainfall correction image). In this case, in the above-described embodiment, “snowfall” is read as “rainfall”, “snowdrop” is read as “raindrop”, and “snowdrop size” is read as “raindrop size”. .

Further, for example, the processing control unit 11 of the image processing apparatus 1 of the present embodiment reads out moving image data stored in the input image storage unit 21 and removes snow or rain in the input image of the present embodiment. The menu screen of the correction (hereinafter referred to as “snow and rain removal”) processing and the image data of the moving image read from the input image storage unit 21 may be displayed on the same screen of the screen display unit 17 (FIG. 7 to FIG. 12). Hereinafter, screen examples displayed on the same screens WD1 to WD6 of the screen display unit 17 will be described with reference to FIGS.

FIG. 7 is a diagram showing a display example of a moving image immediately before reproduction, in which the moving image viewer display area VIW and the image processing menu display area MNU are displayed on the same screen WD1. FIG. 8 is a diagram showing a display example of a moving image immediately after reproduction, with the moving image viewer display area VIW and the image processing menu display area MNU displayed on the same screen WD2. FIG. 9 is a diagram showing a display example of the moving image after the snowfall and rain removal are displayed on the same screen WD3 where the moving image viewer display area VIW and the image processing menu display area MNU are displayed.

FIG. 7 shows the display area VIW of the viewer of the moving image data in the state immediately before reproduction at the time of reading from the input image storage unit 21 and the image processing that can be executed by the image processing apparatus 1 (specifically, A display area MNU of a menu (image processing menu) of spatial gradation correction, multi-image composition NR (Noise Reduction), snowfall / rain removal) is shown as the same screen WD1. In FIG. 7 and FIG. 8, the detailed contents of each of these image processing menus are not displayed, only the names of the respective image processing menus are displayed, and the image of the moving image to be subjected to image processing in the image processing apparatus 1 is displayed. The relationship between the data and a list of image processing menus that can be executed by the image processing apparatus 1 is visible to the user at a glance. Therefore, the user can confirm the image data of the moving image to be subjected to image processing and the image processing menu in a comparative manner.

In the lower part of the screens WD1 to WD7 shown in FIG. 7 to FIG. 13, the operation button set BST related to the playback, pause, stop, fast forward, rewind, recording, rewind operation, etc. A playback bar BAR, a video time area STE indicating the start time and end time of the video, and an actual playback time (that is, an elapsed time from the start time) indicated by the indicator IND displayed in the playback bar BAR. The playback time area CRT shown in FIG. When the playback button RP is pressed by the cursor CSR by a user operation, the processing control unit 11 of the image processing apparatus 1 plays back the moving image data read from the input image storage unit 21 (see FIGS. 8 to 13). . In this case, the processing control unit 11 of the image processing apparatus 1 changes the reproduction button RP to the pause button HL and displays it on the screens WD2 to WD7.

Here, for example, when the snowfall / rain removal menu bar MNU3 in the display area MNU of the image processing menu is pressed once by the cursor CSR by the user's operation, the snowfall correction image generation unit 16 of the image processing apparatus 1 is described above. In accordance with the method of the present embodiment (see, for example, FIG. 3), correction processing for removing snow and rain is performed on the image data of the moving image displayed in the display area VIW. On the other hand, when the menu bar MNU3 of the snowfall / rain removal correction process is further pressed again by the cursor CSR, the snowfall correction image generation unit 16 of the image processing apparatus 1 performs the snowfall / fall for the moving image data displayed in the display area VIW. Cancel correction processing. As a result, the image processing apparatus 1 is displayed in the display area MNU by a simple user operation (that is, whether or not the menu bar MNU3 is pressed) in a state in which the moving image data displayed in the display area VIW is reproduced. The image processing corresponding to any one of the image processing menus can be executed accurately or can be interrupted, and the processing result before and after the image processing can be easily confirmed by the user. Note that the processing control unit 11 of the image processing apparatus 1 may display a cursor CSR1 having a different shape when the cursor CSR is on or near any menu bar of the image processing menu. A cursor CSR may be displayed.

Further, when the marker DT1 displayed on the right end portion of the menu bar MNU3 of the snow and rain removal correction process shown in FIG. 9 is pressed by the cursor CSR1 by the user's operation, the process control unit 11 of the image processing apparatus 1 A detailed operation screen display area MNU3D for setting input parameters (for example, correction strength) relating to the correction process for snow and rain removal is expanded and displayed on the screens WD4 to WD6 (see FIGS. 10 to 12). FIGS. 10 to 12 are diagrams showing examples in which the moving image viewer display area VIW and the image processing menu display area MNU3D are displayed on the same screen WD4 to WD6, respectively, and the details of the snowfall and rain removal menu are shown. It is.

As shown in FIG. 10 to FIG. 12, the processing control unit 11 of the image processing apparatus 1 sets a detailed operation screen SRC (FIG. 4 to FIG. 4) for setting input parameters (for example, correction strength) related to correction processing for snowfall and rain removal. 6) is displayed on the display area MNU3D. In the display area MNU3D shown in FIG. 10 to FIG. 12, the correction mode and the correction intensity, which are input parameters relating to the snow and rain removal correction processing, are respectively set to a predetermined mode and a predetermined value, and an automatically set check box ATC (see FIG. 12) ) Is added and displayed. It should be noted that the correction mode and the correction intensity at the time of automatic setting may be initial values, or may be dynamically set based on, for example, a detection result in the moving area size detection unit 15, and so on. The processing selection options shown in FIGS. 4 to 6 (that is, a selection box for selecting either no afterimage reduction processing or afterimage reduction processing) correspond to the correction modes shown in FIGS. 10 to 12. is doing.

Similarly, in FIGS. 10 and 11, the image processing apparatus 1 is operated by the user moving the cursor CSR individually to the left and right with respect to the seek bar (see the slider 35 shown in FIGS. 4 to 6) provided for the correction strength. Then, with respect to the image data of the moving image being reproduced in the display area VIW, correction processing for removing snow and rain is performed using the input parameter (that is, the correction strength) after the moving operation.

Further, when the user inputs a numerical value (for example, a value from 0 to 32) in an input field (see the numerical value display unit 37 shown in FIGS. 4 to 6) provided for the correction strength, the image processing apparatus 1 displays the display area VIW. In the image data of the moving image being reproduced in, correction processing for removing snow and rain is performed using the correction strength after input.

Although the correction strength is “16” in FIG. 10, the correction strength is changed to “30” in FIG. 11, so that the moving image data shown in FIG. 11 is compared with the moving image data shown in FIG. For example, the removal amount of snow particles at the time of snowfall or raindrops at the time of rain increases, and the user can sufficiently grasp the contents of the moving image.

In addition, when a check box ATC for automatically setting a correction mode and a correction intensity, which are input parameters relating to snow and rain removal shown in FIG. The image processing apparatus 1 performs a snowfall / rain removal correction process on the image data of the moving image being reproduced in the display area VIW by using the correction mode and the default value of the correction intensity. As a result, the image processing apparatus 1 can provide a typical initial value for the correction process for snowfall and rain removal, for example, to a user who does not know what value should be input as the correction mode and correction strength for snow and rain removal. By using the value as the default value, it is possible to easily perform the snow and rain removal correction process on the moving image data.

In addition, the image processing apparatus 1 of the present embodiment displays the moving image data read from the input image storage unit 21 and a plurality of image processing menu screens including correction processing for removing snow and rain on the same screen of the screen display unit 17. May be displayed (see FIG. 13). Hereinafter, a screen example displayed on the same screen WD7 of the screen display unit 17 will be described with reference to FIG.

FIG. 13 shows the display area VIW of the moving image viewer and the display areas MNU1, MNU2, and MNU3 of each image processing menu on the same screen WD7, and each of spatial gradation correction, multi-sheet composite NR, and snowfall removal. It is a figure which shows the example of a display of the moving image after image processing. In FIG. 13, by the user's operation, the menu bar MNU1 for correcting the spatial gradation, the menu bar MNU2 for combining multiple NRs, and the menu bar MNU3 for removing snow and rain are each pressed by the cursor CSR1, so that the image processing apparatus 1 Thus, the image data of the moving image that has been subjected to the image processing corresponding to each menu bar is displayed in the display area VIW. That is, the image processing apparatus 1 according to the present embodiment is not limited to the single image processing (for example, correction processing for removing snow and rain) described with reference to FIGS. A plurality of image processing can be executed in accordance with a simple operation of the user while being displayed, and a plurality of image processing results can be intuitively and visually shown to the user.

It should be noted that spatial gradation correction refers to another parameter (for example, weighting coefficient, etc.) corresponding to input or specification of predetermined parameters (for example, correction method, correction intensity, color enhancement degree, brightness, and correction range). Convert to weighting range, histogram upper limit clip amount, histogram lower limit clip amount, histogram distribution coefficient setting value, distribution start / end position, image blend ratio, color gain), and input parameters using the parameters obtained after conversion Image processing for generating and shaping a local histogram and further generating tone curves to perform tone conversion and color enhancement. The multi-sheet composite NR is an image of the current frame and the frame input immediately before (previous frame) according to the contrast of the image of the input frame (current frame) and a preset motion detection level. This is an image processing for reducing the noise component appearing in the image of the current frame by further calculating the synthesis ratio and synthesizing the image according to the synthesis ratio.

In FIG. 13, as in FIG. 10, a spatial tone correction menu bar MNU 1, a multi-image composite NR menu bar MNU 2, and a snow and rain removal menu bar corresponding to the marker DT 1 shown in FIG. When each marker of the MNU 3 is pressed by the cursor CSR1, the processing control unit 11 of the image processing apparatus 1 is configured to set a plurality of input parameters related to spatial gradation correction, a plurality of composite NRs, and snow and rain removal. The detailed operation screen display area MNU3D is expanded and displayed on the screen. It should be noted that a detailed operation screen display area for setting a plurality of input parameters related to spatial gradation correction and a plurality of composite NRs is not shown. For example, in the spatial gradation correction, the above-described correction method, correction intensity, color enhancement degree, brightness, and correction range) are input or designated by a user operation. Further, in the multi-sheet composite NR, a parameter (that is, the NR level) that specifies the degree to reduce noise by combining the image of the input frame and the image of the previous frame by the user's operation is set in advance as described above. Parameters for specifying the motion detection level, for example, the detection accuracy and detection range of the camera when the image processing apparatus 1 is a camera (that is, detection accuracy and detection range) are respectively input or specified.

As a result, the image processing apparatus 1 reproduces the moving image data displayed in the display area VIW while the user performs a simple operation (that is, whether or not the menu bars MNU1, MNU2, and MNU3 related to a plurality of image processing have been pressed). ), The image processing corresponding to the pressing operation of the display areas MNU1, MNU2, and MNU3 can be accurately executed or the execution can be interrupted, and the user can easily check the processing results before and after the image processing. In addition, the image processing apparatus 1 appropriately performs image processing according to an operation for changing any one of the parameters displayed in the display areas MNU1D, MNU2D, and MNU3D of detailed operation screens for setting each parameter in a plurality of image processing. Or the execution can be interrupted, and the user can easily check the processing results before and after the image processing.

The present invention relates to an image processing apparatus and an image processing method capable of removing snowfall noise and reducing afterimages generated on a relatively large moving object such as a monitoring object when performing snowfall correction by performing median filter processing on an image. Useful as.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 11 Processing control part 12 Image | video input part 13 Median processing part 14 Moving area detection part 15 Moving area size detection part 16 Snowfall correction image generation part 17 Screen display part 21 Input image storage part 22 Median image storage part 23 Moving area Processed

image storage unit

31, 32 Window 33 Radio button 35 Slide bar 37 Numerical display unit WD1, WD2, WD3, WD4, WD5, WD6, WD7 screen

Claims

An image input unit for inputting an image;
A median processing unit that performs median filter processing on the input image input by the image input unit;
A moving region detecting unit for detecting a moving region having a movement of the input image;
An input operation unit that receives input of a snow correction parameter or a rain correction parameter for the input image;
A designated size setting unit for setting a designated size of the moving region corresponding to the snow correction parameter or the rain correction parameter;
A moving region size detection unit that detects a moving region larger than the specified size set by the specified size setting unit;
When the size of the moving area is smaller than the specified size, the image subjected to the median filter processing is used as the moving area image. When the moving area is larger than the specified size, the image input is performed as the moving area image. An image generation unit that generates a snow correction image or a rain correction image using the image input by the unit,
Image processing device.
The image processing apparatus according to claim 1,
A display unit that displays the snowfall correction image or the rain correction image and the menu screen of the snow and rain correction process including the operation screen of the snow correction parameter or the rain correction parameter on the same screen;
Image processing device.
The image processing apparatus according to claim 1,
A display unit that displays the snow correction image or the rain correction image and a plurality of image processing menu screens having a snow and rain correction process including an operation screen of the snow correction parameter or the rain correction parameter on the same screen;
Image processing device.
The image processing apparatus according to claim 2 or 3,
The designated size setting unit
In accordance with the change operation of the snow correction parameter or the rain correction parameter on the operation screen displayed on the display unit, the specified size is set,
The image generation unit
In accordance with the designated size set by the designated size setting unit, the snow correction image or the rain correction image is generated.
Image processing device.
The image processing apparatus according to claim 1,
The input operation unit is
Specify the snow or raindrop size to be removed in pixels.
Image processing device.
The image processing apparatus according to claim 1, wherein:
The moving region size detection unit
A logical sum image of two images obtained by performing contraction processing in the horizontal direction and the vertical direction is generated on the moving region image detected by the moving region detection unit, and further, expansion processing is performed in the horizontal direction and the vertical direction. Use the generated image,
Image processing device.
An image processing method of an image processing apparatus for performing snowfall correction or rain correction,
Inputting an image;
Applying median filtering to the input image, and
Detecting a moving region of the input image;
Receiving an input of a snow correction parameter or a rain correction parameter for the input image;
Setting a designated size of the moving area corresponding to the snow correction parameter or the rain correction parameter;
Detecting a moving region of the specified size or more;
When the size of the moving area is less than the specified size, the image subjected to the median filter process is used as the moving area image, and when the size is equal to or larger than the specified size, the image is input as the moving area image. Generating a snow correction image or a rain correction image using the obtained image,
Image processing method.