JP4169842B2 - Electronic image correction device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an electronic image correction apparatus that erases unnecessary lines and unnecessary points by instructing a specific part of a subject on a specific image in a moving image and performing image processing that makes the part inconspicuous. Is.
[0002]
[Prior art]
There are almost three conventional methods related to this, and the first is when processing is performed before shooting, and the subject to be erased is colored or illuminated so that it has the same color and brightness as the surroundings. Moderate.
The second method is after shooting, and the captured image is corrected by an electronic drawing device so that it is not conspicuous manually.
The third method is also a case after taking an image, which uses a property (for example, spatial frequency) as an image to be erased and applies a two-dimensional filter uniformly to the entire image. Taking the spatial frequency as an example, it is a technique of first analyzing the spatial frequency component of the line or point to be erased, and then excluding the component near the frequency in the image.
[0003]
[Problems to be solved by the invention]
However, Method 1 described above requires a worker who is skilled in the work and cannot achieve the same color and brightness as the surroundings (for example, the color and brightness cannot be adjusted physically). There is also.
In the second method, a more skilled worker is required to work on the basis of the sensitivity, and in the case of a moving image, enormous labor and time are required.
Further, in the third method, the corresponding line or point is at least inconspicuous, but there is a drawback that an image component to be stored having a component near the corresponding frequency may also disappear, and the space of the line or point to be erased. Analyzing frequency components is very difficult when the corresponding line or point is photographed with another subject, and the work is complicated and requires experience. In addition to those using spatial frequency, non-linear filters such as epsilon filters are also used, but this cancels only low-amplitude signals on the image, such as facial wrinkles, and sharp edges such as facial contours. The signal seems to be convenient at first glance because it has a characteristic of storing, but even a video signal other than wrinkles that is to be stored cancels even a low-amplitude video signal that resembles wrinkles, so that the applicable images are limited.
Therefore, the object of the present invention does not require a skilled worker as in the prior art, and does not require the sensitivity of the person who performs the operation as much as possible, and is unnecessary in a moving image without much labor and time. An electronic image correction apparatus capable of erasing dots and lines is provided.
[0004]
[Means for Solving the Problems]
To this end, a first invention according to the electronic image correction apparatus includes a means to specify a series of consecutive images in the site of on the subject of the specific image as a point or a line, time to that particular image A means for tracking the same part on the subject designated in the previous image on the continuous image based on the motion vector detection, and the position based on the position information of the characteristic point of the certain part obtained by the tracking means A regression line that approximates a part is calculated, and pixel information that is located at a predetermined distance in the vertical direction with respect to the regression line is used to weight pixels located between the opposing pixels according to the predetermined distance And means for performing image processing on the same part on the subject of each image by performing replacement correction by distribution .
The electronic image correction apparatus according to the present invention, when specifying the a site as a line may connect a sparse since it was ruptured by connecting between a point near the torn line of distance lines. Also, if the tracked coordinate line has a certain width, a differential image is generated from the image, the width of the estimated line is estimated using this, and the line of the estimated width is represented by pixel information outside the width. substituting corrected, further, when replacing correct than its width of the outside of the pixel information may be a weighted allocation substitution correction of the image information corresponding to this distance. Here, having a certain width on the coordinates means that the line width is expressed by a plurality of pixels.
Further, in the above-described device, when the certain part is designated as a point, and the point on the tracked coordinates has a certain spread, a differential image is generated from the image, and the width of the point is estimated using this, When the pixel information outside the area of the point is replaced and corrected, and when the pixel information outside the area of the point is replaced and corrected, the weighted distribution replacement correction of the image information according to the distance can be performed. Here, a point on a coordinate having a certain spread means that the point is expressed by several pixels or more . The image processing means may be a means for performing high-pass filter processing using image information of the point or line itself and image information outside the point or line, and the second invention is the means for image processing. However, it may be a non-linear filter processing unit that uses the image information of the point or line itself and the image information outside the point or line to smooth only a low-amplitude signal at the certain part. The nonlinear filter may be an epsilon filter, for example.
[0005]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.
It shows a configuration block diagram of the present invention an electronic image correction apparatus non-limiting embodiment in Figure 1. In this embodiment, the target to be erased is a facial wrinkle, but the present invention uses a line or a point as the target, and is not limited to a facial wrinkle.
In the following, the contents of each block will be described first in the configuration block diagram of FIG. 1, and then the overall flow will be described.
(A) Image input unit 1: An NTSC image signal (which may be a moving image or a still image) from a target image source is converted into digital data that can be stored in the first image storage unit 3 in the next stage. Although the present invention is not particularly limited as to the format to be converted, wrinkles on the face are wrinkled on the image due to shadows due to illumination. That is, there is a change in brightness, but there is no change in color. Therefore, the processing is only a luminance signal, and it is not necessary to modify the color information signal. Therefore, in this embodiment, the NTSC image signal is converted into a luminance signal and a color difference signal, and only the luminance signal is corrected in the erasure processing unit at the subsequent stage.
(B) Input instructing unit 2: Instructs the start timing of the image to be corrected, and the first image accumulating unit starts accumulating image information from the image input unit 1 (instructed by the operator). ).
(C) First image storage unit 3: Starts storing image information from the image input unit 1 at the timing of an instruction from the input instruction unit 2. In this embodiment, since the storage end instruction is not given, the storage processing is ended when the first image storage unit 3 has been stored up to the full recording capacity (usually storage for several hundred frames).
(D) Position instruction unit 4: This is a part that performs an erasure correction target part in response to an operator instruction. In this embodiment, the first image that is the earliest in time order among the images stored in the first image storage unit 3 is taken out and displayed, and a target part in this image is displayed with a mouse that is used by a personal computer or the like. Use to indicate. However, the present invention does not limit the method of this work. The position information specified by the position instruction unit 4 is sent to the feature point position storage unit 7 at the next stage.
(E) Feature point tracking unit 5: Acquires position information of the part on the image from the feature point position storage unit 7, and reads the image and temporally subsequent image from the first image storage unit 3. . From this, it is determined where the part of the image is located on the subsequent image. In this embodiment, the block matching method, which is a general method, is used. However, the present invention is not limited to this, and general motion vector detection methods such as an optical flow method can be used. In any case, there is a possibility that the existing motion vector detection method may be erroneously detected, and it is necessary to cope with this. In this embodiment, the corresponding part of the next image is obtained from the image by block matching and obtained. Block matching is performed on the image from the part of the next image, and if it is not equal to the position of the part of the image, it is regarded as a false detection and eliminated. The feature point position information obtained by the above processing is sent to the fragmentation prevention unit 6.
(F) Fragmentation prevention unit 6: When the subject is enlarged or reduced in time order, the distance between feature points changes. Especially when enlarged, a gap is opened between the feature points. Further, a portion that is regarded as an erroneous detection by the feature point tracking unit 5 also becomes a gap. The fragmentation prevention unit 6 connects the gaps that are close to each other with a straight line, and adds the position of the pixel on the straight line as a new feature point. In the present embodiment, the connection is made with a straight line, but the present invention does not limit this connection method. For example, a spline (cloud-line ruler) curve passing through the feature point sent from the feature point tracking unit 5 is calculated, and a pixel position passing through the spline curve is newly determined as a countermeasure for a gap in the feature point. Add as a feature point. Here, as a reason for preventing fragmentation, if a feature point is fragmented for the above-described reason, for example, a part of the wrinkle to be erased is not processed in the subsequent erasure processing unit, and the wrinkle remains. As a countermeasure against this, fragmentation is prevented by utilizing the feature of being continuous on an image such as wrinkles. The feature point position information connected by the fragmentation prevention unit 6 is sent to the feature point position storage unit 7.
(G) Feature point position accumulating unit 7: Stores the position information of the feature points sent from the position instruction unit and the feature point tracking unit and sends them to the erasure processing unit.
(H) Erasing processing unit 8: Position information of the feature point position accumulating unit 7 and an image corresponding thereto are extracted from the first image accumulating unit 3, and five feature points before and after on the screen at each feature point position are used. (In the case of both ends, there may be less than 5 feature points before or after, but in that case, feature points as far as the end are used), for example, the regression line 21 is obtained by the least square method. . Two pieces of luminance information (brightness c and luminance d) at positions separated by about 5 pixels in both directions in the vertical direction 22 with respect to the straight line 21 (brightness c and luminance d) are used. In accordance with the distance from b), the luminance information (luminance c, luminance d) is weighted and replaced with the inner luminance information (see FIG. 2). Here, in this embodiment, the regression line 21 is obtained using five points before and after, but in the present invention, this score is not limited. Further, although luminance information at a position of about 5 pixels from the feature point is used, the present invention does not limit this distance, and the image is differentiated two-dimensionally, and the line width is obtained from the contour image and used. Is also effective.
(I) Second image storage unit 9: A part for storing images processed by the erasing processing unit.
(J) Output instructing unit 10: A part for instructing an opportunity to reproduce a moving image stored in the second image storage unit.
(K) Image output unit 11: A part for converting digital data of a moving image sent from the second image storage unit 9 into an NTSC image signal.
In this embodiment, processing is performed using the above blocks. The processing procedure is as follows.
First, an image source to be erased is connected to the image input unit 1, and an accumulation instruction is given by the input instruction unit 2 at the beginning of the time of the moving image. As a result, the portion of the moving image is stored in the first image storage unit 3. The head image of the accumulated moving image is pulled out by the position instructing unit 4 to instruct the position of the part desired to be deleted. When the position instruction is finished, it is stored in the feature point position storage unit 7 as a feature position. Thereafter, the feature point tracking unit 5 reads the feature point position from the feature point position accumulating unit 7, and the corresponding image (referred to as image 1) and temporally following image (referred to as image 2) are the first image. Read from the storage unit 1. A feature point position corresponding to the feature point position of image 1 is extracted from image 2 from these pieces of information. The feature point positions obtained here are converted into continuous curves by the fragmentation prevention unit 6 and stored in the feature point position storage unit 7. Thereafter, the same processing is performed on the images as long as they are in the first image storage unit 1. When the feature point positions are obtained for all the images, an image in which the part to be erased is erased from the feature point position information in the feature point position accumulation unit 7 and the image information in the first image accumulation unit 1 in the erasure processing unit 8. Is generated and stored in the second moving image storage unit 9. The moving images stored in the second image storage unit 9 are reproduced from the top image in response to an instruction from the output instruction unit 10.
[0006]
Although the embodiments of the present invention have been described above with reference to the embodiments, the present invention is not limited thereto, and it is obvious that various modifications and changes can be made within the gist of the invention defined in the claims. Will.
[0007]
【The invention's effect】
As described above, according to the present invention, the electronic image correction that eliminates the disadvantages that have been problematic in the conventional method and can remove unnecessary lines and points in the photographed image with less effort and in a short period of time. An apparatus can be provided.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of an electronic image correction apparatus according to an embodiment of the present invention.
FIG. 2 is a diagram for explaining the operation of an erasure processing unit of the apparatus of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Image input part 2 Input instruction | indication part 3 1st image storage part 4 Position instruction | indication part 5 Feature point tracking part 6 Fragmentation prevention part 7 Feature point position storage part 8 Erasing process part 9 2nd image storage part 10 Output instruction | indication part 11 Image output unit

Claims (3)

A series of a means to specify the site of on the subject of successive images in specific image as a point or a line, the same site on the subject specified in the previous image on the temporally continuous images for that particular image, motion A means for tracking based on vector detection; and a regression line that approximates the same part based on positional information of the characteristic point of the certain part obtained by the means for tracking , and is perpendicular to the regression line By using the pixel information of the opposing pixels at a predetermined distance, the pixels located between the opposing pixels are replaced and corrected with a weighted distribution according to the predetermined distance, so that the same part on the subject of each image is imaged. An electronic image correction apparatus comprising: means for processing. 2. The apparatus according to claim 1, wherein the certain part is designated as a line. If the tracked line on the coordinate has a certain width, a differential image is generated from the image, and the width of the line is estimated using the differential image. An electronic image correction apparatus configured to replace and correct a line having an estimated width with pixel information outside the width. The electronic image according to claim 1 or 2, wherein the certain part is designated as a line, and the broken line is connected by connecting the sparse and broken lines to each other at a short distance. Correction device.

Images