JPH1153514A - Method and device for setting region in image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To repair an image without limiting a sample region to within a repair region by selecting a frame for setting a 2nd image region independently of a frame where a 1st image region is set when the 1st image region including noise is repaired by referring to the 2nd image region. SOLUTION: A frame selection part 14 moves the repair region as the 1st image region and the sample region as the 2nd image region to a desired frame according to the input of a choice of a frame when the choice is inputted, and outputs information regarding the frame which is switched according to the input to a CRT 25. Namely, the frame for setting the sample area independently of the frame where the repair region is set is selected. Therefore, the sample region can be set in the same frame with or a different frame from the repair region in a frame selecting process. The sample region which is best used for the repair region can be selected even from a different frame without being limited to within the same frame.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing technique for repairing a repair area including a scratch or a wire in an image by using a sample area serving as a reference image. The present invention relates to an area setting method and apparatus.

[0002]

BACKGROUND OF THE INVENTION With the proliferation of available television broadcast channels and the proliferation of multimedia, many old cinema and photographic films are increasingly being used after being converted to video signals and digital formats. I have. However, over the years of use and browsing, many old cinema and photographic films have become scratched.
s: Scratch, dust or other debris
is: debris).

[0003] Defects caused by scratches and debris in images are very noticeable to viewers. In high definition television and digital television formats, as in other digital multimedia formats, there has been a demand for a technique for removing image defects due to scratches and debris without deteriorating or adversely affecting image quality.

As a technique for removing such image defects, Japanese Patent Application No. 7-303420 discloses a "removal method based on projection of noise of a digital image". In this image processing technique, a repair area including scratch / wire noise is repaired with reference to a sample area serving as a reference area, and scratch / wire noise is removed from an image.

[0005] An example of erasing scratch / wire noise using the above image processing technique will be described. Here, the image 31 to be restored is, as shown in A of FIG. 13, an image in which rows of blocks arranged with the horizontal direction as a longitudinal direction are alternately arranged in the vertical direction in the figure. Is present in a part of the frame 31 displaying the scratch / wire noise 35 whose longitudinal direction is the horizontal direction in the figure.

In order to remove the scratch / wire noise 35, the frame 31 is painted with a mask large enough to cover the scratch / wire noise 35 as indicated by A in the figure. Here, this mask is called a mask image. Then, a rectangular area surrounding the scratch / wire noise 35 is set, and is set as a repair area 1 which is an area to be repaired. In the frame 31, a rectangular area that displays an image similar to the repair area 1 is set as a reference area when the scratch / wire noise 35 is removed, and this area is set as a sample area 2.

[0007] With respect to the repair area 1 and the sample area 2, an image of each area is subjected to fast Fourier transform.
The frequency component obtained by the fast Fourier transform is divided into a phase component and an amplitude component. The amplitude component of the sample area 2 is M1, the phase component is P1, and the amplitude component of the repair area 1 is M1.
2. Let P2 be the phase component. Then, the minimum calculation is performed on the amplitude component M1 of the sample area 2 and the amplitude component M2 of the repair area 1, and using the amplitude component obtained by the minimum calculation and the output of the phase component P2 of the repair area 1,
When the inverse fast Fourier transform is performed, a restored image is obtained.

[0008] The restored image and the original image obtained here are mixed using a mask image. A portion with a mask gives the pixel value of the restored image, and a portion without the mask gives the pixel value of the original image. The image obtained as a result is used as an original image to be used for the next restoration, and a series of processing is repeated again from the constraint-freedom transformation. When this process is repeated several times, the scratch / wire noise 35 is restored.

[0009]

In the above-described conventional image processing technology, the setting of the sample area is
It was limited to the frame including the repair area. Therefore, when an image area similar to the repair area 1 including the scratch / wire noise 35 does not exist in the same frame 31 as shown in FIG. 14, a sample area cannot be set. Therefore, when the sample area corresponding to the repair area 1 cannot be set on the same frame, the image of the frame 31 cannot be restored.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described circumstances, and is directed to an image processing technique for specifying a scratch / wire on an image in a repair area and repairing the image by referring to the sample area. It is an object of the present invention to provide a method and an apparatus for setting an area in an image in which an area can be restored without being limited to the same frame as the repair area.

[0011]

In order to solve the above-mentioned problems, a method for setting an area in an image according to the present invention restores a first image area containing noise with reference to a second image area. In the method for setting an area in an image related to image processing, the second image is set independently of the frame in which the first image area is set.
Has a frame selecting step of selecting a frame for setting the image area.

According to a second aspect of the present invention, there is provided an image region setting apparatus for performing image processing for restoring a first image region including noise with reference to a second image region. There is provided a frame selecting means for selecting a frame for setting the second image area independently of the frame in which the first image area is set.

That is, the above-described method of setting an area in an image is performed in image processing using two image areas: a sample area serving as a reference image in image processing and a repair area including scratch / wire noise in the image. A frame for setting a repair area to be repaired including noise and a frame for setting a sample area to be referred to when repairing the repair area can be independently selected.

In the above-described method for setting an area in an image, since a frame is selected in the frame selection step, the repair area and the sample area can be obtained from the same frame as the repair area or from a different frame.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a method and apparatus for setting an area in an image according to the present invention will be described below with reference to the drawings.

As shown in FIG. 1, the apparatus for setting an area in an image includes a control section 10 for controlling the entire apparatus, a central processing section 21 for centralizing data calculations, and a memory for storing data. 22.

The control section 10 is a controller for controlling each section of the apparatus. This control unit 10
The central processing unit 21 and the memory 22 transmit and receive data, and execute a series of procedures related to a method for setting an area in the image integrally with the central processing unit 21 and the memory 22. The control unit 10 receives data from a mouse 23, a pen / tablet 24, and an image input 26, and outputs data to a CRT 25.

The central processing unit 21 is a processor that centrally performs calculations on data. This central processing unit 21
Is connected to the control unit 10 and performs an operation on data provided from the control unit 10 under the control of the control unit 10.

The memory 22 is a storage unit for storing data. The memory 22 is connected to the control unit 10, and writes and reads data under the control of the control unit 10.

The area setting device has a mouse 23 and a pen / tablet 24 as input means, and a CRT 25 as an output means for outputting an image relating to the processing of the area setting method in the image.

The mouse 23 is a pointing device having a casing large enough to be operated by holding it with the palm. The mouse 23 is slid in a desired direction and a desired distance on a flat surface such as a desk, for example, to input a position of a cursor displayed on the CRT 25, for example. Then, the mouse 23 transmits data relating to the input position to the control unit 10.

The pen / tablet 24 has a substantially rectangular parallelepiped flat casing, and has a plurality of sensing elements arranged in a matrix on one of its flat main surfaces. And a pen for inputting a position by tracing the main surface of the tablet with a pen tip. The pen / tablet 24 transmits data relating to the input position to the control unit 10.

The CRT 25 is an output means for displaying data relating to an image as an image on a plurality of pixels arranged on the cathode ray tube surface. The CRT 25 includes the control unit 10
Transmits data relating to an image, and displays the data as an image.

The above-mentioned area setting device has a frame selecting section 14 as shown in FIG. This frame selection unit 1
4 includes, for example, the control unit 10, the central processing unit 21, and the memory 22. This frame selection unit 14
The input is received from the mouse 23 and output to the CRT 25.

The frame selecting section 14 is a frame selecting means for receiving a frame selection and changing the frame in accordance with the input. The frame selection unit 14 moves the repair area as the first image area and the sample area as the second image area to a desired frame according to the input. Then, the frame selection unit 14 outputs information on the frame switched according to the input to the CRT 25.
Output to

As described above, the frame selection unit 14
The frame for setting the sample area is selected independently of the frame in which the repair area is set.

The area setting device has an area determining unit 11 as shown in FIG. This area determination unit 11
Comprises, for example, the control unit 10, the central processing unit 21, and the memory 22.

The area determining section 11 has a condition parameter generating section 12 for generating a constraint condition corresponding to a condition parameter deriving step, and an area shape input section 13 for selectively accepting an input satisfying the constraint condition. ing. The condition parameter generating unit 12 generates a constraint condition between the repair region and the sample region, and inputs the parameters of the constraint condition to the region shape input unit 13.

The area shape input section 13 receives the shape of the user's image area by input means such as a mouse 23 and accepts only an input satisfying the constraint condition given by the condition parameter generating section 12. The area shape input unit 13 causes the CRT 25 to display information relating to the image area. Further, the area shape input unit 13 receives the shape of one of the repair area and the sample area, and outputs the shape of the other area of the repair area or the sample area.

Next, a series of steps according to the method for setting an area in an image will be described with reference to a flowchart shown in FIG.

In the first step S1, image data is input. That is, data representing an image including unnecessary scratch / wire noise is input to the image input 2 shown in FIG.
Input via 6. Then, the process proceeds to step S2.

In step S2, a scratch / wire noise area is specified. That is, the CRT shown in FIG.
Based on the display of 25, the user specifies a scratch / wire noise region using a pointing device such as a mouse 23 or a pen / tablet 24, for example. The scratch / wire noise area specified in this way is, for example, highlighted. Then, the next step S3
Proceed to.

In step S3, the above step S
Binary mask data is generated that distinguishes the scratch / wire noise region identified in 2 from other regions of the image. This binary mask data gives a value “0” to each pixel in the specified scratch / wire noise area, for example.
By giving “1” to each pixel outside the specified scratch / wire noise region, the scratch / wire noise region is specified. As the binary mask data, the value “1” is set in the scratch / wire noise region,
A value “0” can be given outside the scratch / wire noise region. Then, the process proceeds to the subsequent step S4.

In step S4, an image area is specified. That is, a repair area which is a first image area having a rectangular shape, for example, including a scratch / wire noise area is designated. Further, a sample area which is a second image area to be referred to when the repair area is repaired is designated.

Here, as the sample area, an area on the image that is most similar to the repair area in structure, line, color, luminance, texture, gray value, and the like is selected. Then, the process proceeds to the next step S5.

In step S5, the area shape of the repair area or the sample area is specified. That is, the shapes of the repair area and the sample area input in step S4 are deformed under the constraint conditions described later. Then, the process proceeds to step S6.

In step S6, the sample area is moved. That is, the sample area necessary for repairing the repair area is moved to the same frame or another frame of the repair area. Here, the movement of the frame is performed by the frame selection unit 14. Then, the process proceeds to the next step S8.

In step S8, the FFT is calculated for the sample area, and in step S9, the FFT is calculated for the repair area. Then, the process proceeds to the next step S10.

In step S10, a new image spectrum is generated using the DC amplitude of the repair area and the small amplitude of the repair area and the sample area of other frequencies in the frequency spectrum of the repair area. Then, the process proceeds to step S11.

In step S11, a new image data is created by calculating the inverse fast Fourier transform (IFT) of the new image spectrum, and in step S12, the new image data is limited to 0 to 255. Then, the process proceeds to step S13.

In step S13, new image data of the repair area is created, and the subsequent step S14
In, replace the repair area with new image data,
Proceed to step S15.

In step S15, it is determined whether the repair area replaced with the new image data has a good result. If the result is good, the series of steps relating to the image processing is terminated as "YES", and if not, the procedure returns to step S9 as "NO".

Here, of the steps relating to the above-mentioned area setting method, steps S4 to S6 will be described in detail with reference to the flowchart shown in FIG.

In step S4, an image area is designated. That is, in this step S4, the user designates a rectangular image area including a scratch / wire noise in the image using, for example, the input means such as the mouse 23 shown in FIG. Thereby, the repair area to be repaired is specified. When the repair area is specified, a sample area that is referred to when the image of the repair area is restored is created. This sample area has the same size as the repair area. When these processes are completed, the process advances to the next step S5.

In step S5, the area determining section 11 shown in FIG. 3 determines the shape of the image area based on the shape of the image area input by the user.

That is, in the area determination section 11, when the user inputs the movement and deformation of the image area into the area shape input section 13 using the mouse 23 or the like, the shape of the image area is converted into the condition parameter. The image is deformed based on the constraint generated by the unit 12, and the shape of the image area is determined. However, the area shape input unit 13 ignores the input when the shape of the input image area does not comply with the constraint condition. Then, the process proceeds to the next step S6.

In step S6, the sample area is moved. The user can move the sample area to be referred to when repairing the repair area to an arbitrary position in the frame of the repair area, but there is a case where there is no appropriate moving part in the image. In such a case,
The sample area can be set in a frame different from the repair area. Thus, this step S6 is
There is a frame selecting step of moving a frame for setting a sample area.

When a sample area is set in a frame different from the repair area, the area shape input unit 13 is instructed to select a sample area. When another frame is selected using the frame selection unit 14 shown in FIG. 2, the frame selection unit 14 displays another frame on the CRT 25.

Next, a description will be given of a constraint condition that when the shape of the repair region or the sample region is input, one shape of the repair region or the sample region is the other shape. The constraint condition relating to the image area is generated by the condition parameter generation unit 12 of the shape determination unit 11, and the shape input determination unit 13 selectively receives an input based on the constraint condition.

That is, the constraint condition “keeping the repair area and the sample area always the same shape”, which is the constraint condition, is described in FIG.
Are generated from the condition parameter generating section 12 of the area determining section 11 shown in FIG. The area shape input unit 13 provided in the area determination unit 11 inputs the deformation of the area shape input by the user using the mouse 23 or the like only when the area satisfies the constraint condition.

The repair area 1 here has a substantially rectangular shape that is horizontally long in the figure, as shown at A in FIG. The repair area 1 is transformed into the repair area 3 in the figure by dragging the lower right vertex 1A in the figure with the mouse 23 in the upper left direction in the figure to the vertex 3A in the figure. Is done. The deformed repair area 3 has a substantially rectangular shape which is horizontally long in the drawing, and has the same top vertex 1B and 3B as the repair area 1 before the deformation and the upper and lower vertices in the drawing. The sides overlap each other.

On the other hand, the sample area 2 has the same shape as the repair area 1 shown in A in the figure, as shown in B in the figure. When the repair area 1 is transformed into the repair area 3, the sample area 2 is transformed from the sample area 2 to the sample area 4 accordingly. The deformed sample area 4 has the same shape as the repair area 3 after deformation, and has a substantially rectangular shape that is horizontally long in the drawing. Also,
The deformed sample area 4 has the same upper-left vertices 2B and 4B as the sample area 2 before the deformation, and the upper and left sides in the figure overlap each other.

Conversely, when the shape of the sample area is deformed,
The repair area 1 is also a sample area 2 based on the above constraint conditions.
It is transformed into the shape of That is, with respect to the sample area 2 shown in B in the figure, the lower right vertex 2A of this rectangle is dragged by the mouse 23 in the upper left direction in the figure to the vertex 4A in the figure, whereby It is transformed to the sample area 4. At this time, the shape of the sample area before and after this deformation is the repair area 1 corresponding to the sample area 2 described above.
Is changed from the repair area 1 to the repair area 3 in response to the change of the sample area 2 to the sample area 4 as shown in FIG.

Similarly, the shape of the repair area and the sample area may be a parallelogram as shown in FIG. A in the figure is a repair area 1, and B in the figure is a corresponding sample area 2. Here, the repair region 1 and the sample region 2 do not have the same shape, but each shape satisfies the condition that the length of a pair of opposite sides is equal to the interval between the opposite sides.

With respect to the repair area 1 and the sample area 2 of the parallelogram, the condition parameter generator 12
For example, `` When the shape of one region is a parallelogram, the shape of the other region is at least one of the pair of opposing sides, the length and the distance between the sides of the set. Is defined as a parallelogram whose distance is equal to any one of a pair of opposing sides of the parallelogram.

Here, as an example of the modification of the parallelogram under the constraint condition, the constraint condition “Either pair of two sides is parallel to the vertical axis or the horizontal axis of the image or the frame including this image. An operation of deforming the shape of the repair region or the sample region for a certain parallelogram will be described. The following description of the deformation operation is made for the repair area, but the same applies to the sample area.

Here, as shown in FIG. 8, the repair area 1 has one of a pair of opposing sides parallel to the vertical axis of the frame.
The other has a substantially rectangular shape parallel to the horizontal axis of the frame.
At the vertices and the midpoint of each side of the rectangle, handles indicated by “■” symbols are provided. By clicking and dragging the handle with the mouse 23, if the constraint condition is satisfied, the repair area 1 can be deformed into a desired shape.

For convenience of explanation, the upper right vertex in the figure is denoted by a handle 1p, the lower right vertex in the figure is denoted by a handle 1s, and the middle point of the information side is denoted by a handle 1t. To The same reference numerals are given to the handles at the corresponding locations in the following drawings.

The operation on the handle 1p of the repair area 1 is performed by dragging the handle with the mouse 23 to enlarge / reduce the shape of the repair area 1. Does not change to a shape that does not satisfy the condition, the handle 1p can always be operated.

The operation on the handle 1t of the repair area performs a deformation process in the direction of the horizontal axis of the frame in the horizontal direction in the figure, but is possible when one set of sides is parallel to the horizontal axis of the frame. . When a lateral movement operation is performed on the handle 1t, as shown in FIG. 9, the upper side of the repair area 1 moves laterally in the figure with the handle 1t.

When the handle 1s of the repair area 1 is dragged with the mouse 23 and moved in the direction of the vertical axis of the vertical frame in the figure, as shown in FIG. The right side moves in the vertical direction in the figure with the handle 1s.

Further, as shown in FIG. 11, the shape of the repair area 1 can be modified even when one set of sides is parallel to the vertical axis of the frame. However, by moving one of the set of sides parallel to the vertical axis from the rectangle having the set of sides parallel to the vertical axis and the horizontal axis of the frame, which is the horizontal direction and the vertical direction in the figure, by moving it in the vertical axis direction. The parallelogram already deformed in the vertical direction is the handle 1 of this repair area.
Deformation in the horizontal direction using t results in a shape that does not meet the constraint conditions. Therefore, lateral deformation of the handle 1t is not permitted.

In general, when the repair area 1 and the sample area 2 are free shapes, examples of the constraint conditions generated by the condition parameter generating unit 12 include “when the shape of one area is given, The shape of the other region is symmetrical to the shape of the one region. " Here, the symmetry is, for example, point symmetry or line symmetry.

Next, a sample area in the moving image processing using two image areas, a sample area 2 serving as a reference image in the image processing and a repair area 1 including a noise area such as a scratch or a wire in the image. Frame 32 for setting area 2
And the selection of the frame 31 for setting the repair area 1 will be described.

The frame 32 for setting the sample area 2 and the frame 31 for setting one repair area can be independently selected by the frame selection unit 14 regardless of whether they are the same frame or different frames. .

That is, in the image indicated by A in FIG. 12, the image is displayed only at the center of the frame 31, and the scratch / wire noise 35 exists so as to cross this image from the upper left to the lower right in the figure. I have. On the scratch / wire noise 35, a substantially parallelogram-shaped repair region 1 is formed so that the mask image covers the scratch / wire noise 35.

In the frame 31 including the scratch / wire noise 35, a sample area to be referred to when the repair area is repaired cannot be set. In this frame 31, since an image is displayed only at the center,
This is because there is no room to select a sample area 2 having the same shape and an image similar to the repair area 1 referred to when the repair area 1 is repaired.

In such a case, the sample area 2 to be referred to when the repair area is repaired is replaced with the scratch / wire noise 35 by using the frame selecting section 14 as shown by B in FIG. Can be set in the other frame 32 of the frame 31 including. Therefore, the present invention is not limited to the frame 31 including the repair area 1 but may include other frames 32.
Since the sample area 2 can be set also in the above, the range of selection of the sample area 2 is expanded, and the function of restoration is enhanced.

[0069]

As described above, in the method for setting an area in an image, the sample area can be set in the same or different frame as the repair area in the frame selecting step. Therefore, the user can select the optimal sample area to be used in the repair area from not only the same frame as the repair area but also a different frame. The above-described area setting method can be used to select a sample area from a frame different from the repair area, and is therefore significant when a sample area cannot be set in a frame including the repair area.

Further, the apparatus for setting an area in an image can set the sample area in the same or different frame as the repair area by the frame selecting means. Therefore,
The user can select the optimal sample area to be used in the repair area from not only the same frame as the repair area but also a different frame. Since the above-mentioned area setting device can select a sample area from a frame different from the repair area, it is significant when a sample area cannot be set in a frame including the repair area.

[Brief description of the drawings]

FIG. 1 is a block diagram schematically showing an overall configuration of an area setting device in an image.

FIG. 2 is a block diagram showing a frame selection unit of the area setting device.

FIG. 3 is a block diagram schematically showing a configuration of an area determining unit of the area setting device.

FIG. 4 is a flowchart showing a series of steps of an area setting method in an image.

FIG. 5 is a flowchart showing a main part of the area setting method.

FIG. 6 is a diagram illustrating an example of deformation of the shape of an area in an image.

FIG. 7 is a diagram illustrating another example of deformation of the shape of an area in an image.

FIG. 8 is a diagram illustrating an example of an operation for deforming the shape of a region in an image.

FIG. 9 is a diagram illustrating an example of an operation of deforming the shape of a region in an image.

FIG. 10 is a diagram illustrating an example of an operation for deforming the shape of a region in an image.

FIG. 11 is a diagram illustrating an example of an operation for deforming the shape of a region in an image.

FIG. 12 is a diagram illustrating a case where a repair area and a sample area are set to different frames.

FIG. 13 is a diagram illustrating a repair area and a sample area.

FIG. 14 is a diagram illustrating a case where a sample area cannot be set in a frame including a repair area.

[Explanation of symbols]

 10 control unit, 23 mouse, 25 CRT

Claims (7)

[Claims] 1. An area setting method for an image related to image processing for restoring a first image area including noise with reference to a second image area, the method comprising the steps of: Comprises a frame selecting step of independently selecting a frame for setting a second image region. 2. The method according to claim 1, wherein the noise is a scratch or a noise caused by a wire. 3. When the shape of one of the first image area and the second image area is given, the shape of the other area of the first image area or the second image area is 2. The method according to claim 1, further comprising a step of generating a condition parameter for generating a constraint condition for forming one of the regions. 4. When the shape of one of the first image region and the second image region is given, the shape of the other region of the first image region or the second image region is 2. The method for setting an area in an image according to claim 1, further comprising a condition parameter generating step of generating a constraint condition for setting a shape of the one area as a target shape. 5. When the shape of one of the first image area or the second image area is a parallelogram, the shape of the other area of the first image area or the second image area is Is a parallelogram in which at least one of the set of opposing sides has a length and a distance between the sides equal to any one of the set of opposing sides of the parallelogram. 2. The method according to claim 1, further comprising a condition parameter generating step of generating a constraint condition. 6. An area setting device in an image related to image processing for restoring a first image area including noise with reference to a second image area, wherein: a frame in which the first image area is set; An apparatus for setting an area in an image, comprising: a frame selecting means for independently selecting a frame for setting a second image area. 7. The apparatus according to claim 6, wherein the noise is a scratch or a noise caused by a wire.