JP5663951B2 - Image processing program and image processing apparatus - Google Patents

Description

  The present invention relates to an image processing program and an image processing apparatus.

  There are known methods for erasing an undesired object in a digital image as follows. In this method, an image is corrected by erasing an undesired object or the like from the digital image by replacing pixels in an area including the undesired object or the like specified by the user with the generated texture (for example, Patent Document 1).

Japanese Patent Laid-Open No. 2005-63433

  However, in the conventional method, a desired subject is not necessarily deleted from the area designated by the user, and a subject different from the subject desired by the user may be deleted.

An image processing program according to claim 1 of the present invention calculates an area setting procedure for setting an edit target area in an image, a hue frequency in an image of the edit target area, and selects a color having the highest hue frequency. An extraction procedure for extracting as a characteristic color of an unnecessary subject, a generation procedure for generating a pixel value of a pixel having the second highest hue frequency as editing data, and the editing target area using the editing data an image processing procedure for replacing the pixel value of the characteristic color of the pixels, and characterized by causing a computer to execute the.
An image processing program according to claim 2 of the present invention calculates an area setting procedure for setting an edit target area including a subject area and a background area in an image, a hue frequency in the image of the edit target area, When the background area is larger than the subject area and included in the edit target area, the color having the second highest hue frequency is extracted as a characteristic color of the unnecessary subject, and the background area is larger than the subject area. When not included in the edit target area, an extraction procedure for extracting the color with the highest hue frequency as the characteristic color, and when the background area is included in the edit target area more than the subject area A pixel value of a pixel having a color with the highest hue frequency is generated as editing data, and when the background area is larger than the subject area and not included in the editing target area, the hue A generation procedure for generating a pixel value of a pixel having the second highest frequency as the editing data, and an image processing procedure for replacing the pixel value of the characteristic color pixel in the editing target area using the editing data And making the computer execute.

  According to the present invention, it is possible to improve the possibility that the subject desired by the user can be deleted.

It is a block diagram which shows the structure of one Embodiment of a personal computer. It is a figure which shows the specific example in the case of deleting the electric wire A reflected in the image. It is a flowchart figure which shows the flow of an image correction process.

  FIG. 1 is a block diagram showing a configuration of an embodiment of an image processing apparatus according to the present embodiment. As the image processing apparatus, for example, a personal computer (personal computer) 100 is used. The personal computer 100 includes an operation member 101, a connection IF (interface) 102, a control device 103, an HDD (hard disk drive) 104, a monitor. 105.

  The operation member 101 includes various devices operated by the user, such as a keyboard and a mouse. The connection IF 102 is an interface for connecting the personal computer 100 and an external device. In the present embodiment, the personal computer 100 is connected to the digital camera via the connection IF 102, and an image file obtained by photographing is taken from the digital camera. As the connection IF 102, a USB interface for connecting the personal computer 100 and the digital camera by wire, a wireless LAN module for wireless connection, or the like is used.

  The control device 103 includes a CPU, a memory, and other peripheral circuits, and controls the entire personal computer 100. The memory constituting the control device 103 is a volatile memory such as SDRAM, for example. This memory includes a work memory for the CPU to expand the program when the program is executed, and a buffer memory for temporarily recording data.

  The HDD 104 is a recording device for recording image file data fetched via the connection IF 102 and data of various programs executed by the control device 103. The program data recorded in the HDD 104 is provided by being recorded on a storage medium such as a CD-ROM or DVD-ROM, and the user installs the program data in the HDD 104 using the storage medium. The control device 103 can execute the program. The monitor 105 is a liquid crystal monitor, for example, and displays various display data output from the control device 103.

  In personal computer 100 in the present embodiment, a user can execute an image processing application to display an image of an image file recorded in HDD 104 and edit the image being displayed. For example, as shown in FIG. 2A, when the electric wire A is reflected in the image, the user can restore the image by deleting the electric wire A from the image. Specifically, as shown in FIG. 2B, the user designates the region 2a including the electric wire A by using a rectangular tool which is one of editing tools included in the image processing application, As shown in c), the region 2b including the electric wire A is designated by using a repair brush tool which is one of editing tools included in the image processing application.

  The control device 103 identifies an area (2a or 2b) designated by the user as an editing target area, and performs image processing for deleting an unnecessary subject (in this case, the electric wire A) included in the editing target area ( Hereinafter, this is referred to as “image correction processing”. A specific image correction processing method will be described later.

  When the restoration brush tool is used as a method for designating the editing target area, the user selects the size of the brush to be used, and then selects the brush whose size is to be erased on the image, that is, the above-described electric wire A. By moving on an unnecessary subject such as, it is possible to designate an area within the range where the brush has moved as an editing target area. Note that a plurality of sizes of brushes are prepared in a stepwise manner, and the user selects an arbitrary brush size from among them.

  At this time, the user sets the editing target area so as to include the subject to be erased, but the control device 103 may specify which subject in the editing target area is to be erased by the user. Otherwise, as a result of the image correction process, there is a possibility that the subject that the user wanted to delete is not correctly deleted.

  For example, as shown in FIG. 2 (d), the user leaves an object other than the wire A among the subjects in the image shown in FIG. 2 (a), and obtains an image in which only the wire A is erased. Suppose you want to edit and specify the editing area. However, as a result of the image correction process, as shown in FIG. 2E, the flower A in the region 2c may be erased while the electric wire A originally intended to be erased remains. Or as shown in FIG.2 (f), both the electric wire A and a flower may be erase | eliminated.

  In such a case, after that, the user needs to reset the edit target area and execute the image correction process. Furthermore, when the result of the image correction processing at that time is not what the user desires, the user needs to set the editing target area again, and must repeat this until a desired result is obtained. . In the present embodiment, the control device 103 performs the following process so that the subject intended by the user can be erased from the image without the user performing such a complicated operation.

  Specifically, when the editing target area is specified on the image using a brush by the user, the control device 103 extracts an image in the specified editing target area, and within the extracted editing target area The color space of the image, for example, the RGB space is converted into the LCH space. The control device 103 calculates the hue of each pixel with respect to the image in the editing target area after conversion into the LCH space. Then, the control device 103 divides the hue circle into a plurality of regions. For example, the control device 103 divides the hue circle into 10 parts with a 16 degree width or 16 parts with a 22.5 degree width.

  After that, the control device 103 calculates the hue distribution in the editing target region by calculating the appearance frequency of the hue of each pixel in the editing target region for each divided region of the hue circle. Then, based on the calculated hue distribution, the control device 103 extracts a hue having the highest appearance frequency in the editing target area as a characteristic color in the editing target area. Further, the pixel for which the characteristic color is calculated in the editing target area is specified as the editing target pixel.

  Next, the control device 103 generates editing data to be used for erasing unnecessary subjects included in the editing target area based on the colors other than the above-described characteristic colors in the editing target area. Specifically, the control device 103 identifies the hue with the second highest appearance frequency in the editing target area, and edits the pixel for which the hue with the second highest appearance frequency is calculated in the editing target area. It is specified as a pixel. Then, the control device 103 obtains each pixel value of the identified editing pixel, and calculates an average value of the pixel values as editing data. Then, the control device 103 deletes an unnecessary subject included in the editing target area by replacing the pixel value of the editing target pixel specified in the editing target area with the editing data.

  In this way, by calculating the hue with the highest appearance frequency in the edit target area set by the user as the feature color and specifying the pixel for which the feature color is calculated as the pixel to be edited, the user can Considering that there is a high possibility that the edit target area is set so as to largely include the subject to be erased, the user can specify the subject to be erased with high accuracy. Further, the pixel having the second highest appearance hue in the editing target area is specified as the editing pixel, and the pixel value of the editing target pixel is determined using the average value of the pixel values as the editing data. By substituting, it becomes possible to erase the subject that the user wants to erase with high accuracy using the peripheral pixels of the pixel to be edited.

  FIG. 3 is a flowchart showing the flow of image correction processing in the present embodiment. The processing shown in FIG. 3 is executed by the control device 103 as a program that is activated when the user instructs execution of the image correction processing application.

  In step S10, the control device 103 determines whether or not the repair brush tool is selected as the image editing tool by the user, and the repair brush tool is operated to specify the editing target area in the image. When an affirmative determination is made in step S10, the process proceeds to step S20, and the control device 103 extracts an image in the editing target area from the image, and proceeds to step S30. In step S30, the control device 103 converts the color space of the image in the editing target area extracted in step S20, for example, the RGB space into the LCH space, and proceeds to step S40.

  In step S40, as described above, the control device 103 calculates the hue of each pixel for the image in the editing target area after conversion to the LCH space. Then, the control device 103 divides the hue circle into a plurality of regions, and calculates the appearance frequency of the hue of each pixel in the editing target region for each divided region of the hue ring, thereby Calculate the hue distribution. Further, based on the calculated hue distribution, the control device 103 extracts a hue having the highest appearance frequency in the edit target area as a characteristic color in the edit target area. In addition, the control device 103 identifies the pixel whose characteristic color is calculated in the editing target area as the editing target pixel.

  Thereafter, the process proceeds to step S50, and as described above, the control device 103 identifies the hue having the second highest appearance frequency in the editing target area, and calculates the hue having the second highest appearance frequency in the editing target area. The designated pixel is specified as a pixel for editing. Then, the control device 103 obtains each pixel value of the identified editing pixel, and calculates an average value of the pixel values as editing data. Thereafter, the process proceeds to step S60.

  In step S60, the control device 103 deletes an unnecessary subject included in the editing target area by replacing the pixel value of the editing target pixel specified in step S40 with the editing data calculated in step S50. . Thereafter, the process ends.

According to the present embodiment described above, the following operational effects can be obtained.
(1) The control device 103 extracts a feature color from the image in the edit target area set in the image by the user, and is unnecessary in the edit target area based on the color other than the extracted feature color. Editing data used for erasing a subject is generated, and an unnecessary subject included in the editing target area is erased using the generated editing data. Thus, taking into account that the characteristic color in the editing target area is likely to be the color of the subject that the user wants to erase, the subject that the user wants to erase can be identified and erased with high accuracy.

(2) The control device 103 calculates the appearance frequency of the hue of each pixel in the editing target area, and extracts the hue having the highest appearance frequency in the editing target area as a characteristic color in the editing target area. As a result, the user is highly likely to set the edit target area so as to largely include the subject to be deleted, so by extracting the hue with the highest appearance frequency as a characteristic color and specifying the pixel to be edited, The subject that the user wants to erase can be specified with high accuracy.

(3) The control device 103 identifies the pixel for which the hue having the second highest appearance frequency in the editing target area is calculated as an editing pixel, and edits the average value of the pixel values of the identified editing pixel. It was calculated as data. Then, the control device 103 deletes an unnecessary subject included in the editing target area by replacing the pixel value of the editing target pixel specified in the editing target area with the editing data. As a result, it is possible to erase the subject that the user wants to erase with high accuracy using the peripheral pixels of the pixel to be edited.

-Modification-
The image processing apparatus according to the above-described embodiment can be modified as follows.
(1) In the above-described embodiment, the control device 103 extracts a hue having the highest appearance frequency in the editing target area as a characteristic color in the editing target area, and the pixel in which the characteristic color is calculated in the editing target area. Is specified as a pixel to be edited. In addition, the control device 103 identifies the pixel for which the hue with the second highest appearance frequency is calculated in the editing target area as the editing pixel, and determines the average value of the pixel values of the identified editing pixel as the editing data. It was made to calculate as. However, the control device 103 extracts a hue having the second highest appearance frequency in the editing target area as a characteristic color in the editing target area, and uses the pixel in which the characteristic color is calculated in the editing target area as the editing target pixel. The pixel for which the hue having the highest appearance frequency is calculated in the editing target area is specified as the editing pixel, and the average value of the pixel values of the specified editing pixel is calculated as the editing data. It may be. As a result, even when the editing target area is set so as to include more background areas than the area of the subject that the user wants to erase, the subject that the user wants to erase is specified with high accuracy, and the subject is detected using the pixels in the background area. It can be erased.

(2) Moreover, you may combine embodiment mentioned above and the said modification (1). That is, the control device 103 performs the image correction process in the above-described embodiment to delete the subject and the image after the image correction process in the modification (1) to delete the subject. The images displayed side by side on the monitor 105 may be used as the final edited image.

(3) In the above-described embodiment, the example has been described in which the control device 103 extracts the hue having the highest appearance frequency in the editing target area as the characteristic color in the editing target area. However, the control device 103 may specify a color that exists in the central portion of the editing target area based on the color information of the image in the editing target area, and specify this as a characteristic color in the editing target area. Good. Note that the color existing in the central portion of the editing target area is, for example, the central portion of the set editing target area when the set editing target area is a circle or a rectangle whose aspect ratio is not significantly different. Or a color component of a part in the vicinity thereof. Further, when the set edit target area is a rectangle having a greatly different aspect ratio, it means a color component distributed in the center of the set edit target area in the short side direction and in the entire long side direction.

  In this case, the control device 103 replaces the pixel value of the pixel whose characteristic color is calculated in the editing target area in the image with the pixel value of the pixel other than the characteristic color in the editing target area. Thus, an unnecessary subject included in the editing target area may be deleted. This allows the user to specify the subject to be erased with high accuracy and to erase it, considering that the subject to be erased is likely to designate the editing target area so that the subject is located in the center. It becomes possible.

(4) Alternatively, the control device 103 may specify the color that occupies the largest area in the editing target region as a characteristic color. This allows the user to specify the subject to be erased with high accuracy and erase it, taking into account that it is highly likely that the edit target area will be set to include the subject to be erased. It becomes.

(5) Alternatively, the control device 103 presents a plurality of colors included in the edit target area to the user, and specifies the color selected by the user as a characteristic color. Good. As a result, the user can directly specify the color of the subject to be erased, so that the user can specify the subject to be erased with high accuracy and erase it.

(6) In the above-described embodiment, an example in which the control device 103 extracts an image in the editing target area and performs image correction processing on the extracted image in the editing target area has been described. However, the control device 103 may divide the editing target area into a plurality of divided areas and perform image correction processing for each divided area. Thereby, for example, even when the area of the edit target region set by the user is large, the image correction processing can be performed for each divided region, and the processing accuracy can be improved.

(7) In the above-described embodiment, the user has described an example in which the editing target area is specified using a rectangular tool or a repair brush tool which is one of editing tools included in the image processing application. However, the user may designate the editing target area by other methods. For example, it may be specified by drawing a region to be edited on an image freehand using a drawing tool. Alternatively, when the user uses the mouse to click on a point on the subject to be erased in the image, the control device 103 displays a pixel that is continuous with the pixel at the clicked position and has a similar color. An existing area may be automatically recognized, and a recognized area or an area including the recognized area and its surrounding pixels may be set as an editing target area.

(8) In the embodiment and the modification described above, the control device 103 extracts an image in the edit target area designated by the user, and performs image correction processing on the image in the edit target area. An example was described. However, the control device 103 extracts, instead of the images in the editing target area, images in the peripheral area of the editing target area or images in the editing target area and the peripheral area of the editing target area, and these images are extracted. Image correction processing may be performed on the subject.

(9) In the above-described embodiment, the example in which the personal computer 100 is used as the image processing apparatus has been described. However, the present invention is not limited to this, and the present invention can be applied to other devices such as a camera.

  Note that the present invention is not limited to the configurations in the above-described embodiments as long as the characteristic functions of the present invention are not impaired. Moreover, it is good also as a structure which combined the above-mentioned embodiment and a some modification.

100 PC, 101 operation member, 102 connection IF, 103 control device, 104 HDD, 105 monitor

Claims (7)

An area setting procedure for setting an area to be edited in an image,
An extraction procedure for calculating the frequency of hue in the image of the edit target region and extracting the color having the highest hue frequency as a characteristic color of an unnecessary subject;
A generation procedure for generating a pixel value of a pixel of the color having the second highest hue frequency as editing data;
An image processing procedure for replacing pixel values of the characteristic color pixels in the editing target area using the editing data;
An image processing program for causing a computer to execute. An area setting procedure for setting an edit target area including an object area and a background area in the image;
The frequency of the hue in the image of the edit target area is calculated, and when the background area is included in the edit target area more than the subject area, the color of the subject that does not require the second highest color of the hue An extraction procedure for extracting the color having the highest hue frequency as the feature color when the background area is larger than the subject area and not included in the edit target area;
When the background region is included in the editing target region more than the subject region, the pixel value of the pixel having the highest hue frequency is generated as editing data, and the background region is more than the subject region. A generation procedure for generating a pixel value of a pixel of a color having the second highest frequency of hue as the editing data when not included in the editing target region;
An image processing procedure for replacing pixel values of the characteristic color pixels in the editing target area using the editing data;
An image processing program for causing a computer to execute. The image processing program according to claim 1 or 2,
An image processing program characterized in that the extraction procedure extracts a color existing in a central portion of an image of the edit target area as the characteristic color. The image processing program according to claim 1 or 2,
The extraction procedure presents a plurality of colors included in an image of the editing target area to a user, and specifies a color selected by the user as the characteristic color. In the image processing program according to claim 3 or 4,
The generation procedure generates pixel values of pixels other than the characteristic color in the image of the editing target area as the editing data,
In the image processing procedure, the editing data is used to replace pixel values of pixels of the characteristic color in the image of the editing target area. In the image processing program according to any one of claims 1 to 5,
The extraction procedure divides the image of the editing target area into a plurality of divided areas, extracts the characteristic colors for each of the divided areas,
The image processing program characterized in that the generation procedure generates the editing data for each of the divided areas. An image processing apparatus comprising execution means for executing the image processing program according to claim 1.

Images