JP5338248B2 - Image processing apparatus, electronic camera, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus and method, an electronic camera, and an image processing program, for applying image processing to a target image in image data, which reliably apply image processing to a target image without designation of a control point of image processing. <P>SOLUTION: The image processing apparatus includes a detection means for detecting a target image in image data, and a processing means for applying image processing whose control point is the center of the target image detected by the detection means. The image processing may be filtering whose control point is the center of the target image. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

The present invention relates to an image processing apparatus , an electronic camera, and an image processing program that perform image processing on a target image in image data.

2. Description of the Related Art Conventionally, a technique for performing deformation processing such as fisheye processing on a target image in image data is known.
Japanese Patent No. 4063306 JP 2005-25703 A

  However, in the conventional technique, since deformation processing such as fisheye processing is performed with the center of the image data as a reference, if the target image does not exist at the center of the image data, the image is unnaturally deformed. There was a problem. Therefore, there is a technique for designating a reference point for deformation processing and performing deformation processing around the reference point. In this case, however, there is a problem that it is necessary to designate a reference point for each image data, and operation is troublesome. .

The present invention has been made to solve such a conventional problem, and provides an image processing apparatus , an electronic camera, and an image processing program capable of reliably processing an object image without designating a reference point for image processing. The purpose is to provide.

According to a first aspect of the present invention, there is provided an image processing apparatus that detects a size and a position of an object included in an image, and corresponds to the object corresponding to the size of the object detected by the detection means. Setting means for setting the intensity of image processing for performing fisheye processing to be changeable, and processing means for performing the image processing with the center of the object detected by the detection means as a reference point. The detection means sets a detection frame surrounding the object, the setting means uses, as the intensity, a parameter set so that the intensity increases according to the size of the detection frame, and the processing means The processing means performs the image processing based on the intensity set by the setting means.

An image processing apparatus according to a second aspect of the present invention is the image processing apparatus according to the first aspect, wherein a display means for displaying an image including the object subjected to the image processing, and an image displayed by the display means. And a determination unit that determines whether or not to change the intensity according to the operation unit. The processing means repeatedly performs the image processing based on the intensity set by the setting means according to the operation means until the determination means determines that the intensity is not changed. .

According to a third aspect of the present invention, in the image processing apparatus according to the second aspect , in the case of an object photographed with a fisheye lens, the setting means sets the intensity to a minimum value according to the operation means. The intensity is set so as not to perform the fisheye processing, and the processing means performs image processing for contracting the object photographed with the fisheye lens to return to the original state from the image processing for performing the fisheye processing. Features.

An image processing program according to a fourth aspect of the invention is an image processing program for realizing image processing for performing fisheye processing on an object included in an image by a computer, and a detecting step for detecting the size and position of the object. A setting step for setting the intensity of the image processing for applying the fisheye processing to the object in accordance with the size of the object detected in the detection step, and a detection step detected by the detection step. And a processing step of performing the image processing using the center of the object as a reference point, wherein the detection step sets a detection frame surrounding the object, and the setting step sets the detection frame as the intensity. Using the parameters set so that the intensity increases according to the size of the process, the processing step based on the intensity set in the setting step And performing image processing.

According to a fifth aspect of the present invention, there is provided an electronic camera including the image processing apparatus according to any one of the first to third aspects.

  In the present invention, it is possible to reliably perform image processing on a target image without designating a reference point for image processing.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an embodiment of an electronic camera of the present invention.

  The electronic camera includes an imaging optical system 11, a lens driving unit 13, an AF sensor 15, an AE sensor 17, an image sensor 19, an analog signal processing unit 21, a timing generator (TG) 23, a digital signal processing unit 25, an image processing unit 27, It has a buffer memory 29, a recording I / F 31, a CPU 33, a flash memory 35, a monitor drive unit 37, a monitor 39, an operation unit 41, and a bus 43. The digital signal processing unit 25, the image processing unit 27, the buffer memory 29, the recording I / F 31, the CPU 33, the flash memory 35, and the monitor driving unit 37 are connected via a bus 43. Further, the AF sensor 15, the AE sensor 17, the TG 23, and the operation unit 41 are each connected to the CPU 33.

  The photographing optical system 11 includes a plurality of lens groups including a zoom lens 11a and a focusing lens 11b for adjusting a focus position. Each lens constituting the photographing optical system 11 is driven and adjusted in the optical axis direction by the lens driving unit 13.

  The AF sensor 15 detects the focus detection state of the photographing optical system 11 and outputs it to the CPU 33. The AE sensor 17 measures the luminance of the subject and outputs it to the CPU 33.

  The image sensor 19 photoelectrically converts the light beam that has passed through the photographing optical system 11 to generate an analog image signal of the subject image. The image sensor 19 captures the main image at the time of release. The image sensor 19 exposes the subject at predetermined intervals even when it is not released (during shooting standby), and outputs an analog image signal (through image signal) by thinning readout. The through image data is used for image display on the monitor 39 and the like.

  The analog signal processing unit 21 is an analog front end circuit (AFE) that performs analog signal processing on the output of the image sensor 19. The analog signal processing unit 21 performs correlated double sampling, image signal gain adjustment, and A / D conversion of the image signal. The output of the analog signal processing unit 21 is connected to the digital signal processing unit 25.

  The TG 23 supplies timing pulses to the image sensor 19 and the analog signal processing unit 21 based on instructions from the CPU 33. The drive timing of the image sensor 19 and the analog signal processing unit 21 is controlled by the timing pulse of the TG 23.

  The digital signal processing unit 25 is a digital front end circuit (DFE) that performs signal processing such as defective pixel correction and black level offset correction on the digital image signals sequentially input from the analog signal processing unit 21. In the electronic camera using the image sensor 19 that performs multi-channel output, the digital signal processing unit 25 rearranges the image signals output from each channel.

  The image processing unit 27 performs various types of image processing (color interpolation processing, gradation conversion processing, contour enhancement processing, white balance adjustment, etc.) on the digital image signal for one frame. Further, the image processing unit 27 generates a display image (view image) of the monitor 39 from the data of the through image according to an instruction from the CPU 33. Furthermore, the image processing unit 27 also performs processing for compressing the data of the main image in the JPEG format and processing for decompressing and restoring the compressed captured image data.

  The buffer memory 29 temporarily records the data of the main image in the pre-process and post-process of various image processes.

  A connector for connecting the recording medium 45 is formed in the recording I / F 31. The recording I / F 31 executes data writing / reading with respect to the recording medium 45 connected to the connector. The recording medium 45 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the recording medium 45.

  The CPU 33 is a processor that performs overall control of the electronic camera. The CPU 33 controls the operation of each part of the electronic camera according to a predetermined sequence program, and executes various calculations (calculation of the defocus amount at the time of AF, AE calculation, etc.) necessary for shooting. Further, the CPU 33 functions as a face detection unit. The face detection unit extracts feature points from the through image data or the captured image data, and detects a face area, a face size, and the like of the subject. For example, the face detection unit extracts a face region by a feature point extraction process described in Japanese Patent Laid-Open No. 2001-16573. Examples of the feature points include eyebrow, eye, nose, and lip end points, face contour points, head apexes, and chin lower end points.

  The flash memory 35 stores image data after image processing by the image processing unit 27. The flash memory 35 stores a known program for detecting a face, a known program for performing fisheye processing, and the like. In addition, intensity parameters to be described later are stored.

  The monitor drive unit 37 controls the display on the monitor 39 by outputting a drive signal based on an instruction from the CPU 33. The monitor 39 is composed of a liquid crystal monitor, for example. On the monitor 39, a view image based on the through image can be displayed. The monitor 39 can display a menu screen on which various setting items can be input in a GUI (Graphical User Interface) format.

  On the menu screen of the monitor 39, the user can select a playback mode. By selecting the reproduction mode, a plurality of image data stored in the flash memory 35 is displayed on the monitor 39 in a reduced state. When the user selects one image data, only that image data is displayed on the monitor 39. In this selected state, the post-processing menu can be selected. In the post-processing menu, items such as “resizing”, “trimming”, “tone correction”, and “special effect processing” are set. In the “special effect processing”, filter processing items such as “fisheye”, “swirl”, and “mosaic image” are set.

  The operation unit 41 includes a release button 47 and an operation button 49. The release button 47 receives an instruction input such as AF by a half-press operation and an input of a release instruction (exposure start instruction) by a full-press operation. The operation button 49 includes, for example, a command dial, a cross-shaped cursor key, a determination button, and the like. The operation button 49 accepts an electronic camera mode switching operation, an input operation on a menu screen, and the like from the user.

  FIG. 2 is a flowchart for explaining the operation when the fisheye for special effect processing is selected in the post-processing menu of the electronic camera described above.

  Step S1: The CPU 33 detects the object O from the image G displayed on the monitor 39 as shown in FIG. In this embodiment, a human face is set on the object O. The CPU 33 detects the face according to a known program stored in the flash memory 35 for detecting the face.

  Step S2: The CPU 33 determines whether or not the object O exists. As shown in FIG. 3, when a face exists in the image G, it is determined that it exists. On the other hand, when a face cannot be detected in the image G, it is determined that it does not exist. When there are a plurality of faces that are the object O, the largest face is determined as the object O. When the object O exists, the process proceeds to step S3, and when it does not exist, the process proceeds to step S5.

  Step S3: When it is determined that the object O exists, the CPU 33 sets a quadrangular detection frame K surrounding the outer shape in the vicinity of the face as shown in FIG. The detection frame K is set so that the intersection point P of the diagonal line is the center of the face.

  Step S4: The CPU 33 calculates center coordinates (coordinates of point P) as reference coordinates for deformation processing. As shown in FIG. 4, when the position of the upper left corner of the image G is the origin (0, 0) of the X and Y coordinates, the X coordinate of the center of the detection frame K is (x1 + x2) / 2. The Y coordinate is (y1 + y2) / 2.

  Step S5: If it is determined in step S2 that the object O does not exist, the CPU 33 sets “no detection frame”.

Step S6: The CPU 33 reads the intensity parameter. An intensity parameter table for obtaining the intensity parameter is stored in the flash memory 35. As shown in FIG. 5, the strength parameter table has default values and user set values. The user set value can be changed by the user, and the user set value is used after the default value is changed once. In the table shown in FIG. 5, the intensity parameter is set up to 10 levels. The size of the detection frame K is a ratio to the overall size of the image G. For example, when the overall size of the image G is 1000 × 1000 and the size of the detected detection frame K is 100 × 100,
(100 × 100) / (1000 × 1000) = 0.01
It becomes. Therefore, the size of the detection frame K is 1% with respect to the entire image G, and the value of the intensity parameter is 1 which is less than 5%. In step S5, if “no detection frame” is set, the value of the intensity parameter is zero. When the value of the intensity parameter is 0, special effect processing is not performed.

  Step S7: The CPU 33 performs the fisheye process selected in the special effect process. FIG. 6 shows an image G ′ after the fish-eye processing of the face that is the object O shown in FIG. 4. The fisheye processing is performed using the center (P) of the detection frame K shown in FIG. 4 as the processing reference coordinates. Then, the image G including the face is deformed based on a known program stored in advance with the reference coordinates as the center. The deformation process is performed in conjunction with the ASIC of the image processing unit 27. The size of the deformation process of the image G including the face is set based on the value of the intensity parameter obtained in step S6. The default values of the intensity parameters shown in FIG. 5 are set stepwise so that the intensity increases according to the size of the detection frame K. By setting in this way, it is possible to prevent deterioration in image quality. That is, when the detection frame K is small (when the face is small), if the deformation process is more than necessary, the image quality deteriorates and it is difficult to see, but by setting the intensity to increase according to the size of the detection frame K, Degradation of image quality can be prevented.

  Step S8: The CPU 33 displays the image G ′ subjected to the fish-eye process on the monitor 39. At the same time, a question is displayed as to whether or not to change the processing intensity of the fish-eye processed image G ′. The user can select whether or not to change the intensity of the image G by operating the operation button 49.

  Step S9: The CPU 33 determines whether or not to change the processing intensity of the fish-eye processed image G ′. If there is an input from the user to change the intensity for the display in step S8, it is determined that the processing intensity of the image G 'is to be changed. On the other hand, if no input is made or no input is made within a predetermined time, it is determined that no change is made.

  Step S10: When determining that the processing intensity of the image G ′ is to be changed, the CPU 33 displays the current intensity parameter value on the monitor 39. At the same time, a request is displayed to input the value of the requested strength parameter. The user can input a requested strength parameter value by operating the operation button 49.

  Step S11: The CPU 33 changes the value of the intensity parameter. This change is performed when the strength parameter requested by the user is input, and the strength parameter is set to a required value. And until it determines with not changing intensity | strength in step S9, the process of step S7 to step S11 is repeated.

  Step S12: If it is determined in step S9 that the intensity is not changed, it is determined in step S11 whether or not the intensity parameter has been changed.

  Step S13: If the intensity parameter has been changed, the CPU 33 displays on the monitor 39 whether or not to change the value of the intensity parameter table to the value set last in step S11.

  Step S14: The CPU 33 determines whether or not to change the value of the intensity parameter in the intensity parameter table. If there is an input to change the display in step S13 from the user, it is determined that the value of the intensity parameter is to be changed. On the other hand, if no input is made or no input is made within a predetermined time, it is determined that no change is made.

  Step S15: When determining that the value of the intensity parameter is to be changed, the CPU 33 updates the value of the intensity parameter in the intensity parameter table. This update is performed by changing the user setting value as shown in FIG. For example, when the size of the detection frame K is 13%, the default value of the intensity parameter is 3, but when the value of the intensity parameter is changed to 5, the user setting value “10% or more and The value corresponding to “less than 15%” is overwritten with 5. Then, the overwritten strength parameter value is used from the next fisheye process. Thereby, a fish-eye process can be performed according to a user preference. In this embodiment, it is possible to set the value of the intensity parameter to 0 other than when the detection frame K is not present, and when the user sets 0, the fisheye process is performed depending on the size of the detection frame K. Whether or not can be set.

In the above-described electronic camera, the object O is detected from the image G, and the image processing is performed using the detected center P of the object O as a reference point. Can be reliably processed. Accordingly, it is possible to easily and reliably perform deformation processing such as fish-eye processing. In addition, since the center of the detected object O is used as a reference point, an image that is relatively easy to recognize and has a little uncomfortable feeling can be obtained.
(Supplementary items of the embodiment)
As mentioned above, although this invention was demonstrated by embodiment mentioned above, the technical scope of this invention is not limited to embodiment mentioned above, For example, the following forms may be sufficient.

  (1) In the embodiment described above, an example in which the present invention is applied to an electronic camera has been described. However, for example, the present invention can be widely applied to an image processing apparatus using a personal computer (PC) or the like, an image processing program for a personal computer, and the like. it can.

  (2) In the above-described embodiment, the example in which the fisheye process is not performed by setting the strength of the strength parameter to 0 when the object O is not detected in the image G has been described. Fish-eye processing may be performed using coordinates. In this case, the intensity parameter may be set to 5, for example.

  (3) In the above-described embodiment, the example in which the fish-eye process is performed as the special effect process has been described. However, the present invention can be widely applied to filter processes such as spirals and mosaic images.

  (4) In the above-described embodiment, an example in which the present invention is applied to a still image that has already been photographed has been described. However, the present invention can also be applied to a through image, a moving image, and the like.

  (5) In the above-described embodiment, the example in which the largest face is determined as the object O when there are a plurality of faces that are the object O has been described. A specific face may be determined as an object.

  (6) In the above-described embodiment, an example in which the object O is a face has been described, but it may be a person, an animal, a plant, or the like. Further, for example, it may be a nose, eyes, ears, etc., which are part of the face.

  (7) In the above-described embodiment, the example in which the object O is processed with the fisheye process centered on the reference coordinates and the center is not moved has been described. However, for example, trimming is performed by moving the center of the object to the center of the image. Also good.

  (8) In the above-described embodiment, an example in which the object O is fish-eye processed has been described. However, for example, an object photographed with a fish-eye lens may be contracted and returned to the original face.

It is explanatory drawing which shows one Embodiment of the electronic camera of this invention. It is explanatory drawing which shows operation | movement of the electronic camera of FIG. It is explanatory drawing which shows the image in which a face exists. It is explanatory drawing which shows a detection frame. It is explanatory drawing which shows an intensity | strength parameter table. It is explanatory drawing which shows the image which carried out the fish-eye process of the face of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Shooting optical system, 19 ... Image sensor, 33 ... CPU, G, G '... Image, O ... Object, P ... Center.

Claims (5)

Detection means for detecting the size and position of the object included in the image;
Setting means for setting the intensity of image processing for applying fisheye processing to the object in a changeable manner in accordance with the size of the object detected by the detection means;
Processing means for performing the image processing using a center of the object detected by the detection means as a reference point;
The detection means sets a detection frame surrounding the object,
The setting means uses, as the intensity, a parameter set so that the intensity increases according to the size of the detection frame,
The image processing apparatus, wherein the processing means performs the image processing based on the intensity set by the setting means. The image processing apparatus according to claim 1.
Display means for displaying an image including the object subjected to the image processing;
Operating means for accepting the presence or absence of the intensity change based on the image displayed by the display means;
Determination means for determining whether to change the intensity according to the operation means,
The processing means repeatedly performs the image processing based on the intensity set by the setting means according to the operation means until the determination means determines that the intensity is not changed. Image processing device. The image processing apparatus according to claim 2 ,
In the case of an object photographed with a fisheye lens, the setting means sets the intensity not to perform the fisheye processing by minimizing the intensity according to the operation means,
The image processing apparatus according to claim 1, wherein the processing means performs image processing for shrinking an object photographed by the fisheye lens and returning it to an original state from image processing for performing the fisheye processing. In an image processing program for realizing, by a computer, image processing for performing fisheye processing on an object included in an image,
A detecting step for detecting the size and position of the object;
A setting step for setting the intensity of the image processing for applying the fisheye processing to the object in a changeable manner corresponding to the size of the object detected in the detection step;
A processing step of performing the image processing using a center of the object detected in the detection step as a reference point,
The detection step sets a detection frame surrounding the object,
The setting step uses, as the intensity, a parameter set so that the intensity increases according to the size of the detection frame,
The image processing program characterized in that the processing step performs the image processing based on the intensity set in the setting step. An electronic camera comprising the image processing apparatus according to any one of claims 1 to 3.

Images