JP5459072B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program, and in particular, when a plurality of images are displayed in a list on one screen, the user can easily grasp the contents and overall image of each image processing. It relates to technology that makes it possible.

  In recent years, digital photo frames (hereinafter simply referred to as “photo frames”) that display images for viewing have appeared. That is, the user can appreciate the desired image by performing an operation of selecting the desired image from a large number of images using the photo frame.

In many cases, a photo frame has a function of reducing a plurality of images that are candidates for display to display a list in order to support such an image selection operation.
In this way, an image having a fixed resolution (size) including a reduced image obtained by reducing the original image evenly in the vertical and horizontal directions for list display is hereinafter referred to as a “thumbnail image”. In addition, the original image of the reduced image included in the thumbnail image is hereinafter referred to as “original image”.
Note that the reduced image itself is not necessarily a thumbnail image. When the screen aspect ratio of the original image is different from that of the thumbnail image, an image in which a margin area is added to the reduced image having the same screen aspect ratio as the original image and the screen aspect ratio is adjusted is a thumbnail image. It is because it is used as.

FIG. 7 shows an example of a thumbnail image.
The photo frame can display a list of ten thumbnail images 511 to 520 in the screen 500. Here, the “screen” refers to an image displayed on the entire display area of a display unit such as a digital photo frame or a display unit included in the device.

  Some image data that are candidates for display of a photo frame have been subjected to various types of image processing such as gradation processing and image-tone conversion processing for conversion into a painting-like image. For this reason, when image processing is applied to image data that is a candidate for display target, there has recently been a demand for selecting image data to be displayed after grasping the contents of the image processing.

  For this reason, Patent Document 1 discloses a technique (hereinafter referred to as “Method of Patent Document 1”) in which character information and icons indicating a history of image processing and the like are superimposed on an image and displayed. By applying such a method of Patent Document 1, when only one image is displayed on the entire screen of the digital photo frame, the user can display character information or an icon superimposed on the one image. By visually recognizing, it is possible to easily grasp the contents of the image processing performed on the image data of the one image.

JP 2008-42473 A

However, even if the method of Patent Document 1 is applied, when a plurality of thumbnail images are displayed in a list on one screen of a photo frame, character information and icons superimposed on each of the plurality of thumbnail images are extremely Appears small. It is very difficult for the user to visually recognize such very small character information and icons.
In addition, it is very difficult for the user to determine a small difference existing in the original image by looking at the reduced image of the original image included in the thumbnail image.
For this reason, it is very difficult for a user to grasp the contents of image processing using thumbnail images.

  In order to alleviate the difficulty of determining small differences existing in the original image, the conventional method of recognizing the face included in the original image and using only the recognized face image as the thumbnail image is used. A method may be applied. However, when the conventional method is applied, it becomes difficult for the user to achieve the original purpose of selecting a display target because the user does not know the whole image of the original image.

  The present invention has been made in view of such a situation. When a plurality of images are displayed in a list on one screen, the content of each image processing (including the content that the image processing has not been performed) and The purpose is to allow the user to easily grasp the whole picture.

According to one aspect of the invention,
Determining means for determining a processing target from image data of thumbnail images in which a blank area is added to a reduced image of the original image;
From the image data of the original image corresponding to the processing target determined by the determining means, the image includes at least a part of the feature region of the original image, and the image data of the image having the same resolution as the thumbnail image, Feature image extraction means for extracting as feature image data;
By performing a synthesis process for synthesizing the feature image and the thumbnail image on the data of the thumbnail image determined as a processing target by the determination unit and the image data of the feature image extracted by the feature image extraction unit Image synthesizing means for generating image data of a synthesized thumbnail image in which the feature image is superimposed on the margin area of the thumbnail image;
An image processing apparatus is provided.

  According to another aspect of the present invention, an image processing method and an image processing program corresponding to the image processing apparatus according to one aspect of the present invention described above are provided.

  According to the present invention, when a plurality of images are displayed in a list on one screen, the user can easily grasp the contents of each image processing (including the content that the image processing is not performed) and the whole image. Can do.

It is a block diagram which shows the structure of the hardware of the photo frame of 1st Embodiment. It is a functional block diagram which shows the functional structure of the data processing part of the photo frame of FIG. It is a figure explaining the outline | summary of a process of the image synthetic | combination part among the data processing parts of FIG. It is a figure which shows an example of the list image displayed when the data processing part of FIG. 2 performed the thumbnail display process. 3 is a flowchart illustrating an example of a flow of thumbnail display processing executed by a data processing unit in FIG. 2. It is a figure which shows an example of the list image displayed when the data processing part of the photo frame of 2nd Embodiment performed the thumbnail display process. It is a figure which shows an example of the some thumbnail image of a prior art example.

[First Embodiment]
A photo frame 1 according to a first embodiment of an image processing apparatus to which the present invention is applied will be described with reference to FIGS.

FIG. 1 is a block diagram showing an internal hardware configuration of the photo frame 1.
As shown in FIG. 1, the photo frame 1 includes a data processing unit 11 and a user interface unit 12.

  The data processing unit 11 includes a CPU (Central Processing Unit) 31, a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 33, a memory 34, a display control unit 35, and an image processing unit 36. I have.

The CPU 31 executes various processes according to programs recorded in the ROM 32.
That is, the ROM 32 stores programs for realizing various functions such as a function for controlling the display control unit 35, the image processing unit 36, the communication unit 43, and the drive 44. Therefore, the CPU 31 can control the display control unit 35, the image processing unit 36, the communication unit 43, and the drive 44 by executing processing according to a program stored in the ROM 32, and can realize various functions.
The RAM 33 appropriately stores data necessary for the CPU 31 to execute various processes.

  The memory 34 is constituted by a DRAM (Dynamic Random Access Memory) or the like, and temporarily stores image data input to the data processing unit 11 and image data subjected to image processing by the image processing unit 36. The memory 34 also appropriately stores data necessary for various image processing.

  The display control unit 35 reads image data stored in the memory 34 and executes control for causing the display unit 41 to display an image represented by the image data. For example, the display control unit 35 generates an RGB signal based on the image data and supplies the RGB signal to the display unit 41 to display the RGB signal on the display unit 41.

  The image processing unit 36 is configured by a DSP (Digital Signal Processor) or the like, and performs various image processing such as white balance correction processing and γ correction processing on the image data stored in the memory 34.

The user interface unit 12 includes a display unit 41, an operation unit 42, a communication unit 43, and a drive 44.
The display unit 41 is configured by a display or the like provided in the housing of the photo frame 1 and displays various images under the control of the display control unit 35 described above. A display example of the display unit 41 will be described later with reference to FIG.
The operation unit 42 receives a user instruction operation and notifies the CPU 31 of it. The CPU 31 executes various processes according to the notified instruction operation.
The communication unit 20 communicates with other devices under the control of the CPU 31. In this case, the communication unit 20 appropriately connects to a network including the Internet.
The drive 44 reads and writes data from and to the removable recording medium 51 under the control of the CPU 31.

The removable recording medium 51 is composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, and records various data.
For example, the removable recording medium 51 records image data in JPEG (Joint Photographic Experts Group) format in a file in Exif (Exchangeable Image File Format) format.
A file including such image data is hereinafter referred to as an “image file”. The image file can include various meta information related to the image data in addition to the image data. In the present embodiment, meta information indicating the content of image processing performed on the image data is included in the image file as image information.

The data processing unit 11 including the CPU 31 generates image data such as a plurality of thumbnail images when the operation unit 42 is instructed to display a list of thumbnail images and the like, and displays a list of the plurality of thumbnail images and the like. A series of processing until it is displayed on is executed. Such a series of processing is hereinafter referred to as “thumbnail display processing”.
Note that “etc.” with a plurality of thumbnail images and the like is intended to include a composite thumbnail image described later.

  FIG. 2 is a functional block diagram showing a functional configuration of the data processing unit 11 for executing such thumbnail display processing.

  The data processing unit 11 includes an image acquisition unit 61, an image storage unit 62, a composite thumbnail image generation unit 63, and a list image display control unit 64.

The image acquisition unit 61 is configured by, for example, a combination of the CPU 31 in FIG. 1 and a program executed by the CPU 31.
The image acquisition unit 61 acquires an image file supplied from the removable recording medium 51 via the drive 44 in FIG. 1 and an image file supplied from another device (not shown) via the communication unit 43.
The image acquisition unit 61 generates image data of a thumbnail image from the image data included in the image file, and stores it in the thumbnail image storage unit 71 of the image storage unit 62.
As described above, the image data of the thumbnail image is generated based on the image data included in the image file acquired by the image acquisition unit 61. Therefore, the image data included in the image file acquired by the image acquisition unit 61 becomes the image data of the original image. Therefore, the image acquisition unit 61 stores an image file including the image data of the original image in the original image storage unit 72 of the image storage unit 62.

The image storage unit 62 is configured by, for example, the memory 34 of FIG. The image storage unit 62 stores the image data generated or used in the thumbnail display process, the thumbnail image storage unit 71, the original image storage unit 72, the feature image storage unit 73, and the composite thumbnail image storage unit 74. It is equipped with.
The thumbnail image storage unit 71 stores the image data of the thumbnail image generated by the image acquisition unit 61 as described above.
As described above, the original image storage unit 72 stores the image file acquired by the image acquisition unit 61, that is, the image file including the image data of the original image.
The feature image storage unit 73 and the composite thumbnail image storage unit 74 will be described later.

The composite thumbnail image generation unit 63 is configured by, for example, a combination of the CPU 31 and the image processing unit 36 illustrated in FIG. 1 and a program executed by the CPU 31.
The composite thumbnail image generation unit 63 generates image data of an image obtained by combining at least a part of the feature area of the original image with the blank area of the thumbnail image (hereinafter referred to as “composite thumbnail image”).
The combined thumbnail image generating unit 63 includes a combining target determining unit 81, a feature image extracting unit 82, and an image combining unit 83 in order to generate such image data of the combined thumbnail image.

The compositing target determining unit 81 determines a target (hereinafter referred to as “compositing target”) for generating image data of the composite thumbnail image from the image data of the thumbnail image stored in the thumbnail image storage unit 71. The method for determining the composition target will be described later.
The compositing target determination unit 81 reads out the image data of the thumbnail image determined as the compositing target from the thumbnail image storage unit 71 and supplies it to the image composition unit 83, and supplies the image data of the other thumbnail images to the thumbnail image storage unit 71. Is read from the image data and supplied to the list image display control unit 64.
That is, among the image data of the thumbnail images stored in the thumbnail image storage unit 71, the compositing target is supplied to the image compositing unit 83 and converted into the image data of the composite thumbnail image, as will be described later, and the list image display control is performed. Supplied to the unit 64. On the other hand, the image data of the thumbnail images stored in the thumbnail image storage unit 71 other than the synthesis target are supplied to the list image display control unit 64 as the image data of the thumbnail images.
In addition, the synthesis target determined by the synthesis target determination unit 81 is notified to the feature image extraction unit 82.

The feature image extraction unit 82 recognizes the original image corresponding to the notified composition target, reads the image data of the original image from the original image storage unit 72, and detects a feature region from the original image.
Then, the feature image extraction unit 82 is an image including at least a part of the feature region of the original image, and has the same resolution (same size) as the thumbnail image (hereinafter referred to as “feature image”). Are extracted from the image data of the original image.
The image data of the extracted feature image is stored in the feature image storage unit 73.

  Note that the type of feature region detected from the original image and the method for detecting the feature region are not particularly limited. In the present embodiment, a human face image area (hereinafter referred to as a “face area”) is detected as a feature area. The face area detection method will be described later.

The image synthesis unit 83 is supplied with the image data of the thumbnail image determined as the synthesis target by the synthesis target determination unit 81 by being read from the thumbnail image storage unit 71 and the image of the feature image corresponding to the synthesis target. Data is read from the feature image storage unit 73 and supplied.
Then, the image synthesis unit 83 performs a synthesis process for synthesizing the feature image and the thumbnail image on the feature image and the thumbnail image, using the feature image as the background, the thumbnail image as the foreground, and the foreground on the background. Thereby, the image data of the composite thumbnail image is generated.

FIG. 3 is a diagram for explaining the outline of the processing of the image composition unit 83.
The original image 101 is configured such that N pixels are arranged in the horizontal direction (horizontal direction in FIG. 3), and M pixels are arranged in the vertical direction (vertical direction in FIG. 3). Here, N and M are arbitrary integer values different from each other.
Here, if the image is expressed as “an image having the number of pixels in the horizontal direction × the number of pixels in the vertical direction”, the original image 101 is an N × M image. In this case, the screen aspect ratio of the original image 101 is N: M.

In the present embodiment, the image data of the thumbnail image 102 having the screen aspect ratio of 1: 1 (n: n when n described later is used) from the image data of the original image 101 having the screen aspect ratio of N: M. Is generated.
That is, when thumbnailing is performed, the image data of the original image 101 is reduced to 1 / L times (L is an arbitrary value larger than 1) without changing the screen aspect ratio. , N × m reduced image 111 image data is obtained. Here, n is an integer value that satisfies (N / L), and m is an integer value that satisfies (M / L). That is, the screen aspect ratio N: M of the original image 101 matches the screen aspect ratio n: m of the reduced image 111.
On the other hand, in the present embodiment, the screen aspect ratio of the thumbnail image is 1: 1. Therefore, the image data of the reduced image 111 having a screen aspect ratio of n: m (n and m are different from each other and are not 1: 1) cannot be directly used as the image data of the thumbnail image.
Therefore, in the present embodiment, the n × n image 102 in which the margin regions 112U and 112D are added vertically to the n × m reduced image 111 in parallel to the horizontal direction, that is, the screen aspect ratio is 1: 1. The image data of the image 102 adjusted to (= n: n) is adopted as the image data of the thumbnail image.

  The blank area including the blank areas 112U and 112D is an area composed of pixels other than the pixels of the reduced image of the original image in the thumbnail image. Accordingly, the pixel value of the pixel in the blank area is not particularly limited, but it is preferable that the pixel value is the same in order to clarify that it is the blank area at the stage of the thumbnail image. A value indicating is adopted. That is, in this embodiment, the blank area including the blank areas 112U and 112D is a white area at the stage of the thumbnail image.

  Further, since the original image 101 in the example of FIG. 3 is horizontally long, that is, the number of pixels N in the horizontal direction of the original image 101 is larger than the number of pixels M in the vertical direction (N> M), the blank area 112U and 112D are added above and below the reduced image 111 so as to be parallel to the horizontal direction. However, although not shown, in the case of a vertically long original image, that is, in the case of an original image having more pixels in the vertical direction than in the horizontal direction, the blank area is parallel to the vertical direction. Images added to the left and right of the reduced image become thumbnail images (see thumbnail image 211 in FIG. 4).

  In the present embodiment, the image data of the thumbnail image 102 is generated from the image data of the original image 101 in this way. Further, an n × n image 121 including at least a face area in the original image 101 is used as a feature image, and image data of the feature image is extracted from the image data of the original image 101.

Therefore, the image composition unit 83 performs composition processing for compositing the feature image 121 and the thumbnail image 102 on the feature image 121 and the thumbnail image 102, with the thumbnail image 102 as the foreground and the foreground on the background. Thereby, image data of the composite thumbnail image 103 is generated.
Specifically, the portion of the reduced image 111 of the original image 101 in the foreground thumbnail image 102 is synthesized on the background feature image 121 with a transmittance of 0% (non-transparent). On the other hand, the margin areas 112U and 112D in the foreground thumbnail image 102 are combined on the background feature image 121 with a transmittance of 100% (total transparency). As a result, the image data of the image 103 in which the feature image 121 is superimposed on the blank areas 112U and 112D of the thumbnail image 102 is obtained as the image data of the composite thumbnail image.
In summary, the image synthesis unit 83 performs a synthesis process for synthesizing the foreground feature image 121 with the background thumbnail image 102 on the image data of the feature image 121 and the thumbnail image 102, so that the margin area of the thumbnail image 102 is displayed. Image data of the composite thumbnail image 103 in which the feature image 121 is superimposed on 112U and 112D is generated.

  Returning to FIG. 2, the image data of the combined thumbnail image generated by the image combining unit 83 in this way is stored in the combined thumbnail image storage unit 74.

As described above, among the image data of the thumbnail images stored in the thumbnail image storage unit 71, the image data of the thumbnail image determined as the synthesis target by the synthesis target determination unit 81 is converted into the image data of the synthesized thumbnail image. And stored in the composite thumbnail image storage unit 74. Thereafter, the image data of the combined thumbnail image is read from the combined thumbnail image storage unit 74 and supplied to the list image display control unit 64.
On the other hand, among the image data of the thumbnail images stored in the thumbnail image storage unit 71, the image data of the thumbnail images that have not been determined as the synthesis target by the synthesis target determination unit 81 are read from the thumbnail image storage unit 71 as they are. , And supplied to the list image display control unit 64.

  The list image display control unit 64 is based on a plurality of image data supplied from the thumbnail image storage unit 71 or the composite thumbnail image storage unit 74, and is an image in which a plurality of thumbnail images and composite thumbnail images are arranged (hereinafter, “list image”). Is displayed as a screen of the display unit 41.

FIG. 4 shows an example of a list image displayed as a screen of the display unit 41 of the photo frame 1.
The list image 200 (screen 200) in the example of FIG. 4 includes two thumbnail images 211 and 212, and eight composite thumbnail images 221 to 228.
The central portion of each of the eight composite thumbnail images 221 to 228 includes a reduced image of the original image. The user can understand from these reduced images that some image processing has been performed on the image data of the original image, but since the small differences in the reduced images cannot be distinguished, the contents of the image processing It is very difficult to figure out.
However, at least a part of the characteristic area (the face area in the present embodiment) of the original image is included in the upper and lower parts of each of the eight composite thumbnail images 221 to 228, that is, the marginal area of the thumbnail image. Including feature images are superimposed.
Here, the reduced image of the central portion of each of the combined thumbnail images 221 to 228, that is, the reduced image of the central portion of the thumbnail image before combining includes the face area in the original image in a reduced manner. . Therefore, if the viewpoint is changed, it is possible to grasp that the enlarged image of the face area in the thumbnail image before composition is a feature image. From this understanding, it can be said that the combined thumbnail images 221 to 228 are images in which the enlarged image of the face area in the thumbnail image before the combination is superimposed on the blank area of the thumbnail image.
Therefore, the user can easily understand the contents of the image processing performed on the image data of the original image by looking at the feature image (enlarged image of the face area when viewed from the reduced image) among the composite thumbnail images 221 to 228. It becomes possible to grasp.
On the other hand, the user can easily grasp the entire image of the original image by looking at the reduced image at the center of the combined thumbnail images 221 to 228.
In summary, when a plurality of thumbnail images or composite thumbnail images are displayed in a list on one screen as in the list image 200 in the example of FIG. 4, the contents of each image processing (the image processing is not performed). And the whole image can be easily grasped by the user.

Here, as image processing that may be performed on the image data of the original image, for example, there is image tone conversion processing.
The image tone conversion process is a process for converting an image to be processed into an image having a predetermined image tone. Adopt various styles such as "Japanese style", "Western style", "Watercolor style", "Ink style", "Pen style", "Goch style", etc. Can do.
Specifically, for example, a process of updating the value of a predetermined parameter from a value of an image to be processed to a value that matches a predetermined image tone is an example of the image tone conversion process. The parameters used in this case include, for example, contrast, brightness, color density, hue, sharpness, noise reduction filter effect setting, color temperature, and γ correction γ value. These parameters may be used alone or in any combination of two or more. Note that these parameters are merely examples, and any parameters may be used as long as they are appropriate for expressing the tone conversion processing.
The image tone conversion processing is realized by, for example, an editing function predetermined for the device or application software such as Photoshop (registered trademark).

Further, as image processing that may be performed on the image data of the original image, for example, gradation processing exists.
Gradation processing is processing that changes the brightness of an image in stages.
Specifically, for example, it is realized by a process of changing the brightness of an image by changing the light amount stepwise at the time of imaging, an editing function predetermined for the device, or application software such as Photoshop (registered trademark). The process of converting the brightness of the image after imaging by the editing function is an example of the gradation process.

  As described above, the image tone conversion processing and the gradation processing have been described as image processing that may be performed on the image data of the original image, but these are merely examples. That is, there is a possibility that any kind of image processing is performed on the image data of the original image.

Here, as described above, when the user's purpose is to grasp the contents of the image processing performed on the image data of the original image from the composite thumbnail image included in the list image, the image processing is performed. There is no particular need to generate image data of a composite thumbnail image from thumbnail image data corresponding to an original image that has not been subjected to.
Therefore, in such a case, the compositing target determination unit 81 in FIG. 2 may determine only the image data of the thumbnail image corresponding to the original image that has undergone image processing as the compositing target.
Here, a method for determining whether or not image processing is performed on the image data of the original image is not particularly limited. For example, the database is a database regarding a plurality of original images, and the type of image processing performed is determined. When a stored database exists, a method using the database may be employed.
However, in the present embodiment, the following determination method is adopted. That is, in this embodiment, as described above, the image data of the original image is included in the image file and stored in the original image storage unit 72. This image file includes image information indicating the content of image processing performed on the image data of the original image as one piece of meta information. Therefore, the compositing target determining unit 81 determines whether image processing has been performed on the image data of the original image based on the content of the image information.

A flowchart for explaining the flow of the thumbnail display process to which such a determination method is applied is shown in FIG.
That is, FIG. 5 is a flowchart for explaining an example of the flow of thumbnail display processing executed by the data processing unit 11 having the functional configuration of FIG.

  The thumbnail display process is triggered by a user's instruction operation to display a list image (a list of thumbnail images or composite thumbnail images) using the operation unit 42 (FIG. 1). The trigger for starting the thumbnail display process is not particularly limited to this. For example, the thumbnail display process may be automatically started by using the input of image data to the data processing unit 11 as a trigger. Good.

In step S1, the image acquisition unit 61 acquires the image data of the original image.
That is, the image acquisition unit 61 acquires an image file including the image data of the original image as a display target candidate from the removable recording medium 51 via the drive 44 in FIG. Obtained from a device (not shown).
Specifically, for example, the image data of the original image 101 in FIG. 3 is acquired.

In step S <b> 2, the image acquisition unit 61 generates image data of a thumbnail image from the image data of the original image.
That is, the image acquisition unit 61 generates image data of a thumbnail image having a screen aspect ratio of 1: 1 by adding a blank area to the reduced image of the original image.
Specifically, for example, as illustrated in FIG. 3, image data of an image 102 in which blank areas 112U and 112D are added to the reduced image 111 of the original image 101 is generated as image data of a thumbnail image.

  When the image file including the image data of the original image is stored in the original image storage unit 72 and the image data of the thumbnail image is stored in the thumbnail image storage unit 71, the process proceeds to step S3.

In step S3, the compositing target determining unit 81 determines whether image processing has been performed on the image data of the original image.
That is, the compositing target determination unit 81 reads image information from an image file that includes the image data of the original image, and performs image processing on the image data of the original image based on the content of the image information. It is determined whether or not.

In the first place, when the image information does not exist in the image file or the image information read from the image file is information indicating that the image data of the original image has not been subjected to image processing, the image of the original image Assuming that the data is not a synthesis target, NO is determined in step S3, and the process proceeds to step S4.
In step S4, the list image display control unit 64 includes thumbnail images in the list image. That is, image data of thumbnail images is read from the thumbnail image storage unit 71, supplied to the list image display control unit 64, and held in the list image display control unit 64 for generating image data of the list image.
Thereby, a process progresses to step S9. However, the processing after step S9 will be described later.

  On the other hand, when the image information read from the image file is information indicating the content of predetermined image processing, that is, information indicating that image processing is performed on the image data of the original image, Assuming that the image data of the image is to be synthesized, it is determined as YES in Step S3, and the process proceeds to Step S5.

In step S5, the feature image extraction unit 82 reads the image data of the original image from the original image storage unit 72, and detects a face area from the original image.
That is, in the present embodiment, as described above, the face area is adopted as the feature area. Therefore, the feature image extraction unit 82 extracts one or more feature points related to the human face from the original image, and detects a face region from the extracted one or more feature points.
Specifically, for example, the feature image extraction unit 82 performs feature point extraction processing disclosed in Japanese Patent Laid-Open No. 2001-16573 on the image data of the original image, so that each of the eyebrows, eyes, nose, and lips An end point, a face contour point, a head vertex, a lower end point of a chin, and the like are detected as feature points related to a human face.
The feature image extraction unit 82 demarcates the eyebrows, eyes, nose, and lips based on the feature points thus detected, and determines the boundaries of the regions of the eyebrows, eyes, nose, and lips as contours. Thus, the face area is detected.
Note that the method for detecting a face area is not particularly limited to the above-described technique, and various other methods such as a technique for generating luminance image data from image data of an original image and detecting the face area using the luminance image data are available. Can be adopted.

In step S6, the feature image extraction unit 82 extracts image data of an image including at least a part of the face area detected from the original image as image data of the feature image from the image data of the original image.
Specifically, for example, as illustrated in FIG. 3, image data of a part of the image 121 of the face area of the original image 101 is extracted as image data of a feature image.
The image data of the feature image extracted in this way is stored in the feature image storage unit 73. Thereby, a process progresses to step S7.

  In step S7, the image composition unit 83 generates image data of a composite thumbnail image in which the feature image is superimposed on the blank area of the thumbnail image.

  That is, the image composition unit 83 is supplied with the image data of the thumbnail image generated by the process of step S2 and the image data of the feature image extracted by the process of step S6. Therefore, the image composition unit 83 performs composition processing for compositing the feature image and the thumbnail image on the feature image and the thumbnail image with the feature image as a background, the thumbnail image as a foreground, and the foreground on the background. Generate image data.

Specifically, for example, referring to FIG. 3 described above, the image data of the reduced image 111 portion of the original image 101 in the foreground thumbnail image 102 has a transmittance of 0% (non-transparent) and the background image. The image data of the feature image 121 is combined. On the other hand, the image data of the margin areas 112U and 112D in the foreground thumbnail image 102 is combined with the image data of the background feature image 121 with a transmittance of 100% (total transparency). As a result, the image data of the image 103 in which the feature image 121 is superimposed on the blank areas 112U and 112D of the thumbnail image 102 is generated as the image data of the composite thumbnail image.
More specifically, for example, in the present embodiment, the foreground image data including the thumbnail image 102 includes a data area (hereinafter referred to as “alpha channel”) for storing transparency information (hereinafter referred to as “alpha value”). ")" Is provided. In this case, the image composition unit 83 can set the transmittance in units of pixels by setting the alpha value in the alpha channel of each pixel constituting the foreground.
Therefore, the image composition unit 83 sets a low alpha value for each pixel constituting the reduced image 111 of the original image 101 in the foreground thumbnail image 102, thereby realizing a transmittance of 0% (non-transparent). To do. On the other hand, in the foreground thumbnail image 102, the image compositing unit 83 sets a high alpha value for each pixel constituting the margin areas 112U and 112D so that the transmittance is 100% (total transparency). Realize.
In this way, the image data of the combined thumbnail image generated by the image combining unit 83 is stored in the combined thumbnail image storage unit 74. Thereby, a process progresses to step S8.

  In step S8, the list image display control unit 64 includes the composite thumbnail image in the list image. That is, the image data of the combined thumbnail image is read from the combined thumbnail image storage unit 74, supplied to the list image display control unit 64, and held in the list image display control unit 64 for generating image data of the list image. Is done.

In this way, when the combined thumbnail image is included in the list image in the process of step S8, or when the thumbnail image is included in the list image in the process of step S4, the process proceeds to step S9.
In step S <b> 9, the list image display control unit 64 determines whether all thumbnail images or combined thumbnail images corresponding to the display target candidates are included in the list image.
If there is still an original image that is a candidate for display whose thumbnail image or composite thumbnail image is not included in the list image, it is determined NO in step S9, and the process proceeds to step S1. Is returned, and the subsequent processing is repeated.
That is, for each of a plurality of original images that are candidates for display, the loop processing of steps S1 to S9 is executed each time, so that the thumbnail images or synthesized thumbnail images are sequentially included in the list image.
If all thumbnail images or composite thumbnail images corresponding to display target candidates are included in the list image, it is determined as YES in step S9, and the process proceeds to step S10.

In step S <b> 10, the list image display control unit 64 displays images including all thumbnail images and composite thumbnail images on the display unit 41 as list images.
Specifically, for example, the list image 200 shown in FIG. 4 is displayed as the screen of the display unit 41.

As described above, in the first embodiment, image data of a thumbnail image in which a blank area is added to a reduced image of the original image is generated from the image data of the original image subjected to image processing. Furthermore, image data of a combined thumbnail image is generated by superimposing a feature image including at least a part of a feature region (a face region in the present embodiment) of the original image on a margin region of the thumbnail image. Then, a list image including the composite thumbnail image is displayed as a screen of the display unit 41.
Here, from the viewpoint of a thumbnail image, the feature image corresponds to an enlarged image of a feature area (a face area in the present embodiment) of the thumbnail image. Therefore, the user can easily confirm small differences in the original image by looking at the feature image of the composite thumbnail image, as compared with the case where only the thumbnail image is seen. It is possible to easily grasp the contents of the processed image.
On the other hand, the user can easily grasp the whole image of the original image by looking at the reduced image of the original image among the combined thumbnail images.
In summary, when a plurality of thumbnail images or composite thumbnail images are displayed as a list within one screen, such as a list image, the contents of each image process (including the contents that the image process has not been performed) and the whole The user can easily grasp the image.

  Further, in the composite thumbnail image, the place where the feature image is superimposed is a blank area that inevitably occurs due to the difference in the screen aspect ratio between the original image and the thumbnail image. Therefore, the visibility of the list image including a plurality of thumbnail images and composite thumbnail images, that is, the list property is not hindered, and a limited range (display area range) such as the screen of the display unit 41 is wasted. There is no consumption.

[Second Embodiment]
Next, a second embodiment of the image processing apparatus to which the present invention is applied will be described.

  As an image processing apparatus according to the second embodiment, the photo frame 1 having the hardware configuration shown in FIG. 1 can be employed as it is. In addition, the functional configuration for realizing the thumbnail display processing in the photo frame 1 of the second embodiment can be employed as it is. Therefore, description of the hardware configuration and functional configuration of the photo frame 1 of the second embodiment is omitted.

  Therefore, only the differences from the first embodiment of the photo frame 1 of the second embodiment will be described below.

  That is, in the first embodiment, it is assumed that the user's purpose is to grasp the contents of the image processing performed on the image data of the original image from the composite thumbnail image included in the list image as described above. It was done. For this reason, in the first embodiment, the compositing target determining unit 81 in FIG. 2 determines the image data of the thumbnail image corresponding to the original image subjected to the image processing as the compositing target. That is, in the first embodiment, the image data of the combined thumbnail image is generated only from the image data of the thumbnail image corresponding to the original image that has been subjected to image processing.

On the other hand, in the second embodiment, it is assumed that the user's purpose is to grasp the difference in imaging conditions between a plurality of original images taken with brackets from the synthesized thumbnail images included in the list image. . For this reason, in the second embodiment, the compositing target determining unit 81 determines the image data of the thumbnail image corresponding to the original image obtained by bracket shooting as the compositing target. That is, in the second embodiment, the image data of the composite thumbnail image is generated only from the image data of the thumbnail image corresponding to the original image obtained by bracket shooting.
Accordingly, although not shown, the thumbnail display process of the second embodiment is also a process according to the flowchart of FIG. 5 in principle. However, in the thumbnail display process of the second embodiment, only the process of step S3 is replaced with a process of “Is the original image obtained by bracket shooting?”.
Here, bracket shooting refers to shooting each time the setting value is changed by changing one or more setting values such as exposure value, zoom value, focus value, white balance value flash emission amount, etc., in multiple stages. To do. By bracket shooting, image data of an original image when each is shot with different setting values is obtained.

FIG. 6 shows an example of a list image displayed as a screen of the display unit 41 of the photo frame 1 of the second embodiment.
The list image 300 (screen 300) in the example of FIG. 6 includes ten composite thumbnail images 311 to 320.
The central portion of each of the ten composite thumbnail images 311 to 320 includes a reduced image of the original image. From these reduced images, the user can grasp that the image data of the original image is obtained by bracket shooting, but small differences in the reduced images cannot be distinguished, so the difference in imaging conditions can be determined. It is very difficult to grasp.
However, at least a part of the characteristic area (the face area in the present embodiment) of the original image is included in the upper and lower parts of each of the ten composite thumbnail images 311 to 320, that is, the marginal area of the thumbnail image. Including feature images are superimposed.
Here, the reduced image of the central portion of each of the combined thumbnail images 311 to 320, that is, the reduced image of the central portion of the thumbnail image before combining includes the face area in the original image in a reduced manner. . Therefore, if the viewpoint is changed, it is possible to grasp that the enlarged image of the face area in the thumbnail image before composition is a feature image. If grasped in this way, it can be said that the synthesized thumbnail images 311 to 320 are images in which the enlarged image of the face area in the thumbnail image before synthesis is superimposed on the blank area of the thumbnail image.
Therefore, the user compares the characteristic images (enlarged images of the face area when viewed from the reduced image) among the composite thumbnail images 311 to 320, and thus the difference in imaging conditions between the plurality of original images obtained by bracket shooting. Can be easily grasped.
On the other hand, the user can easily grasp the entire image of the original image by looking at the reduced image at the center of the combined thumbnail images 311 to 320.
In summary, when a plurality of thumbnail images or composite thumbnail images are displayed as a list in one screen as in the list image 300 in the example of FIG. 6, the user can easily identify differences in the shooting conditions and the overall image. It becomes possible to grasp.

  Further, in the composite thumbnail image, the place where the feature image is superimposed is a blank area that inevitably occurs due to the difference in the screen aspect ratio between the original image and the thumbnail image. For this reason, just as in the first embodiment, the visibility of the list image including a plurality of thumbnail images and composite thumbnail images, that is, the list property is not hindered, and the screen of the display unit 41 is limited. The range (display area range) is not wasted.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and modifications, improvements, and the like within the scope that can achieve the object of the present invention are included in the present invention.

For example, in the first embodiment described above, only the image data of the thumbnail image corresponding to the original image that has been subjected to image processing is to be combined, and the image data of the combined thumbnail image is generated. On the other hand, in the above-described second embodiment, only the image data of the thumbnail image corresponding to the original image obtained by bracket shooting is the object to be combined, and the image data of the combined thumbnail image is generated.
However, the synthesis target is not particularly limited to the examples of the first embodiment and the second embodiment described above, and may be arbitrary. For example, the first embodiment and the second embodiment may be combined. That is, the image data of the combined thumbnail image may be generated by using the image data of the thumbnail image corresponding to each of the original image subjected to the image processing and the original image obtained by the bracket shooting as a compositing target.

  For example, in the above-described embodiments, the image processing apparatus to which the present invention is applied has been described as an example configured as a photo frame. However, the present invention is not particularly limited to a photo frame, and can be applied to electronic devices in general. Specifically, for example, the present invention is applicable to a video camera, a portable navigation device, a portable game machine, and the like.

  The series of processes described above can be executed by hardware or can be executed by software.

  When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium. The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only constituted by the removable recording medium 51 (FIG. 1) distributed separately from the apparatus main body in order to provide the program to the user, but is also incorporated in the apparatus main body in advance. And a recording medium provided to the user. The removable recording medium 51 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being preinstalled in the apparatus main body is configured by, for example, the ROM 32 in FIG.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.

DESCRIPTION OF SYMBOLS 1 ... Photo frame (image processing apparatus) 11 ... Data processing part 12 ... User interface part 31 ... CPU, 32 ... ROM, 33 ... RAM, 34 ... Memory 35... Display control unit 36... Image processing unit 41... Display unit (display means) 42. ... Removable recording medium, 61 ... Image acquisition unit, 62 ... Image storage unit, 63 ... Composite thumbnail image generation unit, 64 ... List image display control unit, 71 ... Thumbnail image storage , 72 ... Original image storage unit, 73 ... Feature image storage unit, 74 ... Composite thumbnail image storage unit, 81 ... Composite object determination unit, 82 ... Feature image extraction unit, 83 ..Image composition unit

Claims (6)

Determining means for determining a processing target from image data of thumbnail images in which a blank area is added to a reduced image of the original image;
From the image data of the original image corresponding to the processing target determined by the determining means, the image includes at least a part of the feature region of the original image, and the image data of the image having the same resolution as the thumbnail image, Feature image extraction means for extracting as feature image data;
By performing a synthesis process for synthesizing the feature image and the thumbnail image on the data of the thumbnail image determined as a processing target by the determination unit and the image data of the feature image extracted by the feature image extraction unit Image synthesizing means for generating image data of a synthesized thumbnail image in which the feature image is superimposed on the margin area of the thumbnail image;
An image processing apparatus comprising: The determining means determines the image data of the thumbnail image generated from the image data of the original image subjected to image processing as a processing target.
The image processing apparatus according to claim 1. The determination means determines the image data of the thumbnail image generated from the image data of the original image obtained by bracket shooting as a processing target.
The image processing apparatus according to claim 1. The feature image extraction unit detects a human face area included in the original image as the feature area, and extracts image data including at least a part of the detected face area as the feature image data. ,
The image processing apparatus according to claim 1. A determination step of determining a processing target from image data of thumbnail images in which a blank area is added to the reduced image of the original image;
From the image data of the original image corresponding to the processing target determined by the processing of the determination step, the image data includes at least a part of the feature region of the original image and has the same resolution as the thumbnail image A feature image extraction step for extracting the image as feature image data;
The combination processing for combining the feature image and the thumbnail image is the thumbnail image data determined as the processing target by the determination step processing and the feature image image data extracted by the feature image extraction step processing. An image synthesis step for generating image data of a synthesized thumbnail image in which the feature image is superimposed on the margin area of the thumbnail image;
An image processing method including: A computer provided in the image processing apparatus,
Determining means for determining a processing target from image data of thumbnail images in which a blank area is added to a reduced image of the original image;
From the image data of the original image corresponding to the processing target determined by the determining means, the image includes at least a part of the feature region of the original image, and the image data of the image having the same resolution as the thumbnail image, Feature image extraction means for extracting as feature image data;
By performing a synthesis process for synthesizing the feature image and the thumbnail image on the data of the thumbnail image determined as a processing target by the determination unit and the image data of the feature image extracted by the feature image extraction unit Image synthesizing means for generating image data of a synthesized thumbnail image in which the feature image is superimposed on the margin area of the thumbnail image;
Image processing program to make it function.

Images