JP2007020029A - Image processor and method, program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To simply and surely confirm each of a plurality of face images. <P>SOLUTION: When face images 131-133 are displayed in the whole image, one face image 131 on the most lefthand side is zoomed on a face button being turned on. When the face button is further turned on, a face image 132 on the lefthand side is zoomed. When the face button is further turned on, a face image 133 on the lefthand side is zoomed. When the face image button is further turned on, the original whole image is displayed. The present invention is applicable to a digital still camera. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method, a program, and a recording medium, and more particularly, to an image processing apparatus and method, a program, and a recording medium that allow each of a plurality of face images to be easily and reliably confirmed.

  Recently, digital still cameras have become popular. There is no need for development, and an image captured and stored in the internal memory can be immediately read out and displayed. Therefore, when a user goes on a school trip, a trip with family or friends, etc., after taking a commemorative photo with multiple people, the user immediately checks the captured image and closes his eyes at the timing of shooting. If there is, you can retake it.

  Patent Document 1 proposes a digital still camera that automatically detects a human face in a preview image (monitoring image). In this digital still camera, when a plurality of human faces are detected in the preview image, a frame is displayed around each face. The user selects one face by selecting a desired frame. When the user operates the face enlargement display button, the selected face is enlarged and displayed. After confirming the state of the enlarged face, the user turns on the shutter button and records an image.

  When the user selects a plurality of faces, the screen is divided into a plurality of areas according to the selected number, and images of the plurality of faces are displayed simultaneously on one screen. For example, when four faces are selected, one screen is divided into four 2 × 2 areas, and four face images are simultaneously displayed in each divided area.

Japanese Patent Laying-Open No. 2005-102175 (paragraph 0036, FIGS. 2 to 5)

  As described above, in the invention described in Patent Document 1, when a user selects a plurality of faces, images of the plurality of faces are simultaneously displayed in the divided areas of the screen. There is a problem that the size becomes small and it is difficult to check each state.

  The present invention has been made in view of such a situation, and makes it possible to easily and reliably confirm each of a plurality of face images.

One aspect of the present invention is an image data acquisition unit that acquires image data of an entire image to be displayed;
Position information acquisition means for acquiring position information of a plurality of face images in the whole image;
Instruction acquisition means for acquiring an instruction to switch display of the plurality of face images included in the whole image;
Each time a switching instruction is acquired, the image processing apparatus includes display means for sequentially switching and enlarging and displaying the plurality of face images included in the entire image one by one.

  The display means can display the entire image when a switching instruction is further acquired after sequentially displaying all of the plurality of face images.

  The image data of the face image to be enlarged can be further provided with gamma correction means for gamma correction.

  The gamma correction means can perform reverse S-shaped gamma correction when the histogram of the image data luminance of the face image to be enlarged is U-shaped, and can perform S-shaped gamma correction when the histogram is inverted U-shaped.

  The display means can further display the position of the displayed face image in the entire image.

A recording unit that records image data of the captured image together with position information of the plurality of face images on a recording medium;
The image data acquisition unit and the position information acquisition unit can acquire the image data and the position information read from the recording medium.

  The recording means can further record a ratio of the face image to the whole image.

  The display means can calculate a position where OSD information is displayed and does not overlap the face image, and can display the OSD information at the calculated position.

  The display means can display the OSD information, calculates a size that does not overlap the face image, and reduces the calculated size to display the OSD information.

One aspect of the present invention acquires image data of a whole image to be displayed,
Obtaining position information of a plurality of face images in the whole image;
Obtaining an instruction to switch display of the plurality of face images included in the whole image;
This is an image processing method or a program for causing a computer to execute an image processing method for sequentially switching and enlarging and displaying a plurality of face images included in the entire image each time a switching instruction is acquired.

  In one aspect of the present invention, each time a switching instruction is acquired, a plurality of face images included in the entire image are sequentially switched one by one and enlarged and displayed.

  According to one aspect of the present invention, each of a plurality of face images can be easily and reliably confirmed.

  Embodiments of the present invention will be described below. Correspondences between the configuration requirements of the present invention and the embodiments described in the detailed description of the present invention are exemplified as follows. This description is to confirm that the embodiments supporting the present invention are described in the detailed description of the invention. Accordingly, although there are embodiments that are described in the detailed description of the invention but are not described here as embodiments corresponding to the constituent elements of the present invention, It does not mean that the embodiment does not correspond to the configuration requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

An aspect of the present invention is an image data acquisition unit that acquires image data of an entire image to be displayed (for example, the reading unit 71 of FIG. 4 that executes the process of step S141 of FIG. 14),
Position information acquisition means for acquiring position information of a plurality of face images in the whole image (for example, the reading unit 71 of FIG. 4 that executes the process of step S146 of FIG. 14);
An instruction acquisition means for acquiring an instruction to switch display of the plurality of face images included in the entire image (for example, the determination unit 72 in FIG. 4 that executes the processing of steps S148 and S153 in FIG. 14);
Each time a switching instruction is acquired, a plurality of the face images included in the whole image are sequentially switched one by one in order to be enlarged and displayed (for example, the process of executing step S176 in FIG. 16) 4 display unit 75).

  The display means sequentially displays all of the plurality of face images, and when a switching instruction is further acquired (for example, when it is determined that all images are selected in step S149 in FIG. 14), The entire image can be displayed (for example, the process in step S152 in FIG. 14).

  The image data of the face image to be magnified can be further provided with gamma correction means (for example, the correction unit 78 in FIG. 4 that executes the processes of steps S173 and S175 in FIG. 16).

  When the histogram of the image data brightness of the face image to be enlarged is U-shaped, the gamma correction means performs inverse S-gamma correction (for example, the process of step S173 in FIG. 16) and is inverted U-shaped. In this case, S-shaped gamma correction can be performed (for example, the process in step S175 in FIG. 16).

  The display means can further display the position of the displayed face image in the whole image (for example, the frame 191 of the reduced image 172 in FIG. 21).

The figure which performs the process of step S6 of FIG. 5 (for example) which records the image data of the imaged said whole image on a recording medium (for example, the removable memory 50 of FIG. 3) with the positional information on the said some face image 4 recording unit 74),
The image data acquisition unit and the position information acquisition unit can acquire the image data and the position information read from the recording medium.

  The recording means can further record a ratio (for example, a reduction ratio) of the face image to the whole image.

  The display means calculates a position where the OSD information is displayed and does not overlap the face image (for example, the process of step S204 in FIG. 22), and displays the OSD information at the calculated position (for example, the step in FIG. 22). S205).

  The display means calculates OSD information, a size that does not overlap the face image (for example, the process of step S204 in FIG. 22), reduces the calculated size to display the OSD information (for example, , Step S205 in FIG. 22).

The aspect of the present invention acquires image data of the entire image to be displayed (for example, step S141 in FIG. 14),
Acquiring position information of a plurality of face images in the whole image (for example, step S146 in FIG. 14);
An instruction to switch the display of the plurality of face images included in the whole image is acquired (for example, steps S148 and S153 in FIG. 14),
Each time a switching instruction is acquired, the plurality of face images included in the entire image are sequentially switched one by one and enlarged and displayed (for example, step S176 in FIG. 16).
An image processing method or a program for causing a computer to execute these processes.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. 1 and 2 show the configuration of a digital still camera as an embodiment of the present invention. The digital still camera 1 includes a main body 11 and a lens barrel 12 protruding from the main body 11 as necessary.

  A dial 14 is provided near the right end of the upper surface of the main body 11. A power button 13 is provided on the left side of the dial 14. The power button 13 is operated when the power of the digital still camera 1 is turned on or off. The dial 14 is operated to select any one of scene 1 (SCN1) to scene 6 (SCN6), full auto (mark of a camera graphic), and program mode (P).

  Scenes 1 to 6 are associated with various shooting conditions such as landscape, portrait, night view, and night view + person. For example, when scene 1 is selected, shooting conditions suitable for shooting a landscape are set. When the scene 2 is selected, shooting conditions suitable for shooting a portrait are set. When full auto is selected, all shooting conditions such as focusing, exposure, and white balance adjustment necessary for shooting are automatically set. When the program mode is selected, adjustments necessary for shooting are automatically set as in full auto mode, but the user intentionally sets various setting items such as white balance and EV value (exposure compensation value). It becomes possible.

  A release button 15 is disposed at the center of the dial 14. The release button 15 is operated when the user records a captured image.

  An LCD (Liquid Crystal Display) 21 for displaying various images is disposed at the approximate center on the back side of the main body 11. A switch 22 is arranged on the upper right of the LCD 21. This switch 22 is operated when tele or wide is set. A menu button 23 is disposed below the switch 22. The menu button 23 is operated by the user when displaying a menu on the LCD 21. A button 24 is arranged below the menu button 23. The button 24 can be operated in the up / down / left / right directions. When operated on the upper side, the flash is selected, when operated on the lower side, a timer is set, and when operated on the right side, macro shooting is performed. When the mode is set and it is operated to the left, the white balance can be set manually.

  A playback button 25 is arranged on the left side of the LCD 21. The playback button 25 is operated when playing back a captured image. A face button 26 is arranged below the playback button 25. The face button 26 is sequentially displayed every time a face image included in the displayed whole image is operated when the image recorded on the LCD 21 is read and displayed. . A display button 27 is arranged below the face button 26. The display button 27 is operated when displaying OSD (On Screen Display) information or deleting the display. A trash can button 28 is disposed below the display button 27. The trash can button 28 is operated when deleting unnecessary files and the like. Note that the whole image means one whole image including the face image and the other parts with respect to the face image which is a partial image.

  FIG. 3 shows a basic configuration inside the digital still camera 1. The camera block 41 includes a lens, an imaging device such as a CCD (Charge Coupled Device) and a CMOS (Complementary Metal-Oxide Semiconductor), a timing generator, and the like. The image processing block 42 performs processing such as white balance, brightness adjustment, and gamma correction on the image signal output from the camera block 41 and outputs the processed signal to the image display device 44. The image processing block 42 incorporates a face recognition block 43, and this face recognition block 43 recognizes a face image existing in one whole image. The image processing device 44 displays an image based on the image data supplied from the image processing block 42. The image display device 44 includes the LCD 21 of FIG. 2 and an electronic viewfinder.

  An SDRAM (Synchronous Dynamic Random Access Memory) 45 temporarily holds image data and various other data. The operation block 46 includes various buttons and switches. The operation button 46 includes the power button 13, the dial 14, the switch 22, the menu button 23, the button 24, the play button 25, the face button 26, the display button 27, the trash can button 28, and the like in FIG. The operation block 46 outputs a signal corresponding to the operation to the control block 47.

  The control block 47 controls the operation of each block based on a user instruction supplied from the operation block 46. The flash memory 48 stores white balance, parameters for setting various other shooting conditions, and the like. The removable memory controller 49 is controlled by the control block 47, reads out image data stored in the removable memory 50, and stores image data or the like therein. The removable memory 50 is configured by a flash memory such as a Memory Stick (registered trademark), for example.

  The control block 47 has a functional configuration shown in FIG. 4, for example. The reading unit 71 reads various data and information. The determination unit 72 executes various determination processes. The holding unit 73 holds image data. The recording unit 74 stores the image data in the removable memory 50 via the removable memory controller 49. The display unit 75 displays an image. The selection unit 76 selects a face image. The computing unit 77 computes a luminance histogram and computes coordinates. The correction unit 78 performs gamma correction.

  Next, image recording processing of the digital still camera 1 will be described with reference to the flowchart of FIG. When the user operates the power button 13 to turn on the power and instruct to start imaging, the control block 47 controls each block to start imaging. At this time, the camera block 41 captures an image of the subject and outputs a corresponding image signal to the image processing block 42. The image processing block 42 temporarily stores the image signal supplied from the camera block 41 in the SDRAM 45. The image data stored in the SDRAM 45 is read therefrom by the image processing block 42, output to the image display device 44, and displayed on the LCD 21. In this way, a preview image is displayed.

  Further, the user operates the menu button 23 to set the autofocus (AF) mode. As the autofocus mode, any one of a face recognition autofocus mode, a continuous autofocus mode, and a single autofocus mode is set. When the user operates the operation block 46 to set the autofocus mode, the mode is stored in the flash memory 48 by the holding unit 73.

  In step S1, the reading unit 71 reads the focus mode. That is, the focus mode stored in the flash memory 48 is read out. In step S2, the determination unit 72 determines whether the read focus mode is the face recognition autofocus mode. If the face recognition autofocus mode is selected, face recognition processing is executed by the face recognition block 43 in step S3. The details of this face recognition processing will be described later with reference to the flowchart of FIG. 8, whereby the position information of the face image included in the entire image displayed on the LCD 21 is detected. The detected position information is temporarily recorded in the SDRAM 45. In step S4, the determination unit 72 determines whether the release button has been fully pressed.

  If the release button 15 is not fully pressed, the process returns to step S3, and the subsequent processes are repeatedly executed. If it is determined that the release button 15 has been fully pressed, in step S5, the holding unit 73 performs various processes on the image data and holds the image data. That is, the update of the image data held in the SDRAM 45 is prohibited at the timing when the release button 15 is fully pressed. In step S6, the recording unit 74 records the image data and the face position information in the removable memory. That is, the image data recorded in the SDRAM 45 is read and supplied to the removable memory controller 49. The removable memory controller 49 supplies the image data and face position information to the removable memory 50 for recording.

  When the face recognition autofocus mode is set as described above, not only the image data but also the corresponding face position information is recorded in the removable memory 50.

  FIG. 6 shows the structure of the memory area of the removable memory 50. MBR (Master Boot Record) records the location and size of partitions that make up the media, and the ID that indicates the OS type. PBR (Partion Boot Recorder) describes an active partition used when the OS is started. FAT (File Allocation Table) describes information for managing the physical allocation of files. ROOTDIR describes the top directory of the drive in the hierarchical directory structure.

  The image file is configured in units of Exif, thumbnail, face position information, and main image file. Exif (Exchangeable Image File Format) is an image file standard for digital still cameras advocated by the Japanese Electronic Industry Development Association (JEIDA), and records additional information such as image information and shooting date and time. The thumbnail is a display image that is obtained by reducing the corresponding main image of the maximum size for recording so as to fit the size of the LCD or the like to be displayed. The face position information is information representing the position of a face image existing in the main image (entire image) recorded in the main image file. The face position information may be recorded in a manufacturer note in Exif. The main image data is image data of the main image and is recorded in the main image file. Both the main image and the thumbnail are the entire image. Of course, the position of the face in the thumbnail can be easily obtained from the ratio of the main image and the thumbnail image.

  Returning to FIG. 5, when it is determined that the mode set in step S2 is not the face recognition autofocus mode, the determination unit 72 determines in step S7 whether the mode is the continuous autofocus mode. If the set autofocus mode is the continuous autofocus mode, processing corresponding to that mode is executed in step S8. If it is determined that the set mode is not the continuous autofocus mode, the set autofocus mode is eventually the single autofocus mode. Therefore, in this case, processing corresponding to the mode is executed in step S9.

  In the continuous autofocus mode, release is possible regardless of whether or not focus is achieved. In the single autofocus mode, the operation is controlled so that the shutter is released at the time of focusing.

  Next, the face recognition process in step 3 will be described. This face recognition process is executed by the face recognition block 43. In order to execute this processing, the face recognition block 43 is configured as shown in FIG. The input image scale conversion unit 71 of the face recognition block 41 reads the main image and converts the reduction ratio of the main image into a plurality of different scale images. The window cutout unit 72 cuts out a window from the image whose scale has been converted by the input image scale conversion unit 71. The template matching unit 73 matches the window image cut out by the window cutout unit 72 with a template image prepared in advance. The preprocessing unit 74 performs vector conversion on the score image supplied from the template matching unit 73, and converts the obtained vector group into one pattern vector. The pattern identification unit 75 identifies whether the score image supplied from the preprocessing unit 74 is a face image. The overlap determination unit 76 removes an area where the score images overlap.

  Next, details of the face recognition process in step S3 of FIG. 5 will be described with reference to the flowchart of FIG.

  In step S31, the input image scale conversion unit 71 reads a frame image. That is, the main image stored in the SDRAM 45 is read out here. In step S32, the input image scale conversion unit 71 creates a scale image having a reference size for performing face detection processing from the main image, and further converts the scale image into a plurality of scale images having different reduction ratios. For example, when the reference scale image is 320 × 240 pixels (= 76800 pixels), the scale image is sequentially reduced by 0.8 times to be converted into a five-stage scale image. As a result, 1.0 to 0.8, 0.64, 0.51, and 0.41 times of the first to fifth scale images are generated. When the image size to be detected by the face increases, the scale is set to be finer as appropriate, and the reduction is performed in a range where one side does not become 20 pixels or less.

  In step S33, the window cutout unit 72 cuts out a window area. For example, a rectangular area of 20 × 20 pixels (= 400 pixels) is first scanned from the first scale image starting from the upper left corner of the image while sequentially shifting two pixels to the right or lower side to the lower right corner of the image. (Window area) is cut out sequentially.

  In step S34, the template matching unit 73 executes a process of converting into a function curve. That is, the window region image (window image) is converted into a function curve having a peak value by performing arithmetic processing such as a normalized correlation method and an error square method. In step S35, the template matching unit 73 determines whether the peak value of the function is greater than a threshold value. That is, a sufficiently low threshold is set in advance so as not to deteriorate the recognizability (for example, a threshold serving as a determination criterion is set using an average face image consisting of an average of about 100 persons as a template). Whether the window image is a face image is determined based on the threshold value.

  When it is determined in step S35 that the peak value of the function is larger than the threshold value (when it is determined that the human face is included in the window image), the preprocessing unit 74 performs preprocessing in step S36. The details of this pre-processing will be described later with reference to the flowchart of FIG. 9. As a result, a window image (score image) including a human face image is converted into a vector, and the obtained vector group further includes one pattern. Converted to a vector.

  In step S37, the pattern identification unit 75 executes pattern identification processing. Details of this processing will be described later with reference to the flowchart of FIG. 10, whereby the face position information is generated and stored.

  In step S38, the pattern identification unit 75 determines whether all scale images of the scale currently being processed have been processed. For example, when an image of the first scale is being processed, it is determined whether all the scale images of the first scale have been processed. If there is an image of the first scale that has not yet been processed, the window cutout unit 72 selects another scale image of the first scale in step S39. Thereafter, the processing returns to step S33, and the subsequent processing is repeatedly executed.

  If it is determined in step S38 that all scale images of the scale currently being processed have been processed, the pattern identifying unit 75 determines in step S40 whether all scale images have been processed. For example, if there is a first scale to a fifth scale, it is determined whether all images have been processed up to the fifth scale. If there is an image of a scale that has not yet been processed, the window cutout unit 72 selects a scale image of another scale in step S41. Thereafter, the processing returns to step S33, and the subsequent processing is repeatedly executed.

  As described above, when the processing for all the images of the first scale to the fifth scale is completed, in step S42, the overlap determination unit 76 executes the overlap processing. The details of the overlapping process will be described later with reference to the flowchart of FIG. 12, whereby the process of removing the overlapping area is executed.

  In step S35, when it is determined that the peak value of the function is not larger than the threshold value (when it is determined that the human face image is not included in the score image), the processing of steps S36 to S42 is skipped. .

  Next, details of the pre-processing in step S36 in FIG. 8 will be described with reference to the flowchart in FIG.

  In step S71, the preprocessing unit 74 extracts the central portion of the score image. That is, by removing four corners that are unrelated to the human face area from the score image of a rectangular area composed of 20 × 20 pixels, an area of 400 pixels to 360 pixels is extracted. In step S72, the preprocessing unit 74 corrects the gray value. Specifically, in order to eliminate subject condition variation expressed by shading due to illumination at the time of shooting, a mean square error is formed so as to form a plane based on the most suitable part as a face image among 360-pixel score images. The gray value is corrected using the method.

  In step S73, the preprocessing unit 74 performs a histogram smoothing process. That is, the result of enhancing the contrast of the score image is subjected to a histogram smoothing process so that a face image can be detected regardless of the gain of the camera block 41 and the intensity of illumination. Next, in step S74, the preprocessing unit 74 converts the score image into a pattern vector. Specifically, the score image of 360 pixels is converted into a vector by the Gabor filtering process, and the obtained vector group is further converted into one pattern vector.

  Next, details of the pattern identification processing in step S37 in FIG. 8 will be described with reference to the flowchart in FIG.

  In step S91, the pattern identification unit 75 compares the pattern vector with the identification function. That is, the pattern vector generated by the preprocessing unit 74 is compared with a provisional discriminant function obtained in advance using learning data. Specifically, face images are detected using a support vector machine for a score image of 360 pixels as a pattern vector obtained from the preprocessing unit 74.

  In step S92, the pattern identification unit 75 determines whether a face image is detected. If a face image is detected, in step S93, the pattern identification unit 75 lists and stores the coordinates, size (number of vertical and horizontal pixels), and reduction ratio of the center of the score image. Here, the reduction ratio means a reduction ratio of the scale image with respect to the main image.

  FIG. 11 shows an example of coordinate data. The main image 100 that is the entire image is composed of X0 pixels in the horizontal direction and Y0 pixels in the vertical direction. The center of the face image 101 in the main image 100 is the coordinates (p1, q1). The length of the face image 101 in the vertical direction is b1 and the width is a1. In this case, the coordinates of the uppermost point of the face image 101 are (p1, q1-b1 / 2). The coordinates of the lowermost point of the face image 101 are (p1, q1 + b1 / 2). The coordinates of the leftmost point of the face image 101 are (p1−a1 / 2, q1), and the coordinates of the rightmost point are (p1 + a1 / 2, q1).

  When the aspect ratio of the main image 100 is 3 to 4 (when the ratio of Y0 and X0 is 3 to 4), when the face image 101 is enlarged and displayed, the number of vertical margin pixels is α1. At this time, α1 is expressed as a constant ratio to the length b1 of the face image 101. When this face image 101 is enlarged and displayed with an aspect ratio of 3 to 4, the width of the face area 101A as an area to be displayed is (b1 + 2α1) × 4/3, and the length in the vertical direction is b1 +. 2α1.

  Similarly, the center coordinates of the face image 102 are (p2, q2). The length of the face image 102 in the vertical direction is b2, and the length in the horizontal direction is a2. The coordinates of the uppermost point of the face image 102 are (p2, q2−b2 / 2), and the coordinates of the lowermost point are (p2, q2 + b2 / 2). The coordinates of the leftmost point of the face image 102 are (p2-a2 / 2, q2), and the coordinates of the rightmost point are (p2 + a2 / 2, q2).

  Further, the length of the margin calculated corresponding to the vertical length b2 of the face image 102 is α2. When the face image 102 is displayed with an aspect ratio of 3 to 4, the horizontal width of the face region 102A is (b2 + 2α2) × 4/3, and the height is b2 + 2α2.

  The face image 101 is detected by a scale image having a scale in which the length b1 in the vertical direction or the width a1 in the horizontal direction is expressed by about 20 pixels. Similarly, the face image 102 is also detected as a scale image when the length b2 in the vertical direction or the width a2 in the horizontal direction is about 20 pixels. Therefore, the center coordinates (p1, q1) and (p2, q2) are also obtained as the center coordinates of the score image in each scale. The center coordinates are also stored in the SDRAM 45 in the same manner.

  If it is determined in step S92 that no face image has been detected, the pattern identifying unit 75 adds the score image to the learning data as non-face data in step S94. Thereby, learning data with a higher recognition rate is generated.

  Next, details of the overlap processing in step S42 in FIG. 8 will be described with reference to the flowchart in FIG.

  In step S <b> 121, the overlap determination unit 76 reads scale data list data of one scale. That is, the list data stored in the SDRAM 45 is read here. In step S122, the overlap determination unit 76 compares the list data. In step S123, the overlap determination unit 76 determines whether the two score images overlap. If the two score images overlap, in step S124, the overlap determination unit 76 executes a process for removing the overlapping region.

For example, as shown in FIG. 13, when there are two score images P1, P2, the coordinates of the upper left corner are (Xa, Ya), (Xb, Yb). The score image P1 has a width La, a height Ha, and the score image P2 has a width Lb and a height Hb. When the difference between the x coordinates of the upper left points of the two score images is dX (= Xb−Xa) and the difference between the y coordinates is dY (= Yb−Ya),
(La−dX) × (Lb + dX)> 0
And (Ha−dX) × (Hb + dY)> 0
When this condition is satisfied, it can be determined that the score image P1 and the score image P2 overlap. If there is an overlapping area, the overlapping area (area shown by hatching in FIG. 13) is removed.

  If it is determined in step S123 that the two score images do not overlap, the process of step S124 is skipped.

  In step S125, the overlap determination unit 76 determines whether list data of scale images of all scales has been read. If there is list data of scale images that have not been read yet, the process returns to step S121, and the subsequent processing is repeatedly executed. In this way, a plurality of list data is read for each scale image of the first scale image to the fifth scale image, and the overlap of the list data is determined.

  If it is determined in step S125 that all the scale image list data has been read, the overlap determination unit 76 stores the face position information of each scale image in step S126. That is, the position information is stored in the SDRAM 45.

  As described above, in the removable memory 50, the position information (center coordinates, reduction ratio, size, etc.) of the face of the main image is recorded together with the main image file as shown in FIG. That is, information that can calculate the position of the face in the entire image such as the main image or the thumbnail image is recorded.

  Although explanation is omitted, a thumbnail image obtained by reducing the main image file is also recorded at the same time.

  Next, processing for reproducing the image data stored in the removable memory 50 as described above will be described with reference to the flowchart of FIG. When the user operates the operation block 46 to instruct the reproduction of a predetermined main image (entire image), the reading unit 71 is designated by the user from the removable memory 50 via the removable memory controller 49 (initially (for example, default (for example, The data of the main image) is read out. That is, the thumbnail data of the designated main image is read from the removable memory 50 and stored in the SDRAM 45.

  In step S142, the determination unit 72 determines whether there is face position information corresponding to the designated main image data. If there is no face position information, the display unit 75 displays an image in step S143. That is, the thumbnail image data read in step S141 and stored in the SDRAM 45 is read, output to the image display device 44 via the image processing block 42, and displayed as an entire image on the LCD 21. Alternatively, the main image data may be read in step S141, a reduced display image may be generated from the main image data, and the entire image may be displayed in step S143.

  In step S144, the determination unit 72 determines whether another image (entire image) is selected. If another image (entire image) is selected, the process returns to step S141, and the subsequent processing is repeatedly executed. The If it is determined in step S144 that no other image (entire image) has been selected, in step S145, the determination unit 72 determines whether termination has been instructed. If the end has not been instructed yet, the process returns to step S143, and the subsequent processes are repeated. If termination is instructed, the reproduction process is terminated.

  If it is determined in step S142 that there is face position information, the reading unit 71 reads face position information in step S146. As a result, the face position information is read from the removable memory 50 via the removable memory controller 49 and stored in the SDRAM 45 (or, if there is face position information, it is read together with the thumbnail image data or the main image data). Or may be stored in the SDRAM 45). At this time, the calculation unit 77 performs calculation so that the number of pixels in the vertical direction is appropriate from the obtained face position information (for example, in the case of the face image 101 in FIG. 11, the center coordinates (p1, q1)) And a margin α1 from the length b1 in the vertical direction is calculated) and stored in the SDRAM 45. In step S147, the display unit 75 displays an image. That is, the thumbnail image data read in step S141 is supplied to the LCD 21 of the main image display device 44 via the image processing block 42 and displayed as an entire image. Alternatively, a display image generated by reducing the read main image data is displayed as an entire image.

  In step S148, the determination unit 72 determines whether the face button is turned on. When the face button 26 is turned on, in step S149, the determination unit 72 determines whether all face images in the entire image that has been specified have been selected. If not all face images have been selected, the selection unit 76 selects one face image in step S150. For example, when there are four face images in the entire image, one of the leftmost face images is selected. In step S151, the display unit 75 executes a face image display process. The details will be described later with reference to the flowchart of FIG. 16, but the selected face image is enlarged and displayed to an optimum size using the calculated margin. At this time, the face image is enlarged and displayed after being gamma-corrected as necessary.

  In step S153, the determination unit 72 determines whether another image has been selected. If it is determined that no other image (entire image) has been selected, in step S154, the determination unit 72 determines whether termination has been instructed. If the end has not been instructed yet, the process returns to step S148, and the subsequent processes are repeatedly executed. That is, if it is determined that the face button 26 is turned on again, it is determined in step S149 whether all the face images in the entire image have been selected. If all the face images in the entire image have not been selected yet, one face image is selected in step S150. That is, in this case, the second face image on the left side that has not yet been selected among the above-described four face images in the entire image is selected. In step S151, the second face image is enlarged and displayed.

  As described above, each time the face button 26 is turned on, the face images are sequentially enlarged and displayed on the LCD 21 one by one.

  If it is determined in step S149 that all face images in the entire image have been selected, the display unit 75 displays the entire image (thumbnail image or reduced image for display generated from the main image) in step S152. Is displayed. In other words, when all the four face images are displayed by sequentially turning on the face button 26, when the face button 26 is further turned on, the original whole image is displayed.

  FIG. 15 shows a display state that changes each time the face button 26 is operated as described above. As shown in the upper left of FIG. 15, when there is an entire image on which face images 131 to 133 are displayed, when the user turns on the face button 26 once, for example, the face image 131 located on the leftmost side in the drawing. Is enlarged and displayed as shown in the upper right in FIG. At this time, the magnification is also displayed. In this case, it is doubled. When the face button 26 is further turned on from this state, the face image 132 located second from the left in the entire upper left image in FIG. 15 is enlarged and displayed as shown in the lower right in FIG. In this case as well, the magnification is shown in the lower left. In this case, the enlargement magnification is 3 times. When the face button 26 is further turned on from this state, the face image 133 located on the rightmost side is enlarged and displayed as shown in the lower left of FIG. In this case, the enlargement magnification is set to 4 times. Further, when the face button 26 is turned on, the whole image (image including all face images) shown in the upper left of FIG. 15 is returned to the displayed state.

  After the process of step S152, the process proceeds to step S153, and it is determined whether another image (entire image) has been selected. If it is determined in step S153 that another image (entire image) has been selected, the process returns to step S141, and the subsequent processes are repeatedly executed.

  In step S154, when it is determined that the end is instructed, the reproduction process is ended.

  Next, the details of the face image display processing in step S151 in FIG. 14 will be described with reference to the flowchart in FIG.

  In step S171, the calculation unit 77 calculates a luminance histogram. That is, the luminance histogram of the face image that is the object of processing is calculated here. In step S172, the determination unit 72 determines whether the luminance histogram is U-shaped. That is, it is determined whether the luminance histogram pattern calculated in step S171 is substantially U-shaped. If the luminance histogram pattern is substantially U-shaped, the correction unit 78 performs inverse S-gamma correction on the image data of the face image in step S173. That is, the image data of the face image is corrected along the inverted S-shaped characteristic curve 151 as shown in FIG. According to the characteristic curve 151, when the input signal level is small and large, the output gradation is corrected so as to change abruptly. When the input signal level is in the middle, the change rate of the output gradation is small. It is corrected so that As a result, as shown in FIG. 18A, the frequency of brightness is extremely high, such as when the brightness (horizontal axis) is dark and bright (the vertical axis) is large and the intermediate brightness frequency is small. When the distribution is divided into two, the distribution is corrected so that the interval between the peaks of the mutual distribution is narrowed as shown in FIG. 18B. By correcting in this way, for example, an image in the backlit state is mainly displayed clearly.

  If it is determined in step S172 that the luminance histogram is not U-shaped, the determination unit 72 determines in step S174 whether the luminance histogram is inverted U-shaped. If the luminance histogram is substantially inverted U-shaped, in step S175, the correction unit 78 performs S-shaped gamma correction on the image data of the face image.

  That is, as shown in FIG. 19, the image data is corrected along a substantially S-shaped characteristic curve 161. The characteristic curve 161 gradually changes the output gradation when the input signal is small, but increases rapidly with a large change rate of the output gradation when the level of the input signal is near the middle. As the input signal level further increases, the change rate of the output gradation decreases. By performing such correction, for example, as shown in FIG. 20A, when the brightness distribution is concentrated in a narrow range in the center, the brightness distribution is as shown in FIG. 20B. Can be distributed over a wide range. By doing in this way, an image with little contrast can be changed into an image with enhanced contrast.

  If it is determined in step S174 that the luminance histogram is not an inverted U-shape, or after the process of step S173 or step S175, the process proceeds to step S176, where the display unit 75 displays an enlarged image. That is, when the luminance histogram is a pattern that is neither U-shaped nor inverted U-shaped, gamma correction is not performed and the face image is enlarged and displayed as it is. When the processes in steps S173 and S175 are performed, the gamma-corrected face image is enlarged and displayed. Thereby, the face image which is a part of the image is displayed in the same state as the entire image, thereby suppressing the face image from becoming unclear.

  In step S176, the display unit 75 enlarges and displays the face image and reduces the entire image (a diagram schematically showing the positional relationship) in order to make the user recognize the position of the face image. Are displayed at the same time. This state is shown in FIG. As shown in the figure, the face image 181 is enlarged and displayed on the screen 171.

  A reduced image 172 obtained by reducing the entire image is displayed at the lower right of the screen 171. Since the reduced image 172 is an image obtained by reducing the entire image as it is, all the face images 181 to 183 existing in the entire image are displayed. A frame 191 is displayed around the face image 181 so that the user can easily identify where the currently displayed range is in the entire image. In step S176, the magnification of the face image is also displayed at the same time. In the example of FIG. 21, the enlargement magnification is 4 times.

  The magnification that can be enlarged is a discrete value, and since the face areas 101A and 102A in FIG. 11 are enlarged to the size of the screen 171, the magnification is Y0 / (b + 2α1). Or Y0 / (b2 + 2α2). By adjusting the values of α1 and α2, the magnification at which the face image can be displayed in an enlarged manner is selected.

  If the face image with a small number of pixels is enlarged too much, the face cannot be recognized. Therefore, a limit may be set on the maximum magnification according to the number of pixels of the original image.

  Next, the OSD information display process will be described with reference to the flowchart of FIG.

  In step S <b> 201, the determination unit 72 determines whether an image is displayed on the LCD 21. If an image is displayed, in step S202, the determination unit 72 determines whether there is face position information. If there is face position information, the reading unit 71 reads face position information in step S203. As a result, the position information recorded in the removable memory 50 is read out via the removable memory controller 49 and stored in the SDRAM 45. When stored, the position information is read out. In step S204, the calculation unit 77 calculates a position away from the face image. That is, a position where the OSD information is displayed and the face image is not hidden when the OSD information is displayed is calculated. At this time, if necessary, the size of the OSD information that does not hide the face image is also calculated. In step S205, the display unit 75 displays the OSD information at the position calculated in the process of step S204. When the size is calculated so that the face image is not hidden, the OSD information of the reduced size is displayed.

  If it is determined in step S201 that no image is displayed, or if it is determined in step S202 that there is no face position information, in step S206, the display unit 75 sets the OSD to a preset default position. Display information.

  FIG. 23 shows a display example when OSD information is displayed at the default position. In the display example of FIG. 23, a face image 201 and a face image 202 exist, and OSD information 211 to 216 is displayed for the face image 201 and the face image 202. The OSD information 211 represents shutter speed (1/100), ISO value (ISO800), and the like. The OSD information 212 is a luminance value histogram. The OSD information 213 represents the remaining battery level (165 minutes). The OSD information 214 represents the current date and time (February 15, 2005, 3 characters 56 minutes). The OSD information 215 represents the image size (5M). The OSD information 216 indicates the number of sheets (126) recorded in the removable memory 50 and the current number of displays (21st).

  In the display example of FIG. 23, the face image 202 is hidden by the OSD information 212.

  On the other hand, FIG. 24 shows a display example in which the OSD information is displayed at a position where the face image is not hidden. In this case, OSD information 212 is displayed at the lower left of the face image 201. Therefore, as is apparent from comparison with FIG. 24, the face image 202 is prevented from being hidden by the OSD information 212.

  FIG. 25 shows another display example in which the OSD information is displayed at the default position. In this display example, when the face images 231 to 234 exist, the OSD information 212A to 212D hides the face image 233 and the face image 234. The OSD information 212A to 212D are histograms representing red, green, blue, and luminance distributions, respectively.

  In contrast, FIG. 26 shows a state in which OSD information is displayed at the calculated position. In this display example, the OSD information 212A to 212D is reduced and displayed at the lower left of the face image 231. Therefore, the face images 233 and 234 are prevented from being hidden. The OSD information 211 is displayed by moving from the default position (position shown in FIG. 25) in the upper right direction of the face image 231.

  As described above, when the position information of the face image exists, it is possible to display the OSD information by adjusting the position and size so as not to hide the face image.

  In addition, a simple and highly functional slide show has been proposed by automatically connecting multiple still images with various effects. In this case, the zoom position and magnification are determined at random, and a still image is displayed. Alternatively, the scroll direction and moving distance are determined randomly. As a result, even in a photograph showing a person, a part not related to the person may be zoomed up, or only a place where there is no person may be displayed by scrolling. However, since the position and size of the face can be known by using the face position information, an optimum zoom can be performed. It is also possible to scroll so that the human face does not protrude from the screen by scrolling the part where the human face is present.

  As described above, in the embodiment of the present invention, each time the face button 26 is operated, each of a plurality of face images existing in one whole image is sequentially enlarged and displayed one by one. Therefore, it is possible to easily and surely confirm a human face image.

  In addition, OSD display such as shooting information and histograms can be prevented from being applied to the face, and the OSD display position can be changed and displayed based on the face position and size data, so the face is not covered as much as possible OSD information can be displayed at the position.

  FIG. 27 is a block diagram showing an example of the configuration of a personal computer that executes the above-described series of processing by a program. A CPU (Central Processing Unit) 321 executes various processes according to a program stored in a ROM (Read Only Memory) 322 or a storage unit 328. A RAM (Random Access Memory) 323 appropriately stores programs executed by the CPU 321 and data. The CPU 321, ROM 322, and RAM 323 are connected to each other via a bus 324.

  An input / output interface 325 is also connected to the CPU 321 via the bus 324. The input / output interface 325 is connected to an input unit 326 including a keyboard, a mouse, and a microphone, and an output unit 327 including a display and a speaker. The CPU 321 executes various processes in response to commands input from the input unit 326. Then, the CPU 321 outputs the processing result to the output unit 327.

  The storage unit 328 connected to the input / output interface 325 includes, for example, a hard disk, and stores programs executed by the CPU 321 and various data. The communication unit 329 communicates with an external device via a network such as the Internet or a local area network. A program may be acquired via the communication unit 329 and stored in the storage unit 328.

  The drive 330 connected to the input / output interface 325 drives a removable medium 331 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and drives the programs and data recorded therein. Get etc. The acquired program and data are transferred to and stored in the storage unit 328 as necessary.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, the program is installed in a general-purpose personal computer from the program storage medium.

  As shown in FIG. 27, a program storage medium for storing a program that is installed in a computer and can be executed by the computer includes a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only). Memory), DVD (including Digital Versatile Disc)), magneto-optical disk (including MD (Mini-Disc)), or removable media 331 which is a package medium made of semiconductor memory or the like, or the program is temporary or permanent Is configured by a ROM 322 stored in the hard disk, a hard disk constituting the storage unit 328, and the like. The program is stored in the program storage medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 329 which is an interface such as a router or a modem as necessary. Done.

  In the present specification, the step of describing the program stored in the program storage medium is not limited to the processing performed in time series according to the described order, but is not necessarily performed in time series. Or the process performed separately is also included.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

It is a top view which shows the structure of the digital still camera of embodiment of this invention. It is a rear view which shows the structure of the digital camera of embodiment of this invention. It is a block diagram which shows the structure inside the digital camera of embodiment of this invention. It is a block diagram which shows the functional structure of a control block. It is a flowchart explaining an image recording process. It is a figure which shows the structure of a memory area. It is a block diagram which shows the structure of a face recognition block. It is a flowchart explaining a face recognition process. It is a flowchart explaining pre-processing. It is a flowchart explaining a pattern identification process. It is a figure which shows the example of coordinate data. It is a flowchart explaining an overlap process. It is a figure explaining an overlap process. It is a flowchart explaining a reproduction | regeneration process. It is a figure explaining the change of a display when a face button is pushed. It is a flowchart explaining a face image display process. It is a figure which shows a reverse S character gamma characteristic. It is a figure explaining the correction | amendment by a reverse S character gamma characteristic. It is a figure which shows S character gamma characteristic. It is a figure explaining the correction | amendment by S character gamma characteristic. It is an example of an enlarged display of a face image. It is a flowchart explaining an OSD information display process. It is an OSD display example in a default position. It is the example which changed the display position of OSD information. It is another example in which OSD information is displayed at a default position. It is the example which changed the display position of OSD information. It is a block diagram which shows the structure of a personal computer.

Explanation of symbols

1 digital still camera, 11 body, 12 lens barrel, 21 LCD, 22 switch, 25 playback button, 26 face button, 27 display button, 28 trash can button, 41 camera block, 42 image processing block, 43 face recognition block, 44 image Display device, 45 SDRAM, 46 Operation block, 47 Control block, 48 Flash memory, 49 Removable memory controller, 50 Removable memory

Claims (12)

Image data acquisition means for acquiring image data of the entire image to be displayed;
Position information acquisition means for acquiring position information of a plurality of face images in the whole image;
Instruction acquisition means for acquiring an instruction to switch display of the plurality of face images included in the whole image;
An image processing apparatus comprising: display means for sequentially switching and enlarging and displaying the plurality of face images included in the entire image each time a switching instruction is acquired. The image processing apparatus according to claim 1, wherein the display unit sequentially displays all of the plurality of face images, and further displays the entire image when a switching instruction is acquired. The image processing apparatus according to claim 1, further comprising a gamma correction unit that performs gamma correction on image data of the face image to be enlarged and displayed. The gamma correction unit performs reverse S-gamma correction when the histogram of the image data luminance of the face image to be enlarged is U-shaped, and performs S-shaped gamma correction when the histogram is reverse U-shaped. The image processing apparatus described. The image processing apparatus according to claim 1, wherein the display unit further displays a position of the displayed face image in the entire image. A recording unit that records image data of the captured image together with position information of the plurality of face images on a recording medium;
The image processing apparatus according to claim 1, wherein the image data acquisition unit and the position information acquisition unit acquire the image data and the position information read from the recording medium. The image processing apparatus according to claim 6, wherein the recording unit further records a ratio of the face image to the whole image. The image processing apparatus according to claim 1, wherein the display unit calculates OSD information, a position that does not overlap the face image, and displays the OSD information at the calculated position. The image processing apparatus according to claim 1, wherein the display unit displays OSD information, calculates a size that does not overlap the face image, reduces the calculated size, and displays the OSD information. Get the image data of the whole image to be displayed,
Obtaining position information of a plurality of face images in the whole image;
Obtaining an instruction to switch display of the plurality of face images included in the whole image;
An image processing method in which a plurality of face images included in the entire image are sequentially switched one by one and enlarged and displayed each time a switching instruction is acquired. Get the image data of the whole image to be displayed,
Obtaining position information of a plurality of face images in the whole image;
Obtaining an instruction to switch display of the plurality of face images included in the whole image;
A program that causes a computer to execute a process of sequentially switching and enlarging and displaying a plurality of face images included in the entire image each time a switching instruction is acquired.   A recording medium on which the program according to claim 11 is recorded.

Images