JP4340806B2 - Image processing apparatus, method, and program - Google Patents

Description

The present invention relates to an image processing apparatus , method, and program , and more particularly, to an image processing apparatus , method, and program for processing an image file including thumbnail image data.

In general, an image (still image) taken by a digital camera is recorded on a recording medium as an image file to which predetermined attached information is added. The attached information added to the image file includes thumbnail image data obtained by reducing the photographed image to a predetermined size, in addition to photographing information such as the aperture value and shutter speed of the photographed image. Conventionally, thumbnail image data to be added to the image file has been generated by reducing the entire captured image to a predetermined size (for example, Patent Document 1).
JP-A-10-243273

  However, thumbnails generated by reducing the entire image have the drawbacks that if the face of a person is shown small in the entire image, it is not possible to know who the image is, or the facial expression cannot be confirmed. .

The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image processing apparatus , method, and program capable of easily confirming the face of a person being photographed.

In order to achieve the above object, the invention according to claim 1 detects a human face from an input image, and the size of the human face detected by the face detection means is determined by the input. A calculating means for calculating a ratio of the entire image; and a ratio of the person detected by the face detecting means from the input image when the ratio calculated by the calculating means falls below a preset reference value . Image cutout means for cutting a face in a predetermined size, face thumbnail generation means for generating a face thumbnail of a predetermined size from the face image cut out by the image cutout means, image data of the input image, and image of the face thumbnail Image file generating means for generating an image file including data, and means for recording the image file generated by the image file generating means on a storage medium. To provide an image processing apparatus for.

According to the present invention, a face of a person is detected from the input image by the face detection means, and the detected face of the person is cut out to a predetermined size by the image cutting means. Then, a face thumbnail having a predetermined size is generated from the cut face image by the face thumbnail generation unit, and an image file including the generated face thumbnail image data and the input image image data is generated by the image file generation unit. The Further, according to the present invention, the ratio of the size of the face in the entire input image is calculated by the calculation means, and the face thumbnail generation means generates the face thumbnail only when the calculated ratio falls below the reference value. . That is, a face thumbnail is generated only when a person's face is photographed with a certain size or less.

  According to a second aspect of the present invention, in order to achieve the above object, when a plurality of human faces are detected by the face detecting means, the image cutting means is arranged to detect the person's face closest to the center of the input image. The image processing apparatus according to claim 1, wherein the image processing apparatus is cut out at a predetermined size.

  According to the present invention, when a plurality of human faces are detected by the face detecting means, the face of the person closest to the center of the input image is cut by the image cutting means.

  According to a third aspect of the present invention, in order to achieve the above object, when the faces of a plurality of persons are detected by the face detection means, the image cutout means detects each person detected by the face detection means. The face thumbnail generating unit generates a face thumbnail for each face image clipped by the image cutting unit, and the image file generating unit generates a plurality of generated face thumbnail images. The image processing apparatus according to claim 1, wherein an image file including data and image data of the input image is generated.

  According to the present invention, when a plurality of human faces are detected by the face detection means, the detected face of each person is cut by the image cutting means. Then, a face thumbnail is generated for each cut face image, and an image file including a plurality of generated face thumbnail image data and input image image data is generated by the image file generation means.

  According to a fourth aspect of the present invention, there is provided a panoramic thumbnail generating unit that generates a panoramic thumbnail of a predetermined size from the input image in order to achieve the object, and the image file generating unit includes image data of the face thumbnail. 4. The image processing apparatus according to claim 1, wherein the image file includes the image data of the panoramic view thumbnail and the image data of the input image.

  According to the present invention, a panoramic view thumbnail is generated together with a face thumbnail, and an image file including the generated face thumbnail image data, panoramic view thumbnail image data, and input image image data is generated by the image file generating means.

According to a fifth aspect of the present invention, in order to achieve the object, the step of acquiring an image, the step of detecting a human face from the acquired image, and the size of the detected human face A step of calculating a ratio of the entire acquired image, and when the calculated ratio falls below a preset reference value, the face of the person detected from the acquired image is A step of cutting by size, a step of generating a face thumbnail of a predetermined size from the cut face image, a step of generating an image file including the image data of the acquired image and the image data of the face thumbnail, and generation And an image processing method comprising: recording the recorded image file on a storage medium .
According to a sixth aspect of the present invention, in order to achieve the object, when a plurality of human faces are detected in the step of detecting the human faces, the human faces are cut out to a predetermined size. 6. The image processing method according to claim 5, wherein a face of a person closest to the center of the acquired image is cut out with a predetermined size.
According to a seventh aspect of the present invention, in order to achieve the above object, when a plurality of human faces are detected in the step of detecting the human faces, the human faces are cut out to a predetermined size. Cutting the face of each detected person with a predetermined size, and creating the face thumbnail, generating a face thumbnail for each cut face image, and generating the image file The image processing method according to claim 6, wherein an image file including the image data of the plurality of face thumbnails and the image data of the acquired image is generated.
In order to achieve the above object, the invention according to claim 8 includes a step of generating a panoramic view thumbnail of a predetermined size from the acquired image, and the step of generating the image file includes image data of the face thumbnail The image processing method according to claim 5, wherein the image file includes the image data of the panoramic view thumbnail and the image data of the acquired image.
In order to achieve the above object, the invention according to claim 9 includes a step of acquiring an image, a step of detecting a human face from the acquired image, and a size of the detected human face. A step of calculating a ratio of the entire acquired image, and when the calculated ratio falls below a preset reference value, the face of the person detected from the acquired image is A step of cutting by size, a step of generating a face thumbnail of a predetermined size from the cut face image, a step of generating an image file including the image data of the acquired image and the image data of the face thumbnail, and generation There is provided an image processing program characterized by causing a computer to execute a step of recording a recorded image file on a storage medium.
According to a tenth aspect of the present invention, in order to achieve the above object, when a plurality of human faces are detected in the step of detecting the human faces, the human faces are cut out by a predetermined size. The image processing program according to claim 9, wherein a face of a person closest to the center of the acquired image is cut out by a predetermined size.
According to an eleventh aspect of the present invention, in order to achieve the object, when a plurality of human faces are detected in the step of detecting the human faces, the human faces are cut out to a predetermined size. Cutting the detected face of each person in a predetermined size, and creating the face thumbnail, generating a face thumbnail for each cut face image, and generating the image file The image processing program according to claim 9, wherein an image file including the image data of the plurality of face thumbnails and the image data of the acquired image is generated.
In order to achieve the above object, the invention according to claim 12 includes a step of generating a panoramic view thumbnail of a predetermined size from the acquired image, and the step of generating the image file includes image data of the face thumbnail 12. An image processing program according to claim 9, 10 or 11, wherein an image file including the image data of the panoramic view thumbnail and the image data of the acquired image is generated.

According to the image processing apparatus , method, and program of the present invention, it is possible to easily confirm the face of a person being photographed.

The best mode for carrying out an image processing apparatus , method, and program according to the present invention will be described below with reference to the accompanying drawings.

  FIG. 1 is a front perspective view showing a configuration of a digital camera to which the present invention is applied. As shown in the figure, the camera body 12 of the digital camera 10 is formed in a horizontally long rectangular box shape, and a lens 14, a strobe 16, a finder window 18, a self-timer lamp 20, AF assist light on the front thereof. A lamp 22, a strobe light control sensor 24, and the like are provided. A shutter button 26, a power / mode switch 28, a mode dial 30 and the like are provided on the upper surface of the camera body 12.

  FIG. 2 is a rear view of the digital camera 10 shown in FIG. 1. As shown in FIG. 2, a monitor 32, a viewfinder eyepiece 34, a speaker 36, a zoom button 38, and a cross are placed on the back of the camera body 12. A button 40, a MENU / OK button 42, a DISP button 44, a BACK button 46, and the like are provided.

  Note that a battery insertion portion and a memory card slot are provided on the lower surface of the camera body 12 (not shown) via an openable / closable cover, and a battery and a memory card are loaded into the battery insertion portion and the memory card slot.

  The lens 14 is constituted by a retractable zoom lens, and is extended from the camera body 12 by setting the camera mode to the photographing mode by the power / mode switch 28.

  The shutter button 26 is constituted by a two-stage stroke type switch composed of so-called “half-press” and “full-press”. When the shutter button 26 is “half-pressed”, the digital camera 10 activates AE / AF, and “full-press” to execute shooting.

  The power / mode switch 28 has both a function as a power switch for turning on / off the power of the digital camera 10 and a function as a mode switch for setting the mode of the digital camera 10. It is slidably provided between “position” and “photographing position”. The digital camera 10 is turned on by sliding the power / mode switch 28 to the “reproduction position” or “photographing position”, and turned off by setting it to the “OFF position”. Then, the power / mode switch 28 is slid and set to “playback position” to set to “playback mode”, and to the “shooting position” to set to “shooting mode”.

  The mode dial 30 functions as a shooting mode setting means for setting the shooting mode of the digital camera 10. Depending on the setting position of the mode dial, the shooting mode of the digital camera 10 is “auto shooting mode”, “movie shooting mode”, “ "Portrait shooting mode", "Sport shooting mode", "Landscape shooting mode", "Night scene shooting mode", "Program shooting mode", "Aperture priority shooting mode", "Shutter speed priority shooting mode", "Manual shooting mode", etc. Set to

  The monitor 32 is composed of a liquid crystal display capable of color display. The monitor 32 is used as an image display panel for displaying recorded image images in the reproduction mode, and is also used as a user interface display panel for performing various setting operations. In the photographing mode, a through image is displayed as necessary, and is used as an electronic viewfinder for checking the angle of view.

  The zoom button 38 functions as zoom instruction means for instructing zooming, and includes a zoom tele button 38T for instructing zooming to the telephoto side and a zoom wide button 38W for instructing zooming to the wide angle side. In the digital camera 10, when the zoom tele button 38T and the zoom wide button 38W are operated in the shooting mode, the focal length of the lens 14 changes. Further, when the zoom tele button 38T and the zoom wide button 38W are operated in the reproduction mode, the image being reproduced is enlarged or reduced.

  The cross button 40 functions as direction indicating means for inputting instructions in four directions, up, down, left, and right, and is used, for example, for selecting menu items on a menu screen.

  The MENU / OK button 42 functions as a button (MENU button) for instructing a transition from the normal screen to the menu screen in each mode, and also functions as a button (OK button) for instructing selection confirmation, execution of processing, and the like. To do.

  The DISP button 44 functions as a button for instructing the display switching of the monitor 32. When the DISP button 44 is pressed during photographing, the display of the monitor 32 is switched from ON to framing guide display to OFF. If the DISP button 44 is pressed during playback, the playback mode is switched from normal playback to playback without character display to multi playback.

  The BACK button 46 functions as a button for instructing to cancel the input operation or return to the previous operation state.

  FIG. 3 is a block diagram showing an electrical configuration of the digital camera. The overall operation of the digital camera 10 is comprehensively controlled by a central processing unit (CPU) 110. CPU 110 receives operation signals input from operation unit 112 (shutter button 26, power / mode switch 28, mode dial 30, zoom button 38, cross button 40, MENU / OK button 42, DISP button 44, BACK button 46, etc.). Based on the above, the entire digital camera 10 is comprehensively controlled according to a predetermined control program.

  A ROM 116 connected to the CPU 110 via the bus 114 stores a control program executed by the CPU 110 and various data necessary for the control. The EEPROM 118 is a digital camera such as user setting information set by the user. Various setting information relating to the ten operations are stored. The memory (SDRAM) 120 is used as a calculation work area for the CPU 110 and is used as a temporary storage area for image data and the VRAM 122 is used as a temporary storage area dedicated to image data.

  As described above, the digital camera 10 is set to the shooting mode by setting the power / mode switch 28 to the shooting position, and the lens 14 is extended so that the camera can be shot.

  Under this photographing mode, the subject light that has passed through the lens 14 forms an image on the light receiving surface of the CCD 124 via the diaphragm 15. A large number of photodiodes are two-dimensionally arranged on the light receiving surface of the CCD 124, and above them, red (R), green (G), and blue (B) color filters have a predetermined arrangement structure (Bayer, G stripes). The subject light that has passed through the lens 14 is received by a photodiode through a color filter and converted into a signal charge in an amount corresponding to the amount of incident light.

  The signal charge accumulated in each photodiode is sequentially read out as a voltage signal (image signal) corresponding to the signal charge based on the drive pulse supplied from the timing generator (TG) 126, and is analog processing section (CDS / AMP). 128.

  The analog processing unit 128 samples and holds the input R, G, and B signals for each pixel, amplifies them, and outputs them to the A / D converter 130. The A / D converter 130 converts the analog R, G, and B signals output from the analog processing unit 128 into digital R, G, and B signals and outputs the signals, and the A / D converter 130 outputs the signals. The digital R, G, and B signals are taken into the memory 120 via the image input control unit 132.

  The image signal processing unit 134 performs signal processing on the R, G, and B signals fetched into the memory 120 in accordance with a command from the CPU 110 to generate image data including luminance data and color difference data. Further, the image signal processing unit 134 performs trimming processing on the generated image data in accordance with a command from the CPU 110, and performs resizing processing to a predetermined size.

  When outputting a captured image to the monitor 32, the image data generated by the image signal processing unit 134 is temporarily stored in the VRAM 122 and sent from the VRAM 122 to the video encoder 136. The video encoder 136 converts the input image data into a display signal format (for example, an NTSC color composite video signal), and outputs it to the monitor 32. As a result, an image captured by the CCD 124 is displayed on the monitor 32.

  An image captured by the CCD 124 is obtained by periodically taking an image signal from the CCD 124, periodically rewriting data in the VRAM 122 with image data generated from the image signal, and outputting the data to the monitor 32 via the video encoder 136. Is displayed in real time. The photographer can confirm the shooting angle of view by viewing the image (through image) displayed in real time on the monitor 32.

  Note that image data added from the VRAM 122 to the video encoder 136 is added to the synthesis processing unit 138 as necessary, and is synthesized with predetermined characters, graphics, etc., and then added to the video encoder 136. Thereby, required shooting information and the like are displayed superimposed on the through image.

  Shooting is performed by pressing the shutter button 26. When the shutter button 26 is half-pressed, an S1 ON signal is input to the CPU 110. The CPU 110 performs AE / AF processing in response to the input of the S1 ON signal.

  First, an image signal captured from the CCD 124 via the image input control unit 132 is input to the AF detection unit 140 and the AE / AWB detection unit 142.

  The AE / AWB detection unit 142 includes a circuit that divides one screen into a plurality of areas (for example, 16 × 16) and integrates R, G, and B signals for each divided area, and provides the integrated value to the CPU 110. . The CPU 110 detects the brightness of the subject (subject brightness) based on the integrated value obtained from the AE / AWB detection unit 142, and calculates an exposure value (shooting EV value) suitable for shooting. Then, an aperture value and a shutter speed are determined from the obtained photographing EV value and a predetermined program diagram, and an electronic exposure and an aperture drive unit 144 of the CCD 124 are controlled according to the determined aperture value and an appropriate exposure amount.

  The AE / AWB detection unit 142 calculates an average integrated value for each color of the R, G, and B signals for each divided area during automatic white balance adjustment, and provides the calculation result to the CPU 110. The CPU 110 obtains the ratio of R / G and B / G for each divided area from the obtained integrated value of R, integrated value of B, and integrated value of G, and calculates the R / G and B / G values obtained. The light source type is determined based on the distribution in the color space of R / G and B / G. Then, according to the white balance adjustment value suitable for the discriminated light source type, for example, the value of each ratio is approximately 1 (that is, the RGB integration ratio is R: G: B≈1: 1: 1 on one screen). As described above, the gain value (white balance correction value) for the R, G, and B signals of the white balance adjustment circuit is controlled to correct the signal of each color channel.

  The AF detection unit 140 includes a high-pass filter that passes only high-frequency components of the G signal, an absolute value processing unit, an AF area extraction unit that extracts a signal within a predetermined focus area (for example, the center of the screen), and an absolute value within the AF area. The CPU 110 is notified of the integrated value data obtained by the AF detection unit 140. The CPU 110 calculates a focus evaluation value (AF evaluation value) at a plurality of AF detection points while moving the focus lens group of the lens 14 by controlling the lens driving unit 146, and aligns the lens position where the evaluation value becomes the maximum. Determine as the focal position. Then, the lens driving unit 146 is controlled so that the focus lens group moves to the obtained in-focus position.

  As described above, the AE / AF process is performed by half-pressing the shutter button 26. Thereafter, when the shutter button 26 is fully pressed, an S2 ON signal is input to the CPU 110, and the CPU 110 performs shooting and recording processing in response to this.

  That is, first, the CCD 124 is exposed with the shutter speed and aperture value determined based on the photometric result. The image signal output from the CCD 124 is taken into the memory 120 via the analog processing unit 128, the A / D converter 130, and the image input control unit 132, and subjected to predetermined signal processing in the image signal processing unit 134. After being converted to data, it is stored in the memory 120. The image data stored in the memory 120 is added to the compression / decompression processing unit 148, compressed according to a predetermined compression format (for example, JPEG format), and then stored in the memory 120. Then, the image file of a predetermined format is recorded on the storage medium 152 via the media control unit 150.

  Here, as for the image file recorded on the storage medium 152, the attached information is recorded in the tag format in the file header portion. The attached information recorded in the file header portion includes thumbnails as shown in FIG. This thumbnail is created when a person's face is captured in the main image with a predetermined size or less, and a thumbnail (face thumbnail) of the image obtained by cutting out the person's face with a predetermined size is created. If the image is not captured, or even if a person is captured, but the size is larger than a predetermined size, a thumbnail (entire view thumbnail) of the entire main image is created.

  The procedure for creating thumbnails is as follows. In response to the full press of the shutter button 26, image data of the captured image (main image) generated by the image signal processing unit 134 is added from the memory 120 to the face detection unit 154.

  The face detection unit 154 extracts the face area in the main image from the input image data of the main image, and detects the position and size (FIG. 5A, FIG. 6A, FIG. 7). a)). In the extraction of the face area, for example, the face of a person in the image is detected by extracting the skin color data in the image. At this time, the face area may be extracted using information on a focus point at the time of shooting. In other words, in general, when shooting a person, the person's face is shot in focus, and the face position can be accurately and efficiently extracted by using information on the value that is desired to be focused. Can do.

  The detection result (face position and size) of the face detection unit 154 is input to the CPU 110, and the CPU 110 determines based on the detection result of the face detection unit 154 whether a human face is captured in the main image. To do. If a person's face is photographed, it is determined whether or not the person's face is photographed with a predetermined size or less. That is, the ratio of the face size of the person in the main image is calculated from the detection result of the face detection unit 154, and it is determined whether or not the calculated ratio is equal to or less than a preset set value.

  As a result of this determination, if the face of the person is copied at a predetermined size or less, the “face thumbnail” is created (see FIGS. 5 and 6). If the face of the person is not copied, Alternatively, even if a person's face is photographed, if the person's face is photographed larger than a predetermined size, a process of creating a “entire view thumbnail” is performed (see FIG. 7).

  The setting value is, for example, 1/5 (corresponding to the face size at the time of bust-up shooting), and the face size of the person detected by the face detection unit 154 is 1/5 or less of the entire image. Creates a face thumbnail, and if it exceeds 1/5, creates a panoramic thumbnail. That is, as shown in FIG. 7A, when a person's face is photographed up, it is not necessary to create a face thumbnail. In this case, a panoramic view thumbnail is created.

  A panoramic view thumbnail is generated by resizing the entire main image to a predetermined size. In this case, first, the image data of the main image stored in the memory 120 is added to the image signal processing unit 134. The image signal processing unit 134 resizes the image data of the main image to a predetermined size in accordance with a command from the CPU 110, and generates image data of a panoramic view thumbnail. The generated whole-view thumbnail image data is stored in the memory 120.

  On the other hand, the face thumbnail is generated by cutting a person's face shown in the main image to a predetermined size and resizing the cut image to a predetermined size.

  In this case, first, the image data of the main image stored in the memory 120 is added to the image signal processing unit 134. The image signal processing unit 134 cuts out a person captured in the main image with a predetermined size in accordance with a command from the CPU 110, and generates face image image data for a face thumbnail (see FIG. 5B).

  At this time, the CPU 110 calculates an image clipping region for clipping a human face imaged in the main image with a predetermined size from the detection result of the face detection unit 154, and the calculated image clipping region is an image signal processing unit. It specifies to 134. The image signal processing unit 134 cuts the main image in the image cut region designated by the CPU 110, and generates face image image data for the face thumbnail (see FIG. 5B).

  As shown in FIG. 6A, when a plurality of persons are captured in the main image and the faces of the plurality of persons are detected by the face detection unit 154, the CPU 110 displays the information shown in FIG. 6B. As described above, the face area of the person closest to the center of the main image among the face areas of the person photographed with a predetermined size or less is recognized as the face area for image cutting.

  Further, as shown in FIGS. 5A and 6A, the image cut-out area is set to an image cut-out area that is three times as large as the detected face area. When the face is located at the edge of the image, the cropping area is set by shifting to the center side of the image as necessary.

  Image data of the generated face image (see FIG. 5B and FIG. 6B) is temporarily stored in the memory 120 and then added to the image signal processing unit 134 again. The image signal processing unit 134 resizes the image data of the face image clipped in accordance with a command from the CPU 110 to generate a face thumbnail (see FIG. 5C and FIG. 6C). The generated face thumbnail image data is stored in the memory 120.

  The image data of the face thumbnail or the panoramic thumbnail generated in this way is added to the compression / decompression processing unit 148, compressed according to a predetermined compression format (for example, JPEG format), and then stored in the memory 120. Then, it is stored in the image file together with the image data of the main image, and then recorded on the storage medium 152 via the media control unit 150.

  FIG. 8 is a flowchart showing a procedure of processing from photographing to recording in the digital camera 10 of the present embodiment.

  First, it is determined whether or not shooting is performed (step S10). As a result of this determination, when shooting is executed, the face of the person shown in the shot image (main image) is extracted, and its position and size are detected (step S11). Then, from the detection result, it is determined whether or not a person's face is copied in the main image (step S12).

  As a result of this determination, if it is determined that the person's face is not copied, a panoramic view thumbnail creation process is performed (step S13).

  On the other hand, if it is determined that the face of the person is copied, it is determined whether the size of the face of the person is equal to or smaller than a certain size (step S14). If the face size of the person is equal to or smaller than a certain size, face thumbnail creation processing is performed (step S15). If the face size of the person is larger than the certain size, a panoramic view thumbnail is created. Processing is performed (step S13).

  Then, an image file is created from the created full-view or face thumbnail data and the image data of the main image (step S16), and the created image file is recorded in the storage medium 152 (step S17).

  As described above, in the digital camera 10 according to the present embodiment, when a person's face is captured in a photographed image (main image) with a predetermined size or less, “face thumbnail” is added as a thumbnail. An image file is created. If a person's face is not copied, or is copied, but the person's face is larger than a predetermined size, an image file with a “full view thumbnail” added as a thumbnail Created.

  Therefore, when the image file created by the digital camera 10 of the present embodiment is displayed as a thumbnail, the face image of the person captured in the image is enlarged and displayed for the image file to which “face thumbnail” is added. A thumbnail is displayed. As a result, it is possible to grasp who is in the main image and what kind of expression is in the image by simply looking at the thumbnail.

  In the digital camera 10 of the present embodiment, when a person's face is captured in a photographed image with a predetermined size or less, a “face thumbnail” is created and the “face thumbnail” is converted into an image file. If a person's face is captured in a photographed image with a predetermined size or less, both a “face thumbnail” and a “entire view thumbnail” are created as shown in FIG. May be recorded in the image file.

  Further, in the digital camera 10 of the present embodiment, when a plurality of human faces are captured in the captured image, the face area of the person closest to the center of the main image is recognized as the face area for image cutting. Although only the “face thumbnail” of the face area is created, as shown in FIG. 9, “face thumbnails” of all detected persons may be created and recorded in the image file. .

  Next, a method for reproducing recorded images in the digital camera of this embodiment will be described.

  As described above, in the digital camera 10 of the present embodiment, the recorded image is reproduced on the monitor 32 by setting the power / mode switch 28 to the reproduction position and setting the mode of the digital camera 10 to the reproduction mode. Can be displayed.

  Here, in the digital camera 10 according to the present embodiment, as the image reproduction method, a “main image reproduction mode” for reproducing the main image, a “thumbnail reproduction mode” for reproducing the thumbnails, and a list of thumbnails of a plurality of frames are displayed. There are three playback modes of “multi playback mode”, and the user selects one of the playback modes and causes the monitor 32 to play back the image. These three playback modes can be switched by pressing the DISP button 44, for example.

  First, a method for reproducing recorded image images in the main image reproduction mode will be described. When the main image reproduction mode is set, the CPU 110 outputs a command to the media control unit 150, and finally reads the compressed image data of the main image of the image file recorded on the storage medium 152. The read compressed image data of the main image is added to the compression / decompression processing unit 148, and is expanded to non-compressed image data. Then, the image signal processing unit 134 performs a reduction process for display, and the video encoder 136 Is output to the monitor 32 via As a result, the main image recorded in the image file recorded last on the storage medium 152 is reproduced and displayed on the monitor 32.

  The frame advance of the image is performed by operating the left and right keys of the cross button 40. When the right key of the cross button 40 is pressed, the image data of the main image of the next image file is read from the storage medium 152, The process is performed and reproduced and displayed on the monitor 32. When the left key of the cross button 40 is pressed, the image data of the main image of the previous image file is read from the storage medium 152, subjected to necessary processing, and reproduced and displayed on the monitor 32.

  Next, a method for reproducing recorded image images in the thumbnail reproduction mode will be described. When the thumbnail playback mode is set, the CPU 110 outputs a command to the media control unit 150 to read the compressed image data of the thumbnail of the image file recorded on the storage medium 152 last.

  The compressed image data of the read thumbnail is added to the compression / decompression processing unit 148, and is decompressed to non-compressed image data. Then, the image signal processing unit 134 performs an enlargement process for display, and the video encoder 136 Is output to the monitor 32 via As a result, the thumbnail recorded in the image file last recorded in the storage medium 152 is reproduced and displayed on the monitor 32. At this time, as shown in FIG. 11, when a face thumbnail is recorded in the image file, the face thumbnail is displayed on the monitor 32, and when a full view thumbnail is recorded, the monitor displays a full view thumbnail on 32. Is done. For the face thumbnail, a frame is attached around the image to distinguish it as a face thumbnail.

  The frame advance of the image is performed by the left and right key operations of the cross button 40 as described above. When the right key of the cross button 40 is pressed, the thumbnail image data recorded in the next image file is transferred from the storage medium 152. It is read out, subjected to necessary processing, and reproduced and displayed on the monitor 32. When the left key of the cross button 40 is pressed, the thumbnail image data recorded in the previous image file is read from the storage medium 152, subjected to the necessary processing, and reproduced and displayed on the monitor 32. Is done.

  As described above, in the thumbnail reproduction mode, the thumbnails recorded in the image file are displayed, so that it is possible to easily check who is shown in each image.

  In this thumbnail playback mode, when the MENU / OK button 42 is pressed while a desired thumbnail is displayed, the main image of the thumbnail may be played back.

  In addition, when a desired thumbnail is displayed and a non-operation state continues for a certain period of time, the main image of the currently displayed thumbnail may be automatically reproduced.

  When a plurality of face thumbnails are recorded in the image file, the thumbnails are displayed by switching at regular intervals. For example, as shown in FIG. 12, when two face thumbnails of face thumbnail 1 and face thumbnail 2 are recorded in the image file, the monitor is switched between face thumbnail 1 and face thumbnail 2 each time a certain time elapses. 32.

  Hereinafter, a processing procedure in the case of reproducing an image file in which a plurality of face thumbnails are recorded will be described.

  FIG. 13 is a flowchart showing the procedure of a reproduction process of an image file including an image file in which a plurality of thumbnails are recorded.

  First, all the thumbnail image data recorded in the image file is read (step S21). Then, it is determined whether there are a plurality of thumbnails based on the read thumbnail image data (step S22).

  If it is determined that the number of read thumbnails is one, the one read thumbnail is displayed on the monitor 32 (step S23). Then, the presence / absence of a display end instruction from the user is determined (step S24). When it is determined that the display end instruction ant is present, the reproduction process ends.

  On the other hand, if it is determined that there are a plurality of read thumbnails, first, the top thumbnail is displayed on the monitor 32 (step S25). Then, the presence / absence of a display end instruction from the user is determined (step S26). When it is determined that the display end instruction ant is present, the reproduction process is ended.

  If it is determined that there is no instruction to end the display from the user, it is next determined whether or not a fixed time has elapsed from the start of displaying the thumbnail that was reproduced first (step S27). If it is determined that the predetermined time has not elapsed, the process returns to step S25, and the thumbnails reproduced earlier are continuously displayed.

  On the other hand, if it is determined that a predetermined time has elapsed since the start of displaying the previously played thumbnail, the thumbnail display is switched (step S28), the next thumbnail is displayed, and the process returns to step S25.

  As described above, when a plurality of face thumbnails are recorded in the image file, the thumbnails are switched at regular intervals. Thereby, it is possible to easily confirm who is shown in the main image and what kind of expression.

  When a plurality of face thumbnails are recorded in the image file, a plurality of face thumbnails may be displayed at a time as shown in FIG. FIG. 14 shows an example in which two face thumbnails of face thumbnail 1 and face thumbnail 2 are recorded in the image file, and the two face thumbnails are displayed in parallel. Also in this case, it is possible to easily confirm who is shown in the main image and what kind of expression.

  Similarly, when both the face thumbnail and the panoramic thumbnail are recorded in the image file, the face thumbnail and the panoramic thumbnail are switched and displayed on the monitor 32 at regular intervals. Alternatively, the face thumbnail and the full view thumbnail are displayed at a time. As a result, it is possible to easily confirm who the whole view and the person photographed there are.

  Next, a method for reproducing recorded image images in the multi-reproduction mode will be described. In the multi-playback mode, for example, nine frames (3 × 3) of images are displayed as a list on the monitor 32. When the multi playback mode is set, the CPU 110 outputs a command to the media control unit 150 to read out the compressed image data of 9 frames of thumbnails.

  The compressed image data of each read thumbnail is added to the compression / decompression processing unit 148, and after being decompressed to uncompressed image data, the image signal processing unit 134 performs a reduction process for display, and a synthesis processing unit 138. The composition processing unit 138 generates a multi-display image from the input thumbnail image data, and outputs it to the monitor 32 via the video encoder 136. Thereby, as shown in FIG. 15, thumbnails for nine frames are displayed in a list on the monitor 32. At this time, for the face thumbnail, a frame is added around the image to distinguish it as a face thumbnail.

  In the multi-playback mode, when the right key of the cross button 40 is pressed, thumbnail image data for the next nine frames are read from the storage medium 152, subjected to necessary processing, and listed on the monitor 32. Is displayed. When the left key of the cross button 40 is pressed, the thumbnail image data for the previous nine frames are read from the storage medium 152, subjected to necessary processing, and displayed as a list on the monitor 32.

  Further, in the multi-playback mode, when a plurality of face thumbnails are recorded in the image file, the face thumbnails are switched and displayed at regular intervals for that frame. Similarly, when both the face thumbnail and the panoramic thumbnail are recorded in the image file, the face thumbnail and the panoramic thumbnail are switched and displayed on the monitor 32 for the frame at regular intervals.

  In this multi-playback mode, one of the thumbnails displayed in a list may be selected with a cursor or the like, and the main image of the selected thumbnail may be switched and played back.

  In the digital camera 10 described above, a face thumbnail is generated from a photographed image. However, a face thumbnail may be generated from a recorded image and rewritten with a recorded panoramic thumbnail and recorded in an image file. . Hereinafter, a processing procedure when generating a face thumbnail from the recorded image will be described.

  For the creation of the face thumbnail, for example, the thumbnail of the main image is played under the thumbnail playback mode, and the execution of the face thumbnail is instructed when the thumbnail of the main image to be processed is displayed.

  FIG. 16 is a flowchart illustrating a processing procedure when a face thumbnail is generated from a recorded image.

  First, it is determined whether or not the thumbnail reproduction mode is set (step S30). If it is determined that the thumbnail playback mode is set, the thumbnails recorded in the image file last recorded on the storage medium 152 are played back and displayed on the monitor 32 (step S31).

  When face thumbnail creation processing is performed on the main thumbnail image displayed on the monitor 32, the camera is instructed to execute face thumbnail creation processing. Here, the execution instruction of the face thumbnail creation process is performed on the reproduction menu, and the function item is selected from the reproduction menu prepared in advance, and the execution of the process is instructed.

  The CPU 110 determines whether or not there is an instruction to execute the face thumbnail creation process (step S32). If the CPU 110 determines that no face thumbnail creation processing instruction has been issued for the main image on which the thumbnail is currently displayed, the CPU 110 determines whether or not there is a frame advance or frame reverse instruction (step S33). Alternatively, if it is determined that the frame return instruction ant is detected, frame advance or frame return processing is executed (step S34).

  On the other hand, if it is determined that the execution instruction ant of the face thumbnail creation process is selected, the face of the person shown in the main image of the thumbnail currently displayed on the monitor 32 is extracted, and its position and size are detected (step S35). ). Then, based on the detection result, it is determined whether or not a person's face is captured in the main image (step S36). If it is determined that the person's face is not captured, the process is terminated.

  On the other hand, if it is determined that the person's face is photographed, it is determined whether or not the size of the person's face is equal to or smaller than a certain size (step S37). If it is determined that the size of the person's face is larger than a certain size, the process ends.

  On the other hand, when it is determined that the size of the person's face is equal to or smaller than a certain size, a process of creating a face thumbnail is performed (step S38). Then, the thumbnail image data currently recorded in the image file is rewritten with the newly created face thumbnail image data (step S39).

  In this way, a new face thumbnail may be created for a recorded image file.

  When a plurality of human faces are copied in the main image to be processed, a face thumbnail is created with the face of the person closest to the center of the main image. Alternatively, a face thumbnail is created with each person's face.

  Also, both the newly created face thumbnail and full view thumbnail may be recorded in the image file.

  In the above embodiment, an image file to be processed is selected under the thumbnail playback mode. However, an image file to be processed is selected under the main image playback mode or the multi playback mode. You may do it. Alternatively, face thumbnail creation processing may be executed collectively on image files stored in the storage medium 152.

  In the above series of embodiments, the case where the present invention is applied to a digital camera has been described as an example. However, the application of the present invention is not limited to this, and for example, a personal computer according to the present invention is applied. You may make it function as an image processing apparatus. Hereinafter, a case where a personal computer (personal computer) functions as the image processing apparatus according to the present invention will be described.

  FIG. 17 is a block diagram illustrating a hardware configuration example of a personal computer that functions as an image processing apparatus according to the present invention.

  The personal computer 210 is a commonly used type of computer, a central processing unit (CPU) 212 for executing a program, a RAM 214 for temporarily storing calculation results, image data, and the like in the CPU 212, an operating system, and the like. A hard disk drive (HDD) 216 for storing a system (OS), an image classification program according to the present invention, various application programs, image files, etc., a CD-ROM drive 218 for reading data stored in a CD-ROM, A memory card for connecting a display device 220 for displaying calculation results and image data in the CPU 212, a keyboard for inputting commands, numerical values, etc., an input device 222 such as a mouse, and a memory card 224 storing image files. The 226, an input / output terminal (for example, a USB terminal) 230 for connecting to a digital camera or the like via a communication cable, a modem 232 for communicating with a server through a network, and the like, which are mutually connected to a bus 234. Connected with.

  Now, the personal computer 210 configured as described above functions as an image processing apparatus by executing a predetermined image processing program. The image processing program is installed in the hard disk drive 216, loaded from the hard disk drive 216 to the RAM 214, and executed and controlled by the CPU 212.

  When the image processing program is activated, a main window of the image processing program is displayed on the display device 220 as shown in FIG.

  The main window includes a folder tree display area displayed on the left side of the screen and a thumbnail display area displayed on the right side of the screen.

  In the folder tree display area, folders in the personal computer 210 are hierarchically displayed. When a folder displayed in the folder tree display area is selected, thumbnails of image files stored in the selected folder are displayed as a list in the thumbnail display area as shown in FIG.

  If there is no image file in the selected folder, nothing is displayed in the thumbnail display area, and if there is another folder in the selected folder, a folder icon is displayed.

  In this image processing program, when an arbitrary thumbnail is selected from the thumbnails displayed in the thumbnail display area and double-clicked, the main image of the selected thumbnail is enlarged and displayed in another window. When the slide show function is executed, the image files stored in the selected folder are frame-advanced at regular time intervals to perform the slide show.

  When the face thumbnail creation process is executed by this image processing program, the thumbnail to be processed is selected and the execution of the face thumbnail creation process is instructed.

  In order to execute the face thumbnail creation process, first, the mouse pointer is moved over the thumbnail to be processed, and the mouse is right-clicked. This pulls down the file operation menu. When the mouse pointer is moved on the face thumbnail creation item from the pull-down menu and the mouse is left-clicked, the selected process is executed. That is, face thumbnail creation processing is executed.

  First, the face of a person photographed in the main image of the selected thumbnail is extracted, and its position and size are detected. Then, it is determined from the detection result whether or not a person's face is copied in the main image.

  If it is determined that the person's face is not copied, the process ends. If it is determined that the person's face is copied, it is determined whether the size of the person's face is equal to or smaller than a certain size. Is done. If it is determined that the size of the person's face is larger than a certain size, the process is terminated. If it is determined that the size of the person's face is equal to or smaller than the certain size, a face thumbnail creation process is performed.

  Then, the thumbnail image data recorded in the image file is rewritten with the newly created face thumbnail image data.

  When a plurality of human faces are captured in the main image to be processed, a face thumbnail is created with the face of the person closest to the center of the main image, as in the case of the digital camera 10. Alternatively, a face thumbnail is created with each person's face.

  Also, both the newly created face thumbnail and full view thumbnail may be recorded in the image file.

  Further, when displaying an image file in which the face thumbnails created as described above are recorded as thumbnails, the face thumbnails are displayed on the display device 220.

  In this case, when displaying an image file in which a plurality of face thumbnails are recorded or an image file in which both a face thumbnail and a panoramic thumbnail are recorded, the unit time is the same as in the case of the digital camera 10 described above. Switch the thumbnails to be displayed every time. Alternatively, all recorded thumbnails are displayed at once.

  As described above, a personal computer can function as the image processing apparatus according to the present invention to create a face thumbnail. In addition, the present invention can be applied to a digital video camera, a mobile phone with a camera, a PDA with a camera, and the like.

  In the above-described series of embodiments, the case where the present invention is applied to still image processing has been described as an example, but the same can be applied to moving image processing.

Front perspective view of a digital camera to which the present invention is applied Rear view of a digital camera to which the present invention is applied Block diagram showing schematic configuration inside digital camera Diagram showing image file structure Figure showing the procedure for creating a face thumbnail Figure showing the procedure for creating a face thumbnail Diagram showing the procedure for creating a panoramic thumbnail Flow chart showing the procedure of thumbnail creation processing Conceptual diagram when both face thumbnails and panoramic thumbnails are recorded in an image file Conceptual diagram when recording multiple face thumbnails in an image file Illustration of how to play face thumbnails and panoramic thumbnails Explanatory diagram of how to play an image file with multiple face thumbnails recorded The flowchart which shows the procedure of the reproduction | regeneration processing of the image file containing the image file in which the several face thumbnail was recorded Explanatory diagram of how to play an image file with multiple face thumbnails recorded Illustration of playback method in multi playback mode A flowchart showing a processing procedure when generating a face thumbnail from a recorded image Block diagram showing the hardware configuration of a personal computer The figure which shows the main window of the image processing program

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Digital camera, 12 ... Camera body, 14 ... Lens, 16 ... Strobe, 18 ... Finder window, 20 ... Self-timer lamp, 22 ... AF auxiliary light lamp, 24 ... Strobe light control sensor, 26 ... Shutter button, 28 ... Power / mode switch, 30 ... mode dial, 32 ... monitor, 34 ... finder eyepiece, 36 ... speaker, 38 ... zoom button, 40 ... cross button, 42 ... MENU / OK button, 44 ... DISP button, 46 ... BACK 110, CPU, 112, operation unit, 114, bus, 116, ROM, 118, EEPROM, 120, memory (SDRAM), 122, VRAM, 124, solid-state image sensor (CCD), 126, timing generator (TG) 128 ... Analog processing unit (CDS / AMP), 130 ... A / D 132: Image input control unit, 134: Image signal processing unit, 136: Video encoder, 138: Composition processing unit, 140: AF detection unit, 142: AE / AWB detection unit, 144: Aperture drive unit, 146 ... Lens drive unit, 148 ... compression / decompression processing unit, 150 ... media control unit, 152 ... storage medium, 154 ... face detection unit

Claims (12)

A face detection means for detecting a human face from an input image;
Calculating means for calculating a ratio of the face size of the person detected by the face detecting means to the entire input image;
An image cutting means for cutting a face of a person detected by the face detection means from the input image in a predetermined size when the ratio calculated by the calculation means is lower than a preset reference value ; ,
Face thumbnail generating means for generating a face thumbnail of a predetermined size from the face image cut out by the image cutting means;
Image file generating means for generating an image file including image data of the input image and image data of the face thumbnail;
Means for recording the image file generated by the image file generation means on a storage medium;
An image processing apparatus comprising:   The said image cut-out means cuts out the face of the person nearest to the center of the input image with a predetermined size when the face detection means detects a plurality of human faces. Image processing device.   When a plurality of human faces are detected by the face detection means, the image cutout means cuts each person's face detected by the face detection means in a predetermined size, and the face thumbnail generation means Generating a face thumbnail for each face image clipped by the image clipping means, and the image file generating means generates an image file including the generated plurality of face thumbnail image data and the input image image data. The image processing apparatus according to claim 1, wherein:   A panoramic thumbnail generating unit configured to generate a panoramic thumbnail of a predetermined size from the input image, the image file generating unit including image data of the face thumbnail, image data of the panoramic thumbnail, and image data of the input image; The image processing apparatus according to claim 1, wherein the image processing apparatus generates an image file. Acquiring an image;
Detecting a person's face from the acquired image;
Calculating the proportion of the detected face size of the person in the entire acquired image;
Cutting the detected person's face in a predetermined size from the acquired image when the calculated ratio falls below a preset reference value;
Generating a face thumbnail of a predetermined size from the cut face image;
Generating an image file including image data of the acquired image and image data of the face thumbnail;
Recording the generated image file on a storage medium;
An image processing method comprising: When a plurality of human faces are detected in the step of detecting the human faces, the step of cutting the human faces in a predetermined size includes a step of cutting a human face closest to the center of the acquired image to a predetermined size. The image processing method according to claim 5, wherein the image processing method is cut off. If a plurality of human faces are detected in the step of detecting the human faces,
The step of cutting out the face of the person by a predetermined size cuts out the face of each detected person by a predetermined size,
The step of creating the face thumbnail generates a face thumbnail for each cut face image,
The image processing method according to claim 6, wherein the generating the image file includes generating an image file including the generated image data of the plurality of face thumbnails and the image data of the acquired image. Generating a panoramic thumbnail of a predetermined size from the acquired image,
The step of generating the image file generates an image file including the image data of the face thumbnail, the image data of the panoramic view thumbnail, and the image data of the acquired image. Or the image processing method of 7. Acquiring an image;
Detecting a person's face from the acquired image;
Calculating the proportion of the detected face size of the person in the entire acquired image;
Cutting the detected person's face in a predetermined size from the acquired image when the calculated ratio falls below a preset reference value;
Generating a face thumbnail of a predetermined size from the cut face image;
Generating an image file including image data of the acquired image and image data of the face thumbnail;
Recording the generated image file on a storage medium;
An image processing program for causing a computer to execute. When a plurality of human faces are detected in the step of detecting the human faces, the step of cutting the human faces in a predetermined size includes a step of cutting a human face closest to the center of the acquired image to a predetermined size. The image processing program according to claim 9, wherein the image processing program is cut off. If a plurality of human faces are detected in the step of detecting the human faces,
The step of cutting out the face of the person by a predetermined size cuts out the face of each detected person by a predetermined size,
The step of creating the face thumbnail generates a face thumbnail for each cut face image,
The image processing program according to claim 9, wherein the step of generating the image file generates an image file including the generated image data of the plurality of face thumbnails and the image data of the acquired image. Generating a panoramic thumbnail of a predetermined size from the acquired image,
The step of generating the image file generates an image file including image data of the face thumbnail, image data of the panoramic view thumbnail, and image data of the acquired image. Or the image processing program of 11.

Images