JP2012256327A - Image display apparatus, image display method, and image display program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display apparatus, an image display method and an image display program which can quickly perform a search when searching for an image from a plurality of images and displaying the image.SOLUTION: A smiling level is determined of a face which is located in a screen center of each of a plurality of images (S63), a representative image is searched for from the plurality of images on the basis of the determined smiling level and then it is displayed (S73). Furthermore, when searching for the representative image, a similar image is rejected at S71 in addition to selection by the smiling level (S71).

Description

  The present invention relates to an image display device, an image display method, and an image display program, and more specifically, an image display device and an image display method for retrieving a representative image from a plurality of photographed images and displaying the representative image. And an image display program.

  In recent years, due to technological innovations in digital cameras, high sensitivity and expansion of the exposure control range have been progressed, and the capacity of recording media has been increasing. Furthermore, compared to cameras in the film era, the continuous shooting function is also greatly improved, so that camera users can enjoy taking pictures easily and obtain a large amount of images. In recent years, a camera equipped with a function for determining the facial expression of a person as a subject has appeared, and it has become easier to take a picture of a smile. Furthermore, the development of technology for identifying individual faces and determining who the face is is also progressing.

As described above, a large amount of images can be obtained. However, when viewing, it is difficult to find an image to be viewed. Therefore, it is conceivable to search based on the content of the image, such as determining the facial expression of the person using the image data. For example, Patent Document 1 discloses an electronic camera that determines whether a person is a registered person by comparing face recognition data based on the feature points of the person's face and face recognition data registered in advance. It is disclosed. Patent Document 2 discloses an image evaluation apparatus that performs smile evaluation and neatness evaluation for each face image, and allows a subject to easily enjoy an image with a good facial expression.
JP 2007-150604 A JP 2004-46591 A

  The electronic camera disclosed in Patent Document 1 can recognize only a person registered in advance, and has a problem in simplicity and high speed such as a registration procedure. In addition, the image evaluation apparatus disclosed in Patent Document 2 requires scene analysis such as casual and formal, and it takes time to perform image analysis for this. Searching and comparing all faces of all images takes time to search and judge, and cannot immediately start viewing.

  The present invention has been made in view of such circumstances, and an image display device, an image display method, and an image that can be quickly searched for searching for and displaying an image from a plurality of images. An object is to provide a display program.

In order to achieve the above object, an image display apparatus according to a first invention is an image display apparatus that selects and displays a representative image from a plurality of images including a face image, and includes a face image included in each of the plurality of images. A selection unit that selects a representative image based on the degree of smile and composition, a search unit that searches for an image in which a person different from the representative image is smiling as a next representative image, a display control unit that displays the representative image,
It is provided with.

  In order to achieve the above object, an image display device according to a second invention is the image display device according to the first invention, wherein the plurality of images are photographed images, and the smile level and the characteristics of each person at the time of the photographing. Is an image associated with.

  In order to achieve the above object, an image display device according to a third invention is the image display device according to the first invention, wherein the display control unit is configured to store a person who has a high degree of smile between the representative image and the next representative image. It is characterized by displaying images side by side.

  In order to achieve the above object, an image display method according to a fourth aspect of the present invention is an image display method for selecting and displaying a representative image from a plurality of images including a face image, wherein a face included in each of the plurality of images is displayed. Selecting a representative image based on the degree of smile and composition, searching for an image in which a person different from the representative image is smiling as a next representative image, and displaying the representative image. Features.

  In order to achieve the above object, an image display method according to a fifth invention is the image display method according to the fourth invention, wherein the plurality of images are photographed images, and the smile level and the characteristics of each person at the time of the photographing. Is an image associated with.

  In order to achieve the above object, an image display method according to a sixth invention is the image display method according to the fourth invention, wherein the display step includes an image of a person with a high degree of smile of the representative image and the next representative image. Are displayed side by side.

  In order to achieve the above object, an image display program according to a seventh aspect of the present invention is an image display program for selecting and displaying a representative image from a plurality of images including a face image, wherein the face display included in each of the plurality of images is displayed. Selecting a representative image based on the degree of smile and composition, searching for an image in which a person different from the representative image is smiling as a next representative image, and displaying the representative image. Features.

  In order to achieve the above object, an image display program according to an eighth invention is the image display program according to the seventh invention, wherein the plurality of images are photographed images, and the smile level and the characteristics of each person at the time of photographing. Is an image associated with.

  In order to achieve the above object, an image display program according to a ninth invention is the image display program according to the seventh invention, wherein the display step includes an image of a person with a high degree of smile between the representative image and the next representative image. Are displayed side by side.

  According to the present invention, it is possible to provide an image display device, an image display method, and an image display program capable of quickly performing a search when displaying and displaying an image from a plurality of images.

  Hereinafter, a preferred embodiment will be described using a camera to which the present invention is applied according to the drawings. The camera in the present embodiment records the position and number of faces and the degree of smile at the time of shooting, and the degree of smile of the face located at the center of the screen from a plurality of taken pictures at the time of image display reproduction. In addition, an image in which many people are photographed together is displayed as a representative image.

  FIG. 1 is a block diagram showing a configuration of a system including a camera 10 and a television 20 according to the first embodiment of the present invention. The camera 10 is a digital camera, and includes a control unit 1, an imaging unit 2, an optical system control unit 3, a recording unit 4, a face detection unit 5, a cutting unit 6, a composition processing unit 7, a display unit 8, a display control unit 8a, It has a clock unit 9, an operation determination unit 11, a smile level determination unit 12, and a communication unit 14, and can be connected to an external television 20 via wired communication or wireless communication.

  The control unit 1 is connected to each unit in the camera 10 and performs overall control of the camera 10. The control unit 1 includes an image processing unit 1a, an event classification unit 1b, a face classification unit 1c, and a display order determination unit 1d. The image processing unit 1a performs various types of image processing such as compression / decompression processing of images input from the imaging unit 2 or the like.

  In addition, the event classification unit 1b determines which event the image belongs to according to the shooting frequency based on the shooting date / time information of the image. The event classification will be described later with reference to FIGS. The face classification unit 1c detects a portion corresponding to the face of the person detected by the face detection unit 5, and performs classification based on the feature of the face based on the detected portion. The display order determination unit 1d determines the display order based on the face classification by the face classification unit 1c and the smile level by the smile level determination unit 12.

  The imaging unit 2 includes an optical system for forming a subject image, an imaging element that photoelectrically converts the subject image, and the like, and outputs image data. The optical system control unit 3 performs control such as focusing of the optical system in the imaging unit 2. The recording unit 4 compresses the image data output from the imaging unit 2 by the image processing unit 1a in the control unit 1, and records this image data as a captured image. In the recording unit 4, an image vs face storage unit 4a is provided. The image vs face storage unit pairs image data with information on what kind of face is included in the image. Record.

  The face detection unit 5 determines a face pattern and detects a face portion when a captured image includes a human image. Based on the detection result, the optical system control unit 3 performs exposure control, automatic focus adjustment (AF), and the like. The face detection unit 5 includes a position detection unit 5a, a size determination unit 5b, and a feature unit 5c. The position detection unit 5a detects the position of the person's face detected by the face detection unit 5 on the screen. The size determination unit 5b determines the size of the detected face portion. The feature unit 5c detects the feature amount of the face in order to identify the individual.

  The cutout unit 6 cuts out image data corresponding to the face portion of the human image detected by the face detection unit 5. The face image data cut out here is used in the image synthesizing process in the synthesizing processing unit 7. The composition processing unit 7 synthesizes the face image data cut out by the cut-out unit 6 and performs image composition like an image C (FIG. 1) described later.

  The display unit 8 is configured by a display device such as a liquid crystal monitor disposed on the back surface of the camera 10, and displays and reproduces image data recorded in the recording unit 4. In addition, the subject image is displayed instead of the optical viewfinder. Live view display for observation. In addition, normal reproduction of the captured image recorded in the recording unit 4 and reproduction display in a slide show format are performed. The display control unit 8a performs display control of the display unit 8.

  The clock unit 9 has a timekeeping function and a calendar function, and is used for acquiring date and time information at the time of shooting. The date and time information obtained here is recorded as a file together with the photographed image when the photographed image is recorded in the recording unit 4. The operation determination unit 11 determines an operation state of an operation member or the like of the camera 10 by the user. In accordance with this operation state, the control unit 1 controls the camera 10 as described above.

  The smile degree determination unit 12 quantifies the degree of smile of the person's face detected by the face detection unit 5. The determination of the smile level will be described later with reference to FIGS. The communication unit 14 performs transmission / reception with the external television 20 by wired communication or wireless communication. The photographed image recorded in the recording unit 4 can be transmitted to the outside via the communication unit 14.

  Next, before describing the detailed operation of the camera 10 according to the present embodiment configured as described above, a basic method for reproducing and displaying a captured image will be described. As a feature of this embodiment, an image that the user most wants to see is displayed immediately. The photographer shoots a large number of images. Among them, there are images that look good with a smile or the like, and the same person may be captured many times.

  Therefore, when reproducing and displaying on the display unit 8 and the television 20, only the similar photos as shown in the photo group A are not displayed. In the case of similar photos, the best photo B1 is selected. Select to display. When switching to the next image for viewing by a slide show or the like, a photo B2 showing a different person is reproduced and displayed, avoiding a photo showing the same person.

  In addition, as in the image C, only the face of the center person in the image is cut out by the cutout unit 6, and the plurality of cut out faces are combined and displayed by the combination processing unit 7. Is possible.

  The detailed operation of the camera 10 according to the present embodiment along the basic manner as described above will be described with reference to the flowchart shown in FIG. FIG. 2 is a flowchart showing the camera control operation of the camera 10. When this flow is entered, it is first determined whether or not the photographer has set the photographing mode (S1). As a result of the determination, if the shooting mode is set, an image is taken and a through image (also referred to as a live view display) is displayed (S2). In this step, imaging is performed by the imaging unit 2, and the acquired image data of the subject is displayed on the display unit 8 as a through image.

  Subsequently, the face position, size, and number are detected (S3). In this step, the face detection unit 5 detects the position of the face from the subject image using the image data acquired by the imaging unit 2, detects the number of faces in the subject image, and determines each face. Determine the size. Next, smile level determination and determination of the feature of the face located at the center of the screen are performed (S4).

  The determination of the smile level in step S4 will be described later with reference to FIGS. The feature of the face located at the center of the screen is grasped by, for example, linearly approximating the shadow of the face and using the coordinates of the intersection. The example shown in FIG. 10 is characterized by the intersections of three straight lines 81, 82, 83, and 84 surrounding the eyes. By detecting the coordinates of this feature point for each face and recording it for each image, it is determined whether or not the faces of the same person are based on the similarity of the distribution of the feature points of the facial parts of each image can do.

  When the determination of the smile level in step S4 is completed, the release button is then operated to determine whether or not to shoot (S5). If the result of determination in step S5 is that no shooting operation is to be carried out, processing returns to step S1 and the aforementioned operation is performed. In the present embodiment, when the shooting mode is selected and the shooting operation is not performed, the position, size, and number of faces of the subject image acquired by the imaging unit 2 are detected, and the determination of the smile level and the face The feature points are repeatedly detected and executed.

  If the result of determination in step S5 is that the release button has been operated, a shooting operation is executed (S6). In this step, shooting is performed by the imaging unit 2, and image data of a still image is acquired. When the image data of the photographed image is acquired, the number of faces included in the image detected by the face detection unit 5 in step S3 immediately before photographing is tabulated (S7), and face detection is performed in steps S3 and S4. The face position, size, smile level, and features detected by the unit 5 are tabulated (S8).

  In the table created in steps S7 and S8, as shown in FIG. 11A, the number of faces is first recorded for each image (image 1 to image 4 in the example shown in FIG. 11). The event name described later is recorded next to it. Then, for the face of a person near the center, the facial feature, position, size, and smile level are recorded. The position, size, and smile level are recorded for the right face near the center, and the position, size, and size for the left face near the center, other faces near the center, and so on, are sequentially displayed. The smile level is recorded.

  In the present embodiment, the amount of information is increased for the face of the person at the center of the screen, and the amount of information is reduced for the faces of surrounding persons. Although the amount of information on the faces of the surrounding people may be the same as that in the center, the processing time is increased by increasing the amount of information. Therefore, in this embodiment, the amount of information on the periphery is reduced. The smile level may be omitted for faces other than the center. By omitting, the amount of information is reduced, but the processing time can be shortened.

  Next, the photographing result and the table are recorded (S9). In this step, the table created in steps S7 and S8 is paired with the photographing result as shown in FIG. 11B (only the image 1 and the image 2 are shown in the example of FIG. 11B). The address recorded in the table in the header part of the data is recorded in the image vs face recording unit 4a. When the photographing result and the table recording are finished, the process returns and the process is executed from step S1. Hereinafter, “return” means processing to return to step S1.

  As described above, in this embodiment, when the shooting mode is entered, the number and positions of faces are detected in real time using image data for live view display, and the release operation is performed during the shooting operation by operating the release button. The number, position, etc. of the faces detected immediately before are recorded together with the captured image.

  If the result of determination in step S1 is not shooting mode, it is next determined whether or not normal playback is to be carried out (S11). In the present embodiment, two types of reproduction, that is, normal reproduction and smile search are prepared for reproduction display of the captured image. Normal reproduction is a known reproduction display, and is a mode in which a photographed image recorded in the recording unit 4 is selected and displayed.

  Smile search is a characteristic reproduction display in the present embodiment. An image with a high degree of smile is searched from captured images, this image is displayed as shown in photo B1 in FIG. 1, and the next image is displayed. When doing so, the photograph B2 in which a person other than the person in the photograph B1 is shown is displayed. These reproduction displays are performed by the operation determination unit 11 determining the operation state of the operation member.

  If the result of determination in step S11 is normal playback, a known playback display is performed (S12). On the other hand, if the result of determination in step S11 is not normal playback, it is determined whether or not search playback is to be performed, that is, whether or not playback display is performed by a smile search (S13). If the result of determination in step S13 is not search playback, event classification is performed (S15).

  The event classification in step S15 will be described later with reference to FIGS. 3 and 4. In this step, it is determined whether the photographed image belongs to a school display such as an athletic meet or a party such as a wedding. When the event classification is completed, the process returns to step S1 and the above steps are repeated. As described above, in this embodiment, when any of the shooting mode, normal playback mode, and search playback mode is not selected, event classification is performed on the shot image.

  If the result of determination in step S13 is search playback, a smile search is performed (S14). The smile search will be described later with reference to FIG. If the smile search is performed or the reproduction in step S12 is performed, it is next determined whether or not communication is performed (S21). In this step, it is determined whether or not to communicate with an external display device such as the television 20 connected by wired communication or wireless communication via the communication unit 14 of the camera 10. Communication is performed when a communication instruction is given by the user.

  If communication is not performed as a result of the determination in step S21, the process returns and executes from step S1. On the other hand, if communication is performed as a result of the determination in step S21, a display location is designated (S23). Here, an external display device connected to the camera 10 by wired communication or wireless communication is determined. In the example shown in FIG. Subsequently, the selected image is transmitted (S24). In the case of search reproduction, the image searched by the smile search in step S14 is transmitted.

  Next, event classification will be described with reference to FIGS. First, the concept of event classification will be described. FIG. 3 is a graph illustrating how images are classified according to time information. In addition, if a face is detected by the face detection unit 5, it is displayed in halftone.

  As is clear from FIG. 3, when there is a grouped image, it can be assumed that there was an event. For example, on September 21st, shooting is performed intensively from morning to afternoon, and there are fewer photographs of people than on September 24th. On September 24, shooting was performed intensively from the afternoon to the evening, and many people were photographed.

  From these pieces of information, it is assumed that September 21 is shooting at an athletic meet and September 24 is shooting at a wedding. Further, if September 24 is a wedding, when an image taken on September 25 is reproduced, the image is classified and displayed as the previous day image. If the images of September 24 and September 25 are mixed, there will be a problem that, for example, a private image is suddenly mixed in with a work image at the time of playback. Keep it.

  As described above, in the present embodiment, the contents of the shooting (event) are analyzed using the date / time information recorded together with the shot image, and using the presence / absence information of the person such as the presence of a face in the shot image. I do. In the case of a smile search, it is executed within the same event classified here.

  Next, the event classification subroutine will be described with reference to the flowchart shown in FIG. In this flow, first, one day (24 hours) is divided into six, and the captured images are arranged (S101). This image arrangement is performed according to the shooting time zone using date and time information recorded together with the shot image. In the present embodiment, classification is performed by dividing 24 hours into 6 parts. However, the present invention is not limited to this, and the division value may be appropriately changed in 4 divisions or 8 divisions.

  Next, in step S103 to step S105, events of the captured image are classified. First, the degree of concentration of captured images is determined (S103). In this step, it is determined whether or not photographing is performed intensively in a short time with respect to the time zone divided into six. If the result of the determination is that it is concentrated, it is next determined whether or not there is little face-up based on detection by the face detection unit 5 (S104).

  As a result of the determination, if the face-up is small, it is determined that it is a school event such as an athletic meet (S111). On the other hand, if the face-up is large, it is determined that the party is a party (S112). Many school events such as athletic events are concentrated in a short period of time, and parents and the like tend to photograph their children from a distance, so that their faces are less likely to be raised. The party is often held in a short time, which is the same as the school event, but it can be distinguished from the school event because there are many face-up photos.

  If the result of determination in step S103 is that there is no concentration in a predetermined time, it is determined whether there are many landscape images (S105). If the result of determination is that there are many landscape images, it is determined that the trip is a trip (S113). In the case of travel, such a determination is made because shooting is dispersed in a single day and there is little intensive shooting for a short time, and there are many landscape images. If there are few landscape images, it is not a trip, and it is not a school event or party as described above.

  When the event selection is completed, the classified event information is tabulated (S115). That is, the events classified in steps S111 to S114 are recorded in the event column in the table described in FIG. 11A, and are recorded in pairs with the photographed image as shown in FIG. 11B. To do.

  Next, details of the smile search in step S14 will be described using the flowchart shown in FIG. Searching for all the faces of a person in the captured image recorded in the recording unit 4 of the camera 10 takes time and is liable to cause a malfunction. Therefore, in this embodiment, since the face existing at the center of the screen is often the main subject in many cases, the determination is performed mainly using the image at the center of the screen.

  When entering the smile search subroutine, first, a photograph with a face in the center of the screen is selected (S31). That is, a photograph with a face in the center of the screen is searched from a collection of photographs of a specific event. As described above, in the table shown in FIG. 11A, information on the position of the face is recorded. From this table, a photograph with a face in the approximate center of the screen is searched.

  When the selection of a photo with a face in the center of the screen is finished, the smile level is determined (S32). The degree of smile of the face in the center of the selected photo is obtained as a numerical value R. The determination of the smile level may be read from the table recorded at that time if the smile level is required at the time of shooting. If the smile level is not calculated at the time of shooting, the smile level is determined as described later with reference to FIGS.

  When the smile level is determined, the number of persons in each image is multiplied by the smile level of the central person, and this value is obtained as RN (S33). The larger the numerical value RN, the higher the smile level of the central person, and the more the person is in the picture, the better the picture. Note that the number of faces may be the number of faces stored in the table shown in FIG.

  Once the numerical value RN has been obtained, it is next determined whether or not all images have been completed (S34). In this step, it is determined whether or not steps S31 to S33 have been executed for all photographs having a face in the center of the screen in a certain event. If the result of determination in step S34 is that the processing has not been completed for all images, the next candidate is selected (S35), the process returns to step S31, and steps S31 to S33 are repeated until the processing is completed for all images.

  As a result of the determination in step S34, if all the images are completed, the image having the highest numerical value RN obtained in step S33 is set as a display candidate (S36), and this image is displayed on the display unit 8 or the television 20 or the like. Displayed on the display device. Subsequently, the feature amount of the person at the center of the screen is stored (S38). Here, the feature amount of the face of the central person is stored in order to prevent only the same person from being displayed. Note that the feature quantity stored in the table shown in FIG.

  When the feature amount of the central person is stored, it is determined whether or not display of the next image is desired (S39). In this step, when the user wants to see the next image, an operation to that effect is performed, so it is determined whether or not this operation has been performed. If the result of determination in step S39 is that the next image is desired, the image in which the person at the center of the displayed image is located in the center is excluded (S40). Since the feature amount of the center person already displayed is stored in step S38, the feature is excluded by not selecting the image in which the person is located in the center, and the next display candidate is returned to step S36. Search for.

  If the result of determination in step S39 is that the user does not want to display the next image, processing returns to the original routine. In this manner, in the smile search subroutine, the one with a high smile level of the person located at the center is displayed. In the center of the screen, there is often a photographer's center of interest, and the possibility that the image that the user wants to view will be selected is increased. Moreover, since not only the degree of smile but also the number of people in the image is taken into consideration, many people can enjoy the reproduction of captured images such as a slide show.

  Next, determination of the smile level will be described with reference to FIGS. 6A shows the face of a smiling person, and FIG. 6B shows the face of a troubled person. As you can see, there are differences in eyes and lips between smiling faces and troubled faces. In the flowchart shown in FIG. 7, the smile level is obtained from this difference.

  When the smile determination subroutine is entered, first the eyes and mouth are detected (S151). This is based on the output of the imaging unit 2 and the face detection unit 5 detects a face, and the eyes and mouth are extracted from the detected face by pattern recognition.

  Subsequently, in steps S152 to S154, the smile level is detected from the area of the white of the eye. First, the area of the white eye above the line 61 connecting the eyes of both eyes is measured, and this value is set as EA (S152). Similarly, the area of the white eye below the line 61 is measured, and this value is set to EB (S153). Next, the difference between EA and EB obtained in this way is normalized by the sum of these, that is, (EA−EB) / (EA + EB) is obtained to obtain a numerical value RE (S154). It is determined that the larger the numerical value RE, the higher the smile degree.

  Next, in steps S155 to S157, the smile level is detected from the area of the lips. First, the area of the lips above the line 62 connecting both ends of the mouth is measured, and this value is set as LA (S155). Similarly, the area of the lips below the line 62 is measured, and this value is set as LB (S156). The difference between LA and LB thus obtained is normalized by the sum of these, that is, (LB−LA) / (LA + LB) is obtained to obtain a numerical value RL (S157). It is determined that the larger the numerical value RL, the higher the smile degree.

  Subsequently, the smile level R is obtained by adding the values RE and RL obtained in steps S154 and S157 (S158). The larger the value R, the closer to smile. Furthermore, in order to improve the accuracy of determination of the smile level, the state near the arrow 64 in FIG. 6A is determined (S159). That is, if there is a shadow at the edge of the mouth and teeth are visible, the probability of smile is high, and this is determined. If the result of determination in step S159 is that teeth are visible and there is a shadow at the mouth edge, one point is added to smile level R (S160).

  However, if there is a wrinkle between the eyebrows, it cannot be said that it is a smile. That is, it is determined whether or not there is a wrinkle between the eyes near the arrow 65 in FIG. 6B (S161). If the eyelid is detected between the eyebrows as a result of the determination, one point is deducted from the smile level R (S162). In this way, the numerical value (R) can be obtained such that the closer to a smile, the higher the score.

  In the present embodiment, the degree of smile is determined based on whether there are many white eyes above the line connecting the centers of the eyes of both eyes. However, the present invention is not limited to this. Good. That is, in FIG. 6A, the position ET at the highest point of the heel with respect to the width EH of the heel is obtained, and the smile degree can also be obtained by the ratio ET / EH. When ET / EH increases, it becomes close to a smile, and when this ratio decreases and the inclination of the upper eye increases, it tends to move away from the smile.

  As shown in FIG. 8A, the determination of the smile level described above can be made accurately by estimating the emotion of the person when the person is facing the front. However, an accurate determination cannot be made with the upper eye and the looking-down angle as shown in FIGS. Therefore, when the determination result in step S161 is that there is no wrinkle between the eyebrows, or when the processing in step S162 is completed, next, R correction is performed (S163). When the R correction is completed, the process returns to the original routine.

  Next, the R correction subroutine in step S163 will be described with reference to the flowchart shown in FIG. When entering the R correction subroutine, first, the center coordinates of the face are determined (S251), and the center coordinates of the eyes are determined (S252). The face detection unit 5 determines the coordinates of the face center and the eye center. Next, a center-to-center distance ΔY between the center of the face and the center of the eye is obtained (S253).

  Subsequently, it is determined whether or not the center-to-center distance ΔY is smaller than −ΔY1 (S256). As a result of this determination, if the center-to-center distance ΔY between the center of the face and the center of the eye is smaller than −ΔY1, it is assumed that the upper eye is used as shown in FIG. 1 is subtracted from the R value (S257).

  Next, it is determined whether or not the center-to-center distance ΔY is greater than + ΔY2 (S258). As a result of this determination, when the center distance ΔY between the center of the face and the center of the eye is larger than + ΔY2, it is assumed that the person looks down as shown in FIG. 8C, and this state is also far from the smile. Therefore, 1 is subtracted from the R value (S259).

  In this way, the face center position and the eye center position are determined, and when the distance between the centers is between −ΔY1 and + ΔY2 (ΔY0 in FIG. 8A), the front face is determined. If the predetermined value −ΔY1 or + ΔY2 is exceeded, the face is turned away and the R value representing the degree of smile is subtracted because it is far from the smile (S257, 259). When the R correction subroutine ends, the process returns to the original routine.

  As described above, in the first embodiment of the present invention, an image having a high smile level of a person near the center of the screen is searched and displayed. When everyone views the image, it can be said that a smiling photo is a suitable photo because it softens the viewer, but it takes time to determine the smile level of all persons appearing in the captured image. In this regard, in the present embodiment, since only the face near the center of the screen is determined, it is possible to quickly search from a plurality of images.

  In this embodiment, in addition to the high smile level of the face near the center of the screen, an image having a large number of faces appearing in one captured image is selected. The more faces that appear, the more enjoyable the viewers will be.

  Further, in the present embodiment, when the image is continuously reproduced, the face of the person other than the face centered in the previous image is set based on the feature amount of the face. For this reason, when a plurality of images are continuously reproduced and displayed, it is possible to avoid an image in which the same person continues to be near the center of the screen, and an interesting reproduction display is obtained.

  Next, a second embodiment of the present invention will be described with reference to FIG. In the first embodiment of the present invention, the smile search preferentially selects and reproduces and displays a smile having a high degree of smile and a large number of faces on the screen. In the second embodiment, an image with a high smile level is preferentially reproduced and displayed, and the number of faces in the screen is not considered. However, if only the same person appears, it is not interesting, so the face of the same person is not displayed.

  The configuration of the second embodiment is the same as that of the first embodiment except that the smile search subroutine shown in FIG. 5 is changed to the smile search subroutine shown in FIG. This subroutine, which is the difference, will be mainly described.

  If the smile search flow in this embodiment is entered, first, the center of the screen is set as a predetermined part (S60). In the first embodiment, the smile level is compared only for the center of the screen. In the second embodiment, the smile level may also be compared for the peripheral portion. Subsequently, the temporary storage image is cleared (S61). The temporary storage image temporarily stores the display image for comparison in step S71 to be described later. In the initial state, the temporary storage image is cleared. In the present embodiment, instead of storing the image data itself, the feature amount of the face of the predetermined part is stored.

  Next, a photograph having a face in a predetermined part of the screen is selected (S62). In this step, one image having a face in the predetermined portion (initially the center) set in step S60 is selected from the group of photos related to the selected event. Subsequently, the smile level of the face in the predetermined part of the selected image is determined (S63). The determination of the smile level is read from the table when it is recorded at the time of shooting, and is performed using the flow described with reference to FIG. 7 if it is not recorded.

  When the smile level determination is completed, it is next determined whether or not all the images have been completed (S64). In this step, it is determined whether or not steps S62 and S63 have been executed for all photographs having a face in a predetermined part of the screen in a certain event. As a result of the determination in step S64, if all the images have not been completed, the next candidate is selected (S65), the process returns to step S62, and steps S62 to S65 are repeated until the completion for all the images.

  If the result of determination in step S64 is that all the images have been completed, the image with the highest smile level is selected (S66). Next, the face feature amount of the predetermined part of the selected image is determined (S67). If the face is located at the center, it is stored in the table shown in FIG. 11, so this value may be stored as it is.

  Next, it is determined whether the image selected in step S66 (in the case of the next display, the image selected in step S72) and the temporary storage image are similar (S71). This comparison is performed on the basis of the facial feature amount because the facial feature amount of a predetermined portion is stored every time it is displayed in step S74 described later. Here, the reason why it is determined whether or not the images are similar is to prevent the same person's face from being continuously displayed.

  If the result of determination in step S71 is not similar, the image selected in step S66 (in the case of the next display, the image selected in step S72) is displayed (S73). This display is performed on an external display device such as the display unit 8 or the television 20.

  Subsequently, the facial feature amount of a predetermined portion of the image displayed in step S73 is temporarily stored (S74). When temporary storage ends, it is determined whether or not to perform the next display (S75). When the user desires the next display, the operation determination unit 11 determines this because it is performed by the operation member. If the result of determination in step S75 is that there is no next display, processing returns to the original routine.

  As a result of the determination in step S75, when the next display is performed, an image having the next highest value is selected for the smile degree in the predetermined portion (S72). In this step, images are selected in the order of smile levels obtained by repeatedly executing steps S62 to S65. Next, it is determined whether or not there is an image selected in step S72 (S81). If all the images having a face in the predetermined part are selected, it becomes none.

  If the result of determination in step S81 is that an image has been selected, the process returns to step S71 to determine whether or not the images are similar using facial feature amounts. Jumping to step S72, the next image is selected. On the other hand, if the images are not similar, the display and the facial feature amount are temporarily stored as described above.

  If the result of determination in step S81 is that no image has been selected, the predetermined part is changed (S82). That is, at first, the smile level and the feature amount of the face in the center of the screen are determined. If there is no image having a face in the center of the screen, then the right side of the screen, the left side of the screen, and the predetermined part in step S82. To change. Note that the order of the predetermined units is determined in advance in the flowchart, but may be arbitrarily set by the user. In step S82, when the predetermined part is changed, the process returns to step S61, and an image with a high smile degree is searched for by replacing the predetermined part and displayed.

  As described above, in the second embodiment of the present invention, images with a high degree of smile on the face in a predetermined part of the screen are sequentially reproduced and displayed, and the same face is not displayed. Therefore, it is possible to search quickly, and various human faces are sequentially reproduced and displayed.

  In the present embodiment, since the predetermined part can be changed in step S82, the smile level can be determined for a face of a person who is not near the center of the screen, and an image with a high smile level can be reproduced and displayed. it can.

  Next, a third embodiment of the present invention will be described with reference to FIGS. In the first embodiment of the present invention, the smile search preferentially selects an image with a high smile level and a large number of faces in the screen, and in the second embodiment, an image with a high smile level is given priority. The number of faces on the screen was not taken into consideration. In the third embodiment, the degree of smile is taken into consideration, but the display image is selected in consideration of this and the composition of the captured image.

  The configuration of the third embodiment is the same as that of the first embodiment except that the smile search subroutine shown in FIG. 5 is changed to the smile search subroutine shown in FIG. Since the flow shown in FIG. 13 is similar to the flow of FIG. 5, the same steps as those in FIG. 5 are denoted by the same step numbers, and different steps will be mainly described.

  If the smile search flow is entered, a photograph whose face is located at the center of the screen is selected (S31). Subsequently, the smile level is determined (S32). This smile level determination is read from the table if it is recorded at the time of shooting, and is executed by the subroutine shown in FIG. 7 if it is not recorded at the time of shooting.

  Subsequently, a value RK obtained by multiplying the degree of smile obtained in step S32 by the composition coefficient K is obtained (S33b). The composition coefficient is a coefficient corresponding to the analyzed composition after analyzing the composition of the photograph. In this embodiment, since emphasis is placed on the degree of smile, composition analysis is performed based on the face size determined by the size determination unit 5b of the face detection unit 5, for example, as shown in FIG. As shown in the figure, a so-called bust shot in which the upper body is photographed from the chest of a person has a good composition (in the example of the figure, a figure coefficient of 10). Further, in the case of the composition as shown in FIG. 14C where the smile does not look good, the composition coefficient is set low (composition coefficient 2 in the example in the figure). For a very large face, the situation around FIG. 14 (a) is unknown, so the composition coefficient is slightly lowered (composition coefficient 5 in the example in the figure).

  When the value RK of the selected photo is obtained, it is next determined whether or not the processing of steps S31 to S33b has been completed for all images (S34). As a result of the determination, if all the images have not been completed, the next candidate is selected (S35), the process returns to step S31, and steps S31 to S33b are repeated until all the images are completed.

  As a result of the determination in step S34, when processing is completed for all the photos, the photo having the highest value among the values RK obtained in step S33b is selected as a display candidate (S36b). Hereinafter, as in the first embodiment, the selected photograph is displayed on an external display device such as the display unit 8 or the television 20, and if the next photograph is desired to be displayed, the same person other than the photograph located in the center is displayed. A photograph is selected from the images and displayed (S36b to S40). In the first embodiment, the degree of smile is represented by RN indicating the number of people in the image, but in the third embodiment, the display is performed in the order of the degree of smile: the ratio RK of the composition coefficient. In step S40, if the next image is not desired, the process returns to the original routine.

  As described above, in the third embodiment, the order of reproduction and display is determined based on the idea that images with good composition are preferentially displayed in addition to the degree of smile. In this embodiment, the composition coefficient of the bust shot is increased, the composition coefficient is slightly lowered for a very large face, and the small composition coefficient of the face where the smile cannot be seen is further decreased.

  As described above, in each embodiment of the present invention, the smile level of a face located at the center of the screen of each of a plurality of images is determined, and based on the determined smile level, a representative is selected from the plurality of images. It searches for images and displays them. Since the smile level is determined for the face located at the center of the screen, the search can be performed quickly. Note that the position of the face to be searched in the first and third embodiments is not limited to the center of the screen, but may be any position such as the right side, left side, upper side, and lower side of the screen as in the second embodiment.

  Further, in each embodiment of the present invention, when searching for the next image, a search is performed except for an image in which the person who performed smile level determination is in the center among the displayed images. For this reason, it is possible to prevent the same person from being repeatedly reproduced and displayed in a specific range, and it is possible to enjoy a variety of changes.

  In the embodiment of the present invention, the number of faces, the position of the face, the size, the smile level, and the features are tabulated at the time of shooting. You may make it carry out at the timing of.

  In the embodiment of the present invention, an image photographed by the camera is subjected to processing such as a smile search in the camera and reproduced and displayed on the camera display device or an external display device. The image may be stored in a personal computer, a smile search may be performed based on the stored image, and the image may be reproduced and displayed.

  Furthermore, in the embodiment of the present invention, in selecting the representative image, in addition to the smile degree alone or the smile degree, the number of people and the composition are taken into consideration. The representative image may be selected in consideration of whether the background is simple.

  In the present embodiment, the digital camera is used as the photographing device. However, the camera may be a digital single-lens reflex camera or a compact digital camera, and may be a mobile phone or a personal digital assistant (PDA). Of course, a camera built in Personal Digital Assist) may be used.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

It is a block diagram which shows the structure concerning the camera concerning 1st Embodiment of this invention, and its periphery. It is a flowchart which shows the operation | movement of the camera control in the camera concerning 1st Embodiment of this invention. In 1st Embodiment of this invention, it is a figure explaining the view of event determination. It is a flowchart which shows the operation | movement of the event determination in the camera concerning 1st Embodiment of this invention. It is a flowchart which shows the operation | movement of the smile search in the camera concerning 1st Embodiment of this invention. In a 1st embodiment of the present invention, it is a figure which compares a smile of a subject and a troubled face, (a) shows a smile and (b) shows a troubled face. It is a flowchart which shows the operation | movement of smile degree determination in the camera concerning 1st Embodiment of this invention. In the first embodiment of the present invention, it is a diagram showing a state of facing the camera of the subject, (a) shows the face facing front, (b) shows the face of the upper eye, (c) looks down. Showing the face. It is a flowchart which shows the operation | movement of R correction | amendment in the camera concerning 1st Embodiment of this invention. It is a figure which shows the measurement of the feature-value of a person's face in 1st Embodiment of this invention. In the first embodiment of the present invention, data recorded with an image is shown, (a) shows a table of information to be recorded, and (b) shows a relationship between the image and the location of the table. It is a flowchart which shows the operation | movement of the smile search in the camera concerning 2nd Embodiment of this invention. It is a flowchart which shows the operation | movement of the smile search in the camera concerning 3rd Embodiment of this invention. In a 3rd embodiment of the present invention, it is a figure showing an example of composition coefficient, (a) is an example of composition coefficient 5, (b) is an example of composition coefficient 10, (c) is an example of composition coefficient 2. .

DESCRIPTION OF SYMBOLS 1 ... Control part, 1a ... Image processing part, 1b ... Event classification part, 1c ... Face classification part, 1d ... Display order determination part, 2 ... Imaging part, 3 ... Optical system control unit, 4 ... recording unit, 4a ... image vs face storage unit, 5 ... face detection unit, 5a ... position detection unit, 5b ...
Size determining section, 5c ... characteristic section, 6 ... cutting section, 7 ... compositing processing section, 8 ... display section, 8a ... display control section, 9 ... clock section, DESCRIPTION OF SYMBOLS 11 ... Operation determination part, 12 ... Smile degree determination part, 14 ... Communication part, 20 ... Television, 61 ... Line, 62 ... Line, 64 ...
-Arrows, 65 ... arrows, 81 ... intersections, 82 ... intersections, 83 ... intersections, 84 ...
・ Intersection

Claims (9)

An image display device that selects and displays a representative image from a plurality of images including a face image,
A selection unit for selecting a representative image from the smile level and composition of the face included in each of the plurality of images;
A search unit that searches for an image in which a person different from the representative image is smiling as a next representative image;
A display control unit for displaying the representative image;
An image display device comprising: The image display apparatus according to claim 1, wherein the plurality of images are photographed images, and are images associated with the smile degree and the characteristics of each person at the time of photographing. The image display device according to claim 1, wherein the display control unit displays an image of a person with a high smile degree of the representative image and the next representative image side by side. An image display method for selecting and displaying a representative image from a plurality of images including a face image,
Selecting a representative image from the smile level and composition of the face included in each of the plurality of images;
Searching for an image in which a person different from the representative image is smiling as the next representative image;
Displaying the representative image;
An image display method comprising: The image display method according to claim 4, wherein the plurality of images are captured images, and are images in which the smile level is associated with the characteristics of each person at the time of capturing. The image display method according to claim 4, wherein the display step displays the representative image and the next representative image of a person with a high smile level side by side. An image display program for selecting and displaying a representative image from a plurality of images including a face image,
Selecting a representative image from the smile level and composition of the face included in each of the plurality of images;
Searching for an image in which a person different from the representative image is smiling as the next representative image;
Displaying the representative image;
An image display program characterized by comprising: The image display program according to claim 7, wherein the plurality of images are photographed images, and are images associated with the smile degree and the characteristics of each person at the time of photographing. The image display device according to claim 7, wherein the display step displays the image of a person with a high smile level of the representative image and the next representative image side by side.

Images