JP2008277904A - Image processor, image direction discrimination method and image direction discrimination program, reproducing device, reproducing method and reproducing program, and digital still camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately discriminate longitudinal/lateral directions of image data with simple constitution even when image data includes a plurality of person's faces. <P>SOLUTION: A digital still camera 1 of the present invention detects as a detected face area a face area FA of a person included in captured image data as image data, and discriminate longitudinal/lateral directions of respective detected face areas based upon roll face angles as angles of the detected face areas around a vertical direction wit respect to a plane that the captured image data represents as an axis of rotation. Then the digital still camera 1 discriminate the longitudinal/lateral direction of the captured image data by majority based upon discrimination results showing that the detected face areas are longitudinal or lateral to thereby accurately discriminate the longitudinal/lateral direction of the captured image data without using a detecting means such as a posture sensor when a plurality of persons are captured at the same time. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing device, an image direction determination method and an image direction determination program, a playback device, a playback method and a playback program, and a digital still camera, and is suitable for application to a digital still camera, for example.

  Conventionally, as a digital still camera, detection means such as a posture sensor detects whether the digital still camera is in a vertical or horizontal posture at the time of shooting, and captured image data (hereinafter referred to as captured image data) There is information added to the captured image data as image direction information indicating whether the current direction is (see, for example, Patent Document 1).

When displaying the captured image data, the digital still camera rotates and displays the captured image data based on the added image direction information, so that the captured image has the same vertical and horizontal directions as the captured image data at the time of imaging. Is to be presented to the user.
Japanese Patent No. 3192462

  By the way, in the digital still camera having such a configuration, since the posture sensor is required to detect the camera posture, the configuration becomes complicated.

  In addition, a technique for detecting a human face area from captured image data and discriminating the vertical and horizontal directions of the captured image data from the direction of the face has been proposed (Japanese Patent Laid-Open No. 2006-109175). No consideration is given to the case where a human face is present, and in such a case, there is a problem in that the vertical and horizontal directions of the captured image data cannot be correctly determined.

  The present invention has been made in consideration of the above points, and provides an image processing apparatus that has a simple configuration and can accurately determine the vertical and horizontal directions of image data even when a plurality of faces are present. It is what we are going to propose.

  In order to solve such a problem, in the present invention, a human face area included in image data is detected as a detected face area, and the rotation angle of the detected face area when the direction perpendicular to the plane represented by the image data is taken as the rotation axis is used. The vertical and horizontal directions of a plurality of detected face areas included in the image data are determined based on a certain roll face angle, and the vertical and horizontal directions of the image data are determined based on the determination results of the detected face areas.

  Thus, the vertical and horizontal directions of the image data can be determined based on a plurality of detected face areas detected from the image data without a sensor for detecting the camera posture.

  In the present invention, the human face area is detected from the image data as a detected face area, and the detected face area is determined based on the roll face angle with the vertical direction with respect to the plane represented by the image data as the rotation axis. The vertical and horizontal directions of the image data are determined based on the reliability with respect to the detected face area.

  Thus, the vertical and horizontal directions of the image data can be determined based on the detected face area regardless of the number of detected face areas detected from the image data without a sensor for detecting the camera posture.

  According to the present invention, it is possible to determine the vertical and horizontal directions of image data based on a plurality of detected face regions detected from image data without a sensor for detecting the camera posture, and thus a simple configuration and a plurality of people. PROCESSING DEVICE, IMAGE DIRECTION DETERMINING METHOD, IMAGE DIRECTION DETERMINING PROGRAM, REPRODUCING DEVICE, REPRODUCING METHOD AND REPRODUCING PROGRAM, AND DIGITAL STILL CAMERA Can be realized.

  According to the present invention, the vertical and horizontal directions of image data can be determined based on the detected face area, regardless of the number of detected face areas detected from the image data, without a sensor for detecting the camera posture. An image processing apparatus can be realized that has a simple configuration and can accurately determine the vertical and horizontal directions of image data even when there are a plurality of faces.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

(1) Overall Configuration 1 in FIG. 1 shows the overall configuration of a digital still camera as a whole, and a control unit including a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory) not shown. 2 controls the entire digital still camera 1 in an integrated manner. The control unit 2 develops various basic programs and image direction determination programs stored in the ROM or the flash memory 5 in the RAM, and thereby performs various processes based on these programs, image direction determination processing described later, and later described. An image data display process is executed.

  The control unit 2 has a plurality of imaging modes such as a face detection mode used when imaging a person, a landscape mode used when imaging a landscape as a main, and an exercise mode used when imaging a moving subject. In response to an operation input via the operation unit 6, the user is caused to set the imaging mode according to the subject to be imaged.

  The control unit 2 sets imaging conditions such as an aperture, a depth of focus, and a shutter speed that are set in advance so as to be optimal for a subject to be imaged according to the imaging mode, and performs an imaging process according to a user request. Execute.

  Specifically, for example, when an imaging request signal is supplied from the operation unit 6 by a user pressing a shutter button (not shown) included in the operation unit 6, the control unit 2 captures an imaging condition corresponding to the set imaging mode. The imaging process is executed at.

  The control unit 2 controls a shutter mechanism (not shown) to capture imaging light from the lens group 7 and photoelectrically converts the imaging light that has passed through the lens group 7 and formed an image on the image sensor 8 by the image sensor 8. The image signal is sent to the signal processing unit 9.

  The control unit 2 generates captured image data by performing gamma correction processing, AGC (Auto Gain Control) processing, and analog-digital conversion processing on the supplied imaging signal by the signal processing unit 9, and performs imaging The image data is temporarily stored in the image RAM 4, and the captured image data is read from the image RAM 4 and supplied to the image processing unit 11.

  The control unit 2 generates compressed image data by executing a predetermined compression process on the captured image data by the image processing unit 11, supplies the compressed image data to the image storage unit 12, and stores the compressed image data.

  In addition, when the current imaging mode is set to the face detection mode, the control unit 2 executes the above-described imaging process and the image direction determination process. This image direction discrimination process will be described later.

  On the other hand, the control unit 2 receives a display request signal for displaying a captured image based on the compressed image data on a display unit 13 such as an LCD (Liquid Crystal Display) in response to an operation input to the operation unit 6 by the user. , The compressed image data is read from the image storage unit 12 and sent to the image processing unit 11.

  When the image processing unit 11 decompresses the compressed image data to generate captured image data, the control unit 2 executes image data display processing (described later) on the captured image data, and the captured image data is displayed on the display unit 13. Supply. As a result, a captured image based on the compressed image data is displayed on the display unit 13.

  On the other hand, the control unit 2 receives a supply request signal for supplying compressed image data from the external interface 14 in a state where an external device (not shown) such as a personal computer is connected to the external interface 14. The compressed image data is read from the image storage unit 12 and the compressed image data is supplied to an external device via the external interface 14.

(2) Image Direction Discriminating Process in Face Detection Mode As shown in FIG. 2A, the digital still camera 1 having a flat, substantially rectangular parallelepiped shape is a posture of the digital still camera 1 (hereinafter referred to as a camera posture). ) Is a posture in which the longitudinal direction of the housing surface 1A having the display unit 13 is horizontal (hereinafter referred to as a horizontal posture), as shown in FIG. 2B, the longitudinal direction in the captured image is horizontal. (Hereinafter referred to as a horizontal image).

  On the other hand, as shown in FIG. 3A, the digital still camera 1 is shown in FIG. 3B when the casing surface 1A is in a posture in which the longitudinal direction is a vertical direction (hereinafter referred to as a vertical posture). As shown, the image is picked up in a state where the longitudinal direction in the picked-up image is the vertical direction (hereinafter referred to as a vertical image).

  The digital still camera 1 generates captured image data composed of lateral images regardless of the camera posture at the time of imaging.

  However, since the normal user often checks the captured image displayed on the display unit 13 in a state where the camera posture is fixed to the horizontal posture (FIG. 2B) when displaying the captured image on the display unit 13. It is preferable to display the captured image data captured in the horizontal posture as it is as a horizontal image, and display it as a vertical image by rotating the captured image data captured in the vertical posture.

  Here, when a user captures a person, it is extremely rare that the person's face is laid down (eyes and mouth lined up horizontally), and the person's face is upright (eyes and mouth). Are overwhelmingly often imaged in a state where

  Therefore, in the digital still camera 1 of the present embodiment, in the face detection mode, the face angle represented by the face area in the captured image data (hereinafter referred to as the face angle) is detected, and imaging is performed based on the face angle. The vertical and horizontal directions of the captured image at the time are determined. Next, the image direction discrimination process will be described.

  As shown in FIG. 4, when a captured image based on captured image data is displayed on the display unit 13 or the like among the face angles described above, the image rotates about the Y axis perpendicular to the XZ plane having the captured image. An angle in the roll direction is defined as a roll face angle.

  That is, as shown in FIG. 5 (A), the roll face angle is obtained when the vertical direction (that is, the short side direction) in the captured image data that is a lateral image is used as the reference axis SP with the roll face angle being 0 °. This is the angle of the vertical axis of the face area FA with respect to the reference axis SP (hereinafter referred to as the face vertical axis FP). The roll face angle when the face vertical axis FP is rotated clockwise from the reference axis SP is positive, and the roll face angle when the face vertical axis FP is rotated counterclockwise is negative.

  For example, as shown in FIG. 5B, when the face of an upright person is imaged while the digital still camera 1 is rotated by −70 ° (that is, 70 ° counterclockwise), as shown in FIG. 5C. In addition, the roll face angle in the captured image is rotated by 70 ° clockwise, and the face angle is + 70 °.

  Also, as shown in FIG. 4, the direction of rotation about the face vertical axis FP is defined as the yaw direction, and the face angle in this yaw direction is defined as the yaw face angle. The yaw face angle is an angle formed by the face area FA, which is a plane including eyes and mouth, and the XZ plane, as shown in FIG.

  That is, as shown in FIGS. 6A and 6B, when the face area FA is facing the front (the face area FA exists on the XZ plane), the reference angle is 0 °, and the face area shown in FIG. The angle is positive when rotated clockwise as viewed from below the FA (mouth side), and negative when rotated counterclockwise. Since the yaw face angle has the face vertical axis FP as the rotation axis, the direction of the face vertical axis FP changes in the XZ plane according to the change in the roll face angle described above.

  For example, as shown in FIG. 6C, in the face area FA having a yaw angle of −40 ° (present on the virtual plane YP indicated by the broken line), the front of the face area FA is compared with the XZ plane indicated by the solid line. -40 degrees in the yaw direction. Further, as shown in FIG. 6D, even in the face area FA having the same −40 ° yaw face angle, when the face vertical axis FP is changed, the rotation direction (yaw direction) is similarly changed. become.

  When the current imaging mode is set to the face detection mode, the control unit 2 (FIG. 1) of the digital still camera 1 uses the image processing unit 11 to capture the captured image data generated by the signal processing unit 9 in the imaging process. Instead, it is supplied to the image size conversion unit 22 of the face detection block 20.

  In the face detection block 20, the controller 21 controls the image size conversion unit 22, the image memory 23, the correlation determination unit 24, and the reference face data storage unit 25 based on various control signals supplied from the control unit 2. A human face area FA is detected as a detected face area from the captured image data, and an image direction determination process is performed to determine the direction of the captured image based on the roll face angle of the detected face area.

  That is, when the captured image data is supplied from the signal processing unit 9, the image size conversion unit 22 converts the captured image represented by the captured image data into various image sizes in accordance with the image size signal representing the magnification supplied from the controller 21. Then, size-converted image data is generated, and feature image data is generated by calculating a luminance difference from a reference point arbitrarily extracted from the size-converted image data, for example.

  Further, the image size conversion unit 22 cuts out the feature image data into a predetermined size in order from the central portion, generates the cut out feature image data, and expresses the coordinate information indicating the position of the cut out feature image data in the original feature image data Then, magnification information indicating the enlargement ratio or reduction ratio of the cutout feature image data is added to the cutout feature image data, and the cutout feature image data is sent to the image memory 23.

  The image memory 23 temporarily stores the cutout feature image data in accordance with the memory address supplied from the controller 21, and reads out the cutout feature image data in accordance with the timing signal and the memory address supplied from the controller 21. 24.

  In the reference face data storage unit 25, a plurality of reference face data having various face angles (however, the roll face angle is 0 °) as reference face data to be compared with the cut-out feature image data, and the reference The yaw face angle of the face data (hereinafter referred to as reference face angle information) and the reference face area information representing the area of the person's face area FA in the reference face data are stored in association with each other. The reference face data is stored as feature data representing a luminance difference from the reference point, for example.

  The correlation determination unit 24 sends an acquisition command for acquiring the reference face data to the reference face data storage unit 25. The reference face data storage unit 25 sends the requested reference face data to the correlation determination unit 24.

  Then, the correlation determination unit 24 sequentially acquires the reference face data from the reference face data storage unit 25 and, at the same time, sets each reference face data in the positive and negative directions in the roll direction (clockwise and counterclockwise on the XZ plane), for example. The rotation angle is rotated by ± 90 ° in 1 ° increments and compared with the cutout feature image data acquired from the image memory 23. The correlation determination unit 24 determines that the detected face area has been detected from the extracted feature image data when there is reference face data having a correlation value equal to or greater than a predetermined matching threshold.

  At this time, the correlation discriminating unit 24 uses the area of the detected face area (hereinafter referred to as the face area area) P1 from the magnification information and the reference face area information as a parameter P used for calculation of the face area reliability described later. calculate. Further, the correlation determining unit 24 uses the magnification information and the coordinate information of the cut out feature image data, and the distance from the center in the feature image data to the center point of the detected face region (hereinafter referred to as the face region center distance). P2 is calculated. Further, the correlation determination unit 24 uses the reference face angle information and the correlation value added to the reference face data as the face area importance degree parameter P (P3 and P4), respectively.

  Furthermore, the correlation determination unit 24 sets the rotation angle when the detected face area is detected as the roll face angle for the detected face area (hereinafter referred to as the detected roll face angle).

  Then, when the correlation determination unit 24 sends the detected roll face angle, the reference face angle information P3, the face area area P1, the face area center distance P2, and the correlation value to the control unit 2, the correlation determination unit 24 outputs the next value from the image memory 23. The extracted feature image data is acquired and the comparison with the reference face data is continued. At this time, the control unit 2 stores the detected roll face angle, face region area P1, face region center distance P2, reference face angle information P3, and correlation value P4 in the RAM in the control unit 2.

  On the other hand, when all the reference face data is rotated and compared with the cut-out feature image data and there is no reference face data with the correlation value P4 equal to or greater than the predetermined matching threshold, the correlation The determination unit 24 determines that there is no human face area in the cut out feature image data, acquires the next cut out feature image data from the image memory 23, and continues the comparison with each reference face data.

  Then, when the comparison with the reference face data is finished for all the extracted feature image data, the correlation determination unit 24 finishes the face detection for the supplied captured image data.

  At this time, when a plurality of human faces are detected by the correlation determination unit 24, the control unit 2 determines the vertical and horizontal directions of the captured image data by face direction determination processing (details will be described later), and the vertical and horizontal directions (vertical and vertical) Image direction information indicating that the image is a directional image or a horizontal image), and adding the image direction information to the captured image data and sending it to the image processing unit 11, thereby capturing the image with the image direction information added thereto. When compressed image data is generated from image data, it is stored in the image storage unit 12.

  As shown in FIG. 7, when there are a large number (11 or more) of faces in one captured image data, the control unit 2 ends the face detection when detecting 10 faces, By executing the face direction discrimination process, it is possible to prevent an excessive amount of time from being detected in detecting the detected face area. In the figure, the detected face area is indicated by diagonal lines, and the same applies to the following figures.

  On the other hand, when only one person's face is detected by the correlation determination unit 24, the control unit 2 reads the detected roll face angle from the RAM in the control unit 2, and the absolute value of the detected roll face angle is 70 ° or more. If it is within the range of 110 ° or less, it is determined that the captured image data is a vertical image. On the other hand, when the absolute value of the detected roll face angle is not less than 0 ° and less than 70 °, and greater than 110 ° and less than 180 °, the control unit 2 determines that the captured image data is a horizontal image.

  The control unit 2 adds the determined vertical and horizontal directions of the captured image to the captured image data as image direction information, and stores the compressed image data based on these in the image storage unit 12.

  On the other hand, when no detected face area is detected by the correlation determining unit 24, the control unit 2 generates compressed image data as it is from the captured image data and stores the compressed image data in the image storage unit 12.

(3) Face Direction Determination Processing Next, face direction determination processing that is executed when a plurality of human faces are detected in the image direction determination processing will be described.

  As shown in FIG. 8, in the captured image data in which a plurality of persons are captured simultaneously, it is assumed that the detected roll face angles in the plurality of detected face regions are different from each other. Therefore, the digital still camera 1 determines the vertical and horizontal directions of a plurality of detected face areas in the captured image data, and comprehensively determines the vertical and horizontal directions of the captured image data based on the vertical and horizontal directions.

  That is, the face direction determination unit 2A of the control unit 2 in the digital still camera 1 obtains the detected roll face angle, the face region area P1, the face region center distance P2, the reference face angle information P3, and the correlation value P4 from the RAM in the control unit 2. read out.

  In captured image data in which a plurality of persons are captured at the same time, as shown in FIG. 9, generally, a person with a high priority of imaging (that is, a person who wants to image as a subject) often comes to the front. The area of the detected face area is increased. In addition, as shown in FIG. 8, adults (persons B and C) having a larger face area than children (person A) rarely lie down or pose to tilt their faces.

  Therefore, it can be said that the vertical and horizontal directions of the detection face area having a large area are more likely to be the face area of the person who is the main subject of imaging compared with other detection face areas, and the reliability is high. Therefore, the face direction determination unit 2A calculates an area evaluation value so that a value for a detected face area having a large face area area P1 is high.

  That is, the face direction determination unit 2A calculates an area evaluation value for each detected face area by multiplying the face area area P1 by a predetermined weighting coefficient W1.

  Further, in captured image data in which a plurality of persons are imaged simultaneously, as shown in FIG. 9, generally, the person who is the main subject of imaging (person B in the figure) is often located at the center. Accordingly, it can be said that the vertical and horizontal directions of the detection face area located at the center are more likely to be the face area of the person who is the main subject of imaging compared to other detection face areas, and the reliability is high.

  Therefore, the face direction determination unit 2A uses the face area center distance P2 representing the distance from the center point to the center of the detected face area in the captured image data, and sets the center evaluation value so that the value for the detected face area close to the center becomes high. calculate.

  That is, the face direction determination unit 2A divides the face area center distance P2 into, for example, 10 levels so that the level for the detected face area close to the center is increased, and multiplies the level by a predetermined weighting coefficient W2. The center evaluation value for each detected face area is calculated.

  Furthermore, in the captured image data in which a plurality of persons are captured at the same time, as shown in FIG. 9, generally when the person who is the main subject of imaging is facing the front (in the direction of the digital still camera 1) (ie, yaw) When the face angle is 0 °, it is highly possible that an image is captured in accordance with the person B) in the figure, while a person who is not trying to image (person A in the figure) such as a passerby is in a completely different direction. Often it is suitable.

  Therefore, it can be said that the vertical and horizontal directions of the detected face area where the absolute value of the yaw face angle is small are more likely to be the face area of the person who is the main subject of imaging compared with other detected face areas, and the reliability is high. Therefore, the face direction determination unit 2A uses the yaw face angle of the detected face area represented by the reference face angle information P3, and evaluates the front so that the yaw angle is closer to 0 ° and becomes higher as the detected face area faces the front. Calculate the value.

  That is, the face direction discriminating unit 2A divides the absolute value of the yaw face angle into, for example, 10 levels so that the detected face region is higher in front, and a predetermined weighting coefficient W3 is assigned to the level. By multiplying, the front evaluation value for each detected face area is calculated.

  In the digital still camera 1, the higher the correlation value P4 representing the matching rate with the reference face data, the higher the possibility that the detected face area is definitely a person's face (that is, the person's face-likeness). It is highly possible that the face angle is correctly detected, and the vertical and horizontal directions of the detected face area having a high correlation value P4 can be said to have higher reliability than other detected face areas. Therefore, the face direction determination unit 2A uses the correlation value P4 for the detected face region, and calculates a correlation evaluation value so that the value increases as the correlation value P4 increases.

  That is, the face direction determination unit 2A calculates a correlation evaluation value for each detected face region by multiplying the correlation value P4 by a predetermined weighting coefficient W4.

  The face direction discriminating unit 2A multiplies each value by weighting coefficients W1, W2, W3, and W4 to align the levels of the respective evaluation values (area evaluation value, center evaluation value, correlation evaluation value, and front evaluation value). At the same time, the values of the weighting coefficients W1, W2, W3 and W4 are adjusted according to the importance of each evaluation value.

  Then, the face direction determination unit 2A adds each evaluation value, and for each detected face area, a face that comprehensively represents the reliability of the detected face area when compared with other detected face areas in the captured image data The area reliability can be calculated.

  For example, in FIGS. 8 and 9, a person B that is closest to the center, has the largest face area, and faces the front has a correlation value P4 that is almost the same as that of person A or person C. The face detection reliability higher than that of the person C is obtained.

  Next, the face direction determination unit 2A determines the vertical and horizontal directions of each detected face area based on the face area reliability.

  That is, for the detected face area in which the absolute value of the detected roll face angle is in the range of 70 ° to 110 °, the face direction determination unit 2A performs vertical detection in which the detected face area is in an upright state in the vertical image. It is determined that the face area. On the other hand, the face direction discriminating unit 2A, for the detected face area whose absolute value of the detected roll face angle is not less than 0 ° and less than 70 ° and greater than 110 ° and less than 180 °, the detected face region is upright in the lateral image. It is determined that the face detection area is in the detected state.

  The face direction determination unit 2A multiplies the face area reliability for the vertically detected face area by “−1”, and multiplies the face area reliability for the horizontally detected face area by “+1” to thereby obtain the vertically detected face. The face area reliability of the area is expressed as “negative”, and the face area reliability of the side detection face area is expressed as “positive”.

  Then, the face direction determination unit 2A calculates the sum total value by adding the face area reliability, and determines whether the sum total value is “positive” or “negative”.

  When the face direction determination unit 2A determines that the total addition value is “positive”, the total face area reliability for the horizontally detected face area is larger than the total face area reliability for the vertically detected face area, Since the reliability of the lateral detection face area is high as the entire captured image data, it is determined that the captured image data is a lateral image captured in a horizontal posture.

  And the control part 2 produces | generates the image direction information showing that this captured image data is a horizontal direction image, and is made to add to the said captured image data.

  On the other hand, when the face direction determination unit 2A determines that the total addition value is “negative”, the sum of the face area reliability for the vertically detected face area is the sum of the face area reliability for the horizontally detected face area. Since the reliability of the vertical detection face area is high as the entire captured image data, it is determined that the captured image data is a vertical image in which the vertical detection face area is upright.

  At this time, the control unit 2 adds up the detected roll face angles for the vertically detected face, and when the total value is “positive”, it represents that the image is rotated by −90 ° when the captured image data is displayed. While generating image direction information, if the total value is “negative”, image direction information indicating that the imaged image data is rotated by + 90 ° is generated and added to the imaged image data. It is made to do.

  For example, the face direction discriminating unit 2A has three captured persons (persons A, B, and C, detection roll face angles of 0 °, 0 °, and −20 °) with respect to the captured image data shown in FIG. It is determined that the face detection area is a horizontal detection face area, and the captured image data can be determined as a horizontal image.

  Further, the face direction discriminating unit 2A laterally detects two persons (persons B and C, detection roll face angles of 0 ° and −20 °) among the three captured people from the captured image data shown in FIG. A face area, one person (person A, detection roll face angle is −90 °) is determined as a vertically detected face area, and the captured image is determined based on the face area reliability indicating the reliability of each detected face area. The data can be discriminated as a lateral image in which the lateral detection face area is upright.

  Further, with respect to the captured image data shown in FIG. 10A, the face direction determination unit 2A determines that the two captured persons (the detected roll face angles are both −90 °) are both vertical detection face areas. Then, the captured image data is determined as a vertical image in which the vertical detection face area is upright. At this time, since the total value of the detected roll face angles becomes “negative”, the control unit 2 generates image direction information indicating that the display roll is rotated by + 90 °.

  As described above, when a plurality of detected face areas are detected from the captured image data, the digital still camera 1 determines the vertical and horizontal directions of the captured image data when each detected face area is in the upright state. By calculating the face area reliability for each detected face area, the reliability for each detected face area is expressed as a relative numerical value between the detected face areas, and the sum of the face area reliability is defined as a vertically detected face area. Compare with the side detection face area. Then, the digital still camera 1 determines that the vertical and horizontal directions when the detected face area having the higher reliability of the face area and the higher reliability is in the upright state are the vertical and horizontal directions of the captured image data. did.

  That is, the digital still camera 1 basically sets the vertical and horizontal directions of the captured image data determined that the majority of the detected face areas are in an upright state as the vertical and horizontal directions of the captured image data, and the reliability of each detected face area. Is calculated as the face area reliability, and the vertical and horizontal directions of the captured image data when the detected face area with the high face area reliability is in an upright state is more important than the detected face area with the low face area reliability. By viewing, it is possible to accurately determine the direction of the captured image.

(4) Display of Captured Image Data Next, a process for displaying captured image data to which image direction determination processing is applied and image direction information is added will be described.

  When the control unit 2 (FIG. 1) of the digital still camera 1 recognizes that a request to display the captured image data on the display unit 13 is made by an operation input via the user operation unit 6, the image storage unit The compressed image data corresponding to the captured image data is read from 12 and supplied to the image processing unit 11.

  The control unit 2 decompresses the compressed image data by the image processing unit 11 to generate captured image data, and confirms the image direction information added to the captured image data.

  At this time, when image direction information indicating that the captured image data is a horizontal image is added, the control unit 2 displays the captured image data as a horizontal image on the display unit 13 as it is (FIG. 2). (B)).

  On the other hand, when the image direction information indicating that the captured image data is a vertical image is added, the control unit 2 determines that the image is a vertical image as illustrated in FIG. The captured image data is rotated by 90 ° or −90 ° in accordance with the image direction information, and the captured image data is reduced and displayed in accordance with the vertical size of the display unit 13 as shown in FIG. Displayed on the unit 13.

  On the other hand, when the imaging direction information is not added to the captured image data, the control unit 2 determines that the captured image data is captured in an imaging mode other than the face detection mode and thus the vertical and horizontal directions are unknown. The captured image data is displayed on the display unit 13 as it is as a lateral image that is highly likely to be captured.

  As a result, the control unit 2 does not bother to rotate the digital still camera 1 with respect to the user who is viewing the display unit 13 with the digital still camera 1 in the horizontal posture. In this way, a captured image consisting of both vertical and horizontal directions can be viewed in an appropriate direction.

(5) Image Direction Determination Processing Procedure Next, an image direction determination processing procedure executed according to the image direction determination program will be described with reference to the flowchart shown in FIG.

  When a shutter (not shown) is pressed by the user, the digital still camera 1 executes an imaging process and starts an image direction determination processing procedure RT1, and proceeds to the next step SP1.

  In step SP1, the digital still camera 1 determines whether or not the current imaging mode is the face detection mode. If a negative result is obtained, the digital still camera 1 proceeds to an end step and ends the process, while an affirmative result is obtained. If it is obtained, the process proceeds to the next step SP2.

  In step SP2, when the digital still camera 1 detects a human face area from the captured image data, the digital still camera 1 proceeds to the next step SP3.

  In step SP3, the digital still camera 1 determines whether or not there is a detected face area detected as a human face area from the captured image data. If a negative result is obtained, the process proceeds to an end step. On the other hand, if a positive result is obtained, the process proceeds to the next step SP4.

  In step SP4, the digital still camera 1 determines whether or not the detected face area is 2 or more.

  If a negative result is obtained here, this indicates that the direction of the captured image can be simply determined based on one detected face area. At this time, the digital still camera 1 performs the next step SP5. If the direction of the captured image is determined based on the detected roll angle of the detected face area, the process proceeds to the next step SP15.

  On the other hand, if an affirmative result is obtained in step SP4, this indicates that there are a plurality of detected face regions. At this time, the digital still camera 1 proceeds to the next step SP7 and performs face direction determination processing. Execute.

  In step SP7, the digital still camera 1 determines the face area area P1 representing the area of each detected face area (that is, the size of the detected face area), the distance from the center of the captured image to each detected face area (that is, in the captured image data). A face area center distance P2 representing the position of the detected face area), a yaw face angle (reference face angle information P3) representing rotation about the face vertical axis FP of each detected face area (that is, the front degree of the detected face area) Based on the correlation value P4 representing the matching rate (that is, human face-likeness) with respect to the reference face data of each detected face area, the value of each evaluation value (area evaluation value, center evaluation value, front evaluation value and correlation evaluation value) is calculated. Once calculated, the process proceeds to the next step SP8.

  In step SP8, when the digital still camera 1 calculates the face area reliability representing the total reliability for the detected face area for each detected face area by adding the evaluation values, the process proceeds to the next step SP9. .

  In step SP9, based on the detected roll face angle, the digital still camera 1 captures image data when the detected face area is in an upright state for each detected face area as a vertical image or a horizontal image. If it is discriminate | determined, it will move to following step SP10.

  In step SP10, the digital still camera 1 multiplies the face area reliability of the vertical detection face area determined to be a vertical image by -1, while the horizontal detection face area determined to be a horizontal image. When the face area reliability of +1 is multiplied by +1, the process proceeds to the next step SP11.

  In step SP11, when the digital still camera 1 adds the face area reliability for all the detected face areas to calculate the sum total value, the process proceeds to the next step SP12.

  In step SP12, the digital still camera 1 determines whether the added total value is positive or negative.

  If a negative result is obtained here, this means that if the direction of the captured image determined for each detected face area and the reliability of each detected face area are considered in a majority manner, the captured image data is This indicates that there is a high possibility that the image is a directional image. At this time, the digital still camera 1 proceeds to step SP13, and if the captured image data is determined to be a vertical image, the process proceeds to the next step SP15.

  On the other hand, if an affirmative result is obtained in step SP12, this means that the majority of the reliability of each detected face area is taken into consideration, and the captured image data is likely to be a horizontal image. At this time, the digital still camera 1 proceeds to step SP14. When the captured image data is determined to be a horizontal image, the process proceeds to the next step SP15.

  In step SP15, the digital still camera 1 generates image direction information indicating that the captured image data is a captured image in the vertical and horizontal directions determined in step SP5, step SP13, or step SP14, and this is used as the captured image data. If it is added, the process proceeds to an end step and the process is terminated.

(6) Image Data Display Processing Procedure Next, an image data display processing procedure executed according to the image direction determination program will be described with reference to the flowchart shown in FIG.

  When the user requests that the captured image data be displayed on the display unit 13, the digital still camera 1 starts the image data display processing procedure RT2, and proceeds to step SP31.

  In step SP31, the digital still camera 1 determines whether or not image direction information is added to the captured image data generated by reading and decompressing the corresponding compressed image data from the image storage unit 12.

  If a negative result is obtained here, this means that the captured image data is captured in an imaging mode other than the face detection mode, and the possibility of a lateral image is high. At this time, the digital still camera 1 Then, the process proceeds to the next step SP33.

  On the other hand, when a positive result is obtained in step SP31, the digital still camera 1 proceeds to the next step SP32 and determines whether or not the image direction information represents a horizontal image.

  If a positive result is obtained here, this indicates that the captured image data is a horizontal image, and at this time, the digital still camera 1 proceeds to the next step SP33.

  In step SP33, when the digital still camera 1 displays the captured image data as it is on the display unit 13 (FIG. 1), the process proceeds to an end step and ends.

  On the other hand, if a negative result is obtained in step SP32, this indicates that the image direction information is a vertical image. At this time, the digital still camera 1 has an angle (± 90) represented by the image direction information. When the captured image data is rotated and reduced and displayed on the display unit 13 by (°), the process proceeds to an end step and the process ends.

(7) Operation and Effect In the above configuration, the digital still camera 1 detects the human face area FA included in the captured image data as the image data as a detected face area, and is perpendicular to the plane represented by the captured image data. Based on the roll face angle, which is the angle of the detected face area when the axis is the rotation axis, the vertical and horizontal directions of the plurality of detected face areas in the captured image data are respectively determined, and based on the determination results of the plurality of detected face areas The vertical and horizontal directions of the captured image data are determined.

  As a result, even if the digital still camera 1 discriminates the vertical and horizontal directions of the detected face area due to a face detection error or the like, the digital still camera 1 determines the captured image data by majority decision with the discrimination result of other detected face areas. Since the vertical and horizontal directions can be comprehensively determined, the vertical and horizontal directions of the captured image data can be determined more accurately than in the case of determining the vertical and horizontal directions of the captured image data based on one detected face area.

  Further, the digital still camera 1 discriminates the vertical and horizontal directions of the captured image data by placing more importance on the vertical and horizontal directions of the detected face area representing the reliability of the detected face area than the vertical and horizontal directions of the other detected face areas. By doing so, the vertical and horizontal directions of the captured image data can be more accurately determined as compared with the case where the vertical and horizontal directions of the detected image data are simply determined by the majority of the detected face area in the vertical and horizontal directions.

  Further, the digital still camera 1 determines the absolute value of the detected roll face angle when determining the range of the absolute value of the detected roll face angle (70 ° or more and 110 ° or less) when it is determined as the vertical detection face region as the horizontal detection face region. It was narrower than the range of values (0 ° or more and less than 70 ° and more than 110 ° and less than 180 °).

  Here, when lying down or leaning on something, it is assumed that the person tilts a large face (20 ° or more), and conversely, when standing upright, it is difficult to assume a large tilt. Also, when lying down, it is assumed that the whole body of the person fits in the captured image data, and when something is leaning on, it is assumed that “something” is captured at the same time. There is a high possibility that data is captured as a lateral image.

  Therefore, the digital still camera 1 preferentially uses the lateral detection face area by increasing the absolute value range of the detection roll face angle used when determining that the face detection area has a higher possibility of tilting the face. By discriminating, the accuracy of discrimination of the detected face area in the vertical and horizontal directions can be further improved.

  According to the above configuration, the digital still camera 1 determines the vertical and horizontal directions of each detected face area from the roll face angle of the detected face area detected from the captured image data, and the vertical detection face area determined as the vertical direction. And the horizontal detection face area determined as the horizontal direction, and the vertical and horizontal directions of the captured image data are determined in a majority manner, so that it is accurate even when there are multiple faces. In addition, it is possible to realize an image processing apparatus, an image direction determination method and an image direction determination program, a playback apparatus, a playback method and a playback program, and a digital still camera that can determine the vertical and horizontal directions of image data.

(8) Other Embodiments In the above-described embodiment, the vertical and horizontal directions of captured image data at the time of imaging are determined using the face area reliability calculated as the reliability for each detected face area. Although the case has been described, the present invention is not limited to this. For example, the vertical and horizontal directions of the captured image data at the time of imaging may be determined by simply voting the vertical detection face area and the horizontal detection face area.

  In this case, the digital still camera 1 executes an image direction determination process according to the flowchart RT3 shown in FIG. That is, when the digital still camera 1 enters from the start step and executes steps SP1 to SP4 as in the image direction determination processing procedure RT1 (FIG. 11), the process proceeds to step SP5 or step SP9. In step SP9, the digital still camera 1 that has determined whether each detected face area is a vertically detected face area or a horizontally detected face area. In step SP20, the digital still camera 1 sets the determination value for the vertically detected face area to “−1” and the horizontally detected face area. The discrimination value for the detected face area is held as “+1”, and the process proceeds to the next step SP21.

  In step SP21, the digital still camera 1 adds the discriminant values for each detected face area and calculates the sum total value. Then, the digital still camera 1 proceeds to the next step SP12 and performs the same steps as in the image direction discrimination processing procedure RT1 (FIG. 11). Processing up to SP15 is executed.

  As described above, the digital still camera 1 discriminates the vertical and horizontal directions, which occupy the majority in the determination results of the plurality of detected face areas, as the vertical and horizontal directions of the captured image data, so that there are a plurality of detected face areas in the captured image data. Even so, it is possible to appropriately determine the vertical and horizontal directions of the captured image data.

  Further, in the above-described embodiment, the case where up to 10 detected face areas are detected for captured image data in which 10 or more detected face areas exist is described. However, the present invention is not limited to this. For example, up to 5 detected face areas may be detected, or all detected face areas may be detected.

  Furthermore, in the above-described embodiment, the case where the face area FA that is a circle or an ellipse previously added to the reference face data is detected has been described, but the present invention is not limited to this, and for example, a part of the face May be detected in a square shape, or a face area FA larger than the face may be detected as the face area FA.

  Furthermore, in the above-described embodiment, the case where the detected face area is detected from the captured image data by comparing with the reference face data has been described. However, the present invention is not limited to this, and for example, contour extraction processing and skin color The detected face area may be detected by the detection process.

  Further, in the above-described embodiment, detection is performed based on the size of the detected face area, the position of the detected face area in the captured image data, the front degree (yaw face angle) of the detected face area, and the human face-likeness. Although the case where the face area reliability that comprehensively represents the reliability of the face area is calculated has been described, the present invention is not limited to this, and if the face area reliability is calculated based on at least one of them, There are no restrictions on the combination. Further, the face area reliability may be calculated using other reliability factors (for example, the coincidence rate of focal lengths, etc.).

  Furthermore, in the above-described embodiment, the case where the area of the detected face area is used as the size of the detected face area has been described. However, the present invention is not limited to this, for example, the vertical or horizontal direction of the detected face area. The maximum length or the like may be used.

  Furthermore, in the above-described embodiment, the case where the center distance from the center of the captured image to each detected face region is used as the position of the detected face region in the captured image data is described, but the present invention is not limited to this. For example, the coordinates of the detected face area may be used.

  Furthermore, in the above-described embodiment, the case where the yaw face angle is used as the frontal degree of the detected face area has been described. However, the present invention is not limited to this. From the relative positional relationship between the part and the white eye part (i.e., where the black eye is in the whole eye) and the yaw face angle, the direction in which the person having the detected face area is actually looking is calculated as the eye angle, This may be used as the degree of front of the detected face area.

  Furthermore, in the above-described embodiment, the case where the correlation value P4 representing the matching rate with the reference face data is used as the human face-likeness has been described. However, the present invention is not limited to this. It is also possible to compare face data such as dogs and cats that are relatively similar to a human face, and subtract the matching rate from the correlation value P4.

  Further, in the above-described embodiment, the case has been described in which it is determined that the detected roll face angle is the vertical detection face area when the absolute value of the detected roll face angle is in the angle range of 70 ° to 110 °. However, the present invention is not limited to this, and for example, when it is within an angle range of 45 ° or more and 135 ° or less, it may be determined as a vertical detection face region.

  Furthermore, in the above-described embodiment, the case where the face evaluation process is executed only when two or more detected face areas are detected is described, but the present invention is not limited to this, and one detected face area is provided. The face evaluation process may be executed in the same manner even when only the image is detected.

  In this case, the digital still camera 1 executes an image direction determination process according to the flowchart RT4 shown in FIG. That is, the digital still camera 1 enters from the start step in the same manner as the image direction determination processing procedure RT1 (FIG. 11) and executes steps SP1 to SP3. If an affirmative result is obtained in step SP3, steps SP7 to SP11 are performed. Execute the process.

  In step SP25, the digital still camera 1 that has calculated the total addition value in step SP11 determines whether or not the absolute value of the total addition value is equal to or greater than a predetermined certain threshold value.

  If an affirmative result is obtained here, this means that there is a sufficient difference in the sum of the face area reliability in the vertical and horizontal directions, so that the vertical and horizontal direction discrimination of the captured image data based on the added total value is reliable. At this time, the digital still camera 1 proceeds to the next step SP12, and when the processes of steps SP12 to SP15 are executed, the process proceeds to an end step and the process is terminated.

  On the other hand, if a negative result is obtained in step SP25, this means that the sum of the face area reliability in the vertical and horizontal directions is antagonized, and thus the vertical and horizontal discrimination of the captured image data based on the added total value is reliable. In this case, the digital still camera 1 moves to an end step and ends the process.

  As described above, the digital still camera 1 can detect the vertical and horizontal directions of the detected face area determined based on the roll face angle and the vertical and horizontal directions of the captured image data based on the face area reliability. Regardless of the number of detected face areas, the vertical and horizontal directions of the captured image data can be accurately determined based on the face area reliability.

  In addition, the digital still camera 1 determines the vertical and horizontal directions of the captured image data only when the absolute value of the total addition value is equal to or greater than the certain threshold and the face area reliability has a sufficient difference between the vertical and horizontal directions of the detected face area. Thus, the vertical and horizontal directions of the captured image data can be determined only when certainty can be ensured. If it is not certain, the vertical and horizontal directions of the captured image data are not discriminated, and as a result, the image is processed as a horizontal image having a high possibility of being captured.

  Furthermore, in the above-described embodiment, the case where the image direction information is added to the captured image data has been described. However, the present invention is not limited to this, and for example, a request from the user to display the captured image data. The image direction discrimination processing procedure RT1 may be executed when there is an error. In this case, the digital still camera 1 does not necessarily need to add the image direction information to the captured image data, and can display the captured image data on the display unit 13 according to the determined vertical and horizontal directions.

  Further, in the above-described embodiment, the case has been described in which the image direction information is added by determining the vertical and horizontal directions of the captured image data depending on whether the total addition value is “positive” or “negative”. Is not limited to this. For example, it is determined whether or not the total addition value is less than a predetermined total threshold, and since there is a difference in the vertical and horizontal directions in the total facial area reliability, it is determined when the total addition value is equal to or greater than the certain threshold While the vertical and horizontal directions are determined and the image direction information is added as it is, the total of the face area reliability in the vertical and horizontal directions is antagonized. It may be possible to confirm whether the data is correct or not.

  Furthermore, in the above-described embodiment, the case has been described in which the image direction information is added to the captured image data even when the captured image data is in either the vertical or horizontal direction. Not limited to this, it may be added only when the image is a vertical image. In this case, the digital still camera 1 rotates and displays captured image data determined to have image direction information added thereto.

  Furthermore, in the above-described embodiment, the case has been described in which image direction information is automatically added to captured image data when the vertical and horizontal directions of the captured image data are determined. However, the present invention is not limited to this. When the vertical / horizontal direction confirmation of the determined captured image data is displayed on the display unit 13 and the vertical / horizontal direction of the captured image data is determined according to the user's operation, the image direction information indicating the vertical / horizontal direction is displayed as the image data You may make it add to. Thereby, the vertical and horizontal directions of the captured image data can be more accurately determined.

  Further, in the above-described embodiment, the case where the image direction determination processing procedure RT1 is executed on the captured image data as the image data has been described. However, the present invention is not limited to this, and for example, a personal computer or the like You may make it perform with respect to the various image data edited and produced.

  Further, in the above-described embodiment, when the digital still camera 1 itself executes the image direction determination processing procedure RT1 and the image data display processing procedure RT2 on the captured image data captured by the digital still camera 1. However, the present invention is not limited to this. For example, the image direction determination processing procedure RT1 and the image data display processing procedure RT2 are executed on various image data acquired by various playback devices such as a hard disk recorder and a cellular phone from the outside. You may make it do.

  Further, in the above-described embodiment, the case where the digital still camera 1 itself executes the image direction determination processing procedure RT1 on the captured image data captured by the digital still camera 1 has been described. However, the present invention is not limited to this. For example, the image direction determination processing procedure RT1 may be executed when various image data acquired from the outside by an image processing apparatus such as a personal computer or a television is acquired. .

  Furthermore, in the above-described embodiment, the case where the digital still camera 1 displays captured image data has been described. However, the present invention is not limited to this, and for example, a television that acquires captured image data from the digital still camera 1. Various display devices such as John and a mobile phone may display the captured image data in accordance with the image data display processing procedure RT2.

  Further, in the above-described embodiment, the case where the image orientation determination program or the like is stored in the ROM or the flash memory 5 in advance has been described. However, the present invention is not limited to this, and the memory stick (registered trademark of Sony Corporation) is used. You may make it install in the flash memory 5 etc. from external storage media, such as. In addition, an image orientation determination program or the like is acquired from the outside via a wireless LAN (Local Area Network) such as USB (Universal Serial Bus) or IEEE (Institute of Electrical and Electronics Engineers) 802.11a / b / g. Also good.

  Further, in the above-described embodiment, the digital still camera 1 as the image processing apparatus is configured by the face detection block 20 as the face detection unit and the face direction determination unit 2A as the face direction determination unit. As described above, the present invention is not limited to this, and the image processing apparatus of the present invention may be configured by a face detection unit and a face direction determination unit having various other configurations.

  Furthermore, in the above-described embodiment, the digital still as a playback device is constituted by the face detection block 20 as the face detection unit, the face direction determination unit 2A as the face direction determination unit, and the image processing unit 11 as the image processing unit. Although the case where the camera 1 is configured has been described, the present invention is not limited to this, and the playback device of the present invention is configured by a face detection unit, a face direction determination unit, and an image processing unit having various other configurations. You may make it comprise.

  Furthermore, in the above-described embodiment, the signal processing unit 9 as the signal processing unit, the face detection block 20 as the face detection unit, the face direction determination unit 2A as the face direction determination unit, and the image as the image processing unit Although the case where the digital still camera 1 as a digital still camera is configured by the processing unit 11 has been described, the present invention is not limited thereto, and other signal processing units having various configurations, a face detection unit, You may make it comprise the digital still camera of this invention by a face direction discrimination | determination part and an image process part.

  The present invention can be used, for example, in various electronic devices having an image processing function.

It is a basic diagram which shows the whole structure of a digital still camera. It is a basic diagram with which it uses for description of the imaging by a horizontal posture. It is a basic diagram with which it uses for description of the imaging by a vertical attitude | position. It is an approximate line figure used for explanation of a definition of a direction. It is an approximate line figure used for explanation of a definition of a roll face angle. It is an approximate line figure used for explanation of a definition of a yaw face angle. It is a basic diagram with which it uses for description when many faces exist. It is a basic diagram with which it uses for description of calculation (1) of face area reliability. It is a basic diagram with which it uses for description of calculation (2) of face area reliability. It is a basic diagram with which it uses for description of the discrimination | determination and display of a longitudinal image. It is a flowchart with which it uses for description of an image direction discrimination | determination processing procedure. 5 is a flowchart for explaining an image data display processing procedure. It is a basic diagram which shows the image direction discrimination | determination processing procedure (1) by other embodiment. It is a basic diagram which shows the image direction discrimination | determination processing procedure (2) by other embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Digital still camera, 2 ... Control part, 2A ... Face direction discrimination part, 5 ... Flash memory, 6 ... Operation part, 9 ... Signal processing part, 11 ... Image processing part, 12 ... Image storage unit, 20 ... face detection block, FP ... face vertical axis.

Claims (18)

A face detection unit that detects a human face area included in the image data as a detection face area;
Based on the roll face angle, which is the rotation angle of the detected face area when the direction perpendicular to the plane represented by the image data is the rotation axis, the vertical and horizontal directions of the plurality of detected face areas included in the image data are respectively determined. An image processing apparatus comprising: a face direction determining unit that determines the vertical and horizontal directions of the image data based on the determination results of the plurality of detected face regions. The image processing apparatus according to claim 1, wherein the face direction determination unit determines a vertical and horizontal direction that occupies a majority of the determination results of the plurality of detected face regions as a vertical and horizontal direction of the image data. The face direction discrimination unit
2. The image processing apparatus according to claim 1, wherein the vertical and horizontal directions of the image data are discriminated by placing importance on the vertical and horizontal directions of the detected face area having higher reliability than the vertical and horizontal directions of the other detected face areas. The face direction discrimination unit
The image processing apparatus according to claim 1, wherein the size of the detected face area is used as the reliability of the detected face area. The face direction discrimination unit
The image processing apparatus according to claim 1, wherein the position of the detected face area in the image data is used as the reliability with respect to the detected face area. The face direction discrimination unit
The image processing apparatus according to claim 1, wherein the degree of front of the detected face area is used as the reliability of the detected face area. The face direction discrimination unit
The image processing apparatus according to claim 1, wherein the face-likeness of the detected face area is used as the reliability of the detected face area. The face direction discrimination unit
The image processing apparatus according to claim 1, wherein the detection face area in the horizontal direction is determined with priority over the detection face area in the vertical direction. The image processing apparatus according to claim 1, further comprising: an image processing unit that adds image direction information indicating a vertical and horizontal direction of the image data determined by the face direction determining unit to the image data. The image processing unit
The image processing apparatus according to claim 9, wherein the image direction information is added to the image data only when the image data is determined to be in the vertical direction. A display unit that displays the vertical and horizontal directions of the image data determined by the face direction determination unit and confirms whether the image data is correct;
The image processing unit according to claim 3, further comprising: an image processing unit that determines a vertical / horizontal direction of the image data in accordance with an operation of the user and adds image direction information representing the vertical / horizontal direction to the image data. Image processing device. A face detection step of detecting a face area of a person included in the image data as a detected face area;
Based on the roll face angle, which is the rotation angle of the detected face area when the direction perpendicular to the plane represented by the image data is the rotation axis, the vertical and horizontal directions of the plurality of detected face areas included in the image data are respectively determined. And a face direction determining step for determining the vertical and horizontal directions of the image data based on the determination results of the plurality of detected face areas. Against the computer,
A face detection step of detecting a face area of a person included in the image data as a detected face area;
Based on the roll face angle, which is the rotation angle of the detected face area when the direction perpendicular to the plane represented by the image data is the rotation axis, the vertical and horizontal directions of the plurality of detected face areas included in the image data are respectively determined. And a face direction determining step for determining a vertical and horizontal direction of the image data based on the determination results of the plurality of detected face regions. A face detection unit that detects a human face area included in the image data as a detection face area;
Based on the roll face angle, which is the rotation angle of the detected face area when the direction perpendicular to the plane represented by the image data is the rotation axis, the vertical and horizontal directions of the plurality of detected face areas included in the image data are respectively determined. A face direction determination unit that determines the vertical and horizontal directions of the image data based on the determination results of the plurality of detected face regions;
An image processing unit that displays the image data in the vertical and horizontal directions on a display unit by rotating the image data according to the vertical and horizontal directions of the image data by the face direction determination unit. Playback device. A face detection step of detecting a face area of a person included in the image data as a detected face area;
Based on the roll face angle, which is the rotation angle of the detected face area when the direction perpendicular to the plane represented by the image data is the rotation axis, the vertical and horizontal directions of the plurality of detected face areas included in the image data are respectively determined. A face direction determining step for determining the vertical and horizontal directions of the image data based on the determination results of the plurality of detected face regions;
An image processing step of displaying the image data in the vertical and horizontal directions on a display step by rotating the image data in accordance with the vertical and horizontal directions of the image data in the face direction determining step. Playback method. Against the computer,
A face detection step of detecting a face area of a person included in the image data as a detected face area;
Based on the roll face angle, which is the rotation angle of the detected face area when the direction perpendicular to the plane represented by the image data is the rotation axis, the vertical and horizontal directions of the plurality of detected face areas included in the image data are respectively determined. A face direction determining step for determining the vertical and horizontal directions of the image data based on the determination results of the plurality of detected face regions;
An image processing step of displaying the image data in the vertical and horizontal directions on a display step by rotating the image data in accordance with the vertical and horizontal directions of the image data in the face direction determining step. Playback program. A signal processing unit that generates captured image data from imaging light captured via a lens;
A face detection unit for detecting a face area of a person included in the captured image data as a detection face area;
The vertical and horizontal directions of the plurality of detected face areas included in the image data are respectively determined based on a roll face angle that is a rotation angle of the detected face area when a vertical direction with respect to a plane represented by the captured image data is a rotation axis. A face direction determination unit that determines the vertical and horizontal directions of the image data based on the determination results of the plurality of detected face regions;
A digital still camera comprising: an image processing unit that adds image direction information representing the vertical and horizontal directions of the captured image data determined by the face direction determining unit to the image data. A face detection unit for detecting a human face area as a detection face area from image data;
The vertical and horizontal directions of the image data are determined based on the vertical and horizontal directions of the detected face area determined based on the roll face angle with the vertical direction with respect to the plane represented by the image data as the rotation axis, and the reliability with respect to the detected face area. An image processing apparatus comprising: a face direction determining unit for determining.

Images