JP2011134173A - Apparatus, method and program for displaying image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent a sub-image from being superimposed on the face of a target person when there are a lot of faces in a main image and the sub-image overlaps any of the faces. <P>SOLUTION: The image display apparatus includes: a face region detection part 52 that detects a face region of a person displayed in the main screen; a face importance determination part 55 that determines the importance of the detected face region in the main screen; and a display position setting part 56 that sets a display position of the sub-image in the main screen based on the importance of the face region. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image display device, an image display method, and an image display program.

  2. Description of the Related Art Conventionally, for example, in a photo viewer or the like, there are some which display a character string such as an icon or shooting information as a sub image superimposed on a main image that is a main display target. For example, in Patent Document 1, a main screen that displays a main image can be overlaid with a sub screen that displays a sub image, and the face can be seen according to the position of the face in the image displayed on the main screen. Controls the display of screens and sub-screens.

JP 2009-89051 A

  However, in Patent Document 1, in order to prevent the main screen face and the sub-screen from overlapping, the position and size of the sub-screen are controlled, or the sub-screen transparency is set so that the face can be seen when they overlap. Or control. If there are multiple human faces on the main screen and the sub screen overlaps one of these faces, the sub screen may overlap the face of an important person who wants to avoid hiding. is there. Further, even if the face can be transmitted from the sub-screen, the face becomes difficult to see.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following forms or application examples.

  Application Example 1 An image display device capable of displaying a main image and a sub-image that overlaps the main image, and a face area detection unit that detects a face area of a person displayed in the main image; A face importance level determination unit for determining the importance level in the main image for the detected face area, and a display position for setting the display position of the sub-image in the main image based on the importance level of the face area An image display device comprising: a setting unit.

  According to the image display device described above, the face importance level determination unit determines the importance level of the face area displayed in the main image. Then, the display position setting unit sets the display position of the sub image in the main image based on the importance of the face area. By setting the display position of the sub image in the main image based on the importance of the face area, the sub image can be displayed at a position that does not overlap with the face area having a high importance. As a result, the face of an important person who wants to avoid hiding can be seen without being hidden in the sub-image.

  Application Example 2 A storage unit that stores face feature information in which feature values of a person's face image are registered, and person importance information in which the importance of a person corresponding to the face feature information is registered, and the face area detection A face feature detection unit that detects a feature amount from the face region detected by a part; and the feature amount detected by the face feature detection unit and the stored face feature information A face recognizing unit for identifying the person, and the face importance determining unit is configured to determine the importance of the face region based on the stored person importance information corresponding to the identified person. The image display device according to claim 1, wherein:

  According to the above-described image display device, the face recognition unit identifies a person in the face area by comparing the feature amount detected from the face area with the face feature information. Then, the face importance level determination unit determines the importance level of the face area based on the person importance level information corresponding to the specified person. Thereby, the importance of the face area displayed on the main image can be obtained based on the pre-registered face feature information and person importance information.

  Application Example 3 In the image display device, a plurality of sub-image display areas that are candidate areas for displaying the sub-image are defined in advance in the area of the main image.

  According to the image display device described above, a plurality of sub-image display areas are defined in advance, and the display position setting unit displays sub-images based on the importance of the face area from among these sub-image display areas. The position can be set. Thereby, the sub image is displayed at a substantially constant position, and the user can easily confirm the sub image.

  Application Example 4 An image display method capable of displaying a main image and a sub-image that overlaps the main image, and a face area detection step of detecting a face area of a person displayed in the main image; A face importance level determining step for determining the importance level in the main image for the detected face area, and a display position for setting the display position of the sub image in the main image based on the importance level of the face area And a setting step.

  According to the image display method described above, the importance of the face area displayed in the main image is determined in the face importance determination step. In the display position setting step, the display position of the sub image in the main image is set based on the importance of the face area. By setting the display position of the sub image in the main image based on the importance of the face area, the sub image can be displayed at a position that does not overlap with the face area having a high importance. As a result, the face of an important person who wants to avoid hiding can be seen without being hidden in the sub-image.

  Application Example 5 An image display program capable of displaying a main image and a sub-image overlapping with the main image, a face area detection function for detecting a face area of a person displayed in the main image, And a display position for setting the display position of the sub-image in the main image based on the importance level determination function for determining the importance level in the main image and the importance level of the face area for the detected face area. An image display program for causing a computer to execute a setting function.

  According to the image display program described above, the importance of the face area displayed in the main image is determined by the face importance determination function. Then, the display position setting function sets the display position of the sub image in the main image based on the importance of the face area. By setting the display position of the sub image in the main image based on the importance of the face area, the sub image can be displayed at a position that does not overlap with the face area having a high importance. As a result, the face of an important person who wants to avoid hiding can be seen without being hidden in the sub-image.

The block diagram which shows the hardware constitutions of an image display apparatus. The block diagram which shows the function structure of an image display apparatus. The flowchart which shows operation | movement of an image display apparatus. The figure which shows the example of each sub image display area displayed in the area | region of a main image. The figure which shows the example in which a person is included in a main image. The figure which shows the example of the person table based on face feature information and person importance information. The figure which shows the example which calculates the importance of each sub image display area. The figure which shows the example which made the display position and area | region size of a sub image display area variable.

  Hereinafter, the image display apparatus according to the present embodiment will be described with reference to the drawings.

<Configuration of image display device>
First, the configuration of the image display apparatus according to the present embodiment will be described.
FIG. 1 is a block diagram showing a hardware configuration of the image display apparatus 1 according to the present embodiment. This image display device can be implemented as a function of an information processing device such as a photo viewer, a printer, a projector, a personal computer, a digital camera, or a mobile phone.

  The image display device 1 includes a CPU 11, an internal memory 12, an operation unit 13, a display unit 14, a communication interface 15, a storage unit 16, and the like.

  A CPU (Central Processing Unit) 11 controls the above-described units, and executes various processes according to programs stored in the internal memory 12. The internal memory 12 includes a ROM (Read Only Memory) that stores various programs executed by the CPU 11 and various data, and a RAM (Random Access Memory) that temporarily stores programs and data to be executed by the CPU 11. Yes.

  The operation unit 13 includes buttons, a touch panel, and the like, and notifies the CPU 11 of the operation content from the user. The display unit 14 is configured by a liquid crystal display, for example, and displays an image corresponding to data supplied from the CPU 11. The communication interface 15 is an interface for exchanging image data and the like from an information device such as a digital camera.

  The storage unit 16 includes, for example, a nonvolatile memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory) or a flash memory, or a hard disk. The storage unit 16 stores in advance face feature information 16a in which the feature amount of the person's face image is registered, and person importance information 16b in which the importance of the person corresponding to the face feature information 16a is registered. .

  FIG. 2 is a block diagram illustrating a functional configuration of the image display apparatus 1. As shown in the figure, the image display device 1 includes a main image display unit 51, a face area detection unit 52, a face feature detection unit 53, a face recognition unit 54, a face importance degree determination unit 55, a display position setting unit 56, a sub Each function of the image display unit 57 and the like is provided. These functions are realized by the CPU 11 executing a predetermined program stored in the internal memory 12.

The main image display unit 51 displays, for example, a person image or a landscape image over the entire surface of the display unit 14 as a main image. This main image is obtained from an information device such as a digital camera via the communication interface 15, for example.
The sub image display unit 57 displays, for example, icons, character strings such as shooting information, histograms, and the like as sub images so as to overlap a part of the main image. For this reason, a plurality of sub-image display areas that are candidate areas for displaying sub-images are defined in advance in the main image area. For each sub-image display area, definition information such as the display position, area size, and priority in the main image area is stored in the storage unit 16 in advance. This definition information can be appropriately set by the user via the operation unit 13.

  The face area detection unit 52 detects all face areas included in the image displayed by the main image display unit 51. In the present embodiment, for example, a face area is detected using a known detection method such as a pattern matching method using a template (see Japanese Patent Application Laid-Open No. 2006-279460).

  The face feature detection unit 53 detects a face feature amount from the face region detected by the face region detection unit 52. This feature amount is used to perform face recognition of a person in the face area. Here, the organ outlines of the face, eyes, and mouth are detected from the face area, and the relative positions in these face areas are detected. In the present embodiment, the face organ is detected using a known detection method such as a pattern matching method using a template, for example, in the same manner as the detection of the face area described above.

  The face recognition unit 54 is detected by the face region detection unit 52 by collating the feature amount detected from the face region in the face feature detection unit 53 with the face feature information 16a stored in the storage unit 16. Identify the person in the face area.

  The face importance level determination unit 55 refers to the person importance level information 16b stored in the storage unit 16 for the person specified by the face recognition unit 54, thereby determining the importance level of the person's face area in the main image. Ask. Here, a face area having a high importance level is a face area that is less likely to be hidden by overlapping sub-images in the main image than a face area having a low importance level. On the other hand, a face area with low importance is a face area that is more likely to be hidden by overlapping sub-images in the main image than a face area with high importance.

  Based on the face area detected by the face area detection unit 52, the display position setting unit 56 sets which sub-image display area to display the sub-image in among the plurality of sub-image display areas. At this time, the display position setting unit 56 may set the sub image display region based on the priority of each sub image display region, or based on the importance of the face region obtained by the face importance determination unit 55. is there.

<Operation of image display device>
Next, the operation of the image display apparatus according to this embodiment will be described.
FIG. 3 is a flowchart showing the operation of the image display apparatus 1 according to the present embodiment.

  First, the image display apparatus 1 displays the main image acquired via the communication interface 15 on the display unit 14 by the main image display unit 51 (step S110). In this flowchart, an example in which a person is included in the main image will be described.

  Next, in the image display apparatus 1, the face area detection unit 52 detects the face areas of all persons included in the main image displayed in step S110 (step S120).

  Next, the image display device 1 acquires the display position, area size, and priority of each sub-image display area to be displayed in the main image area from the storage unit 16 (step S130).

  FIG. 4 is a diagram illustrating examples of the sub image display areas A, B, C, and D that are area candidates to be displayed in the area of the main image M. In the figure, in step S130, each display position (four corners of the main image M) of each sub image display area A, B, C, D, each area size, and each sub image display area A, B, C, D. Priority (A> B> C> D) is acquired. 5A to 5C are diagrams illustrating an example in which a person is included in the main image M. Hereinafter, description will be made with reference to FIGS. 4 and 5 as an example.

  Next, the image display device 1 determines whether or not the face area detected in step S120 overlaps each sub-image display area (step S140). Here, it is determined whether or not the face coordinates of the face area are included in each sub image display area. In addition, the face coordinates of the face area to be compared with each sub-image display area are detected by detecting the face area of a person from the main image, a reduced image of the main image, or a thumbnail embedded in the image file in advance. As a result, the face coordinates of the obtained face area may be coordinate-converted according to the size of the display medium. Further, when the main image is reduced, the face area may be detected using an image reduced to the size of the display medium, and the obtained coordinates may be used as they are. The face area is represented by a rectangle, an ellipse, or the like, and the face coordinates of the face area indicate the coordinates of the internal points, and the coordinates are coordinates from a reference point such as the upper left pixel of the image.

  When the face area does not overlap any of the sub image display areas (step S140: Yes), the image display apparatus 1 sets the sub image display area at the initial position as the display area by the display position setting unit 56. Then, the sub image display unit 57 displays the sub image (step S160), and the process is terminated. In the example of FIG. 5A, the face area of the person P1 does not overlap any of the sub-image display areas A, B, C, and D. Accordingly, the sub image is displayed in the sub image display area A at the initial position.

  On the other hand, when the face area overlaps any of the sub image display areas (step S140: No), the image display apparatus 1 determines whether there is at least one sub image display area that does not overlap the face area. Is determined (step S150).

  If there is at least one sub-image display area that does not overlap the face area (step S150: Yes), the image display device 1 uses the display position setting unit 56 to give the highest priority from the sub-image display areas that do not overlap the face area. A sub-image display area with a high degree is set as the display area. Then, the sub image display unit 57 displays the sub image (step S170), and the process is terminated. In the example of FIG. 5B, the face areas of the persons P1 and P2 do not overlap the sub image display areas B and C. Furthermore, the sub image display area B has a higher priority than the sub image display area C. Accordingly, the sub image is displayed in the sub image display area B.

  On the other hand, if there is no sub-image display area that does not overlap with the face area, that is, if any face area overlaps with all the sub-image display areas (step S150: No), the process proceeds to the next step S180.

  In step S180, the image display device 1 performs organ contour detection in order to detect a feature amount from the face area overlapping each sub-image display area by the face feature detection unit 53 (step S180). Then, the image display device 1 uses the face recognition unit 54 and the face importance degree determination unit 55 to detect the feature amount of each face area detected in step S180, the face feature information 16a stored in the storage unit 16, and the person importance. Based on the degree information 16b, face recognition of each face area is performed to determine the importance of each face area in the main image (step S190).

In the example of FIG. 5C, the face area of any one of the persons P1, P2, P3, P11, and P12 overlaps the sub-image display areas A, B, C, and D. For this reason, the feature amount of the face area of each person P1, P2, P3, P11, P12 is detected in step S180, and the importance of the face area of each person P1, P2, P3, P11, P12 is obtained in step S190.
FIG. 6 is a diagram illustrating an example of a person table based on the face feature information 16a and the person importance information 16b. In the figure, for persons P1, P2, and P3, feature amounts T1, T2, and T3 that are face characteristic information of each person and importance levels 100, 50, and 100 that are person importance level information of each person are registered. . In addition, the person who does not correspond is registered as the importance 10. When the feature amounts of the face regions of the persons P1, P2, P3, P11, and P12 detected in step S180 are T1, T2, T3, T11, and T12, respectively, by comparing with the feature amounts of the person table in FIG. The importance levels 100, 50, 100, 10, and 10 of the face regions of the persons P1, P2, P3, P11, and P12 can be obtained.

  In step S200, the image display device 1 calculates the importance of each sub-image display area based on the importance of each face area obtained in step S190. Here, in each sub image display area, the importance is calculated by taking the sum of the importance of the face area overlapping with each sub image display area. In the image display device 1, the display position setting unit 56 sets the sub image display area having the lowest importance among the sub image display areas as the display area. Then, the sub image display unit 57 displays the sub image (step S210), and the process ends.

  FIG. 7 is a diagram illustrating an example of calculating the importance of each sub image display area. This figure shows calculation of importance in the examples of FIGS. 5C and 6. For example, in the sub image display area B, only the face area of the person P11 overlaps, and the importance level 10 of the person P11 is calculated as the importance level of the sub image display area B. In the sub image display area C, the face areas of the persons P3 and P12 overlap, and the importance 110 of the sub image display area C is calculated by taking the sum of the importance 100 and 10 of the persons P3 and P12. The In the example of FIG. 7, since the importance of the sub image display area B is the lowest, the sub image is displayed in the sub image display area B.

  In the embodiment described above, when any face area overlaps all the sub-image display areas as in the example of FIG. 5C, face recognition is performed by detecting the feature amount of each face area. Then, the importance of each sub image display area is calculated based on the importance of each face area, and the sub image is displayed in the sub image display area having the lowest importance. As a result, the sub image is displayed so as to overlap the face image of the less important person, avoiding the face image of the important person in the main image. Therefore, an important person who wants to avoid hiding his face can be seen without being hidden by the sub-image by registering it in the image display device 1 in advance.

(Modification 1)
In the embodiment described above, a plurality of sub-image display areas in which the display position, area size, and priority in the area of the main image are defined are set. However, the present invention is not limited to this, and a sub-image is displayed by setting one sub-image display area whose display position is variable and whose area size is fixed, or one sub-image display area whose display position and area size are both variable. You may make it do.

  In this case, the sub image display area is set at a position that does not overlap with the face area detected from the main image. Here, if the sub image overlaps any of the face areas of the main image, the sub image display area is set based on the importance of the face area. If the region size is variable, the region size may be reduced so as not to overlap the face region. As a result, it is possible to prevent the face image of the person displayed in the main image and the sub image from overlapping each other more finely.

  Further, the display position of the sub image display area may be set so as to be far from the face area having high importance. FIG. 8 is a diagram illustrating an example in which the display position and area size of the sub image display area are variable. When the person table of FIG. 6 is applied to the same figure, the importance of the face areas of the persons P1 and P3 becomes the highest. Accordingly, the reduced sub image display area E is set at a position farthest from the face areas of the persons P1 and P3, and the sub image is displayed. Thereby, when looking at the face of an important person in the main image, it is possible to suppress an undesirable effect such as anxiety that the sub-image is nearby.

(Modification 2)
In the embodiment described above, when calculating the importance of the sub image display area, the calculation is performed by taking the sum of the importance of the face area overlapping the sub image display area. However, the present invention is not limited to this. Based on the average importance level of the overlapping face areas, the highest importance level of the overlapping face areas, the area of the overlapping face areas or the ratio of the overlapping area parts, the number of overlapping face areas, etc. Thus, the importance of the sub image display area may be calculated. Also, the importance may be calculated by weighting each of these elements and the priority of the sub-image display area.

(Modification 3)
In the embodiment described above, the importance of the face area is obtained according to the importance registered for each person. However, the present invention is not limited to this. For example, identification of a child or adult, male or female is detected from the face area, and based on these identifications, the face area is pre-registered according to settings such as child priority or female priority. You may calculate the importance of.

  DESCRIPTION OF SYMBOLS 1 ... Image display apparatus, 11 ... CPU, 12 ... Internal memory, 13 ... Operation part, 14 ... Display part, 15 ... Communication interface, 16 ... Memory | storage part, 16a ... Face feature information, 16b ... Person importance information, 51 ... Main image display section 52... Face area detection section 53. Face feature detection section 54. Face recognition section 55 55 Face importance determination section 56 56 Display position setting section 57 57 Sub image display section

Claims (5)

An image display device capable of displaying a main image and a sub image overlapping the main image,
A face area detection unit for detecting a face area of a person displayed in the main image;
A face importance determination unit for determining the importance in the main image for the detected face area;
An image display device comprising: a display position setting unit that sets a display position of the sub image in the main image based on the importance of the face area. A storage unit for storing face feature information in which the feature amount of a person's face image is registered and person importance information in which the importance of the person corresponding to the face feature information is registered;
A face feature detection unit that detects a feature amount from the face region detected by the face region detection unit;
A face recognition unit for identifying a person in the face region by comparing the feature amount detected by the face feature detection unit with the stored face feature information;
The image according to claim 1, wherein the face importance level determination unit determines the importance level of the face area based on the stored person importance level information corresponding to the identified person. Display device.   The image display device according to claim 1, wherein a plurality of sub-image display areas that are candidate areas for displaying the sub-image are defined in advance in the main image area. An image display method capable of displaying a main image and a sub-image overlapping the main image,
A face area detection step of detecting a face area of a person displayed in the main image;
A face importance level determining step of determining the importance level in the main image for the detected face area;
A display position setting step of setting a display position of the sub-image in the main image based on the importance of the face area. An image display program capable of displaying a main image and a sub image overlapping the main image,
A face area detection function for detecting a face area of a person displayed in the main image;
A face importance determination function for determining the importance in the main image for the detected face area;
An image display program for causing a computer to execute a display position setting function for setting a display position of the sub image in the main image based on the importance of the face area.

Images