JP5267183B2 - Image layout apparatus and program - Google Patents

Description

  The present invention relates to an image layout apparatus and a program for laying out a plurality of images as one image.

  There is known a digital camera capable of laying out and recording captured images in a desired arrangement (see, for example, Patent Document 1).

JP 2008-211705 A

  However, it is not interesting to look at a layout image in which a plurality of images having no relevance are arranged at the same size, and it is not easy to create a layout image that is fun to see.

The invention of claim 1, in the image layout apparatus for creating a layout image formed by arranging a plurality of person images, and recording medium for recording a plurality of person image data, the person image data read out from said recording medium On the other hand, a facial expression acquisition means for acquiring facial expression information of a person in the image, and a first human image whose facial expression information is a first facial expression degree in a predetermined facial expression are closer to the center in the layout image. A second human image that is arranged in a large size and the facial expression information has a second facial expression level lower than the first facial expression level in the predetermined facial facial expression is placed around the center of the layout image. If there are a plurality of person images having the same facial expression degree in the predetermined facial expression and the arranging means that arranges with a size smaller than the first human image, the same facial expression degree Display means for displaying a list of a plurality of person images having a selection, and selection means for selecting at least one person image from the plurality of person images displayed in the list. The person image selected by the means is arranged as the first person image or the second person image .
According to a second aspect of the present invention, there is provided an image layout apparatus for creating a layout image formed by arranging a plurality of person images, a recording medium for recording a plurality of person image data, and the person image data read from the recording medium On the other hand, the facial expression acquisition means for acquiring facial expression information of a person in the image, and the first person image whose facial expression information is a smile or joy face are arranged in a large size near the center in the layout image, The second person image whose facial expression information is an angry face or a crying face comprises placement means for placing the second person image around the center of the layout image in a size smaller than the first person image. To do.
According to a third aspect of the present invention, there is provided an image layout apparatus for creating a layout image formed by arranging a plurality of person images, a recording medium for recording a plurality of person image data, and the person image data read from the recording medium On the other hand, a facial expression acquisition means for acquiring facial expression information of a person in the image, and a first human image whose facial expression information is a first facial expression degree in a predetermined facial expression are closer to the center in the layout image. A second human image that is arranged in a large size and the facial expression information has a second facial expression level lower than the first facial expression level in the predetermined facial facial expression is placed around the center of the layout image. Arrangement means for arranging in a size smaller than that of the first person image.

  According to the present invention, it is possible to easily create a layout image that is more enjoyable than a conventional layout image.

The control block diagram of the digital camera in one Embodiment of this invention. The figure which shows an example of the image recording availability setting by a facial expression. The flowchart which shows the process sequence at the time of smile layout mode setting. FIG. 4 is a flowchart showing details of layout processing of FIG. 3. FIG. The figure which shows the example of an operation screen in smile layout mode. The figure which shows the example of a screen after layout.

An embodiment when the present invention is applied to a digital camera will be described with reference to FIGS.
FIG. 1 is a control block diagram of the digital camera in the present embodiment. The digital camera refers to all devices having a digital camera function (for example, including a mobile phone with a camera function).

  In FIG. 1, the subject luminous flux that has passed through the photographic lens 1 is imaged by the imaging device 2, and the imaging signal is input to the image processing unit 3. The image processing circuit 3a constituting the image processing unit 3 performs various processes on the input imaging signal to generate image data. The image data is displayed on a liquid crystal monitor 4 provided on the back of the camera or the like through processing by the display circuit 3b. When the shooting mode is set, the above imaging and image display are repeated, and so-called live view display (through image display) is performed. When an imaging instruction is given, imaging is performed again, and the generated image data is recorded on the recording medium 5 such as a memory card via the recording / reproducing circuit 3c.

  On the other hand, in the reproduction mode, the image data recorded on the recording medium 5 can be read out by the recording / reproduction circuit 3c and displayed on the liquid crystal monitor 4 through the processing by the image processing circuit 3a and the display circuit 3b.

  The CPU 6 controls the image processing unit 3 and a circuit (not shown) in response to an input from the operation unit 7. The operation unit 7 includes a power button and a release button, various operation members used for reproduction operation and information input, a touch panel disposed on the liquid crystal monitor 4, and the like.

Next, face recognition and facial expression recognition will be described.
The camera has a well-known face recognition function. When a person is present as a subject in the image data, the position and size of the person's face area are detected by image analysis, and the face portion is focused and exposed. Take control. A plurality of faces can be recognized in one image data, and a face recognition frame corresponding to the face size is displayed at the face position in the live view image. This is notified to the user.

  Further, in face recognition, it is possible to specify who is the person in the image data by performing a database search based on the recognized face feature information. In order to make this possible, the camera is provided with a database for face recognition, and in a predetermined face registration mode, image data obtained by imaging a person, facial feature points obtained from the image data, and the person Information (names, etc.) can be registered for a plurality of people in association with each other. The facial feature information includes feature points such as eyebrow, eye, nose, lip end points, face contour points, head apex, lower chin end points, and relative distances between the feature points. Then, the facial feature information obtained from the image data at the time of imaging is compared with the feature information in the database, and the person is specified when the similarity is found to exceed a predetermined threshold.

  Facial expression recognition is a function for recognizing facial expressions in the image data recognized by the face recognition by image analysis. In this embodiment, emotional expressions such as “smile”, “joy”, “sadness”, “anger”, and “surprise” can be detected as facial expressions. In particular, with regard to “smile”, the level (hereinafter referred to as smile level) can be recognized in several stages from a slight smile to a big smile. Level detection may be performed for facial expressions other than smiles (joy level, sadness level, anger level, surprise level, etc.). Furthermore, “wink” can be detected as an action of the subject person, and it can also be detected whether or not the line of sight faces the camera (lens).

  Each detection result is reflected in the presence or absence of image recording. The camera repeats imaging in the imaging mode and performs face recognition on the image data obtained each time. When a human face is recognized in the image data, facial expression recognition is performed, and imaging, facial recognition, and facial expression recognition are repeated until it is detected that the person has a preset facial expression. When it is detected that the set facial expression is reached, image data captured with the facial expression is recorded in the recording medium 5 as an image file. As a result, an image of the facial expression desired by the photographer can be obtained.

  In this embodiment, a facial expression that determines whether or not image recording is possible can be set for each person as a subject, and the setting contents can be registered in the face recognition database. FIG. 2 shows an example of the image recording permission / inhibition setting. In this example, information for three persons is registered. In the figure, a circle indicates “selected” (recording is possible), a cross indicates “non-selection” (recording is not possible), and the smile degree percentage indicates a range where a smile is recognized. For example, “smile degree of 10%” means a smile expression of 10% or more, and image recording is permitted if even a little smile. In the case of “smile degree of 50%”, image recording is not permitted with a slight smile level, but is permitted for the first time with a moderate smile level. “Smile level 0%” means that smile is not selected (images cannot be recorded with a smile). Smiles with different percentages are positioned as different facial expressions even with the same smile.

  Unlike other facial expressions of emotion, “wink” can be considered as a signal for permission to shoot or a shooting prompt from the subject person. Since we do not want to acquire a winking face image separately, if a wink is detected, the image is taken again after a predetermined time (for example, 3 to 5 seconds), and the resulting image data is recorded. It is desirable to do. “Gaze” means that the subject person is facing the camera, and when this is selected, when another selected facial expression or AND condition between “wink” and “gaze” is satisfied Image recording is allowed. When “line of sight” is not selected, image recording is permitted when any of the selected facial expressions is detected regardless of the direction of the line of sight.

  For example, in the case of the person A, any one of “smile degree” other than “sadness”, “joy”, “anger”, “surprise”, “wink” is detected, and the line of sight is facing the camera. Image recording is permitted when is detected. For those who don't usually laugh a little, “smile degree 10%” that allows image recording even with a little laugh may be effective. Also, for those who do not want to take the expression of “sadness”, it is better not to select “sadness”.

  In the case of the person B, an image is recorded when any expression of “smiling degree 50%”, “sadness”, “anger”, “wink” is detected and the line of sight is facing the camera. Is allowed. In the case of the person C, image recording is not permitted even if any emotional expression is detected unless the line of sight is directed toward the camera and winking is performed. This setting is effective when leaving the facial expression to the subject.

  Here, the result of facial expression detection is recorded in association with the image. For example, although an image is recorded as an image file, it is conceivable to describe a facial expression detection result as header information of the image file. According to this, when an image file is transferred to an external device, facial expression information can be read on the transfer device side and reflected in display or the like.

Next, an image layout function during image reproduction will be described.
The image layout function is to create a single layout image by laying out a plurality of images in a desired arrangement. In particular, in the present embodiment, a smile layout mode using the smile degree is possible as one form of the image layout.

  In the smile layout mode, the image size is determined depending on the level of smile. That is, an image with a high smile level is arranged in a large size, and an image with a low smile level is set in a small size. According to this, it is possible to make an image with a high smile degree stand out in the completed layout image. In order to stand out more, it is desirable to place an image with a high smile level (large size image) in the center and several images with a low smile level (small size images) around it. As described above, the image size and the arrangement position of the image are automatically determined based on the facial expression of the person in the image, so that it is possible to easily create an enjoyable layout image.

  The image used for the layout may be automatically determined by the camera or may be determined by the user. However, the layout image can be created more easily if the camera determines it. When the camera is determined, the CPU 6 selects, for example, an image with the highest smile level from among the candidate images, arranges it in a large size, and sets some of the smile levels below a predetermined value (for example, less than 20%). Select the images and place them around the large image.

  The candidate image refers to an image in which a smile is detected among all the images in the recording medium 5 or in a folder designated in advance. Further, if all images with smile detection are stored in the same folder (for example, a folder named as a smile folder), all the images in the folder become candidate images.

  For example, an image with a high smile level may be placed in a large size, an image with a smile level of “medium” in a medium size, and an image with a smile level of “low” in a small size. Good. Here, for example, the smile degree “high” may be determined as a smile degree of 90% or more, the smile degree “medium” as a smile degree of 20% or more and less than 50%, and the smile degree “low” as a smile degree of less than 20%. The smile degree “high” may be the highest smile degree, the smile degree “medium” may be the second highest smile degree, and the smile degree “low” may be the third highest smile degree. Also in this case, it is desirable to place the large size image in the center and determine the placement position so that the smaller size image is placed on the outer side.

  On the other hand, when the user himself / herself selects an image to be used for the layout, for example, when the user selects a smile image, the CPU 6 determines the image size (and arrangement position) based on the smile level for the image. It is possible to do.

FIG. 3 is a flowchart showing an example of a processing procedure when the smile layout mode is set.
In this example, a layout template is used for image arrangement, and an image to be arranged in the image frame is determined by selecting an image frame on the template.

  The program shown in FIG. 3 is activated by the CPU 6 when the smile layout mode is selected, and first displays a list of layout templates in step S1. In step S2, the user waits for any template to be selected by the user, and when selected, an operation screen including the selected template is displayed on the liquid crystal monitor 4 in step S3.

  FIG. 5 shows an example of the operation screen. In this template, a large image frame 51 is displayed at the center of the layout image frame 50, a medium image frame 52 is displayed at the upper left, and a small image frame 53 is displayed at the lower right. The small and medium image frames 52 and 53 are images that partially overlap the large image frame 51 and the large image frame 51 is the lowest.

  Next, in step S4, layout processing is performed. Referring to FIG. 4 showing the details of the layout processing, in step S41, the user waits for any image frame to be selected by the user. The selection method of the frame may be a method in which the selected frame is cyclically switched every time a predetermined operation member is pressed, and the selection is confirmed with the determination button. The selection of the frame may be confirmed by touching the image frame.

  If step S41 is negative, and if three images to be laid out have already been determined, the process returns from step S55. If step S41 is affirmed, the process proceeds to step S42 to determine which image frame has been selected.

  When the large image frame 51 is selected, the process proceeds to step S43, and it is determined whether or not there are a plurality of images having a smile level “high” among the candidate images. An image with a smile level of “high” is, for example, an image with a smile level of 90% or more. An image with the highest smile level may be used. When there is only one image with a smile level of “high”, the image automatically becomes an image to be placed in the large image frame. That is, step S43 is denied and the process proceeds to step S46, where the image is recognized as an image to be placed in the large image frame 51 and reflected in the display of FIG. The user can confirm that the image to be arranged in the large image frame 51 has been determined by looking at the operation screen of the liquid crystal monitor.

If there are multiple images with a smile level of “high”, all thumbnail images with a smile level of “high” are displayed on the thumbnail list page in step S44 to prompt the user to select. The user selects a favorite image from the thumbnail list by performing a predetermined operation. For example, in a camera employing a touch panel, image selection can be performed by touching a thumbnail image. When an image is selected, the process proceeds from step S45 to step S46, where the image is recognized as an image to be placed in the large image frame 51 and reflected in the display of FIG.
The thumbnail list page and the operation screen of FIG. 5 can be switched alternately by a predetermined button operation.

  The same processing is performed when the medium / small image frames 52 and 53 are selected (steps S47 to S50 and steps S51 to S54). Here, the image with the smile level “middle” to be displayed in the middle image frame 52 is an image with a smile level of 50% or more and less than 90%, for example, and the image with the smile level “low” to be displayed in the small image frame 53 is For example, the image has a smile level of less than 20%.

After steps S46, S50, and S54, the process proceeds to step S55, where it is determined whether or not three images to be laid out have already been determined. If NO, the process returns to step S41, and if YES, the process returns.
FIG. 6 shows an operation screen after image placement, which allows the user to check the layout image at a glance.

  In step S5 of FIG. 3, it is determined whether or not a layout adjustment operation has been performed. If the determination is affirmative, layout adjustment is performed in step S6.

  The layout adjustment is manual layout adjustment and is performed when the user determines that adjustment is necessary by looking at the image frame arrangement in FIG. On the operation screen, a movement start button 61, a movement stop button 62, a peripheral image switching button 63, and an image advance button 64 are displayed. When the movement start button 61 is operated, the image frames 51 to 53 on the display start moving little by little, and the movement is stopped by operating the movement stop button 62. Note that the movement is limited to the layout image frame. The peripheral image switching button 63 is used to interchange two peripheral images with each other, and is alternately switched every time an operation is performed. The image advance button 64 is used to replace the central image and the peripheral image with other images, and the images are sequentially switched every time an operation is performed.

  In step S7 of FIG. 3, the presence / absence of the determination operation is determined. If the determination operation is not confirmed, the process returns to step S4. If the determination operation is confirmed, the layout image is recorded on the recording medium 5 in step S8 and the process is terminated. The recorded layout image can be recalled at any time and displayed on the liquid crystal monitor 4 or transferred to an external device in the same manner as a normal image.

  In the process of FIG. 4, an example is shown in which the image selection is left to the user when there are a plurality of images with a smile level of “high” (“medium”, “low”), but this is selected by the camera (CPU 6). May be. For example, in a case where “favorite degree” can be embedded in an image file, an image having the highest “favorite degree” among a plurality of images having the same smile degree may be automatically selected. Alternatively, the image may be evaluated by detecting the focus state or the degree of camera shake by image analysis, and the image having the highest evaluation may be selected.

  The arrangement and number of images in the layout image are not limited to those shown in FIG. For example, a layout in which an image with a smile level of “high” is arranged large at the center and images with a smile level of “low” are arranged at the four corners may be used. Alternatively, more images (for example, all of smile images) may be extracted and arranged in a radial shape, a mandala shape, or a spiral shape. According to this, many images can be browsed at once.

  Although the layout based on the smile level has been described above, the layout process may be performed based on other facial expression levels (for example, joy level, sadness level, anger level, surprise level, etc.). At that time, in the case of an unfavorable expression, an image with a lower expression level may be displayed in a larger size. Further, the layout is not limited to the expression level, and a layout in which images of a plurality of facial expressions are mixed may be used. For example, an image of a preferable facial expression such as a smile or joy may be arranged in the center with a large size, and an undesirable facial expression such as anger or sadness may be arranged in a small size around the periphery. This makes it possible to enjoy a variety of layout images.

  You may change the color at the time of displaying a layout image according to a person's facial expression, the size or arrangement position of each image. For example, an image having a predetermined size or larger (an image to be emphasized) may be displayed in color, and other images may be displayed in monochrome. The monochrome display includes sepia and other single color displays in addition to the monochrome display. Further, the saturation may be increased for an image to be emphasized.

  Furthermore, when a layout image is displayed, each image can be moved like a screen saver. For example, the central image may be rotated around the central image without moving the central image. A plurality of image movement variations may be preset and configured so that the user can select them. Further, the center and peripheral images may be switched automatically. Furthermore, BGM may be played when moving an image.

  Although the above has been described using a digital camera, the present invention can also be applied to an electronic device that does not have an imaging function. For example, if an image layout program having the layout function described above is incorporated into a personal computer or the like, layout processing can be performed using an image transferred from a camera. At that time, the facial expression can be recognized by reading facial expression information associated with the image (for example, embedded in the image file). Alternatively, a face recognition function and an expression recognition function by image analysis may be incorporated in the image layout program.

DESCRIPTION OF SYMBOLS 1 Shooting lens 2 Image pick-up element 3 Image processing part 4 Liquid crystal monitor 5 Recording medium 6 CPU
51-53 Image frame 61 Movement start button 62 Movement stop button 63 Peripheral image switching button 64 Image forward button

Claims (18)

In an image layout apparatus for creating a layout image formed by arranging a plurality of person images,
A recording medium for recording a plurality of person image data;
Facial expression acquisition means for acquiring facial expression information of a person in the image for the person image data read from the recording medium ;
The first human image whose facial expression information is a first facial expression level in a predetermined facial expression is arranged in a large size near the center of the layout image, and the facial expression information is in the predetermined facial expression, A second person image having a second facial expression level lower than the first facial expression level is arranged at a size smaller than the first human image around the center of the layout image;
Display means for displaying a list of a plurality of person images having the same expression level when there are a plurality of person images having the same expression level in the predetermined facial expression;
Selecting means for selecting at least one person image from the plurality of person images displayed in a list;
The image layout apparatus , wherein the arrangement unit arranges the person image selected by the selection unit as the first person image or the second person image . In an image layout apparatus for creating a layout image formed by arranging a plurality of person images,
A recording medium for recording a plurality of person image data;
Facial expression acquisition means for acquiring facial expression information of a person in the image for the person image data read from the recording medium ;
The first person image whose facial expression information is a smile or joy face is arranged in a large size near the center of the layout image, and the second person image whose facial expression information is an angry face or a crying face is the layout. An image layout apparatus comprising: arrangement means for arranging the image at a size smaller than that of the first person image around a center of the image. In an image layout apparatus for creating a layout image formed by arranging a plurality of person images,
A recording medium for recording a plurality of person image data;
Facial expression acquisition means for acquiring facial expression information of a person in the image for the person image data read from the recording medium ;
The first human image whose facial expression information is a first facial expression level in a predetermined facial expression is arranged in a large size near the center of the layout image, and the facial expression information is in the predetermined facial expression, The second person image having a second expression level lower than the first expression level includes arrangement means for arranging the second person image at a size smaller than the first person image around the center of the layout image. An image layout apparatus characterized by that. The facial expression information, the image layout apparatus according to claim 1 or 3, characterized in that information indicating a facial expression degree of specific emotions. It said positioning means, the image layout apparatus according to claim 1 or 3, wherein the expression degree is arranged in a large size higher person image. Based on an external operation, the image layout apparatus according to any one of claims 1 to 5, further comprising an arrangement changing means for changing the position of the deployed person image by said arrangement means. When there are a plurality of person images having the same facial expression level in the predetermined facial expression , the apparatus further comprises image selection means for selecting the person image to be arranged based on the facial expression information ,
The image layout apparatus according to claim 3 , wherein the arrangement unit arranges the person image selected by the image selection unit as the first person image or the second person image . Based on an external operation, the image layout apparatus according to claim 7, wherein the image selecting means further includes an image changing means for changing the person selected image to another person image. 9. The color determination unit according to claim 1, further comprising a color determination unit that determines a color of each person image when the layout image is displayed based on a size of the person image arranged by the arrangement unit . The image layout apparatus according to claim 1. 10. The image layout apparatus according to claim 9, wherein the color determining means displays a person image having a predetermined size or more in a color display and other person images in a monochrome display.   The image layout apparatus according to claim 1, further comprising display means for displaying the layout image.   The image layout apparatus according to claim 1, further comprising recording means for recording the layout image on a recording medium.   The image layout apparatus according to claim 1, further comprising an imaging unit that captures an image of a subject to obtain an image.   The image layout apparatus according to claim 1, wherein the facial expression acquisition unit obtains a facial expression of a person in the image by image analysis.   The image layout apparatus according to claim 1, wherein the facial expression acquisition unit obtains facial expressions of a person in the image by reading facial expression information associated with the image. In an image layout program for causing a computer to execute a process of creating a layout image formed by arranging a plurality of person images,
A procedure for recording multiple person image data;
To the person image data, a step of acquiring the facial expression information of a person in the image,
The first human image whose facial expression information is a first facial expression level in a predetermined facial expression is arranged in a large size near the center of the layout image, and the facial expression information is in the predetermined facial expression, A second human image having a second facial expression level lower than the first facial expression level is arranged in a smaller size than the first human image around the center of the layout image;
When there are a plurality of person images having the same expression level in the predetermined facial expression, a procedure for displaying a list of the plurality of person images having the same expression level;
Selecting at least one person image from the plurality of person images displayed in the list;
An image layout program including a procedure for arranging the selected person image as the first person image or the second person image . In an image layout program for causing a computer to execute a process of creating a layout image formed by arranging a plurality of person images,
A procedure for recording multiple person image data;
To the person image data, a step of acquiring the facial expression information of a person in the image,
The first person image whose facial expression information is a smile or joy face is arranged in a large size near the center of the layout image, and the second person image whose facial expression information is an angry face or a crying face is the layout. An image layout program including a procedure for arranging the image at a size smaller than that of the first person image around the center of the image. In an image layout program for causing a computer to execute a process of creating a layout image formed by arranging a plurality of person images,
A procedure for recording multiple person image data;
To the person image data, a step of acquiring the facial expression information of a person in the image,
The first human image whose facial expression information is a first facial expression level in a predetermined facial expression is arranged in a large size near the center of the layout image, and the facial expression information is in the predetermined facial expression, An image layout including: a second person image having a second expression level lower than the first expression level is arranged at a size smaller than the first person image around a center in the layout image. program.

Images