JP2016081319A - Edition support device for image material, album creation method and program for layout - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an edition support device for image material which reduces the number of times of operation by a user at the time of edition.SOLUTION: There are displayed, on a display device, a material display area 20 in which display images 21 of a plurality of image material recorded in a recording medium 14 are displayed and which allows acceptance of selection of first image material by a user, a layout selection area 40 in which a plurality of display layouts corresponding to a layout page where second image material satisfying a predetermined selection condition with respect to the selected first image material is placed around the first image material are displayed according to the number of predetermined arrangement combination patterns and which allows acceptance of selection of a display layout by the user, and a layout edition area 50 which allows acceptance of edition operation with respect to the selected display layout.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an apparatus and an album generation method that support editing of a plurality of image materials.

A technique for creating an album (electronic album) using image materials such as still images taken with a digital camera or the like is widespread. When creating an album, it is necessary to select a plurality of image materials and determine their layout. For this reason, various applications for laying out a plurality of image materials using a personal computer or the like have been developed.
For example, Patent Document 1 discloses an image forming apparatus that creates a layout image. In this image forming apparatus, an image having a minimum or maximum human face area is selected as a central image from a plurality of image materials selected by the user, and an image material excluding the central image is determined as a peripheral image. Then, the layout image is created by adjusting the central image and the peripheral image to different sizes.

Japanese Patent No. 4588627

In the image forming apparatus disclosed in Patent Document 1, it is said that a layout image in which a balance between subjects can be obtained can be obtained. However, since it is necessary for the user (operator) to select image materials to be laid out, particularly peripheral images one by one, there is a problem that the number of user operations increases when the types and sizes of image materials are large.
The main object of the present invention is to provide an image material editing support apparatus that reduces the number of user operations during editing.

  The editing support apparatus according to the present invention includes an analysis unit that extracts an image feature of a specific first image material among a plurality of image materials recorded on a predetermined recording medium, and an image feature of the extracted first image material. Acquisition means for selectively acquiring a plurality of second image materials having image features satisfying a predetermined selection condition from the recording medium, and a layout page in which the second image materials are arranged around the first image material. Editing means for creating a plurality of corresponding display layouts according to the predetermined number of combination patterns of the arrangement, and displaying the created display layouts on a display screen of a predetermined display device, through the display Control means enabling selection of any display layout and replacement of the selected display layout or change of the arrangement.

  According to the present invention, when a first image material is specified, a plurality of second image materials arranged around the first image material are acquired, and a plurality of display layouts are displayed. Since the user can select or edit any of the display layouts, the number of operations during editing can be reduced.

1 is an overall configuration diagram of an image editing support apparatus according to a first embodiment. The function block diagram of an image editing assistance apparatus. FIG. 3 is an exemplary diagram of a display screen in the first embodiment. Explanatory drawing of the process sequence of the album production | generation method which concerns on 1st Embodiment. FIG. 6 is an exemplary diagram of layout information. The illustration figure of the display screen at the time of replacing a center image. Explanatory drawing of the process sequence of the album production | generation method which concerns on 2nd Embodiment. The illustration figure of the display screen in 2nd Embodiment.

Hereinafter, an embodiment in which the present invention is applied to an editing support apparatus for image material used for album generation will be described.
[First Embodiment]
FIG. 1 is a configuration diagram of an editing support apparatus according to the first embodiment. The editing support apparatus according to the present embodiment includes a processing apparatus 12 that can access the recording medium 14. A display device 11, a mouse 13 and an internet line 15 are connected to the processing device 12. The display device 11 is a device that displays an editing screen for album generation on a display device such as a liquid crystal display. The processing device 12 is a computer that performs various processes related to album generation by reading and executing the layout program of the present invention. The mouse 13 is an example of an operation unit that accepts a user's selection operation. As the operation means, a pointing device other than the mouse 13 such as a touch sensor or a keyboard can be used.
The recording medium 14 is a storage device such as a hard disk or a semiconductor memory. In addition to the above-described layout program, image materials used for album generation, various conditions described later, and the like are recorded. The Internet line 15 is used to acquire image material on a server on the Internet. The image material acquired via the Internet line 15 can be stored in the recording medium 14 and used for layout.

FIG. 2 is a functional configuration diagram of the processing device 12. The processing device 12 executes the layout program of the present invention, thereby allowing the input reception unit 121, the image feature analysis unit 122, the image acquisition unit 123, the editing unit 124, the display control unit 125, the communication control unit 126, and the main control unit. It operates as 128. In addition, a plurality of image materials 141 for album, layout information 142, selection conditions 143, and editing rules 144 are stored in the recording medium 14.
The image material 141 is still image data or moving image data. The layout information 142 is information representing the relationship between the arrangement and size of image materials. A specific example of the layout information 142 will be described later. The selection condition 143 is a condition for automatically selecting the other image based on one image.

  For example, the selection condition 143 may be a similarity of image features within a predetermined range or a degree of image relevance. The predetermined range can be set arbitrarily. The image features include, for example, when the subject shown in the image is a person, the features of the person's face, the features of the person's wearing items, the ratio of the area in which the person is shown to the total area of the central image, etc. . In addition to the arrangement of the eyes, nose and ears, the characteristics of a person's face include skin color, hair color and its length, facial expressions (smiles, smooth faces), and the like. The characteristics of the worn items include the shape and size of the glasses worn, the color of the frame and lens, the type of clothing (coat, suit, skirt, kimono, etc.), color, and pattern. Note that the image feature can include the image shooting date and time or GPS information. A plurality of selection conditions 143 are also prepared according to the type of image feature (person, animal, building, landscape, etc.). For example, when the subject shown in the image is a person, the degree of association is, for example, whether there is a parent-child relationship, marital relationship or friend relationship with the person, whether there is a relationship between a group (multiple persons) and the individuals included in the group, When the image feature is the line-of-sight direction, the relationship is whether or not the line-of-sight directions match.

  The editing rule 144 defines the relationship between the layout page (individual pages of the album to be laid out; the same applies hereinafter) and the display layout, the relationship between the actual image material placed on the layout page and the display image, and the like. It is. The display image is an image obtained by reducing the resolution (display size) of the image material, for example, a thumbnail image. The display layout is a layout image in which the resolution of the layout page is reduced at the same ratio as the arrangement position and size of the image material on the actual layout page. When the arrangement position and size of the display image in the display layout are edited, the editing result is reflected on the actual layout page. The editing rule 144 can be changed as appropriate by learning or the like.

  The input reception unit 121 receives an operation of the mouse 13 by the user through display of the display screen of the display device 11. For example, the specification of an image material to be arranged at the center portion of the layout page is received from a plurality of image materials recorded on the recording medium 14. The center portion does not strictly mean only the center point of the layout page, but may be near the center point. In addition, a layout that stands out on the page, such as one that is laid out larger on the page, is also included. In this specification, the image material selected by the user is referred to as a “first image material” or a “center image” in order to distinguish it from the image material specified by the apparatus. The input receiving unit 121 may receive specification of a plurality of first image materials at the same time. In addition, selection of a display layout, replacement of the selected display layout, change in the arrangement of image materials, and the like are also accepted.

  The image feature analysis unit 122 functions as an analysis unit that extracts image features of a specific first image material among a plurality of image materials recorded on the recording medium 14. For example, the image of one or more of the characteristics of a person's face reflected in the first image material selected by the user, the characteristics of the person's wearing goods, and the area in which the person is reflected in the total area of the image material Extract as a feature. Extraction of image features can be performed by a known image analysis method. Note that, when a plurality of first image materials are specified, the image feature analysis unit 122 acquires a second image material for each first image material. The shooting date and time or GPS information of the first image material may be included in the image feature of the image material and extracted.

  The image acquisition unit 123 functions as an acquisition unit that selectively acquires, from the recording medium 14, a plurality of image materials having image features that satisfy a predetermined selection condition 143 with respect to the image features of the extracted first image material. “Selectively acquiring” means selecting sequentially by searching. The acquired image material is arranged around the first image material. In this specification, in order to distinguish from the “first image material”, it is referred to as a “second image material” or a “peripheral image”. For example, when the image feature of the extracted first image material is a person, the image acquisition unit 123 uses the selection condition 143 for the person and the first person in which the same person as the person shown in the first image material is shown. 2 Image material is acquired.

  The editing unit 124 functions as an editing unit that creates a plurality of display layouts corresponding to the layout page in which the second image material is arranged around the first image material in accordance with a predetermined number of combination patterns. That is, a plurality of display layouts having different arrangement positions and sizes of the first image material (center image) and the second image material (peripheral image) are created for each layout page. The editing unit 124 is configured to be able to change any of the arrangement, the arrangement position, and the size of the image materials arranged in each display layout. Alternatively, the image material arranged in each display layout can be individually replaced.

The display control unit 125 functions as a control unit that controls information display together with the input reception unit 121 and the main control unit 128. In the present embodiment, the area illustrated in FIG. 3 is displayed on the display screen of the display device 11 that is a display device visible to the user.
Referring to FIG. 3, a material display area 20, an image display area 30, a layout selection area 40, and a layout editing area 50 are displayed on the display screen.
The material display area 20 is an area that displays a display image for each of a plurality of image materials and allows the user to accept selection of a center image. The image display area 30 is an area for displaying a display image of the central image selected by the user. The layout selection area 40 is an area that displays a plurality of display layouts according to the number of combination patterns of a predetermined arrangement, and allows a user to accept a selection. The layout editing area 50 is an area that allows an editing operation to be accepted for the display layout selected by the user. Each area may be switched and displayed, but may be displayed simultaneously.

  In addition to the display control of each area, the main control unit 128 displays a plurality of display layouts, enables selection of any display layout through the display, and replacement of the selected display layout. Or it functions also as a control means which enables change of arrangement. In addition, the operation of each of the units 121 to 126 is comprehensively controlled.

[Album generation method]
Next, an album generation method by the editing support apparatus configured as described above will be described. This method is realized by the processing device 12 having access to the recording medium 14 executing the following steps.
(1) A step of receiving selection of a first image material to be used for an album among a plurality of image materials 141 recorded on the recording medium 14. The first image material is the above-described central image.
(2) A step of extracting image features of the received first image material.
(3) A step of acquiring, from the recording medium 14, a second image material having an image feature that satisfies a predetermined selection condition 143 for the image feature when the image feature of the first image material is extracted. The second image material is the peripheral image described above.
(4) A step of creating a plurality of display layouts corresponding to a layout page in which the second image material is arranged around the first image material according to the number of combination patterns of a predetermined arrangement.
(5) A plurality of created display layouts are displayed on the display screen of the display device 11 and any one of the display layouts can be selected through the display, and the selected display layout is replaced or arranged. Steps that allow changes.

  The specific contents of the above steps will be described with reference to FIG. FIG. 4 is an explanatory diagram of a processing procedure performed by the processing device 12. The layout editing process starts with a start instruction from the user. First, the processing device 12 reads an image material stored in the recording medium 14 (S101). Next, the processing device 12 displays a display image corresponding to the read image material in the image material display area 20 (first stage in FIG. 3: S102). The user inputs (selects) the center image by clicking the left button of the mouse 13 from the displayed display images. When detecting the input of the center image (S103: Yes), the processing device 12 displays the display image 31 of the center image in the center image display area 30 (second stage in FIG. 3: S104). When the user inputs (selects) the center image to be used for the layout from the image material display area 20 in order with the mouse 13, the processing device 12 repeats the processes of S103 and S104. In this example, it is assumed that three central images are input. In this case, as shown in the second row of FIG. 3, display images 31 to 33 for the three central images are displayed in the central image display area 30.

When finishing the input detection of the central image (S103: No), the processing device 12 performs image feature analysis on the central image (S105). The end of the center image input detection may be detected by an explicit instruction input from the user, or may be detected when a certain time has passed after the last center image is selected. Here, it is assumed that the selected center image is an image material on which a person is reflected, and the feature analysis of the person's face image is performed as the image feature analysis.
Next, the processing device 12 acquires a peripheral image for each central image using the result of image feature detection (S106). Specifically, the selection condition 143 for the person is selected from the recording medium 14, and a peripheral image satisfying the selection condition 143 is acquired from the image material 141 of the recording medium 14. The peripheral image satisfying the selection condition 143 is, for example, an image material 141 in which a person having a face image feature similar to a feature of the face image extracted from the center image is shown. At this time, the face size of the person shown in each peripheral image is classified into large, medium, and small, and the number and arrangement of the faces appearing are detected, and the classification / detection results are stored. .

Thereafter, the processing device 12 reads the layout information 142. As illustrated in FIG. 5, the layout information 142 is information describing the arrangement position (x, y), size (vertical H, horizontal W) of each peripheral image with respect to the central image, and arrangement conditions of the peripheral image. It is. A plurality of types of layout information are prepared for each central image.
The arrangement position (x, y) is the horizontal (x) coordinate and the vertical (y) coordinate of the image in the display layout. The vertical size H is the vertical image length in the display layout, and the horizontal width W is the horizontal image length. However, these numerical values are appropriately changed to values according to the reduction ratio of the display layout after the editing is completed.

In accordance with the description of the first layout information 1421 for the first center image, the processing device 12 sets a peripheral image having a relatively small face among the peripheral images at the arrangement position (x11, y11) and the size (vertical H11, horizontal W11). ). In addition, an image material having a relatively large face is arranged at the arrangement position (x31, y31) in the size (vertical H31, horizontal W31). Then, the central image is arranged at the arrangement position (x21, y21) with the size (vertical H2, horizontal W2), and a first display layout is created. Further, according to the description of the next layout information 1422, a peripheral image having a relatively small face is arranged at the arrangement position (x22, y22) in the size (vertical H22, horizontal W22). Further, an image material having a relatively large face is arranged at the arrangement position (x32, y32) in the size (vertical H31, horizontal W31). Then, the center image is arranged at the arrangement position (x12, y22) with the size (vertical H22, horizontal W22), and a second display layout is created. For other layout information 1423 and 142n, a display layout is created in the same procedure.
Also, a plurality of display layouts are created for other center images in the same procedure.

  After creating a plurality of display layouts for each central image in this way, the processing device 12 displays a corresponding display layout for each of the central images in the layout selection area 40 (FIG. 3: third row, S107). In the example of FIG. 3, the display layouts corresponding to the first center image are 1-A, 1-B, and 1-C. The display layouts corresponding to the second center image are 2-A, 2-B, and 2-C. The display layouts corresponding to the third central image are 3-A, 3-B, and 3-C. In FIG. 3, the arrangement position and size of the central image and the peripheral image are different because the layout information differs depending on the face detection result of the central image. This is because, for example, the arrangement position and size differ depending on the size and number of face images of the person shown in the central image.

  The user selects one of the display layouts by clicking the left button of the mouse 13 from the display layouts 1-A, 1-B, and 1-C in the layout selection area 40. The selected display layout is called a “user selected layout”. Here, it is assumed that the user selection layout 1-B is selected. When detecting the input of the user-selected layout 1-B (S108: Yes), the processing device 12 displays the display layout for the user-selected layout 1-B in the layout editing area 50 (S109). In the same procedure, the user selection layout 2-C corresponding to the second center image (display image 32) and the user selection layout 3-A corresponding to the third center image (display image 33). Are sequentially displayed in the layout editing area 50 (FIG. 3: fourth row).

  When the input of all user-selected layouts has been detected (S108: No), the processing device 12 detects the input of the user-selected layouts (S110). The end of input detection may be detected by an explicit instruction from the user, or may be detected by displaying the last display layout. When the user wants to change the arrangement (order) of the user-selected layout, the user clicks the user-selected layout to be changed in the layout editing area 50 with the left button of the mouse 13. Then, drag it as it is, move it to the position you want to rearrange, and drop it. When detecting such an operation (S110: Yes), the processing device 12 rearranges and displays the display layout (S111).

  As described above, when the user selects the central image from the display image 21 in the material display area 20, the processing device 12 automatically acquires the peripheral image arranged around the central image and displays the display corresponding to the layout page. Multiple layouts are presented. The user can edit the layout only by selecting a desired display layout from among them. This eliminates the need for the user to select peripheral images one by one, thus facilitating editing work.

In this embodiment, when selecting a center image, an example of selecting the center image by clicking the left button of the mouse 13 from the image material display area 20 has been shown. Is as follows.
The display image 31 to be deleted is clicked with the right button of the mouse 13 from the display images 31 to 33 of the central image displayed in the central image display area 30 of FIG. As a result, the display image 31 disappears from the screen. Next, the center image to be replaced is selected by clicking with the left button of the mouse 13 from the display image 21 in the image material display area 20. Then, as shown in FIG. 6, the display image 34 of the newly selected center image is placed in the center image display area 30 at the position where the deleted center image 31 was located. In conjunction with this, the display layouts 1-A, 1-B, 1-C in the layout selection area 40 correspond to the display layouts 4-A, 4-B, 4-C corresponding to the display image 34 of the center image. Replaced by C. Further, the user-selected layout 1-B in the layout editing area 50 is replaced with 4-B.
When the display layout corresponding to the display image 34 of the center image may remain as 4-B, the user can omit the operation of selecting the layout again.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. The hardware configuration and functional configuration are the same as those in the first embodiment. Therefore, parts having the same function are denoted by the same reference numerals. In the second embodiment, when the display layout is presented to the user, a display layout having the same arrangement position and size, but different only in the central image and the peripheral image is created to facilitate switching of the display layout An example will be described. Hereinafter, a description will be given centering on differences from the first embodiment.

FIG. 7 is an explanatory diagram of the processing procedure of the album generation method according to the second embodiment. FIG. 8 is an exemplary view of a display screen according to the second embodiment.
In FIG. 7, the process until a plurality of display layouts are displayed for each of the center images 31 to 33 in the layout selection area 40 (S <b> 107) is the same as that in FIG. 4 of the first embodiment. Here, the display layout corresponding to the first central image (display image) 31 by the user from the display layouts 1-A, 1-B, and 1-C in the layout selection area 40 of FIG. Suppose that 1-B is selected. The processing device 12 detects the content of the selected display layout (user selected layout) 1-B (S108), and specifies the arrangement position and size of each image material.

Next, the processing device 12 creates display layouts 1-B-1, 1-B-2, and 1-B-3 having the same arrangement position and size as the user-selected layout 1-B and only different peripheral images. Are displayed (S121). The user selects the user-selected layout 1-B-2 by clicking with the left button of the mouse 13 from the display layouts 1-B-1, 1-B-2, 1-B-3. And The processing device 12 detects the user-selected layout 1-B-2 (S122) and displays it in the layout editing area 50 (S109). This makes it possible to select display layouts that differ only in the surrounding image at the same arrangement position.
Similarly, the user selects a display layout corresponding to other central images (display images 32 and 33). Subsequent operations are the same as those in the first embodiment, and will be omitted.

  As described above, in the second embodiment, after a display layout is once selected, a plurality of display layouts that differ only in the peripheral image at the same arrangement position and size are displayed, so the user selects the display layout. Fewer processing steps.

[Other Examples]
In the first and second embodiments, the example in which the image material editing support apparatus is applied to album generation has been described, but the embodiment of the present invention is not limited to the above example. For example, it can be applied to editing support for organizing documents.
The present invention can also be realized by supplying the above layout program to a system or apparatus via a network or a storage medium, and reading and executing the program by one or more processors in the computer of the system or apparatus. It is. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Claims (12)

Analyzing means for extracting image features of a specific first image material among a plurality of image materials recorded on a predetermined recording medium;
Acquisition means for selectively acquiring, from the recording medium, a plurality of second image materials having image features that satisfy a predetermined selection condition with respect to the image features of the extracted first image material;
Editing means for creating a plurality of display layouts corresponding to a layout page in which the second image material is arranged around the first image material according to a predetermined number of combination patterns of the arrangement;
A plurality of the created display layouts are displayed on a display screen of a predetermined display device, and one of the display layouts can be selected through the display, and the selected display layout can be replaced or the arrangement can be changed. Control means that allow changes,
An image material editing support device comprising: Receiving means for receiving a plurality of identifications of the first image material;
The analysis means extracts image features of each first image material received by the reception means,
The acquisition means acquires the second image material for each first image material based on the extracted image features,
The editing means creates the layout page and a plurality of display layouts corresponding to the layout page for each first image material,
The control means causes the display screen to simultaneously display a plurality of display layouts corresponding to a plurality of layout pages.
The editing support apparatus according to claim 1. The analysis means includes a feature of a person's face reflected in the first image material, a feature of the person's wearing goods, a ratio of the area of the person reflected in the total area of the central image, Extracting one or more of the shooting date and time or GPS information as an image feature of the first image;
The editing support apparatus according to claim 1 or 2. The acquisition means acquires, as the second image material, an image material in which a person whose image characteristics are similar to a person shown in the first image material is reflected in a predetermined range.
The editing support apparatus according to claim 3. The editing means arranges the first image material as a central image at the center of the layout page, and creates the display layout in which the second image material is arranged as a peripheral image around the central image.
The editing support apparatus according to claim 1. The editing unit creates a plurality of display layouts having different arrangement positions and sizes of the central image and the peripheral image for each layout page.
The editing support apparatus according to claim 5. The editing means is configured to be able to change any one of the arrangement, the arrangement position, and the size of each of the central image and the peripheral image arranged in each of the display layouts.
The editing support apparatus according to claim 6. The editing means is configured to be able to individually replace the central image or the peripheral image arranged in each of the display layouts.
The editing support apparatus according to claim 7. After any one of the display layouts is selected, the editing unit further creates a plurality of display layouts in which the central image or the peripheral images different from each other in the same layout as the selected display layout are arranged. ,
The editing support apparatus according to claim 6. A material display area for displaying a display image of each of a plurality of image materials recorded on a predetermined recording medium and enabling a user to accept selection of a first image material;
Around the first image material selected by the user, a display layout corresponding to a layout page in which a second image material satisfying a predetermined selection condition is arranged for the first image material is determined in advance. A layout selection area that displays a plurality according to the number of combination patterns, and that allows the user to accept selection of a layout for display;
A layout editing area that allows an editing operation to be accepted for the display layout selected by the user;
Control means for controlling display of these areas for a predetermined display device;
An image material editing support apparatus. A method executed by a processing apparatus capable of accessing a predetermined recording medium,
Receiving a selection of a first image material to be used for an album among a plurality of image materials recorded on the recording medium;
Extracting image features of the received first image material;
Acquiring from the recording medium a second image material having an image feature that satisfies a predetermined selection condition for the image feature triggered by the extraction of the image feature;
Creating a plurality of display layouts corresponding to a layout page in which the second image material is arranged around the first image material according to a predetermined number of combination patterns of the arrangement;
A plurality of the created display layouts are displayed on a display screen of a predetermined display device, and one of the display layouts can be selected through the display, and the selected display layout can be replaced or the arrangement can be changed. Steps to allow changes,
Having
Album generation method.   A layout program for causing a computer to operate as the editing support apparatus according to any one of claims 1 to 10.

Images