JP4991614B2 - Layout determining method, program and apparatus - Google Patents

Description

  The present invention relates to determination of a layout for arranging images.

In Patent Document 1, a plurality of images in an album are laid out in a plurality of different layouts, and a plurality of album information is provided to the album orderer. In Patent Document 2, in the operation of the touch panel, a plurality of predetermined pattern images are arranged at predetermined intervals on a straight line or a curve having the end point of the position where the designation input is started and the end position.
JP 2007-312058 A JP 2003-141558 A

  The method of determining the layout by allowing the user to select the layout prepared in advance is lacking in originality. When seeking originality in the layout of an image, it is optimal to let the user decide the layout freely. However, the free layout of images is complicated because the user needs to set various parameters such as the arrangement position and size of each image from scratch.

  In Patent Document 1, the parameters for layout creation are only images and system-fixed information, and cannot reflect the user's layout preference and generally popular layout. In Patent Document 2, the position of the image can be freely specified, but other parameters, for example, the size and shape of the image cannot be freely specified, and these need to be specified separately, and the user creates an original image layout. It's too complicated.

  An object of the present invention is to easily provide an image layout with high originality.

  The layout determination method according to the present invention includes a step of storing a basic layout, a step of creating at least one layout candidate different from the basic layout by changing a parameter defining the basic layout, and a layout candidate created A step of outputting; a step of accepting a layout candidate selection instruction; and a step of determining a final layout from the layout candidates based on the selection instruction.

  Preferably, the basic layout includes a layout used by a specific user in the past or a layout used by a user other than the specific user in the past.

  Preferably, the method includes a step of repeatedly scrolling and displaying each of the created layout candidates.

  Preferably, the method includes a step of inputting image data obtained by imaging the subject, a step of analyzing the facial expression of the subject based on the image data, and a step of determining the presence or absence of a selection instruction based on the analyzed facial expression of the subject. .

  Preferably, the parameter includes at least one of the number of arranged images, the arranged position of the image, the size of the image, and the shape of the image.

  A program for causing a computer to execute the above layout determination method is also included in the present invention.

  The layout determination apparatus according to the present invention includes means for storing a basic layout, means for creating at least one layout candidate different from the basic layout by changing a parameter that defines the basic layout, and a layout candidate that has been created. Means for outputting, means for receiving an instruction for selecting a layout candidate, and means for determining a final layout from the layout candidates based on the selection instruction.

  According to the present invention, a layout candidate is created by changing parameters of an existing basic layout, and a desired layout is selected from the layout candidates, so that a highly original layout can be easily used by a user.

  FIG. 1 is a block diagram showing a schematic configuration of a photo album creating apparatus 10 to which the present invention is applied. The photo album creating apparatus 10 reads, for example, moving image data and still image data (including still images obtained by capturing moving images) stored in various recording media 40 such as a CD and a DVD, a memory card 42, and the like. A photo album (data for displaying an image on a display device with a predetermined layout and design or a print medium such as paper on which an image is printed with a predetermined layout and design) is created. The photo album creating apparatus 10 includes an operation unit 26 including a pointing device capable of instructing movement of a pointer (cursor) such as a mouse, a touch pad, and a drag ball, and various operation devices such as a keyboard and a touch panel (or connected from the outside via the Internet). It can be operated with the keyboard of the terminal that has been used.

  The photo album creating apparatus 10 includes a CPU 12 that controls the entire photo album creating apparatus 10, a drive 11 that reads and writes data from and to a recording medium 40, a card slot 16 that reads and writes data from and to a memory card 42, and a photo album creating apparatus 10. A hard disk drive (storage medium) 18 (hereinafter referred to as HDD 18) in which programs to be controlled, moving images and still images are stored, a RAM 20 that functions as a working memory when the CPU 12 performs various processes, a RAM 20 and the like A list of stored images and other graphical user interfaces related to image editing, a video signal of a pointer moving on the window according to an input operation including an instruction of a moving direction and a moving speed from the operation unit 26 is displayed on the display 50 (or the Internet The video encoder 22 can be output to and displayed on a display connected to an externally connected terminal or the like (the same applies hereinafter), and communication for connecting to the Internet and other networks and various electronic devices such as the digital camera 15. It comprises an I / F 28 and a data recording unit (operation history recording means) 30 each responsible for specialized processing. These units are connected to each other via a bus 36, and the CPU 12 controls the respective units of the apparatus according to an operation program stored in the HDD 18.

  The photo album creating apparatus 10 is connected with a printer 44 for printing the designated still image on a mount made of a paper medium such as plain paper or photographic paper in a predetermined arrangement via the printer driver 38. Yes. Alternatively, still images arranged in a predetermined arrangement can be output to the display 50, the recording medium 40, a personal computer connected via the Internet, or the like as an electronic medium such as “PDF” of Adobe Systems, Inc. In the document, an electronic medium creation form on which a desired image can be arranged at a desired position is also called a mount.

  The card slot 16 detachably holds, for example, a memory card 42 such as a compact flash (registered trademark), an SD card, and smart media, and reads or writes still image data recorded on the memory card 42. Thereby, still image data captured by a digital still camera or the like can be read. In addition to the multi-drive 11 and the card slot 16, for example, a communication port such as USB or IEEE1394 may be provided so that direct communication with an electronic device such as a digital still camera or a PDA can be performed.

  The HDD 18 stores, in addition to one or a plurality of still image data serving as material for the photo album, decoration image data combined with these still image data when the photo album is generated. The decorative image is a mask image or an illustration that covers an unnecessary portion of the target image. These decoration images add color to the photo album, such as decorating the background of still images as materials and adding one-point decorations.

  Each still image data is stored in the HDD 18 in the form of an image file such as JPEG, TIFF, or EXIF standardized for a digital still camera. These image files are provided with a data area for storing still image data and a tag area for storing metadata (typically EXIF tag information) indicating the contents of the data. Hereinafter, still image data is simply referred to as “image”.

  The moving image data corresponds to a compression recording standard such as motion JPEG or MPEG4.

  The communication I / F 28 connects the photo album creating apparatus 10 to the Internet, displays a home page on the Internet on the display 50, and transmits and receives image data via an electronic mail (hereinafter referred to as mail). The communication I / F 28 is a broadband compatible modem or the like. The communication I / F 28 may be connected to the Internet via an optical communication network or a cable network. The photo album generated by the photo album generation unit 34 can be distributed to a specific terminal via the Internet. The distribution-side terminal can output the photo album to a monitor or printer and use it freely. The communication I / F 28 can be used not only as a network adapter but also as a general-purpose interface for connecting peripheral devices such as USB.

  The data recording unit 30 and the photo album generation unit 34 are so-called coprocessors, and each assists the processing operation of the CPU 12 by taking charge of specialized processing. The data recording unit 30 records history information of operation input to the operation unit 26 such as a staying area, staying position coordinates, staying time of a pointer, time and number of input operations, and the like.

  When the photo album creation apparatus 10 is a storefront print terminal, the photo album created by the photo album generation unit 34 is output to the display 50 and displayed as a preview. If the content is confirmed, the printer 44 is set according to a user instruction. Can be output to a print medium, or the photo album such as a DVD can be transferred to a recording medium 40 that can be recorded as video data. Further, when the photo album creating apparatus 10 accepts a photo album order from a terminal connected from the outside via the Internet, the photo album preview is converted into a Web page that can be displayed on the terminal instead of the display 50 and the like. You may transmit to a terminal. Alternatively, the completed photo album may be displayed as one of the photo album display functions of various electronic devices such as a mobile phone and a digital camera without being premised on outputting to a recording medium or a print medium.

  The system configuration in FIG. 1 is an example, and the present invention can be implemented with other electronic devices (for example, mobile phones and PDAs) having the same configuration. Further, the individual blocks shown in the figure are not necessarily configured integrally with one electronic device, and each block is a part of the configuration of another electronic device, and they are connected by some communication means such as a network. If information transmission / reception between the electronic devices is ensured, the present invention can be implemented.

  The flow of the photo album creation process will be described below with reference to the flowchart of FIG. This process is executed by the photo album generation unit 34 (or CPU 12). A program that causes the photo album generation unit 34 (or the CPU 12) to execute this processing is recorded in a computer-readable recording medium such as the HDD 18 or the RAM 20. Such a program may be imported from the outside via the Internet.

  In S <b> 1, images to be arranged in the photo album are read from the recording medium 40, the memory card 42, or a terminal connected to the communication I / F 28 from the outside via the Internet and stored in the HDD 18. The image includes a captured image of a moving image file in addition to a still image file such as JPEG.

  In S2, selection of an image to be arranged in the album among the read images is accepted. Hereinafter, unless otherwise specified, the image selected in this step is simply referred to as an image.

  In S3, by changing the parameters of the past layout stored in the layout table, one or a plurality of layout candidates (arrangement position, shape, size, background color and other images of one or a plurality of images on the mount are arranged on the mount. Data for defining various parameters necessary to create a photo album. This process will be described later.

  In S4, the plurality of created layout candidates are sequentially output and displayed on the display 50. This process will be described later.

  In S5, a desired layout is selected from the sequentially displayed layout candidates.

  In S6, a photo album in which images are arranged in the selected layout is created and output to the mount.

  Next, details of the process of S3 will be described.

  A user who is currently selecting a layout or another user who has been determined as a final layout from layout candidates in the past or a freely created layout, except for a substantially overlapping layout, is stored in the HDD 18. A layout obtained by modifying a part of the layout (basic layout) stored in the layout table is displayed as a layout candidate. If the user wants to see the finished image, the image may actually be arranged in the layout candidate. In order to distinguish between a user who wants to select a desired layout from the layout candidates and a user who has selected a layout in the past, the user is requested to input user identification information and a password as a set at the start of the album creation process, If the stored user identification information and password pair match, the album creation process is started, and after that, the user identification information input is compared with the identification information of the user who has selected the layout in the past. If not, it is good to judge that the past layout was used by someone else. If you change what others have used in the past, you can easily create originality based on the layout of others. If you change the layout you used in the past, your originality will increase further.

  For example, as shown in FIG. 3A, if a basic layout (layout A) in which three images are arranged in a small size along a diagonal line from the upper right to the lower left is stored in the layout table, Two layout candidates (layout B and layout C) are generated by changing the size of each image without changing the arrangement position of the images. In the layout B shown in FIG. 3B, three images are arranged in small, large, and small sizes along diagonal lines from the upper right to the lower left. In the layout C shown in FIG. 3C, three images are arranged in a large, small, and large size along a diagonal line from the upper right to the lower left.

  Next, the detail of S4 is shown.

  For example, as shown in FIG. 5, the image arranged according to the layout candidate is scrolled from the upper part of the screen to the lower part (or vice versa) like a roulette in the candidate display area R1 provided in the display 50. By doing so, an animation in which images arranged according to the respective layout candidates appear in the candidate display area R sequentially is displayed. The animation display is controlled by the CPU 12. The display order is the layout candidate creation order. Instead of scrolling layout candidates, a predetermined number of layouts that have been used in the past may be scroll-displayed in a predetermined number (for example, 10).

  FIG. 5A shows a state where an image arranged according to the layout B is displayed, and FIG. 5B shows a state where an image arranged according to the layout C is displayed. Although illustration is omitted, if other layout candidates are created, images arranged in accordance with the layout candidates are also sequentially displayed in a roulette (slot machine) shape. When the display of the last layout candidate (layout C in FIG. 5) ends, the display returns to the display of the first layout candidate (layout B in FIG. 5), and thereafter, until the “OK” button is selected. The roulette-like display is repeated.

  When the user displays a desired layout candidate, when the touch sensor included in the operation unit 26 detects a touch of the display position of the “decision” button displayed near the candidate display area R1 of the display 50, The layout candidate displayed at the time of detection is determined as the final layout. The parameters of the final layout are stored in the layout table together with the layout identification information, and are used to create subsequent layout candidates. The correspondence between the number of images for each layout, the arrangement position of each image, the arrangement size, the shape of the image, and other parameters is stored in the HDD 18 as a layout table.

  FIG. 4 shows that layout A, layout B, and layout C were determined as final layouts in the past in the layout table. In addition, the utilization factor of each layout may be associated.

  Alternatively, layout candidates can be created according to user operations, and layout candidates can be created and selected simultaneously.

  For example, as shown in FIG. 6, when an image is picked up by a simulated cue object C displayed on the display 50 in a simulated billiard table R2 provided on the display 50, the operation speed and operation of the cue object C are displayed. The projected image moves like a billiard ball depending on the direction. At this time, if the struck image collides with another image, the collided image is determined according to an algorithm that defines the relationship between the incident velocity / angle and the bounce velocity / angle of objects (in accordance with rigid body dynamics law). Move. The hitting speed and hitting direction of the cue object C can be input from the operation unit 26, and the CPU 12 performs display control of an animation of collision rebound of the image according to the input. The user repeatedly hits the image with the queue object C, and determines the arrangement of each image. When the image moves to a desired position, the “determine” button near the table R2 is pressed to determine the position as the final layout of the image. The determined final layout is stored in the layout table. Note that the past layout of the layout table may be used as the default arrangement position of the image. For example, the number of times of use is accumulated for each other's layout or one's own layout, and the other person's or one's layout most selected in the past is set as the default arrangement position.

  Alternatively, a pseudo ball object and a block that move autonomously in the table R2 are displayed like a block break, and a pseudo racket object is applied to the ball object to change the moving direction of the ball object. When the ball object hits a block, the block disappears from the table R2. The user operates the operation unit 26 to move the racket object, repeatedly changing the moving direction of the ball object, and deletes only a desired block. If the desired block remains, the “determine” button is pressed there to determine the final layout with the position of the remaining block as the image placement position. The determined final layout is stored in the layout table.

  Alternatively, by operating the touch sensor included in the operation unit 26, the position and size of the image can be changed, and if a desired arrangement can be realized, the final arrangement position can be determined there.

  For example, as shown in FIG. 7, the images I1 to I5 are arranged and displayed at a default position on the display 50 stacked with the touch sensor (possible to be arranged according to the past layout), and the images I1 to I5 are displayed. Is traced from the display 50 and moved. At this time, the image is gradually enlarged and displayed as it moves to the central portion of the display 50, and conversely, the image is gradually reduced and displayed as it moves away from the central portion. In FIG. 7, since the image I5 is in the central portion, the image I5 is relatively enlarged as compared with the other images I1 to I4. Thereby, the position and size of the image can be arbitrarily changed. When the image is moved to a desired position, a “determine” button is pressed there to determine the position and size of the image at that time as the final layout. The determined final layout is stored in the layout table. In other words, the layout stored in the layout table is not necessarily limited to the layout candidates created in the processes of S1 to S5, and any layout that has been used in the past may be used.

  Alternatively, as shown in FIG. 8, a simulated mount object M is arranged on the display 50, and the mount is traced from the display 50 at a default position (possible according to a past layout). An image is placed on or removed from the object M, an arbitrary image is placed at an arbitrary position on the mount object M, and the arrangement of the images on the actual mount is determined. If the image is moved to a desired position, the “determine” button is pressed there to determine the position of the image at that time as the final layout. The determined final layout is stored in the layout table.

  Alternatively, as shown in FIG. 9, the arrangement and the size of the image are determined in accordance with the direction of tracing the display 50, the speed, the starting point where the tracing starts, and the finishing point where the tracing ends. For example, as shown in FIG. 9A, if the display 50 is traced in a straight line from the lower left to the upper right, the images are arranged along the straight line. Here, as shown in FIG. 9B, if the user traces quickly, so many images are arranged in a small size. On the contrary, as shown in FIG. 9C, if slowly tracing, fewer images are arranged in a larger size. If an image is arranged in a desired size at a desired position, the “determine” button is pressed there to determine the position and size of the image at that time as the final layout of the image. The determined final layout is stored in the layout table. If you do not like the layout and size, you can press the “Redo” button (not shown) to clear the position and size of the image so far and specify them from the beginning.

  However, the final layout is not necessarily determined by the input of the operation unit 26. For example, as shown in FIG. 10, a digital camera 15 is installed on the top of a case terminal-type photo album creation apparatus 10, the digital camera 15 is connected to a communication I / F 28 of the photo album creation apparatus 10, and digital Images obtained by the camera 15 taken periodically are sent to the photo album creating apparatus 10. The image from the digital camera 15 is distinguished from the image arranged in the album, but of course, it may be selected in S2 as the image arranged in the album.

  FIG. 11 is a flowchart for determining the presence / absence of a layout candidate selection instruction according to facial expression determination. A program for causing the CPU 12 to execute this processing is recorded on the HDD 18 or other computer-readable recording medium.

  As shown in FIG. 11, the CPU 12 analyzes the facial expression of the person (typically, the person who ordered the photo album) in front of the housing from the received image (S11). As a result of facial expression analysis, when it is determined that the person has a favorite facial expression such as a smile (S12), at that time, the above-mentioned layout candidate displayed in the roulette, image position determined by billiard block breaking, touch panel operation The parameters such as the image arrangement position, size, shape, etc., determined in (1) are determined as final layout parameters (S13).

  As a result of the facial expression analysis, when it is determined that the person has an unpleasant facial expression such as an awkward face or a troubled face (S14), the display of the displayed layout candidates is finished (S15). Then, if it is the above-mentioned roulette display, the next layout candidate is displayed. Alternatively, the billiard block is broken, the parameters such as the image position, size and shape determined by the touch panel operation are cleared, the image is returned to the default position, and the final layout determination operation is performed again from the beginning.

  A known expression can be used to detect an expression. For example, the expression is analyzed and determined by matching with generalized expression data as described in JP-A-2007-299297.

  In this way, layout candidates are created by changing the parameters of the existing basic layout, and by allowing the user to select a desired layout candidate from among them, a highly original layout can be easily used by the user. .

Second Embodiment
In the first embodiment, based on a basic layout (for example, layout A), only specific parameters among parameters (number of sheets, shape, size, arrangement, background color, etc.) that determine the layout based on the operation of the operation unit 26 are used. It is possible to change and create layout candidates.

  For example, as shown in FIG. 12, when a cross button (or various operation members capable of inputting instructions in multiple directions such as a touch pad of a circular wheel and up / down / left / right cursor keys) is provided on the operation unit 26 and the cross button is pressed, The layout candidates increased in number based on the layout A, that is, the images are arranged along the diagonal line from the upper right to the lower left, and the size, number, and shape of each image are maintained, but the size of each image is reduced and arranged instead. Create and display a larger number of images.

  When the right side of the cross button is pressed, the size of each image is changed evenly or only the size of a specific image is changed while maintaining the current arrangement of images, the number of images, and the shape. If you keep pressing right, the size of the image will increase, and if it exceeds the allowable range of the mount, it will return to the size of the basic layout.

  If the left side of the cross button is pressed, the arrangement, size and number of each image are maintained, but the shape of each image is changed. For example, if the left button is kept pressed, the shape of each image is changed in the order of circle, ellipse, diamond, square, and rectangle.

  If the bottom of the cross button is pressed, the size, number, and shape of each image are maintained, but the arrangement is changed. For this, several arrangement patterns are prepared (for example, images gathered in the center, images dispersed in the four corners), and the arrangement patterns are sequentially switched by continuously pressing down. If an image is arranged in a desired size and shape at a desired position, the “position” button is pressed there, or the position and size of the image at that time according to the facial expression analyzed from the image from the camera 15 And determine the final layout.

  The parameter may be changed within a range specified by the user. For example, if the user arranges images on only one page in a range of up to five, layout candidates for arranging six or more images are not created.

  In this way, layout candidates can be created by changing only the parameters that the user wants to change, so that layout candidates that meet the user's preference can be quickly created.

Configuration diagram of photo album creation device Photo album creation process flowchart The figure which shows an example of the past layout (layout A), a layout candidate (layout B), and a layout candidate (layout C) Diagram showing an example of a layout table The figure which shows an example of the layout candidate displayed on a roulette style The figure which shows an example of the layout candidate displayed on billiard style The figure which shows the 1st example of the layout candidate produced by operation to a touch sensor The figure which shows the 2nd example of the layout candidate produced by operation to a touch sensor The figure which shows the 3rd example of the layout candidate produced by operation to a touch sensor The figure which shows the color scheme and the decoration which emphasizes an image without a tint) Diagram showing an example of camera installation Flowchart for determining whether or not there is a layout selection instruction according to an expression The figure which illustrated the parameter of the layout changed corresponding to the pushing direction of a cross button

Explanation of symbols

10: Photo album creation device, 18: HDD, 34: Photo album generation unit

Claims (8)

Means for storing the basic layout of the image ;
Means for creating at least one layout candidate different from the basic layout by varying parameters relating to the position and size of the image defining the basic layout according to a touch on a touch sensor ;
Means for outputting the created layout candidate;
Means for accepting a layout candidate selection instruction;
Means for determining a final layout from the layout candidates based on the selection instruction;
Only including,
The layout determination apparatus , wherein the parameter related to the position of the image varies according to a touch position on the touch sensor, and the parameter related to the size of the image varies according to at least one of a moving direction and a moving amount of the touch position on the touch sensor .   Means for storing the basic layout of the image;
  Means for creating at least one layout candidate different from the basic layout by varying parameters relating to the number, position, and size of the image defining the basic layout according to a touch on a touch sensor;
  Means for outputting the created layout candidate;
  Means for accepting a layout candidate selection instruction;
  Means for determining a final layout from the layout candidates based on the selection instruction;
  Including
  The parameter related to the position of the image varies according to the touch position on the touch sensor, and the parameter related to the number and size of the image varies according to at least one of the moving direction, moving amount, and moving speed of the touch position on the touch sensor. Layout determination device. The basic layout is a layout determination device according to claim 1 or 2 including a layout that is specific users other than the layout or the specific user who has used in the past users have used in the past. The layout determination apparatus according to any one of claims 1 to 3, further comprising means for repeatedly scrolling and displaying each of the created layout candidates. Means for inputting image data obtained by imaging a subject;
Means for analyzing the facial expression of the subject based on the image data;
Means for determining the presence or absence of the selection instruction based on the analyzed facial expression of the subject;
Layout determination apparatus according to any one of claims 1 to 4 containing. A program that causes a computer to function as the layout determination device according to claim 1. A step of layout determining device stores the basic layout of the image,
A step wherein the layout determining device parameters about the position and size of the image that defines the basic layout, by varying in accordance with touch to the touch sensor, to create at least one different layout candidates and the basic layout,
And outputting a layout candidates said layout determination device described above created,
A step of the layout determining unit accepts the selection instruction of layout candidates,
A step of the layout determining device determines a final layout from the layout candidates based on the selection instruction,
Only including,
The layout determination method in which the parameter related to the position of the image varies according to a touch position on the touch sensor, and the parameter related to the size of the image varies according to at least one of a moving direction and a moving amount of the touch position on the touch sensor . A step of layout determining device stores the basic layout of the image,
A step wherein the layout determining device the number of images to define the basic layout, the position and the parameters relating to the size, by varying in accordance with touch to the touch sensor, to create at least one different layout candidates and the basic layout,
And outputting a layout candidates said layout determination device described above created,
A step of the layout determining unit accepts the selection instruction of layout candidates,
A step of the layout determining device determines a final layout from the layout candidates based on the selection instruction,
Only including,
The parameter related to the position of the image varies according to the touch position on the touch sensor, and the parameter related to the number and size of the image varies according to at least one of the moving direction, moving amount, and moving speed of the touch position on the touch sensor. layout determining how to.

Images