JP2011024052A - Image processing apparatus, image processing control method, and image processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To readily generate photography instruction information to instruct a photographer to photograph an image inserted in a photo album. <P>SOLUTION: The processing apparatus acquires image data where photographic images are arranged, determines the area of them, generates a template for defining the layout area of the defined photograph images, calculates a feature value of each image of the photograph images, and generates the photography instruction information for giving an instruction on the contents to be photographed based on the feature value. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method for supporting photo album creation.

  In recent years, with the widespread use of digital cameras, it has become easy to create photo albums by operating various applications from captured images. However, in order to create a good-looking photo album, it is necessary to work by a skilled engineer or designer to determine which image should be arranged and laid out in which template.

  Patent Document 1 discloses a method for assisting layout work. That is, first, composition information of an image to be inserted into the image insertion area is given, and the editor selects the image. Then, processing information indicating the change of the enlargement ratio and the trimming position is generated so that the selected image matches the given composition information, and a method of processing the selected image according to the processing information is presented.

JP 2005-182767 A

  However, in Patent Document 1, composition information can be given in advance, and a method for creating composition information itself has not been mentioned. In addition, Patent Document 1 merely processes an image that has been shot using the composition information, and it has not been considered to use the composition information at the time of shooting. In order to solve such a problem, an object of the present invention is to generate photographing instruction information for instructing a photographer of contents such as a composition to be photographed as an image to be inserted into a photo album.

  An image processing apparatus according to the present invention includes an acquisition unit that acquires image data in which a photographic image is arranged, a determination unit that determines a region of the photographic image in the image data, and an arrangement of the determined region of the photographic image. A template generation unit that generates a template to be defined; a shooting instruction information generation unit that measures a feature amount of each of the photographic images and generates shooting instruction information that indicates contents to be shot based on the feature amount; An image processing apparatus comprising:

  It is possible to easily generate shooting instruction information for instructing a photographer to take a picture to be inserted into a photo album.

It is a figure showing the structure of an image processing apparatus. It is a flowchart showing the flow of main processes. It is a figure which shows the example of the album mount in which the image was inserted. 10 is a flowchart for creating a template and shooting instruction information from an album. It is a flowchart which calculates the feature-value of an image by analyzing a photograph part. It is a figure which shows the example of a template and imaging | photography instruction information. It is a figure which shows the example of the structured document which recorded the template and imaging | photography instruction information. It is a flowchart of the operation | movement which image | photographs using imaging | photography instruction information. It is a figure which shows the example of a display of the imaging | photography information using face information. It is a figure which shows the example of a display of the imaging | photography information using scene information. It is a figure which shows the example of a display of the imaging | photography information using color information. It is a figure which shows the example of a display of the imaging | photography information using the information of all the feature-values. It is a figure which shows the example of a display of the imaging | photography information using image information.

<Embodiment 1>
FIG. 1 is a diagram schematically illustrating an image processing apparatus in an image processing system that implements the present embodiment. The image processing system includes an image processing device and an image photographing device (photographing device). The CPU 101 controls the operation of the entire computer system and executes programs stored in the primary storage 102.

  The primary storage 102 is a main memory, and reads and stores programs stored in the secondary storage 103. The secondary storage 103 corresponds to, for example, a hard disk. Generally, the capacity of the primary storage is smaller than the capacity of the secondary storage, and programs and data that cannot be stored in the primary storage are stored in the secondary storage. Data that must be stored for a long time is also stored in the secondary storage. A computer program that implements the processing procedure in the present embodiment is stored in the secondary storage 103, read into the primary storage 102 when the program is executed, and the CPU 101 executes the execution process.

  Examples of the input device 104 include a mouse, a keyboard, a touch panel device, and buttons. These input devices 104 are used when sending an interrupt signal to a program or the like.

  Examples of the output device 105 include a liquid crystal panel, an external monitor, and a printer.

  The photographing device 106 corresponds to a camera or the like. The imaging device 106 receives an object light incident through the imaging lens, converts the imaging light into an imaging signal, and outputs the imaging signal. The imaging device 106 converts the imaging signal output from the imaging element into image data and outputs the image data. A conversion unit and a storage unit that stores image data output from the A / D conversion unit are provided. Further, an image photographed by the photographing device 106 is read directly or indirectly into the primary storage 102 or the secondary storage 103.

  The I / O device 107 corresponds to an external memory input / output device such as a memory card and an input / output unit such as a USB cable. For example, information can be acquired from the outside by connecting a scanner or the like.

  FIG. 2 shows an overall main flow in the present embodiment.

  In step S201, the CPU 101 first scans an existing album including one or more photographic images via the I / O device 107 connected to a scanner or the like, and inputs album information. The existing album here refers to an album in which a photographic image is already inserted, which is referred to at the time of shooting. This album may be an album mount in which a photograph image of a booklet actually bound as shown in FIG. 3 is inserted, or may be electronic data such as a PDF file or an image file.

  In step S <b> 202, the CPU 101 generates template data indicating the composition of the inserted image and generation of shooting instruction information from the acquired album information data. In step S203, the CPU 101 transfers the generated shooting instruction information to the imaging device (camera) 106 and displays the shooting instruction information on the display unit of the camera.

  In step S <b> 204, the user performs shooting according to the shooting instruction information displayed on the camera. Next, a detailed description will be given of the above-described reading of album information to generation of shooting instruction information with reference to the flowchart of FIG.

  In step S401, the album is read as described in step S201. In step S402, the CPU 101 determines each of the album mount area and the image area inserted into the mount area in the read album information, and divides it into two. For the determination of this area, for example, a known area recognition / area division technique disclosed in Japanese Patent Laid-Open No. 09-223240 may be used.

  In step S403, the CPU 101 determines a blank portion of the album mount and a portion determined to be a character or graphics portion as template data, and generates a template. A portion determined to be an insertion image is used as photographic image data. The template here is for defining the arrangement of the determined region of one or more photographic images.

  Here, the data portion of the photographic image in the template data is called a placeholder (hereinafter referred to as “PH”). This PH is a portion into which an image is inserted when creating an album, and is shown, for example, as shown in FIG.

  Various other information is included in the template data, but since it is not the subject of the present invention, description thereof is omitted here.

  In step S404, the CPU 101 analyzes photographic image data in the read album information. The analysis of the photographic image data will be described in detail with reference to the flowchart of FIG.

  In step S501, the CPU 101 first determines whether a human face has been detected in the photographic image data. If a face is detected, the process proceeds to step S502, and if no face is detected, the process proceeds to step S504.

  In step S502, the CPU 101 counts the number of human faces, and measures the position, size, orientation, facial expression, and the like using known techniques. In step S503, the CPU 101 determines whether all faces have been measured. If an unmeasured face still remains, the process proceeds to step S502, and an unmeasured photographic image is measured. If all faces have been measured, the process proceeds to step S504.

  In step S504, the CPU 101 detects and determines a scene of each image for determining an image scene. For example, the image scene is determined as an image scene corresponding to a portrait scene, a landscape / scene, a night view / scene, a group photo / scene, and the like, and whether the image scene is a vertical position or a horizontal position is detected.

  The method of determining the image scene is that when the number of faces in the image is greater than a certain value, the group photo is taken. When the face size is greater than a certain size, the portrait and the average brightness of the image are lower than the threshold. If it is a night view, it is determined as a night view, otherwise it is a landscape.

  In step S <b> 505, the CPU 101 measures the color of the image and detects the pixel arrangement of the color. Here, a representative color in the image is measured, and a color having a high appearance frequency equal to or higher than a predetermined threshold is used. In addition, for a color having a high appearance frequency that is measured in a photographic image, color pixel arrangement acquisition is performed to determine which pixel in the image the color is arranged.

  As described above, according to the flowchart of FIG. 5, the CPU 101 analyzes the data portions of all the photographic images. The analysis results of all the photographs are collected as shown in FIG. 6B, for example, and stored in the primary storage 102 or the secondary storage 103.

  In step S <b> 405, the CPU 101 generates shooting instruction information for assisting shooting when the user takes a picture with the shooting apparatus from the template data and the analysis results of all the photos, and stores the shooting instruction information in the primary storage 102 or the secondary storage. Store in the memory 103. Here, the shooting instruction information is information for instructing what kind of photograph should be taken at the PH in the template data.

  The template data and the shooting instruction information are stored in the memory of the primary storage 102 or the secondary storage 103 as shown in FIG. Here, an area 701 in FIG. 7 represents template data, and an area 702 represents PH shooting instruction information.

  It may be recorded as a structured document such as XML or HTML, or may be added as metadata to the image data of each page. The shooting instruction information according to the present embodiment is an instruction for which PH of the template what kind of subject is to be shot.

  Finally, in S406, the CPU 101 determines whether all album pages have been read. If not all pages have been read, the process returns to step S401, and the process ends when reading is completed.

  The method for creating the template data and the shooting instruction information has been described above.

  Next, an operation of photographing using the created photographing instruction information in the photographing apparatus will be described with reference to the flowchart of FIG. The imaging device here refers to the imaging device 106 directly connected to the image processing apparatus. However, this imaging apparatus does not need to be directly connected to the image processing apparatus shown in FIG. 1, and a camera (not shown) connected indirectly via a network via the I / O device 107, and imaging instruction information A camera (not shown) equipped with a transferred memory card may be used. Here, a case where shooting instruction information is transmitted to the shooting device 106 will be described.

  In step S <b> 801, the photographing device 106 receives a template and photographing instruction information by an internal receiving unit, and loads the template and photographing instruction information into a memory (not shown) in the photographing device 106. In step S802, the control unit (not shown) of the imaging device 106 decodes the first PH shooting instruction information and displays it on the display unit (not shown) of the imaging device. Here, the shooting instruction information includes information regarding a face, information regarding a scene, information regarding a color, and the like shown in FIG. 6B.

  These display methods will be described with reference to FIG. FIG. 9 shows the back side of a digital camera which is the photographing device 106. The camera operation unit 901 performs switching of shutter speed and mode, display switching, and the like, and the shooting through image 902 displays an image being shot. The shooting instruction information display unit 903 displays shooting instruction information regarding an image to be shot to the user. The album layout display unit 904 displays the entire album mount and the inserted image.

  FIG. 9 shows shooting instruction information using face information. Here, regarding the image to be put in PH1, shooting instruction information regarding the face is displayed. A photographed through image 902 is also displayed, and a photographer who is a user photographs while comparing both images.

  FIG. 10 displays shooting instruction information using scene information. Here, scene information is displayed regarding the image to be placed in PH1. Here, a photographed through image 902 is also displayed, and the photographer photographs while comparing the two.

  FIG. 11 shows photographing instruction information using color information. Here, the arrangement of the colors of the image to be put in PH1 is displayed. A photographed through image 902 is also displayed, and the photographer shoots while comparing the two. As described above, the shooting instruction information is displayed in three parts, but all the shooting instruction information may be displayed together.

  FIG. 12 shows shooting instruction information using all information of face, scene, and color. The photographer shoots while comparing all the information with the shot through image. In addition to the feature amount, an image may be displayed. FIG. 13 displays shooting instruction information for each of the images contained in the album originally scanned with PH1. The photographer compares this PH shooting instruction information with the shooting through image 902 to be shot.

  In step S803, the photographer performs photographing while referring to the photographing instruction information. In step S <b> 804, after shooting, the CPU 101 analyzes the image shot by the shooting device 106 and calculates the feature amount of the image. The feature amount here is face information, scene information, and color information as shown in FIG.

  In step S <b> 805, the CPU 101 compares the captured image with the feature amount of the PH shooting instruction information. Here, the photographed image is associated with the PH that has the highest degree of matching with the photographing instruction information in the PH and has not yet been photographed.

  In step S806, the CPU 101 determines whether a captured image has been inserted into all PHs. Here, if all of the photographing has not been completed and there are those that have not yet been inserted into the PH, the process returns to step S802, and if all the PHs have been inserted, the process proceeds to step S807.

  In step S <b> 807, the CPU 101 displays a message that all the images in the album are filled up on the display unit of the shooting device, for example, the shooting instruction information display unit 903, and ends shooting. As described above, in this embodiment, by acquiring information on an existing album, shooting instruction information and a template can be created, and shooting suitable for the album can be performed using the created shooting instruction information.

  In the present embodiment, the image shooting is described as still image shooting. However, it goes without saying that the present invention can be implemented not only for still images but also for moving image shooting. In this way, in the present embodiment, by acquiring the information of the existing album, the shooting instruction information and the template are created, and by using the created shooting instruction information, the photographer is based on the album without error. Shooting can be performed.

<Other examples>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (7)

Acquisition means for acquiring image data in which photographic images are arranged;
Determining means for determining a region of the photographic image in the image data;
Template generating means for generating a template that defines the layout of the determined region of the photographic image;
An image processing apparatus comprising: photographing instruction information generating means for measuring a feature amount of each of the photographic images and generating photographing instruction information for instructing contents to be photographed based on the feature amount.   The image processing apparatus according to claim 1, wherein the template and the imaging instruction information are transmitted to an imaging apparatus. The photographing instruction information generating means includes
Counting means for counting the number of human faces for each of the photographic images;
Image scene determining means for determining an image scene of the photographic image;
2. The image processing according to claim 1, further comprising at least one color pixel arrangement acquisition unit configured to acquire information relating to an arrangement of pixels of the color with respect to a color having an appearance frequency equal to or higher than a predetermined threshold in the photographic image. apparatus.   The image scene determining means determines an image scene corresponding to a region of the photographic image based on an image scene including at least one of a portrait scene, a landscape / scene, a night view / scene, and a group photo / scene. The image processing apparatus according to claim 3. An image processing system comprising an image processing device and a photographing device,
The image processing apparatus includes:
Acquisition means for acquiring image data in which photographic images are arranged;
Determining means for determining a region of the photographic image in the image data;
Template generating means for generating a template that defines the layout of the determined region of the photographic image;
A shooting instruction information generating unit that measures the feature amount of each image of the photographic image and generates shooting instruction information for instructing the content to be shot based on the feature amount, the template, and the shooting instruction information as the shooting device Transmission means for transmitting to
The imaging device
Receiving means for receiving the transmitted template and the shooting instruction information;
An image processing system comprising: the template and display means for displaying the photographing instruction information. An acquisition step of acquiring image data in which photographic images are arranged;
A determination step of determining a region of the photographic image in the image data;
A template generation step of generating a template that defines the layout of the determined region of the photographic image;
An image processing apparatus comprising: a shooting instruction information generation step of measuring a feature amount of each image of the photographic image and generating shooting instruction information for instructing contents to be shot based on the feature amount. Control method.   The computer program for making a computer perform each process in the control method of the image processing apparatus of Claim 6.