JP2011070276A - Image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve efficiency of identifying the relationship between people from images. <P>SOLUTION: The image processing apparatus processing a plurality of target images arranged in time series includes: a face-area detection unit for detecting a face area, i.e., an image area containing a face image, from each of the target images, for each face image; a subject distance estimation unit for estimating a subject distance, i.e., the distance between the face of a subject and an imaging device, by using the face area; an inter-subject distance estimation unit for estimating an inter-subject distance, i.e., the actual distance between faces in the face area, by using the subject distance; and an affinity calculation unit for calculating an affinity that indicates affinity between people, using the estimated inter-subject distance for people whose inter-subject distance estimated from a predetermined number of consecutive target images are equal to or less than a predetermined value, so that the smaller the inter-subject distance is, the greater the value of the affinity becomes. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus.

  2. Description of the Related Art Conventionally, a technique for identifying the relationship between people using an image showing a plurality of people is known. For example, a technique is known in which a closeness representing the degree of intimacy between two people is calculated using the distance between the people represented in the image (Patent Document 1). Also, there are known image classification methods and image search methods that use relationships between people specified from images.

JP 2006-79457 A

  However, there is still room for improvement in the efficiency of processing for identifying the relationship between people from the image.

  The present invention has been made in order to solve at least a part of the above-described conventional problems, and an object thereof is to improve the efficiency of processing for specifying the relationship between people from an image.

  In order to solve at least a part of the above problems, the present invention employs the following aspects.

  The first aspect provides an image processing apparatus that processes a plurality of target images arranged in time series. The image processing apparatus according to the first aspect of the present invention uses a face area detection unit that detects, for each face image, a face area that is an image area including a face image from each of the target images, and the face area. A subject distance estimation unit that estimates a subject distance that is a distance from the subject's face to the photographing apparatus, and a subject that is an actual distance between human faces represented in the face area using the subject distance An inter-subject distance estimation unit that estimates an inter-distance, and people who have all the inter-subject distances estimated from a predetermined number of consecutive target images that are less than or equal to a predetermined value, using the estimated inter-subject distance A closeness calculating unit that calculates a closeness representing a degree of intimacy between people so that the closeness between the subjects becomes smaller as the distance between the subjects becomes smaller.

  According to the image processing apparatus according to the first aspect, since the intimacy is calculated for people whose subject-to-subject distances are all equal to or less than a predetermined value in a predetermined number of consecutive target images, It is possible to improve the efficiency of processing for specifying the relationship.

  In the image processing device according to the first aspect, the closeness calculation unit is configured to perform the following operation on the target image that is next to the target image on which the inter-subject distance is estimated. Even if the distance between subjects is not estimated, if the distance between subjects is estimated in a target image that is next to the target image for which the distance between subjects is not estimated, the target whose distance between subjects is estimated If the total number of images is a predetermined number and all the estimated distances between the subjects are less than or equal to a predetermined value, the closeness between the persons may be calculated. In this case, in order to calculate intimacy for people whose estimated inter-subject distances are all equal to or less than a predetermined value in a predetermined number of target images that are not continuous, the process of specifying the relationship between people from the target image Efficiency can be improved.

  The image processing apparatus according to the first aspect further includes a feature discriminating unit that discriminates human characteristics from each person's face image whose intimacy is calculated by the intimacy calculating unit, and the intimacy and the person And a relationship determination unit that determines a relationship between people whose intimacy has been calculated. In this case, in order to determine the relationship between the persons whose intimacy is calculated using the intimacy and the characteristics of the person, it is possible to improve the efficiency of the process of identifying the relationship between the persons from the target image. it can.

  In the image processing apparatus according to the first aspect, the feature determination unit further determines, for each person whose intimacy is calculated, whether or not the person has already determined the characteristics of the person, and the characteristics of the person The determination may be performed only for a person who has not been determined. In this case, once the person whose characteristics have been determined is not determined again, it is possible to improve the efficiency of the process of specifying the relationship between persons from the target image.

  In the image processing apparatus according to the first aspect, the feature determination unit may determine the age of a person from a face image included in the face region. In this case, since the relationship between the persons whose intimacy has been calculated is determined using the intimacy and the age of the person, it is possible to improve the accuracy of specifying the relationship between the persons from the target image.

  In the image processing apparatus according to the first aspect, the feature determination unit may determine the sex of a person from a face image included in the face area. In this case, since the relationship between the persons whose intimacy has been calculated is determined using the intimacy and the person's gender, it is possible to improve the accuracy of identifying the relationship between the persons from the target image.

  In the image processing apparatus according to the first aspect, the interstitial determination unit stores two or more interstitial determination information indicating a correspondence relationship between the intimacy and the characteristics of the person and the interpersonal relation. And a interstitial determination information selection unit that selects one interstitial determination information from the storage unit based on information on the location where the target image was taken, and is selected by the interstitial determination information selection unit The relationship between the persons whose intimacy has been calculated may be determined using the determined relationship determination information. In this case, using the relationship determination information selected by the relationship determination information selection unit, the relationship between the people whose intimacy is calculated is determined, so that the accuracy of identifying the relationship between the people from the target image is improved. be able to.

  In the image processing device according to the first aspect, the information related to the location where the target image was taken may be specified using GPS information. In this case, since the information regarding the location where the target image was taken is specified using GPS information, it is possible to improve the accuracy of specifying the relationship between people from the target image.

  The image processing apparatus according to the first aspect further includes a relationship diagram representing a relationship between all the persons whose intimacy is calculated by the intimacy calculating unit based on the relationship between the persons determined by the relationship determining unit. An interstitial relationship diagram creation unit may be provided. In this case, it is possible to display the relationship of all the persons whose intimacy has been calculated.

  In the image processing apparatus according to the first aspect, the interstitial relationship diagram represents a human face image for which the familiarity is calculated and a relation line connecting the facial images, and the familiarity is high. The distance between the face images may be smaller as the number of people increases, and the width of the relationship line may be larger as the numbers of people who appear more frequently in the target image. In this case, it is possible to easily display the relationships of all persons whose intimacy has been calculated.

  In the image processing device according to the first aspect, the subject distance estimation unit may estimate the subject distance based on the size of the face area with respect to the size of the target image. In this case, it is possible to easily estimate the subject distance, which is the distance from the face represented in the face area to the photographing apparatus.

  In the image processing device according to the first aspect, the closeness calculating unit is represented in at least a part of a plurality of target images arranged in time series, and the distance between subjects is less than or equal to a predetermined value, The intimacy may be calculated for people whose frequency appears in the target image is equal to or greater than a predetermined number, and the intimacy may not be calculated for people whose frequency appears in the target image is less than the predetermined number. In this case, it is possible to improve the efficiency of the process of specifying the relationship between people from the target image.

  In the image processing device according to the first aspect, the closeness calculating unit is configured such that the distance between the face images included in the first target image is equal to the distance between the face images included in the second target image. If the actual distance between the faces represented in the first target image is smaller than the actual distance between the faces represented in the second target image, it is represented in the first target image. The intimacy may be calculated so that the intimacy of the people represented in the second target image is higher than that of the inferred people. In this case, it is possible to accurately specify the relationship between people from the target image.

  In the image processing device according to the first aspect, the interstitial relationship diagram creating unit is configured such that the distance between the face images included in the first target image is the face image included in the second target image. If the actual distance between the faces represented in the first target image is smaller than the distance between each other and greater than the actual distance between the faces represented in the second target image, The interstitial relationship diagram may be created so that the distance between human face images appearing in the second target image is smaller than the distance between human face images appearing in the first target image. In this case, it is possible to accurately display the relationships of all the persons whose intimacy has been calculated.

  The present invention can be realized in various modes, for example, a printer, a digital still camera, a personal computer, a digital video camera, or the like. Further, an image processing method and apparatus, an affinity calculation method and apparatus, a relationship relationship display method and apparatus, a computer program for realizing the functions of these methods or apparatuses, a recording medium on which the computer program is recorded, and the computer program It can be realized in the form of a data signal embodied in a carrier wave.

FIG. 2 is an explanatory diagram schematically illustrating a configuration of a printer as an image processing apparatus according to a first embodiment. 6 is a flowchart showing a flow of image search / print processing in the first embodiment. It is explanatory drawing which illustrated a part of each target image represented by a moving image file. It is explanatory drawing for demonstrating the moving image file stored in the internal memory. It is explanatory drawing which shows an example of the detection result of face area FA. It is explanatory drawing for demonstrating the person information database stored in the internal memory. It is explanatory drawing for demonstrating matching with an image identifier and a person identifier. It is a flowchart which shows the flow of the person information setting process in 1st Example. It is explanatory drawing which shows the estimation method of object distance Sd. It is explanatory drawing which shows the estimation method of the to-be-photographed object distance Sbd. It is explanatory drawing for demonstrating the state which set to-be-photographed object distance Sbd in the person information database HC. It is explanatory drawing for demonstrating the state which set frequency information to the person information database HC. It is explanatory drawing for demonstrating the state which set the person information about age and sex in a person information database. It is explanatory drawing for demonstrating the state which set familiarity Di to the person information database HC. It is a flowchart which shows the flow of an interim determination process. It is explanatory drawing for demonstrating the content of 1st design determination information. It is explanatory drawing for demonstrating the content of 2nd design determination information. It is explanatory drawing for demonstrating the state which set the relationship information in the person information database HC. It is explanatory drawing which illustrated the relationship relationship figure. It is explanatory drawing which illustrated the user interface containing a relationship relationship figure. It is explanatory drawing which illustrated the 1st printing screen. It is explanatory drawing which illustrated the 2nd printing screen. It is explanatory drawing for demonstrating the arrangement order of the target image contained in a target image list. It is explanatory drawing which illustrated the 3rd printing screen. It is a flowchart which shows the flow of the person information setting process in 2nd Example. It is explanatory drawing for demonstrating the content of 3rd relationship determination information.

  Hereinafter, a printer which is an aspect of an image processing apparatus according to the present invention will be described based on examples with reference to the drawings.

A. First embodiment:
A1. Configuration of image processing device:
FIG. 1 is an explanatory diagram schematically showing the configuration of a printer as an image processing apparatus in the first embodiment. The printer 100 of this embodiment is an ink jet color printer that supports so-called direct printing, in which an image is printed based on image data acquired from a memory card MC or the like. The printer 100 includes a CPU 110 that controls each unit of the printer 100, an internal memory 120 configured by a ROM and a RAM, an operation unit 140 configured by buttons and a touch panel, a display unit 150 configured by a liquid crystal display, and printing. A mechanism 160 and a card interface (card I / F) 170 are provided. The printer 100 may further include an interface for performing data communication with other devices (for example, a digital still camera or a personal computer). Each component of the printer 100 is connected via a bus so that bidirectional communication is possible.

  The printing mechanism 160 performs printing based on the print data. The card interface 170 is an interface for exchanging data with the memory card MC inserted into the card slot 172. In this embodiment, a moving image file is stored in the memory card MC. This moving image file includes, for example, a plurality of image files that are sequentially and temporally generated in accordance with the Exif (Exchangeable Image File Format) standard by a photographing device such as a digital still camera. In addition to image data as RGB data generated by shooting, it includes additional data such as shooting date and time, aperture value (F value) at the time of shooting, shutter speed, lens focal length, latitude and longitude information at the time of shooting. . For the moving image file, for example, a compression format such as Motion JPEG can be adopted. For example, a compression format such as JPEG can be adopted for the image file constituting the moving image file. The printer 100 acquires a moving image file stored in the memory card MC via the card interface 170.

  The internal memory 120 stores an image processing unit 200, a display control unit 310, and a print control unit 320. The image processing unit 200 is a computer program, and performs image search / printing processing by being executed by the CPU 110 under a predetermined operating system. In the image search / print processing of the present embodiment, a closeness representing the degree of intimacy between people is calculated from a plurality of images to be searched (hereinafter also referred to as “target images”), and this closeness is used. This is a process of performing an image search for detecting a specific image and printing the detected image. The image search / print process will be described in detail later. The display control unit 310 and the print control unit 320 are also executed by the CPU 110 to realize the respective functions.

  The image processing unit 200 includes a face area detection unit 210, a subject distance estimation unit 220, an inter-subject distance estimation unit 230, a closeness calculation unit 240, a feature determination unit 250, and a specific information acquisition unit 260 as program modules. And an image search unit 270, a interstitial determination unit 280, and a interstitial relationship diagram creation unit 290. The interstitial determination unit 280 includes a interstitial determination information selection unit 281. The functions of these units will be described in detail in the description of image search / print processing described later.

  The display control unit 310 is a display driver that controls the display unit 150 to display processing menus, messages, images, and the like on the display unit 150. The print control unit 320 is a computer program for generating print data from image data, controlling the printing mechanism 160, and printing an image based on the print data. The CPU 110 reads out and executes these programs (the image processing unit 200, the display control unit 310, and the print control unit 320) from the internal memory 120, thereby realizing the functions of these units.

  The internal memory 120 also stores relationship determination information RDI. The interstitial determination information RDI is information set in advance to be referred to in an image search / print process described later. The contents of the interstitial determination information RDI will be described in detail in the description of the image search / print processing described later.

A2. Image Search / Print Processing FIG. 2 is a flowchart showing the flow of image search / print processing in the first embodiment. The image search / printing process of the present embodiment creates a interstitial relationship diagram representing the interstitial relationship between people based on the intimacy calculated for the people represented in the moving image, and is selected by a user who refers to the interstitial relationship diagram The printed image is printed.

  The image processing unit 200 acquires a target image that is a target of image search (step S100). In this embodiment, all images represented by moving image files stored in the memory card MC will be described as target images to be subjected to image search / print processing. The target image may be an image represented by a moving image file stored in a storage device other than the memory card MC, or a part of an image represented by a moving image file stored in the memory card MC. It may be. In the present embodiment, when the memory card MC is inserted into the card slot 172, the printer 100 acquires a target image from the memory card MC. That is, the printer 100 stores the moving image file stored in the memory card MC in the internal memory 120.

  FIG. 3 is an explanatory diagram illustrating a part of each target image represented by the moving image file. Each of the target images TI0 to TI3 is a temporally continuous image in the order of the target image TI0, the target image TI1, the target image TI2, and the target image TI3. That is, the target images TI0 to TI3 are images arranged in time series in the order of the target image TI0, the target image TI1, the target image TI2, and the target image TI3. The target image TI corresponds to each frame of the moving image represented by the moving image file. There is no particular limitation on the frame rate of the moving image represented by the moving image file, and 30 fps is adopted in this embodiment. However, a value other than 30 fps may be adopted for the frame rate.

  FIG. 4 is an explanatory diagram for explaining a moving image file stored in the internal memory. FIG. 4 shows each target image TI represented by a moving image file stored in the internal memory 120 and an image identifier for identifying each target image TI. The image processing unit 200 associates an image identifier with each image file corresponding to each acquired target image TI and stores the image identifier in a predetermined area of the internal memory 120. Here, the image identifier is an identifier that can uniquely identify each of the image files stored in the internal memory 120, and may be a character string such as a file name in addition to numbers and symbols.

  The image processing unit 200 sets an attention image OI (step S110). Specifically, the image processing unit 200 selects one image file from the image files stored in the internal memory 120. The image processing unit 200 sets the target image TI corresponding to the selected image file as the target image OI. In the present embodiment, an example in which the target image TI1 shown in FIG. 4 is set as the target image OI will be described.

  The face area detection unit 210 detects the face area FA in the target image OI (step S120). Here, the face area FA is an image area on the attention image OI and means an area including at least a partial image of the face. The detection of the face area FA by the face area detection unit 210 is performed using a known face detection method such as a pattern matching method using a template (see Japanese Patent Application Laid-Open No. 2004-318204).

  FIG. 5 is an explanatory diagram showing an example of the detection result of the face area FA. In the example of FIG. 5, the attention image OI includes two face images. Therefore, here, two face areas FA1 and FA2 are detected from the target image OI. As shown in FIG. 5, the face area FA is a rectangular area including images of both eyes, nose and mouth. Further, the face area detection unit 210 detects a head area TA that is a rectangular area including the entire head based on the face area FA. The position and range of the head area TA in the target image OI are set in advance so as to be predetermined with respect to the position and range of the detected face area FA in the target image OI. Therefore, the position and range of the head area TA are uniquely defined by detecting the face area FA. The method for defining the head area TA is not particularly limited. For example, the head area TA may be outside the face area FA by a predetermined number of pixels, or a predetermined constant is set for the number of pixels in the vertical and horizontal directions of the face area FA. It is good also as a range calculated by multiplying. In this embodiment, two head areas TA1 and TA2 are detected.

  Note that the face area detecting unit 210 detects information indicating the positions of the face area FA and the head area TA in the target image OI (for example, four vertices of the face area FA) as the detection results of the face area FA and the head area TA. And the coordinates of the four vertices of the head area TA). Further, as shown in FIG. 5, in this embodiment, the width of the target image OI is represented as Wwi (unit is the number of pixels), and the width of the face area FA is represented as Wfi1 and Wfi2 (unit is the number of pixels). To do. If the face area FA is not detected in the detection of the face area FA in step S120, another target image TI is newly selected as the attention image OI, and the face area FA is detected.

  In step S120, the face area FA is detected from the target image OI by pattern matching using a template. A known face detection method such as a pattern matching method using such a template generally does not detect in detail the position and inclination (angle) of the entire face or part of the face (eyes, mouth, etc.) An area that is considered to contain a face image from the attention image OI is set as the face area FA.

  FIG. 6 is an explanatory diagram for explaining the person information database stored in the internal memory. The image processing unit 200 generates image data of an image corresponding to the head area TA of the target image OI. Since the head region TA includes an image representing the entire human head, the generated image data represents a person image PI for each person included in the target image OI. As shown in FIG. 6, the image processing unit 200 stores, in a predetermined area of the internal memory 120, image data representing the person image PI and a person information database HC associated with a person identifier for identifying each person image. To do. Here, since two person images are included in the target image TI1 set as the attention image OI, the image data of the person image PI1 of the person represented by the head area TA1 is associated with the person identifier aa, and the head image The image data of the person image PI2 of the person represented by the area TA2 is associated with the person identifier bb. In the present embodiment, the image processing unit 200 generates the image data of the person image PI corresponding to the head area TA of the target image OI, but does not generate the image data and specifies the position of the head area TA. For example, the person information database HC associated with the person identifier and the image identifier of the target image OI in which the head area TA is detected may be constructed.

  FIG. 7 is an explanatory diagram for explaining the association between an image identifier and a person identifier. The image processing unit 200 associates the image file corresponding to the target image OI stored in the internal memory with the image data of the person image PI included in the target image OI. Specifically, as shown in FIG. 7, the image processing unit 200 includes an image identifier IA of an image file corresponding to the target image TI1, and person identifiers aa and bb respectively corresponding to the two person images PI1 and PI2. Associate.

  The image processing unit 200 performs person information setting processing (step S130). The person information setting process refers to the intimacy and relationship between the attention image OI, the age and sex of the person represented in the attention image OI, and the degree of intimacy between the persons represented in the attention image OI. Is acquired and set in the person information database HC. In addition to the above, the person information may include, for example, the facial expression or face orientation of the person represented in the attention image OI.

  FIG. 8 is a flowchart showing the flow of the person information setting process in the first embodiment. The subject distance estimation unit 220 estimates the subject distance Sd (step S200). Here, the subject distance Sd is the distance from the photographing device (more specifically, the principal point of the lens of the photographing device) to the face of the person who is the subject at the time of photographing the target image OI.

  FIG. 9 is an explanatory diagram showing a method for estimating the subject distance Sd. FIG. 9 shows the positional relationship between the imaging plane IS of the photographing apparatus and the face of the person P as the subject at the time of photographing the attention image OI. As shown in FIG. 9, the subject distance Sd, which is the distance between the principal point UP of the lens and the face of the person P, is a plane including the position of the face of the person P and parallel to the imaging plane IS (hereinafter referred to as “subject plane”). It is determined by the width Ww of the shooting range and the angle of view θ1). The angle of view θ1 is specified by the relationship between the focal length f of the lens and the width Wx of the imaging plane IS. That is, the following formula (1) is established.

  Further, the width Ww of the imaging range on the subject surface SS is specified based on the size that the face image of the person P occupies in the attention image OI (FIG. 5). That is, the ratio between the width Ww on the subject surface SS and the width Wf of the face of the person P is considered to be equal to the ratio between the width Wwi of the entire image in the target image OI and the width Wfi of the face area FA (formula (2)). reference).

  From the above formulas (1) and (2), the following formula (3) is derived.

  The subject distance estimation unit 220 acquires information necessary for calculating the subject distance Sd using Expression (3). Specifically, the subject distance estimation unit 220 acquires the value (number of pixels) of the entire width Wwi of the target image OI added as metadata to the image file representing the target image OI, and also detects the face area FA ( The value (number of pixels) of the width Wfi in FIG. 5) is calculated. The width Wfi of the face area FA is calculated, for example, by calculating the distance between the two vertices using the coordinates of the two vertices of the four vertices of the face area FA described above.

  In addition, the subject distance estimation unit 220 approximates the typical human face width (actual face size) stored in the internal memory 120 in advance as the value of the face width Wf of the person P. For example, 200 mm).

  Further, the subject distance estimation unit 220 acquires the value of the focal length f of the lens at the time of imaging included in the additional data of the image file of the target image OI. Here, the value of the focal length f of the lens acquired is a 35 mm film equivalent value and may be different from the actual focal length (actual focal length) of the imaging apparatus. In such a case, the subject distance estimation unit 220 acquires a preset value of the width of the 35 mm film as the width Wx of the imaging plane IS. If the additional data of the image file includes data on the actual focal length and data on the width of the image sensor of the imaging device, the subject distance estimation unit 220 sets the actual focal length as the lens focal length f. A value may be acquired, and the value of the width of the image sensor may be acquired as the width Wx of the imaging plane IS. If the additional data of the image file includes data indicating the angle of view, the subject distance estimation unit 220 may acquire data indicating the angle of view.

  The subject distance estimation unit 220 obtains each of the acquired information (the value of the entire width Wwi of the target image TI, the value of the width Wfi of the face area FA, the value of the face width Wf of the person P, and the value of the focal length f of the lens). The object distance Sd is calculated (estimated) by using the value of the width Wx of the image plane IS and the above equation (3). In the present embodiment, a subject distance Sd1 that is a distance from the photographing apparatus to a human face corresponding to the face area FA1 and a subject distance Sd2 that is a distance from the photographing apparatus to a human face corresponding to the face area FA2 are estimated. The

  Note that the ratio of the subject distance Sd1 that is the distance from the photographing apparatus to the face of the person corresponding to the face area FA1 and the subject distance Sd2 that is the distance from the photographing apparatus to the face of the person corresponding to the face area FA2 is the face Since it is considered to be equal to the ratio of the width Wfi1 of the area FA1 and the width Wfi2 of the face area FA2, the subject distance Sd1 is calculated using the above formula (3), and the subject distance Sd2 is calculated by the following formula (4). May be.

  When the subject distance estimation unit 220 estimates the subject distance Sd, the subject distance estimation unit 230 estimates the subject distance Sbd (step S210). Here, the inter-subject distance Sbd is an actual distance from one face to the other face of the two faces represented in the attention image OI. In other words, the inter-subject distance Sbd is an actual distance at the time of photographing the faces represented in the face area FA.

  FIG. 10 is an explanatory diagram showing a method for estimating the inter-subject distance Sbd. FIG. 10 shows the positional relationship between the imaging device PD and the faces of the persons P1 and P2 as subjects when the attention image OI is captured. The person P1 is a person corresponding to the face area FA1 shown in FIG. 5, and the person P2 represents a person corresponding to the face area FA2. As shown in FIG. 10, the inter-subject distance Sbd, which is the distance between the face of the person P1 and the face of the person P2, is a depth direction distance Sh that is a distance component on a straight line connecting the photographing apparatus PD and the person P1. , And is specified by the orthogonal direction distance Sv which is a distance component in a direction orthogonal to the depth direction. That is, Expression (5) is established.

  Further, the orthogonal distance Sv is a formation angle θ2 formed by a straight line connecting the photographing apparatus PD and the person P1, a straight line connecting the photographing apparatus PD and the person P2, and a distance from the photographing apparatus to the face of the person P2. Is specified by the subject distance Sd2. That is, Expression (6) is established.

  Further, the orthogonal direction distance Sv and the depth direction distance Sh are specified by the subject distance Sd1 which is the distance from the photographing apparatus to the face of the person P1 and the formation angle θ2. That is, Expression (7) is established.

  The depth direction distance Sh is specified by using the subject distances Sd1 and Sd2 and the formation angle θ2 by Expression (6) and Expression (7). That is, Expression (8) is established.

  From the equations (5), (6), and (8), the subject distance Sbd can be specified using the subject distances Sd1, Sd2 and the formation angle θ2. That is, Expression (9) is established.

  The inter-subject distance estimation unit 230 acquires information necessary for calculating the inter-subject distance Sbd using Expression (9). Specifically, the inter-subject distance estimation unit 230 acquires the subject distances Sd1 and Sd2 estimated by the subject distance estimation unit 220 and calculates the value of the formation angle θ2. The formation angle θ2 is calculated by, for example, the ratio of the length of the line segment connecting the center of the face area FA1 and the center of the face area FA2 in the target image OI to the width Wwi of the entire image, the formation angle θ2, It can be calculated by using the ratio to the angle θ1.

  The inter-subject distance estimation unit 230 sets the calculated inter-subject distance Sbd in the person information database HC. FIG. 11 is an explanatory diagram for explaining a state in which the subject distance Sbd is set in the person information database HC. The inter-subject distance estimation unit 230 associates the calculated inter-subject distance Sbd with the image data of the two person images PI that are the calculation targets in the person information database HC. Specifically, when the subject distance Sbd between the person represented by the head area TA1 and the person represented by the head area TA2 in the attention image OI is “50”, as shown in FIG. In the image data of the person image PI1 corresponding to the head area TA1, the image identifier “IB” of the target image TI set as the target image OI, the partner person identifier “bb” for which the inter-subject distance Sbd is calculated, and the subject The distance Sbd “50” is associated with each other. In addition, in the image data of the person image PI2 corresponding to the head area TA2, the image identifier “IB” of the target image TI set as the attention image OI, and the person identifier “aa” of the partner who calculated the inter-subject distance Sbd , “50”, which is the inter-subject distance Sbd, is associated with each other.

  The image processing unit 200 determines whether or not the inter-subject distance Sbd is smaller than the threshold Th1 (Step S220). Specifically, the image processing unit 200 compares the threshold Th1 set in advance in the internal memory 120 with the inter-subject distance Sbd calculated by the inter-subject distance estimation unit 230 in step 210, and the inter-subject distance It is determined whether Sbd is smaller than the threshold Th1. When the inter-subject distance Sbd is equal to or greater than the threshold Th1 (step S220: NO), the image processing unit 200 sets “frequency n = 0” as frequency information to be described later in the person information database HC (step S225). When “frequency n = 0” is set as frequency information in the person information database HC, the image processing unit 200 ends the person information setting process.

  When the inter-subject distance Sbd is smaller than the threshold Th1 (step S220: YES), the image processing unit 200 determines whether each person who calculated the inter-subject distance Sbd is a person represented in the previous target image. Determination is made (step S230). That is, the image processing unit 200 determines whether or not each person who has calculated the inter-subject distance Sbd this time is represented in any target image set as the target image before the target image set as the target image this time. Judgment is made. For example, in the target image TI1 set as the target image OI, two persons (a person represented by the head area TA1 and a person represented by the head area TA2) whose inter-subject distance Sbd has been calculated from the target image TI1. It is determined whether or not it is represented in the target image TI0 that is the previous image in time. For example, the image processing unit 200 calculates the similarity between the face image included in the head region TA detected from the target image TI1 and the person image PI represented in the person information database HC, and detects the detected head region. By determining whether or not the person represented by TA is already set in the person information database HC, whether or not each person whose inter-subject distance Sbd has been calculated is a person represented in the previous target image. Determine whether.

  When each person who has calculated the inter-subject distance Sbd is not a person represented in the previous target image TI (step S230: NO), the image processing unit 200 stores “frequency n = frequency” as frequency information in the person information database HC. 1 "is set (step S235). The frequency information is set for each person set in the person information database HC, and the number of target images represented in a state where the subject distance Sbd between each person and the specific person is smaller than the threshold value Th1. (Frequency n). If each person who calculated the inter-subject distance Sbd is not a person represented in the previous target image TI, the images appear in the target image for the first time this time. Frequency n = 1 is set as frequency information corresponding to the person.

  FIG. 12 is an explanatory diagram for explaining a state in which frequency information is set in the person information database HC. In the person information database HC, the image processing unit 200 associates “1” as frequency information with the image data of the two person images PI for which the inter-subject distance Sbd is calculated. Specifically, the image data of the person image PI1 corresponding to the head area TA1 is associated with the partner person identifier “bb” for which the inter-subject distance Sbd is calculated and the frequency information “1”. Similarly, the image data of the person image PI2 corresponding to the head area TA2 is associated with the partner person identifier “aa” for which the inter-subject distance Sbd has been calculated and the frequency information “1”. When “frequency n = 1” is set as frequency information in the person information database HC, the image processing unit 200 ends the person information setting process.

  When each person whose inter-subject distance Sbd is calculated is a person represented in the previous target image TI (step S230: YES), the image processing unit 200 sets each person set in the person information database HC. The frequency n of the frequency information corresponding to the person is incremented by one (step S240).

  The image processing unit 200 determines whether or not the frequency n incremented in step S230 is greater than the threshold value Th2 (step S250). When the frequency n is equal to or less than the threshold value Th2 (step S250: NO), the image processing unit 200 ends the person information setting process. When the frequency n is greater than the threshold Th2 (step S250: YES), the feature determination unit 250 determines whether the age and sex have already been determined for each person for which the inter-subject distance Sbd has been calculated (step S260). ). Specifically, the feature determination unit 250 refers to the person information database HC and determines whether age information and gender information to be described later is set for each person whose inter-subject distance Sbd has been calculated. When age and sex determination has already been performed for each person whose inter-subject distance Sbd has been calculated (step S260: YES), age determination and gender determination (step S270) are skipped. When the age and sex are not determined for each person for which the inter-subject distance Sbd is calculated (step S260: NO), the feature determination unit 250 performs age determination and sex determination (step S270).

  The age determination and the gender determination by the feature determination unit 250 are estimated based on facial feature amounts such as positions, sizes, ratios, and the like for eyes, noses, hairstyles, colors, and the like included in the face area FA (Japanese Patent Laid-Open No. 2007-2007). This is executed using a known discrimination method such as -206921). In this embodiment, the age estimated in the age determination is estimated at the time when the image search / print process is performed. That is, the age estimated from the image of the person represented in the target image OI has passed from the shooting date / time added as metadata to the image file representing the target image OI to the date / time when the image search / print processing was performed. The age at the time of image search / print processing is estimated by aging. In this embodiment, the age at the time of performing the image search / printing process is estimated. Instead, the birth year or the birth date of the person represented in the target image OI is estimated. There may be.

  FIG. 13 is an explanatory diagram for explaining a state in which person information regarding age and gender is set in the person information database. In the person information database HC, the feature determination unit 250 associates the person information regarding the determined age and sex with the image data of the person image PI corresponding to the person to be determined. Specifically, as shown in FIG. 13, the age (35 years old) and gender (male) determined from the face area FA1 of the target image OI are associated with the image data of the person image PI1 corresponding to the head area TA1. Further, the age (30 years old) and the sex (female) determined from the face area FA2 of the attention image OI are associated with the image data of the person image PI2 corresponding to the head area TA2.

  The familiarity calculation unit 240 calculates the familiarity Di (step S280). Here, the familiarity Di is a value representing the degree of intimacy between the people represented in the target image OI, and the higher the closer the people are to each other. The familiarity calculation unit 240 calculates the familiarity Di using the inter-subject distance Sbd estimated in step S210. Specifically, the familiarity calculation unit 240 calculates the familiarity Di so that the value of the familiarity Di increases as the inter-subject distance Sbd decreases. This is because, at the time of taking a picture, people who are intimate with each other are closer to each other, and those who are not intimate are less likely to be close to each other. The relationship between the familiarity Di and the inter-subject distance Sbd can be arbitrarily set, and the closeness Di does not need to have a linear correlation (linear correlation) with the inter-subject distance Sbd. It may be a curved correlation (non-linear correlation). Further, for example, as long as the inter-subject distance Sbd is equal to or larger than a predetermined value, the closeness Di value may be constant.

  The familiarity calculating unit 240 sets the calculated familiarity Di in the person information database HC. FIG. 14 is an explanatory diagram for explaining a state in which the familiarity Di is set in the person information database HC. The familiarity calculating unit 240 associates the calculated familiarity Di with the image data of the two person images PI that are the objects of calculation in the person information database HC. Specifically, in the attention image OI, when the closeness Di between the person represented by the head area TA1 and the person represented by the head area TA2 is “21”, as shown in FIG. The image data of the person image PI1 corresponding to the area TA1 is associated with the partner person identifier “bb” for which the familiarity Di is calculated and “21” which is the familiarity Di. Further, the image data of the person image PI2 corresponding to the head area TA2 is associated with the partner person identifier “aa” for which the familiarity Di is calculated and “21” which is the familiarity Dii.

  The interstitial determination unit 280 performs interstitial determination processing (step S290). The relationship determination process is a process for determining the relationship between people represented in the target image OI. FIG. 15 is a flowchart showing the flow of the interstitial determination process. The relationship determination unit 280 is represented in the attention image OI from the person information database HC using personal information such as the age and sex of each person who is the target of relationship determination and the closeness Di of the target people. Judge the relationship between the people who are.

  The interstitial determination information selection unit 281 of the interstitial determination unit 280 acquires position information (step S310). Here, the position information is information related to the location where the target image TI selected as the target image OI is captured, and includes information regarding the type of the captured location such as a street, a house, or a company. The relationship determination information selection unit 281 uses the latitude and longitude information at the time of shooting included in the additional data of the image file and the geographic information stored in the internal memory 120 to determine the location where the target image TI was shot. Identify. Here, the latitude / longitude information is obtained by embedding GPS information acquired by a GPS unit integral with or separate from the photographing apparatus in the image file as additional data. In other words, the interstitial determination information selection unit 281 specifies information related to the place where the target image TI is taken by using GPS information. Note that the printer 100 does not need to store geographic information in the internal memory 120. For example, the printer 100 may acquire information regarding a place taken from latitude and longitude information using map information such as a Google map. .

  The interstitial determination information selection unit 281 of the interstitial determination unit 280 selects one interstitial determination information from a plurality of interstitial determination information stored in the internal memory 120 (step S320). The relationship determination information indicates the correspondence between the personal information such as the age and sex of the person who is the target of the relationship determination and the closeness Di and the relationship. The plurality of interstitial determination information stored in the internal memory 120 indicates the correspondence set for each place where the image was taken. FIG. 16 is an explanatory diagram for explaining the contents of the first interstitial determination information. FIG. 17 is an explanatory diagram for explaining the contents of the second relationship determination information. The interstitial determination information selection unit 281 selects one interstitial determination information from a plurality of interstitial determination information stored in the internal memory 120 based on the information regarding the location where the target image was captured acquired in step S310. In other words, the interstitial determination information selection unit 281 selects interstitial determination information corresponding to the location where the target image was captured, which is indicated in the information regarding the location where the target image was captured.

  FIG. 16 exemplifies the contents of the relationship determination information corresponding to the image photographed on the street, which is one of the relationship determination information stored in the internal memory 120. FIG. 17 illustrates one example of the content determination information stored in the internal memory 120 and corresponding to the image photographed at the company. The contents of the respective pattern determination information will be described later. The contents of the interstitial determination information shown in FIGS. 16 and 17 are merely examples of interstitial determination information that can be set in the internal memory 120, and other interstitial determination information can be arbitrarily set. For example, the relationship determination information corresponding to the images taken at each place such as home, park, entertainment facility, or the relationship determination information selected when the position information is not acquired in step S310 is set. can do.

  The interstitial determination unit 280 determines interstitial relationships between the persons according to the interstitial determination information (step S330). First, the case where the interstitial determination information whose contents are shown in FIG. 16 is selected by the interstitial determination information selection unit 281 will be described. The relationship determination unit 280 compares the familiarity Di with the threshold Th3 (step S500). When the familiarity Di is equal to or less than the threshold Th3 (step S500: NO), the persons who are the targets of the relationship determination are “others”. (Step S505).

  When the familiarity Di is greater than the threshold Th3 (step S500: YES), the interstitial determination unit 280 determines whether or not the age difference between the persons who are subject to interstitial determination is smaller than the threshold Th4 (step S515). When the age difference is smaller than the threshold Th4 (step S515: YES), the interstitial determination unit 280 determines whether or not the age of the person who is the target of the interstitial determination is greater than 12 (step S520).

  When the age of at least one of the persons who are the targets of the relationship determination is 12 or less (step S520: NO), the relationship determination unit 280 determines that the persons who are the targets of the relationship determination are “friends” (step) S525). When the ages of the persons who are the targets of the relationship determination are each greater than 12 (step S520: YES), the relationship determination unit 280 determines whether the persons who are the targets of the relationship determination are the same (step S535).

  When the persons who are the targets of the relationship determination are homosexual (step S535: YES), the relationship determination unit 280 determines that the persons who are the targets of the relationship determination are “friends” (step S540). When the persons who are the targets of the relationship determination are heterosexual (step S535: NO), the relationship determination unit 280 determines that the persons who are the targets of the relationship determination are “lovers / couples” (step S545).

  In step S515, when the age difference is greater than or equal to the threshold Th4 (step S515: NO), the interrogation determination unit 280 performs face authentication (step S550). Here, in face authentication, in order to calculate the degree of similarity (face similarity) between face images of people who are the targets of relationship determination, a feature amount is calculated from each person's face image. This is a process of calculating the face similarity from the feature amount similarity. For example, when the target image is expressed in RGB, the pixel frequency of each color bin of the color histogram in the RGB color space can be used as the feature amount of the face image used here. Any known technique other than those described above may be used as long as it can calculate the similarity between human face images.

  When the face similarity calculated in step S550 is smaller than the threshold Th5 (step S555: YES), the interstitial determination unit 280 determines that the persons who are the targets of the interstitial determination are “parent and child” (step S560). When the face similarity calculated in step S550 is greater than or equal to the threshold Th5 (step S555: NO), the interstitial determination unit 280 determines that the persons who are the targets of the interstitial determination are “others” (step S565). .

  Next, a case where the interstitial determination information selection unit 281 selects the interstitial determination information whose contents are shown in FIG. 17 will be described. The relationship determination unit 280 compares the familiarity Di with the threshold Th6 (step S600). If the familiarity Di is equal to or less than the threshold Th6 (step S600: NO), the persons who are the targets of the relationship determination are “others”. (Step S605).

  When the familiarity Di is greater than the threshold Th6 (step S600: YES), the interstitial determination unit 280 determines whether or not the age difference between the persons who are subject to the interstitial determination is smaller than the threshold Th7 (step S615). When the age difference is equal to or greater than the threshold Th7 (step S615: NO), the interpersonal determination unit 280 determines that the persons who are the targets of the interpersonal determination are “boss and subordinate” (step S617).

  When the age difference is smaller than the threshold Th7 (step S615: YES), the interstitial determination unit 280 determines whether or not the familiarity Di is larger than the threshold Th8 (> threshold Th6) (step S620). When the intimacy Di is equal to or less than the threshold Th8 (step S620: NO), the relationship determination unit 280 determines that the persons who are the targets of the relationship determination are “colleagues” (step S625). When the familiarity Di is greater than the threshold Th8 (step S620: YES), the interstitial determination unit 280 determines whether or not the persons who are the targets of the interstitial determination are the same (step S635).

  When the persons who are the targets of the relationship determination are the same sex (step S635: YES), the relationship determination unit 280 determines that the persons who are the targets of the relationship determination are “friends” (step S640). When the persons who are the targets of the relationship determination are heterosexual (step S635: NO), the relationship determination unit 280 determines that the persons who are the targets of the relationship determination are “lovers” (step S645).

  The relationship determination unit 280 sets information related to the determined relationship (hereinafter also referred to as “relationship information”) in the person information database HC. FIG. 18 is an explanatory diagram for explaining a state in which relationship information is set in the person information database HC. The interstitial determination unit 280 associates the image data of the two person images PI to be determined with the interstitial information in the personal information database HC. Specifically, in the attention image OI, when the relationship between the person represented by the head area TA1 and the person represented by the head area TA2 is “lover / couple”, as shown in FIG. The image data of the person image PI1 corresponding to the area TA1 is associated with the person identifier “bb” of the partner who has determined the relationship and “lover / couple” which is the relationship information. In addition, the image data of the person image PI2 corresponding to the head area TA2 is associated with the person identifier “aa” of the partner who has determined the relationship and “lover / couple” which is the relationship information. The interstitial determination process ends as described above. In addition, the personal information setting process is terminated by the termination of the relationship determination process.

  Returning to FIG. 2, the image processing unit 200 sequentially sets the target image TI corresponding to the image file stored in the internal memory 120 as the attention image OI, and executes the processing from step S110 to step S130. (Step S135: NO). At this time, if the person represented by the head area TA detected in step S120 is already set in the person information database HC, the image processing unit 200 creates a new record in the person information database HC for the person. Is not set. For example, the image processing unit 200 calculates the similarity between the face image included in the head area TA detected in step S120 and the person image PI represented in the person information database HC, thereby detecting the detected head. It is determined whether or not the person represented by the area TA has already been set in the person information database HC.

  The image processing unit 200 sets the target image TI for all target images TI, and when the processing from step S110 to step S130 is completed (step S135: YES), the interstitial relationship diagram creation unit 290 creates the interstitial relationship diagram. Perform (step S150). The relationship diagram is a diagram showing the relationship between all the persons whose intimacy Di is calculated by illustrating the relationships between the persons whose intimacy Di is calculated. The interstitial relationship diagram creating unit 290 generates image data representing the interstitial relationship diagram.

  FIG. 19 is an explanatory diagram illustrating a relationship relationship diagram. The interpersonal relationship diagram RM includes a person image PI (PI1 to PI6), a person image frame Fi formed on the outer periphery of the person image PI, and the age information ag and gender information Ss of the person represented by the person image PI. And a relationship line Rl that connects the person image frames Fi to each other.

  The interstitial relationship diagram creating unit 290 creates the person image PI of the interstitial relationship diagram RM using the image data of the person image PI set in the person information database HC. Further, the interstitial relationship diagram creating unit 290 creates age information ag and gender information Ss of the interstitial relationship diagram RM using the personal information about the age and sex set in the personal information database HC. The gender information Ss is represented as a symbol in a part of the person image frame Fi as shown in FIG. The interstitial relationship diagram creation unit 290 determines the length Lr of the relationship line Rl using the familiarity Di set in the person information database HC. Specifically, the interstitial relationship diagram creation unit 290 increases the length of the relationship line Rl such that the greater the familiarity Di, the shorter the length Lr of the relationship line Rl that connects the person image frames Fi of the persons concerned. The length Lr is determined. That is, in the interpersonal relationship diagram RM, the person image frames Fi are set closer to each other as the closeness Di is greater. Thereby, the user can grasp | ascertain the degree of closeness of persons easily through vision.

  The interstitial relationship diagram creation unit 290 determines the width Wr of the relationship line using the frequency n set in the person information database HC. Specifically, the interstitial relationship diagram creation unit 290 increases the width Wr of the relationship line Rl that connects the person image frames Fi of the persons who become the target as the frequency n increases. As a result, in the interpersonal relationship diagram RM, the relationship line Rl that connects the person image frames Fi of the persons increases with increasing frequency in the target image TI, and therefore the reliability of the relationship line Rl is high. Recognize. The interstitial relationship diagram creating unit 290 stores image data representing the generated interstitial relationship diagram in the internal memory 120.

  The display control unit 310 displays the interstitial relationship diagram RM (step S155). FIG. 20 is an explanatory diagram illustrating a user interface including a relationship relationship diagram. The display control unit 310 causes the display unit 150 to display a user interface including the interstitial relationship diagram RM using the image data representing the interstitial relationship diagram RM generated in step S150. The user interface shown in FIG. 20 includes a display indicating that the user is requested to select a relationship line Rl or a person image PI, and an input box Bo, in addition to the relationship relationship diagram RM. The printer 100 according to the present embodiment searches for a target image including a person specified by selecting a relation line Rl by the user or inputting a keyword or the like in the input box Bo.

  The specific information acquisition unit 260 acquires specific information (step S160). Here, the specific information is information for specifying a person to be searched. Here, the relationship line Rl selected by the user corresponds to the specific information. In addition, the identification information may be a person image PI or a person image frame selected by the user in the interpersonal relationship diagram RM, or information for identifying a person input in the input box Bo. It may be. The information for identifying a person that is input to the input box Bo is, for example, input information such as gender and age, or when personal information and name information are associated with the personal information database HC. Input information such as name corresponds to the specific information. When the file name of an image file representing a person to be searched is input in the input box Bo, this file name corresponds to the specific information. That is, it can be said that the specific information is input information input by the user.

  The image search unit 270 performs image search (step S170). The image retrieval unit 270 identifies the image identifier of the target image TI including the two person images PI connected by the relationship line Rl from the relationship line Rl (specific information) selected by the user. Specifically, as shown by the oblique lines in FIG. 20, when the relationship line Rla between the person represented by the person image PI1 and the person represented by the person image PI2 is selected by the user, the image search unit 270 The familiarity Di corresponding to the selected relationship line Rla is specified from the person information database HC shown in FIG. The image search unit 270 identifies the image identifiers IB to ID of the target image TI for which the inter-subject distance Sbd that is the basis for calculating the familiarity Di is calculated from the identified familiarity Di. The image search unit 270 selects the target images TI1 to TI4 corresponding to the identified image identifiers IB to ID from the internal memory 120. As described above, since the two persons connected by the relationship line are specified by the relationship line Rl that is the specific information, the image search unit 270 searches the target image TI based on the specific information. It can be said that the target image TI including the face image is searched.

  The display control unit 310 displays a print image (step S180). FIG. 21 is an explanatory diagram illustrating the first print screen. The display control unit 310 displays the target images TI1 to TI4 searched in step S170 on the display unit 150. At this time, the display control unit 310 displays the objects in a descending order of the subject distance Sbd between the person represented by the person image PI1 and the person represented by the person image PI2. Specifically, as shown in FIG. 18, the inter-subject distance Sbd in the target image TI1 is “50”, the inter-subject distance Sbd in the target image TI2 is “55”, and the inter-subject distance in the target image TI3. Since the distance Sbd is “60”, the display control unit 310 displays the target image TI1, the target image TI2, and the target image TI3 in this order (here, the first image is on the left side). In addition to the print image, four buttons BN1 to BN3 and 5 are displayed on the print screen. When one (or a plurality of) target images TI are selected by the user and the print button BN3 is selected, a print process for printing the selected images is executed.

  The print controller 320 performs a printing process (step S190). The print control unit 320 generates print data for the selected target image TI. Specifically, for the selected target image TI, the print control unit 320 dot-converts the color conversion process for matching the pixel value of each pixel with the ink used by the printer 100 and the gradation of the pixel after the color conversion process. Print data is generated by performing halftone processing for representing the distribution of the image data, rasterizing processing for rearranging the data arrangement of the image data subjected to the halftone processing to the order to be transferred to the printer 100, and the like. The printing mechanism 160 prints the selected target image TI based on the print data generated by the print control unit 320. Thus, the image search / print process ends.

  In the above description, in step S160, the relationship line R1 selected by the user is used as the specific information. However, a case where the person image PI is selected by the user in the user interface shown in FIG. 20 will be described. Here, a case where the person image PI1 is selected will be described.

  The image search unit 270 performs image search (step S170). The image search unit 270 uses the person information database HC to specify the person identifier aa of the person corresponding to the person image PI1 (specific information) selected by the user. As illustrated in FIG. 7, the image search unit 270 selects a target image TI associated with the identified person identifier aa from the target images TI stored in the internal memory 120. The target image TI selected at this time is included in a target image list Rst1 described later.

  The image search unit 270 uses the person information database HC to identify the person identifier bb of the person corresponding to the person image PI1 and the person with the highest familiarity Di. The image search unit 270 selects a target image TI associated with the identified person identifier bb from the target images TI stored in the internal memory 120. The target image TI selected at this time is included in a target image list Rst2 described later. At this time, the target image TI included in the target image list Rst1 may overlap and be included in the target image list Rst2.

  The image search unit 270 specifies person identifiers in descending order of the closeness Di between the person image PI1 and the corresponding person, and sequentially stores the target images TI associated with the specified person identifiers in the internal memory 120. The selected target image TI is selected. The target images TI selected for each person identifier are included in different target image lists Rst. At this time, the same target image TI may be included in different target image lists Rst. Hereinafter, in each target image list Rst, the person represented by the person identifier used for the search is referred to as “reference person”. Here, the reference person in the target image list Rst1 is a person corresponding to the person image PI1, and the reference person in the target image list Rst2 is a person corresponding to the person image PI2.

  As described above, the image search unit 270 includes a target image including a face image of a person (here, the person corresponding to the person image PI1) specified by the specific information from the plurality of target images TI. In addition to searching for an image TI, a search is also made for a target image TI that includes an image of the face of a person to be searched and a person having a predetermined familiarity Di. Here, the person having the predetermined familiarity Di is, for example, that the familiarity Di between the person to be searched and the person having the highest familiarity Di or the person to be searched is a set value or more. A person.

  The image search unit 270 uses the object identifiers of all persons included in the person information database HC, and uses the object images TI associated with each person identifier from the object images TI stored in the internal memory 120, respectively. Alternatively, the target image TI associated with the person identifier may be selected from the target images TI stored in the internal memory 120 using only the human identifiers for a preset number of people. In addition, the target image TI associated with the person identifier is stored in the internal memory 120 using only the person identifier of the person whose closeness Di with the person to be searched is a predetermined value or more. You may select from image TI.

  The display control unit 310 displays a print image (step S180). FIG. 22 is an explanatory diagram illustrating the second print screen. The display control unit 310 causes the display unit 150 to display the target image TI searched in step S170. At this time, the display control unit 310 displays the target image TI for each target image list Rst, as shown in FIG. The display control unit 310 displays the target images TI included in each target image list Rst in a stacked manner, and displays the person image PI of the reference person on the top of the stacked target images TI. When at least a part of the stacked target images TI is selected by the user, the display control unit 310 displays the entire selected target image TI.

  The display control unit 310 sets the target image list Rst1 of the person to be searched (the person represented by the person image PI1) at the top (here, the leftmost side), and has a greater closeness Di with the person to be searched. The target image list Rst with a high person as a reference person is displayed in order (on the right side). Specifically, as shown in FIG. 22, the target image list Rst2 of the person represented by the person image PI1 and the person with the highest familiarity Di (person represented by the person image PI2) is represented by the person image PI1. The target image list Rst3 of the person who has the second highest intimacy Di (the person represented by the person image PI5) and the person represented by the person image PI1 and the third highest intimacy Di (the person image PI4) Are displayed in the order of the target image list Rst4 (in this case, in order from the left side).

  FIG. 23 is an explanatory diagram for explaining the arrangement order of the target images included in the target image list. In each target image list Rst, the display control unit 310 displays the target images TI in order from the group of target images TI including images of faces of people with a higher familiarity Di with the reference person. Specifically, as in the target image list Rst1 illustrated in FIG. 23, the display control unit 310 includes a target image including the person image PI2 of the person represented by the person image PI1 and the person having the highest familiarity Di. Target image group including target images TId, TIE, and TIf including target image group Grp1 including TIa, TIb, and TIc, and a person image PI5 of a person represented by person image PI1 and a person having the second highest intimacy Di The group Grp2, the target image group Grp3 including the target image TIg and the target image group Grp3 including the person image PI4 of the person having the third highest familiarity Di and the person represented by the person image PI1 (in this case, from the left side) The target image TI is displayed side by side.

  In each target image group Grp in the target image list Rst, the display control unit 310 displays the target images TI in order starting from the target image TI having the smallest subject distance Sbd between the person included in the target images of the group and the reference person. Specifically, as illustrated in FIG. 23, in the target image group Grp1 of the target image list Rst1, the display control unit 310 includes a person (a person represented by the person image PI2) that is commonly included in the target images of the group. And the target image TI having the smallest inter-subject distance Sbd with the reference person (the person represented by the person image PI1) are displayed in order. That is, among the three target images TIa, TIb, and TIc, the display control unit 310 has a target image TIa having the smallest subject distance Sbd, a second target image TIb having a smallest subject distance Sbd, and a third subject distance. The target images TIc with small Sbd are displayed in order (in this case, in order from the left side). Similarly, for the target image groups Grp2 and Grp3, the display control unit 310 also includes a person (a person represented by the person image PI5 or a person image PI4) that is commonly included in the target images of each group. The subject images TI and the reference person are displayed side by side in order from the target image TI having the smallest subject distance Sbd.

  In the second print screen, each target image list Rst is arranged in a line in the left-right direction (horizontal direction), and each target image list Rst is arranged in the target image TI included in each target image list Rst. They are displayed side by side (stacked) in the depth direction perpendicular to the direction in which they are positioned. In addition to the print image, five buttons BN1 to BN3, 5, and 7 are displayed on the second print screen. When one (or a plurality of) target images TI are selected by the user and the print button BN3 is selected, a print process for printing the selected images is executed.

  FIG. 24 is an explanatory diagram illustrating a third print screen. When the display change button BN7 is selected by the user on the second print screen, the display control unit 310 causes the display unit 150 to display the third print screen illustrated in FIG. Specifically, as in the second print screen, the display control unit 310 sets the target image list Rst1 of the person to be searched (the person represented by the person image PI1) to the top (here, the leftmost). The target image list Rst with a person having a high degree of closeness Di with the person to be searched as a reference person is displayed in order (on the right side). In addition, the display control unit 310 displays the target image lists Rst in order from the group of target images TI that include images of faces of people with a high familiarity Di with the reference person. In addition, the display control unit 310 displays the target image groups Grp in the target image list Rst side by side in order from the target image TI having the smallest subject distance Sbd to the reference person.

  In the third print screen, the target image lists Rst are arranged in a line in the left-right direction (horizontal direction), and the target images TI included in the target image lists Rst are arranged in the respective target image lists Rst. Are displayed side by side in the up-down direction (vertical direction) which is a direction orthogonal to the current direction. In addition to the print image, five buttons BN1 to BN3, 5, and 8 are displayed on the third print screen. When one (or a plurality of) target images TI are selected by the user and the print button BN3 is selected, a print process for printing the selected images is executed. When the display change button BN8 is selected by the user, the display control unit 310 causes the display unit 150 to display the second print screen illustrated in FIG.

  In this embodiment, the person information corresponds to “personal characteristics” in the claims. In the present embodiment, the closeness Di corresponds to “closeness” in the claims. In the present embodiment, the internal memory 120 in which a plurality of interstitial determination information is stored corresponds to a “storage unit” in the claims.

  According to the printer 100 according to the first embodiment described above, the persons whose subject distances Sbd respectively estimated in the consecutive predetermined number (threshold Th2) of the target images TI are all equal to or smaller than the predetermined value (threshold Th1). Since the intimacy Di is calculated for, the efficiency of the process of specifying the relationship between people from the target image TI can be improved. Specifically, not only can the processing load for calculating the intimacy be avoided by not calculating the intimacy between persons whose inter-subject distance Sbd is greater than a predetermined value, but also the inter-subject distance Sbd in the target image TI that is less than the predetermined number. Since the intimacy is not calculated even for persons having a value equal to or less than a predetermined value, the processing load for calculating the intimacy can be suppressed, and the efficiency of the processing for calculating the intimacy can be improved.

  Techniques for calculating intimacy have been known for some time. However, for example, when the intimacy is to be calculated from a moving image or the like, the number of images serving as the target image TI is enormous, and thus an increase in the processing load for calculating the intimacy has been a problem. However, according to the present invention, since the closeness is calculated only for persons who satisfy a predetermined condition among all persons represented in the target image, it is possible to improve processing efficiency.

  In other words, the printer 100 is represented in at least a part of a plurality of target images arranged in time series, and the distance between subjects is less than or equal to a predetermined value. Can calculate intimacy, and can not calculate intimacy between people who appear less than a predetermined number in the target image. This is because, in the prior art, it is considered that intimacy is calculated for people whose subject distances are all equal to or smaller than a predetermined value. Therefore, the efficiency of processing for calculating intimacy is improved by the present invention. I understand that.

  According to the printer 100 according to the first embodiment, the degree of intimacy between people is determined using the inter-subject distance Sbd that is the actual distance between the faces represented in the face area FA of the target image TI. Since the closeness Di to be expressed is calculated, it is possible to improve the accuracy of specifying the relationship between people from the target image TI. Specifically, by calculating the actual distance between people at the time of taking a photo, the closer to each other, the closer to each other at the time of taking a photo, and the case of people who are not intimate Since the closeness Di can be calculated using the characteristics at the time of shooting that the state of being close to each other is small, the accuracy of the closeness Di can be increased.

  A technique for calculating intimacy from the distance between facial images represented in the target image TI has been known. However, when the intimacy is calculated using the distance between the face images represented in the target image TI, the distance between people in the depth direction of the target image TI is not reflected in the intimacy. However, even if the distance between the persons is long, if the distance between the face images represented in the target image TI is short, the closeness may be calculated to be high. However, according to the present invention, since the distance of the depth method of the target image TI can be reflected in the closeness Di, the degree of intimacy between people can be accurately reflected in the closeness Di.

  That is, the distance between the face images included in the first target image is smaller than the distance between the face images included in the second target image, and the face represented in the first target image. When the actual distance between the faces is larger than the actual distance between the faces represented in the second target image, the printer 100 determines that the second target image is more than the people represented in the first target image. The intimacy of each person can be calculated so that the intimacy between the people represented is higher. This is represented in the second target image by the people represented in the first target image in the conventional technique for calculating the intimacy from the distance between the facial images represented in the target image TI. Since it is considered that the intimacy of each person is calculated so that the intimacy becomes smaller, it is understood that the present invention improves the accuracy of calculating the intimacy.

  According to the printer 100 according to the first embodiment, the subject distance estimation unit estimates the subject distance based on the size of the face area with respect to the size of the target image, and thus the face represented in the face area. The subject distance, which is the distance from the camera to the photographing apparatus, can be easily estimated. Specifically, as shown in Expression (3), the subject distance estimation unit can easily estimate the subject distance by using the entire image width Wwi and the width Wfi of the face area FA in the target image OI. .

  According to the printer 100 according to the first embodiment, since the relationship between the persons represented in the target image is determined using the closeness Di and the characteristics of the person, the relationship between the persons from the target image is determined. The accuracy of specifying can be improved. Specifically, in order to specify the relationship representing the relationship between people, the parent calculated using the inter-subject distance Sbd, which is the actual distance between the faces represented in the face area FA of the target image TI. Since the human characteristic determined from the target image TI is used in addition to the density Di, different patterns can be determined even if the familiarity is the same.

  According to the printer 100 according to the first embodiment, since the relationship between persons represented in the target image is determined using the closeness and the age of the person, the relationship between the persons is determined from the target image. The accuracy of identification can be improved. Specifically, in order to specify the relationship representing the relationship between people, in addition to the familiarity Di, the age of the person determined from the target image TI is used, so even if the familiarity is the same, the parent or child or friend It is possible to discriminate specific differences such as. Further, since the relationship between the people represented in the target image is determined using the closeness and the sex of the person, it is possible to improve the accuracy of specifying the relationship between the people from the target image. Specifically, in order to specify the relationship representing the relationship between people, in addition to the familiarity Di, the sex of the person determined from the target image TI is used. It is possible to discriminate specific differences such as whether or not a friend.

  According to the printer 100 according to the first embodiment, since the relationship determination information selected by the relationship determination information selection unit is used to determine the relationship between the people represented in the target image, The accuracy of specifying the relationship can be improved. Specifically, even if the positional relationship, gender, age, and other characteristics of people are the same, there is a possibility that the personality of people differs depending on the location where the image was taken. However, according to the present embodiment, since the relationship between people is determined using the relationship determination information corresponding to the place where the image was taken, it is possible to improve the accuracy of specifying the relationship between people from the target image TI. .

  According to the printer 100 according to the first embodiment, since the interstitial relationship diagram representing the interstitial relationship of all the persons whose intimacy has been calculated is created, the user can easily display the interpersonal relationships represented in the target image. I can grasp it. In addition, the user can identify a desired target image from among a plurality of target images by arbitrarily selecting a relationship or a person image shown in the relationship relationship diagram.

  In addition, the interstitial relationship diagram shows a human face image appearing in the target image and a relationship line connecting the human images, and the higher the intimacy between people, the smaller the distance between the face images. Since the frequency of the relationship line increases as people appear more frequently, the relationship of all the people whose intimacy is calculated can be easily displayed. Specifically, in the interpersonal relationship diagram RM, the person image frames Fi of the persons are set closer to each other as the closeness Di is greater. Thereby, the user can grasp | ascertain the degree of closeness of persons easily through vision. Moreover, since the relationship line Rl which connects the person image frames Fi of the persons becomes thicker as the frequency of appearing in the target image TI increases, the user can easily grasp the reliability of the relationship line Rl. . In other words, if the frequency of appearance in the target image is low, if there are target images that are close to each other by chance even though they are non-intimate people, the effect that the target image has on the intimacy Becomes larger. On the other hand, if people who appear frequently in the target image are people who are not intimate, even if some of the target images are close to each other, many other mutual distances Since there are target images that are far from each other, the influence of the target images that are close to each other on the closeness can be suppressed. Therefore, it can be said that the thicker the relationship line Rl, the higher the reliability of the relationship line Rl.

  According to the printer 100 according to the first embodiment, the distance between the face images included in the first target image is smaller than the distance between the face images included in the second target image, When the actual distance between the faces represented in the first target image is larger than the actual distance between the faces represented in the second target image, it appears in the first target image in the relationship diagram. The interstitial relationship diagram can be created so that the distance between the human face images appearing in the second target image is smaller than the distance between the human face images. This is because, when a related relationship diagram is created using a conventional technique for calculating intimacy from the distance between face images represented in the target image TI, the face of a person appearing in the first target image is displayed. Since it is considered that the interstitial relationship diagram is created so that the distance between the human face images appearing in the second target image is larger than the distance between the images, the present invention improves the accuracy of the interstitial relationship diagram. You can see that

  According to the printer 100 according to the first embodiment, the searched target images are displayed side by side in order from the target image group including the face image of the person having a higher familiarity with the person to be searched. Therefore, the image search by the user can be facilitated. Specifically, as illustrated in FIG. 23, the printer 100 includes a target image TI that includes an image of a person's face having a higher closeness Di with the person to be searched in the target image list Rst. The image groups are displayed in order from the image group Grp. More specifically, the printer 100 includes the target image group Grp1 including the target image TI including the person image PI of the person to be searched and the person having the highest familiarity Di, the person to be searched and the second person. Includes a target image group Grp2 including a target image TI including a person image PI of a person having a high degree of closeness Di, and a person image PI of a person to be searched and a person having the third closest degree of closeness Di. The target image TI is displayed in the order of the target image group Grp3 including the target image TI. Accordingly, the user can check the target image TI in order from the target image TI including the person image PI of the person having a high familiarity Di with the person to be searched, thereby facilitating the image search. be able to.

  According to the printer 100 according to the first embodiment, in each target image group constituting the target image list, the distance between subjects between the person included in the target image for each target image group and the person to be searched for. Since the target images are displayed side by side in order of increasing Sbd, the user can easily search for images. Specifically, as illustrated in FIG. 23, the printer 100 includes, in each target image group Grp in the target image list Rst, a person who is commonly included in the target images of the group and a person (reference person) to be searched. Are displayed side by side in order from the target image TI with the smallest subject distance Sbd. More specifically, in the target image group Grp1 of the target image list Rst1, the printer 100 includes a person (a person represented by the person image PI2) that is commonly included in the target images of the group and a reference person (a person image PI1). Are displayed side by side in order from the target image TI having the smallest subject-to-subject distance Sbd. In other words, among the three target images TIa, TIb, and TIc, the printer 100 has the target image TIa having the smallest subject distance Sbd, the second target image TIb having the smallest subject distance Sbd, and the third subject distance Sbd. The small target images TIc are displayed in order (in this case, in order from the left side). As a result, the user can check the target images TI including the same person in order from the target image TI having the smallest inter-subject distance Sbd, thereby facilitating the image search.

  According to the printer 100 according to the first embodiment, a target image including a face image of a person to be searched and a person having a predetermined familiarity is searched from the target image, and the searched target image is searched. Since a target image list in which target images are arranged in order from a target image group including a face image of a person with a higher familiarity with a person having a predetermined familiarity is displayed, image search by a user can be facilitated. it can. Specifically, the printer 100 not only searches for a target image that includes an image of a person's face to be searched, but also includes an image of a person's face having a predetermined familiarity with the person to be searched. A search is also performed for the included target images, and the target images are displayed side by side in order from a target image group including a face image of a person with a higher familiarity with a person having a predetermined familiarity. . Accordingly, the user can confirm the target image TI in order from the target image TI including the person image PI of the person having a high degree of closeness Di with the person having the predetermined closeness. Can be. The person having the predetermined familiarity Di is, for example, a person who has the highest familiarity Di with the person to be searched, or a person whose familiarity Di with the person to be searched is a set value or more. And so on.

  According to the printer 100 according to the first embodiment, the target images included in the target image list Rst are arranged in a line in the first direction, and the plurality of target image lists Rst are orthogonal to the first direction. Since the images are displayed side by side, the user can easily search for images. Specifically, as shown in FIGS. 22 and 24, the target image list Rs to which each displayed target image belongs is visually determined by orthogonalizing the direction in which the target images are arranged and the direction in which the target image list is arranged. Since it can be easily determined, image search by the user can be facilitated.

  According to the printer 100 according to the first embodiment, a plurality of target image lists Rst are displayed side by side in order from the target image list Rst of a person with a higher familiarity with the person to be searched, so that the image search by the user Can be made easier. Specifically, as shown in FIG. 22 and FIG. 24, image search by the user can be performed by displaying the target image list Rst arranged in order from the target image list Rst with a person having a higher closeness to the person to be searched as a reference person. Can be easily. For example, if a person who is a search target wants to search for an image of a friend or child who is close to him / her, he / she can set himself / herself as a search target without setting the friend or child as a search target. Thus, the target image list for friends and children with high intimacy can be displayed in the order of intimacy with themselves, so that a desired image can be easily acquired.

B. Second embodiment:
FIG. 25 is a flowchart showing the flow of person information setting processing in the second embodiment. In the first example, in the person information setting process (FIG. 8) in step S130 (FIG. 2) of the image search / print process, the age determination / sex determination is performed for persons whose frequency n is greater than the threshold Th2. For those who have not, age and sex have been determined by the feature discriminating unit 250 (step S270). In this example, as shown in FIG. 25, the frequency n is greater than the threshold Th2. There will be described an embodiment that is performed in the relationship determination process in step S290 without immediately determining the age and sex even if the person has not been subjected to age determination / gender determination. Steps S200 to S280 are the same as those in the first embodiment, and a description thereof will be omitted.

  FIG. 26 is an explanatory diagram for explaining the contents of the third interstitial determination information. FIG. 26 exemplifies the content of the relationship determination information corresponding to the image taken on the street, which is one of the relationship determination information stored in the internal memory 120 as in the first embodiment. That is, the contents of the relationship determination information shown in FIG. 26 correspond to the contents of the relationship determination information shown in FIG. Since the interstitial determination process described in the second embodiment is the same as that of the first embodiment except for the contents of the interstitial determination information, the description thereof is omitted.

  A case where the interstitial determination information selection unit 281 selects interstitial determination information whose contents are shown in FIG. 26 will be described. Steps S500 and S505 are the same as in the first embodiment. When the familiarity Di is larger than the threshold Th3 (step S500: YES), the feature determination unit 250 determines whether or not age determination has already been performed for each person for whom the familiarity Di has been calculated (step S507). If the age determination is not performed for each person whose intimacy Di is calculated (step S507: NO), the feature determination unit 250 determines the age of each person (step S510). The age determination method is the same as in the first embodiment. The feature determination unit 250 skips age determination when age determination has already been performed for each person whose intimacy Di has been calculated (step S507: YES). Thereafter, the interstitial determination unit 280 performs Steps S515 to S525. Steps S515 to S525 are the same as in the first embodiment.

  In step S520, if the age of the person who is the subject of the relationship determination is greater than 12 (step S520: YES), the feature determination unit 250 has already determined gender for each person whose intimacy Di has been calculated. Is determined (step S527). If the gender determination is not performed for each person whose intimacy Di is calculated (step S527: NO), the feature determination unit 250 performs the gender determination for each person (step S530). The gender determination method is the same as in the first embodiment. The feature determination unit 250 skips gender determination when sex determination has already been performed for each person whose intimacy Di has been calculated (step S527: YES). Thereafter, the interstitial determination unit 280 performs Steps S535 to S545. The interstitial determination process ends as described above. In addition, the personal information setting process is terminated by the termination of the relationship determination process.

  According to the printer 100 according to the second embodiment described above, since it is possible to determine a relationship without performing at least one of age determination and gender determination, it is possible to improve processing efficiency of image search / print processing. Can be planned. Specifically, in the above step S500, when the familiarity Di is equal to or less than the threshold Th3 (step S500: NO), the persons who are the targets of the relationship determination are “others” without performing age determination or gender determination. Can be determined. In step S515, when the age difference is greater than or equal to the threshold Th4 (step S515: NO), or in step S520, the age of at least one of the persons who are subject to the relationship determination is 12 or less (step S520: NO), it is possible to determine the relationship between persons who are the targets of the relationship determination without performing the gender determination. In addition, by performing sex determination after performing age determination, it is possible to efficiently determine the relationship between persons who are targets of relationship determination. This is thought to be because the difference in age has a greater influence on the determination of personal relationships than the difference in gender.

C. Variation:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

C1. Modification 1:
The printer 100 according to the present embodiment includes a feature determination unit 250 that determines human characteristics such as age and gender from a target image, a relationship determination unit 280 that determines a relationship between people, and a relationship that creates a relationship relationship diagram RM. Although the relationship diagram creation unit 290 is provided, the printer 100 can be realized even in an aspect in which at least some of these are not provided. That is, even if it does not have the above-described configuration, it is possible to calculate intimacy for persons whose distances between subjects estimated in a predetermined number of consecutive target images are all equal to or less than a predetermined value. It is possible to improve the accuracy of specifying the relationship between each other.

C2. Modification 2:
The printer 100 according to the present embodiment performs image search / print processing for a moving image file that employs the Motion JPEG compression format. The compression format of the moving image file that the printer 100 performs image search / print processing is based on this. For example, a moving image file adopting a compression format such as MPEG may be used. In this case, the image for each frame calculated using the difference data is the target image.

C3. Modification 3:
Although the printer 100 according to the present embodiment performs processing on a moving image file including a predetermined number of target images, the printer 100 may perform processing sequentially on sequentially input image files. Good. Specifically, when image data is sequentially input from a video camera or the like, the input image data may be sequentially set as the attention image OI, and the face area FA may be detected or the person information setting process may be performed. At this time, whether or not the input target image is the last image may be determined by an input from the user.

C4. Modification 4:
In step S230 of the person information setting process (FIG. 8), the printer 100 according to the present embodiment determines whether each person whose inter-subject distance Sbd is calculated is a person represented in the previous target image. However, it may be an aspect in which it is determined only whether or not the person is represented in the previous target image. In this case, since the frequency n is not incremented and n = 1 is set again even for a person represented in the previous target image two times before, a predetermined number of target images that are continuous in time are used. The intimacy can be determined only for persons whose estimated subject distances Sbd are all equal to or less than a predetermined value.

C5. Modification 5:
The printer 100 according to the present embodiment displays a print screen as illustrated in FIGS. 22 and 24 on the display unit 150 and prints a target image selected by the user, but the printer 100 is electrically connected. In this case, a print screen as shown in FIGS. 22 and 24 may be displayed on the monitor, and the display control unit may be realized by a CPU of the computer. Further, the printer 100 may print a target image as shown in FIG. 22 or FIG.

C6. Modification 6:
22 and 24, the printer 100 according to this embodiment sets the target image list Rst1 of the person to be searched at the top (here, the leftmost side), and is the parent of the person to be searched. The target image list Rst with the higher density Di as the reference person is displayed in order (on the right side). The target image list Rst1 of the person to be searched is placed on the rightmost side, and the person to be searched May be displayed side by side on the left side in order from the target image list Rst using a person with a higher familiarity Di as a reference person. Further, the target image list Rst1 of the person to be searched is placed on the uppermost side, and the target image list Rst with a person having a higher closeness Di with the person to be searched as a reference person is displayed in order from the lower side. Also good.

C7. Modification 7:
The printer 100 according to the present embodiment includes a person relationship PI RM, a person image PI, a person image frame Fi formed on the outer periphery of the person image PI, the age information ag of the person represented by the person image PI, and Although the gender information Ss and the relationship line Rl that connects the person image frames Fi are created, the interstitial relationship diagram RM may be in a form in which some of these are not present. In the present embodiment, the relationship line width Wr is determined using the frequency n set in the person information database HC, and the length Lr of the relationship line Rl is determined using the familiarity Di. However, these are examples, and other elements constituting the interpersonal relationship diagram RM may be changed based on information such as the person information database HC. For example, the size of the person image PI may be changed depending on the frequency of appearance in the target image TI.

C8. Modification 8:
Although the printer 100 according to the present embodiment determines the relationship using the relationship determination information, the content of the relationship determination information shown in the present embodiment is an example, and the relationship is determined based on information other than these. May be. Further, in this embodiment, the interstitial determination information is determined using the interstitial determination information set for each place where the target image is photographed, but one interstitial determination information is used regardless of the place where the target image is photographed. It is also possible to determine the relationship.

C9. Modification 9:
The printer 100 according to the present embodiment specifies a person to be searched by selecting a relation line Rl by the user or inputting a keyword or the like in the input box Bo. Other methods may be used. For example, the printer 100 searches for a person represented by a person image PI that is most similar to the query image by using an image of an arbitrary face set by the user as a query image and performing face authentication using the query image. It may be set as a target person.
C10. Modification 10:
The printer 100 according to the present embodiment selects the interstitial determination information using the position information of the target image TI set as the target image OI, but selects the interstitial determination information using other positional information. May be. For example, the printer 100 selects the relationship determination information using the position information of the target image TI for which the inter-subject distance Sbd having the smallest value is calculated among the inter-subject distances Sbd used for calculating the familiarity Di. Also good.

C11. Modification 11:
In the printer 100 according to the present embodiment, a part of the configuration realized by hardware in the above embodiment may be replaced with software, and conversely, a part of the configuration realized by software is replaced with hardware. It may be replaced with wear.

C12. Modification 12:
The present invention can be realized by a digital still camera, a personal computer, a digital video camera, or the like in addition to a printer. Further, an image processing method and apparatus, an affinity calculation method and apparatus, a relationship relationship display method and apparatus, a computer program for realizing the functions of these methods or apparatuses, a recording medium on which the computer program is recorded, and the computer program It can be realized in the form of a data signal embodied in a carrier wave.

100 ... Printer 110 ... CPU
DESCRIPTION OF SYMBOLS 120 ... Internal memory 140 ... Operation part 150 ... Display part 160 ... Printing mechanism 170 ... Card interface 172 ... Card slot 200 ... Image processing part 210 ... Face area detection part 220 ... Subject distance estimation part 230 ... Inter-subject distance estimation part 240 ... Intimacy calculation unit 250 ... feature determination unit 260 ... specific information acquisition unit 270 ... image search unit 280 ... interstitial determination unit 281 ... interstitial determination information selection unit 290 ... interstitial relationship diagram creation unit 310 ... display control unit 320 ... print control unit MC ... Memory card HC ... Personal information database RM ... Interest relationship diagram Rst ... Target image list Grp ... Target image group FA ... Face area TA ... Head area OI ... Attention image TI ... Target image Di ... Intimacy

Claims (14)

An image processing apparatus that processes a plurality of target images arranged in time series,
A face area detection unit that detects a face area that is an image area including a face image from each target image for each face image;
A subject distance estimation unit that estimates a subject distance, which is a distance from the subject's face to the imaging device, using the face region;
An inter-subject distance estimation unit that estimates an inter-subject distance, which is an actual distance between human faces represented in the face area, using the subject distance;
A degree of intimacy representing a degree of intimacy between people using the estimated distance between subjects for each of the subjects whose distance between the subjects estimated from a predetermined number of consecutive target images is not more than a predetermined value. An intimacy calculating unit that calculates a larger value as the distance between the subjects is smaller. The image processing apparatus according to claim 1.
The intimacy calculation unit may be configured such that, even if the subject-to-subject distance is not estimated in the target images that are next to the target image in which the subject-to-subject distance has been estimated, When the inter-subject distance is estimated in the target image that is next to the target image for which the inter-subject distance has not been estimated, the total of the target images in which the inter-subject distance is estimated is a predetermined number, and is estimated. An image processing apparatus that calculates the intimacy between the persons if the distances between the subjects are all equal to or less than a predetermined value. The image processing apparatus according to claim 1 or 2, further comprising:
A feature discriminating unit that discriminates human characteristics from each person's face image for which the intimacy calculation unit calculates
An image processing apparatus comprising: a relationship determination unit that determines a relationship between people whose intimacy is calculated using the affinity and the characteristics of the person. The image processing apparatus according to claim 3 further includes:
The feature determination unit determines, for each person whose intimacy is calculated, whether or not the person has already determined the characteristics of the person, and performs the determination only for a person who has not determined the characteristics of the person. Image processing device. The image processing apparatus according to claim 3 or 4,
The feature determination unit is an image processing apparatus that determines a person's age from a face image included in the face region. The image processing apparatus according to any one of claims 3 to 5,
The feature determination unit is an image processing apparatus that determines a gender of a person from an image of a face included in the face area. The image processing apparatus according to any one of claims 3 to 6,
The interstitial determination unit
A storage unit storing two or more relationship determination information indicating a correspondence relationship between the intimacy and the characteristics of the person and the relationship between the persons;
Based on information on the location where the target image was taken, a interstitial determination information selection unit that selects one interstitial determination information from the storage unit,
An image processing apparatus that determines a relationship between persons whose intimacy has been calculated using the relationship determination information selected by the relationship determination information selection unit. The image processing apparatus according to claim 7.
The information regarding the place where the target image was taken is an image processing device that is specified using GPS information. The image processing apparatus according to any one of claims 3 to 8, further comprising:
An image processing apparatus comprising a interstitial relationship diagram creating unit that creates a interstitial relationship diagram representing the interstitial relationship of all of the persons whose intimacy has been calculated by the intimacy calculating unit from the interpersonal relationships determined by the interstitial determining unit . The image processing apparatus according to claim 9.
The interstitial relationship diagram represents an image of a person's face for which the familiarity is calculated, and a relationship line connecting the face images.
The image processing apparatus in which the distance between the face images is smaller as the degree of intimacy is higher, and the width of the relationship line is larger as the frequency of appearing in the target image is higher. The image processing apparatus according to any one of claims 1 to 10,
The subject distance estimation unit is an image processing apparatus that estimates the subject distance based on a size of the face area with respect to a size of the target image. The image processing apparatus according to any one of claims 1 to 11,
The intimacy calculation unit is represented in at least a part of a plurality of target images arranged in time series, and the frequency of appearing in the target image is greater than or equal to a predetermined number of persons whose subject distances are all equal to or less than a predetermined value. An image processing apparatus that calculates the intimacy for each other and does not calculate the intimacy for people who appear less than a predetermined number in the target image. The image processing apparatus according to any one of claims 1 to 11,
The closeness calculating unit is configured such that the distance between the face images included in the first target image is smaller than the distance between the face images included in the second target image, and When the actual distance between the faces represented is larger than the actual distance between the faces represented in the second target image, the second target image is more than the people represented in the first target image. An image processing apparatus that calculates intimacy so that the intimacy between the people represented in the above is higher. The image processing apparatus according to claim 10.
The interstitial relationship diagram creating unit is configured such that the distance between the face images included in the first target image is smaller than the distance between the face images included in the second target image, and the first target image Is larger than the actual distance between the faces represented in the second target image, in the relationship diagram, the human face appearing in the first target image is displayed. An image processing apparatus that creates the interstitial relationship diagram so that a distance between human face images appearing in a second target image is smaller than a distance between images.

Images