JP2014174782A - Image processing device, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately identify a person in an image in relation to a subject executing processing to the image.SOLUTION: A name database d1 stores for example "a name" and "a face image and a feature" as person information for each of a plurality of persons (e.g. relatives). A person relationship database d2 stores a relationship (a relative structure) between the persons (the relatives). A control section 1 refers to the name database d1, recognizes the person on the basis of a face of the person in the image, and identifies a person relationship to the recognized person on the basis of the person relationship database d2 from a perspective of a person who is the subject executing the processing to the image (e.g. the subject seeing the image).

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for processing an image.

  In general, an imaging device such as a digital camera acquires date / time information of a shooting date and position information of a shooting location at the time of shooting, and adds and stores the information as Exif (Exchangeable Image File Format) information of the image file. I am doing so. Further, conventionally, by recognizing a person's face in the captured image, identifying the person from the face, and associating the person's name and age with the person in the captured image, A technique is disclosed in which the name and age are displayed in association with each person (see Patent Document 1).

JP 2009-110147 A

  However, even if the technique of the above-mentioned patent document displays the name and age in association with each person in the image, even if it is possible to know who is the image taken at what age, Regardless of whether the person viewing the image is a relative or another person, displaying the additional information uniformly is formal and lacks interest, is not understood, or is not easy to get to know , Etc. problems remain. This has the same problem when performing some processing based on the additional information in addition to viewing the image.

  An object of the present invention is to enable appropriate identification of a person in an image in relation to a subject of processing for the image.

In order to solve the above-described problems, the image processing apparatus of the present invention provides:
Person storage means for storing a correlation between a plurality of persons;
Recognition means for recognizing the person based on the face of the person in the image;
A determination means for determining a person who is a subject of processing for an image;
Specifying means for specifying a person relationship with the person recognized by the recognition means based on the correlation of the plurality of persons, with the subject person determined by the determination means as a viewpoint;
An image processing apparatus comprising:

In order to solve the above-described problem, the image processing method of the present invention
Recognizing the person based on the person's face in the image;
Specifying a person relationship with the recognized person based on a correlation between a plurality of persons, with a person who is a subject of processing on an image as a viewpoint;
The image processing method characterized by including.

In order to solve the above-described problems, the program of the present invention
Against the computer,
The ability to recognize a person based on the person's face in the image,
A function to determine the person who will be the subject of processing for the image;
A function of specifying a person relationship with the recognized person based on a correlation between a plurality of persons, with the determined subject person as a viewpoint;
It is a program for realizing.

  According to the present invention, it is possible to appropriately identify a person in an image in relation to the subject of processing for the image.

The block diagram which showed the basic component of the digital camera applied as an image processing apparatus. The figure for demonstrating the name database d1. The figure for demonstrating the person relationship database d2. The figure for demonstrating the related analysis basic database d3. In (1) and (2), the information (relationship) indicating what kind of person each person in the image has is related to each person from the viewpoint of the subject of processing for the image at the time of image reproduction. The figure which illustrated the case where additional display (identification display) was carried out. The flowchart which showed the registration process to the name database d1 and the person relationship database d2. 10 is a flowchart that starts execution when a person relationship additional display function is designated in a playback mode. 8 is a flowchart following the operation of FIG. 8 is a flowchart for explaining in detail the subject determination process (step B3 in FIG. 7). (1) is a diagram illustrating a person relationship according to the subject (A or F), and (2) and (3) are character strings (according to the person relationship in the case of the subject (A or C)) ( The figure which illustrated the connection). (1), (2) is the figure which showed the specific example in the case of adding and displaying a person relationship.

Hereinafter, embodiments of the present invention will be described with reference to FIGS.
This embodiment exemplifies a case where the present invention is applied to a digital camera as an image processing apparatus. FIG. 1 is a block diagram showing basic components of the image processing apparatus (digital camera).
This digital camera is a digital compact camera that can shoot moving images in addition to still images. It has an image capturing function for capturing images, a time counting function for measuring the current date and time and timer time, and recorded and saved images ( In addition to an image reproduction function for arbitrarily reading out and reproducing (saved images) and reproduction, a face recognition function for recognizing a person from the face of the person in the image, etc., in this embodiment, a person relationship additional display function is provided.

  This face recognition function is a function for identifying a person by analyzing the image and recognizing the person's face. Such a face recognition function is a technique commonly used in cameras, Since the known technique is used in the embodiment, the description thereof is omitted. The person relationship addition display function is information indicating how each person in the image has a relationship from the viewpoint of the subject of the processing of the image (in this embodiment, the subject of viewing the image) during playback of the captured image ( For example, in the case of a relative, this is a function of additionally displaying (identifying display) a character string indicating a relationship) in association with each person in the image.

  The control unit 1 operates by supplying power from the power source unit (secondary battery) 2 and controls the overall operation of the digital camera in accordance with various programs in the storage unit 3. A CPU (Central Processing Unit) and a memory (not shown) are provided. The storage unit 3 includes, for example, a ROM, a flash memory, and the like, and stores programs and various applications for realizing the present embodiment in accordance with operation procedures shown in FIGS. A program memory m1, a work memory m2 for temporarily storing various kinds of information (for example, flags) necessary for the operation of the digital camera, an image memory m3 for recording and saving captured images, It has a name database d1, a person relation database d2, and a related analysis basic database d3, which will be described later. The storage unit 3 may include a removable portable memory (recording medium) such as an SD card or an IC card, and is connected to a network via a communication function (not shown). In the state, a storage area on a predetermined server device side may be included.

  Although not shown, the operation unit 4 includes a mode change key for arbitrarily switching between a shooting mode for enabling shooting and a playback mode for reproducing a shot image, a shutter key for instructing start of shooting, a shooting parameter setting key, and the like. I have. The control unit 1 performs, for example, a mode change process, a shooting process, a shooting condition setting process, and the like according to an input operation signal output from the operation unit 4 corresponding to the operation key. The display unit 5 is a high-definition liquid crystal display or an organic EL (Electro Luminescence) display, and serves as a monitor screen that displays a live view image or a playback screen that reproduces a captured image. In addition, you may make it comprise the touch display part provided with the touch panel and the display panel by the operation part 4 and the display part 5. FIG.

  The imaging unit 6 is capable of taking a high-definition still image by imaging a subject image from an optical lens on an imaging device (CCD, CMOS, etc.). And various sensors, analog processing units, and digital processing units. The image captured by the imaging unit 6 is recorded and stored in the storage unit 3 after being subjected to image compression processing for compression (reversible compression) by reducing the size of the image to form a file.

FIG. 2 is a diagram for explaining the name database d1 used in the person relationship addition display function.
The name database d1 stores personal information (person information) corresponding to each of a plurality of persons (each person constituting a group), and “ID”, “name”, “age”, “face image” Each item of “feature data” has contents that are arbitrarily input and registered in advance from the operation unit 4 by a user operation. Here, in the illustrated example, personal information (person information) of each person constituting a plurality of persons (relatives) is stored, and “ID” is person identification information, and in the illustrated example, “A”. , “B”,... Are registered as person IDs.

  “Name” is the name (real name) of the person. “Age” is the current actual age calculated based on the birth date information input in advance from the operation unit 4, and the control unit 1 periodically updates the “age” (for example, every day ) To run. The “face image / feature” is face feature information obtained by analyzing the face image together with the person's face image, and is basic information referred to for collation during face recognition. Note that the control unit 1 tries to collate again with the face image when the person cannot be recognized as a result of the collation with the feature information first of the “face image / feature”.

FIG. 3 is a diagram for explaining the person relation database d2 used in the person relation additional display function.
The person relation database d2 defines a correlation between a plurality of persons stored in the name database d1 described above, and the contents are arbitrarily input and registered in advance by a user operation. In the example shown in the figure, relatives are exemplified as a plurality of persons, and it is arbitrary what data structure is used to define the correlation (configuration of relatives) of the relatives. FIGS. 3 (1) to (3) show correlation diagrams showing the correlation of relatives in a tree structure (hierarchical structure), and the control unit 1 uses a person who is the subject of viewing an image as a viewpoint, The relationship (person relationship: relationship) with a person is specified based on the correlation (relative structure) of a plurality of persons (relatives) defined in the personal relationship database d2. In the figure, a black circle indicates the main person (A) (the same applies hereinafter).

  FIG. 3 (1) shows that the person (ID: B, female) is the wife of the person (A), and the person (ID: C, male) is the person (ID: A, male) as the viewpoint. It is a correlation diagram showing that the son of the person (A) and the person (ID: D, woman) are the sisters of the person (C). Similarly, in FIG. 3 (2), the person (C) is the son of the person (B) and the person (D) is the younger sister of the person (C) when the person B (female) serving as a reference is viewed. It is the correlation diagram which showed that. FIG. 3 (3) shows that the person (B) is the wife of the person (A), the person (C) is the son of the person (A), and the person (D) when the reference person A (male) is the viewpoint. Is a correlation diagram showing that the younger sister of the person (C) and the person (E) is the younger brother of the person (D).

FIG. 4 is a diagram for explaining the relation analysis basic database d3 used in the person relation addition display function.
The related analysis basic database d3 is a database that stores general definitions of relationships called in place of the “name” of each person in the name database d1, and is a reference person among a plurality of persons (relatives) in the name database d1. A character string (information indicating a person relationship) indicating a relationship (relationship) with another person is stored. In other words, because the name of the relationship is different depending on who the standard person is, for example, “the reverse of the wife is the husband”, “the reverse of the son is the father”,…, “the father of the wife is the father-in-law”,…, "My father's wife is my mother", "My father's father is my grandfather" ...

  Also, from the same viewpoint, for example, “Father” = “Dad”, “Grandfather” = “Grandpa”, etc. are stored so that the relationship name (expression format) varies depending on the age of the subject. (Not shown). For example, in the case of a person under the age of 18, a character string is stored so that the expression is easy to understand or familiar to the person in relation to the subject. When the control unit 1 specifies what kind of person relationship each person in the image has from the viewpoint of the subject who views the image during image reproduction by referring to the person relation database d2, the correlation A character string indicating a relationship according to the relationship is read from the related analysis basic database d3.

FIG. 5 is a diagram showing a display example when the person relationship addition display function is executed.
In other words, from the viewpoint of the subject who views the image during image reproduction, information indicating what kind of person relationship (relationship) each person in the image is associated with each person is additionally displayed (identified display) FIG. The control unit 1 determines the person who will be the subject to view the image during image reproduction. At that time, the control unit 1 determines the person who is designated to be input by the user operation as the subject, or the imaging unit 6 identifies the user himself / herself. A person in the captured image is determined as a subject.

  FIG. 5A is a display example in the case where the subject is a person (A), in which “wife” is additionally displayed as a relationship of the person (B) and “son” is additionally displayed as a relationship of the person (C). In this case, the additional display (identification display) of the continuation pattern is such that the character string of the continuation pattern is arranged in the background rectangular frame and this rectangular frame is synthesized in the image. The case where it synthetically arranges in the body is illustrated. FIG. 5B is a display example in the case where the subject is a person (C), in which “mother” is additionally displayed as a relationship of the person (B), and “I” is additionally displayed as a relationship of the person (C). In this case, as in FIG. 5 (1), the additional display (identification display) of the continuation pattern is such that a rectangular frame is synthesized in the image, but the whole person (whole body) is avoided and it corresponds to that person. The case where it adjoins and is arranged is illustrated.

  In the display examples of FIGS. 5 (1) and 5 (2), the background color (background color) of the rectangular frame is unified to a predetermined color such as white or black. Moreover, although the case where the background rectangular frame is arranged is shown in the above-described example, a character string indicating a related pattern may be directly synthesized in the image, or the character string indicating the related pattern is directly arranged on the image. In this case, the color of the character string is determined according to the contrast with the image color at the arrangement position. As described above, in the present embodiment, the identification display in which the character string of the related pattern is arranged in the rectangular frame for background is performed, or the identification display in which the character string is directly combined in the image without using the rectangular frame is performed. Can be specified arbitrarily. In addition, when identification display using a rectangular frame is designated, it is further possible to arbitrarily designate whether the rectangular frame is to be synthesized and arranged on a body other than the face or at a position other than the body. .

  As described above, in this embodiment, the image processing apparatus (digital camera) includes a person storage unit (person relation database d2) for storing a correlation (configuration of relatives) of a plurality of persons (relatives), and a person's face in the image. Based on the recognition means (control unit 1, name database d1) and determination means (control unit 1, operation unit 4 or imaging) that determines the person who will be the subject of processing on the image (subject to watch the image) 6) and a specifying unit (control unit) for specifying a person relationship (relationship) with the person recognized by the recognition unit based on the correlation of a plurality of persons with the subject person determined by the determination unit as a viewpoint 1 and a person relation database d2).

  Next, the operation concept of the image processing apparatus (digital camera) in the present embodiment will be described with reference to the flowcharts shown in FIGS. Here, each function described in these flowcharts is stored in the form of a readable program code, and operations according to the program code are sequentially executed. In addition, the operation according to the above-described program code transmitted via a transmission medium such as a network can be sequentially executed. In other words, in addition to the recording medium, an operation unique to the present embodiment can be executed using a program / data supplied externally via a transmission medium. 6 to 9 are flowcharts showing an outline of the operation of the characteristic part of the present embodiment among the entire operations of the camera. When the flow is left from any of the flows of FIGS. Return to the main flow of operation (not shown).

FIG. 6 is a flowchart showing person registration processing in the name database d1 and the person relation database d2, and is started when an item “person registration processing” is selected and specified from the menu screen.
First, the control unit 1 obtains an image obtained by photographing a person (relative) from the imaging unit 6 (step A1), records the image as a file, and stores it in an image memory m3 (for example, an SD card) (step A2). After that, the image is analyzed, the face of any one person is specified as a processing target and the face image is extracted, and the face image is further analyzed to obtain feature data such as an outline, eyes, nose, and mouth. Detect (step A3).

  Then, referring to the “face image / feature” in the name database d1 based on the detected face image and feature data, basically the same (featurely identical) “face image / feature” is registered. It is checked whether it has been registered (step A4). If the “face image / feature” has already been registered (YES in step A4), the content of the “face image / feature” is updated by overwriting the newly detected face image / feature data this time (step After performing A5), it is checked whether there are other unprocessed faces in the image (step A11). If other unprocessed faces remain (YES in step A11), the process proceeds to step A3 described above. Returning, the face image is extracted and analyzed.

  If “face image / feature” has not been registered (NO in step A4), a screen for inquiring whether or not a new registration has been made is displayed (step A6). In this state, it is checked whether a new registration is instructed by a user operation (step A7). If no new registration is instructed (NO in step A7), the process proceeds to step A11 described above, but when a new registration is instructed ( YES in step A7), the face image and feature data detected this time are additionally registered in the name database d1 (step A8).

  Further, when the name and date of birth are input from the operation unit 4 as personal information (person information) of the person (step A9), the control unit 1 calculates the age from the date of birth, and the name database It is additionally registered in “name” and “age” of d1 (step A10). Then, it is checked whether or not there is another unprocessed face in the image (step A11). If another unprocessed face remains (YES in step A11), the process returns to the above-described step A3, and the above-described operation is performed. repeat. If no other unprocessed face remains in the image (NO in step A11), it is checked whether or not the registration end to the name database d1 is instructed (step A12). (NO in A12), the process returns to the first step A1, the next captured image is acquired, and the above-described operation is repeated thereafter.

  When the end of registration in the name database d1 is instructed (YES in step A12), the process proceeds to registration processing in the person relation database d2 (steps A13 to A15). That is, an input screen is displayed in which the correlation between persons is defined as a tree structure correlation diagram based on the person relationship database d2 (step A13). When the correlation of the newly registered person is input by designating the position of the person newly registered this time or inputting the name or ID in the correlation diagram of the tree structure (step) A14) A process of updating and registering the content is performed by overwriting the correlation diagram reflecting the input in the person relation database d2 (step A15). Thereafter, the flow of FIG. 6 is exited.

7 and 8 are flowcharts that are started when the person relationship additional display function is designated in the reproduction mode.
First, the control unit 1 checks whether the subject to view the image has been determined (step B1 in FIG. 7). If the subject has not been determined (NO in step B1), the control unit 1 proceeds to subject determination processing (step B3). However, if the subject has been determined (YES in step B1), it is checked whether the change of the subject has been instructed by a user operation (step B2). Here, when the change of the subject is instructed (YES in step B2), the process proceeds to subject determination processing (step B3).

FIG. 9 is a flowchart for explaining the subject determination process (step B3 in FIG. 7) in detail.
First, the control unit 1 checks whether the subject is within a relative, that is, whether or not the user has instructed the subject to be a subject belonging to the correlation (relative structure) of a plurality of persons (relatives) (step C1). ), If a person in the relative is not designated as the subject (NO in step C1), another person is determined as the subject (step C2). When a person in the relative is instructed as a subject (YES in step C1), it is checked whether or not a user operation has instructed to automatically determine the subject (step C3).

  When automatic determination is instructed (YES in step C3), the imaging unit 6 is activated to capture the subject person (step C4), the captured image is acquired, the face in the image is analyzed, and the image is analyzed. A feature is detected, the name database d1 is referenced based on the detection result (face image, feature data), and the person of the face is determined as the subject (step C5). If automatic determination of the subject is not instructed (NO in step C3), a list of “names” in the name database d1 is displayed (step C6), and the person whose name is selected by the user operation from the list screen is the subject. Determine (step C7). The main body determined in this way is temporarily stored in the work memory m2 (step C8), and then the flow of FIG. 9 is exited.

  When such a subject determination process (step B3 in FIG. 7) ends, the process returns to step B1 described above. If the subject has already been determined (YES in step B1) and no change is instructed (NO in step B2), the subject temporarily stored in the work memory m2 is read in the subject determination process described above. (Step B4). Then, an image list screen is displayed (step B5). When an image to be displayed is arbitrarily selected from the list screen by a user operation (step B6), the face of each person in the selected image is detected. (Step B7), referring to the name database d1 based on the detection result (face image, feature data), and specifying each person (Step B8).

  Next, the flow proceeds to the flow of FIG. 8, and it is checked whether the subject determined by the subject determination process (step B3 in FIG. 7) is a relative (step B9). If another person other than the relative is the subject (step B9) NO), and the character string of “name” of each person identified in the above-mentioned step B8 (for example, XXX Taro, XX Hanako, XX Ichiro, XX Yuko) is read out from the name database d1 (step B10). Then, the process proceeds to a display process (steps B15 to B18) described later. If the subject is a relative (YES in step B9), the person relation database d2 is referred to based on each person identified in step B8 described above, and the human relationship with each person is identified from the subject as a viewpoint ( Step B11). For example, as shown in FIG. 10A, in the case of the subject (reference A), the person (B) is the wife of the person (A), the person (C) is the son of the person (A), and the person (D) Identifies a human relationship that is a younger sister of the person (C) and a person (E) is the younger brother of the person (D). In the case of the subject (reference F), the person (G) is the wife of the person (F), the person (A) is the son of the person (F), and the person (H) is the younger sister of the person (A). Identify relationships.

  Then, the “age” of the subject is read from the name database d1, and the age group (for example, under 18 years old / above) is specified (step B12). Next, the related analysis basic database d3 is referred based on the age group of the subject and the human relationship with each person (step B13), and a character string representing the relationship of each person is obtained from the subject as a viewpoint (step B14). . For example, as shown in FIG. 10 (2), in the case of the subject (reference A), character strings indicating the relationships between the persons (A), (B), to (H) are “I”, “wife”, “ Acquire the eldest son, daughter, second son, father, mother, and sister. Further, as shown in FIG. 10 (3), in the case of the subject (standard C), if the age of the person (C) is less than 18 years old, the person (A), (B), to (H) “Dad”, “Mom”, “I”, “Sister”, “Brother”, “Grandpa”, “Grandma”, “Aunt” are acquired as a character string indicating the relationship. If the person (C) is 18 years old or older, “Father”, “Mother”, “I”, “Sister”, “Brother”, “Grandfather,“ Grandmother ”, and“ Aunt ”are acquired.

  Then, the process proceeds to display processing (steps B15 to B18), and first, the display format is determined (step B15). In other words, whether the identification display that arranges the character string of the related pattern within the rectangular frame for the background is specified by the user operation, or the identification display that directly synthesizes the character string in the image without using the rectangular frame is the user operation Determines the display format as specified in. Next, the position of the acquired character string in the image is determined (step B16). In other words, whether or not the user's operation specifies that the acquired character string should be placed in the body while avoiding the face of the person, or the acquired character string is closely placed in association with the person while avoiding the entire person (whole body) Is determined by the user operation.

  Further, the display color is determined (step B17). In other words, in the case of a display format in which a character string of a related pattern is placed in a rectangular frame for background, white or black is determined as the background color (background color) of the rectangular frame, and it is acquired without using the rectangular frame. In the case of a display format in which the character string is directly arranged in the image, the color of the character string is determined according to the contrast with the image color at the arrangement position. The acquired character string is added and displayed (identified display) in association with the corresponding person in the image with the display format, arrangement position, and display color determined in this way (step B18).

FIG. 11 is a diagram showing a specific example of the person relation addition display. As a display format, the character string is directly synthesized in an image without using a rectangular frame, and the entire person (whole body) is set as an arrangement position. This is a display example when it is specified that the acquired character string is arranged close to the person in association with the person.
FIG. 11 (1) is a diagram showing an example of a person relationship addition display when a person other than the relative is the subject. In this case, “Taro XXX” is the person (A), “Ichiro XX” is the person (C), “Yu Yuko” is the person (D), “Hanako XX” is the person (B). The person (E) is additionally displayed (identified display) “Jiro XX”. FIG. 11 (2) is a diagram showing an example of a personal relationship addition display when a relative (C: under 18 years old) is the subject. In this case, the person (A) is “dad”, the person (C) is “I”, the person (D) is “sister”, the person (B) is “mother”, and the person (E) is “ "Brother" is additionally displayed (identified).

  In this state where the personal relationship is additionally displayed, it is checked whether the user has instructed to save the image (step B19), or whether switching to the next image is instructed (step B21). Then, it is checked whether or not the end of the person relationship additional display is instructed (step B22). When an instruction to save an image is given (YES in step B19), an image obtained by combining information (acquired character string) indicating personal relationships (relationships) is filed and recorded in an image memory m3 (for example, an SD card). (Step B20).

  When an instruction to switch to the next image is given (YES in step B21), the process proceeds to step B2 in FIG. 7, and if the subject is not changed (NO in step B2), the above operation is repeated. When the change of the subject is instructed (YES in step B2), the subject determination process (step B3) is executed, and then the above operation is repeated. When an instruction to end the personal relationship additional display function is given (YES in step B22), the flow of FIGS. 7 and 8 is exited.

  As described above, in the present embodiment, the control unit 1 refers to the name database d1, recognizes the person based on the face of the person in the image, and recognizes the person as the viewpoint of processing on the image. Since the person relation with the person is specified based on the person relation database d2, the person in the image can be appropriately specified in relation to the subject of the processing for the image.

  Since the control unit 1 identifies and displays information indicating the specified personal relationship on the display unit 5 in association with the person in the image, a personal relationship that differs depending on who is the same person depends on the identification display. I can inform you.

  Since the control unit 1 identifies and displays based on the correlation when the subject person belongs to the correlation among a plurality of persons, what kind of relationship the person has for the subject Can be easily known. In this case, it is possible to identify and display in an easy-to-understand manner the relationship of the person to the subject, or to identify and display in a friendly expression, for example, useful for creating a family introduction photo. It will be something.

  The control unit 1 reads out the name from the name database d1 and identifies and displays the name in association with the person in the image when the subject is another person who does not belong to the correlation among a plurality of persons. It is possible to identify and display with an easily understandable name.

  When the control unit 1 identifies and displays the information indicating the personal relationship, the control unit 1 arranges the information indicating the personal relationship in an appropriate position in the image so as not to hide the human face. The subject can clearly understand the correspondence with the identification display while confirming the face of the person.

  When the control unit 1 identifies and displays information indicating a person relationship, the controller 1 is arranged in association with the person at a position that does not hide the entire person in the image, so that the person in the image is affected by the identification display. The subject can clearly understand the correspondence with the identification display while confirming the entire person.

  When the control unit 1 identifies and displays information indicating a personal relationship, the control unit 1 is colored in a color selected according to the color of the ground on which the information is arranged in the image, so that it is possible to display the information easily using a contrasting color or the like. Thus, an editing operation for changing the color scheme is also unnecessary.

  Since the control unit 1 changes the information indicating the personal relationship based on the age group of the subject, the control unit 1 can display information that is easy to understand and familiar to the subject.

  Since the control unit 1 determines the person as the subject by recognizing the person from the face in the image taken by the imaging unit 6, an operation for designating the subject is not required, and convenience is improved. To do.

  Since the control unit 1 identifies the relationship of the person in the image based on the structure of the relative with the subject person as the viewpoint, it is possible to change the way the relationship is called by the subject. In addition, since the group of relatives is used as a unit, it is possible to easily specify the personal relationship (relationship).

  When the personal information of each person is arbitrarily input as personal information together with the correlation between a plurality of persons from the operation unit 4, the control unit 1 stores the input personal correlation and personal information in the name database d 1 and the personal relationship database. Since it is stored in d2, the user's unique person correlation and person information can be freely registered at any time.

  Since the control unit 1 adds information indicating the personal relationship to the image and records and saves it, it is possible to freely know the relationship with the person in the image at any time simply by reproducing the image. For example, family introduction photos can be freely created at any time.

In the above-described embodiment, the information (relationship) indicating the relationship with the person in the image is identified and displayed from the subject person as the viewpoint. However, in addition to the person relationship, the personal information (for example, (Such as age) may also be identified and displayed. In this case, an item of personal information may be selected and displayed according to who the subject is.
In the above-described embodiment, the case where the subject of processing on an image is the subject of viewing an image is exemplified, but when performing some processing based on additional information of a captured image, the subject of the processing may be used.

  In the above-described embodiment, a relative correlation diagram is illustrated as a correlation between a plurality of persons. However, the correlation diagram may be a personnel configuration diagram, an organization chart, or the like of a company or another organization.

  In the above-described embodiment, the name (real name) of the person is stored in the “name” of the name database d1, but a nickname, a brush name, a stage name, an icon, and the like may be stored.

  In the above-described embodiment, an image obtained by synthesizing information (acquired character string) indicating personal relationships is filed and stored. However, information indicating personal relationships is used as Exif information of an image file together with its arrangement position and the like. You may make it store.

  In the above-described embodiment, information indicating the specified personal relationship is identified and displayed on the display unit 5 in association with the person in the image. However, the information may be printed without being displayed.

  In the above-described embodiment, the case where the present invention is applied to a digital camera as an image processing apparatus is illustrated. However, the present invention is not limited to a camera, and a mobile phone such as a recorder, personal computer, PDA (personal portable information communication device) A music player, a tablet terminal, a television receiver, or the like may be used.

  Further, the “device” and “unit” shown in the above-described embodiment may be separated into a plurality of cases according to function, and are not limited to a single case. In addition, each step described in the above-described flowchart is not limited to time-series processing, and a plurality of steps may be processed in parallel or separately.

The embodiment of the present invention has been described above. However, the present invention is not limited to this, and includes the invention described in the claims and the equivalent scope thereof.
Hereinafter, the invention described in the claims of the present application will be appended.
(Appendix)
(Claim 1)
The invention described in claim 1
Person storage means for storing a correlation between a plurality of persons;
Recognition means for recognizing the person based on the face of the person in the image;
A determination means for determining a person who is a subject of processing for an image;
Specifying means for specifying a person relationship with the person recognized by the recognition means based on the correlation of the plurality of persons, with the subject person determined by the determination means as a viewpoint;
An image processing apparatus comprising:
(Claim 2)
The invention according to claim 2 is the image processing apparatus according to claim 1,
Processing means for performing a predetermined process on the image based on information indicating the personal relationship specified by the specifying means;
This is an image processing apparatus characterized by being configured as described above.

(Claim 3)
The invention according to claim 3 is the image processing apparatus according to claim 2,
The processing means identifies and displays information indicating the personal relationship specified by the specifying means in association with a person in the image.
This is an image processing apparatus characterized by being configured as described above.
(Claim 4)
The invention according to claim 4 is the image processing apparatus according to claim 3,
The processing means performs identification display based on the correlation when the subject person determined by the determination means belongs to the correlation among the plurality of persons.
This is an image processing apparatus characterized by being configured as described above.
(Claim 5)
The invention according to claim 5 is the image processing apparatus according to claim 3 or 4,
The person storage means stores a face and a name of each person together with a correlation between a plurality of persons,
The processing means reads out the name of the person recognized by the recognition means from the person storage means if the subject person determined by the determination means is another person who does not belong to the correlation of the plurality of persons. Display in association with the person in the image,
This is an image processing apparatus characterized by being configured as described above.
(Claim 6)
The invention according to claim 6 is the image processing apparatus according to claim 3,
The identification display of the information indicating the person relationship is arranged in association with the person at a position where the face of the person is not hidden in the image.
This is an image processing apparatus characterized by being configured as described above.
(Claim 7)
The invention according to claim 7 is the image processing apparatus according to claim 6,
The identification display of information indicating the person relationship is arranged in association with the person at a position that does not hide the entire person in the image.
This is an image processing apparatus characterized by being configured as described above.
(Claim 8)
The invention according to claim 8 is the image processing apparatus according to any one of claims 3 to 7,
The identification display of information indicating the person relationship is colored in a color selected according to the color of the background on which the information is arranged in the image.
This is an image processing apparatus characterized by being configured as described above.
(Claim 9)
The invention according to claim 9 is the image processing apparatus according to any one of claims 3 to 8,
The person storage means stores a face and age of each person together with a correlation between a plurality of persons,
The processing means changes information indicating the person relationship based on the age group of the subject person determined by the determining means.
This is an image processing apparatus characterized by being configured as described above.
(Claim 10)
The invention according to claim 10 is the image processing apparatus according to any one of claims 1 to 9,
Further comprising an imaging means,
The determining unit determines a person as a subject by recognizing the person from the face in the image captured by the imaging unit;
This is an image processing apparatus characterized by being configured as described above.
(Claim 11)
The invention according to claim 11 is the image processing apparatus according to any one of claims 1 to 10,
The person storage means stores a relative structure as a correlation between a plurality of persons,
The specifying means specifies a person's relationship recognized by the recognizing means based on the configuration of the relatives, with the person who is the subject determined by the determining means as a viewpoint.
This is an image processing apparatus characterized by being configured as described above.
(Claim 12)
The invention according to claim 12 is the image processing apparatus according to any one of claims 1 to 11,
It further comprises an input means for arbitrarily inputting personal information of each person as personal information together with a correlation between a plurality of persons,
The person storage means stores the correlation and person information of the person input from the input means;
This is an image processing apparatus characterized by being configured as described above.
(Claim 13)
The invention according to claim 13 is the image processing apparatus according to any one of claims 1 to 12,
Image storage means for adding information indicating the personal relationship specified by the specifying means to the image and recording and saving the image is further provided.
This is an image processing apparatus characterized by being configured as described above.
(Claim 14)
The invention according to claim 14
Recognizing the person based on the person's face in the image;
Specifying a person relationship with the recognized person based on a correlation between a plurality of persons, with a person who is a subject of processing on an image as a viewpoint;
The image processing method characterized by including.
(Claim 15)
The invention according to claim 15 is:
Against the computer,
The ability to recognize a person based on the person's face in the image,
A function to determine the person who will be the subject of processing for the image;
A function of specifying a person relationship with the recognized person based on a correlation between a plurality of persons, with the determined subject person as a viewpoint;
It is a program for realizing.

DESCRIPTION OF SYMBOLS 1 Control part 3 Memory | storage part 4 Operation part 5 Display part 6 Imaging part m1 Program memory m3 Image memory d1 Name database d2 Person relation database d3 Related analysis basic database

Claims (15)

Person storage means for storing a correlation between a plurality of persons;
Recognition means for recognizing the person based on the face of the person in the image;
A determination means for determining a person who is a subject of processing for an image;
Specifying means for specifying a person relationship with the person recognized by the recognition means based on the correlation of the plurality of persons, with the subject person determined by the determination means as a viewpoint;
An image processing apparatus comprising: Processing means for performing a predetermined process on the image based on information indicating the personal relationship specified by the specifying means;
The image processing apparatus according to claim 1, wherein the image processing apparatus is configured as described above. The processing means identifies and displays information indicating the personal relationship specified by the specifying means in association with a person in the image.
The image processing apparatus according to claim 2, which is configured as described above. The processing means performs identification display based on the correlation when the subject person determined by the determination means belongs to the correlation among the plurality of persons.
The image processing apparatus according to claim 3, which is configured as described above. The person storage means stores a face and a name of each person together with a correlation between a plurality of persons,
The processing means reads out the name of the person recognized by the recognition means from the person storage means if the subject person determined by the determination means is another person who does not belong to the correlation of the plurality of persons. Display in association with the person in the image,
5. The image processing apparatus according to claim 3, wherein the image processing apparatus is configured as described above. The identification display of the information indicating the person relationship is arranged in association with the person at a position where the face of the person is not hidden in the image.
The image processing apparatus according to claim 3, which is configured as described above. The identification display of information indicating the person relationship is arranged in association with the person at a position that does not hide the entire person in the image.
The image processing apparatus according to claim 6, which is configured as described above. The identification display of information indicating the person relationship is colored in a color selected according to the color of the background on which the information is arranged in the image.
The image processing apparatus according to claim 3, wherein the image processing apparatus is configured as described above. The person storage means stores a face and age of each person together with a correlation between a plurality of persons,
The processing means changes information indicating the person relationship based on the age group of the subject person determined by the determining means.
The image processing apparatus according to claim 3, wherein the image processing apparatus is configured as described above. Further comprising an imaging means,
The determining unit determines a person as a subject by recognizing the person from the face in the image captured by the imaging unit;
The image processing apparatus according to claim 1, wherein the image processing apparatus is configured as described above. The person storage means stores a relative structure as a correlation between a plurality of persons,
The specifying means specifies a person's relationship recognized by the recognizing means based on the configuration of the relatives, with the person who is the subject determined by the determining means as a viewpoint.
The image processing apparatus according to claim 1, wherein the image processing apparatus is configured as described above. It further comprises an input means for arbitrarily inputting personal information of each person as personal information together with a correlation between a plurality of persons,
The person storage means stores the correlation and person information of the person input from the input means;
The image processing apparatus according to claim 1, wherein the image processing apparatus is configured as described above. Image storage means for adding information indicating the personal relationship specified by the specifying means to the image and recording and saving the image is further provided.
The image processing apparatus according to claim 1, wherein the image processing apparatus is configured as described above. Recognizing the person based on the person's face in the image;
Specifying a person relationship with the recognized person based on a correlation between a plurality of persons, with a person who is a subject of processing on an image as a viewpoint;
An image processing method comprising: Against the computer,
The ability to recognize a person based on the person's face in the image,
A function to determine the person who will be the subject of processing for the image;
A function of specifying a person relationship with the recognized person based on a correlation between a plurality of persons, with the determined subject person as a viewpoint;
A program to realize

Images