JP5164767B2 - Image processing apparatus, image processing apparatus control method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, a control method for an image processing apparatus, and a program that improve the efficiency of image search by grouping the same person, for example, when collecting human face images from a plurality of digital cameras or the like.

  Generally, the person recognition device acquires individual identification information of a person to be recognized, and collates the acquired individual identification information with the individual identification information of a person to be recognized that is recorded in a recording unit in advance. Thus, it is recognized whether or not the person is the person himself / herself.

It is not assumed that a conventional person recognition apparatus performs acquisition and verification of individual identification information with a single device, collects personal information from a plurality of devices, and performs verification (Patent Document 1).
JP 2004-310287 A

  By the way, for example, a face recognition device is incorporated in a portable device such as a digital camera, and it is sufficiently conceivable to exchange human face information and the like between a plurality of person recognition devices.

  However, when exchanging human face information and the like between a plurality of person recognition devices, the following problem occurs.

  That is, when information on a person is received from a plurality of devices and information on the same person is included in each device, the information is recorded as a different person on the device side that has received the information.

  For example, using individual identification information as a face image, as shown in FIG. 31, the face images registered in the cameras A and B are transmitted from the digital camera A and the digital camera B to a personal computer (hereinafter referred to as a PC). Take the case of an example.

  32 is a face image table registered in the digital camera A, and FIG. 33 is a face image table registered in the digital camera B, and the face image tables are transmitted from both devices to the PC.

  In this case, the face image table shown in FIG. 34 is generated on the PC. However, even if “Yamada” registered in digital camera A and “Yama-chan” registered in digital camera B are the same person, they are not recognized as the same person as in FIG. Will be registered.

  Further, it is assumed that the image data shown in FIG. 35 is transmitted from the digital camera A to the PC. Here, “Yamada” is shown in the image data of FIG. 35, and the face image “Yamada” is associated therewith.

  In this case, when searching for an image on the PC, the image of FIG. 35 is hit only when searching for “Yamada”, and even if searching for “Yama-chan” of the same person, the image of FIG. 35 is not hit. ,ineffective. That is, in the prior art, there is a problem that classification can be performed only in units of face images.

  Therefore, the present invention classifies individual identification information collected from a plurality of image recording devices in units of persons, and enables an efficient image search when searching for an image showing a person, An object of the present invention is to provide an image processing apparatus control method and program.

In order to achieve the above object, an image processing apparatus of the present invention collects the image data and the individual identification information from a plurality of image recording apparatuses capable of recording image data and individual identification information of the image data. A collection unit, a collation unit that collates a plurality of pieces of individual identification information collected by the collection unit to determine whether or not they are the same individual, and a plurality of individuals that are determined to be the same individual by the collation unit Recording means for grouping and recording identification information, and setting a common name for a plurality of individual identification information grouped by the recording means, and recording the image for each of the plurality of individual identification information A setting means for associating the name set in the apparatus with the common name; and after the common name is set by the setting means, the plurality of individual identification information Retrieval means for accepting input of the name set in the image recording apparatus and the common name for each of them and searching for image data related to the input name from the image data collected by the collection means And .

An image processing apparatus control method according to the present invention includes: a collecting step of collecting the image data and the individual identification information from a plurality of image recording apparatuses capable of recording image data and individual identification information of the image data; A collation step for collating a plurality of individual identification information collected in the collecting step to determine whether or not they are the same individual, and a plurality of individual identification information determined to be the same individual in the collation step are grouped A common name is set for the plurality of individual identification information grouped by the recording step and the recording step, and each of the plurality of individual identification information is set in the image recording apparatus. A setting step for associating the common name with the common name, and after the common name is set by the setting step, Accepts input of the name set in the image recording apparatus for each piece of identification information and the common name, and searches for image data related to the input name from the image data collected by the collecting step And a search step .

The program of the present invention is a collection step for collecting the image data and the individual identification information from a plurality of image recording devices capable of recording the image data and the individual identification information of the image data, and collected in the collection step. A collation step for collating a plurality of individual identification information to determine whether or not they are the same individual, and a recording step for grouping and recording a plurality of individual identification information determined to be the same individual in the collation step And a common name for the plurality of individual identification information grouped by the recording step, and the common name and the name set in the image recording apparatus for each of the plurality of individual identification information A setting step for associating with the names of the plurality of individual identification information after the common name is set by the setting step. Receiving the input of the name set in the image recording apparatus and the common name for each of them, and searching the image data related to the input name from the image data collected by the collecting step And a search step .

  According to the present invention, individual identification information collected from a plurality of image recording devices can be classified by person, so that an efficient image search can be performed when searching for an image of a person.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
As shown in FIG. 1, in the notebook PC 10 which is the first embodiment of the image processing apparatus of the present invention, two digital cameras (image recording apparatuses) 1 and 2 are registered in the respective cameras 1 and 2. The face image is sent. In the present embodiment, a person face image is used as the individual identification information of the person, but other individual identification information other than the person face image may be used.

  FIG. 2 is a block diagram for explaining a configuration example of the digital cameras 1 and 2.

  In FIG. 2, an optical system 501 includes a focus lens, a diaphragm, and the like, and light incident on the optical system 501 forms an image on the light receiving surface of the image sensor 502 and is output as an analog signal. The analog signal is converted into a digital signal by the A / D conversion circuit 503 and input to the signal processing unit 504.

  The signal processing unit 504 converts the input digital signal into arbitrary image data, and then displays it on the display unit 505 or records it on the recording medium 506. The display unit 505 can realize the function of an electronic viewfinder (EVF) by sequentially displaying the image data processed by the signal processing unit 504.

  The recording medium 506 records not only image data but also face image data or face feature data described later. The recording medium 506 is generally an SD card or a CF card, but is not limited thereto.

  The external communication unit 507 communicates with an external device (in this embodiment, the notebook PC 10). The physical layer of the external communication unit 507 may be either wired or wireless. A face detection circuit 508 detects a face area from the captured image data.

  The technique for detecting the face area is not particularly limited as long as the face position and size information can be acquired. For example, a method using learning typified by a neutral network and a method of extracting a part having a characteristic physical shape such as eyes and nose from an image region by template matching are known.

  In addition, there is a method of detecting an image feature amount such as a skin color or an eye shape and analyzing it using a statistical method (see, for example, JP-A-10-232934 and JP-A-2000-48184). ).

  The face image matching circuit 509 extracts the feature amount of the face detected by the face detection circuit 508, and checks whether each face image recorded on the recording medium 506 is the same person.

  The face image matching technique is not particularly limited as long as it is a technique for determining whether or not each data is the same person. For example, a method is known in which one or more of face eyes, nasal cavity, ears, and the like are selected as feature points, and the person is collated by parameterizing the absolute length, pattern direction, center of gravity, etc. No. -283472).

  The operation unit 510 includes up / down / left / right operation buttons and a determination button for performing various settings. The control unit 511 controls processing of the entire digital camera. The memory 512 is used as a temporary image storage device. Further, the face image or the facial feature amount data may be recorded in the memory 512.

  FIG. 3 is a block diagram for explaining a configuration example of the notebook PC 10.

  In FIG. 3, an external communication unit 601 communicates with an external device. The physical layer of the external communication unit 601 may be either wired or wireless. The face image matching unit 602 has the same configuration as that of the face image matching circuit 509 in FIG. 2, and checks whether or not the input face images are the same person. The face image detection unit 603 has the same configuration as the face detection circuit 508 in FIG. 2, and detects a face area from image data.

  The recording medium 605 records image data, a face image, or facial feature data. In general, in a PC, a hard disk is used as the recording medium 605, but this is not a limitation. The memory 604 is used as a temporary image storage device. Further, the face image or the facial feature amount data may be recorded in the memory 604.

  The operation unit 606 is configured with a mouse, a keyboard, and the like. The display unit 607 is configured by a CRT, a liquid crystal display, or the like. The control unit 608 controls the entire notebook PC 10.

  Next, processing when the notebook PC 10 collects image data and a face table from the digital cameras 1 and 2 will be described.

  First, with reference to FIG. 4, processing until the digital cameras 1 and 2 register image data and a face table in the recording medium 506 will be described. Each process in FIG. 4 is executed by the CPU of the control unit 511 by loading a program stored in the memory 512 of the digital cameras 1 and 2 into the RAM.

  In the present embodiment, a case will be described in which a face table is first created in the face image registration mode, and then image data is captured and image data is captured.

  In FIG. 4, in step S2000, the control unit 511 performs shooting processing in the face image registration mode, and proceeds to step S2001. Specifically, the control unit 511 displays a menu on the display unit 505, and sets the face image registration mode when the user selects a button for the face image registration mode from the menu using the operation unit 510.

  Further, the control unit 511 converts the light incident on the optical system 501 into an electric signal by the image sensor 502 when the user presses a release button or the like, converts the light into a digital signal by the A / D conversion circuit 503, and then converts the light into a signal processing unit. Image data is created at 504 and output to the face detection circuit 508.

  In step S2001, the control unit 511 controls the face detection circuit 508 to detect a face image from the image data, and the process proceeds to step S2002.

  In step S2002, the control unit 511 records (registers) the face image detected in step S2001 on the recording medium 506, and proceeds to step S2003. FIG. 5 shows a face table recorded on the recording medium 506 of the digital camera 1, and FIG. 6 shows a face table recorded on the recording medium 506 of the digital camera 2. Here, it is assumed that “Yamada” in the face table in FIG. 5 and “Yama-chan” in the face table in FIG. 6 are the same person. The names “Yamada” and “Yama-chan” are digitally displayed by displaying a character input menu on the display unit 505 when the face image is registered in the recording medium 506 or when the face image registration mode is started. The user of the cameras 1 and 2 is input.

  With the above processing, the face image registration processing ends, and in step S2003, the control unit 511 performs normal photographing processing, and proceeds to step S2004.

  Specifically, when the user presses a release button or the like, light incident on the optical system 501 is converted into an electrical signal by the image sensor 502, converted into a digital signal by the A / D conversion circuit 503, and then converted by the signal processing unit 504. Image data is created and recorded on the recording medium 506.

  In step S2004, the control unit 511 controls the face detection circuit 508 to detect a face image from the image data. In addition, the control unit 511 compares the detected face image with the registered face table by the face matching circuit 509, and when the detected face image matches the face information of the registered face table, the process proceeds to step S2005. .

  In step S2005, the control unit 511 associates the face information matched in step S2004 with the image data captured in step S2003, and proceeds to step S2006. In the present embodiment, the digital camera 1 captures the image data shown in FIG. 7, and the subject of the image data matches “Yamada” registered in the face table shown in FIG. 5, and “Yamada” is shown in FIG. Assume that it is associated with image data. This association is managed by the association table shown in FIG. It is assumed that no image data exists in the digital camera 2.

  In step S2006, the control unit 511 records the image data associated in step S2005 on the recording medium 506, and ends the process. Through the above processing, the digital cameras 1 and 2 register the image data and the face table in their recording media.

  Next, a process when the notebook PC 10 collects image data and a face table from the digital cameras 1 and 2 will be described for each of the transmission source digital cameras 1 and 2 with reference to FIG. In the present embodiment, processing is performed in the order of the digital camera 1 and the digital camera 2. 9 is executed by the CPU of the control unit 608 by loading a program stored in the memory 604 or the like of the notebook PC 10 into the RAM.

  First, on the digital camera 1 side, for example, a menu is displayed on the display unit 505, and a button for transmission is selected from the display unit 505 to transmit image data and a face table to the notebook PC 10.

  On the notebook PC 10 side, in step S200, the control unit 608 receives the image data and the face image table transmitted from the digital camera 1 via the external communication unit 601, and proceeds to step S201.

  In step S201, the control unit 608 proceeds to step S203 on the assumption that image data is first processed from the received image data and face table.

  In step S203, the control unit 608 performs image data association processing 2 and proceeds to step S206.

  Here, the image data association processing 2 in step S203 will be described with reference to FIG.

  In FIG. 10, in step S2201, the control unit 608 determines whether face information is already associated with the image data. If not, the control unit 608 proceeds to step S2202, and if associated, the image data is acquired in step S2207. Record and exit.

  In the present embodiment, as described above, since the face information “Yamada” is associated with the image data in FIG. 7, the process proceeds to step S <b> 2207, the image data is recorded on the recording medium 605, and the process ends. Note that the processing when the face information is not associated with the image data (the processing after step S2202) will be described in a second embodiment of the present invention described later.

  Returning to FIG. 9, in step S206, the control unit 608 determines whether all received data has been processed. Here, since face table processing still remains, the process returns to step S201, and then proceeds to registration processing of step S202 to process the face table.

  Here, the registration process in step S202 will be described with reference to FIG.

  First, in step S300, the control unit 608 determines whether all the face information of the received face table has been processed. Here, since no face image of the face table shown in FIG. 5 has been processed, the process proceeds to step S301.

  In step S301, the control unit 608 performs processing for reading the next input face information, and proceeds to step S302. In this embodiment, the face information of the face table in FIG. 5 is processed in order from the top. Here, the face information “Yamada” in the face table of FIG. 5 is read out.

  In step S302, the control unit 608 determines whether a face table has already been registered.

  Here, the face table registered in the notebook PC 10 at the start of the processing in FIG. 11 is hereinafter referred to as an old face table. Before the face table is received from the digital camera 1, since the face table is not registered in the notebook PC 10, the control unit 608 advances the process to step S <b> 309 here.

  In step S309, the control unit 608 creates a new face table, registers face information as a new person, that is, face information “Yamada”, and returns to step S300. Here, the newly created face table is hereinafter referred to as a new face table.

  In step S300, the control unit 608 determines whether all the face information of the received face table has been processed. Here, since all the face information of the received face table has not yet been processed, the process proceeds to step S301.

  In step S301, the control unit 608 reads the face information “Suzuki” from the received face table, and proceeds to step S302.

  In step S302, since the old face table does not exist, the control unit 608 proceeds to step S309.

  In step S309, the control unit 608 registers the face information “Suzuki” in the new face table, and returns to step S300.

  In step S300, the control unit 608 determines whether all the face information of the received face table has been processed. Here, since all the face information of the received face table has not yet been processed, the process proceeds to step S301.

  In step S301, the control unit 608 reads the face information “Sato” from the received face table, and proceeds to step S302.

  In step S302, since the old face table does not exist, the control unit 608 proceeds to step S309.

  In step S309, the control unit 608 registers the face information “Sato” in the new face table, and returns to step S300.

  In step S300, the control unit 608 determines that all face information in the received face table is being processed, and proceeds to step S310.

  In step S310, the control unit 608 records the new face table in the storage medium 605, and ends the process. As described above, the face table shown in FIG. 5 is recorded on the recording medium 605 of the notebook PC 10.

  Returning to FIG. 9, in step S204, the control unit 608 performs group name determination processing.

  Here, the group name determination process will be described with reference to FIG.

  First, in step S800, the control unit 608 determines whether a new group has been generated in the previous registration process. In the previous registration process, “Yamada”, “Suzuki”, and “Sato” are only registered as new persons, and no group has been generated, and thus the process proceeds to step S803.

  In step S803, the control unit 608 determines whether face information has been added to the existing group in the previous registration process. Here, since the existing group did not exist before the previous registration process, the name determination process is terminated without doing anything.

  Returning to FIG. 9, in step S <b> 205, the control unit 608 performs image data association processing 1.

  Here, the image data association processing 1 will be described with reference to FIG.

  First, in step S900, the control unit 608 determines whether a new group has been generated in the previous registration process or whether face information has been added to an existing group. In the previous registration process, “Yamada”, “Suzuki”, and “Sato” are only registered as new persons, and no group is generated. Therefore, here, the image data association process is terminated without doing anything.

  Returning to FIG. 9, in step S206, the control unit 608 determines whether or not all data has been processed. Here, since all data is processed, a series of processes when collecting image data and a face image table from the digital camera 1 are ended.

  Next, similarly to the case of the digital camera 1 described above, a process when the notebook PC 10 collects image data and a face table from the digital camera 2 will be described with reference to FIG.

  First, since there is no image data on the digital camera 2 side as described above, only the face table shown in FIG. 6 is transmitted to the notebook PC 10.

  In step S200, when the control unit 608 receives a face table from the digital camera 2, the process proceeds to step S201.

  In step S201, the control unit 608 proceeds to step S202 to process only the face table and performs registration processing.

  Here, the registration process will be described with reference to FIG.

  First, in step S300, the control unit 608 determines whether all the face information of the received face table has been processed. Here, since no face information in the face table of FIG. 6 has been processed, the process proceeds to step S301.

  In step S301, the control unit 608 reads the face information “nabeyan” in the face table of FIG. 6, which is input face information, and proceeds to step S302.

  In step S302, the control unit 608 determines whether the old face table is registered. Here, since the face table of FIG. 5 received from the digital camera 1 has already been registered as the old face table, the process proceeds to step S303.

  In step S303, the control unit 608 determines whether or not the face information read in step S301 is collated with all face information recorded in the old face table. Here, since it has not collated with all the face information of an old face table, it progresses to step S304.

  In step S304, the control unit 608 reads “Yamada” as the first face information from the old face table, and proceeds to step S305.

  In step S305, the control unit 608 determines whether the information attached to the face information matches, or whether there is no information attached to the face information.

  Then, the control unit 608 returns to step S303 if the supplementary information does not match or if the supplementary information is present, and returns to step S303 if the supplementary information matches or if the supplementary information does not exist. Proceed to In the present embodiment, since there is no supplementary information in the face information, the process proceeds to step S306.

  In step S306, the control unit 608 controls the face image matching unit 602 to determine whether the received face table face information “Nabeyan” and the old face table face information “Yamada” are the same person. Therefore, the process proceeds to step S307.

  In step S307, the control unit 608 determines whether the two persons to be collated are the same person as a result of the collation process in step S306. If the two persons are the same person, the control unit 608 proceeds to step S308. Return to. Here, since “Nabeyan” and “Yamada” are not the same person, the process returns to step S303.

  In step S303, the control unit 608 determines again whether or not all face information recorded in the old face table has been collated. Here, since it has not collated with all the face information yet, it progresses to step S304.

  In step S304, the control unit 608 reads the next face information “Suzuki” from the old face table (FIG. 5), and proceeds to step S305.

  In step S305, the control unit 608 proceeds to step S306 because there is no incidental information in the face information as described above.

  In step S306, the control unit 608 controls the face image matching unit 602 to determine whether the received face information “Nabeyan” in the face table and the face information “Suzuki” in the old face table are the same person. Therefore, the process proceeds to step S307.

  In step S307, the control unit 608 determines whether the two persons to be collated are the same person as a result of the collation process in step S306. If the two persons are the same person, the control unit 608 proceeds to step S308. Return to. Here, “Nabeyan” and “Suzuki” are not the same person, so the process returns to step S303.

  In step S303, the control unit 608 determines again whether or not all face information recorded in the old face table has been collated. Here, since it has not collated with all the face information yet, it progresses to step S304.

  In step S304, the control unit 608 reads the next face information “Sato” from the old face table (FIG. 5), and proceeds to step S305.

  In step S305, the control unit 608 proceeds to step S306 because there is no incidental information in the face information as described above.

  In step S306, the control unit 608 controls the face image matching unit 602 to determine whether the received face table face information “Nabeyan” and the old face table face information “Sato” are the same person. Therefore, the process proceeds to step S307.

  In step S307, the control unit 608 determines whether the two persons to be collated are the same person as a result of the collation process in step S306. If the two persons are the same person, the control unit 608 proceeds to step S308. Return to. Here, “Nabeyan” and “Sato” are not the same person, so the process returns to step S303.

  In step S303, the control unit 608 determines again whether or not all face information recorded in the old face table has been collated. Here, since all face information in the old face table has been collated, the condition is satisfied, and the process proceeds to step S309. At this stage, the face information of the old face table of the digital camera 1 and the face information “Nabeyan” of the face table of the digital camera 2 did not match.

  In step S309, the control unit 608 registers “nabeyan” as a new person in the new face table, and returns to step S300.

  After step S300, the control unit 608 performs the same processing as described above. That is, the control unit 608 reads the next input face information “Yama-chan” from the face table of FIG. 6 in step S301, performs the same processing as described above in steps S302 and S303, and performs face information from the old face table in step S304. Read “Yamada”.

  In the same manner as described above, the control unit 608 proceeds from step S305 to step S306, controls the face image matching unit 602, and the face information “Yama-chan” and the face information “Yamada” in the old face table are the same person. A collation process is performed to determine whether or not there is, and the process proceeds to step S307.

  In step S307, the control unit 608 determines whether the two persons to be collated are the same person as a result of the collation process in step S306. If the two persons are the same person, the control unit 608 proceeds to step S308. Return to. Here, since “Yama-chan” and “Yamada” are the same person, the process proceeds to step S308.

  In step S308, the control unit 608 groups “Yama-chan” and “Yamada” determined to be the same person, registers them in the new face table, and returns to step S300.

  The new face table at this time is shown in FIG. The group name “A” in FIG. 14 is a name that is provisionally assigned until the name is determined in the group name determination process described later.

  In step S300, since the input face information still remains, the control unit 608 proceeds to step S302 and reads the face information “branch”. Since there is no match between the “branch” and the face information in the old face table, the processing of “branch” is the same as “nabeyan”, and is therefore omitted.

  In step S309, after registering “branch” in the new face table, the control unit 608 completes the processing for all input information, and thus satisfies the conditions of step S300 and proceeds to step S310.

  Finally, in step S310, the control unit 608 records the new face table on the recording medium 605, and ends the process. As described above, the face table shown in FIG. 15 is recorded on the recording medium 605 of the notebook PC 10.

  Next, returning to FIG. 9, in step S204, the control unit 608 performs group name determination processing.

  Here, the group name determination process will be described with reference to FIG.

  First, in step S800, the control unit 608 determines whether a new group has been generated in the previous registration process. Here, since “Yama-chan” and “Yamada” are grouped and registered as the same person in the previous registration process, the process proceeds to step S801.

  In step S801, the control unit 608 determines a common group name determination method to be assigned to each grouped face information. Specifically, as shown in FIG. 16, the control unit 608 displays a UI of “newly name” or “select from existing name” on the display unit 607, and the user selects either the operation unit 606. Select.

  Here, when the user selects “newly name”, the control unit 608 proceeds to step S802, waits for input of a new name by the user on the screen shown in FIG. move on. The user inputs a new name using the keyboard of the operation unit 606 or the like.

  On the other hand, when the user selects “select from existing name” in step S801, the control unit 608 proceeds to step S805 and waits for the user to select a group name from the existing name on the screen of FIG.

  Specifically, the control unit 608 displays the name of each grouped face information (“Yama-chan” and “Yamada”) on the display unit 607, and when the user determines any of them with the operation unit 606, the step The process proceeds to S803.

  The group name is determined by the above processing. In this embodiment, the group name “Taro Yamada” is newly set in step S802, and the face table shown in FIG. 19 is created.

  In step S803, the control unit 608 determines whether face information has been added to the existing group. If the face information has been added, the process proceeds to step S804, and if not, the process ends. Here, since the existing group did not exist before the previous registration process, the name determination process is terminated without doing anything.

  In step S803, if an existing group exists and face information is added thereto, the control unit 608 newly adds the common group name added to the existing group in step S804. An update process for adding and recording the face information is performed. With the above processing, the group name determination processing is completed.

  Next, returning to FIG. 9, in step S <b> 205, the control unit 608 performs image data association processing 1.

  Here, the image data association processing 1 will be described with reference to FIG.

  First, in step S900, the control unit 608 determines whether a new group has been generated in the previous registration process or whether face information has been added to an existing group. Here, since “Yama-chan” and “Yamada” are registered as new groups in the previous registration process, the process proceeds to step S901.

  In step S901, the control unit 608 associates another grouped face image with the image data associated with each grouped face image.

  Specifically, as described above, the face data “Yamada” is associated with the image data in FIG. 7 as shown in FIG. Here, the face image “Yama-chan” grouped with the face image “Yamada” is also associated with the image data in FIG. 7.

  Next, in step S902, the control unit 608 associates the group name added at the time of grouping with the image data associated with each grouped face image.

  Specifically, the group name “Taro Yamada” added when grouping here is also associated with the image data of FIG.

  As described above, as shown in FIG. 20, a file table in which “Taro Yamada”, “Yamada”, and “Yama-chan” are associated is created and displayed as shown in FIG.

  Since all the names of the same person are associated with this image data, even if “Yama-chan” is searched on the search screen shown in FIG. 22, the image of FIG. 21 will be hit, and “Yamada” or “Taro Yamada” The search is successful even if a search is performed with “”, so that an efficient image search can be performed.

  Although the digital camera 2 has been described as having no image data, if image data associated with the face image “Yama-chan” exists in the digital camera 2, the image data is also associated with the image data. Is done. Specifically, in step S901, the face image “Yamada” is also associated with the image data associated with the face image “Yama-chan”, and in step S902, the group name “Taro Yamada” is further associated.

  Returning to FIG. 9, in step S <b> 206, the control unit 608 finishes a series of processing when collecting the face table from the digital camera 2 because all the data is processed.

  As described above, in the notebook PC 10 of the present embodiment, the face information collected from the two digital cameras 1 and 2 can be classified by person, so when searching for an image in which a person is photographed. Efficient image search becomes possible.

(Second Embodiment)
Next, a digital camera which is a second embodiment of the image processing apparatus of the present invention will be described with reference to FIGS. Note that portions that overlap or correspond to the first embodiment will be described with reference to the drawings and symbols.

  As shown in FIG. 23, the digital camera 4 which is the second embodiment of the image processing apparatus of the present invention includes three digital cameras (image recording apparatuses) 1, 2, 3 to each camera 1, 2, The face image registered in 3 is transmitted.

  The basic configuration of the digital cameras 3 and 4 is the same as that of the digital cameras 1 and 2 already described with reference to FIG. In the present embodiment, a person face image is used as the person identification information. However, other individual identification information other than the person face image may be used.

  The processes until the digital cameras 1, 2, and 3 register the image data and the face table in the recording medium 506 are the same as the processes shown in FIG. In contrast to the first embodiment, in the present embodiment, when face registration is performed in step S2002 of FIG. 4, the user is prompted to input supplementary information about male or female, and the gender information is also reflected in the file name. The point is different.

  Specifically, the control unit 511 of the digital camera 1, 2, 3 displays a button for selecting male or female on the display unit 505, and the user selects with the operation unit 510 and presses the enter button to select the gender. Make input. In the present embodiment, the incidental information is gender, but the present invention is not limited to this.

  As a result, the face table shown in FIG. 24 is recorded on the recording medium 506 of the digital camera 1, and the face table shown in FIG. 25 is recorded on the recording medium 506 of the digital camera 2.

  It is assumed that the image data shown in FIG. 26 is recorded on the recording medium 506 of the digital camera 3. As in the image data of FIG. 7, the image data of FIG. 26 records an image including the same person as “Yamada” registered in the digital camera 1 and “Yama-chan” registered in the digital camera 2. . However, the digital camera 3 is premised on that no face table is recorded on the recording medium 506 and no one is associated with the image data (see FIG. 27).

  Next, the processing when the digital camera 4 collects the image data and the face table from the digital cameras 1 to 3 will be described for each of the transmission source digital cameras 1 to 3 with reference to FIG. In this embodiment, processing is performed in the order of the digital camera 1, the digital camera 2, and the digital camera 3.

  The processing when the digital camera 4 collects the face table from the digital cameras 1 and 2 is the same as that in the first embodiment except that the supplementary information is added to the face table. Only will be described.

  9 in this embodiment is executed by the CPU of the control unit 511 after a program stored in the memory 512 or the like of the digital camera 4 is loaded into the RAM.

  The processing of the control unit 511 when the digital camera 4 receives the image data and the face table transmitted from the digital camera 1 in step S200 of FIG. 9 will be described. The control unit 608 of the notebook PC 10 of the first embodiment. Since this is the same as the above process, the description thereof is omitted.

  Next, when the digital camera 4 receives the face table from the digital camera 2 in step S200 of FIG. 9, the control unit 511 of the digital camera 4 proceeds from step S201 to step S202, and performs the registration process shown in FIG.

  In FIG. 11, the control unit 511 proceeds to step S <b> 300 and step S <b> 301, and reads the first face information “Nabeyan” from the received face table. Next, the control unit 511 proceeds to step S302, step S303, and step S304, and reads the first registered face information “Yamada”.

  Next, in step S305, the control unit 511 determines whether the supplementary information of the face information “Nabeyan” and the face information “Yamada” matches, and if they match, the process proceeds to step S306. If not, the process returns to step S303.

  Here, since the genders of the face information “Nabeyan” and the face information “Yamada” are both male and the accompanying information matches, the control unit 511 proceeds to step S306 and controls the face matching circuit 509 to control the face information “ Matching processing of “Nabeyan” and face information “Yamada” is performed.

  Next, in step S307, the control unit 511 determines whether the face information “Nabeyan” and the face information “Yamada” are the same person as a result of the collation process in step S306. If not the same person, the process returns to step S303.

  Here, since face information “Nabeyan” and face information “Yamada” are not the same person, the process returns to step S303.

  Then, the control unit 511 again proceeds to step S303 and step S304, and reads “Suzuki” as the next registered face information.

  Next, in step S305, the control unit 511 proceeds to step S306 because the face information “Nabeyan” and the face information “Suzuki” are both male and the gender information matches, and the face matching circuit 509 is controlled. The face information “Nabeyan” and the face information “Suzuki” are collated.

  In step S307, the control unit 511 determines whether the face information “Nabeyan” and the face information “Suzuki” are the same person as a result of the collation process in step S306. If not the same person, the process returns to step S303.

  Here, the face information “Nabeyan” and the face information “Suzuki” are not the same person, so the process returns to step S303.

  Then, the control unit 511 again proceeds to step S303 and step S304, and reads “Sato” that is the next registered face information.

  Next, in step S305, the control unit 511 omits the matching process in step S306 because the face information “Nabeyan” is male and the face information “Sato” is female, and the incidental information does not match and does not satisfy the condition. Return to step S303.

  In step S303, since the control unit 511 has collated all the face information in the old face table, the control unit 511 proceeds to step S309, registers the face information “Nabeyan” as a new person in the new face table, and returns to step S300.

  As described above, it is different from the first embodiment that the number of collation processes in step S306 can be reduced by determining whether or not the incidental information matches in step S305. The same processing is performed for the face information “Yama-chan” and “Edachi”. As a result, the face table shown in FIG. 28 is recorded on the recording medium 506 of the digital camera 4.

  Next, processing when the digital camera 4 collects image data from the digital camera 3 will be described with reference to FIG. Note that the digital camera 3 transmits only image data to the digital camera 4.

  In step S200, when the control unit 511 of the digital camera 4 receives the image data transmitted from the digital camera 3, the process proceeds to step S201.

  In step S201, the control unit 511 proceeds to step S203 because only image data is received.

  In step S <b> 203, the control unit 511 performs image data association processing 2.

  Here, the image data association processing 2 in step S203 will be described with reference to FIG.

  First, in step S2201, the control unit 511 determines whether face information is already associated with the received image data.

  If the face information is not associated with the received image data, the control unit 511 proceeds to step S2202, and if associated with the received image data, the control unit 511 proceeds to step S2207, records the image data, and ends the process. In this embodiment, since the image data of FIG. 26 collected from the digital camera 3 is not associated, the process proceeds to step S2202.

  Here, with reference to FIG. 29, the face detection association process in step S2202 will be described.

  First, in step S2101, the control unit 511 controls the face detection circuit 508 to perform face information detection processing from the received image data, and proceeds to step S2102.

  In step S2102, the control unit 511 determines whether or not a face can be detected in step S2101. If a face can be detected, the process proceeds to step S2103. If a face cannot be detected, the process ends. Here, since face information exists in the received image data, the process proceeds to step S2103.

  In step S2103, the control unit 511 determines whether or not the old face table is registered in the digital camera 4. If the old face table is registered, the process proceeds to step S2104. If not registered, the process ends. Here, since the digital camera 4 records the face table of FIG. 28 on the recording medium 506, the process proceeds to step S2104.

  In step S2104, the control unit 511 determines whether all face information in the old face table has been processed. If all face information has been processed, the process ends, and all face information is processed. If not, the process proceeds to step S2105. Here, since all face information has not yet been processed, the process proceeds to step S2105.

  In step S2105, the control unit 511 reads face information (here, “Yamada”) from the old face table, and proceeds to step S2106.

  In step S2106, the control unit 511 controls the face collation circuit 509 to collate the face image detected in step S2102 with the face image read in step S2105, and the process proceeds to step S2107. Here, the face information detected from the received image data (FIG. 26) and the face information “Yamada” read from the old face table are collated.

  In step S2107, the control unit 511 determines whether the two persons to be collated are the same person as a result of the collation process in step S2106. Return. Here, since the face information detected from the image data of FIG. 26 and the face information “Yamada” read from the old face table are determined to be the same person, the process proceeds to step S2108.

  In step S2108, the control unit 511 associates the face information “Yamada” with the image data in FIG. 26 and ends the process. Specifically, no image is associated with the image data in FIG. 26 as shown in FIG. 27, but the face information “Yamada” is associated therewith.

  Returning to FIG. 10, in step S2203, the control unit 511 determines whether a matching face image is found during the series of processing in step S2202.

  If the matching face image is not found or the image is not associated, the control unit 511 proceeds to step S2207, records image data, and ends the process. Here, since a matching face image is found, the control unit 511 proceeds to step S2204.

  In step S2204, the control unit 511 determines whether the face images determined to match in step S2203 are grouped.

  If it is grouped, the control unit 511 proceeds to step S2205. If it is not grouped, the control unit 511 proceeds to step S2207, records image data, and ends the process. Here, since face information “Yamada” is grouped, the process proceeds to step S2205.

  In step S2205, the control unit 511 associates each face information grouped with “Yamada” with the image data, and the process advances to step S2206. Here, the face image “Yama-chan” grouped with the face image “Yamada” is also associated with the image data.

  In step S2206, the control unit 511 associates the group name with the image data associated with the face image in step S2205, and the process proceeds to step S2207. Specifically, the group name “Taro Yamada” of the face information “Yamada” is also associated with the image data.

  In step S2207, the control unit 511 records the image data and ends the process. As described above, “Taro Yamada”, “Yamada”, and “Yama-chan” shown in FIG. 30 are associated with the image data shown in FIG. 26, and the file table shown in FIG. 30 is created.

  Returning to FIG. 9, in step S206, the control unit 511 determines whether or not all data has been processed. Here, since all the data is processed, the control unit 511 ends a series of processes when collecting image data from the digital camera 3.

  As described above, in this embodiment, in addition to the first embodiment, it is possible to obtain an effect of reducing the load of collation processing by determining whether or not incidental information matches. Even when image data that is not associated with each other is acquired, it is possible to efficiently search for an image by collating it with face information already recorded.

  In addition, this invention is not limited to what was illustrated by said each embodiment, In the range which does not deviate from the summary of this invention, it can change suitably.

  The object of the present invention can also be achieved by executing the following processing. That is, a storage medium that records a program code of software that realizes the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU, MPU, etc.) of the system or apparatus is stored in the storage medium. This is the process of reading the code.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code and the storage medium storing the program code constitute the present invention.

  Moreover, the following can be used as a storage medium for supplying the program code. For example, floppy (registered trademark) disk, hard disk, magneto-optical disk, CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW, magnetic tape, nonvolatile memory card, ROM or the like. Alternatively, the program code may be downloaded via a network.

  Further, the present invention includes a case where the function of the above-described embodiment is realized by executing the program code read by the computer. In addition, an OS (operating system) running on the computer performs part or all of the actual processing based on an instruction of the program code, and the functions of the above-described embodiments are realized by the processing. Is also included.

  Furthermore, a case where the functions of the above-described embodiment are realized by the following processing is also included in the present invention. That is, the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing.

It is a figure which shows the state from which the face image registered into each camera is transmitted from two digital cameras to notebook PC which is 1st Embodiment of the image processing apparatus of this invention. It is a block diagram for demonstrating the structural example of a digital camera. It is a block diagram for demonstrating the structural example of notebook PC. It is a flowchart for demonstrating the process until a digital camera registers image data and a face table in a recording medium. It is a figure which shows the face table recorded on the recording medium of the digital camera. It is a figure which shows the face table recorded on the recording medium of the digital camera. It is a figure which shows an example of image data. It is a figure which shows an example of an association table. It is a flowchart for demonstrating the process at the time of collecting image data and a face table from a digital camera. FIG. 12 is a flowchart for explaining image data association processing 2; It is a flowchart for demonstrating the registration process of a face table. It is a flowchart figure for demonstrating a group name determination process. FIG. 10 is a flowchart for explaining image data association processing 1; It is a figure which shows a new face table. It is a figure which shows the face table before group name determination. It is a figure which shows an example of UI displayed when determining the determination method of a group name. It is a figure which shows an example of UI displayed when inputting a group name. It is a figure which shows an example of UI displayed when selecting a group name. It is a figure which shows the face table after group name determination. It is a figure which shows an example of an association table. It is a figure which shows an example of image data. It is a figure which shows an example of UI displayed at the time of an image search. It is a figure which shows the state in which the face image registered into each camera is transmitted from three digital cameras to the digital camera which is 2nd Embodiment of the image processing apparatus of this invention. It is a figure which shows the face table recorded on the recording medium of the digital camera. It is a figure which shows the face table recorded on the recording medium of the digital camera. It is a figure which shows an example of image data. It is a figure which shows an example of an association table. It is a figure which shows the face table after group name determination. It is a flowchart for demonstrating a face detection correlation process. It is a figure which shows an example of an association table. It is explanatory drawing for demonstrating a prior art example. It is a figure which shows the face table currently recorded on the digital camera. It is a figure which shows the face table currently recorded on the digital camera. It is a figure which shows the face table produced | generated by PC. It is a figure which shows an example of image data.

Explanation of symbols

1-4 Digital camera 10 Notebook PC
501 Optical system 502 Image sensor 503 A / D conversion circuit 504 Signal processing unit 505 Display unit 506 Recording medium 507 External communication unit 508 Face detection circuit 509 Face image matching circuit 510 Operation unit 511 Control unit 512 Memory 601 External communication unit 602 Face image Collation unit 603 Face image detection unit 604 Memory 605 Recording medium 606 Operation unit 607 Display unit 608 Control unit

Claims (10)

Collecting means for collecting the image data and the individual identification information from a plurality of image recording devices capable of recording the image data and the individual identification information of the image data;
Collating means for collating a plurality of individual identification information collected by the collecting means to determine whether or not they are the same individual;
A recording means for grouping and recording a plurality of pieces of individual identification information determined to be the same individual in the matching means;
A common name is set for a plurality of pieces of individual identification information grouped by the recording unit, and the name set in the image recording apparatus for each of the plurality of pieces of individual identification information and the common name A setting means for associating with
After the common name is set by the setting means, the name set in the image recording apparatus for each of the plurality of individual identification information and the input of the common name are accepted and input Search means for searching image data related to the name from the image data collected by the collecting means;
An image processing apparatus comprising: The setting means, when said set Tei was name in the image recording apparatus against the name of a plurality of individual identification information grouped by the recording means are different, the a is a name selected from the plurality of identification information The image processing apparatus according to claim 1 , wherein a common name is set for a plurality of pieces of individual identification information. When the individual identification information is associated with the image data in the image recording device, the recording means adds the individual identification information and the grouped individual identification information to the image data collected by the collecting means. recorded in association for image processing apparatus according to claim 1 or 2, characterized in that. When the individual identification information is associated with the image data in the image recording apparatus, the recording means sets a common name set when the individual identification information is grouped by the setting means. 4. The image processing apparatus according to claim 1, wherein the image data is collected in association with the image data and recorded. Comprising detection means for detecting individual identification information of a person from the image data collected by the collection means,
The collation means determines whether the individual identification information acquired by the detection means and the individual identification information recorded in the recording means are the same individual,
The recording means associates the individual identification information recorded in the recording means and the grouped individual identification information with the image data when the collating means determines that they are the same individual. recording Te, the image processing apparatus according to any one of claims 1 to 4, characterized in that. Comprising detection means for detecting individual identification information of a person from the image data collected by the collection means,
The collation means determines whether the individual identification information acquired by the detection means and the individual identification information recorded in the recording means are the same individual,
When the recording unit determines that the collating unit is the same individual, the recording unit sets a common name set by the setting unit when the individual identification information recorded in the recording unit is grouped. Recording in association with the image data;
The image processing apparatus according to claim 1 , wherein the image processing apparatus is an image processing apparatus. The image recording apparatus includes input means for inputting incidental information,
A determination unit that determines whether or not to perform collation processing by the collation unit based on the incidental information collected from the image recording apparatus by the collection unit;
The image processing apparatus according to any one of claims 1 to 6, characterized in that. The image processing apparatus according to claim 7 , wherein the incidental information is included in a file name of the individual identification information recorded by the recording unit. Collecting the image data and the individual identification information from a plurality of image recording devices capable of recording the image data and the individual identification information of the image data;
A collation step of collating a plurality of individual identification information collected in the collection step to determine whether or not they are the same individual;
A recording step of grouping and recording a plurality of individual identification information determined to be the same individual in the matching step;
A common name is set for the plurality of individual identification information grouped by the recording step, and the name set in the image recording apparatus for each of the plurality of individual identification information and the common name A configuration step for associating with
After the common name is set by the setting step, the name set in the image recording apparatus for each of the plurality of individual identification information and the input of the common name are accepted and input A retrieval step of retrieving image data related to the name from the image data collected by the collecting step;
An image processing apparatus control method comprising: Collecting the image data and the individual identification information from a plurality of image recording devices capable of recording the image data and the individual identification information of the image data;
A collation step of collating a plurality of individual identification information collected in the collection step to determine whether or not they are the same individual;
A recording step of grouping and recording a plurality of individual identification information determined to be the same individual in the matching step;
A common name is set for the plurality of individual identification information grouped by the recording step, and the name set in the image recording apparatus for each of the plurality of individual identification information and the common name A configuration step for associating with
After the common name is set by the setting step, the name set in the image recording apparatus for each of the plurality of individual identification information and the input of the common name are accepted and input A retrieval step of retrieving image data related to the name from the image data collected by the collecting step;
A program that causes a computer to execute.

Images