JP6833571B2 - Image processing equipment, image processing system, and control method - Google Patents

Description

The present invention relates to an image processing device and an image processing system having an individual recognition function.

A personal recognition method is known in which a person's characteristic information is extracted from an image and compared with the characteristic information as dictionary data registered in advance to recognize who the person in the image is (for example). See Patent Document 1).

JP-A-2010-73108

When the dictionary data prepared in advance in one image processing device is not sufficient, a method of improving the accuracy of personal recognition by sharing the dictionary data with another image processing device can be considered.

However, if feature information used for personal recognition is transmitted and received between a plurality of image processing devices, there is a risk that information that can identify an individual will be leaked if the security management of the image processing device is not sufficient. Occurs. For example, if the face image itself is used as feature information for personal recognition and is transmitted to and received from another image processing device, the face image is handed by a third party when the security management of the image processing device is insufficient. There is a risk of crossing over.

Therefore, there is room for improvement in suppressing leakage of personally identifiable information when transmitting and receiving dictionary data between image processing devices.

In order to solve the above problems, the present invention presents an individual recognition means for personally recognizing image data, characteristic information used for the individual recognition for each person to be recognized, and who is the characteristic information. An external device includes a storage means for storing identification information indicating the presence, a transmission / reception means for transmitting / receiving information, and the transmission / reception means for storing the characteristic information excluding the identification information among the information stored in the storage means. Provided is an image processing device characterized by having a control means for receiving feature information that can be regarded as the same person as the feature information from the external device.

Similarly, in order to solve the above problems, the present invention provides a first personal recognition means for personally recognizing an image, characteristic information used for personal recognition for each person to be recognized, and the same. A first image processing device including a first storage means for storing identification information indicating who the feature information is, a first transmission / reception means for transmitting / receiving information, and a feature used in the personal recognition process. An image processing system including a second storage means for storing information and a second image processing device including a second transmission / reception means for transmitting and receiving information, wherein the first image processing device is Of the information stored in the first storage means, the feature information excluding the identification information is transmitted to the second image processing apparatus by using the first transmission / reception means, and the second image. Among the feature information stored in the second storage means, the processing device uses the second transmission / reception means to transmit feature information that can be regarded as the same person as the feature information transmitted from the first image processing device. Provided is an image processing system characterized by being used and transmitted to the first image processing apparatus.

According to the present invention, it is possible to improve the accuracy of personal recognition among a plurality of image processing devices and suppress the leakage of data in which an individual is easily identified.

It is a block block diagram of the image processing system which concerns on embodiment of this invention. It is a flowchart which shows the procedure executed by the 1st image processing apparatus of FIG. It is a flowchart which shows the procedure executed by the 2nd image processing apparatus of FIG.

(First Embodiment)
Hereinafter, the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block configuration diagram of a first image processing device 120 and a second image processing device 121 constituting the image processing system according to the embodiment of the present invention.

In the embodiment of the present invention, the first image processing device 120 acquires new feature information necessary for personal recognition from an external image processing device including the second image processing device 121, so that the accuracy of personal recognition is high. An example of how to achieve the improvement will be described.

In FIG. 1, the first image processing device 120 is composed of a digital camera, a network camera, a tablet terminal, or the like. The first image processing device 120 includes an image pickup unit 101 including a zoom lens, a focus lens, and an aperture blade, and an image pickup element 102 such as a CCD or CMOS sensor. The output signal from the image sensor 102 is converted into image data digitized by the A / D converter 103 and stored in the memory 104.

The personal recognition unit 105 is an ASIC for performing personal recognition for identifying who is the image of a person included in the image data stored in the memory 104. The temporary storage unit 106 is a memory that temporarily records feature information as dictionary data acquired from an external image processing device via a communication unit. The feature information storage unit 109 is a memory for storing feature information as dictionary data used for personal recognition for each person. The feature information includes, for example, information indicating the relative relationship between the positions of the eyes and the nose, information indicating the shape and positional relationship of facial parts such as the position of the mole, and information indicating the distribution of irregularities on the surface of the face. The identification information storage unit 110 is associated with the feature information stored in the feature information storage unit 109, and identifies an individual such as the face image data, name, and personal ID of the person corresponding to each feature information. It is a memory that stores identification information for the purpose. The feature information storage unit 109 and the identification information storage unit 110 are non-volatile memories, and for example, a hard disk, a flash ROM, or the like is used. The feature information is information processed so as to be used for matching processing in the case of personal recognition, and even if only the feature information is referred to without referring to the identification information associated with this feature information, any person is shown. It is difficult to identify if it is.

The communication unit 107 is a communication circuit that transmits / receives data to / from an external image processing device by wire or wirelessly. The display unit 111 is a display that displays an image of a subject in real time based on image data stored in a memory, and displays an individual recognition result by the personal recognition unit 105 and identification information superimposed on the image. The operation unit 112 includes a button, a touch panel, and the like, and the control unit 108 issues an instruction to each unit in response to a user operation using the operation unit 112.

The control unit 115 is a processor composed of a CPU and an MPU, and controls and controls the entire operation of the first image processing device 120 by operating according to a program recorded in a ROM (not shown).

The first image processing device 120 does not necessarily have to include the image pickup unit 101 and the image pickup element 102, and may be configured to receive image data captured by an external camera and perform personal recognition. ..

Further, instead of configuring the personal recognition unit 105 with an ASIC, the control unit 115 may be configured to perform the same function by software processing.

Further, the feature information storage unit 109 and the identification information storage unit 110 are not separated, and the feature information and the identification information of the same person are stored in the same storage unit so that only the feature information can be extracted as needed. It may be configured.

The second image processing device 121 includes a communication unit 113, an individual recognition unit 114, a control unit 115, a feature information storage unit 116, an identification information storage unit 117, and a memory 118. These have the same functions as the communication unit 107, the personal recognition unit 105, the control unit 108, the feature information storage unit 109, the identification information storage unit 110, and the memory 104 of the first image processing device 120. The second image processing device 121 is a digital camera, a network camera, a tablet terminal, a personal computer, a server, or the like.

FIG. 2 is a flowchart showing the procedure of the first image processing device 120, which is executed under the control of the control unit 108 of the first image processing device 120 in FIG.

In step S201, the control unit 108 detects a face area from the captured image data or the read image data, and sets the detected face area as a target area for personal recognition. When a plurality of face areas are detected, the priority is set based on the size and the coordinates in the image data, and the face area having the highest priority is set as the target area for personal recognition. By operating the operation unit 112, the user may arbitrarily specify the target area for personal recognition. Then, the control unit 108 causes the personal recognition unit 105 to extract the feature information from the target area of the image data. As described above, this feature information is information indicating the relative relationship between the positions of the eyes and the nose, the shape and positional relationship of facial parts such as the position of the mole, or information indicating the distribution of unevenness on the surface of the face. ..

In step S202, the control unit 108 causes the personal recognition unit 105 to match the feature information extracted in step S202 with the feature information as dictionary data stored in the feature information storage unit 109, and determines the degree of similarity. .. The feature information storage unit 109 stores feature information for each person as dictionary data, and when there is a person whose similarity is equal to or higher than the threshold value, it is determined that the target area is that person, and individual recognition is successful. Judge that it was done. If there are a plurality of persons whose similarity is equal to or higher than the threshold value, it is determined that the person has the highest similarity. Then, when the individual recognition is successful, the identification information corresponding to the person is read out and the personal recognition result is displayed on the display unit 111.

In step S203, the control unit 108 proceeds to step S204 if the individual recognition fails in step S202, and ends the flow of FIG. 2 if it succeeds.

In step S204, the control unit 108 has the number and types of feature information stored in the feature information storage unit 109 corresponding to each person equal to or greater than the threshold value required for matching for personal recognition. Judge about. When storing feature information as dictionary data, if the person included in the image data for extracting this feature information is dark or part of it is hidden behind another object, it is necessary for personal recognition. Characteristic information may not be sufficiently extracted. Therefore, depending on the person, the feature information storage unit 109 may not store a sufficient amount of feature information for performing personal recognition with high accuracy. Then, the control unit 108 sends the feature information corresponding to the person determined to be lacking in the feature information, the external ID, and the reply request of the new feature information to the communication unit 113 of the second image processing device 121. It is transmitted to the communication unit of a plurality of external image processing devices including.

The external ID is given to distinguish the information from each other when the feature information of a plurality of persons is transmitted to the external image processing device. A fixed value may be set for this external ID for each person, or a random value may be set. As will be described later, the same external ID is assigned to the reply data received from the external image processing device, so that it is possible to determine which reply data corresponds to the characteristic information of which person. The threshold value used to determine whether the number and types of feature information are sufficient depends on the number and type of identification information stored in the identification information storage unit 110 and the feature information stored in the feature information storage unit 109. It may be changed accordingly. Further, when it is determined that the feature information as the dictionary data of any person is sufficient and not insufficient, the flow shown in FIG. 2 may be terminated.

In step S205, the control unit 108 determines whether the communication unit 107 has received the reply data from the communication units of the plurality of external image processing devices including the second image processing device 121, and when the reply data is received. The process proceeds to step S206.

In step S206, the control unit 108 determines whether the feature information included in the reply data includes new feature information that is not stored in the feature information storage unit 109 of the first image processing device 120. If there is no new feature information, it waits for the reply data including the new feature information to be received. If new feature information cannot be acquired as a result of receiving replies from all the external image processing devices that have sent the reply request in step S204, the processing flow of FIG. 2 may be terminated. Alternatively, the processing flow of FIG. 2 may be terminated even when new feature information cannot be acquired within a specific time.

If the reply data contains new feature information, the process proceeds to step S207, and the control unit 108 adds an external acquisition flag indicating that the new feature information has been acquired from the external image processing device. This external acquisition flag indicates that it has not been confirmed which identification information corresponds to the feature information. The control unit 108 deletes the external acquisition flag when the relationship between the newly acquired feature information and the identification information is established.

In step S208, the control unit 108 stores the new feature information acquired from the external image processing device in the temporary storage unit 106.

In step S209, the control unit 108 performs matching with the personal recognition unit 105 using the new feature information stored in the temporary storage unit 106 and the feature information extracted in step S201, and determines the degree of similarity. If the degree of similarity is equal to or greater than the threshold value, it is determined that the individual recognition of the person corresponding to the feature information transmitted to the external image processing device in step S204 has been successful. At this time, matching may be performed by using not only the new feature information stored in the temporary storage unit 106 but also the feature information of the same person stored in the feature information storage unit 109. Further, instead of the feature information extracted in step S201, a plurality of image data may be read from a recording medium (not shown) of the first image processing device 120, and the feature information may be extracted from these image data. By re-extracting the feature information from a plurality of image data, the possibility of successful personal recognition can be increased.

In step S210, the control unit 108 proceeds to step S211 if the individual recognition is successful in step S209, and proceeds to step S212 if it fails.

In step S211 the feature information stored in the temporary storage unit 106 is stored in the feature information storage unit 109 in association with the corresponding identification information. As a result, the feature information corresponding to the person is added, and the accuracy of personal recognition is improved.

In step S212, the new feature information received from the external image processing device is likely to correspond to a person different from the person whose feature information and identification information are stored in the first image processing device 120, and thus is temporarily stored. Delete from part 106.

In step S213, the external acquisition flag is deleted, and this flow ends.

In the present embodiment, when it is determined in step S203 that personal recognition has failed, a reply request is transmitted to the external image processing device, but the present invention is not limited to this. A reply request may be sent to the external image processing device by instructing the user at an arbitrary timing, or a reply request is automatically sent to the external image processing device when the first image processing device is started. It may be configured to transmit.

Further, when the reply data received from the external image processing device contains new feature information, the new feature information is stored in the feature information storage unit 109 instead of the temporary storage unit 106 together with the external acquisition flag. You may do so. Even if it is stored in the feature information storage unit 109, it can be determined whether or not the feature information has been confirmed to have succeeded in personal recognition by observing the presence or absence of the external acquisition flag.

In addition, if the external acquisition flag is left unerased and individual recognition is successful using the feature information to which the external acquisition flag is added, an image indicating that is added and the individual recognition is successful. The data may be recorded on a recording medium (not shown).

FIG. 3 is a flowchart showing the procedure of the second image processing device 121, which is one of the plurality of external image processing devices, and is executed under the control of the control unit 115 of the image processing device 121 in FIG. ..

In step S301, the control unit 115 determines whether the reply request and the feature information have been received from the external image processing device (here, the first image processing device 120) via the communication unit 113, and receives the reply request. If so, the process proceeds to step S302.

In step S302, the control unit 115 stores the feature information received by the communication unit 113 in the memory 118.

In step S303, the control unit 115 performs a comparison process between the feature information stored in the memory 118 and the feature information stored in the feature information storage unit 116, and the feature information having high similarity is the feature information storage unit 116. Determine if it is stored in.

In step S304, if the control unit 115 determines that the feature information having a high similarity to the feature information stored in the memory 118 is stored in the feature information storage unit 116, the process proceeds to step S305, otherwise the process proceeds to step S305. The process proceeds to step S306.

In step S305, the control unit 115 reads out the feature information determined to have high similarity separately from the identification information, and assigns the external ID assigned to the reply request to the feature information separated from the identification information tail. ..

In step S306, when it is determined in step S304 that there is similar feature information, the control unit 115 sends the reply data including the feature information to which the external ID is given at the reply source via the communication unit 113. Send to an external image processing device. If it is determined in step S304 that there is no similar feature information, that fact is transmitted as reply data.

As described above, according to the embodiment of the present invention, when the feature information as dictionary data is not sufficient, the feature information presumed to be the same person can be collected and added from the external image processing device. Therefore, the accuracy of personal recognition can be improved. Further, the feature information transmitted and received between the image processing devices is information that is separated from the identification information and processed so as to be used for matching processing at the time of personal recognition. Therefore, even if the feature information is leaked, it is difficult to identify which person is indicated even if only the feature information is referred to, so that the privacy of the individual can be protected.

Further, by adopting the present invention for a surveillance camera or a network, identification information and feature information can be accumulated regardless of the security state of the external image processing device.

Further, the above-described embodiment can be applied to other than the imaging device. An image processing device having a function of extracting personal feature information and a means for communicating with another image processing device may be a printer, a personal computer, an image search device, or the like.

The present invention also supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads the program. It can also be realized by the processing to be executed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

101 Image sensor 102 Image sensor 103 A / D converter 104, 118 Memory 105, 114 Personal recognition unit 106 Temporary storage unit 107, 113 Communication unit 108, 115 Control unit 109, 116 Feature information storage unit 110, 117 Identification information storage unit 111 Display unit 112 Operation unit 120 First image processing device 121 Second image processing device

Claims (12)

Personal recognition means for personal recognition of image data and
A storage means for storing feature information used in the personal recognition for each person to be personally recognized, and identification information indicating who the feature information is.
Sending and receiving means for sending and receiving information,
The transmission / reception means is made to transmit the feature information excluding the identification information among the information stored in the storage means to an external device, and the feature information that can be regarded as the same person as the feature information is transmitted from the external device. An image processing device comprising: a control means for receiving information. The claim is characterized in that the control means stores the feature information received from the external device in the storage means as feature information corresponding to the same person as the feature information transmitted to the transmission / reception means. Item 1. The image processing apparatus according to item 1. When the personal recognition means succeeds in personal recognition using the feature information received from the external device, the control means causes the transmission / reception means to transmit the feature information received from the external device. The image processing apparatus according to claim 1, wherein the feature information corresponding to the same person as the feature information is stored in the storage means. The control means causes the transmitting / receiving means to transmit the flagged feature information to the external device, and the flagged feature information that can be regarded as the same person as the feature information is transmitted to the external device. The image processing apparatus according to claim 3, wherein the image processing apparatus receives from. When the personal recognition means succeeds in personal recognition using the feature information received from the external device, the control means transfers the feature information received from the external device to a person corresponding to the flag. The image processing apparatus according to claim 4, wherein the corresponding feature information is stored in the storage means. The personal recognition means personally recognizes the image data using the feature information received from the external device.
The control means adds information indicating that the feature information received from the external device is used to the image data for which personal recognition is performed using the feature information received from the external device. The image processing apparatus according to any one of claims 1 to 5, wherein the image processing apparatus is stored in the storage means. A first personal recognition means for personally recognizing an image, characteristic information used in the personal recognition for each person to be personally recognized, and identification information indicating who the characteristic information is are stored. A first image processing apparatus including a first storage means for transmitting and receiving information and a first transmitting / receiving means for transmitting / receiving information.
An image processing system including a second storage means for storing feature information used in personal recognition, and a second image processing device including a second transmission / reception means for transmitting / receiving information.
The first image processing apparatus uses the first transmitting / receiving means to obtain the feature information excluding the identification information from the information stored in the first storage means, the second image processing apparatus. Send to
Among the feature information stored in the second storage means, the second image processing device uses the feature information that can be regarded as the same person as the feature information transmitted from the first image processing device. An image processing system characterized in that transmission to the first image processing apparatus is performed using the transmission / reception means of 2. A memory that stores personal recognition means for personally recognizing image data, characteristic information used for personal recognition for each person to be personally recognized, and identification information indicating who the characteristic information is. A control method for an image processing device having means and transmission / reception means for transmitting / receiving information.
A step of transmitting the feature information excluding the identification information among the information stored in the storage means to an external device, and
A control method for an image processing device, which comprises a step of receiving feature information that can be regarded as the same person as the feature information from the external device. A claim characterized by further comprising a step of storing the feature information received from the external device in the storage means as feature information corresponding to the same person as the feature information transmitted to the external device. Item 8. The method for controlling an image processing apparatus according to item 8. A step of causing the personal recognition means to perform personal recognition using the feature information received from the external device.
A step of storing the feature information received from the external device in the storage means as feature information corresponding to the same person as the feature information transmitted to the external device when the individual recognition is successful. The control method for an image processing apparatus according to claim 8, further comprising. A first personal recognition means for personally recognizing an image, characteristic information used in the personal recognition for each person to be personally recognized, and identification information indicating who the characteristic information is are stored. A first image processing apparatus including a first storage means for transmitting and receiving information and a first transmitting / receiving means for transmitting / receiving information.
A control method for an image processing system including a second storage means for storing feature information used in personal recognition and a second image processing device including a second transmission / reception means for transmitting / receiving information.
A step of transmitting the feature information excluding the identification information among the information stored in the first storage means to the second image processing apparatus by using the first transmission / reception means.
Of the feature information stored in the second storage means, the feature information that can be regarded as the same person as the feature information transmitted from the first transmission means is used in the first transmission / reception means. A control method of an image processing system, which comprises a step of transmitting to an image processing apparatus of the above. A program for causing a computer to execute each step of the control method according to any one of claims 8 to 11.

Images