JP2020190850A - Image processing device, imaging device, imaging processing method and program - Google Patents

Abstract

To reduce a processing load while maintaining the accuracy of collation processing of an object in an image.SOLUTION: An image processing device includes: a detection unit for detecting an object included in a picked-up image; a collation unit for calculating similarity between the detected object and a prescribed registered object, and determining whether or not the detected object is the prescribed registered object on the basis of the similarity; and an optimum area calculation unit for determining a second area included in a first area in accordance with the similarity calculated by the collation unit to the object detected by the detection unit in the first area in the picked-up image. The image processing device determines whether or not the object detected by the detection unit in the second area is the prescribed registered object.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image processing device, an imaging device, an image processing method and a program.

Conventionally, the area of a person is detected from the image from the camera, the feature amount such as a face is acquired from the detected area, and it is compared with the feature amount of the registered image to check whether the person in the image is the same. There is known a technique for determining whether the person is the same as the person corresponding to the registered image based on the collation result.
When such a technique is applied to a surveillance camera, it is difficult to continue taking images so as to always obtain stable features from the person due to the influence of the traveling direction and speed of the person.

Patent Document 1 discloses a technique for determining whether or not a region can stably acquire a feature amount and supporting improvement in accuracy.

JP-A-2017-76288

However, in the technique disclosed in Patent Document 1, although there is a description regarding a region in which a feature amount can be stably acquired, there is no description regarding the stability of collation for determining a person. That is, since the processing for stably lowering the false rejection rate (probability of not recognizing the person as the person) and the false acceptance rate (probability of mistaking another person as the person) is not included, the collation accuracy may decrease.

In the case of monitoring use, it is necessary to notify the collation result before the person is far away from the place. As the number of collations increases, the processing time increases and the processing load increases.

The present invention has been made in view of the above problems, and an object of the present invention is to reduce a processing load while maintaining the accuracy of collation processing of objects in an image.

In order to solve the above problems, according to an aspect of the image processing apparatus according to the present invention, the detection means for detecting an object contained in a captured image and the degree of similarity between the detected object and a predetermined registered object. A calculation means for calculating the above, a determination means for determining whether or not the detected object is the predetermined registered object based on the similarity calculated by the calculation means, and the captured image. A determination means for determining a second region included in the first region according to the similarity calculated by the calculation means for an object detected by the detection means in the first region, and the first region. Provided is an image processing device including a control means for controlling the determination means so as to determine whether or not the object detected by the detection means in the region 2 is a predetermined registered object. Will be done.

According to the present invention, it is possible to reduce the processing load while maintaining the accuracy of the collation processing of the objects in the image.

The figure which shows the configuration example of the image processing system of Embodiment 1. FIG. The block diagram which shows the example of the function of the image processing apparatus of Embodiment 1. FIG. The block diagram which shows the hardware configuration example of the image processing apparatus of Embodiment 1. FIG. The flowchart which shows the example of the optimum area calculation process. Explanatory drawing which shows an example of a frame image. The figure which shows the example of the information of metadata. The figure which shows the example of the optimization level. The figure which shows the example of the locus of movement of a person. The figure which shows the example of the change of the collation score. The figure which shows the example of the collation score for each area in a frame image. A flowchart showing an example of face image collation processing. The figure which shows the example of the optimum area. The figure which shows the example of the installation of the image pickup apparatus in the image processing system of Embodiment 2. The block diagram which shows the structural example of the image processing system of Embodiment 2. The figure which shows the example of the frame image. The figure which shows the example of the information of metadata. The figure which shows the example of the optimum area. The conceptual diagram which shows the process in Embodiment 3.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. It should be noted that each embodiment described below is an example as a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions. Is not necessarily limited to the embodiment of. Moreover, not all combinations of features described in the present embodiment are essential for the means of solving the present invention. The same processing will be described with the same reference numerals.

(Embodiment 1)
FIG. 1 shows a configuration example of the image processing system of the first embodiment.
This image processing system includes an image processing device 1 that performs image processing, an image pickup device 2 that performs image pickup processing, and a display 4. The image processing device 1 and the image pickup device 2 are connected via a network 3. The network 3 is composed of a plurality of routers, switches, cables and the like conforming to communication standards such as Ethernet (registered trademark). The network 3 may be configured by the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), or the like.

The image processing device 1 executes image processing for performing an authentication process for determining whether or not the same object as a predetermined object such as a person registered in advance in the image data captured by the image pickup device 2 is displayed. It is a device. The image processing device 1 is realized by, for example, a personal computer or the like in which a program for realizing an image processing function described later is installed. The imaging device 2 is a device that images a subject within the imaging range of the imaging device 2. The image pickup device 2 includes, for example, image data based on the captured image, identification information (device ID) for identifying the image pickup device 2, and information (frame ID) regarding the time when the image is captured, at predetermined time intervals. Is associated and transmitted to the image processing device 1 via the network 3.

The display 4 is composed of an LCD (Liquid Crystal Display) or the like. Further, the display 4 is connected to the image processing device 1 via a display cable compliant with a communication standard such as HDMI (registered trademark) (High Definition Multimedia Interface). In addition, the display 4 functions as a display means and displays an image captured by the image pickup apparatus 2 and a setting screen related to image processing described later. The display 4 and the image processing device 1 may be provided in a single housing.

(Configuration of image processing device 1)
FIG. 2 is a block diagram showing an example of the function of the image processing device 1.
FIG. 3 is a block diagram showing a hardware configuration example of the image processing device 1.
The image processing device 1 includes a CPU (Central Processing Unit) 31, a RAM (Random Access Memory) 32, and a ROM (Read Only Memory) 33. Further, the image processing device 1 includes a hard disk device (HDD) 34 for storing programs, data, and the like, and an interface (I / F) 35 for input / output with the outside such as communication. The RAM 32 temporarily stores programs and data. The ROM 33 stores programs and parameters that do not need to be changed. The CPU 31 controls the entire image processing device 1. The CPU 31 uses the RAM 32, which is the main memory, as a work memory to execute various programs stored in a storage medium such as the ROM 33 or the HDD 34. Each function shown in FIG. 2 is realized, for example, by executing a computer program stored in the ROM 33 of the image processing device 1 by the CPU 31 of the image processing device 1.

As shown in FIG. 2, the image processing device 1 includes a communication unit 11 that communicates with the image pickup device 2 via the network 3, a setting unit 12 that sets the processing of the entire image processing device 1, and an image or the like. It includes a display unit 13 for displaying and a detection unit 14 for detecting an object in an image. An example of detecting a person as an object will be described below. Further, the image processing device 1 includes a feature amount extraction unit 15 that calculates a feature amount from an image including a person's face detected by the detection unit 14, a face image extracted by the feature amount extraction unit, and a registered image of the storage unit 20. It is provided with a collation unit 16 that collates feature quantities and calculates a collation score indicating the degree of similarity with the same person. Further, the image processing device 1 is based on the metadata generation unit 17 that generates metadata from the face position information obtained by the detection unit 14 and the collation score obtained by the collation unit 16, and the metadata 24 and the collation score. It is provided with an optimum area calculation unit 18 for calculating an optimum area. Further, the image processing device 1 includes an operation unit 19 for inputting a user's operation and a storage unit 20 for storing data and the like necessary for the processing.

The communication unit 11 is composed of the I / F 35 of FIG. 3, and communicates with the image pickup device 2 via the network 3. For example, the communication unit 11 receives image data of an image captured by the image pickup device 2 and transmits a control command for controlling the image pickup device 2 to the image pickup device 2. The control command includes, for example, a command for giving an imaging instruction to the imaging device 2. The storage unit 20 is composed of the RAM 32, the HDD 34, and the like shown in FIG. 3, and stores information and data related to image processing by the image processing device 1. The operation unit 19 inputs information related to the operation performed by the user via an input device (not shown) such as a keyboard or a mouse.

The setting unit 12 makes settings related to information processing by the image processing device 1. The setting unit 12 sets, for example, a passing line (a line used for determining the passage of a predetermined area) designated by the user's operation based on the user's operation input via the operation unit 19. The display unit 13 causes the display 4 to display an image captured by the image pickup apparatus 2, a setting screen for making settings related to image processing, information indicating the result of image processing, and the like.

The detection unit 14 executes a detection process for detecting a person in the image. The detection unit 14 in the present embodiment detects a person included in the image by performing pattern matching or the like using a collation pattern (dictionary). The detection result of the detection process includes face position information indicating at which position of the image the face image is located. The detection unit 14 may be provided with a function (tracking function) for tracking the detected person. Here, the tracking function means that if the same person as the person detected by the detection unit 14 from the image of the frame one or more before the current frame exists in the image of the current frame, the people in each frame are used. It means the function to be associated. That is, the tracking function means a function of tracking a person between images of a plurality of frames that are close in time. When the tracking function is provided, the detection unit 14 records the face position in each frame image of the same person associated with the person in association with the time when the corresponding frame image was acquired or the above-mentioned frame ID, and moves the person. Is stored in the storage unit 20 as a locus of. That is, the detection unit 14 stores the face positions of the same person in the storage unit 20 in association with each of the plurality of frame images.

The feature amount extraction unit 15 extracts an image (face image) including the face of the person detected by the detection unit 14, and calculates the feature amount of the person. The collation unit 16 collates the face image extracted by the feature amount extraction unit 15 with the feature amount of the registered image stored in the storage unit 20, and calculates a collation score indicating the degree of similarity as to whether or not the person is the same person. When the calculated collation score is equal to or higher than a predetermined threshold value, the collation unit 16 determines that the person and the person in the registered image are the same person. The predetermined threshold value is stored in the storage unit 20. In addition, it is desirable to appropriately set a predetermined threshold value according to the imaging environment, registered image, purpose of use, and the like.

The metadata generation unit 17 generates metadata of the captured image from the face position information obtained by the detection unit 14 and the collation score obtained by the collation unit 16, and stores the metadata in the storage unit 20. .. When there are a plurality of imaging devices, the metadata is recorded in association with the identification information of the imaging device 2 that captured the captured image, that is, the device ID.

The optimum area calculation unit 18 calculates the optimum area based on the face position information in the metadata 24 stored in the storage unit 20 and the collation score by the collation unit 16, and stores the calculated optimum area in the storage unit 20. It is stored as the optimum area 22. Here, the optimum region is an region in which the same person can be easily collated within the imaging range corresponding to the captured image obtained by the imaging device 2. By performing collation processing on the face image in such an optimum region, the processing load can be reduced. Further, the area where the collation score of the same person is high is appropriate as the optimum area, and the area where the collation score of the same person is low is inappropriate as the optimum area. As an example showing this suitability, an optimization level according to a collation score is defined as shown in FIG. Here, the collation score is a numerical value in which, when the numerical value is high, the degree of personal matching in which the same person is determined to be the same is high.

The optimum area is calculated based on the face position information and the collation score by setting the optimization levels A and B, which have high numerical values, as the threshold value when the optimization level as shown in FIG. 7 is set. , The optimum area can be calculated. The optimization level is set by the setting unit 12 and stored in the storage unit 20 as the optimization level 23. The optimum area calculation unit 18 reads out the optimization level 23 from the storage unit 20 and uses it as needed.

Further, the optimum area is calculated by the optimum area calculation unit 18 based on the collation score and the face position information calculated in the past at predetermined time intervals, and is stored in the storage unit 20 as the optimum area 22. The optimum region may be calculated for each time zone, or may be performed on the condition that the lighting is turned on or off. The process of calculating the optimum region will be described later.

FIG. 4 is a flowchart showing an example of the optimum area calculation process by the image processing device 1. The process shown in FIG. 4 is executed every time the image data of the captured image is supplied from the image capturing device 2 at predetermined time intervals.
First, in S1, the communication unit 11 acquires the image data of the captured image from the imaging device 2 in units of frame images. Here, an example of a frame image is shown in FIG. 5 (a). It is assumed that a frame ID is assigned to each frame image as time information. Further, it is assumed that the device ID of the corresponding image pickup device 2 is assigned to each frame image.

Next, in S2, the detection unit 14 executes a detection process for detecting a person in the frame image acquired by the communication unit 11 in S1. If a person is detected from the frame image in S2, the process proceeds to S3. On the other hand, when no person is detected from the frame image, the image processing device 1 ends the processing for the current frame image and waits for the acquisition of the next frame image. When the next frame image is supplied from the image pickup device 2, the image processing device 1 executes the process again from S1, and the communication unit 11 acquires the next frame image.

When a person is detected from the frame image, the detection accuracy can be expected to be improved by using both the collation patterns used when the person is facing forward and when the person is facing sideways. For example, a collation pattern for collating with an image of a human body facing in front (rear) and a collation pattern for collating with an image of a person facing sideways are stored in the storage unit 20, and the installation state of the image pickup device 2 and the state of installation. Both collation patterns can be used as specified by the user.

Further, the collation pattern may be prepared from another angle such as from an oblique direction or from an upward direction. In addition, when detecting a person, it is not always necessary to prepare a collation pattern (dictionary) showing the characteristics of the whole body, and a collation pattern for a part of the person such as the upper body, lower body, head, face, and legs is prepared. You may. The process of detecting a person by the detection unit 14 may have a function of detecting a person from an image, and may be, for example, a process of known motion detection or skin color detection, and is limited to pattern matching processing. is not.

Next, in S3, the detection unit 14 detects the face in the frame image. Specifically, the detection unit 14 executes the detection process of the image (face image) of the face portion of the person detected in S2. Here, an example showing the position where the face image is detected is shown in FIG. 5 (b). In this example, the position where the face image is detected is displayed as the face position 41 with respect to the acquired frame image. It is assumed that the detected face position is given an ID that can identify each face position. Further, the detection unit 14 transitions to S4 when a face is detected in the frame image. On the other hand, when the face is not detected in the frame image, the image processing device 1 ends the processing for the acquired current frame image and waits for the acquisition of the next frame image. When the next frame image is supplied from the image pickup device 2, the image processing device 1 executes the process again from S1, and the communication unit 11 acquires the next frame image.

In S4, the feature amount extraction unit 15 extracts the feature amount, which is information indicating the facial features of the person, from the face image detected in S3. As the feature amount of the person, there is information such as color / edge / texture, hair color, shape of facial organs, presence / absence of sunglasses, presence / absence of beard, and the like.

In S5, the collation unit 16 performs collation processing of the face image. Specifically, the collation unit 16 compares the feature amount of the face image extracted from the feature amount extraction unit 15 with the registered image feature amount 21 stored in the storage unit 20, and creates the person and the registered image. Calculate a matching score that indicates the degree of similarity with the corresponding person. The higher the collation score, the higher the similarity. Then, when the collation score calculated by comparing the feature amount of the face image and the feature amount of the registered image exceeds a predetermined threshold value, the collation unit 16 is the same person as the person in the face image and the person in the registered image. Is determined to be. That is, in the collation unit 16, when the similarity between the feature amount of the face image and the feature amount of the registered image exceeds a predetermined threshold value, the person in the face image and the person in the registered image are the same person. Is determined. That is, the collation unit 16 determines whether or not the person in the face image, which is the detected object, is the person in the registered image, which is a predetermined object.

Next, in S6, the metadata generation unit 17 generates the metadata 24 based on the face position information obtained by the detection unit 14 and the collation score obtained by the collation unit 16. The metadata generation unit 17 stores the generated metadata in the storage unit 20. The image processing device 1 repeats the processes S1 to S8 until an end instruction is received from the user via the operation unit 19. As a result, the image processing device 1 continues to generate the metadata 24 of each frame image of the captured image continuously received from the image pickup device 2, and accumulates the metadata 24 in the storage unit 20. FIG. 6 shows an example of the information of the accumulated metadata 24. FIG. 6 shows an example in which the face positions are associated and accumulated as the metadata 24 in descending order of the collation score. The face position includes, for example, X and Y indicating the coordinates in the frame image and W and H indicating the size of the face position. Here, X indicates the X coordinate of the reference position (for example, the upper left) of the face area in the frame image in pixel units of the frame image, and Y indicates the Y coordinate of the reference position of the face area. Further, W indicates the width of the face region in pixel units of the frame image, and H indicates the height of the face region. Further, the metadata 24 may be accumulated for each person corresponding to the registered image, and the optimum area may be calculated for each specific person as described later. Alternatively, the metadata 24 may be accumulated without being limited to a specific person, and the same optimum region may be calculated without being limited to a specific person. Further, the metadata 24 may be stored in association with the time when the corresponding frame image was acquired or the above-mentioned frame ID.

Next, in S7, the optimum area calculation unit 18 calculates the optimum area.
The optimum area calculation unit 18 calculates the optimum area from the metadata 24 stored in the storage unit 20 based on the collation score for each face position. When a person (subject) moves in each frame image, the collation score changes according to the movement. This shows the change in the degree of similarity between the registered image and the face image obtained from the frame image. This change is due to the influence of the moving direction, moving speed, lighting, etc. of the person (subject) and the collation algorithm.

Here, the change in the collation score will be described with reference to FIGS. 8 and 9.
FIG. 8A shows an example of the registered image 50. FIG. 8B shows an example of the movement locus 55 of the same person as the registered image 50 when the collation unit 16 tracks the same person. The locus 55 is calculated based on the face positions of the same person collated in each frame image. In FIG. 8B, face positions 51, 52, and 53 indicate face positions used for face image collation processing.

FIG. 9 shows an example of a change in the collation score with respect to the locus 55 shown in FIG. 8 (b).
FIG. 9 shows the collation score for each consecutive frame image as a graph of the collation score corresponding to the frame ID of the frame image. Even for the same person, the collation score changes depending on the face position obtained for each frame image. Further, the above-mentioned accumulation of the collation score and the face position is targeted for those corresponding to the collation score above the boundary value 61 indicating a predetermined threshold value. The boundary value 61 is determined by the optimization level. For example, the optimization level B or higher in FIG. 7 may be set.

Next, the calculation of the optimum region will be described.
The optimum area calculation unit 18 calculates the optimum area based on the collation score and the face position accumulated as described above. Specifically, the optimum region may be the number of times (appearance rate) of the positions of the face positions where the collation score exceeds the boundary value (predetermined threshold value) 61, or may be the average value of the collation scores. FIG. 10 shows an example of the optimum area calculated by the optimum area calculation unit 18. In FIG. 10, the part with dark shading indicates the part with high collation score.

Next, in S8, the optimum area calculation unit 18 stores the optimum area calculated in S7 in the storage unit 20 as the optimum area 22. When the optimum area 22 is already stored in the storage unit 20, the stored optimum area 22 is overwritten with the value of the optimum area 22 obtained this time. As a result, the optimum area is updated at predetermined time intervals, and the optimum area 22 can always store the optimum area close to the processing time point in response to changes in the environment over time. As described above, the image processing device 1 updates the optimum area by periodically repeating the optimum area calculation process. In the above description, an example of updating the optimum area at predetermined time intervals has been shown, but the optimum area may be prepared for each time zone to be used and switched appropriately.

FIG. 11 is a flowchart showing a face image collation process using the optimum area. Hereinafter, the processing having the same content as that of FIG. 4 will be indicated by the same reference numerals, and the description thereof will be omitted.
First, in S10, the collation unit 16 acquires the optimum area 22 from the storage unit 20.
Next, in S11, when the communication unit 11 acquires the frame image from the image pickup device 2, the collation unit 16 selects the region in the frame image to be collated with the face image from the acquired optimum region and the frame image. Optimal area). FIG. 12 shows an optimum region to be collated with the face image. In the following S12, the detection unit 14 executes the person detection process only in the optimum region for the frame image obtained in S11. As a result, it is possible to reduce the number of times the person detection process is executed for the area where the collation score is low. If a person is detected in S12, the process proceeds to S3. Since the processes from S3 to S5 are the same as those in FIG. 4, they are omitted. After S5, proceed to S16.

Next, in S16, the collation unit 16 determines a person based on the collation score obtained in S5. Specifically, the collation unit 16 determines whether or not the person is the same as the person corresponding to the registered image based on the collation score indicating the degree of similarity between the registered image and the face image. Here, the collation score and the face position information are stored in the storage unit 20 as metadata 24 according to the determination result of whether or not they are the same, and the optimum area calculation unit 18 calculates the optimum area. The optimum area may be updated. That is, the collation process of FIG. 11 and the optimum area calculation process of FIG. 4 may be executed in a series of flow of the authentication process. Further, the processes from S12 (face detection) to S16 (determination) may be executed for the frame image in the optimum region. As a result, the processing load can be further reduced.

In the following S17, the image processing device 1 determines whether or not an end instruction has been input from the user via the operation unit 19. If the end instruction is not input, the process for the current frame image is ended, the process returns to S10, and the image pickup device 2 waits for the next frame image to be supplied. When the next frame image is supplied from the image pickup device 2, the image processing device 1 repeats the processing after S10. On the other hand, in S17, when the end instruction is input, the image processing device 1 ends the collation process of the face image of FIG.

As described above, in the image processing apparatus of the present embodiment, the number of collations is limited by selecting an area in the frame image to be collated with the registered image according to the optimum area obtained based on the collation score. .. That is, in the present embodiment, first, a region (optimal region) included in the region (first region) defined by the frame image is determined based on the collation score. Then, by setting the optimum region whose size (area) is smaller than that of the first region as the target region (second region) for collation, the number of collations is limited as compared with the case where the entire frame image is targeted. be able to. Thereby, in the present embodiment, it is possible to reduce the processing load while maintaining the accuracy of the collation processing of the objects in the image. Further, in the present embodiment, since the setting of the collation score and the optimum area to be collated can be automated, the installation load of the image processing system can be reduced.

(Embodiment 2)
In the second embodiment, an example of obtaining the residence time, which is the time to stay in the facility, will be described for each person who has entered the facility such as a store or a theme park. The residence time is measured, for example, by collating an individual person from two types of images captured by an entrance camera and an exit camera, and obtaining the residence time from the difference between the times when the collation was obtained. It should be noted that the staying time may be obtained from the difference in the acquisition time of the frame image used when the collation is obtained.

FIG. 13 is a diagram showing an example of installation of an image pickup apparatus in an image processing system for measuring residence time.
The entrance camera 81 captures a person passing through the entrance gate 82. The entrance direction 83 indicates a direction passing through the entrance. Similarly, the exit camera 86 captures a person passing through the exit gate 87. The exit direction 88 indicates a direction passing through the exit.

FIG. 14 is a block diagram showing a configuration example of the image processing system of the present embodiment. This image processing system includes one image pickup device 5 in addition to the configuration of the image processing system shown in FIG. 1 described above. Here, an example in which the image pickup device 2 is the entrance camera 81 and the image pickup device 5 is the exit camera 86 will be described. Further, in the present embodiment, the image processing device 1 includes a measuring unit for measuring the residence time in addition to the configuration of FIG. 2 described above.

FIG. 15A is a diagram showing an example of a frame image from the entrance camera 81, and FIG. 15B is a diagram showing an example of a frame image from the exit camera 86.
The feature amount extraction unit 15 of the image processing device 1 associates the feature amount of the face image detected from the frame image from the entrance camera 81 with the face position information and the camera identifier (device ID), and provides metadata to the storage unit 20. 24. Save as the registered image feature amount 21. On the other hand, the collation unit 16 is a registered image feature amount registered according to the feature amount of the face image detected from the frame image from the exit camera 86 and the face image detected from the frame image from the entrance camera 81. The collation with 21 is executed, and the collation score is calculated. The collation unit 16 stores the calculated collation score and the face position as metadata 24. FIG. 16 shows an example of the information of the metadata 24 accumulated according to the plurality of frame images. Separately, the collation unit 16 stores in the storage unit 20 the times when the face images are collated for the entrance and the exit. In addition, the measuring unit obtains the residence time from the difference between the times when the face images are collated for the entrance and the exit stored in the storage unit 20.

The optimum area calculation unit 18 calculates the optimum area based on the collation score and the face position of the accumulated metadata 24. FIG. 17 shows an example of the calculated optimum region. FIG. 17A is an optimum area for the frame image from the entrance camera 81, and FIG. 17B is an optimum area for the frame image from the exit camera 86. In the face image collation process for measuring the residence time, the face image of a person is extracted from the optimum area for the frame image from the entrance camera 81, and the face image extracted from the optimum area for the frame image from the exit camera 86. Is used to execute the collation process.

As described above, in the image processing apparatus of the present embodiment, when a plurality of imaging devices for measuring the residence time are used, the processing load is reduced while maintaining the accuracy of the collation processing of the objects in the image. be able to.

(Embodiment 3)
The third embodiment describes an image processing apparatus capable of reducing the processing load as compared with the second embodiment. In the third embodiment, the processing load is further reduced by using the determination result using the collation priority for the face detected in the optimum region.
FIG. 18 is a conceptual diagram showing the processing in the present embodiment. FIG. 18 shows the locus 95 of the same person when the detection unit 14 tracks the same person, the face image 91, the face image 92, and the face image 93 detected in the optimum region in the frame images of different frames. An example is shown.

Here, the collation unit 16 rearranges the face image 91, the face image 92, and the face image 93 detected in the optimum region in the order of regions having higher collation scores. For example, suppose that the order is face image 92, face image 93, and face image 91.

As described above, in the face image matching process, the feature amount extraction (S4), the face image matching process (S5), and the determination (S16) as shown in FIG. 11 are performed on the face image obtained by the face detection. ) Is executed. In the present embodiment, this process is executed for the face images in the order of regions with the highest collation scores. In the above example, first, the processes S4, S5, and S16 are executed on the face image 92. When the collation score exceeds a predetermined threshold value by the determination (S16), the collation unit 16 determines that the person is the same person. If it is determined that the person is the same person, the subsequent collation process of the person is not executed. That is, the processes of S4, S5, and S16 are not executed for the face image 93 and the face image 91 which are images of the same person. Thereby, in the present embodiment, the processing load can be further reduced. That is, in the present embodiment, the processing load of the collation process can be further reduced by preferentially performing the collation process for the area in the optimum area in descending order of the collation score.

Alternatively, in the present embodiment, a predetermined number of face images may be collated and determined with the highest collation score. For example, when the predetermined number is set to 2, the processing is executed for the face image 92 and the face image 93 based on the order of arrangement. For example, when the collation score of the face image 92 is 71 and the collation score of the face image 93 is 65, the collation score 71 is used as the determination result.

(Embodiment 4)
In the first to third embodiments, the optimum region may be modified. The case where the optimum region is modified will be described below as the fourth embodiment.
For example, when the angle of view of the frame image supplied from the image pickup apparatus 2 is changed by the user, the information of the optimum region is reset. When the information of the optimum area is reset, the detection unit 14 does not perform face detection narrowing down to the optimum area, but performs face detection on the entire area of the frame image or a predetermined area. As a result, the accuracy of the collation process can be maintained even when the angle of view of the image pickup apparatus 2 is changed. When the collation score obtained by the collation unit 16 and the metadata 24 of the face position by the metadata generation unit 17 are accumulated after the reset, the optimum area may be set again and the collation process may be executed. Alternatively, the previous optimum area may be stored, and when the angle of view of the frame image returns to the same angle of view as before, the previous optimum area may be restored.

As another correction of the optimum area, for example, when the average of the collation scores obtained by the accumulated information of the collation scores of the optimum area falls below a certain value, a process of correcting the optimum area is executed. Specifically, when the average of the collation score falls below a certain value, for example, by expanding the range of the optimum area, the range of accumulating the metadata of the collation score and the face position is expanded to correct the optimum area. As a result, the accuracy of the collation process can be maintained.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and modifications can be made within the scope of the gist thereof. For example, although the object to be detected is a person, an object other than the person may be a detection target.

(Other embodiments)
Although each embodiment has been described in detail above, the present invention can take an embodiment as a system, an apparatus, a method, a program, a recording medium (storage medium), or the like. Specifically, it may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or it may be applied to a device composed of one device. Good.

The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions. ASIC is an abbreviation for Application Specific Integrated Circuit.

1 ... image processing device, 2 ... imaging device, 3 ... network, 4 ... display, 11 ... communication unit, 12 ... setting unit, 13 ... display unit, 14 ... detection unit, 15 ... feature amount extraction unit, 16 ... collation unit , 17 ... Metadata generation unit, 18 ... Optimal area calculation unit, 19 ... Operation unit, 20 ... Storage unit

Claims (14)

A detection means that detects objects contained in the captured image,
A calculation means for calculating the similarity between the detected object and a predetermined registered object, and
A determination means for determining whether or not the detected object is the predetermined registered object based on the similarity calculated by the calculation means.
A determination to determine a second region included in the first region according to the similarity calculated by the calculation means with respect to an object detected by the detection means in the first region in the captured image. Means and
A control means that controls the determination means so as to determine whether or not the object detected by the detection means in the second region is the predetermined registered object.
An image processing device characterized by comprising. The calculation means is
The similarity is calculated by obtaining the feature amount of the object detected by the detection means and comparing it with the feature amount of the predetermined registered object.
The image processing apparatus according to claim 1. The determination means
When the similarity calculated by the calculation means is higher than a predetermined threshold value, it is determined that the object detected by the detection means is the registered object.
The image processing apparatus according to claim 1 or 2. The determination means is
The position of the object detected by the detection means and the similarity calculated by the calculation means are accumulated in association with each other, and the second region is determined according to the position of the accumulated object and the similarity. ,
The image processing apparatus according to any one of claims 1 to 3, wherein the image processing apparatus is characterized by the above. The detection means
Detecting the object included in the captured image for the first imaging range,
The object contained in the captured image for the second imaging range is detected, and the object is detected.
The calculation means is
The similarity between the detected object and the registered object for the first imaging range is calculated.
The second similarity between the detected object for the second imaging range and the detected object for the first imaging range was calculated.
The determination means
Depending on the similarity, the second region for the first imaging range is determined.
The second region for the second imaging range is determined according to the second similarity.
The image processing apparatus according to any one of claims 1 to 4, wherein the image processing apparatus is characterized by the above. The captured image has a plurality of frames and has a plurality of frames.
The detection means
The object contained in the captured image is detected for each of the plurality of frames of the captured image, and the object is detected.
The calculation means is
The position of the same object detected in the plurality of frames and the similarity in the frame are stored in association with each other.
The control means
The determination means is controlled so as to determine whether or not the object is a predetermined registered object according to the similarity stored in association with the position of the object.
The image processing apparatus according to any one of claims 1 to 5, wherein the image processing apparatus is characterized in that. The control means
The determination means is controlled so that the object whose similarity stored in association with the position of the object exceeds a predetermined threshold value is not subsequently determined whether or not it is the predetermined registered object. To do
6. The image processing apparatus according to claim 6. The control means
The determination means is controlled so as to determine whether or not the object is a predetermined registered object in descending order of similarity stored in association with the position of the object.
6. The image processing apparatus according to claim 6. The determination means is
When the angle of view of the captured image is changed, the second region is reset.
The image processing apparatus according to any one of claims 1 to 8, wherein the image processing apparatus is characterized in that. The determination means is
When the angle of view of the captured image returns to the same angle of view as before the reset, it returns to the second region before the reset.
9. The image processing apparatus according to claim 9. The determination means is
When the average of the similarity falls below a predetermined threshold, the second region is expanded.
The image processing apparatus according to any one of claims 1 to 10. An imaging means that captures the subject and outputs the captured image,
A detection means for detecting an object included in an image captured by the image pickup means,
A calculation means for calculating the similarity between the detected object and a predetermined registered object, and
A determination means for determining whether or not the detected object is the predetermined registered object based on the similarity calculated by the calculation means.
A determination to determine a second region included in the first region according to the similarity calculated by the calculation means with respect to an object detected by the detection means in the first region in the captured image. Means and
A control means that controls the determination means so as to determine whether or not the object detected by the detection means in the second region is the predetermined registered object.
An imaging device characterized by comprising. Steps to detect objects contained in the captured image,
The step of calculating the similarity between the detected object and the predetermined registered object, and
A step of determining whether or not the detected object is the predetermined registered object based on the calculated similarity.
A step of determining a second region included in the first region according to the calculated similarity to the detected object in the first region in the captured image.
A step of controlling the detected object in the second region so as to determine whether or not the detected object is the predetermined registered object.
An image processing method characterized by having. A program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 11.

Images