JP2016197345A - Image analyzer, image analysis method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the accuracy in tracking processing.SOLUTION: The image analyzer acquires plural images constituting a moving image, associates at least either one of a feature value and a position of a subject which is detected from the acquired image with a subject in another image, updates a feature value of the associated subject, and determines whether the subject detected from the acquired image is similar to the another subject. The image analyzer does not update the feature value of the subject which is determined to be similar in the subjects detected from the acquired image.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image analysis method using feature information of a subject.

  As an image analysis technique for analyzing an image captured by a camera connected to a network, for example, a face recognition technique is known. In addition, a person search apparatus constructed using face recognition technology is used for narrowing down images in which a specific person or a lost child is reflected.

  In the person search device, the face of the person to be searched and the feature information (feature amount) obtained from the clothes are acquired and analyzed in advance and recorded in a format that is easy to search. In order to satisfy specifications such as response speed and search accuracy, an upper limit number of persons to be searched is determined in the person search apparatus. If each frame of the image acquired from the camera and the feature quantity extracted for each detected result are registered in the person search device as they are, a lot of similar information is recorded and the ability of the person search device is compressed. Will be. As a result, the number of searchable people is reduced and the search time is prolonged, which leads to a decrease in system performance.

  In the person search device, the face of the person to be searched and the feature information (feature amount) obtained from the clothes are acquired and analyzed in advance and recorded in a format that is easy to search. In order to satisfy specifications such as response speed and search accuracy, an upper limit number of persons to be searched is determined in the person search apparatus. If each frame of the image acquired from the camera and the feature quantity extracted for each detected result are registered in the person search device as they are, a lot of similar information is recorded and the ability of the person search device is compressed. Will be. As a result, the number of searchable people is reduced and the search time is prolonged, which leads to a decrease in system performance.

  To deal with this problem, Patent Document 1 describes a system in which a first server that selects a candidate image with the best appearance of a person and a second server that generates an image search key are described. . Patent Document 1 describes that an image selected by a first server is transmitted from the first server to the second server and processed by the second server. Patent Document 2 discloses a method in which a plurality of detected moving objects are classified into types such as a person and a vehicle based on the size, and stored in a storage unit for each type. In any of the methods, it is determined that the person is the same person by some means, and the extracted feature values are collectively sent to the person search device, thereby improving the efficiency of the registration process and the search process.

Japanese Patent No. 3612220 JP 2013-239205 A

  However, when the tracking process is performed using the methods of Patent Document 1 and Patent Document 2, there may be a case where the accuracy of the tracking process decreases. The tracking process is a process for identifying a person detected in a plurality of frames. In other words, Patent Document 1 and Patent Document 2 each identify a person across frames using very simple information such as a difference between two consecutive frames and a detected moving object size. On the other hand, the tracking process is not good at the intersection or crowded environment of the person being tracked. In such an environment, tracking may be interrupted, or the tracking target may be switched with another person. Therefore, when the tracking process is performed by the methods of Patent Documents 1 and 2, the accuracy of the tracking process may be reduced. Further, if the tracking target feature amount extracted in such an environment is recorded in the person search device, there is a possibility that the accuracy of the person search system may be reduced.

  The present invention has been made in view of the above problems, and an object thereof is to improve the accuracy of tracking processing.

  As a means for achieving the above object, the image analysis apparatus of the present invention has the following configuration. That is, an acquisition unit that acquires a plurality of images constituting a moving image, and a subject of another image using at least one of the feature amount and position of the subject detected from the image acquired by the acquisition unit A first integrating means for associating, an updating means for updating the feature quantity of the subject associated by the first integrating means, and a subject detected from the image acquired by the acquiring means intersect with another subject. A crossing judging means for judging whether or not the subject has been detected by the crossing judging means among the subjects detected from the image obtained by the obtaining means. The feature amount is not updated.

  According to the present invention, the accuracy of the tracking process can be improved.

The figure which shows the network connection structure in 1st Embodiment. The figure which shows an example of the functional block of the person search system in 1st Embodiment. The figure which shows an example of the hardware constitutions of the person search system in 1st Embodiment. FIG. 6 is a processing flowchart in the image analysis apparatus 300. The figure explaining the crossing determination implemented in the crossing determination part 303. FIG. The figure explaining the feature-value extracted by the image analysis apparatus 300. FIG.

  Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the accompanying drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

[First Embodiment]
FIG. 1 is a diagram illustrating a network connection configuration of a person search system (image analysis system) according to the first embodiment. As components of the person search system in the first embodiment, a network camera (hereinafter referred to as camera) 100, a network storage apparatus (hereinafter referred to as storage apparatus) 200, an image analysis apparatus 300, a person search apparatus 400, and an image display apparatus 500 are provided. It is connected by a LAN 600 that is a network line. The camera 100 is an imaging device and has a function of distributing encoded image data via the LAN 600. The storage device 200 receives and records image data distributed from the camera 100 via the LAN 600.

  The image analysis device 300 reads image (moving image) data recorded in the camera 100 or the storage device 200 via the LAN 600, and generates registration information of a person search system described later. The person search device 400 acquires feature quantities of a person to be searched and records them. The person search device 400 executes person search processing based on a search instruction from the user. The image display device 500 reproduces and displays image data distributed from the camera 100 and image data recorded in the storage device 200. Further, the image display device 500 displays a search user interface, and transmits a search target and person attribute information to the person search device 400 based on a user instruction.

  FIG. 2 is a diagram illustrating an example of functional blocks of the person search system in the present embodiment. The camera 100 includes an image acquisition unit 101, an encoding unit 102, and a communication unit 103. Image data acquired by the image acquisition unit and subjected to image processing is converted into a format that can be communicated by the LAN 600 by the encoding unit 102. The converted image data is distributed to the storage device 200, the image analysis device 300, and the image display device 500 through the communication unit 103. The storage apparatus 200 receives the distribution image from the LAN 600 via the communication unit 201 and records it in the recording unit 202.

  The image analysis device 300 reads images recorded in the camera 100 and the storage device 200 from the LAN 600 via the communication unit 301. In the image analysis apparatus 300, the person detection unit 302, the first integration unit 303, the first determination unit 304, and the second integration unit 305 identify a person across frames. Next, in the image analysis apparatus 300, the second determination unit 306 and the feature amount extraction unit 307 perform feature amount extraction / update. Next, the image analysis apparatus 300 sends the person feature amount registered by the person registration unit 308 including the registration determination unit 3081 and the registered feature amount transmission unit 3082 to the person search device 400.

  The person search apparatus 400 receives registered person information and a person search request from the LAN 600 via the communication unit 401, and executes processes in the recording unit 402 and the search unit 403, respectively. Finally, the image display apparatus 500 receives the distribution image and the person search result from the LAN 600 via the communication unit 501 and displays them on the screen in the display unit 502.

  FIG. 3 is a diagram illustrating an example of a hardware configuration of each component of the person search system according to the present embodiment. The CPU 31 controls the operation of each component. The ROM 32 stores control instructions, that is, programs. The RAM 33 is used for temporary storage of work memory and data when executing a program. The communication unit 34 performs control for communicating with an external device. The display unit 35 performs various displays. A user I / F (interface) 36 receives user operations. The processing by each component 100 to 500 of the person search system may be performed by the CPU 31 executing a program stored in the ROM 32, or may be performed by dedicated hardware.

  Next, details of the processing of the person search system in the present embodiment will be described. First, processing performed by the camera 100 will be described. The image acquisition unit 101 performs predetermined pixel interpolation and color conversion processing on a digital electric signal acquired from an image sensor such as a CMOS, and develops a digital image such as RGB or YUV. In addition, the image acquisition unit 101 performs image correction processing such as white balance, sharpness, contrast, and color conversion on the developed digital image to generate image data. The image data generated by the image acquisition unit 101 is passed to the encoding unit 102. The encoding unit 102 encodes image data in a compression format such as JPEG, Motion JPEG, or H.264 in order to distribute an image via a network. The image data encoded by the encoding unit 102 is passed to the storage device 200, the image analysis device 300, and the image display device 500 via the communication unit 103 and the LAN 600.

  In the storage device 200, the communication unit 201 receives the image data, and the recording unit 202 records the received image data in the storage. The recording unit 202 may record the image file format at the time of distribution in the storage as it is, or may divide it into a plurality of files or a plurality of disks and record it in the storage in order to improve read / write access performance. The storage device 200 also performs processing for distributing recorded image data via the communication unit 201 for image analysis / display of past video in response to requests from the image analysis device 300 and the image display device 500. Do.

  The image analysis apparatus 300 performs integration processing on the data received from the camera 100 and the storage apparatus 200 and extracts a person feature amount to be transmitted to the person search apparatus 400. FIG. 4 shows a flow of image analysis processing. In the present embodiment, an example in which the camera 100, the storage device 200, the image analysis device 300, the person search device 400, and the image display device 500 are different devices has been described. However, the present embodiment is not limited to this example. Absent. For example, the camera 100 and the image analysis device 300 may be an integrated device, the camera 100 storage device 200 may be an integrated device, or the image analysis device 300 and the person search device 400 may be integrated. It may be.

  First, in step S4001, the communication unit 301 reads image data from the camera 100 and the storage apparatus 200 as necessary. In step S4002, the person detection unit 302 and the first integration unit 303 perform a first integration process. In the first integration process, identification of a person appearing between successive image frames, that is, a tracking target is performed. For the first integration process, first, the person detection unit 302 detects a feature quantity such as an SIFT feature quantity using a tracking technique. The first integration unit 303 identifies the same tracking target between successive image frames using the feature amount detected by the person detection unit 302, and assigns a unique ID to the identified tracking target. . That is, the first integration unit 303 associates the same tracking target between successive image frames.

  In step S4003, the first determination unit 304 refers to the detection target to which the ID is given by the first integration unit 303, and confirms whether a new tracking target has occurred. Since the first integration unit 303 assigns a unique ID to all tracking targets, the determination unit records a new tracking target ID by recording the ID when the image of the previous frame is processed. Whether or not has occurred can be determined. When a new tracking target is generated (NO in step S4003), the second integration unit 305 performs a second integration process in step S4004. Note that the second integration process may be omitted, and the process may proceed to step S3005 after step S3003.

  The second integration unit 305 determines whether the newly generated tracking target matches any of the tracking targets detected in the past. Specifically, the second integration unit 305 calculates the score value of the newly generated tracking target using the face recognition technology, and based on the score value, the newly generated tracking target is It is determined whether or not it matches any of the detected tracking targets. When it is determined that the newly generated tracking target matches any of the tracking targets detected in the past, the second integration unit 305 assigns a unique ID, assuming that the two tracking targets are the same. That is, the second integration unit 305 associates the same tracking target between the current image frame and the past image frame. For example, the second integration unit 305 gives the tracking target ID detected in the past to the newly generated tracking target. On the other hand, if it is determined that the newly generated tracking target does not match any of the tracking targets detected in the past, the newly generated tracking target has a state having the ID assigned by the first integration unit 303. Become.

  Next, in step S4005, the second determination unit 306 determines whether or not the detected tracking target is interlaced. In a general tracking process, in order to deal with the case where the tracking target is temporarily hidden behind some object, the target before and after being temporarily hidden is integrated even if the same target is somewhat doubtful. Accordingly, there is a possibility that the person information is mixed in the intersection of people and in a crowded environment. Therefore, the second determination unit 306 in the present embodiment detects position information of the tracking target, and determines whether or not they are interlaced by using the position information. As conditions for determining whether or not they are interlaced, there may be a distance from other tracking targets and how many other tracking targets exist.

  FIG. 5 shows the concept of the crossing area. As the tracking target position information, rectangular information of the face area and the person frame can be acquired. For example, the second determination unit 306 calculates the coordinates of the rectangle center, and calculates the distance between the tracking targets by calculating the Euclidean distance from the other coordinates. Further, the second determination unit 306 can calculate how many other tracking targets exist by counting the number of detection frames that are close to a predetermined threshold 1 or more. The second determination unit 306 can determine, for each tracking target frame, whether or not it is a crossing region by comparing this value with a predetermined threshold 2. As a result of the crossing determination, if it is determined that they are in the crossing state (YES in step S3005), the first integration process is interrupted in step S3006 (step S4006). That is, when it is determined that the two are in a mixed state, even if a feature amount is extracted later, the update process is not performed using the extracted amount. This process can be realized by, for example, the tracking result detected by the first integration unit 303 not being passed to the subsequent processes. Thereby, the tracking target determined to be in the intersecting state is not a processing target after step S4007, and the feature amount is not updated.

  In step S4007, the feature amount extraction unit 307 extracts and updates the person feature amount for each tracking target. FIG. 6 shows an example of the extracted person feature amount. In addition to low-level features such as SIFT, LBP, and HOG that represent gradient information of facial organ positions and surrounding colors, there are high-level features such as age, sex, glasses, beard, hair color, and clothing color. Since the result of the recognition process is not stable only with the extraction result in one frame and may fluctuate, an analysis result with higher accuracy can be obtained by comprehensively judging the result over a plurality of frames. For example, it can be realized by performing majority processing including the past frame and adopting the most detected result. If there are variations such as facial organ position, age, and color information, a stable analysis result can be obtained in the same manner by obtaining an average value.

  In step S4008, the registration determination unit 3081 determines whether the feature amount of the tracking target person can be registered. The registration determination unit 3081 is, for example, the number of seconds elapsed from the start of tracking obtained as a result of the first integration process, whether the crossing condition is not entered immediately before being obtained as a result of the intersection determination, or the result of the second integration process It is possible to determine whether or not registration is possible based on information such as whether a certain time has not passed since the crossing condition obtained. Also, the registration determination unit 3081 refers to the person feature amount acquisition result, and refers to how long the extracted / updated person attribute has not changed, thereby determining whether a sufficient number of feature amounts has been acquired. And can be used as a criterion for registration.

  If it is determined that the feature quantity of the tracking target person can be registered (YES in step S4008), finally, in step S4009, the registered feature quantity transmission unit 3082 sends the person feature quantity to be registered to the person search device 400. Transmit and complete the process.

  The person search apparatus 400 performs a process for registering person information sent from the image analysis apparatus 300 via the communication unit 401 and a process for searching for person information sent from the image display apparatus 500. The passed person information is stored in the recording unit 402 in a format that can be easily searched later. Specifically, the person search device 400 performs clustering processing in advance based on the feature amount passed as registration information so that the search response time is as short as possible, or stores the search index and search target data in the memory. Like in advance.

  Finally, the image display apparatus 500 displays the distribution image data read through the communication unit 501, the person search result, and the search user interface on the display unit 502. Based on the information returned from the person search device 400, the person search result is displayed in a format that is easy for the user to understand, such as chronological order, similarity order, and overlay display on the map. Further, the image display apparatus 500 acquires the search target and person attribute information by a user operation via a search user interface, and uses the acquired search target and person attribute information when making an inquiry to the person search apparatus 400. .

  As described above, according to the present embodiment, the tracking target feature quantity extracted in a crowded environment where tracking is interrupted or the tracking target is replaced with another person is not updated, and accuracy as a person search system is not updated. It is possible to prevent the decrease. In other words, by switching the person feature amount integration method depending on whether or not it is at the time of crossing, the efficiency of the feature amount registered in the person search device is improved, the performance of the person search system is improved, the response time is shortened, and the search accuracy Improvement is realized. In the present embodiment, the tracking target has been described as a person, but a subject other than a person may be used.

[Other Embodiments]
Although the embodiment has been described in detail above, the present invention can take an embodiment as a system, apparatus, method, program, recording medium (storage medium), or the like. Specifically, the present invention may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or may be applied to a device composed of a single device. good.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 network camera, 101 image acquisition unit, 102 encoding unit, 103 communication unit, 200 network storage device, 201 communication unit, 202 recording unit, 300 image analysis device, 301 communication unit, 302 human detection unit, 303 first integration , 304 first determination unit, 305 second integration unit, 306 second determination unit, 307 feature amount acquisition unit, 308 person registration unit, 3081 registration determination unit, 3082 registered feature amount transmission unit, 400 person search bar Device 401 communication unit 402 recording unit 403 search unit 500 image display device 501 communication unit 502 display unit 600 LAN

Claims (9)

Acquisition means for acquiring a plurality of images constituting the video;
First integration means for associating with a subject of another image using at least one of the feature amount and position of the subject detected from the image acquired by the acquisition means;
Updating means for updating the feature quantity of the subject associated by the first integration means;
Crossing determination means for determining whether or not the subject detected from the image acquired by the acquisition means crosses with another subject,
The update means does not update the feature quantity of the subject determined to be interlaced by the crossing determination means among the subjects detected from the image acquired by the acquisition means.   The image analysis apparatus according to claim 1, wherein the first integration unit performs a tracking process for identifying a subject appearing in a continuous image acquired by the acquisition unit. A second integration unit that associates a subject that has not been associated with the first integration unit with the same subject that has been detected in the past based on a feature amount of the subject;
The image analysis apparatus according to claim 1, wherein the update unit further updates the feature amount of the subject associated by the second integration unit.   The image analysis apparatus according to claim 3, wherein the second integration unit performs face recognition processing for identifying a subject appearing in a past image acquired by the acquisition unit.   The crossing determining means crosses whether or not the images acquired by the acquiring means are crossed based on the number of subjects associated with the first integrating means and the number of other subjects and the distance between them. The image analysis apparatus according to claim 1, wherein the determination is performed.   The image analysis apparatus according to claim 1, further comprising a transmission unit configured to transmit the updated feature amount to the external apparatus in order to register the updated feature quantity.   The image analysis apparatus according to claim 6, further comprising a registration determination unit that determines whether a predetermined condition for registering with the external apparatus is satisfied. An acquisition step of acquiring a plurality of images constituting the video;
A first integration step of associating with a subject of another image using at least one of the feature amount and the position of the subject detected from the image acquired in the acquisition step;
An update step of updating the feature amount of the subject associated in the first integration step;
A crossing determination step of determining whether or not the subject detected from the image acquired in the acquisition step is crossing with another subject,
In the update step, the feature amount of the subject determined to be interlaced in the crossing determination step among the subjects detected from the image acquired in the acquisition step is not updated. .   A program that causes a computer to function as each unit of the image analysis apparatus according to claim 1.

Images