JP2018120283A - Information processing device, information processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve detection accuracy of an object.SOLUTION: An information processing device comprises estimation means to estimate, on the basis of a first image of an object taken by a first imaging device as well as correspondence between a coordination system of the first image and a coordination system of a second image of the object taken by a second imaging device, an imaging direction of the object in the second image, determination means to determine, on the basis of the imaging direction estimated by the estimation means, a recognition model of a learning subject used for detection of the object in the second image, and learning means to learn, on the basis of an image of the object contained in the second image, the recognition model determined by the determination means.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

In recent years, installation of an imaging device such as a monitoring camera in a facility such as a store is rapidly spreading for security. Using such an imaging device, not only to acquire images, but also to measure the degree of congestion by detecting objects such as people in the image, or to analyze the flow lines of objects, for marketing research of stores It has been proposed to use it. There is also a desire to search for a specific object from an image taken by an imaging device installed on the street.
When such image analysis is performed, useful information cannot be obtained unless the object detection accuracy is sufficient. As a method for detecting an object such as a person from an image, a method as described in Non-Patent Document 1 has been proposed. However, depending on the scene to be photographed, the object is standing far from the camera, so that sufficient resolution cannot be obtained, or the objects overlap and part of them are hidden, so that sufficient detection accuracy cannot be obtained. is there.

To deal with this problem, Non-Patent Document 2 discloses that a plurality of imaging devices having overlapping fields of view detect objects with each other and integrate the detection results to obtain a more accurate detection result. .
On the other hand, the appearance of a person varies greatly depending on the shooting direction and posture. Therefore, Non-Patent Document 3 discloses a technique for improving detection accuracy by dividing a human body into parts and using a plurality of recognition models for each part according to shooting directions.

Dalal and Triggs. Histograms of Oriented Gradients for Human Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2005) Leistner et al. VISUAL ON-LINE LEARNING IN DISTRIBUTED CAMERA NETWORKS. Proceedings of the International Conference on Distributed Smart Cameras (2008) Yang and Ramanan. Articulated dose estimation with flexible mixes-of-parts. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2011) By Xugang and Saburo Saburo. "3D vision". Kyoritsu Publishing (1998) Bagarinao et al. Adapting SVM Image Classes to Changes in Imaging Conditions Usage Incremental SVM: An Application to Car Detection. Proceedings of the 9th Asian Conference on Computer Vision (2009) Oza and Russell. Online Bagging and Boosting. Proceedings of the Artificial Intelligence and Statistics (2001) Szeliski, translated by Tamaki et al. “Computer Vision Algorithms and Applications”. Kyoritsu Publishing (2013)

Non-Patent Document 2 further discloses the following technique. That is, the position of the object in the image photographed by another camera is estimated using the detection result of the object for the image photographed by a certain camera. And it is the technique which learns the recognition model of the object about the image image | photographed with another camera by using the image of the estimated position in the image image | photographed with another camera as a learning sample. As a result, the accuracy can be improved by adapting the object recognition model to the installation scene.
On the other hand, when detecting an object such as a person walking in a facility such as a store or on the street, the change in posture is not so great, but the appearance changes greatly depending on the direction of shooting. Therefore, it is desirable to have a plurality of recognition models depending on the shooting direction. At that time, it is desirable to adapt the recognition model to the installation scene.
However, when a plurality of recognition models are used for object recognition, the technique described in Non-Patent Document 2 is suitable for which recognition model a learning sample obtained from an object detection result for an image captured by a camera is suitable. Could not be identified as a learning sample. Therefore, the recognition model cannot be learned with a suitable learning sample, and the object detection accuracy cannot be improved.

  An object of the present invention is to make it possible to accurately detect an object from images taken by a plurality of imaging devices.

  The information processing apparatus according to the present invention includes a first image in which an object is photographed by a first imaging device, and a second image in which the object is photographed by a coordinate system of the first image and a second imaging device. And a second image based on the photographing direction estimated by the estimating means and an estimating means for estimating a photographing direction of the object in the second image based on the correspondence relationship with the coordinate system of the second image. Learning the recognition model determined by the determining means based on a determination means for determining a recognition model to be used for detection of the object in the image and an image of the object included in the second image Learning means.

  According to the present invention, the object detection accuracy can be improved.

It is a figure which shows an example of the hardware constitutions of information processing apparatus. It is a figure showing an example of functional composition etc. of an information processor. It is a figure which shows an example of an imaging | photography scene. It is a flowchart which shows an example of a process of information processing apparatus. It is a figure which shows the detail of an example of a detection part. It is a flowchart which shows an example of a process of a detection part. It is a figure which shows an example of the image corresponding to a recognition model. It is a figure which shows an example of a calibration pattern. It is a figure explaining an example of the correspondence of the area | region in an image. It is a figure showing an example of functional composition etc. of an information processor. It is a flowchart which shows an example of a process of information processing apparatus. It is a figure which shows the detail of an example of a visual field control part. It is a flowchart which shows an example of a process of a visual field control part. It is a figure explaining an example of the evaluation method of an overlap area. It is a figure which shows an example of the image | photographed image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Embodiment 1>
FIG. 1 is a diagram illustrating an example of a hardware configuration of the information processing apparatus 10 according to the present embodiment. The information processing apparatus 10 includes a CPU (Central Processing Unit) 11, a storage device 12, an input device 13, and an output device 14. In addition, each component is connected via a bus | bath etc. so that communication is mutually possible.
The CPU 11 is a central processing unit that controls operations of the information processing apparatus 10 and executes programs stored in the storage device 12. The storage device 12 is a storage device such as a magnetic storage device or a semiconductor memory, and stores a program that is read based on the operation of the CPU 11, data that must be stored for a long time, and the like. In the present embodiment, the CPU 11 performs processing according to the procedure of the program stored in the storage device 12, so that the functions of the information processing apparatus 10 described later in FIG. 2 and the processing related to the flowcharts described later in FIGS. Realized. The storage device 12 also stores an image to be processed by the information processing apparatus 10 according to the present embodiment and a detection result.

The input device 13 is an input device such as a mouse, a keyboard, a touch panel device, or a button, and accepts various inputs from the user. The input device 13 also includes an interface for exchanging information with imaging devices such as cameras 111 and 112 described later in FIG. The output device 14 is a display device such as a liquid crystal panel and an external monitor, an audio output device, and the like, and outputs various types of information.
Note that the hardware configuration of the information processing apparatus 10 is not limited to the configuration of FIG. For example, the information processing device 10 may include an I / O device for performing communication between various devices. For example, the I / O device is a memory card, an input / output device such as a USB cable, a wired / wireless transmission / reception device, or the like.

FIG. 2 is a diagram illustrating an example of a functional configuration of the information processing apparatus 10. The processing and functions of the information processing apparatus 10 are realized by the CPU 11.
The information processing apparatus 10 includes shooting information acquisition units 121 and 122, image acquisition units 131 and 132, detection units 141 and 142, a positional relationship calculation unit 150, a position estimation unit 160, an extraction unit 170, a learning unit 180, a selection unit 190, An integration unit 200 is included.
The shooting information acquisition units 121 and 122 acquire shooting information from the cameras 111 and 112, respectively. The image acquisition units 131 and 132 acquire images taken by the cameras 111 and 112, respectively. The detection units 141 and 142 detect a person from images taken by the cameras 111 and 112, respectively. The positional relationship calculation unit 150 calculates a correspondence relationship between the positional coordinates of the image captured by the camera 111 and the positional coordinates of the image captured by the camera 112. The position estimation unit 160 estimates position coordinates corresponding to the position of the person detected by the detection unit 141 in the image captured by the camera 112. The extraction unit 170 extracts a partial image from the image captured by the camera 112. The learning unit 180 learns a recognition model used for detecting a person. The selection unit 190 determines a recognition model that is a learning target. The integration unit 200 integrates the detection results from the detection units 141 and 142.

The cameras 111 and 112 are imaging devices that capture a scene to be monitored.
The shooting information acquisition units 121 and 122 acquire shooting information of the camera 111 and the camera 112, respectively. The shooting information is information that associates the position coordinates of the image shot by the imaging device with the three-dimensional space coordinates of the shot scene, and is determined from the shooting magnification of the imaging device and the shooting direction of the imaging device. .
The image acquisition units 131 and 132 acquire images captured by the camera 111 and the camera 112, respectively. The image acquisition units 131 and 132 store the acquired information on each image in the storage device 12.
The detection units 141 and 142 detect objects to be detected from the images acquired by the image acquisition unit 131 and the image acquisition unit 132, respectively. In the present embodiment, the detection target object is a person, but may be an object such as a luggage, a vehicle, or a drone, or may be an animal such as a dog, a cat, or a domestic animal.
The positional relationship calculation unit 150 performs the following processing based on the shooting information acquired by the shooting information acquisition unit 121 and the shooting information acquisition unit 122. That is, the positional relationship calculation unit 150 calculates a coordinate conversion parameter for converting a coordinate system in an image captured by the camera 111 into a coordinate system in an image captured by the camera 112. The calculated coordinate conversion parameter is an example of correspondence information that associates the coordinate system in the image captured by the camera 111 with the coordinate system in the image captured by the camera 112.

The position estimation unit 160 uses the coordinate conversion parameters calculated by the positional relationship calculation unit 150 to estimate coordinates corresponding to the position of the person detected by the detection unit 141 in the image captured by the camera 112.
The extraction unit 170 extracts a partial image from the image captured by the camera 112 acquired by the image acquisition unit 132.
The learning unit 180 uses the partial image extracted by the extraction unit 170 to learn a recognition model used for the person detection process by the detection unit 142.
The selection unit 190 selects a recognition model learned by the learning unit 180 from a plurality of recognition models used for the person detection process by the detection unit 142.
The integration unit 200 integrates the detection processing results by the detection unit 141 and the detection unit 142.

Below, the process of the information processing apparatus 10 is demonstrated. The camera 111 and the camera 112 are installed so that the fields of view overlap so that the same person can be photographed with respect to the photographing scene as shown in FIG. In this embodiment, it is assumed that the fields of view of the camera 111 and the camera 112 are fixed. In addition, shooting information of the camera 111 and the camera 112 is also fixed. The shooting information of the camera 111 and the camera 112 is set in advance. In addition, it is assumed that the coordinate conversion parameter used for conversion from the position coordinates in the image captured by the camera 111 to the position coordinates in the image captured by the camera 112 is calculated in advance by the positional relationship calculation unit 150.
FIG. 4 is a flowchart illustrating an example of processing of the information processing apparatus 10.
In S401, the image acquisition unit 131 and the image acquisition unit 132 acquire images captured by the camera 111 and the camera 112, respectively. The image acquired in S401 is, for example, bitmap data represented by 8 bits for each of RGB. The cameras 111 and 112 are synchronized so that the shooting timings coincide. Therefore, the image acquisition unit 131 and the image acquisition unit 132 acquire images captured at the same timing by the camera 111 and the camera 112, respectively. However, the image acquisition unit 131 and the image acquisition unit 132 acquire images captured within a period set by the camera 111 and the camera 112 (for example, a period of 0.1 seconds centered on a certain time, respectively). It is good as well. The image acquisition unit 131 and the image acquisition unit 132 store the acquired images in the storage device 12. Hereinafter, an image taken by the camera 111 acquired by the image acquisition unit 131 is referred to as a first image. Hereinafter, an image taken by the camera 112 acquired by the image acquisition unit 132 is referred to as a second image.

In S402, the detection unit 141 detects a person who is an object to be detected from the first image. Details of the detection unit 141 are shown in FIG.
The detection unit 141 includes a partial image acquisition unit 1411, a feature extraction unit 1412, a pattern identification unit 1413, a parameter acquisition unit 1414, and a detection result output unit 1415. The partial image acquisition unit 1411 acquires a partial image for detecting a person from the first image. The feature extraction unit 1412 extracts a feature amount from the partial image acquired by the partial image acquisition unit 1411. The pattern identification unit 1413 determines whether the partial image acquired by the partial image acquisition unit 1411 is an image of a person based on the feature amount extracted by the feature extraction unit 1412 and the identification parameter acquired by the parameter acquisition unit 1414. Identify whether or not. The parameter acquisition unit 1414 acquires an identification parameter from a recognition model for recognizing a person. The detection result output unit 1415 outputs the position coordinates of the four vertices representing the rectangle of the partial image when the pattern identification unit 1413 identifies the person.

FIG. 6 is a flowchart illustrating an example of processing of the detection unit 141. Hereinafter, the details of the person detection process in S402 will be described with reference to FIG.
In step S601, the parameter acquisition unit 1414 sets one of recognition models for recognizing a person, and acquires parameters such as an identification parameter and a search range according to the set recognition model. The identification parameter is a parameter used for identification processing of an object corresponding to the recognition model, and is, for example, a feature amount extracted from the recognition model. In the present embodiment, there are three recognition models used for human recognition: a front model M1, a side model M2, and a planar model M3. Information on each model is stored in advance in the storage device 12 and managed by the parameter acquisition unit 1414. FIG. 7 shows an example of a person image to be recognized by each model. The front model M1 is a recognition model whose recognition target is an image obtained by photographing a person from the front direction as shown in FIG. In the present embodiment, the information processing apparatus 10 performs identification processing using a feature amount that captures the outline of a person. Therefore, the front model M1 is obtained by prior learning so that, in addition to an image obtained by photographing a person from the front, an image obtained by photographing a person having a similar outline from the rear can also be recognized. Further, the side model M2 and the plane model M3 are recognition models for which the human images shown in FIGS. 7B and 7C are recognized.

In this way, the appearance of a person varies depending on the direction in which the person is photographed. In this embodiment, the information processing apparatus 10 uses a plurality of recognition models with different directions in which the person is photographed to cope with fluctuations in the appearance of the person. doing. As shown in FIG. 7 (c), the plane model M3 does not strictly recognize only an image taken from above a person, but also recognizes an image taken from a peripheral direction above the person. Learned to target.
For example, the planar model M3 is learned by using a person photographed at a depression angle within a set range (for example, a range of 70 to 90 degrees from the horizontal direction) as a recognition target. Further, the front model M1 is not limited to an image that is strictly photographed from the front of the person, but is learned so that an image photographed from the peripheral direction in front of the person is also a recognition target. ing. For example, the front model M1 recognizes a person photographed at an azimuth angle within a set range (for example, a range of −10 degrees to 10 degrees, a range of 170 degrees to 190 degrees, etc.) with the forward direction of the person as a reference direction. Learned as a target. In addition, the side model M2 is not limited to an image captured strictly from the side surface of the person, but is learned so that an image captured from the peripheral direction of the side surface of the person is also recognized. Has been. For example, the side model M2 recognizes a person photographed at an azimuth angle within a set range (for example, a range of −10 degrees to 10 degrees and 170 degrees to 190 degrees, etc.) with the side direction of the person as a reference direction. Learned as a target.

  In addition, as shown in FIG. 7, a person image has not only the appearance but also the shape of the region surrounding the person depending on the situation at the time of shooting. In the present embodiment, the information processing apparatus 10 performs pattern matching on a rectangular area in an image, identifies whether or not the rectangular area is a person, and recognizes the vertical and horizontal sizes of the rectangular area to be used for matching. Change by model. Furthermore, in an image taken by a camera that takes a specific scene, there is a bias in the direction in which a person can be seen according to the position in the image. Therefore, in the present embodiment, the information processing apparatus 10 changes the search range, which is a range in an image for searching for a person, according to the recognition model. In order to perform the processing as described above, each human recognition model (recognition models M1 to M3) includes, in addition to the identification parameters used in the pattern identification processing, the vertical and horizontal sizes of the rectangular area to be collated, the search range in the image Parameters such as information. These parameters are acquired by the parameter acquisition unit 1414. In the present embodiment, the information processing apparatus 10 sequentially switches the three recognition models and repeats the following processes of S602 to S605. It is more advantageous to divide the corresponding shooting direction more finely to construct a human model with many recognition models, but the processing amount increases. Even in such a configuration, the processing of the present embodiment can be applied, but in the present embodiment, a person model is configured by three recognition models.

In step S602, the partial image acquisition unit 1411 acquires a partial region image for detecting a person from the first image. In order to detect an object from an image, for example, an image of a predetermined partial area called a search window is acquired from the image, and the acquired image is compared with a model representing an object to be detected. Then, the search window is sequentially moved in the image, and the object is detected from the image by repeating the collation. Also in this embodiment, the information processing apparatus 10 detects an object by such a method. In other words, the information processing apparatus 10 repeatedly acquires the partial region image in S602 and performs the processes in S603 to S605 described below.
In step S603, the feature extraction unit 1412 extracts a feature amount for detecting a person from the partial region image acquired in step S602. In the present embodiment, the feature extraction unit 1412 extracts gradient direction histogram features (Histograms of Oriented Gradients) as described in Non-Patent Document 1. The gradient direction histogram feature is one of the feature amounts that express the edge shape by obtaining the sum of the luminance gradients of the image for each direction, and is useful for capturing the contour mainly in a human image. The information processing apparatus 10 is not limited to the gradient direction histogram feature as a feature amount used for identifying a person. In addition to the gradient direction histogram feature, the information processing apparatus 10 may use, for example, a Haar-like feature, an LBPH feature (Local Binary Pattern Histogram), or a feature amount that is a combination thereof.

In step S604, the pattern identification unit 1413 performs the following process based on the feature amount extracted by the feature extraction unit 1412 in step S603 and the identification parameter acquired by the parameter acquisition unit 1414 in step S601. That is, the pattern identifying unit 1413 identifies whether or not the partial area image acquired by the partial image acquiring unit 1411 in S602 is a person. Non-Patent Document 1 discloses a method of learning a person classifier using a support vector machine and using it for identification. The information processing apparatus 10 also applies this method in this embodiment. However, Non-Patent Document 1 discloses a method for identifying a human image viewed from various directions with one recognition model. However, in the present embodiment, the information processing apparatus 10 uses a plurality of recognition models for each direction to be photographed, collates partial images to be collated with the respective recognition models, and photographs from the corresponding directions. Identified people were identified. In step S604, the pattern identification unit 1413 performs matching with one recognition model using the identification parameter acquired by the parameter acquisition unit 1414 in step S601. The pattern identification unit 1413 performs pattern identification processing by a linear support vector machine using, for example, the following Equation 1.
y = sign <x, w> (Formula 1)
In Equation 1, x is the gradient direction histogram feature extracted by the feature extraction unit 1412 in S603. Further, w is an identification parameter acquired by the parameter acquisition unit 1414 in S601, and is a vector having the same number of dimensions as x. <> Is an operator indicating an inner product operation of vectors. Further, sign is an operator representing a sign operation, and returns +1 for a positive value and -1 for a negative value. Moreover, y is an identification result, and if it is +1, it represents a person, and if it is -1, it represents a person other than a person. The pattern identifying unit 1413 identifies whether or not the image is a person based on the y value that is the calculation result. In addition to the identification method using Equation 1 in S604, the pattern identification unit 1413 may perform an identification method using a support vector machine using kernel calculation, an Adaboost classifier, a random classification tree, or the like. .

In S605, when the partial image acquired in S602 is identified as a person by the pattern identifying unit 1413 in S604, the detection result output unit 1415 positions the four vertices representing the rectangle of the partial image in the first image. Output coordinates.
In step S <b> 606, the partial image acquisition unit 1411 determines whether the processes in steps S <b> 603 to S <b> 605 have been performed for all partial images that can be acquired from the range indicated by the search range information acquired in step S <b> 601 in the first image. If the partial image acquisition unit 1411 determines that it has been performed, the process proceeds to S <b> 607. If it is determined that the partial image acquisition unit 1411 has not performed the process, the process proceeds to S602, and a partial image that has not yet been acquired is acquired.
In step S607, the parameter acquisition unit 1414 determines whether the processing in steps S601 to S606 has been performed for all recognition models. If it is determined that the parameter acquisition unit 1414 has performed, the process of FIG. 6 ends, and the process proceeds to S403. If it is determined that the parameter acquisition unit 1414 has not performed the parameter acquisition unit 1414, the process proceeds to S601, and an identification parameter or the like is acquired from a recognition model that has not been used yet.

In S403, the position estimation unit 160 estimates the position coordinates in the second image corresponding to the person detected by the detection unit 141 in S402 using the coordinate conversion parameters calculated by the positional relationship calculation unit 150. First, the position estimation unit 160 applies the coordinate transformation parameter to the position coordinates in the first image of the four vertices representing the rectangle of the partial image that is the person output by the detection unit 141, and applies the second image. Estimate each epipolar line inside. Then, the position estimation unit 160 performs correlation by the correlation method along the estimated epipolar line and calculates the position coordinates of the corresponding second images.
The position estimation unit 160 performs, for example, an epipolar line estimation using a coordinate transformation parameter disclosed in Non-Patent Document 4 and a correlation method using a correlation method. However, depending on the difference in the appearance of the person between the camera 111 and the camera 112, the association by the correlation method becomes difficult. In such a case, the image acquisition units 131 and 132 perform distance image acquisition in synchronization with the acquisition of the first image and the second image. And the position estimation part 160 estimates the position coordinate of the person area | region of a 2nd image based on a coordinate transformation parameter and the acquired distance image. As means for acquiring the distance image, for example, there are a distance sensor of a TOF (Time of Flight) method, a pattern light projection method, and the like.

Here, an example of a method in which the positional relationship calculation unit 150 calculates the coordinate conversion parameters in advance will be described.
First, the image acquisition unit 131 acquires an image of a calibration pattern whose shape captured by the camera 111 is known. The calibration pattern is, for example, a board on which dots as shown in FIG. 8 are drawn in a grid pattern, and the positional relationship between the points is known. The calibration pattern of FIG. 8 is arranged in the shooting scene, and the image acquisition unit 131 acquires an image captured by the camera 111. The imaging information acquisition unit 121 extracts position coordinates in the image of each point of the calibration pattern from the image acquired by the image acquisition unit 131. For example, the imaging information acquisition unit 121 displays the acquired image on the output device 14 and acquires the position coordinates by receiving the pointing of each point by the user via the input device 13. The position coordinate is the position of the center of gravity in the image of each point. Further, the imaging information acquisition unit 121 represents the position coordinates of each point as relative coordinates from the point O shown in FIG. The shooting information acquisition unit 121 acquires shooting information that associates the position coordinates of the image shot by the camera 111 with the three-dimensional space coordinates of the scene to be shot.

The calibration pattern is arranged as it is, and the imaging information acquisition unit 122 performs the following processing based on the image of the calibration pattern captured by the camera 112, as in the imaging information acquisition unit 121. That is, the shooting information acquisition unit 122 acquires shooting information that associates the position coordinates of the image shot by the camera 112 with the three-dimensional space coordinates of the scene to be shot.
Next, the positional relationship calculation unit 150 calculates a coordinate conversion parameter that associates the coordinate system in the first image with the coordinate system in the second image from the shooting information acquired by the shooting information acquisition units 121 and 122, respectively. The positional relationship calculation unit 150 calculates coordinate conversion parameters using, for example, the binocular camera calibration method disclosed in Non-Patent Document 4. In the present embodiment, the positional relationship calculation unit 150 performs coordinate system conversion on the assumption that the three-dimensional spatial coordinates and the positional coordinates in the image have a linear relationship. When is large, the photographing information and the positional relationship may be calculated in consideration of distortion.

In S404, the selection unit 190 selects a recognition model learned by the learning unit 180 based on the position coordinates estimated by the position estimation unit 160 in S403. The recognition model selected here is one of a plurality of recognition models managed by the detection unit 142. In the present embodiment, details of the detection unit 142 are the same as the details of the detection unit 141 shown in FIG. The detection unit 142 manages three recognition models, that is, a front model, a side model, and a planar model, as recognition models.
Hereinafter, a recognition model selection method according to the present embodiment will be described with reference to FIG. Points A, B, C, and D in FIG. 9 are vertices of a person area in the first image detected by the detection unit 141. Points A ′, B ′, C ′, and D ′ are vertices in the second image corresponding to the points A, B, C, and D estimated by the position estimation unit 160, respectively. In the present embodiment, the selection unit 190 estimates from which direction the person was photographed based on the aspect ratio of the area specified by the points A ′, B ′, C ′, and D ′, and in the estimated direction. Based on this, a recognition model is selected. The direction in which an object such as a person in the image is photographed is defined as the object photographing direction.
The area occupied by the person in the image is a vertically long area when the person is photographed from the front as compared to when photographed from above. In addition, the area occupied by the person in the image is an area having a longer vertical width than the horizontal width when shooting from the side, compared to when shooting from the front. Thus, the direction in which the person in the image is photographed is correlated with the aspect ratio of the area occupied by the person in the image.

The position coordinates of the points A ′, B ′, C ′, and D ′ in the second image are (uA, vA), (uB, vB), (uC, vC), and (uD, vD), respectively. In the present embodiment, the selection unit 190 calculates a pseudo aspect ratio R of the quadrangle A′B′C′D ′ using Equation 2 below.
R = | ((v_A + v_B) − (v_C + v_D)) / ((u_B + u_D) − (u_A + u_C)) | (Formula 2)
|| in Expression 2 is an operator representing an absolute value. Then, the selection unit 190 selects a recognition model corresponding to the calculated R. In the present embodiment, the storage device 12 stores in advance information indicating the range of R corresponding to each recognition model. The selection unit 190 determines which recognition model the calculated R value corresponds to based on information indicating the range of R corresponding to each recognition model stored in the storage device 12. The information indicating the range of R corresponding to each recognition model indicates that, for example, if the value of R is less than the set first threshold, it corresponds to a planar model. Further, if the information indicating the R range corresponding to each recognition model is less than the second threshold that is greater than the first threshold and greater than the first threshold, Indicates that the model is supported. Also, the information indicating the range of R corresponding to each recognition model indicates that it corresponds to the side model if the value of R is equal to or greater than the set second threshold.
In the example of FIG. 9, the detection unit 141 detects a person based on the front model. When the person area corresponds to a rectangle A′B′C′D ′ in FIG. 9, it is estimated that the camera 112 has photographed the person from above. This is because the aspect ratio R obtained from the quadrangle A′B′C′D ′ matches the aspect ratio of the planar model.

The selection unit 190 selects a recognition model corresponding to the calculated value of R, and determines the determined recognition model as a recognition model learned by the learning unit 180.
In the present embodiment, the selection unit 190 selects the recognition model based on the aspect ratio of the person area after conversion in the second image. However, for example, the selection unit 190 is based on the person area detected by the detection unit 141, the recognition model used for detection by the detection unit 141, and the coordinate conversion parameter calculated by the positional relationship calculation unit 150. The following processing may be performed. That is, the selection unit 190 estimates from which direction the person was photographed in the first image, based on the recognition model used for detection by the detection unit 141. Based on the position where the person is detected by the detection unit 141 and the coordinate conversion parameter calculated by the positional relationship calculation unit 150, the selection unit 190 determines whether the person in the first image has the orientation in the second image. Identify how it will change. Then, the selection unit 190 directly selects the second image based on the direction in which the person was captured in the specified first image and the change in the orientation of the person in the second image in the first image. The direction from which the image was taken is estimated. Then, the selection unit 190 may select a recognition model corresponding to the estimated direction, and may determine the selected recognition model as a recognition model learned by the learning unit 180.

  In S405, the extraction unit 170 extracts a partial image from the second image acquired by the image acquisition unit 132 in S401 based on the vertex coordinates estimated by the position estimation unit 160 in S403. The image extracted by this process is a partial image in the second image corresponding to the partial image detected as a person area from the first image in S402, and is an area in which a person may be captured. It is an image. It should be noted that the aspect ratio of the extracted partial image is a value corresponding to the recognition model selected in S404.

  In S406, the learning unit 180 learns the recognition model determined as the recognition model learned by the learning unit 180 by the selection unit 190 in S404 using the partial image extracted by the extraction unit 170 in S405. In the present embodiment, the learning unit 180 performs the same processing as the feature extraction unit 1412 from the partial image extracted by the extraction unit 170, extracts the gradient direction histogram feature, and adds the extracted gradient direction histogram feature to the additional learning sample. And Then, the learning unit 180 performs additional learning on the recognition model selected by the selection unit 190. The learning unit 180 performs additional learning using an additional learning method of a support vector machine disclosed in Non-Patent Document 5, for example. The learning process applied in S406 follows the identification process performed by the detection unit 142. For example, when the identification process by the detection unit 142 is an identification process using an Adaboost classifier, the learning unit 180 learns using online boosting disclosed in Non-Patent Document 6. As described above, in the present embodiment, the learning unit 180 selects the recognition model corresponding to the partial image extracted from the image captured by the camera 112 and performs additional learning. Therefore, the learning unit 180 performs additional learning of the corresponding recognition model with a sample that looks similar according to the direction in which the person is photographed, so that the advantage of the recognition model composed of a plurality of recognition models is not impaired. The accuracy of the identification process can be improved.

  In S407, the detection unit 142 detects a person from the second image. Details of the detection unit 142 are the same as the details of the detection unit 141 shown in FIG. The processing of the detection unit 142 is the same as the processing of the detection unit 141 shown in FIG. However, in the process of S407, the detection unit 142 uses the recognition model additionally learned in S406. Therefore, in the present embodiment, the detection unit 142 is adapted to the scene photographed by the camera 112 in consideration of the influence of the human image in the second image corresponding to the human region detected by the detection unit 141. In addition, since human detection processing is performed, an improvement in accuracy can be expected.

In S <b> 408, the integration unit 200 integrates the detection processing results by the detection unit 141 and the detection unit 142. The result of the detection process integrated in S408 is detected from the first image by the detection unit 141, and the person region in the second image estimated by the position estimation unit 160, and the second image by the detection unit 142. And the person area detected from When the person area is not detected or only one is detected, the integration unit 200 outputs the result as it is without performing any processing. When there are a plurality of detection results as a person region, the integration unit 200 detects the result by applying a process (Non-Maximum Suppression) that combines the overlapping regions disclosed in Non-Patent Document 1 into one. Integrate people areas.
For example, the integration unit 200 outputs the information by displaying information indicating the result of the integration process on the output device 14 of the information processing apparatus 10. For example, the integration unit 200 outputs information by storing information indicating the result of the integration process in the storage device 12. For example, the integration unit 200 outputs the information by transmitting information indicating the result of the integration process to the set transmission destination.

In step S409, the integration unit 200 determines whether shooting with the camera 111 and the camera 112 has ended. If the integration unit 200 determines that shooting by the camera 111 and the camera 112 has ended, the integration unit 200 ends the processing of FIG. If the integration unit 200 determines that shooting by the camera 111 and the camera 112 has not ended, the integration unit 200 proceeds to the process of S401.
With the above processing, the information processing apparatus 10 repeats the processing of S401 to S408 every time an image is captured by the camera 111 and the camera 112.

In the present embodiment, the information processing apparatus 10 learns a recognition model used for an image photographed by the camera 112 using a person detection result for an image photographed by the camera 111, improves detection accuracy, and It was decided to integrate the detection results. The information processing apparatus 10 may learn a recognition model used for an image photographed by the camera 111 using a result of person detection for the image photographed by the camera 112. In addition, the information processing apparatus 10 may learn the recognition model used for each camera by using the human detection results of the images captured by the cameras 111 and 112 mutually.
In the present embodiment, the recognition model includes a recognition model for each person image taken from different directions. Even when the recognition model is configured to include a recognition model for each of a high-resolution image and a low-resolution image of a person or a configuration including a recognition model for each hidden area where a person is hidden by another object, the information processing apparatus 10 The processing of this embodiment can be applied. The information processing apparatus 10 may perform the processing of this embodiment and select an appropriate recognition model for the learning sample for additional learning when learning the recognition model.

  As described above, in the present embodiment, the information processing apparatus 10 learns based on the detection result of the person for the image captured by the camera 111 and the correspondence between the visual field of the camera 111 and the visual field of the camera 112. A recognition model for an image taken by the camera 112 was determined. Then, the information processing apparatus 10 learns the determined recognition model using the partial image of the image captured by the camera 112 corresponding to the detection result of the person regarding the image captured by the camera 111. In this way, the information processing apparatus 10 can learn a recognition model for an image captured by the camera 112 more appropriately. Thereby, the information processing apparatus 10 can improve the accuracy of the person detection process from the image photographed by the camera 112.

<Embodiment 2>
In the first embodiment, the visual field of the camera 111 and the visual field of the camera 112 are overlapped in advance so that the same object can be photographed. However, when a new camera 112 is installed in a situation where the camera 111 is installed, the field of view of the camera 112 does not necessarily overlap so that the same object as the field of view of the camera 111 can be photographed.
Therefore, the information processing apparatus 10 according to the present embodiment performs processing for controlling the visual field of the camera 111 so that the visual fields of the camera 111 and the camera 112 are appropriately overlapped.
The hardware configuration of the information processing apparatus 10 of this embodiment is the same as that of the first embodiment. In the present embodiment, the CPU 11 performs processing according to the procedure of the program stored in the storage device 12, whereby the functions of the information processing device 10 described later in FIG. 10 and the processing related to flowcharts described later in FIGS. Etc. are realized.

FIG. 10 is a diagram illustrating an example of a functional configuration of the information processing apparatus 10 according to the present embodiment. The functional configuration of the information processing apparatus 10 according to the present embodiment illustrated in FIG. 10 is different from the functional configuration according to the first embodiment illustrated in FIG. 2 in that the PTZ control unit 300 is included.
The PTZ control unit 300 is a control unit that controls the drive system of the camera 111. Among the functional components in FIG. 10, those common to FIG. 2 are the same as those in FIG. 2.

Hereinafter, processing of the information processing apparatus 10 in the present embodiment will be described. FIG. 11 is a flowchart illustrating an example of processing of the information processing apparatus 10 according to the present embodiment. In the following, a description will be given of a process of optimizing the detection unit 142 of the camera 112 for a scene to be photographed and integrating the detection result of the camera 111 when the camera 112 is newly installed in a situation where the camera 111 is installed. To do.
In step S1101, the PTZ control unit 300 controls the field of view of the camera 111 by controlling the pan, tilt, and zoom drive systems of the camera 111 so that the field of view of the camera 111 and the field of view of the camera 112 overlap. .
In step S <b> 1102, the positional relationship calculation unit 150 calculates a coordinate conversion parameter that represents the positional relationship between the camera 111 and the camera 112. Details of S1101 to S1102 will be described later with reference to FIG. Thereafter, in the processing of S401 to S409, the information processing apparatus 10 performs processing based on the image captured by the camera 111 whose field of view is controlled in S1102. The processing of S401 to S409 is the same as that of the first embodiment.

Details of the PTZ control unit 300 are shown in FIG.
The PTZ control unit 300 includes a corresponding point extraction unit 310, an overlapping region evaluation unit 320, and a control signal generation unit 330. The corresponding point extraction unit 310 extracts corresponding points between the image captured by the camera 111 and the image captured by the camera 112. The overlapping area evaluation unit 320 evaluates the size of the overlapping area between the image captured by the camera 111 and the image captured by the camera 112 based on the points extracted by the corresponding point extraction unit 310. To do. The control signal generation unit 330 generates a control signal for controlling the visual field of the camera 111 based on the evaluation by the overlapping region evaluation unit 320.
The details of the PTZ control process performed by the PTZ control unit 300 are shown in FIG. Hereinafter, the details of the PTZ control process of S1101 will be described with reference to FIG.
In step S <b> 1301, the corresponding point extraction unit 310 acquires an image captured by the camera 112 from the image acquisition unit 132.

In S1302, the corresponding point extraction unit 310 extracts a local feature amount from the image acquired in S1301. The local feature amount is a feature that can be distinguished from other parts such as an edge extracted by paying attention to a local region in the image. In the present embodiment, the corresponding point extraction unit 310 extracts SIFT feature values from the image acquired in S1301. The SIFT feature value is a feature value obtained by calculating a brightness gradient for each direction from a region near the position where the brightness distribution in the image takes an extreme value and forming a histogram, and is excellent in invariance to image shift, scaling, and rotation. It is. The corresponding point extraction unit 310 temporarily stores the local feature amount extracted in S1302 in the storage device 12 during the PTZ control process.
In step S <b> 1303, the corresponding point extraction unit 310 acquires an image captured by the camera 111 from the image acquisition unit 131. Hereinafter, the image captured by the camera 111 acquired in S1303 is referred to as a third image. Hereinafter, the image captured by the camera 112 acquired in S1301 is referred to as a fourth image.
In S1304, the corresponding point extraction unit 310 extracts a local feature amount from the image acquired in S1303 by the same processing as in S1302.

In S1305, the corresponding point extraction unit 310 extracts corresponding points from the third image and the fourth image based on the local feature amount extracted in S1302 and S1304. For example, the corresponding point extraction unit 310 calculates a similarity between a local feature corresponding to a certain point in the third image and a local feature corresponding to a certain point in the fourth image, and the similarity is predetermined. If it is equal to or greater than the threshold value, the points are associated. The associated point is set as a corresponding point. The corresponding point extraction unit 310 uses, for example, an inner product value between vectors or a histogram intersection as the similarity between local feature amounts. In addition, the corresponding point extraction unit 310 can delete the corresponding points with low reliability by the RANSAC algorithm with respect to the extracted set of corresponding points.
Details of the processing of the corresponding point extraction unit 310 described above are disclosed in Non-Patent Document 7.

In S1306, the overlapping area evaluation unit 320 overlaps the third image and the fourth image from the set of corresponding points between the third image and the fourth image extracted in S1305. Evaluate the size of. Hereinafter, a method for evaluating the overlapping area will be described.
Assume that a certain person is photographed by the camera 111 and the camera 112. An example of the third image and the fourth image in that case is shown in FIGS. 14A and 14B, respectively. In the example of FIG. 14, (P11, P21), (P12, P22), (P13, P23), (P14, P24), (P15, P25) are a set of corresponding points.
An area indicated by R1 in FIG. 14C is an example of an overlapping area between the third image and the fourth image. Based on the set of corresponding points, the overlapping area evaluation unit 320 acquires an overlapping area R1 between the third image and the fourth image, for example, by the following process. In other words, the overlapping area evaluation unit 320 calculates a conversion parameter from a set of corresponding points to a position coordinate in the fourth image from a position coordinate in the fourth image. Then, the overlapping region evaluation unit 320 converts the position coordinates of the four corner points of the fourth image into the position coordinates in the third image, and the region surrounded by the four converted points and the third image Is obtained as an overlapping area between the third image and the fourth image. In addition, the overlapping area evaluation unit 320 may acquire a set area that includes all corresponding points in the third image as an overlapping area between the third image and the fourth image. That is, the overlapping area evaluation unit 320 estimates an area that can be captured by the camera 112 in the third image from the coordinates of the corresponding points, and obtains the area thereof.
Then, when the overlap area size between the acquired third image and the fourth image is larger than the set threshold value, the overlap area evaluation unit 320 sufficiently overlaps the fields of view of the camera 111 and the camera 112. If so, the PTZ control process in FIG. 13 is terminated. In addition, when the size of the overlap region between the acquired third image and the fourth image is equal to or less than a set threshold, the overlap region evaluation unit 320 sufficiently overlaps the fields of view of the camera 111 and the camera 112. If not, the process proceeds to S1307.

In S <b> 1307, the control signal generation unit 330 obtains the PTZ drive amount of the camera 111 based on the overlapping area acquired in S <b> 1306 so that the overlap between the visual field of the camera 111 and the visual field of the camera 112 increases. For example, in the situation as shown in FIG. 14C, the control signal generation unit 330 drives the pan and tilt of the imaging system of the camera 111 so that the field of view of the camera 111 becomes the center of the field of view R1. Ask for. Further, the control signal generation unit 330 obtains the zoom driving amount of the camera 111 so that the region R1 covers the entire field of view.
In step S <b> 1308, the control signal generation unit 330 sends a control signal for driving the imaging system to the camera 111 according to the PTZ drive amount obtained in step S <b> 1307. The camera 111 drives the pan, tilt, and zoom mechanisms of the imaging system in accordance with the created PTZ drive amount control signal. Then, the control signal generation unit 330 proceeds to the process of S1301. Since the subject may move while controlling the orientation of the camera 111, the process returns to S1301 instead of S1303.
As described above, the information processing apparatus 10 controls the field of view of the camera 111 by controlling the pan, tilt, and zoom drive systems of the camera 111 so that the fields of view of the camera 111 and the camera 112 overlap by the processing of FIG. To do. An image photographed by the camera 111 after the processing of FIG. 13 is a scene as shown in FIG. 15, for example, and an entire image of a person is captured, and the scene overlaps with FIG. I understand.

  In step S1102, the positional relationship calculation unit 150 performs processing similar to that in the first embodiment after the processing illustrated in FIG. 13 and coordinates from the coordinate system of the image captured by the camera 111 to the coordinate system of the image captured by the camera 112. Obtain conversion parameters. Further, the positional relationship calculation unit 150 extracts corresponding points from the image captured by the camera 111 and the image captured by the camera 112, for example, by the same processing as in S1305. Then, the positional relationship calculation unit 150 calculates a conversion parameter from the position coordinates of the image photographed by the camera 111 to the position coordinates of the image photographed by the camera 112 based on the extracted position coordinates of the corresponding points. It is good.

As described above, the information processing apparatus 10 can appropriately overlap the visual fields of the camera 111 and the camera 112 by the processing of this embodiment. For example, when a camera 112 is newly installed in a situation where the camera 111 is already installed, it is unclear whether or not the fields of view of the installed camera 112 and the existing camera 111 overlap. Even in such a case, the information processing apparatus 10 can appropriately overlap the visual fields of the camera 111 and the camera 112 so that the same object can be photographed.
In the present embodiment, the camera 111 is controlled by PTZ so that the field of view overlaps with the camera 112. However, when a plurality of cameras are already installed in the vicinity of the camera 112, the information processing apparatus 10 The process of this embodiment can be applied by selecting an optimal camera. Further, the information processing apparatus 10 may perform additional learning by controlling a plurality of cameras to overlap the visual field.

<Other embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
As mentioned above, although preferable embodiment of this invention was explained in full detail, this invention is not limited to the specific embodiment which concerns.
For example, a part or all of the functional configuration of the information processing apparatus 10 described above may be implemented in the information processing apparatus 10 as hardware.

10 Information processing apparatus 11 CPU
111 camera 112 camera

Claims (9)

A correspondence between a first image in which an object is captured by a first imaging device, a coordinate system of the first image, and a coordinate system of a second image in which the object is captured by a second imaging device; , Based on the estimation means for estimating the shooting direction of the object in the second image,
Determining means for determining a recognition model to be used for detection of the object in the second image based on the shooting direction estimated by the estimating means;
Learning means for learning the recognition model determined by the determining means based on an image of the object included in the second image;
An information processing apparatus. The image processing apparatus further includes an acquisition unit configured to acquire the object area in the second image based on the object area in the first image and the correspondence relationship.
The information processing apparatus according to claim 1, wherein the estimation unit estimates a shooting direction of the object in the second image based on the region acquired by the acquisition unit.   The information processing apparatus according to claim 2, wherein the object is a person, and the estimation unit estimates a shooting direction of the object in the second image based on an aspect ratio of the area of the person in the second image. .   The information processing apparatus according to claim 2, wherein the learning unit learns the recognition model determined by the determination unit based on a portion of the area acquired by the acquisition unit in the second image.   The determination unit determines the recognition model of the learning target by selecting the recognition model of the learning target from a plurality of set recognition models based on the shooting direction estimated by the estimation unit. The information processing apparatus according to any one of 1 to 4.   Based on the third image photographed by the first imaging means and the fourth image photographed by the second imaging means, the visual field of the first imaging means and the second imaging The information processing apparatus according to claim 1, further comprising a control unit that controls a field of view of the first imaging unit so that a region overlapping with the field of view of the unit increases.   The control unit determines whether the field of view of the first imaging unit and the field of view of the second imaging unit are based on the local feature amount in the third image and the local feature amount in the fourth image. The information processing apparatus according to claim 6, wherein a visual field of the first imaging unit is controlled so that an overlapping area increases. An information processing method executed by an information processing apparatus,
A correspondence between a first image in which an object is captured by a first imaging device, a coordinate system of the first image, and a coordinate system of a second image in which the object is captured by a second imaging device; Based on the estimation step of estimating the shooting direction of the object in the second image;
A determination step of determining a learning target recognition model used for detection of the object in the second image based on the shooting direction estimated in the estimation step;
A learning step of learning the recognition model determined in the determination step based on an image of the object included in the second image;
An information processing method including:   The program for functioning a computer as each means of the information processing apparatus of any one of Claims 1 thru | or 7.

Images