JP2019008480A - Image analysis program, image analysis apparatus, and image analysis method - Google Patents

Abstract

To correlate objects with each other even when an appropriate feature amount cannot be obtained from an object region.SOLUTION: An image analysis apparatus 10 determines a type of a feature quantity to be used for associating a person region among a plurality of images according to a state of the person region detected from each of the plurality of images. Then, the image analysis apparatus 10 specifies a corresponding person region out of each of the plurality of images, according to a matching degree of the feature quantity of the type extracted from each of the person regions.SELECTED DRAWING: Figure 1

Description

  The disclosed technology relates to an image analysis program, an image analysis apparatus, and an image analysis method.

  There is known an information processing apparatus that estimates a correspondence between persons appearing in a video. This information processing apparatus was photographed by a plurality of video cameras at a second time later than the first time, and at least one person appearing in images captured by the plurality of video cameras at a first time. A scale indicating the possibility that one or more persons appearing in the video are associated with each other is determined. Then, the information processing apparatus considers all the scales related to at least one person shown at one of the first time and the second time and all persons related to the other time, The correspondence relationship between the person shown at the time and the person shown at the second time is estimated.

  There is also known an information processing apparatus that reduces the amount of data transmitted in order to derive the movement trajectory of a subject based on images captured by imaging apparatuses installed in a plurality of places. The plurality of information processing devices extract a plurality of types of feature amounts for the first subject imaged by the first imaging device corresponding to the information processing device, and determine subject differences for each of the plurality of types of feature amounts. A part of the feature quantity is selected by determining the suitability to do so. Then, the information processing apparatus transmits information for identifying the selected feature amount to the other information processing apparatus, and the second subject imaged by the second imaging apparatus corresponding to the other information processing apparatus The feature amount indicated by the transmitted information extracted for is received. Then, the information processing apparatus determines the difference between the first and second subjects based on the feature amount extracted in the extracting process and the received feature amount.

International Publication No. 2014/050432 JP 2017-63266 A

  When associating objects reflected in different images using image analysis, there is a case where the objects cannot be associated with each other because an appropriate feature amount cannot be obtained from the object region in the image.

  An object of the disclosed technique is to associate objects with each other even when an appropriate feature amount cannot be obtained from the object region.

  As one aspect, the disclosed technology determines the type of feature amount used for the association of the object areas in accordance with the state of the object area detected from each of the plurality of images. Then, the corresponding object region of each of the plurality of images is specified according to the degree of coincidence of the types of the feature amounts extracted from each of the object regions.

  As one aspect, there is an effect that the objects can be associated with each other even when an appropriate feature amount cannot be obtained from the object region.

It is a functional block diagram which shows schematic structure of the image analysis apparatus which concerns on embodiment. It is explanatory drawing for demonstrating the outline | summary of the process of this embodiment. It is explanatory drawing for demonstrating specification of a person area. It is a figure which shows an example of the table stored in a detection result memory | storage part. It is explanatory drawing for demonstrating the tracking process of a person area. It is a figure which shows an example of the table stored in a tracking result memory | storage part. It is explanatory drawing for demonstrating the integration process of a detection result and a tracking result. It is explanatory drawing for demonstrating a coordinate system. It is explanatory drawing for demonstrating the overlap of a person area | region. It is a figure which shows an example of the table stored in a direction overlap result memory | storage part. It is explanatory drawing for demonstrating the determination process of a matching candidate. It is explanatory drawing for demonstrating the setting of the order of matching. 1 is a block diagram illustrating a schematic configuration of a computer that functions as an image analysis apparatus according to an embodiment. It is a flowchart which shows an example of the image analysis process in this embodiment. It is a flowchart which shows an example of the person detection process in this embodiment. It is a flowchart which shows an example of the person tracking process in this embodiment. It is a flowchart which shows an example of the integration process in this embodiment. It is a flowchart which shows an example of the person direction determination process in this embodiment. It is a flowchart which shows an example of the overlap determination process in this embodiment. It is a flowchart which shows an example of the matching candidate determination process in this embodiment. It is a flowchart which shows an example of the feature distribution production | generation process in this embodiment. It is a flowchart which shows an example of the feature selection process in 1st Embodiment. It is a flowchart which shows an example of the process order determination process in this embodiment. It is a flowchart which shows an example of the matching process in this embodiment. It is explanatory drawing for demonstrating the feature selection in 2nd Embodiment. It is a flowchart which shows an example of the feature selection process in 2nd Embodiment. It is explanatory drawing for demonstrating the determination of the process order in 3rd Embodiment.

  Hereinafter, an exemplary embodiment related to the disclosed technology will be described in detail with reference to the drawings. In the present embodiment, a case where the target object is a person will be described as an example.

<First Embodiment>

  As shown in FIG. 1, the image analysis apparatus 10 according to the first embodiment includes a camera A, a control unit 14, and a display device 48.

  The camera A is installed at a predetermined location as a surveillance camera, for example, and sequentially captures moving images at the predetermined location. In the present embodiment, a case where the camera A is installed so as to look down on the surrounding area will be described as an example.

  The control unit 14 includes a person detection unit 16, a detection result storage unit 18, a person tracking unit 20, a tracking result storage unit 22, an integration unit 24, an integration result storage unit 26, a person direction determination unit 28, An overlap determination unit 32 and an orientation overlap result storage unit 30 are provided. In addition, the control unit 14 includes an other camera detection result acquisition unit 34, an other camera detection result storage unit 36, an association candidate determination unit 38, a feature distribution generation unit 40, a feature selection unit 42, and a processing order determination unit. 44 and an association unit 46. The feature distribution generation unit 40 and the feature selection unit 42 are examples of a disclosure technology determination unit, and the association unit 46 is an example of a disclosure technology identification unit.

  FIG. 2 is an explanatory diagram for explaining processing of the image analysis apparatus 10 of the present embodiment. The image analysis apparatus 10 associates each person area included in the frame image captured by the camera A with each person area included in the frame image captured by another camera B different from the camera A.

  When associating human regions in moving images captured by different cameras, the human regions may not be associated with each other because appropriate feature amounts are not extracted from the individual regions.

  Therefore, when the image analysis apparatus 10 according to the present embodiment associates the person regions, the type of feature amount used for the association of the person regions is determined according to the state of the person region detected from each of the plurality of images. To do. The image analysis apparatus 10 according to the present embodiment extracts each of the plurality of images according to the degree of coincidence of the types of feature amounts extracted from each person region and determined to be used for matching the person regions. The corresponding person area is specified.

  Specifically, the image analysis apparatus 10 determines the orientation of the person indicated by the person area of each frame image of the moving image captured by the camera A and the person area of each frame image of the moving image captured by the other camera B. The type of feature amount is selected according to the orientation of the person indicated by. In the present embodiment, an example will be described in which the color of the upper body area of the person area, the color of the lower body area of the person area, the color of the head area of the person area, and the color of the shoe area of the person area are used as feature amounts.

  For example, as shown in FIG. 2, the orientation of the person JB1 in the person area of the frame image taken by another camera B and each person JA1, JA2, JA3 in the person area of the frame image taken by the camera A The type of feature quantity is selected according to the direction of JA4 and JA5. In the example shown in FIG. 2, the image analysis apparatus 10 does not extract the feature quantity of the head area and the upper body area of the person JA2 because the orientation of the person JB1 and the orientation of the person JA2 are different. Extract features of lower body area and shoe area. The image analysis apparatus 10 projects each feature quantity extracted from each of a plurality of person regions detected from one frame image onto the feature space for each type of feature quantity, and distributes the feature quantity for each type. Is generated. For example, as shown in FIG. 2, the image analysis apparatus 10 includes a color feature amount distribution F1 of the upper body region of the person region, a color feature amount distribution F2 of the lower body region of the person region, and a head region of the person region. A color feature amount distribution F3 and a color feature amount distribution F4 of the shoe region of the person region are generated. One point in the distribution of each type of feature value corresponds to the feature value of the corresponding type extracted from one person region.

  Further, in the present embodiment, when the person areas are associated, the type of feature amount is selected according to the overlap of the person areas of the frame images of the moving image captured by the camera A.

  For example, in the example shown in FIG. 2, the image analysis apparatus 10 has an overlap in the lower body area of the person area of the person JA3. Therefore, without extracting the feature amount of the lower body area, the upper body area, the hair area, and The feature value of the shoe region is extracted. Further, since there is an overlap in the head area of the person area of the person JA4, the image analysis apparatus 10 does not extract the feature quantity of the head area, and calculates the feature quantities of the upper body area, the lower body area, and the shoe area. Extract.

  The distribution of the feature amount generated by the image analysis apparatus 10 that analyzes the moving image captured by the camera A is transmitted to an image analysis apparatus (not shown) that analyzes the moving image captured by the other camera B, and the person Used for area association.

  Hereinafter, the configuration of the image analysis apparatus 10 of the present embodiment will be specifically described.

  The person detection unit 16 detects a person region from each frame image of the moving image captured by the camera A. For example, the person detection unit 16 detects a person region from each frame image of a moving image using a method described in a patent document (Japanese Patent Laid-Open No. 2013-117865).

  FIG. 3 is an explanatory diagram for explaining an example of detection of a person area in the person detection unit 16. As shown in FIG. 3, when the frame image It-1 at time t-1 and the frame image It at time t are input, the person detection unit 16 and the frame image It-1 at time t-1 and the time A background difference Dif from the frame image It of t is obtained.

  Next, the person detection unit 16 specifies a person area candidate Ca based on the background difference Dif. Next, the person detection unit 16 inputs the person area candidate Ca from the correct person area to a discriminator learned in advance, and obtains a determination result indicating whether the person area candidate Ca is a person area. Then, the person detection unit 16 obtains a person detection result Re determined to be a person area.

  Then, the person detection unit 16 assigns a person area ID to the detection frame of the area determined as the person area. The person detection unit 16 then detects the person area ID, the image information in the person area detection frame, the upper left x coordinate of the person area detection frame, the upper left y coordinate of the person area detection frame, and the person area detection frame. The width and the height of the detection frame of the person area are taken as a person detection result. The person detection unit 16 stores the person detection result in the detection result storage unit 18.

  The detection result storage unit 18 stores a person detection result detected by the person detection unit 16. The person detection result is stored in the form of a table, for example, as shown in FIG. The table shown in FIG. 4 stores the person detection result detected by the person detection unit 16 in association with the image ID of the frame image in which the person region is detected and the imaging time of the frame image.

  The person tracking unit 20 tracks the person area of the same person according to the position of the person area in each frame image.

  FIG. 5 shows an explanatory diagram for explaining the tracking processing of the same person. As illustrated in FIG. 5, the person tracking unit 20 sets a person region detected from the frame image It-1 at time t−1 as a template used for tracking. FIG. 5 shows an example in which a template T1 and a template T2 are set.

  Then, the person tracking unit 20 specifies a person region similar to the template T1 from the frame image It at time t by template matching. Further, the person tracking unit 20 specifies a person area similar to the template T2 from the frame image It at time t by template matching. When performing template matching, for example, a normalized cross-correlation function R (x, y) represented by the following equation (1) can be used as the similarity. In this case, the person tracking unit 20 identifies, as a person area, an area in which the normalized cross-correlation function R (x, y) is greater than a threshold among areas of the frame image at time t. Then, for the person area of the frame image at time t that is determined to be the same as the person area of the frame image at time t, the same person area ID as the person area ID of the person area of the frame image at time t is set. Give.

(1)

  In the above formula (1), T (x ′, y ′) represents the pixel value of the pixel (x ′, y ′) in the template, h represents the height of the template, and w represents the width of the template. I represents a frame image at time t. (X, y) represents the position on the xy coordinate of the pixel of the frame image.

  Thereby, the same person region is specified from the frame image at time t−1 and the frame image at time t. Then, the person tracking unit 20 stores the tracking result in the tracking result storage unit 22.

  The tracking result storage unit 22 stores the tracking result obtained by the person tracking unit 20. The tracking result is stored in the form of a table, for example, as shown in FIG. The table shown in FIG. 6 includes the person area ID, the image information in the person area detection frame, the upper left x coordinate of the person area detection frame, the upper left y coordinate of the person area detection frame, and the person area detection frame. The width and the height of the detection frame of the person area are stored in association with each other. Like the shaded portion shown in the table of FIG. 6, the same person area ID is assigned to the person areas determined to be the same person at different times.

  The integration unit 24 integrates the detection result stored in the detection result storage unit 18 and the tracking result stored in the tracking result storage unit 22. In the detection process using one of the detection result of the person area and the tracking result of the person area, a detection error of the person area may occur. Therefore, in the present embodiment, the person area is specified with high accuracy by combining the person area detection process by the person detection unit 16 and the tracking process by the person tracking unit 20.

  Specifically, for each frame image, the integration unit 24 includes the detection frame of the person area in the detection result stored in the detection result storage unit 18 and the tracking result stored in the tracking result storage unit 22. The detection result and the tracking result are integrated according to an overlapping area between the detection frames of the person area.

  FIG. 7 is an explanatory diagram for explaining the integration of the detection result and the tracking result. As shown in FIG. 7, in the frame image It at time t, when the overlapping rate between the detection result detection frame KR and the tracking result detection frame TR is larger than the threshold value, those areas are the same person. As a result, the detection result and the tracking result are integrated. In this embodiment, when the duplication rate is larger than the threshold value, priority is given to the information stored in the tracking result.

  On the other hand, as shown in FIG. 7, when there is no overlap between the detection frame KR of the detection result and the detection frame TR of the tracking result (in the example of FIG. 7, there is no tracking result), a new one is newly added. As a person area corresponding to an appearing person, the person area corresponding to the detection frame in which the person is detected. Then, the integration unit 24 stores the integration result in the integration result storage unit 26.

  The integration result storage unit 26 stores the integration result obtained by the integration process of the integration unit 24. As the integration result, for example, information in a table format as shown in FIG. 4 or FIG. 6 is stored.

  The person direction determination unit 28 estimates the moving direction of each person area based on each person area of the same person associated between the frame images stored in the integration result storage unit 26. Then, the person direction determination unit 28 specifies the direction of each person area based on the movement direction of each person area of the same person. In the present embodiment, the coordinate system in the image is set as shown in FIG. In the present embodiment, a case where a person moves toward the near side or the far side in the depth direction of the imaging range of the camera will be described as an example. Therefore, the moving direction of the person on the image corresponds to the y-axis direction.

  Specifically, the person direction determination unit 28 reads each person area of the frame image at each time from the integration result stored in the integration result storage unit 26. Then, the person direction determination unit 28 sets the y coordinate value of the person area of the frame image at time t−1 as p (t−1) for each person area having the same person area ID, and the person area at time t. Let y (y) coordinate value be p (t). In addition, the person direction determination unit 28 determines the moving direction of the person appearing in the person area according to the following equations (2) to (4). Then, the person direction determination unit 28 determines the direction of the person according to the moving direction of the person shown in the person area.

p (t) -p (t-1) <0 (2): the person faces backward p (t) -p (t-1)> 0 (3): the person faces forward p (t) -p (t-1) ) = 0 (4): Same direction as determined in frame image at time t−1

  When the above formula (2) is satisfied, the person direction determination unit 28 indicates that the person area is moving upward on the frame image, and the person reflected in the person area is moving toward the back side. Therefore, it is determined that the person shown in the person area is facing backward.

  Further, the person direction determining unit 28 determines that the person area is moving downward on the frame image and the person appearing in the person area is moving toward the front when the above expression (3) is satisfied. Therefore, it is determined that the person shown in the person area is positive.

  Further, when the above formula (4) is satisfied, the person direction determination unit 28 does not change the y coordinate from the previous time t−1, and here, the person direction determination unit 28 has the same direction as the direction at the previous time t−1. It is determined that Then, the person direction determination unit 28 stores the orientation of the person area determined for each person area in the orientation overlap result storage unit 30.

  Based on the integration result stored in the integration result storage unit 26, the overlap determination unit 32 determines whether or not there is an overlap between the person regions for each person region. FIG. 9 shows an explanatory diagram for explaining the overlapping of the person regions.

9, the minimum value of the x coordinate of the detection frame KA of the person area JA is x1 _s , the maximum value of the x coordinate is x1 _e , and the minimum value of the y coordinate of the detection frame KA of the person area JA is y1 _s. The maximum value is y1 _e . Further, the minimum value of the x coordinate of the detection frame KB of the person area JB is x2 _s , the maximum value of the x coordinate is x2 _e , the minimum value of the y coordinate of the detection frame KB of the person area JB is y2 _s , and the maximum value is y2 _e. And In this case, the overlap determination unit 32 determines whether or not there is an overlap between the person regions according to the following equations (5) to (12).

x1 _s <x2 _s <x1 _e (5)
x1 _s <x2 _e <x1 _e (6)
y1 _s <y2 _s <y1 _e (7)
y1 _s <y2 _e <y1 _e (8)

x2 _s <x1 _s <x2 _e (9)
x2 _s <x1 _e <x2 _e (10)
y2 _s <y1 _s <y2 _e (11)
y2 _s <y1 _e <y2 _e (12)

  When the overlap determination unit 32 satisfies at least one of the above formula (5) and the above formula (6) and satisfies at least one of the above formula (7) and the above formula (8), the person area JA and the person area It is determined that there is an overlap with JB. In addition, the overlap determination unit 32 satisfies the human area JA when satisfying at least one of the above formula (9) and the above formula (10) and at least one of the above formula (11) and the above formula (12). It is determined that there is an overlap with the person area JB. Then, the overlap determination unit 32 stores the overlap determination result in the orientation overlap result storage unit 30.

  The direction overlap result storage unit 30 stores information on the orientation of each person area obtained by the person direction determination unit 28 and information on the overlap of each person area obtained by the overlap determination unit 32. It should be noted that the four corner coordinates of the overlapping area of the person area are stored as overlapping information. The orientation overlap result storage unit 30 stores information in a table format as shown in FIG. 10, for example. As shown in FIG. 10, the information on the orientation of the person area and the information on the overlap of the person areas are added to the table stored in the integration result storage section 26 in the table of the orientation overlap result storage section 30.

  The other camera detection result acquisition unit 34 acquires information on the person area of each frame image of the moving image captured by the other camera B representing a camera different from the camera A. The information acquired by the other camera detection result acquisition unit 34 is the same information as the information stored in the orientation overlap result storage unit 30. Further, the information acquired by the other camera detection result acquisition unit 34 is information obtained by an image analysis device (not shown) that analyzes the moving image of the other camera B.

  The other camera detection result storage unit 36 stores information in the same format as the table acquired by the other camera detection result acquisition unit 34 and stored in the orientation overlap result storage unit 30. In the following, a person region included in each frame image of a moving image captured by the camera A is a person region X, and a person region included in each frame image of a moving image captured by another camera B is an association candidate Y. And

  The association candidate determination unit 38 sequentially reads the person area X corresponding to the newly appearing person stored in the orientation overlap result storage unit 30. Next, for each of the read person areas X, the association candidate determination section 38 stores the other camera detection result storage section 36 existing within a predetermined past time range from the imaging time of the frame image including the person area X. The person area is set as a matching candidate Y for the person area X.

  FIG. 11 is an explanatory diagram for explaining the processing of the association candidate determination unit 38. As shown in FIG. 11, for example, consider a case where a person JA1 is newly imaged by the camera A. In this case, it is assumed that the person areas JB1, JB2, JB3, and JB4 imaged by the camera B are the same person candidates of the person JA1 within a predetermined time range range from the time t when the person JA1 was imaged by the camera A. It is set as the association candidate Y. One person area ID corresponds to one person area X1.

  The feature distribution generation unit 40 determines the type of feature amount used for the association of the person areas according to the orientation of the person area and the overlap in the person area, which is the state of the person area. Specifically, the feature distribution generation unit 40 determines a feature amount extracted from a region different from the overlapping region of the person regions as a feature amount used for associating the object region. In addition, when the orientations of one person area and the other person area are not the same, the feature distribution generation unit 40 extracts feature amounts extracted from areas different from the head and upper body of the person area. It is determined as a feature value used for association. In the present embodiment, the feature distribution generation unit 40 extracts the color of the upper body area of the person area, the color of the lower body area of the person area, the color of the shoe area of the person area, and the color of the hair area of the person area as feature amounts. To do.

  Therefore, the feature distribution generation unit 40, for each pair of the person region X and the association candidate Y, the orientation of the person region X and the orientation of the association candidate Y stored in the orientation overlap result storage unit 30, and the person Based on the overlap of the region X, the type of feature amount used for the association is determined.

  Specifically, the feature distribution generation unit 40 extracts feature amounts from each person region according to the following table. When a person region having the same person region ID exists in each of the frame images at a plurality of times within the predetermined time range, the feature amount is extracted from the person region X of the frame image at the latest time.

  The feature distribution generation unit 40 matches the person region X and the association candidate Y when the orientation of the person region X matches the orientation of the association candidate Y and there is no overlap in the person region X and the association candidate Y. All types of feature quantities are extracted from the candidate Y.

  Also, the feature distribution generation unit 40, when the orientation of the person region X and the orientation of the association candidate Y are different and there is no overlap in the person region X and the association candidate Y, the person region X and the association candidate From the candidate Y, the feature amount of the lower body region and the feature amount of the shoe region are extracted.

  In addition, the feature distribution generation unit 40, when the orientation of the person region X matches the orientation of the association candidate Y and there is an overlap in at least one of the person region X and the association candidate Y, the person region X From X and the association candidate Y, a feature amount of a region where there is no overlap is extracted.

  The feature distribution generation unit 40 is set in advance when the orientation of the person region X does not match the orientation of the association candidate Y and there is an overlap in at least one of the person region X and the association candidate Y. The feature value of the selected area. Specifically, the feature distribution generation unit 40 extracts a feature amount of a region that is at least one of the feature amount of the lower body region and the feature amount of the shoe region and has no overlap.

  Further, the feature distribution generation unit 40 generates a distribution of each type of feature amount extracted from each association candidate Y. The feature distribution generation unit 40 generates a distribution of each type of feature quantity so that one type of feature quantity extracted from one association candidate Y becomes one point. Then, the feature distribution generation unit 40 calculates the distribution of each type of feature amount distribution.

  The feature selection unit 42 further determines the type of feature amount used for the association of the person regions according to the distribution state of the feature amount extracted from each of the plurality of person regions, which is the state of the person region. Specifically, the feature selection unit 42 selects, for each of a plurality of person regions, the type of feature amount used for matching the person regions according to the distribution of the feature amounts extracted from each of the plurality of person regions. decide.

  More specifically, the feature selection unit 42 selects, as a feature amount to be used for association, a type of feature amount in which the distribution of the feature amount distribution calculated by the feature distribution generation unit 40 is larger than a threshold value. It can be considered that the greater the variance of the feature amount distribution, the more suitable the feature amounts of the person regions are arranged in the feature amount distribution, which is more suitable for the association of the person regions. For this reason, the feature selection unit 42 selects a feature amount of a type in which the distribution of the feature amount distribution is larger than the threshold value as the feature amount used for the association.

  The processing order determination unit 44 determines the order of collation between person areas. Specifically, the processing order determination unit 44 sets the order of association of the plurality of person areas according to the distribution of the plurality of feature amounts extracted from the person area. For example, it is determined that the processing order determination unit 44 generates a distribution of each type of feature amount extracted from each frame image for a plurality of times for each association candidate Y and uses it for associating person regions. The matching is performed in order from the association candidate Y with less variation in the types of feature amounts. Here, “there is little variation” means that the variance of the feature amount extracted from the person area of each frame image is small. In accordance with the order determined by the processing order determination unit 44, collation is performed for each pair of the person region X and the association candidate Y.

  FIG. 12 is a diagram for explaining the processing of the processing order determination unit 44. As shown in FIG. 12, a case where the person A1 and the person B1 are collated and the person A1 and the person B2 are collated will be described as an example. In the example shown in FIG. 12, the feature amount distribution extracted from the candidate for association of the person B1 detected from each frame image includes the feature amount distribution FP1 of the upper body region, the feature amount distribution FP2 of the lower body region, shoes A distribution FP3 of the feature amount of the region and a distribution FP4 of the feature amount of the hair region. In each feature quantity distribution, one point represents one type of feature quantity extracted from one frame image association candidate.

  As a result of selecting the type of feature amount by the feature selection unit 42, when it is determined that the matching is performed using the feature amount of the upper body region, the processing order determination unit 44 distributes the feature amount distribution F1 of the upper body region of the person B1. And the variance of the feature amount distribution F1 in the upper body region of the person B2. Then, the processing order determination unit 44 sets the order so that the person B2 with the smaller variance of the distribution of the feature quantity of the upper body is collated with the person A1 first. It can be considered that the feature quantity is more stably extracted as the variance of the distribution of the feature quantity extracted from the frame image correspondence candidate Y at each time is smaller. For this reason, the processing order determination unit 44 performs setting so that the association candidate Y having the smaller variance of the distribution of the feature amount extracted from the association candidate Y of the frame image at each time is processed earlier. In the example shown in FIG. 12, the collation order is such that the collation between the person A1 and the person B1 is performed after the collation between the person A1 and the person B2.

  The associating unit 46 associates the person region X with the associating candidate Y based on the type of feature amount selected by the feature selecting unit 42 according to the collation order determined by the processing order determining unit 44. I do.

  Specifically, the associating unit 46 calculates the feature amount (R1, G1, B1) of the color of the person 1 and the feature amount (R2, G2, B2) of the color of the person 2 according to the following equation (13). The correlation value between them is obtained. Further, the associating unit 46 calculates the distance between the color feature amount (R1, G1, B1) of the person 1 and the color feature amount (R2, G2, B2) of the person 2 according to the following equation (14). Ask for. Then, for example, the association unit 46 associates the person 1 and the person 2 when the correlation value between the person areas is larger than the threshold and the distance between the person areas is smaller than the threshold.

(13)

(14)

  The display device 48 displays the result of association between person areas output by the control unit 14.

  The control unit 14 of the image analysis apparatus 10 can be realized by, for example, a computer 50 illustrated in FIG. The computer 50 includes a CPU 51, a memory 52 as a temporary storage area, and a nonvolatile storage unit 53. The computer 50 also has an input / output interface (I / F) 54 to which a display device 48, an input device and the like (not shown) are connected, and read / write (R / R) that controls reading and writing of data with respect to the recording medium 59. W) A portion 55 is provided. The computer 50 includes a network interface (I / F) 56 connected to a network such as the Internet. The CPU 51, memory 52, storage unit 53, input / output device 54, R / W unit 55, and network I / F 56 are connected to each other via a bus 57.

  The storage unit 53 can be realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or the like. An image analysis program 60 for causing the computer 50 to function as the image analysis device 10 is stored in the storage unit 53 as a storage medium. The image analysis program 60 includes a person detection process 63, a person tracking process 64, an integration process 65, a person direction determination process 66, an overlap determination process 67, another camera detection result acquisition process 68, and an association candidate determination process. 69 and a feature distribution generation process 70. Further, the image analysis program 60 includes a feature selection process 71, a processing order determination process 72, and an association process 73. Further, the storage unit 53 stores information constituting the detection result storage unit 18, the tracking result storage unit 22, the integration result storage unit 26, the orientation overlap result storage unit 30, and the other camera detection result storage unit 36. The information storage area 74 is provided.

  The CPU 51 reads the image analysis program 60 from the storage unit 53 and develops it in the memory 52, and sequentially executes the processes included in the image analysis program 60. The CPU 51 operates as the person detection unit 16 illustrated in FIG. 1 by executing the person detection process 63. Further, the CPU 51 operates as the person tracking unit 20 shown in FIG. 1 by executing the person tracking process 64. Further, the CPU 51 operates as the integration unit 24 illustrated in FIG. 1 by executing the integration process 65. Further, the CPU 51 operates as the person direction determination unit 28 illustrated in FIG. 1 by executing the person direction determination process 66. Further, the CPU 51 operates as the overlap determination unit 32 illustrated in FIG. 1 by executing the overlap determination process 67. Further, the CPU 51 operates as the other camera detection result acquisition unit 34 illustrated in FIG. 1 by executing the other camera detection result acquisition process 68. Further, the CPU 51 operates as the association candidate determination unit 38 illustrated in FIG. 1 by executing the association candidate determination process 69. Further, the CPU 51 operates as the feature distribution generation unit 40 shown in FIG. 1 by executing the feature distribution generation process 70. Further, the CPU 51 operates as the feature selection unit 42 illustrated in FIG. 1 by executing the feature selection process 71. Further, the CPU 51 operates as the processing order determination unit 44 illustrated in FIG. 1 by executing the processing order determination process 72. Further, the CPU 51 operates as the association unit 46 illustrated in FIG. 1 by executing the association process 73. Further, the CPU 51 reads information from the information storage area 74 and develops the detection result storage unit 18 in the memory 52. Further, the CPU 51 reads information from the information storage area 74 and develops the tracking result storage unit 22 in the memory 52. Further, the CPU 51 reads information from the information storage area 74 and develops the integrated result storage unit 26 in the memory 52. Further, the CPU 51 reads information from the information storage area 74 and develops the orientation overlap result storage unit 30 in the memory 52. Further, the CPU 51 reads information from the information storage area 74 and develops the other camera detection result storage unit 36 in the memory 52. As a result, the computer 50 that has executed the image analysis program 60 functions as the image analysis apparatus 10. The CPU 51 that executes the image analysis program 60 that is software is hardware.

  Note that the functions realized by the image analysis program 60 can be realized by, for example, a semiconductor integrated circuit, more specifically, an application specific integrated circuit (ASIC).

  Next, the operation of the image analysis apparatus 10 according to the present embodiment will be described.

  First, when a moving image is captured by the camera A of the image analysis apparatus 10 and another camera detection result for the moving image captured by the other camera B starts to be acquired from another image analysis apparatus (not shown), the image analysis apparatus 10, the process shown in FIG. 14 is executed. Hereinafter, each process is explained in full detail.

  In step S <b> 100, the person detection unit 16 acquires a moving image captured by the camera A.

  Next, in step S102, the person detection unit 16 detects a person region from each frame image of the moving image acquired in step S100. Step S102 is realized by the process shown in FIG.

  In step S200 illustrated in FIG. 15, the person detection unit 16 obtains a background difference between the frame image at time t-1 and the frame image at time t for the frame image at each time.

  In step S202, the person detection unit 16 specifies a person area candidate based on the background difference obtained in step S200. Next, the person detection unit 16 inputs a person area candidate to a classifier that has been learned in advance, and obtains a determination result indicating whether the person area candidate is a person area.

  In step S <b> 204, the person detection unit 16 stores the detection result determined as the person region in step S <b> 202 in the detection result storage unit 18.

  Next, in step S104 of FIG. 14, the person tracking unit 20 tracks the person area of the same person according to the person area in each frame image. Step S104 is realized by the processing shown in FIG. The process of FIG. 16 is executed for each person area detected from each frame image.

  In step S300, the person tracking unit 20 reads the detection result of the person region in the frame image at time t-1 from the detection result storage unit 18.

  In step S301, the person tracking unit 20 sets one person area from each person area in the frame image at time t-1 read in step S300.

  In step S302, the person tracking unit 20 sets the image information of the person area set in step S301 as a template, performs template matching on the frame image at time t, and for each position of the frame image at time t. Calculate similarity.

  In step S304, the person tracking unit 20 calculates a position having the highest similarity calculated in step S302.

  In step S306, the person tracking unit 20 determines each person area in the frame image at time t-1 and the person in the frame image at time t based on the position having the highest similarity calculated in step S304. Identify the same person area between each of the areas. Then, the person tracking unit 20 assigns the same person area ID to the person areas determined to be the same and stores them in the tracking result storage unit 22.

  In step S307, the person tracking unit 20 determines whether or not the processes in steps S301 to S306 have been executed for all the person areas in the frame image at time t-1 read in step S300. If the processes in steps S301 to S306 have been executed for all the person regions in the frame image at time t-1 read in step S300, the process proceeds to step S308. On the other hand, if there is a person area for which the processes in steps S301 to S306 are not performed, the process returns to step S301.

  In step S308, the person tracking unit 20 determines whether or not the processing in steps S300 to S307 has been performed on the frame images at all times stored in the detection result storage unit 18. When the processes in steps S300 to S307 are executed for the frame images at all times, the process ends. If there is a frame image at a time when the processes in steps S300 to S307 are not executed, the process returns to step S300.

  Next, in step S106 of FIG. 14, the integration unit 24 integrates the detection result stored in the detection result storage unit 18 in step S102 and the tracking result stored in the tracking result storage unit 22 in step S104. To do. Step S106 is realized by the process shown in FIG. The process of FIG. 17 is executed for each person area detected from each frame image.

  In step S <b> 310, the integration unit 24 reads the detection result at time t stored in the detection result storage unit 18.

  In step S312, the integration unit 24 reads the tracking result at time t stored in the tracking result storage unit 22.

  In step S314, the integration unit 24 sets one person area from each person area in the frame image at time t read in step S310.

  In step S316, the integration unit 24 calculates an overlap rate between the person area set in step S314 and the person area in the frame image at time t read in step S312.

  In step S318, the integration unit 24 determines whether the duplication rate calculated in step S316 is equal to or greater than a threshold value. If the duplication rate calculated in step S316 is greater than or equal to the threshold, the process proceeds to step S320. On the other hand, if the duplication rate calculated in step S316 is less than the threshold, the process proceeds to step S321.

  In step S320, the integration unit 24 integrates the detection result and the tracking result for the person area set in step S314 and stores the integration result in the integration result storage unit 26.

  In step S321, the integration unit 24 stores the person area set in step S314 in the integration result storage unit 26 as a newly appearing person.

  In step S322, the integration unit 24 determines whether or not the processing in steps S314 to S321 has been executed for all person regions in the frame image at time t read in step S310. If the processes in steps S314 to S321 have been executed for all the human regions in the frame image at time t read in step S310, the process proceeds to step S324. On the other hand, if there is a person area that has not been subjected to the processes in steps S314 to S321, the process returns to step S314.

  In step S324, the integration unit 24 determines whether or not the processing in steps S310 to S322 has been performed on the frame images at all times stored in the detection result storage unit 18. When the processes in steps S310 to S322 are executed for the frame images at all times, the process ends. If there is a frame image at a time when the processes in steps S310 to S322 are not executed, the process returns to step S310.

  Next, in step S108 of FIG. 14, the person direction determination unit 28 is based on each person area of the same person stored in the integration result storage unit 26 in step S106 and associated between the frame images. The movement direction of each person area is estimated. Then, the person direction determination unit 28 specifies the direction of the person indicated by each person area based on the movement direction of each person area of the same person. Step S108 is realized by the process shown in FIG. The processing in FIG. 18 is executed for each person area detected from each frame image.

  In step S326, the person direction determination unit 28 reads the integration result at time t-1 and the integration result at time t from the integration result stored in the integration result storage unit 26.

  In step S328, the person direction determination unit 28 sets one person area from each person area in the frame image at time t-1 read in step S326. In addition, the person direction determination unit 28 reads the person area having the same person area ID as the person area ID of the person area set in the frame image at the time t−1 from the frame at the time t read in step S326. Set from each person area in the image.

  In step S330, the person direction determination unit 28 sets the y coordinate p (t-1) of the person area at time t-1 set in step S328 and the y coordinate p (t) of the person area at time t. Based on the movement direction of the person area on the image, the direction of the person shown in the person area is calculated. Then, the person direction determination unit 28 stores the orientation calculated with respect to the person region set in step S328 in the orientation overlap result storage unit 30.

  In step S332, the person direction determination unit 28 determines whether or not the processes in steps S328 to S330 have been executed for all the person regions in the frame image at time t-1 read in step S326. . If the processes in steps S328 to S330 have been executed for all the person areas in the frame image at time t-1 read in step S326, the process proceeds to step S334. On the other hand, if there is a person area that has not been subjected to the processes in steps S328 to S330, the process returns to step S328.

  In step S334, the person direction determination unit 28 determines whether or not the processes in steps S326 to S332 have been executed for the frame images at all times stored in the integration result storage unit 26. When the processes in steps S326 to S332 are executed for the frame images at all times, the process ends. If there is a frame image at a time when the processes in steps S326 to S332 are not executed, the process returns to step S326.

  Next, in step S110 of FIG. 14, the overlap determination unit 32 determines whether or not there is an overlap between human regions based on the integration result stored in the integration result storage unit 26 in step S106. Step S110 is realized by the process shown in FIG. The process of FIG. 19 is executed for each person area detected from each frame image.

  In step S <b> 400, the overlap determination unit 32 reads the integration result at time t from the integration result stored in the integration result storage unit 26.

  In step S401, the overlap determination unit 32 sets one person area from each person area in the frame image of the integration result at time t read in step S400.

  In step S402, the overlap determination unit 32 reads position information of a person area different from the person area set in step S401.

  In step S404, the overlap determination unit 32 determines whether or not there is an overlap between the position information of the person area set in step S401 and the position information of the person area read in step S402. judge. If there is an overlap, the process proceeds to step S406. On the other hand, if there is no overlap, the process proceeds to step S408.

  In step S406, the overlap determination unit 32 determines the person whose y coordinate value is smaller between the position information of the person area set in step S401 and the position information of the person area read in step S402. It is determined that there is an overlap in the area. Then, the overlap determination unit 32 stores information indicating that there is an overlap in the orientation overlap result storage unit 30.

  In step S408, the overlap determination unit 32 determines whether or not the processes in steps S402 to S406 have been executed for all person areas different from the person area set in step S401. When the processes in steps S402 to S406 have been executed for all person areas different from the person area set in step S401, the process proceeds to step S410. On the other hand, if there is a person region that has not been subjected to the processes in steps S402 to S406, the process returns to step S402.

  In step S412, the overlap determination unit 32 determines whether or not the processing in steps S401 to S408 has been executed for all person regions in the frame image at time t read in step S400. If the processes in steps S401 to S408 have been executed for all the human regions in the frame image at time t read in step S400, the process proceeds to step S412. On the other hand, if there is a person area that has not been subjected to the processes in steps S401 to S408, the process returns to step S401.

  In step S412, the overlap determination unit 32 determines whether or not the processing in steps S400 to S410 has been performed on the frame images at all times stored in the integration result storage unit 26. If the processes in steps S400 to S410 have been executed for the frame images at all times, the process ends. If there is a frame image at a time when the processes in steps S400 to S410 are not executed, the process returns to step S400.

  Next, in step S <b> 112 shown in FIG. 14, the association candidate determination unit 38 sequentially reads each of the person regions X corresponding to the newly appearing person stored in the orientation overlap result storage unit 30. Then, the association candidate determination unit 38 sets an association candidate Y with the person region X from each of the person regions stored in the other camera detection result storage unit 36. Step S112 is realized by the processing shown in FIG.

  In step S <b> 500, the association candidate determination unit 38 reads the person area X from the orientation overlap result storage unit 30. The association candidate determination unit 38 reads a person area having only one person area ID within a predetermined time interval from the current time as a person area X corresponding to a new person.

  In step S502, the association candidate determination unit 38 selects the person region in the past predetermined time interval from the time when the frame image including the person region X read in step S500 is captured, and the other camera detection result storage unit 36. Read from. Then, the association candidate determination unit 38 sets each person region in the predetermined time interval read from the other camera detection result storage unit 36 as the association candidate Y.

  Next, in step S114 shown in FIG. 14, the feature distribution generation unit 40 sets the orientation of the person area and the overlap in the person area for each of the pair of association candidates Y and the person area X set in step S112. Accordingly, the type of feature amount used for association is determined. Step S114 is realized by the processing shown in FIG. The process of FIG. 21 is executed for each person area detected from each frame image.

  In step S600, the feature distribution generation unit 40 sets one association candidate from each of the association candidates Y obtained in step S112.

  In step S602, the feature distribution generation unit 40, based on the information stored in the direction overlap result storage unit 30 and the other camera detection result storage unit 36, the person region X and the association candidate Y set in step S600 above. It is determined whether or not there is an overlap. If there is no overlap between the person region X and the association candidate Y, the process proceeds to step S604. On the other hand, if there is an overlap in at least one of the person region X and the association candidate Y, the process proceeds to step S608.

  In step S604, the feature distribution generation unit 40 determines the direction and person of the association candidate Y set in step S600 based on the information stored in the orientation overlap result storage unit 30 and the other camera detection result storage unit 36. It is determined whether or not the direction of the region X is the same. If the direction of the association candidate Y set in step S600 is the same as the direction of the person area X, the process proceeds to step S606. On the other hand, if the direction of the association candidate Y set in step S600 is different from the direction of the person area X, the process proceeds to step S608.

  In step S606, the feature distribution generation unit 40 extracts all types of feature amounts from the association candidate Y and the person region X set in step S600.

  In step S <b> 608, the feature distribution generation unit 40 determines the type of feature amount used for association according to the orientation of the person region X and the orientation of the association candidate Y, and the overlap of the person region X and the association candidate Y. Then, the feature quantity of the determined type is extracted. Specifically, the feature distribution generation unit 40, when the orientation of the person region X and the orientation of the association candidate Y are different and there is no overlap in either the person region X or the association candidate Y, the lower body The feature amount of the region and the feature amount of the shoe region are extracted. In addition, the feature distribution generation unit 40 determines that if the orientation of the person region X matches the orientation of the association candidate Y and there is an overlap in either the person region X or the association candidate Y, the overlap occurs. Extract feature quantities of non-existing areas. Further, when the orientation of the person area X and the orientation of the association candidate Y do not match and there is an overlap in either the person area X or the association candidate Y, the feature amount of the lower body area and the feature of the shoe area A feature amount of an area that is at least one of the feature amounts and does not have an overlap is extracted.

  In step S610, the feature distribution generation unit 40 determines whether or not the processing in steps S600 to S608 has been executed for all the matching candidates. If the processing of step S600 to step S608 has been executed for all the association candidates, the process proceeds to step S612. On the other hand, if there is an association candidate for which the processes in steps S600 to S608 are not executed, the process returns to step S600.

  In step S612, the feature distribution generation unit 40 generates a distribution of feature amounts of each of the association candidates Y extracted in step S606 or step S608. Then, the feature distribution generation unit 40 calculates the distribution of the distribution of each feature amount extracted from the plurality of association candidates Y.

  Next, in step S116 shown in FIG. 14, the feature selection unit 42 further selects the type of feature amount used for associating person regions from the feature amounts extracted in step S114. Step S116 is realized by the process shown in FIG. The process of FIG. 22 is executed for each person area detected from each frame image.

  In step S700, the feature selection unit 42 sets one association candidate from the association candidates obtained in step S112.

  In step S702, the feature selection unit 42 selects the feature amount having the largest distribution of the feature amount distribution among the feature amounts extracted from the association candidate in step S700, the association candidate set in step S700, and the person region. It is selected as a feature value used for association with X.

  In step S704, the feature selection unit 42 determines whether or not the processing in steps S700 to S702 has been executed for all the matching candidates. When the processes in steps S700 to S702 are executed for all the association candidates, the process ends. On the other hand, if there is an association candidate for which the processes in steps S700 to S702 are not executed, the process returns to step S700.

  Next, in step S <b> 118 shown in FIG. 14, the processing order determination unit 44 determines the order of collation between the person region X and the association candidate Y. Step S118 is realized by the processing shown in FIG. The process of FIG. 23 is executed for each person area detected from each frame image.

  In step S800, the processing order determination unit 44 reads the feature amount selected for each association candidate in step S116.

  In step S801, the processing order determination unit 44 sets one association candidate.

  In step S802, the processing order determination unit 44 acquires the feature amount for each association candidate read in step S800 and obtained from the association candidate for the frame image at each time. Then, the processing order determination unit 44 calculates the distribution of the feature amount distribution of the association candidate set in step S801 based on the feature amount obtained from the frame image association candidate at each time.

  In step S804, the processing order determination unit 44 determines whether or not the processing in steps S801 to S802 has been executed for all the association candidates. When the processing of step S801 to step S802 has been executed for all the association candidates, the process proceeds to step S806. On the other hand, if there is an association candidate for which the processes in steps S801 to S802 are not performed, the process returns to step S801.

  In step S806, the processing order determining unit 44 determines the matching order based on the distribution of the feature amount distribution for each association candidate calculated in step S802. The processing order determination unit 44 sets a matching candidate with a small variance so that the matching order comes before a matching candidate with a large variance.

  Next, in step S120 shown in FIG. 14, the associating unit 46 associates persons using the type of feature amount selected in step S116 in accordance with the collation order determined in step S118. . Step S120 is realized by the process of FIG. The process of FIG. 24 is executed for each person area detected from each frame image.

  In step S900, the associating unit 46 acquires the feature amount of the person region X.

  In step S902, the association unit 46 sets one association candidate Y according to the collation order determined in step S118.

  In step S904, the associating unit 46 acquires the feature amount of the association candidate Y set in step S902, which is the feature amount selected for the association candidate Y in step S116.

  In step S906, the associating unit 46 collates the feature amount of the person region X acquired in step S900 with the feature amount of the association candidate Y acquired in step S904, and calculates the matching degree of matching. .

  In step S908, the associating unit 46 determines whether or not the degree of coincidence of the association candidates calculated in step S906 is equal to or greater than a threshold value. If the matching candidate matching degree calculated in step S906 is greater than or equal to the threshold, the process proceeds to step S912. On the other hand, if the matching degree of the matching candidates calculated in step S906 is less than the threshold value, the process proceeds to step S910.

  In step S910, the associating unit 46 determines whether or not the processes in steps S902 to S908 have been executed for all the association candidates. When the processes in steps S902 to S908 are executed for all the association candidates, the process ends. On the other hand, if there is an association candidate for which the processes in steps S902 to S908 are not executed, the process returns to step S902.

  In step S912, the associating unit 46 associates the person region of the other camera detection result with the associating candidate set in step S902 as the same person.

  In step S122, the association unit 46 outputs the association result as a result. For example, the associating unit 46 outputs the person area ID associated as the same person as the associating result.

  In step S124, the control unit 14 determines whether or not to continue the process. When the process is continued, the process returns to step S100, and when the process is not continued, the process is terminated.

  As described above, the image analysis apparatus according to the first embodiment determines the type of feature amount used for the association of the person areas according to the state of the person area detected from each of the plurality of images. . Then, a corresponding person region in each of the plurality of images is specified according to the degree of coincidence of the determined types of feature amounts. As a result, the person areas can be associated with each other with high accuracy. Further, even when a part of the feature amount of the person area cannot be obtained from each of the images captured by the plurality of cameras, the same person can be specified.

  In addition, since it is possible to use the unique feature amount of the matching candidate by selecting the feature used for matching after determining in advance whether or not the feature can be obtained correctly from the person area, Matching is possible, and the matching accuracy can be improved.

<Second Embodiment>

  Next, a second embodiment will be described. In the second embodiment, from past frame images including association candidates, feature amounts obtained from association candidates in the same direction as the direction of the person region of the other camera detection result, and association candidates that do not have an overlap The point which uses the obtained feature-value for collation differs from 1st Embodiment. In addition, since the structure of 2nd Embodiment becomes a structure similar to 1st Embodiment, it attaches | subjects the same code | symbol and abbreviate | omits description.

  The feature selection unit 42 according to the second embodiment associates a person area with each person area according to the state of the person area detected from each of the plurality of frame images captured at each time. Determine the type of feature to use. Specifically, the feature selection unit 42 of the second exemplary embodiment obtains a feature obtained from an association candidate included in a past frame image from an association candidate having the same orientation as the direction of the person region of the other camera detection result. An amount and a feature amount obtained from an association candidate with no overlap are set.

  FIG. 25 is a diagram for explaining selection of feature amounts according to the second embodiment. In the second embodiment, the feature area extracted from the association candidates whose orientation is different from that of the person region X1 from the past frame images FY1 and FY2 of the association candidates Y1 and Y2 is not used, and the orientation is the same as that of the person region X1. The feature amount of the matching candidate is as follows. In addition, the feature selection unit 42 does not extract the feature amount of each of the association candidates obtained from the past frame images of the association candidates Y1 and Y2, and does not extract the feature amount. Feature values are extracted from non-existing association candidates. The past frame images FY1 and FY2 are frame images within a predetermined time range range used when setting the association candidates.

  As a result, since the direction of the matching candidate is the same as the direction of the person area of the other camera detection result, and the feature quantity that does not overlap the matching candidate is used for matching, the matching can be performed with high accuracy. it can.

  Next, the operation of the feature selection unit 42 of the image analysis apparatus 10 according to the second embodiment will be described. In step S116 illustrated in FIG. 14, the feature selection unit 42 further selects the type of feature amount to be used for associating person regions from the feature amounts extracted in step S114. Step S116 of the second embodiment is realized by the process shown in FIG.

  In step S700, the feature selection unit 42 sets one association candidate from the association candidates obtained in step S112.

  In step S752, the feature selection unit 42 determines whether or not the association candidate set in step S700 is present in the past frame image. If the association candidate exists in the past frame image, the process proceeds to step S754. On the other hand, if the association candidate does not exist in the past frame image, the process proceeds to step S756.

  In step S754, the feature selection unit 42 corresponds to the person frame of the other camera detection result that has the same direction and does not overlap from the past frame image including the association candidate set in step S700. Search for attachment candidates.

  In step S756, the feature selection unit 42 selects the feature amount having the largest distribution of the feature amount distribution calculated in step 612 among the feature amounts extracted from the association candidates searched in step S754. It is selected as a feature value used for matching people.

  In step S704, the feature selection unit 42 determines whether or not the processing in steps S700 to S756 has been executed for all the matching candidates. When the processes in steps S700 to S756 are executed for all association candidates, the process ends. On the other hand, if there is an association candidate for which the processes in steps S700 to S756 are not performed, the process returns to step S700.

  As described above, the image analysis apparatus according to the second embodiment is characterized in that the feature amount obtained from the association candidate having the same orientation as the direction of the new person region from the past frame image including the association candidate, And the feature quantity obtained from the matching candidate having no overlap is used for collation. Thereby, a person area can be associated with high accuracy.

<Third Embodiment>

  Next, a third embodiment will be described. The third embodiment differs from the first or second embodiment in that collation is performed from a matching candidate having a feature amount that is largely deviated from the average in the distribution of feature amounts extracted from each of the matching candidates. . In addition, since the structure of 3rd Embodiment becomes a structure similar to 1st Embodiment, it attaches | subjects the same code | symbol and abbreviate | omits description.

  The processing order determination unit 44 according to the third embodiment determines, for each of a plurality of person regions, between the average of the distribution of feature amounts extracted from each of the plurality of person regions and the feature amount extracted from the person region. The correspondence order of each person area is set in accordance with the deviation. Specifically, the processing order determination unit 44 performs the collation order so that collation is performed from the association candidates having the feature amount having a large deviation from the average in the distribution of the feature amounts extracted from each of the association candidates. Set.

  FIG. 27 is a diagram for explaining the setting of the processing order of the third embodiment. In the example shown in FIG. 27, the distribution of feature amounts extracted from each association candidate is shown. One point in the feature amount distribution represents a feature amount extracted from one association candidate. The processing order determination unit 44 of the image analysis apparatus 10 according to the third embodiment sets the matching candidate collation order in accordance with the deviation from the average of the distribution of feature amounts extracted from the matching candidates. In the example shown in FIG. 27, since the feature amount of the person region Y2 is shifted from the average than the feature amount of the person region Y1, the processing order determination unit 44 matches the person region Y2 to that of the person region Y1. The collation order is set so that the collation is performed before the collation. Because the association candidate having a feature amount that is largely deviated from the average indicates that it has a unique feature compared to other association candidates, the possibility of performing high-precision collation is increased. , Collation is preferentially performed.

  As described above, the image analysis apparatus according to the third embodiment sets the collation order based on the average of the distribution of feature amounts obtained from a plurality of association candidates. This increases the possibility of reducing the amount of processing for matching feature quantities between person regions.

  In the above description, the mode in which the image analysis program 60 is stored (installed) in the storage unit 53 in advance has been described. However, the present invention is not limited to this. The program according to the disclosed technology can be provided in a form recorded on a recording medium such as a CD-ROM, a DVD-ROM, or a USB memory.

  Next, a modification of the embodiment will be described.

  In the present embodiment, the case where the object is a person has been described as an example, but the present invention is not limited to this. For example, the object may be a moving body such as a vehicle or an animal.

  Moreover, although the said embodiment demonstrated as an example the case where the person area imaged from the other camera B was transmitted as another camera detection result, it is not limited to this. For example, as the other camera detection result, selection feature information (for example, the above-mentioned Patent Document 2) used for association may be sent together. In this case, in consideration of the selected feature information, the person area captured by the camera A can be associated with the person area of the other camera detection result.

  In addition, the image analysis device 10 according to the present embodiment may transmit the feature amount for each person area selected by the feature selection unit 42 to another image analysis device. In this case, in another image analysis apparatus, the association can be performed in consideration of the direction and overlap of the person area. Similarly, the image analysis apparatus 10 can also acquire information regarding the orientation and overlap of the person area obtained by another image analysis apparatus, and perform association in consideration of the orientation and overlap of the person area.

  Regarding the above embodiments, the following additional notes are disclosed.

(Appendix 1)
According to the state of the object area detected from each of the plurality of images, determine the type of feature amount used for associating the object area between the plurality of images,
Identifying the corresponding object region in each of a plurality of images according to the degree of coincidence of the type of the feature amount extracted from each of the object regions;
An image analysis program that causes a computer to execute processing.

(Appendix 2)
When determining the type of feature value used for the association of the object area,
According to at least one of the orientation of the object indicated by the object area and the overlap between the object indicated by the object area and an object different from the object, which is the state of the object area, Decide the type of feature used to match object areas,
The image analysis program according to attachment 1.

(Appendix 3)
When determining the type of feature value used for the association of the object area,
For each of the object region pairs detected from each of the different images,
When at least one of the object regions includes the overlap, the feature amount extracted from a region different from the overlap region is determined as a feature amount to be used for associating the object region.
The image analysis program according to attachment 2.

(Appendix 4)
When determining the type of feature amount used for associating the person area as the object area,
For each of the plurality of pairs of person regions,
When the orientation of one person area and the other person area does not correspond, the feature amount extracted from an area different from the upper body of the person area is used for associating the object area To be determined as the feature quantity
The image analysis program according to Supplementary Note 2 or Supplementary Note 3.

(Appendix 5)
When determining the type of feature value used for the association of the object area,
For each of the plurality of target areas, used for associating the target areas in accordance with the state of distribution of feature values extracted from each of the plurality of target areas, which is the state of the target area. Determine the type of feature
The image analysis program according to any one of appendix 1 to appendix 4.

(Appendix 6)
When determining the type of feature value used for the association of the object area,
For each of the plurality of object regions, the state of the object region detected from each of the plurality of images captured at each time and the direction of the object indicated by the object region and the object region Determining the type of feature amount used for associating the object area according to at least one of the overlap between the object indicated by and an object different from the object;
The image analysis program according to any one of supplementary notes 1 to 5.

(Appendix 7)
When identifying the corresponding object region in each of a plurality of images,
The degree of coincidence with the feature amount of the target area detected from another image is determined in order from the target area having a small variance of the type of feature quantity used for the target area association.
The image analysis program according to any one of supplementary notes 1 to 6.

(Appendix 8)
When identifying the corresponding object region in each of a plurality of images,
Detected from other images in order from the object region in which the deviation between the average of the distribution of the feature amount extracted from each of the plurality of target regions and the feature amount extracted from the target region is large Determining the degree of coincidence with the feature amount of the object area,
The image analysis program according to any one of supplementary notes 1 to 6.

(Appendix 9)
According to the state of the object area detected from each of the plurality of images, determine the type of feature amount used for associating the object area between the plurality of images,
Identifying the corresponding object region in each of a plurality of images according to the degree of coincidence of the type of the feature amount extracted from each of the object regions;
An image analysis method for causing a computer to execute processing.

(Appendix 10)
When determining the type of feature value used for the association of the object area,
According to at least one of the orientation of the object indicated by the object area and the overlap between the object indicated by the object area and an object different from the object, which is the state of the object area, Decide the type of feature used to match object areas,
The image analysis method according to appendix 9.

(Appendix 11)
When determining the type of feature value used for the association of the object area,
For each of the object region pairs detected from each of the different images,
When at least one of the object regions includes the overlap, the feature amount extracted from a region different from the overlap region is determined as a feature amount to be used for associating the object region.
The image analysis method according to attachment 10.

(Appendix 12)
When determining the type of feature amount used for associating the person area as the object area,
For each of the plurality of pairs of person regions,
When the orientation of one person area and the other person area does not correspond, the feature amount extracted from an area different from the upper body of the person area is used for associating the object area To be determined as the feature quantity
The image analysis method according to appendix 10 or appendix 11.

(Appendix 13)
When determining the type of feature value used for the association of the object area,
For each of the plurality of target areas, used for associating the target areas in accordance with the state of distribution of feature values extracted from each of the plurality of target areas, which is the state of the target area. Determine the type of feature
The image analysis method according to any one of appendix 9 to appendix 12.

(Appendix 14)
When determining the type of feature value used for the association of the object area,
For each of the plurality of object regions, the state of the object region detected from each of the plurality of images captured at each time and the direction of the object indicated by the object region and the object region Determining the type of feature amount used for associating the object area according to at least one of the overlap between the object indicated by and an object different from the object;
The image analysis method according to any one of appendix 9 to appendix 13.

(Appendix 15)
When identifying the corresponding object region in each of a plurality of images,
The degree of coincidence with the feature amount of the target area detected from another image is determined in order from the target area having a small variance of the type of feature quantity used for the target area association.
The image analysis method according to any one of appendix 9 to appendix 14.

(Appendix 16)
When identifying the corresponding object region in each of a plurality of images,
Detected from other images in order from the object region in which the deviation between the average of the distribution of the feature amount extracted from each of the plurality of target regions and the feature amount extracted from the target region is large Determining the degree of coincidence with the feature amount of the object area,
The image analysis method according to any one of appendix 9 to appendix 14.

(Appendix 17)
A determining unit that determines a type of feature amount used for associating the target area between the plurality of images according to a state of the target area detected from each of the plurality of images;
A specifying unit for specifying the corresponding object region in each of a plurality of images according to the degree of coincidence of the type of the feature amount extracted from each of the object regions;
An image analysis apparatus comprising:

(Appendix 18)
When determining the type of feature value used for the association of the object area,
According to at least one of the orientation of the object indicated by the object area and the overlap between the object indicated by the object area and an object different from the object, which is the state of the object area, Decide the type of feature used to match object areas,
The image analysis device according to appendix 17.

(Appendix 19)
When determining the type of feature value used for the association of the object area,
For each of the object region pairs detected from each of the different images,
When at least one of the object regions includes the overlap, the feature amount extracted from a region different from the overlap region is determined as a feature amount to be used for associating the object region.
The image analysis apparatus according to appendix 18.

(Appendix 20)
According to the state of the object area detected from each of the plurality of images, determine the type of feature amount used for associating the object area between the plurality of images,
Identifying the corresponding object region in each of a plurality of images according to the degree of coincidence of the type of the feature amount extracted from each of the object regions;
A storage medium storing an image analysis program that causes a computer to execute processing.

DESCRIPTION OF SYMBOLS 10 Image analysis apparatus 14 Control part 16 Person detection part 18 Detection result storage part 20 Person tracking part 22 Tracking result storage part 24 Integration part 26 Integration result storage part 28 Person direction determination part 30 Result storage part 32 Overlap determination part 34 Other camera detection Result acquisition unit 36 Other camera detection result storage unit 38 Candidate determination unit 40 Feature distribution generation unit 42 Feature selection unit 44 Processing order determination unit 46 Association unit 48 Display device 50 Computer 51 CPU
52 Memory 53 Storage 59 Recording medium

Claims (10)

According to the state of the object area detected from each of the plurality of images, determine the type of feature amount used for associating the object area between the plurality of images,
Identifying the corresponding object region in each of a plurality of images according to the degree of coincidence of the type of the feature amount extracted from each of the object regions;
An image analysis program that causes a computer to execute processing. When determining the type of feature value used for the association of the object area,
According to at least one of the orientation of the object indicated by the object area and the overlap between the object indicated by the object area and an object different from the object, which is the state of the object area, Decide the type of feature used to match object areas,
The image analysis program according to claim 1. When determining the type of feature value used for the association of the object area,
For each of the object region pairs detected from each of the different images,
When at least one of the object regions includes the overlap, the feature amount extracted from a region different from the overlap region is determined as a feature amount to be used for associating the object region.
The image analysis program according to claim 2. When determining the type of feature amount used for associating the person area as the object area,
For each of the plurality of pairs of person regions,
When the orientation of one person area and the other person area does not correspond, the feature amount extracted from an area different from the upper body of the person area is used for associating the object area To be determined as the feature quantity
The image analysis program according to claim 2 or 3. When determining the type of feature value used for the association of the object area,
For each of the plurality of target areas, used for associating the target areas in accordance with the state of distribution of feature values extracted from each of the plurality of target areas, which is the state of the target area. Determine the type of feature
The image analysis program according to any one of claims 1 to 4. When determining the type of feature value used for the association of the object area,
For each of the plurality of object regions, the state of the object region detected from each of the plurality of images captured at each time and the direction of the object indicated by the object region and the object region Determining the type of feature amount used for associating the object area according to at least one of the overlap between the object indicated by and an object different from the object;
The image analysis program according to any one of claims 1 to 5. When identifying the corresponding object region in each of a plurality of images,
The degree of coincidence with the feature amount of the target area detected from another image is determined in order from the target area having a small variance of the type of feature quantity used for the target area association.
The image analysis program of any one of Claims 1-6. When identifying the corresponding object region in each of a plurality of images,
Detected from other images in order from the object region in which the deviation between the average of the distribution of the feature amount extracted from each of the plurality of target regions and the feature amount extracted from the target region is large Determining the degree of coincidence with the feature amount of the object area,
The image analysis program of any one of Claims 1-6. According to the state of the object area detected from each of the plurality of images, determine the type of feature amount used for associating the object area between the plurality of images,
Identifying the corresponding object region in each of a plurality of images according to the degree of coincidence of the type of the feature amount extracted from each of the object regions;
An image analysis method for causing a computer to execute processing. A determining unit that determines a type of feature amount used for associating the target area between the plurality of images according to a state of the target area detected from each of the plurality of images;
A specifying unit for specifying the corresponding object region in each of a plurality of images according to the degree of coincidence of the type of the feature amount extracted from each of the object regions;
An image analysis apparatus comprising: