JP5713655B2 - Video processing apparatus, video processing method, and program - Google Patents

Description

  The present invention relates to a video processing apparatus, a video processing method, and a program for detecting an object in a video.

  Conventionally, when a specific target such as a face or a human body is detected from a camera image, the target is detected by pattern matching processing. For example, in the conventional example, a plurality of local patterns are extracted using the collation pattern over the entire image while changing the size of the image, and each local feature is calculated. And it is determined whether it is a human body from the sum total of the result of having weighted the local feature-value (for example, patent document 1). As a conventional example, it is known to change the processing method of face detection processing according to the posture of the camera body (for example, Patent Document 2). Further, as a conventional example, it is known to perform tracking of an object using template matching (for example, Patent Document 3).

JP 2009-211311 A JP 2010-080993 A JP 2002-373332 A

  However, for example, in the video of the surveillance camera, the subject moves in all directions, and the subject's face and human body characteristics are not always reflected in a direction that is easy to detect. In this environment, human body detection using a specific matching pattern does not always detect a human body with high detection accuracy regardless of how the subject appears. Further, even if the processing method of the face detection process is changed according to the posture of the camera body, it is not always possible to detect the face area with high accuracy.

  The present invention has been made to solve the above-described problems, and an object thereof is to provide a video processing apparatus and a video processing method that can efficiently detect a target regardless of the angle and size of the subject. To do.

In order to achieve the above object, the present invention provides detection means for detecting an object from a video, determination means for determining a movement direction of the object detected by the detection means, and movement direction of the object detected by the detection means. Is the first direction, it is detected whether the object is a specific object by pattern matching processing for matching the object with a predetermined matching pattern, and the moving direction of the object detected by the detecting means is the second direction. In the case of, the detection means detects whether the object is the specific object by a detection method that is different from the pattern matching process and uses a detected aspect ratio, size, or shape of the object .

  According to the present invention having the above-described configuration, it is possible to efficiently detect a target according to the moving direction of the subject.

1 is a diagram illustrating a configuration example of a video processing apparatus according to Embodiment 1. FIG. The figure which shows the example of a collation pattern. The figure for demonstrating the moving direction of an object. The figure for demonstrating the moving direction angle | corner of an object. The figure which shows a collation pattern table. FIG. 3 is a process flowchart of the video processing apparatus according to the first embodiment. The flowchart for demonstrating collation pattern determination processing.

  Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the accompanying drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

<Example 1>
The configuration of the video processing apparatus of this embodiment will be described with reference to FIG. Each component shown in FIG. 1 may be configured as physically independent hardware, or may be configured as one or more instances of software having equivalent functions.

  The acquisition unit 101 acquires images captured by the processing target camera in chronological order. The acquisition unit 101 may acquire an image stored in a server or an external memory.

  The detection unit 102 detects an object from the image acquired by the acquisition unit 101. The detection unit 102 can detect a plurality of objects. The object information detected by the detection unit 102 includes the position of the object on the screen, a rectangle circumscribing the object, and the size of the object. For example, a background subtraction method can be used as an object detection method. The detection method is not limited to the background subtraction method, and any method that can acquire the above-described information may be used.

  The tracking unit 103 associates the object detected by the detection unit 102 from the past image acquired by the acquisition unit 101 and the object detected by the detection unit 102 from the current image acquired by the acquisition unit 101. The association can be performed by assigning an object ID assigned to an object detected from a past image to an object detected from the current image. Also, a new object ID is assigned to a newly appearing object.

  Based on the size of the object detected by the detection unit 102 and the tracking result of the tracking unit 103, the determination unit 104 determines a verification pattern to be used in a human body detection process described later. An example of the collation pattern is shown in FIG. FIG. 2A shows an example of a collation pattern for detecting a human body facing the front. The collation pattern 201 is a luminance pattern composed of, for example, 30 × 80 pixels, and an internal rectangle as a part thereof is a local pattern. The local pattern 202 is a pattern for extracting the edge strength in the horizontal direction of the top of the head. The local pattern 203 and the local pattern 204 are patterns for extracting the edge strength of the shoulder line. The local pattern 205 and the local pattern 206 are patterns for extracting the edge strength in the vertical direction of the arm, and the local pattern 207 and the local pattern 208 are patterns for extracting the vertical edge of the leg. The above-mentioned local pattern is an example, and is not limited to the four places shown in FIG. Next, an example of a collation pattern for detecting a horizontal human body is shown in FIG. In the collation pattern shown in FIG. 2B, the position and size of the forward collation pattern and the local pattern, the local feature amount calculation method described later, and the like are different. The collation pattern is not limited to the front direction and the horizontal direction, and may be from other angles such as diagonally or upwardly. Further, the matching pattern is not limited to the whole body, but may be limited to the upper body, face, and feet. Furthermore, the collation pattern is not limited to the human body, and when detecting other than the human body, a pattern created for detecting a specific target such as a car, a train, or an airplane may be used.

  Next, a determination method in which the determination unit 104 determines a collation pattern used in the human body detection process will be described with reference to FIGS.

  FIG. 3 is an example showing the moving direction of the object in the nth frame. Since the object with the object ID 12 has moved from the position 401 of the (n−1) th frame to the position 402 of the nth frame, the movement of the object ID 12 can be expressed as a position vector 403. Further, since the object with the object ID 6 has moved from the position 404 of the (n−1) th frame to the position 405 of the nth frame, the movement of the object ID 6 can be expressed as a position vector 406. In this way, the determination unit 104 determines the moving direction in which the object detected by the detection unit 102 moves.

  FIG. 4 is an example showing the moving direction angle of the object. In this embodiment, the right direction is the positive x-axis direction, and the downward direction is the positive y-axis direction. The moving direction angle 502α of the position vector 501 is set so that the positive y-axis direction is 0 °. The determination unit 104 determines a verification pattern used for the human body detection process according to the movement direction angle α. When the moving direction angle α is 0 ° or more and less than 45 °, 135 ° or more and less than 225 °, or 315 ° or more and less than 360 °, the determination unit 104 determines the collation pattern used for the detection process as a front-facing collation pattern. In addition, when the moving direction angle α is 45 ° or more and less than 135 ° or 225 ° or more and less than 315 °, the determination unit 104 determines the collation pattern used for the detection process to be a horizontal collation pattern. In this way, when the object detected by the detection unit 102 moves in the vertical direction, the determination unit 104 can determine the verification pattern used for the detection process as a front-facing verification pattern. Further, when the object detected by the detection unit 102 moves in the left-right direction, the determination unit 104 can determine the verification pattern used for the detection process as a horizontal verification pattern. Therefore, the determination unit 104 can appropriately determine the collation pattern used for the detection process according to the detected movement of the object. The angle range used for dividing the movement direction angle α is not limited to the above example, and can be set as appropriate. Further, the determination unit 104 may determine not to perform the detection process when the movement direction angle α is a predetermined angle or an angle within a predetermined angle range. In this embodiment, the moving direction angle is determined from the position vector between two frames. However, the moving direction angle may be determined from the inner product of the position vectors between a plurality of frames. When the detection unit 102 detects an object based on a motion vector, the moving direction of the object may be determined from the motion vector.

  FIG. 5 shows an example of a result obtained when the determination unit 104 determines a collation pattern used for human body detection. The information shown in FIG. 5 is held in a storage unit (not shown). The content held in the storage unit is not limited to the information illustrated in FIG. 5, and may be any information that can understand the correspondence between the object detected by the detection unit 102 and the matching pattern. Further, the relationship between the object detected by the detection unit 102 and the matching pattern does not necessarily have to correspond one-to-one, and may correspond one-to-many. By associating a plurality of collation patterns with a specific movement direction angle 502, the object detected by the detecting unit 102 can be associated with the plurality of collation patterns.

  The detection unit 105 uses the matching pattern determined by the determination unit 104 to perform human body detection processing on the area detected by the detection unit 102. The detection unit 105 extracts edge strengths detected by the detection unit 102 based on the local pattern of the collation pattern determined by the determination unit 104. Then, the determination unit 104 obtains a local feature amount u by performing an inner product operation between the extracted edge strength and an edge in a predetermined direction set in the local pattern. Next, the detection unit 105 obtains the sum total of each local feature amount with a weight w, and compares it with a predetermined threshold th to detect a human body. Further, the local feature amount calculation method is not limited to the inner product calculation, and other methods such as a predetermined linear determination analysis may be used.

  In this embodiment, the detection target is a human body, but the detection target is not limited to the human body. For example, automobiles, animals, etc. may be detected. Furthermore, the detection unit 105 may detect a plurality of types of specific objects. The method by which the detection unit 105 detects a specific object from the image is not limited to the pattern matching process. For example, a method of estimating a human body based on the aspect ratio, size, or shape of the object may be used. When human body detection is performed, a characteristic part such as a face is not photographed when a person moves in the horizontal direction, so that the accuracy of detecting a human body is low in pattern matching. Therefore, detection may be performed using pattern matching when the detected object moves in the vertical direction, and detection may be performed using the aspect ratio, size, or shape of the object when the object moves in the horizontal direction. . In this way, the detection unit 105 can perform detection processing using a matching pattern that is highly likely to be able to detect the subject according to the direction of the subject that is expected from the moving direction of the object detected by the detection unit 102. In addition, when the determination unit 104 determines not to perform object image processing, the detection unit 105 can be configured not to perform detection. In this case, the detection process can be performed by excluding the object moving in the direction of the predetermined movement direction angle α or the direction of the predetermined movement direction range from the detection target. As described above, the detection unit 105 switches detection processing for detecting whether the object detected by the detection unit 102 is a specific object according to the moving direction of the object determined by the determination unit 104. Alternatively, the detection unit 105 can be configured not to perform the detection process when the object moves in a predetermined movement direction. Therefore, the probability of detecting a desired subject from the video can be improved.

  Furthermore, in the present embodiment, the case where the matching pattern is changed according to the movement direction angle has been described, but the local feature amount calculation method may be changed according to the movement direction. For example, the weight added to the local feature amount may be changed according to the moving direction. When detecting a human image from the video, if the detected object is moving in the vertical direction, if the object is a human, it is considered that the image of the front of the human being is captured, Increase the weight added to the local feature of the upper body. On the other hand, if the detected object is moving in the horizontal direction, if the object is a human, it is considered that a human sideways image has been taken. Can be bigger. In this way, a more preferable pattern can be generated for detecting a human image according to the direction of the object. Accordingly, the probability of detecting a desired subject can be improved.

  The abort unit 106 determines the abort of the human body detection process by the detection unit 105 according to the processing elapsed time. The elapsed time threshold value for determining the discontinuation is a value set in advance in accordance with an instruction from the user or the like. The decision to abort the detection process is not limited to that performed based on the elapsed time from the start of the process. The detection process abortion may be determined from the number of human bodies detected by the detection unit 105, the number of times the detection unit 105 has executed human body detection processing, and the like. The output unit 107 outputs the detection result of the detection unit 102, the tracking status of the tracking unit 103, and the detection result of the detection unit 105 to the outside.

  The operation of the video processing apparatus according to this embodiment is shown in the flowchart of FIG. In the form in which the video processing apparatus includes a processor and a memory, the processing flow of FIG. 6 shows a program for causing the processor to execute the procedure shown in FIG. The processor built in the video processing apparatus is a computer, and executes a program read from a memory built in the video processing apparatus. The memory built in the video processing apparatus is a recording medium on which the program is recorded so that the processor can read it.

  First, the acquisition unit 101 determines whether to continue or end image acquisition (f1). The determination of step f1 can be made based on a user instruction input from an input unit (not shown). Alternatively, the image acquisition may be terminated when a desired object is detected. The determination method in step f1 is not limited to these. When the detection process is continued, the acquisition unit 101 acquires an image from the outside (f2). Next, a detection process in which the detection unit 102 detects an object is performed on the image acquired by the acquisition unit 101 (f3). As an object detection method, for example, a background difference method or an inter-frame difference method can be used. The method for detecting an object is not particularly limited, and a conventional detection method can be used. Subsequently, the detection unit 102 determines the detection processing result (f4). If the detection unit 102 has not detected the object (if f4 is no), the process returns to f1. When the detection unit 102 detects an object (if yes in f4), the tracking unit 103 performs object tracking processing (f5). The tracking unit 103 associates an object detected by the detection unit 102 between a plurality of images acquired by the acquisition unit 101. And the determination part 104 determines the collation pattern used for a detection process (f6). The determination unit 104 switches the collation pattern used for the detection process according to the moving direction of the detected object. Alternatively, the determination unit 104 changes the weight of the local feature amount of the matching pattern according to the detected moving direction of the object. Details of the operation of step f6 will be described later with reference to FIG. Next, it is determined whether the truncation unit 106 continues or ends the process (f7). For example, when the detection unit 102 detects a plurality of objects in the screen, the truncation unit 106 determines whether to perform detection processing for the second object after performing detection processing for the first object. To do. The truncation unit 106 can determine whether or not to continue the detection process according to the number of human bodies detected by the detection unit 105, the number of times the detection unit 105 executes the human body detection process, and the like. Alternatively, the truncation unit 106 can make the determination in step f7 based on a user instruction input from an input unit (not shown). Alternatively, the truncation unit 106 may end the detection process when the detection unit 105 detects a desired object. The determination of step f7 is not limited to these. When the detection process is continued (if yes in f7), the detection unit 105 performs the human body detection process using the determined collation pattern (f8). When the detection process is terminated (if f7 is no), the process returns to f1.

  Next, details of the collation pattern determination operation of f6 in the present embodiment will be described using the flowchart of FIG. The determination unit 104 first determines the moving direction angle of the object detected by the detection unit 102 (f10). Next, the determination unit 104 determines whether or not the moving direction angle 502 is within a predetermined angle range (f11). In this embodiment, it is determined whether the angle α shown in FIG. 4 is in the range of 0 ° or more and less than 45 °, 135 ° or more and less than 225 °, or 315 ° or more and less than 360 °. When the moving direction angle 502 is within the predetermined angle range (if f11 is yes), the determination unit 104 determines the collation pattern used for the detection process as a front-facing collation pattern (f13). When the moving direction angle 502 is out of the predetermined angle range (if f11 is no), the determination unit 104 determines the collation pattern used for the detection process to be the horizontal collation pattern (f14). The weighting of the local feature amount of the pattern used for the detection process may be changed according to the determination result of step f11. For example, when the movement direction angle 502 is within a predetermined angle range, the weight of the local feature amount of the upper body is increased, and when the movement direction angle 502 is out of the predetermined angle range, the local feature amount of the lower body is increased. It is also possible to increase the weight of. The angle range used above is an example and is not limited to this. Furthermore, in the present embodiment, the case is divided into two cases in step f11, but three or more cases may be divided and a different detection method may be executed for each case. Then, the determination unit 104 stores the determined verification pattern in a storage unit (not shown) in association with the object for which the verification pattern has been determined as shown in FIG. 5 (f15).

  With the above configuration, the video processing apparatus according to the present embodiment can detect a desired object using a more appropriate matching pattern according to the movement of the object in the video. Therefore, the efficiency of the detection process for detecting a specific object from the video is improved.

<Other examples>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

101 detection unit 104 determination unit 105 detection unit

Claims (15)

Detection means for detecting an object from the image;
Determining means for determining a moving direction in which the object detected by the detecting means moves;
When the moving direction of the object detected by the detection means is the first direction, the detection means detects whether the object is a specific object by pattern matching processing that matches the object with a predetermined matching pattern. When the moving direction of the detected object is the second direction, the detection method is different from the pattern matching process, and the object is the specific object by the detection method using the aspect ratio, size, or shape of the detected object. An image processing apparatus comprising: detecting means for detecting whether or not there is . When the detection unit detects a plurality of objects, the number of times that the detection process for detecting whether the object detected by the detection unit is the specific object is executed, or the specification detected by the detection unit The video processing apparatus according to claim 1, further comprising an abort unit that determines whether or not the detection unit continues the detection process based on the number of objects. When the object moves in a predetermined direction, the detecting means has the specific object having a predetermined local feature amount, a predetermined aspect ratio, a predetermined size, or a predetermined shape. The video processing apparatus according to claim 1, wherein no detection process for detecting whether or not the image is detected. When the moving direction of the object detected by the detecting means is a vertical direction on the screen, the detecting means detects whether the object is a specific object by pattern matching processing that matches the object with a predetermined matching pattern. When the moving direction of the object detected by the detecting means is a horizontal direction on the screen, the object is detected by a detection method that is different from the pattern matching process and uses the aspect ratio, size, or shape of the detected object. The video processing apparatus according to claim 1, wherein the video processing apparatus detects whether the specific object is the specific object. Detection means for detecting an object from the image;
Determining means for determining a moving direction in which the object detected by the detecting means moves;
Whether the object is a specific object is detected based on the sum total weighted to each local feature amount of the object detected by the detection unit, and the object detected by the detection unit is An image processing apparatus comprising: a detecting unit that changes a weight added to each local feature of an object. A detection step in which the detection means detects an object from the video;
A determining step for determining a moving direction in which the object detected in the detecting step moves;
When the detection unit detects the movement direction of the object detected by the detection unit is the first direction, the detection unit detects whether the object is a specific object by pattern matching processing that compares the object with a predetermined verification pattern; When the moving direction of the object detected by the detecting means is the second direction, the object is detected by the detection method that is different from the pattern matching process and uses the aspect ratio, size, or shape of the detected object. And a detection step of detecting whether the object is a specific object . When the detection means detects a plurality of objects, the aborting means performs the number of times that the detection process for detecting whether the object detected by the detection means is the specific object, or the detection means The video processing method according to claim 6, further comprising: an aborting step of determining whether or not the detection means continues the detection process based on the number of the detected specific objects. In the detection step, when the object moves in a predetermined direction, the object moving in the predetermined direction has the predetermined local feature, the predetermined aspect ratio, the predetermined size, or the predetermined shape. The video processing method according to claim 6, wherein a detection process for detecting whether or not the image is detected is not performed. In the detecting step, when the moving direction of the object detected by the detecting means is a vertical direction on the screen, it is detected whether the object is a specific object by pattern matching processing that matches the object with a predetermined matching pattern. When the moving direction of the object detected by the detecting means is a horizontal direction on the screen, the object is detected by a detection method that is different from the pattern matching process and uses the aspect ratio, size, or shape of the detected object. The video processing method according to claim 6, further comprising: detecting whether the specific object is the specific object. A detection step in which the detection means detects an object from the video;
A determining step for determining a moving direction in which the object detected by the detecting means moves; and
The detection means detects whether the object is a specific object based on the sum total weighted to each local feature amount of the object detected by the detection means, and the moving direction in which the object detected by the detection means moves And a detecting step of changing a weight added to each local feature amount of the object according to the method. On the computer,
A detection procedure for detecting an object from the image;
A determination procedure for determining a moving direction in which the object detected in the detection procedure moves;
When the moving direction of the object detected in the detection procedure is the first direction, it is detected whether the object is a specific object by pattern matching processing that matches the object with a predetermined matching pattern. When the moving direction of the detected object is the second direction, the detection method is different from the pattern matching process, and the object is the specific object by the detection method using the aspect ratio, size, or shape of the detected object. A program for executing a detection procedure for detecting whether there is any . In the computer,
When a plurality of objects are detected in the detection procedure, the number of executions of detection processing for detecting whether the object detected in the detection procedure is the specific object, or the specific detected in the detection procedure The program according to claim 11 for executing an abort procedure for determining whether or not to continue detection processing in the detection procedure based on the number of objects. In the computer,
In the detection procedure, when the object moves in a predetermined direction, the object moving in the predetermined direction has the predetermined local feature amount, the predetermined aspect ratio, the predetermined size, or the predetermined shape. The program according to claim 11, for executing a process that does not perform a detection process for detecting whether or not. In the computer,
In the detection procedure, when the moving direction of the object detected in the detection procedure is the vertical direction on the screen, it is detected whether the object is a specific object by pattern matching processing that matches the object with a predetermined matching pattern. When the moving direction of the object detected in the detection procedure is the horizontal direction on the screen, the object is detected by a detection method that is different from the pattern matching process and uses the aspect ratio, size, or shape of the detected object. The program according to claim 11 for executing a process of detecting whether or not the object is the specific object. On the computer,
A detection procedure for detecting an object from the image;
A determination procedure for determining a moving direction in which the object detected in the detection procedure moves;
Whether the object is a specific object is detected based on a sum total weighted to each local feature amount of the object detected in the detection procedure, and the object detected in the detection procedure is A program for executing a detection procedure for changing a weight added to each local feature of an object.

Images