JP2015080220A - Video processing method and apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To process a person area reflected in a video so that the action of a person can be monitored practically, while taking account of privacy, in a realistic processing time.SOLUTION: A person is detected from a captured video, and if a person other than the person thus detected exists in a predetermined range from the person thus detected, the whole region occupied by the person thus detected is determined as a region to be processed. If a person other than the person thus detected does not exist in a predetermined range from the person thus detected, the region occupied by the person thus detected from which a local area is removed is determined as a region to be processed, and the region thus determined is subjected to processing.

Description

  The present invention relates to a method and apparatus for switching video processing depending on the number of people detected per area in a captured video.

  With the increase in crime rates, the number of cameras installed to monitor public spaces is increasing. Although it has a crime prevention effect, there is also a concern that it may infringe on the privacy of the person shown in the video.

Japanese Patent Application Laid-Open No. 2004-228561 discloses that when a face of a person detected in a captured image does not match face information registered in advance, a process for abstracting the face area is performed.
Patent Document 2 discloses a method of switching analysis processing based on the number of people shown in the video captured by the surveillance camera. For example, when there are a large number of people in the captured video, the load of analysis processing can be reduced.

JP 2004-62560 A JP2007-264706 US Published Patent 2007/0237387

  In the technique disclosed in Patent Document 1, when the processing is performed for each person shown in the captured video, there is a problem that the processing load becomes excessive when the number of persons to be processed is large. In addition, there is a problem that it is impossible to determine a person other than the recorded person, and there is a problem that the relationship between the person shown in the photographed image and the other person is not taken into consideration.

The technique disclosed in Patent Document 2 has a problem in that privacy protection is not sufficient because the relationship between a person shown in a captured image and other persons is not taken into consideration.
In view of the above, in order to select an appropriate processing load while taking privacy protection into consideration, an object of the present invention is to switch video processing processing depending on the number of people detected per area in a captured video.

  The above object can be achieved by the following method.

Detection means for detecting a person from the captured video image;
When there is a person other than the detected person in an area within a predetermined range from the detected person, the entire area occupied by the detected person is determined as a processing target area, When there is no person other than the detected person in the area within the predetermined range from the detected person, the area excluding the local area of the area occupied by the detected person is the processing target area Determining means to determine as,
An image processing apparatus comprising: processing means for performing processing on the target area determined by the determining means.

  The above object can also be achieved by the following method.

Detecting means for dividing a captured image into a plurality of areas and detecting a person for each of the divided areas;
Acquisition means for acquiring the number of persons detected by the detection means for each region;
A determining unit that determines a target area for performing processing on the detected person area according to the number of the acquired persons for each of the areas;
An image processing apparatus comprising: processing means for performing processing on the target area determined by the determining means.

  By switching the video processing depending on the number of people detected per area in the captured video, it is possible to select an appropriate processing load while taking privacy protection into consideration.

It is a functional block diagram of the image processing apparatus of the first embodiment. It is a whole processing flow figure of the image processing device of a 1st embodiment. It is a figure explaining the relationship between a division area and a person. It is a figure explaining a mode that a person is distributed and concentrated. It is a figure explaining the processing method of a person area. It is a functional block diagram of the image processing apparatus of the second embodiment. It is a whole processing flow figure of the image processing device of a 2nd embodiment. It is a figure explaining the relationship between a person and a peripheral region. It is a figure which shows the structural example of a computer.

  Hereinafter, description will be made based on embodiments of the present invention.

[First Embodiment]
FIG. 1 is a functional configuration diagram for realizing the video processing apparatus 100 according to the first embodiment.

  An overview of each processing unit constituting the function shown in FIG. 1 will be described below.

  As shown in FIG. 1, the video processing apparatus 100 includes an imaging unit 110, a detection unit 120, a division unit 130, a counting unit 140, a selection unit 150, and a processing unit 160.

  The imaging unit 110 captures a real space using a video input device such as a camera. The captured image captured by the imaging unit 110 is output to the detection unit 120.

  For example, the video input device may be suspended from the ceiling, or may be installed on a floor or a stand. In the following description, it is assumed that a camera suspended from a ceiling in a station premises or a store is used. Camera parameters such as pan / tilt and zoom may be fixed or variable. When the camera parameter is fixed, the parameter is measured in advance or is in a referable state. When the camera parameter is variable, the variable value is measured and output by the video input device.

  The detection unit 120 receives a captured image from the image capturing unit 110 and performs processing for detecting a person from the captured image. For detection, the technique described in Patent Document 3 may be used. The detection unit 120 acquires person position information from the result of detecting a person from the captured video, and outputs the person position information to the counting unit 140. In addition, the detection unit 120 acquires the person area information and outputs it to the selection unit 150. The person area information here is an area where a person obtained from a person detection result is detected, and the person position information is a center position coordinate in the person area obtained from a person detection result.

  The dividing unit 130 performs processing for receiving the captured video from the imaging unit 110 and dividing it into several regions. For example, the captured video input by the imaging unit 110 is divided so that each divided area has an equal size. Hereinafter, the area divided by the dividing unit 130 is referred to as a divided area. The divided areas are output to the counting unit 140.

  The counting unit 140 counts the number of persons in which a person is shown in each person area. Then, the number of persons for each person area is output to selection section 150. A divided area is received from the dividing unit 130, and the number of divided areas in which a person is detected in the divided areas is counted among the divided areas. Then, the area where the person is detected (person area) is output to the counting unit 140, and the number of divided areas where the person is detected (number of person areas) is output to the selection unit 150.

  The selection unit 150 receives the number of person areas from the counting unit 140, receives the number of people per area from the counting unit 140, and selects an algorithm used for the processing of the person area. The person area information is received from the detection unit 120, the algorithm information for each area is input from the selection unit 150, and the processing area is set. The processing area is set by associating a person area detected by the detection unit 120 with an algorithm for processing the area, and associating the person area with the processing method. Then, the algorithm for processing the person area set for each person area detected from the detection unit 120 and the algorithm for each person area are output to the processing unit 160.

  The processing unit 160 receives a captured image from the image capturing unit 110, receives an algorithm for each person area from the selection unit 150, and processes an area in which the person in the captured image is shown. As a result, a processed image can be created in the person area, and the image can be made in consideration of privacy.

  Processing performed by the video processing apparatus 100 according to the first embodiment will be described using the flowchart shown in FIG. The program code according to the flowchart is stored in a memory 902 such as a RAM or a ROM in the apparatus of the first embodiment, and is read and executed by the CPU 901 or the like.

  In step S <b> 201, shooting of the real space is performed by the shooting unit 110. Then, the captured video is sent to the detection unit 120, and processing for detecting a person from the captured video is performed.

  In step S202, the input video is divided into a plurality of regions. At this time, the method of dividing the input video can be arbitrarily determined. For example, as shown in FIG. 3, the input video 300 may be divided into three equal parts in the vertical and horizontal directions, and one frame image may be divided into nine. Here, (1) to (9) in FIG. 3 are divided areas having a horizontal size x and a vertical size y, respectively. FIG. 3 also shows the persons A to F shown in the input video and the center x of each person area for later explanation.

  In step S203, steps S204 to S207 described later are repeated for each divided region.

  In step S204, the number of persons shown in each divided area is counted. Among the divided areas received from step S202, the number of divided areas (number of person areas) in which a person is shown is counted. Whether or not a person is shown in each divided area is determined by using a person detection result obtained from the detection unit 120. The person detection result obtained from the detection unit 120 is, for example, a rectangular rectangular area that encompasses the entire person area. Therefore, the center of the person is defined as the center of the rectangular area obtained from the detection result of the person, and by determining whether or not the person center point is included in each of the divided areas, the presence / absence of the person in the divided area is determined. I can decide. In the example shown in FIG. 3, it is assumed that person A belongs to area (1), person B and person C belong to area (3), person D belongs to area (6), and person E and person F belong to area (9). To be judged. If the person area does not fall within one divided area, such as person D or person E, and spans several divided areas, the area having the center X of the person is set as the area to which the person belongs. Will be judged. Therefore, in the case of the example shown in FIG. 3, the number of areas in which a person is reflected is counted as four (1), (3), (6), and (9). Similar to the determination of whether or not a person exists in the divided area, the number of persons appearing in the divided area is counted using the center point of the person area included in the divided area. In the example shown in FIG. 3, one person is counted in the area (1) and the area (6), two persons are counted in the area (3) and the area (9), and zero persons are counted in the other areas.

  In step S205, it is determined whether a person exists in each of the divided areas. If no person exists, the number of persons is counted in step S204 for the next divided area. If there is a person, the process proceeds to step S206.

  In step S206, a processing method is selected using the number of divided areas (number of person areas) in which a person is shown and the number of persons in the divided areas (number of people per area).

  Using the number of persons in the divided area (number of persons per area), it is determined whether the number of persons is 0 or more. If the person is not shown in the divided area, it is not necessary to perform special processing on the person area, and the process proceeds to the next divided area without performing the processing. If a person appears in the divided area, some processing is performed on the person area.

  Here, a method is selected such that the processing time for each frame of the input video is approximately the same regardless of the number of people shown in the frame and the distribution of the people shown. 4A and 4B are captured images input from the imaging unit 110. FIG. FIG. 4A is a screen shot when the distribution of people shown in the input video is collected, and FIG. 4B is a screen shot when the distribution of people shown in the input video is dispersed. It is. Here, when an algorithm is selected according to the number of persons shown in the captured video so that the processing time required for processing the captured video is approximately the same, 11 persons are shown in FIG. 4A, and FIG. Since 10 people are shown, the same algorithm is selected. Alternatively, an algorithm with a lighter load is selected in FIG. However, since the person in FIG. 4A has an overlap between persons, the size of the video area to be actually processed is smaller in FIG. 4A than in FIG. 4B. Then, in order to make the processing time for each of the same level, a high-load algorithm is selected for FIG. 4A where the area where the person is reflected is small, and FIG. For b), select a light load algorithm. In step S206, selection is performed using the number of divided areas (number of person areas) and the number of persons in the divided areas (number of people per area). A specific algorithm selection method will be described with reference to FIGS.

  Hereinafter, a method for selecting an algorithm from among different processing region setting methods will be described. Regarding the video conversion method after the processing area is set, a plurality of methods can be selected from painting with a specified color, mosaic processing, and video conversion using a different method for each body part. This is the same as the selection of the processing area setting method.

  FIG. 5 shows a method of setting a video processing region using a different algorithm. Specifically, a method for setting a processing region based on a person detection or recognition algorithm is shown in order of increasing processing load. Here, since the processing time for the processing area setting method is longer than the processing time for the video conversion method, the processing time of the processing method including the two methods depends on the processing area setting method. Therefore, the order in which the processing load of the processing method is light and the order in which the processing load of the processing area setting method is light are the same. In order from the top of FIG. 5 are a human detection base processing method 500, a silhouette detection base processing method 510, a human body part detection base processing method 520, a posture recognition base processing method 530, and an action recognition base processing method 540.

  The person detection base processing method 500 is a method of performing processing such as painting on a rectangular region including a region where a person is detected. The silhouette detection base processing method 510 is a method for detecting a silhouette of a person and performing processing on the region in the silhouette. The human body part detection base processing method 520 is a method of recognizing parts such as a hand, a foot, and a head of a human body and performing processing for each recognized part region. For example, if a painting process is performed on a part such as a “hand” that plays an important role in monitoring a person, the behavior cannot be monitored. Therefore, only the parts other than “hand” are processed. The posture recognition base processing method 530 is a method for recognizing the posture of a person and performing a processing process for each recognized posture. For example, a normal posture and an abnormal posture are distinguished from each other, and a painting process is performed when the posture is a normal posture that does not need to be monitored. The action recognition based processing method 540 is a method for recognizing a person's action and performing a process for each recognized action. For example, normal behavior and abnormal behavior are differentiated, and the painting process is performed when a normal behavior that does not need to be monitored is performed. The higher the processing method is, the more the processing can be performed selectively only on the portion to be processed or at the timing to be processed, instead of simply processing the human region every time.

  Here, the persons A to F in FIG. 3 are processed by selecting any one of the processing methods 500 to 540 shown in FIG. The processing method is selected based on the relationship between the number of persons in the divided area to be processed (number of people per area), the processing time of each processing method 500 to 540, and the processing time spent for one frame. For example, the processing time of the person detection base processing method 500 is 10 and the processing time of the silhouette detection base processing method 510 is 20. Further, the processing time of the human body part detection base processing method 520 is set to 30, the processing time of the posture recognition base processing method 530 is set to 40, and the processing time of the action recognition base processing method 540 is set to 50. The processing time used for one frame is set to 100 or less. Since there are two divided areas in which one person is shown and two divided areas in which two persons are shown, if processing time is allocated at the same rate, the processing time used for one divided area is 100/4 = 25. Therefore, the processing method of the person A in the area (1) and the person D in the area (6) shown by one person is a silhouette detection base processing method 510 that is a processing method with a processing time of 25 or less. Further, the processing method of the person B and the person C in the area (3) and the person E and the person F in the area (9) shown by two people is a processing method with a processing time of 25/2 = 12.5 or less. This is a person detection base processing method 500. For example, the accuracy of the mosaic may be changed according to the processing time.

  As described above, when the processing method is selected for the divided area where the person is shown, the process proceeds to step S207.

  In step S207, each person region is associated with the processing method using the region-by-region processing method received from step S206.

  With the above processing, when the number of people per video area in the captured video is large, the light-load processing method is used, and when the number of people per video area in the captured video is small, the high-load processing method is used. It can be used to process human images.

[Second Embodiment]
FIG. 6 is a functional configuration diagram for realizing the image processing apparatus 600 according to the second embodiment. An overview of each processing unit constituting the function shown in FIG. 6 will be described below.

  In the second embodiment, in contrast to a person who exists as a group within a certain range, a person who does not exist within a certain range and is acting alone Change the processing method. For example, it is assumed that people who are present in a flock are within reach of each other's eyes, and therefore need not be monitored in detail. On the other hand, a person who is alone is out of reach of others even if he / she takes suspicious behavior. Thus, a simple processing method is selected for a person who is present in a group, and a processing method that can be grasped in detail even if time is required for a person who is acting alone.

  As shown in FIG. 6, the video processing apparatus 600 includes an imaging unit 110, a detection unit 120, a counting unit 140, a selection unit 150, a processing unit 160, and a classification unit 610.

  The classification unit 610 identifies one person (specific person) from the person area received from the detection unit 120. As for the specific person, any person in the input video may be used, but it is assumed that the person area has not been processed yet at the time of identification. The specific person may use the recognized content such as parts and actions used in the detection of the person.

  The peripheral area of the person is set from the specific person. The peripheral region setting method may be an arbitrary divided region such as a circular region or a rectangular region centered on a specific person. In the second embodiment, the peripheral region is set within a circular region having a distance of a predetermined radius r centered on the center position of the specific person.

  A person in the peripheral area of the specific person is searched from the peripheral area and classified into the same group as the specific person. When searching for a person in the surrounding area, the center point of the person area obtained from the detection unit 120 is used. If there is a center point of a person area other than the specific person in the peripheral area, the person is classified into the same group as the specific person. Then, the classification result of the specific person and the surrounding persons is output to the selection unit 640. In addition, the classification result of persons other than the specific person and other than the surrounding persons may be output to the selection unit 640.

  The selection unit 150 selects a processing method for each classified person area from the classification result of the person received from the classification unit 610. The group-by-group processing method is a method for processing the person area of the group selected for each group classified by the classification unit 610.

  Processing performed by the video processing apparatus 600 according to the second embodiment will be described with reference to the flowchart shown in FIG. The program code according to the flowchart is stored in a memory 902 such as a RAM or a ROM in the apparatus of the second embodiment, and is read and executed by the CPU 901 or the like.

  In step S701, imaging is performed in the same manner as in step S201 of the first embodiment, and a person area is detected.

  In step S702, one person area is arbitrarily specified from the person areas received from the detection unit 120. The identification of the first person area can be arbitrarily performed, but the identification of the person area for the second and subsequent times is selected from person areas that are unclassified and have not been processed.

  In step S703, whether the peripheral area is a square or a circle, what kind of figure the area is, and the size of the area are defined. In the second embodiment, the peripheral region is a circular region having a predetermined radius r. Using a specific person, a circular area having a predetermined radius r around the center point of the person area of the specific person is defined as a peripheral area.

  In step S704, a person in the peripheral area of the specific person is searched from the peripheral area and classified into the same group as the specific person. When searching for a person in the surrounding area, the center point of the person area obtained from the detection unit 120 is used. If there is a center point of a person area other than the specific person in the peripheral area, the person is classified into the same group as the specific person. Then, the classification result of the specific person and the surrounding persons is output to the selection unit 640. In addition, the classification result of persons other than the specific person and other than the surrounding persons may be output to the selection unit 640.

  In step S705, the number of persons present in the peripheral area of any specific person is counted. The number of classified peripheral persons is counted, and the process proceeds to step S706.

  In step S706, it is determined whether the number of surrounding people is zero or more. The processing contents differ between the case where there are one or more neighboring persons and the case where there are no neighboring persons. Specifically, if there are one or more neighboring persons, the process proceeds to step S707 and step S708. If no one exists, the process proceeds to step S709 and step S710.

  In step S707, the classification unit 610 classifies the neighboring persons counted in step S705 and the specific person specified in step S702 as the same group.

  A specific classification example will be described with reference to FIG. FIG. 8 shows the input video 300, the photographed persons A to F, the center X of the person area, and the groups α to γ added from the second embodiment, as in the first embodiment. All the people shown in the input video 300 belong to three groups α to γ. Hereinafter, a method for grouping the persons A to F will be described.

  One person is identified from the persons in the input video 300. In the second embodiment, the person A is set as a specific person. Then, a circular area having a predetermined radius r is defined as the peripheral area of the person A, and the persons in the peripheral area are classified into the same group α as the person A. In the input video 300, since there are no people in the peripheral area of the person A, only the person A belongs to the group α.

  Next, the person B is set as a specific person from the unclassified person group (persons B to F) in the input video 300, and the persons in the peripheral area of the person B are classified into the group β. As a result, the persons classified into group β are person B, person C, and person D. Similarly, when the group E is classified with the person E as the specific person, the persons classified into the group γ are the person E and the person F.

  As described above, the persons in the input video 300 are divided into three groups, and the persons belonging to each group are as follows.

Group α: Person A
Group β: Person B Person C Person D
Group γ: Person E Person F
In step S <b> 708, the processing unit 160 performs processing for the surrounding person using the processing method for each person area set by the selection unit 150.

  For example, it is assumed that people who are present in a flock are within reach of each other's eyes, and therefore need not be monitored in detail. On the other hand, a person who is alone is out of the reach of others even if he / she takes suspicious behavior. Processing is performed according to the need for such monitoring. For example, when processing by a simple method, the processing may be performed on a rectangular region including a region where a person exists.

  In step S709, the specific person specified in step S702 is recognized in more detail than the detection performed in the detection unit 120. Here, information necessary for the processing in step S710 is recognized. For example, person recognition, clothes recognition, posture recognition, motion recognition, and the like are performed. Or it recognizes who it is, what kind of person it is (man or woman, how old, etc.). Furthermore, it recognizes what kind of clothes, what posture it is, what movement it is doing, what action it is doing, what it has.

  In step S710, the processing unit 160 performs the processing for the specific person specified in step S702 using the processing method for each person area set in the selection unit 150. Here, the processing method related to a person who does not have any surrounding person but exists alone is determined before the start of the process. For example, if the silhouette information of a specific person can be obtained from step S709, it is used to process the silhouette area of that person. If any recognition result or the like can be obtained from step S709, the processing process may be performed on a video area in which a person is selectively or partially shown after determining the area and timing to be processed using them. .

  In step S711, each time the processing in steps S708 and S710 ends, it is determined whether there is a person whose person area is not processed in the captured video. If the processing process has not been completed for all the person areas in the captured video, the process returns to step S702 again to continue the process, but if there is no unprocessed person, the process ends.

  With the above processing, it is possible to easily select a processing process for a person who is present in a group, and for a person who is acting alone, a processing process that can be seen from a captured image can be selected. .

[Other Embodiments]
FIG. 9 is a diagram illustrating a configuration example of a computer. The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to the system or apparatus via the network 907 or various computer-readable storage media 902 and 903. Then, the computer 901 (or CPU, MPU, etc.) of the system or apparatus reads and executes the program.

Claims (12)

Detection means for detecting a person from the captured video image;
When a person other than the detected person exists in an area within a predetermined range from the detected person, the entire detected person area is determined as a processing target area and the detected person is detected. If there is no person other than the detected person in an area within a predetermined range from the detected person, a determination is made to determine an area excluding the local area of the detected person area as the processing target area Means,
An image processing apparatus comprising: processing means for performing processing on the target area determined by the determining means.   The said determination means determines the area | region except the area | region of the detected person's hand as said object area | region, when there is no person different from the said detected person. Video processing equipment.   The determination means further determines the entire other person area as a processing target when another person different from the detected person exists in an area within a predetermined range from the detected person. The video processing apparatus according to claim 1, wherein the video processing apparatus is a video processing apparatus.   The processing means includes a case where there is a person different from the detected person in a region within a predetermined range from the detected person, and a case where there is no person different from the detected person. The video processing apparatus according to claim 1, wherein the processing to be performed on the target area is different.   When the detected person is different from the detected person in an area within a predetermined range from the detected person, the processing means, for the other person and the detected person, The video processing apparatus according to claim 1, wherein the same processing is performed. Detecting means for dividing a captured image into a plurality of areas and detecting a person for each of the divided areas;
Acquisition means for acquiring the number of persons detected by the detection means for each region;
A determining unit that determines a target area for performing processing on the detected person area according to the number of the acquired persons for each of the areas;
An image processing apparatus comprising: processing means for performing processing on the target area determined by the determining means.   The video processing apparatus according to claim 6, wherein the determination unit determines a larger region as the target region as the number acquired by the acquisition unit increases.   The video processing apparatus according to claim 1, wherein the processing process is a painting process.   The video processing apparatus according to claim 1, further comprising a photographing unit that photographs the photographed video. A detection step of detecting a person from the captured video,
When there is a person other than the detected person in an area within a predetermined range from the detected person, the entire area occupied by the detected person is determined as a processing target area, When there is no person other than the detected person, a determination step of determining an area excluding a local area of the area occupied by the detected person as a processing target area;
And a processing step of performing processing on the target area determined by the determining means. A detection step of dividing the captured video into a plurality of areas and detecting a person for each of the divided areas;
An acquisition step of acquiring the number of persons detected by the detection means for each region;
A determination step for determining a target area for performing processing on the detected person area according to the number of the acquired persons for each area;
And a processing step of performing processing on the target area determined by the determining means.   The program for functioning a computer as each means of the image processing apparatus of any one of Claim 1 thru | or 9.

Images