JP7182893B2 - Image processing device, imaging device, image processing method, and program - Google Patents

Description

The present invention relates to image processing technology.

2. Description of the Related Art In recent years, attention has been focused on an attribute estimation technique for estimating a person's attribute information (gender, age, etc.) by analyzing a person present in an image captured by an imaging device. Patent Literature 1 describes a technique for estimating attributes of a person from an image.

Patent No. 4888217

However, in the above-mentioned Patent Literature 1, there is no consideration of a situation where a person can be detected from an image captured by an imaging device, but attribute estimation cannot be performed because the size of the person in the image is small. On the other hand, when estimating the attributes of multiple people included in the shooting range, if the angle of view of the imaging device is changed so that the attributes can be estimated, the user will not be able to see the images of the people who are outside the angle of view of the imaging device. It was necessary to re-control the angle and estimate the attribute.

SUMMARY OF THE INVENTION The present invention has been made in view of such problems, and it is an object of the present invention to estimate the attributes of a plurality of people over a wider range while changing the angle of view so that the attributes can be estimated.

In order to solve the above problems, the image processing apparatus of the present invention has the following configuration. That is, detection means for detecting a plurality of persons included in an image captured by a photographing means; determination means for determining two or more areas containing at least one of the plurality of persons detected by the detection means; Control means for outputting a control command for sequentially photographing each of the areas determined by the determination means by the photographing means; and an estimating means for estimating attribute information of a person existing in the area, wherein one of the areas determined by the determining means is included in the photographing range of the photographing means, and the one An image processing apparatus, wherein the control command is output based on photographing conditions that enable estimation of the attribute information of a person existing in one area.

According to the present invention, it is possible to estimate the attributes of a plurality of persons over a wider range while changing the angle of view so that the attributes can be estimated.

1 is a schematic diagram showing a system configuration; FIG. 1 is an external view of an imaging device; FIG. It is a functional block diagram of an imaging device. 1 is a functional block diagram of an image processing device; FIG. 4 is a flowchart of image processing; It is a schematic diagram for demonstrating image processing. It is a table used in image processing. Fig. 3 is a table resulting from image processing; 4 is a flowchart of image processing; It is a schematic diagram for demonstrating image processing. 1 is an example of a functional block diagram of an imaging device and a client device having some functions of an image processing device; FIG. 2 is a schematic diagram showing the hardware configuration of an image processing apparatus; FIG.

The image processing apparatus according to the present embodiment is an image processing apparatus that performs attribute estimation for estimating human attribute information (sex and age) for each of a plurality of estimation regions in successive images captured by an imaging device. .

Embodiments of the present invention will be described below with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and are not limited to the illustrated configurations.

(this embodiment)
FIG. 1 is a diagram showing a system configuration according to this embodiment. The image processing device 100 is a device that executes image processing, which will be described later. Note that the image processing apparatus 100 is realized by, for example, a personal computer or the like in which a program for realizing image processing functions described later is installed.

The display device 101 is connected to the image processing device 100, and is a display device for a user to view data output by image processing described later, a UI (user interface), and the like.

The photographing device 102 is a device for photographing an image, and is configured to be capable of Pan-Tilt-Zoom (hereinafter referred to as “PTZ”) control. Also, the imaging device 102 is, for example, a network camera capable of transmitting image data of an image via a network.

A recording device 103 is a device capable of recording image data of an image captured by the imaging device 102 . Also, the image processing apparatus 100 , the image capturing apparatus 102 , and the recording apparatus 103 communicate via the network 104 . The network 104 is composed of a plurality of routers, switches, cables, etc. that satisfy communication standards such as Ethernet (trademark). In this embodiment, any communication standard, scale, or configuration is acceptable as long as the image processing apparatus 100, image capturing apparatus 102, and recording apparatus 103 can communicate with each other. For example, the network 104 may be configured by the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), or the like.

In the configuration of FIG. 1, image data (live video) of an image captured by the image capturing apparatus 102, image data (images captured in the past) recorded by the recording apparatus 103, and the like are transmitted to the image processing apparatus 100. be.

Next, with reference to FIG. 12, the hardware configuration of the image processing apparatus 100 for realizing each function of this embodiment, which will be described later, will be described. A RAM (Random Access Memory) 1201 temporarily stores computer programs executed by a CPU (Central Processing Unit) 1200 . The RAM 1201 also temporarily stores data (commands and image data) obtained from outside via the communication interface 1203 . The RAM 1201 also provides a work area used when the CPU 1200 executes image processing, which will be described later. Also, the RAM 1201 functions, for example, as a frame memory or as a buffer memory.

CPU 1200 executes computer programs stored in RAM 1201 . In addition to the CPU, a processor such as a DSP (Digital Signal Processor) or an ASIC (Application Specific Integrated Circuit) may be used.

A HDD (Hard Disk Drive) 1202 stores an operating system program and image data.

Computer programs and data for executing the flowcharts of FIGS. 5 and 9, which will be described later, are stored in the HDD 1202, loaded into the RAM 1201 as appropriate under the control of the CPU 1200, and executed by the CPU 1200. FIG. In addition to the HDD, other storage media such as flash memory may be used.

Next, the imaging device 102 will be described with reference to FIGS. 2 and 3. FIG. FIG. 2 is an external view of the imaging device 102 according to this embodiment. Also, FIG. 3 is a functional block diagram of the imaging device 102 according to this embodiment.

The direction in which the optical axis of the lens 202 faces is the imaging direction of the imaging device 102 , and the light flux passing through the lens 202 forms an image on the imaging element of the imaging unit 205 . Note that the lens 202 includes a focus lens, a zoom lens, and the like. Also, the lens drive unit 210 includes drive systems such as a focus lens and a zoom lens, and changes the focal length of the lens 202 . The lens drive unit 210 is controlled by the pan/tilt/zoom control unit 208 .

The pan drive unit 200 includes a mechanical drive system that performs a pan operation and a motor as a drive source, and drives to change the photographing direction of the photographing apparatus 102 to a pan direction 203 . Also, the pan drive unit 200 is controlled by a pan tilt zoom control unit 208 .

The tilt drive unit 201 includes a mechanical drive that performs a tilt operation and a motor as a drive source, and drives the image pickup device 102 to change the shooting direction to a tilt direction 204 . The tilt drive unit 201 is controlled by a pan/tilt/zoom control unit 208 .

The imaging unit 205 has an imaging element such as a CCD (charge coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor. The imaging unit 205 photoelectrically converts the subject image formed through the lens 202 to generate an electric signal.

The image processing unit 206 performs processing for converting the electrical signal photoelectrically converted by the imaging unit 205 into a digital signal, compression encoding processing, and the like, and generates image data.

The pan/tilt/zoom control unit 208 controls the pan drive unit 200 , the tilt drive unit 201 and the lens drive unit 210 based on the instructions transmitted from the system control unit 207 .

A communication unit 209 is an interface that communicates with the image processing apparatus 100 . For example, the communication unit 209 transmits generated image data to the image processing apparatus 100 . The communication unit 209 also receives control commands for controlling pan/tilt/zoom operations of the image capturing apparatus 102 transmitted from the image processing apparatus 100 and transmits the control commands to the system control unit 207 . Also, the communication unit 209 has a wired LAN interface and a wireless LAN interface.

The system control unit 207 controls the entire imaging device 102 and performs, for example, the following processing. That is, the system control unit 207 analyzes the control command transmitted from the communication unit 209 and performs processing based on the analyzed content. For example, the system control unit 207 instructs the pan/tilt/zoom operation to the pan/tilt/zoom control unit 208 .

Note that the pan angle is the angle of the imaging direction (optical axis) in the pan direction 203 of the imaging apparatus 102 when one driving end of the pan driving unit 200 is 0°. The tilt angle is the angle of the photographing direction (optical axis) in the tilt direction 204 of the photographing apparatus 102 when one drive end of the tilt drive unit 201 is 0°.

Next, the image processing apparatus 100 according to this embodiment will be described with reference to FIG. FIG. 4 is a functional block diagram of the image processing apparatus according to this embodiment. Note that each functional block shown in FIG. 4 may be realized by a software module or by a hardware module. Alternatively, a software module and a hardware module may be implemented in cooperation. Note that in the following description, each function of the image processing apparatus 100 is assumed to be executed by the CPU 1200 of the image processing apparatus 100. FIG.

The image processing apparatus 100 according to this embodiment includes a communication unit 400, a detection area setting unit 401, an operation reception unit 402, a detection unit 403, an estimation area determination unit 404, a command management unit 405, and an attribute estimation unit. 406 , an output control unit 407 , and a storage unit 408 .

The communication unit 400 performs communication for receiving image data of an image captured by the imaging device 102 and for transmitting control commands for controlling panning, tilting, and zooming of the imaging device 102 . The communication unit 400 also receives image data stored in the recording device 103 or image data stored in a storage device of another external device, for example, transmitted from each device via the network 104 .

The detection area setting unit 401 sets a detection area, which is an area to be processed by the detection unit 403 in the image obtained via the communication unit 400 . Note that an area within the image set by the user via the operation reception unit 402 may be set as the detection area, or a predetermined area set in advance may be set as the detection area. Note that if there is no detection area specified by the user or a previously set area, the entire image indicated by the image data obtained via the communication unit 400 may be used as the detection area.

An operation reception unit 402 receives an operation performed by a user via an input device (not shown) such as a mouse and keyboard. The operation reception unit 402 in this embodiment receives, for example, information on the position of the detection area set by the user via the input device for the image output to the display device 101 by the output control unit 407 .

The detection unit 403 detects a plurality of persons included in the detection area in the image obtained via the communication unit 400 . Note that the detection unit 403 according to the present embodiment holds a discriminator that is created after learning a human body feature amount (Haar-Like feature amount, HOG feature amount, etc.) from a learning image. Then, the detection unit 403 receives an image as input to the classifier that is the result of learning, and detects a human body from the image. Note that the detection unit 403 in this embodiment is described as performing human body detection for detecting a human body, but the present invention is not limited to this. For example, the detection unit 403 may perform face detection for detecting a person's face or head detection for detecting a person's head. In the following description, the human body means the whole body of a person, including the face and body of the person.

Based on the result of detection by the detection unit 403 , the estimated region determination unit 404 determines a plurality of estimated regions including at least one of the plurality of persons detected by the detection unit 403 . A person included in the estimation area is subject to estimation of attribute information by the attribute estimation unit 406 . Note that information about the position of the estimated region is stored in the storage unit 408 .

A command management unit 405 generates a control command for controlling the image capturing apparatus 102 to sequentially capture each of the plurality of estimated areas determined by the estimated area determination unit 404 . At this time, the imaging device 102 is controlled so that one estimated region is included in the imaging range of the imaging device 102, and the imaging conditions are such that the attribute estimation unit 404 can estimate the attribute information of a person existing in the estimated region. A control command is generated. At this time, the control command generated by the command management unit 405 is, for example, a control command for controlling at least one of panning, tilting, and zooming of the imaging device 102 . Note that the control command generated by the command management unit 405 in this embodiment is sent to the system control unit 207 in the image capturing apparatus 102 via the communication unit 400 and the communication unit 209, and the image capturing apparatus is controlled based on the control command. 102 control is performed.

The attribute estimating unit 406 estimates the attribute information of a person present in the estimation area for each captured image obtained from the imaging device 102 . The attribute estimating unit 406 according to the present embodiment estimates attribute information (information regarding sex and age) from a person's face region using a classifier learned from learning images. When estimating attribute information from a person's face area, it may not be possible to estimate the attribute information if the size (number of pixels) of the face area included in the image is small. In other words, when estimating attribute information from a person's face area, there is a minimum required number of pixels in the face area. Also, although the attribute information in the present embodiment is explained as information about a person's sex and information about a person's age, it is not limited to this. For example, it may be information about race, clothing, hairstyle, and the like.

The output control unit 407 outputs the image data of the image obtained from the imaging device 102 and the attribute information estimation result obtained by the attribute estimation unit 406 to the display device 101 .

A storage unit 408 stores information about the positions of the detection area and the estimation area.

As described above, the imaging device 102 is controlled so that each of the plurality of estimation regions is sequentially photographed while satisfying the photographing conditions under which the attribute estimating unit 406 can estimate the attribute information of the person present in the estimation region. The attribute information of a person existing in each of the estimation regions is estimated. By doing so, it is possible to estimate the attributes of a plurality of people over a wider range.

Next, image processing in this embodiment will be described with reference to the flowchart shown in FIG. FIG. 5 is a flowchart showing the flow of image processing according to this embodiment. The processing of the flowchart shown in FIG. 5 is mainly executed by each functional block shown in FIG. The processing of the flowchart shown in FIG. 5 is executed by the CPU 1200 of the image processing apparatus 100 according to the computer program stored in the HDD 1202 . Processing executed by the CPU 1200 of the image processing apparatus 100 will be described below.

In S<b>501 , the communication unit 400 receives image data of an image captured by the imaging device 102 .

Next, in S502, the detection area setting unit 401 detects an area, which is an area to be subjected to processing for detecting a plurality of people included in the image by the detection unit 403, in the image obtained via the communication unit 400. Set a region. The operation reception unit 402 in this embodiment receives information about the position of the area in the image set by the user for the image output to the display device 101 by the output control unit 407 . Then, the detection area setting unit 401 sets the position of the area in the image, which is the information received by the operation receiving unit 402, as the position of the detection area. Note that the position of the detection area in this embodiment is determined by the pan angle and tilt angle at the center of gravity of the detection area, and the zoom magnification that is the angle of view of the detection area. Information about the position of the detection area is stored in the storage unit 408 .

Next, in S<b>503 , the detection unit 403 detects a plurality of human bodies included in the detection area set by the detection area setting unit 401 . At this time, the detection unit 403 also detects the number of pixels in the face area of each of the plurality of human bodies detected within the detection area and the position information. Note that the detection unit 403 in this embodiment performs human body detection, and the number of pixels in the face area can be estimated from the number of pixels of the detected human body.

Next, in S504, if a plurality of human bodies do not exist in the detection area ('No' in S504), the process of S501 is performed.

In S504, when a plurality of human bodies exist in the detection area ('Yes' in S504), the process of S505 is performed. In S<b>505 , estimated region determining unit 404 determines a plurality of estimated regions including at least one of the plurality of human bodies detected by detecting unit 403 based on the result of detecting unit 403 . The processing of the estimated region determination unit 404 will be described in more detail below with reference to FIG.

6A and 6B are diagrams for explaining the processing of the estimated region determination unit 404. FIG. In FIG. 6A, an image 600 is an image captured by the imaging device 102 . A detection area 601 is an area within the image set by the user, and a plurality of human bodies detected by the detection unit 403 are present in the area.

A range 602 is a photographing range when photographing is performed at a zoom magnification that allows the attribute estimation unit 406 to acquire the minimum number of pixels in the face area necessary for estimating the attribute information of a person included in the range. In this case, a range 602 is a range photographed at a zoom magnification that can obtain the minimum number of pixels in the face region necessary for estimating the attribute information of each of the six persons included in the range. When the photographing range is reached, the attribute information cannot be estimated for the six persons. Note that the range 602 is determined by the estimated area determination unit 404 based on the number of pixels in the human face area detected by the detection unit 403 . For example, it is assumed that the number of pixels required for the attribute estimation unit 406 to estimate the attribute information of a person is 40 pixels or more in the vertical direction and 30 pixels or more in the horizontal direction. At this time, the estimated area determination unit 404 determines, as a range 602 , an imaging range photographed at the minimum zoom magnification for which the number of pixels of the human face area detected by the detection unit 403 satisfies the condition.

Then, the estimated area determining unit 404 determines an area 602a having an area equal to or less than the area of the range 602 and including at least one of the plurality of persons detected by the detecting unit 403 as an estimated area. At this time, the estimated area determination unit 404 assigns an estimated area ID of “1” for specifying the estimated area to the area 602 a that is the estimated area corresponding to the range 602 . As described above, the estimated area determination unit 404 determines the range 602 based on the number of pixels in the human face area detected by the detection unit 403, and determines the estimated area 602a based on the range 602. FIG.

Next, the estimated area determination unit 404 determines the range 603. FIG. As with the range 602, the range 603 is a photographing range when photographed at a zoom magnification that allows the attribute estimation unit 406 to acquire the minimum number of pixels in the face area necessary for estimating the attribute information of the person included in the range. be. Then, the estimated area determination unit 404 determines an area 603 b that is an area within the area 603 that is equal to or less than the area of the area 603 and that includes at least one of the plurality of persons detected by the detection unit 403 . At this time, the estimated area determining unit 404 assigns an estimated area ID of “2” to the area 603 b that is the estimated area corresponding to the range 603 .

Next, the estimated area determination unit 404 determines the range 604. FIG. As with the range 602, the range 604 is a shooting range when the image is taken at a zoom magnification that allows the attribute estimation unit 406 to acquire the minimum number of pixels in the face area necessary for estimating the attribute information of the person included in the range. be. Then, the estimated area determining unit 404 determines an area 604 c that is an area within the range 604 that is equal to or less than the area of the range 604 and that includes at least one of the plurality of persons detected by the detecting unit 403 . At this time, the estimated area determination unit 404 assigns an estimated area ID of “3” to the area 604 c that is the estimated area corresponding to the range 604 .

Next, the estimated area determination unit 404 determines the range 605. FIG. As with the range 602, the range 605 is a photographing range when photographed at a zoom magnification that allows the attribute estimation unit 406 to acquire the minimum number of pixels in the face area necessary for estimating the attribute information of the person included in the range. be. Then, the estimated area determining unit 404 determines an area 605 d that is an area within the range 605 that is equal to or less than the area of the range 605 and that includes at least one of the plurality of persons detected by the detecting unit 403 . At this time, the estimated area determination unit 404 assigns an estimated area ID of “4” to the area 605 d that is the estimated area corresponding to the range 605 .

In addition, the estimation area determination unit 404 includes an estimation area ID for specifying the estimation area, the position of the estimation area, the minimum necessary zoom ratio for estimating the attribute information of the person included in the estimation area, and the number of people included in the estimated area are stored in the table 700 in association with each other. Note that a table 700 shown in FIG. 7 is an example of a data structure diagram in which data is stored by the estimated region determination unit 404. FIG.

The estimated area ID identifies the estimated area and indicates the order of the estimated areas to be targeted when controlling the imaging device to sequentially capture images for each of the plurality of estimated areas. Also, the position of the estimation area is determined by the pan angle, the tilt angle, and the zoom factor, which is the angle of view of the estimation area, at the center of gravity of the estimation area. The required zoom magnification shown in FIG. 7 is the minimum required zoom magnification for estimating the attribute information of a person included in the estimation area. For example, the minimum necessary zoom magnification for estimating the attribute information of a person in the area 602a whose estimated area ID is "1" is the zoom magnification (gg) when the range 602 is the imaging range. At this time, the attribute estimating unit 406 cannot estimate the attribute information of the person in the area 602a when the photographing range is photographed at a zoom magnification (gg) or less. FIG. 6E is a diagram showing a plurality of estimated areas determined by the estimated area determination unit 404 and the estimated area IDs. 6E may be displayed on the display device 101 by the output control unit 407. FIG.

As described above, the estimated region determination unit 404 in this embodiment includes at least one of the plurality of persons detected by the detection unit 403 based on the number of pixels in the human face region detected by the detection unit 403. Multiple estimation regions are determined. Although the estimation area determination unit 404 in this embodiment determines the estimation area so as to scan the detection area 601 from the lower left to the right, the present invention is not limited to this. For example, the estimation area may be determined so as to scan from the upper left of the detection area 601 to the right.

Note that, when determining a plurality of estimation areas including at least one person among the plurality of persons detected by the detection unit 403, the estimation area determination unit 404 in this embodiment sets the estimation area so that the person's face area is included in the estimation area. determined to be, but not limited to. For example, it may be determined that the upper body of a person is included in the estimation area.

In addition, the estimation area determination unit 404 in this embodiment determines one estimation area for each of the ranges 602, 603, 604, and 605 so as to reduce the number of estimation areas. , but not limited to. For example, the estimated region determiner 404 may determine two estimated regions in the range 602 that include at least one person. By reducing the number of estimation regions determined by the estimation region determination unit 404, the time for the attribute estimation unit 406 to estimate the attribute information of the plurality of people detected by the detection unit 403 can be shortened. can be shortened.

Next, return to the description of the flowchart shown in FIG. In step S<b>506 , the command management unit 405 generates a control command for controlling the imaging device 102 to sequentially capture each of the plurality of estimation areas determined by the estimation area determination unit 404 . At this time, the imaging device 102 is controlled so that one estimated region is included in the imaging range of the imaging device 102 and the imaging conditions are such that the attribute estimation unit 406 can estimate the attribute information of a person existing in the estimated region. A control command is generated.

The command management unit 405 in this embodiment first controls the imaging device so that the estimated area 602a with the estimated area ID of "1" is captured. At this time, a control command is generated to control the imaging device 102 so that the estimated area 602a is included in the imaging range of the imaging device 102 and the zoom magnification of the imaging device 102 is equal to or greater than the required zoom magnification as imaging conditions. In this case, a control command is generated for controlling the imaging device 102 so that the zoom magnification of the imaging device 102 is equal to or greater than the required zoom magnification gg as the imaging condition.

Note that the command management unit 405 in this embodiment may generate a control command so that the center of gravity of the captured image captured by the imaging device 102 and the center of gravity of the estimated region are substantially the same. By doing so, even if the periphery of the photographed image is distorted, the distortion can be reduced in the estimation area, and the accuracy of estimating the attribute information can be further improved.

Next, in step S<b>507 , the attribute estimation unit 406 estimates the attribute information of a person existing in the estimation area for the captured image obtained from the imaging device 102 . In this case, the attribute information of the person present in the estimation area 602a is estimated.

In S507, when the attribute estimation unit 406 estimates the attribute information of a person existing in one estimation area, the process may proceed to S508 after a predetermined time (for example, 5 seconds) has elapsed.

When the attribute estimating unit 406 estimates the attribute information of a person existing in an estimated area in the captured image obtained from the imaging device 102, a person existing in another estimated area may appear in the captured image. For example, in FIG. 6C, assume that a photographed image is obtained in a photographing range having substantially the same position and size as the range 604 in order to estimate the attribute information of a person in the estimation region 604c. Suppose. At this time, the photographed image includes three persons in the estimation area 602a whose attribute information has already been estimated. At this time, the attribute estimating unit 406 performs the following processing so as not to redundantly estimate attribute information. For example, the attribute estimating unit 406 cuts out an estimated region 604c from the captured image 604, and estimates the attribute information of a person included in the image of the cut out estimated region 604c. Alternatively, the storage unit 408 may store the position information of the person whose attribute information has been estimated by the attribute estimation unit 406 . Then, for example, when estimating the attribute information of the estimation area 604c in the captured image, a person whose attribute information has already been estimated from the position information stored in the storage unit 408 may be excluded from the processing target. good.

Next, in S508, if there is an estimated area for which attribute information has not been estimated by the attribute estimation unit 406 ('No' in S508), the processes in S506 and S507 are repeated. In the case of this embodiment, the processing in S506 and S507 is repeated in the order of estimated area ID "2", estimated area ID "3", and estimated area ID "4".

Next, in S<b>508 , if there is no estimated area whose attribute information has not been estimated by the attribute estimation unit 406 (“Yes” in S<b>508 ), the process ends.

As described above, in the present embodiment, the imaging device 102 is controlled so that each of the plurality of estimation regions is sequentially photographed, and the attribute information of the person present in each of the plurality of estimation regions is estimated. By doing so, it is possible to estimate the attributes of a plurality of people over a wider range while changing the angle of view so that the attributes can be estimated.

Next, referring to FIG. 8, the output of the result of the image processing according to this embodiment is shown. FIG. 8 is a table in which the result of image processing according to this embodiment is output to the display device 101 by the output control unit 407. As shown in FIG.

For example, from 10:00 to 11:59, when the attribute estimation unit 406 estimates attribute information for a person in the estimation area 602a in S507 shown in FIG. When it is estimated, 1 is added to the number of counts from 20 to 29 for men. Similarly, in S507, when the attribute estimating unit 406 estimates that one person in the estimation area 602a is a 30-year-old woman, 1 is added to the count number of 30 to 39 women. In this way, in the processing in S507 shown in FIG. 5, the count may be added in real time to the graph displayed on the display device 101 each time the attribute information of the person is estimated. Further, after the image processing according to the present embodiment shown in FIG. 5 is completed, the attribute information estimation result may be output to the display device 101 .

In this embodiment, the total number of people is output as a graph for each piece of attribute information estimated by the attribute estimation unit 406 at predetermined time intervals, but the present invention is not limited to this. For example, the results estimated by the attribute estimation unit 406 may be aggregated for each estimated region ID.

In addition, for example, the number of people detected by the detection unit 403 may be output together with the total number of people whose attribute information is estimated by the attribute estimation unit 406 at predetermined time intervals. Thereby, the user can grasp how many people's attribute information has been estimated for the number of people detected by the detection unit 403 . Note that the number of people whose attribute information is estimated and the number of people detected by the detection unit 403 may be displayed for each estimated area ID.

Note that in S502 in the present embodiment, after the detection area setting unit 401 sets the detection area in the image obtained via the communication unit 400, the angle of view of the imaging device 102 is changed with respect to the detection area. The imaging device 102 may be controlled so as to do so. For example, after the user sets the detection area 601 for the image 600 shown in FIG. The angle of view may be changed by increasing the zoom magnification. This makes it possible to further improve the accuracy of detection by the detection unit 403 in S503.

Further, in the present embodiment, the imaging device 102 is controlled so as to sequentially capture a plurality of estimation regions in the order of estimation region IDs "1", "2", "3", and "4", but the present invention is not limited to this. . For example, the following processing may be performed.

In S503, when detecting a person included in the detection area 601, the detection unit 403 performs face detection to detect a person's face, and further calculates a detection score, which is an index of whether it is easy to estimate a person's attribute information. Output for each human face area. It should be noted that the higher the detection score, the easier it is to estimate attribute information for a person's face area. be done.

In S504, if a plurality of face areas exist in the detection area, it is assumed that a plurality of human bodies exist in the detection area, and the process proceeds to S505. In S<b>505 , estimated region determining unit 404 determines a plurality of estimated regions including at least one of the plurality of persons detected by detecting unit 403 based on the result of detecting unit 403 . Furthermore, the estimation area determination unit 404 adds up the detection scores output by the detection unit 403 for each estimation area.

Then, the processes of S506 and S507 may be performed for each of the plurality of estimation areas in accordance with the order of the highest detection score added up for each estimation area. For example, when the summed detection scores are highest in the order of estimated region IDs "2", "3", "1", and "4", the processing of S506 and S507 may be performed according to the order. In this way, by controlling the imaging device 102 so as to preferentially capture an estimated region with a high detection score, attribute estimation is preferentially executed for an estimated region in which many people are facing the front. attribute information can be estimated.

Further, when the processes of S506 and S507 are performed for each of the plurality of estimation regions, they may be performed in the following order.

In S503, when detecting a person included in the detection area 601, the detection unit 403 performs face detection for detecting a person's face, and further outputs a detection score for each person's face area.

In S504, if a plurality of face areas exist in the detection area, it is assumed that a plurality of human bodies exist in the detection area, and the process proceeds to S505. In S<b>505 , estimated region determining unit 404 determines a plurality of estimated regions including at least one of the plurality of persons detected by detecting unit 403 based on the result of detecting unit 403 . Furthermore, the estimation area determination unit 404 adds up the detection scores output by the detection unit 403 for each estimation area.

Then, the processing of S506 and S507 is performed on the estimated region with the highest combined detection score. At this time, it is assumed that the summed detection score is the highest in the estimation area 603b corresponding to the estimation area ID "2", and the processes of S506 and S507 are performed on this estimation area.

After that, the command management unit 405 generates a control command for controlling the imaging device 102 to capture an image so that the detection area 601 is included in the imaging range of the imaging device 102 . After the image capturing apparatus 102 is controlled based on the generated control command, the detection unit 403 performs human face detection again in the detection area 601 and outputs a detection score for each human face area.

Then, the estimated area determining unit 404 again sums up the detection scores output by the detecting unit 403 for each estimated area, targeting the estimated areas whose attribute information has not been estimated by the attribute estimating unit 406 . In this case, the estimated area determining unit 404 sums the detection scores for each estimated area for the estimated areas corresponding to the estimated area IDs "1", "3", and "4". Then, the processing of S506 and S507 is performed on the estimated region with the highest combined detection score.

As described above, while the human detection score in the detection area is repeatedly output, the detection score for each estimated area is summed each time the detection score is output, and the sum of the detection scores for the estimated area with the highest total detection score is Alternatively, the processing of S506 and S507 may be performed. In this way, by controlling the imaging device 102 to preferentially shoot the estimated area with the highest detection score, attribute estimation is preferentially executed for the estimated area in which more people are facing the front. , the attribute information can be estimated more efficiently. In addition, in the present embodiment, in S508 shown in FIG. 5, if there is no estimated area for which attribute information has not been estimated by the attribute estimation unit 406 ('Yes' in S508), the process is terminated. is not limited to After the processes in S506 and S507 are performed for each of the plurality of estimation areas determined by the estimation area determination unit 404, the estimation areas in which there are persons whose attribute information was not estimated by the attribute estimation unit 406 are subjected to S506 and S507 again. may be performed. For example, when the processes in S506 and S507 are performed in the order of the estimated area IDs "1", "2", "3", and "4", the estimated area of the estimated area ID "2" in which six people are present Assume that attribute information is estimated only for five persons in 603b. At this time, the processing in S506 and S507 is performed again for the estimated area ID "2". By doing so, even if there is a person whose attribute information cannot be estimated by the attribute estimation unit 406, the attribute information of the person can be estimated again by the attribute estimation unit 406. FIG.

After performing the processes in S506 and S507 for each of the plurality of estimated regions, the estimated region determination unit 404 re-determines at least one estimated region for the person whose attribute information was not estimated by the attribute estimation unit 406. You may decide. Then, the processes of S506 and S507 may be performed for each of at least one estimation area re-determined by the estimation area determination unit 404. FIG. The process of re-determining the estimated area will be described in more detail below with reference to FIGS. 9 and 10. FIG.

FIG. 9 is a series of flowcharts showing the process of re-determining the estimated region. The processing of the flowchart shown in FIG. 9 is mainly executed by each functional block shown in FIG. 9 is executed by the CPU 1200 of the image processing apparatus 100 according to the computer program stored in the HDD 1202. The processing of the flowchart shown in FIG. Processing executed by the CPU 1200 of the image processing apparatus 100 will be described below. The same reference numerals are assigned to the steps of processing that have the same functions as in FIG. 5, and the description of the steps of processing that are functionally the same is omitted. In S909, if attribute information has been estimated for all persons detected by the detection unit 403 ('Yes' in S909), the process ends. In S909, if attribute information has not been estimated for all of the plurality of persons detected by the detection unit 403 ('No' in S909), the process of S910 is performed.

In S910, estimated area determining unit 404 determines, based on the result of detection by detecting unit 403, at least an estimated area including at least one person whose attribute information could not be estimated among the plurality of persons detected by detecting unit 403. Decide on one. A person included in the estimation area is subject to estimation of attribute information by the attribute estimation unit 406 . Also, information about the position of the estimated region is stored in the storage unit 408 . The process of re-determining the estimation region by the estimation region determining unit 404 will be described in more detail below with reference to FIG.

FIG. 10 is a diagram for explaining the processing of S910. In FIG. 10A, persons 1000 and 1001 are persons whose attribute information was not estimated by the attribute estimation unit 406 in the area 602a, which is the estimation area corresponding to the estimation area ID "1". A person 1002 is a person whose attribute information has not been estimated by the attribute estimation unit 406 in the area 603b, which is the estimation area corresponding to the estimation area ID "2". A person 1003 is a person whose attribute information was not estimated by the attribute estimation unit 406 in the region 605d, which is the estimation region corresponding to the estimation region ID "4".

Based on the result of detection by the detection unit 403, the estimation area determination unit 404 determines at least one estimation area including at least one of the persons 1000 to 1003 whose attribute information has not been estimated by the attribute estimation unit 406. FIG.

A range 1004 in FIG. 10B is a photographing range when the photographing is performed at a zoom magnification that allows the attribute estimation unit 406 to acquire the minimum number of pixels in the face region necessary for estimating the attribute information of the persons 1000 and 1001. be. Note that the range 1004 is determined by the estimated area determination unit 404 based on the number of pixels in the human face area detected by the detection unit 403 . Then, the estimated area determination unit 404 determines an area within the range 1004 that is equal to or less than the area of the range 1004 and that includes at least one of the person 1000 and the person 1001 as an estimated area. In this case, an area 1004b is determined as an estimated area corresponding to range 1004. FIG. At this time, the estimated area determining unit 404 assigns an estimated area ID of “5” to the area 1004 b that is the estimated area corresponding to the range 1004 .

Next, the estimated area determination unit 404 determines the range 1005. FIG. A range 1005 is a photographing range when photographing is performed at a zoom magnification that allows the attribute estimation unit 406 to acquire the minimum number of pixels in the face area necessary for estimating the attribute information of the person 1002 and the person 1003 . Then, the estimated area determination unit 404 determines an area within the range 1005 that is equal to or less than the area of the range 1005 and that includes at least one of the person 1002 and the person 1003 as an estimated area. In this case, the estimated area determination unit 404 determines an area 1005 b as the estimated area corresponding to the range 1005 . In addition, the estimated area determination unit 404 assigns an estimated area ID of “6” to the area 1005 b that is the estimated area corresponding to the range 1005 .

As described above, the estimated region determining unit 404 determines at least one estimated region including at least one person whose attribute information was not estimated by the attribute estimating unit 406, based on the detection result of the detecting unit 403. do. Although the estimation area determining unit 404 determines the estimation area so as to scan from the lower left to the right of the detection area 601, the present invention is not limited to this. For example, the estimation area may be determined by operating from the upper left of the detection area 601 to the right.

Then, the processes of S506 and S507 are performed for each of the plurality of estimation areas re-determined by the estimation area determination unit 404. FIG. By doing so, even if there is a person whose attribute information cannot be estimated by the attribute estimation unit 406, the attribute information of the person can be estimated again by the attribute estimation unit 406. FIG. Note that in S910, the number of estimation areas re-determined by the estimation area 404 may be one.

Although it has been described that the image processing according to the present embodiment is performed by the image processing apparatus 100, the present invention is not limited to this. The imaging device 102 may have one or more functions of the image processing device 100 according to this embodiment. That is, in the present embodiment, the CPU 1200 of the image processing apparatus 100 executes the image processing according to the present embodiment, but the image processing may be executed by CPUs of the client apparatus and the imaging apparatus 102, which will be described later.

For example, the detection area setting unit 401, the detection unit 403, the estimation area determination unit 404, the command management unit 405, the attribute estimation unit 406, and the storage unit 408 shown in FIG. good too. At this time, the operation reception unit 402 and the output control unit 407 are included in the client device 1100 implemented by the hardware configuration shown in FIG. 12 . This case will be described in more detail with reference to FIG. Components having the same functions as those in FIGS. 2 to 4 are denoted by the same reference numerals, and descriptions of components having the same functions are omitted.

FIG. 11 is an example of a functional block diagram of the imaging device 102 and the client device 1100 having some functions of the image processing device 100 . The imaging device 102 and the client device 1100 communicate via the network 104 .
Image data generated by the image processing unit 206 is sent to the output control unit 407 in the client device 1100 via the communication units 209 and 1101 . The output control unit 407 outputs the image data sent from the imaging device 102 to the display device 101, for example. At this time, it is assumed that the display device 101 is connected to the client device 1100 .

The detection area setting unit 401 sets a detection area, which is an area to be processed by the detection unit 403, in the generated image data. Note that an area within an image set by the user via the operation reception unit 402 of the client device 1100 may be set as the detection area, or a predetermined area set in advance may be set as the detection area. If there is no user's indication of a detection area or a previously set area, the entire image indicated by the image data may be used as the detection area.

An operation reception unit 402 in the client device 1100 receives an operation performed by a user via an input device (not shown) such as a mouse or keyboard. Information on the user's operation received by the operation receiving unit 402 is sent to the detection area setting unit 401 in the imaging device 102 via the communication unit 1101 and the communication unit 209 .

The detection unit 403 detects a human body included in the detection area in the generated image data. Based on the result of detection by the detection unit 403 , the estimated region determination unit 404 determines a plurality of estimated regions including at least one of the plurality of persons detected by the detection unit 403 .

A command management unit 405 generates a control command for controlling the image capturing apparatus 102 to sequentially capture each of the plurality of estimated areas determined by the estimated area determination unit 404 . Note that the control command generated by the command management unit 405 is sent to the system control unit 207, and the imaging device 102 is controlled based on the control command.

The attribute estimating unit 406 estimates the attribute information of a person present in the estimation area for each captured image obtained from the imaging device 102 . A storage unit 408 stores information about the positions of the detection area and the estimation area.

An image processing output result (for example, the table shown in FIG. 8) is sent from the imaging device 102 to the client device 1100 via the communication units 209 and 1101 . At this time, the output result of the image processing sent from the imaging device 102 may be sent to, for example, the output control unit 407 and output to the display device 101 by the output control unit 407 .

As described above, one or more functions of the image processing apparatus 100 may be included in the imaging apparatus 102 .

Note that the present invention can also be implemented by processing in which one or more processors read and execute a program that implements one or more functions of the above-described embodiments. The program may be supplied to a system or device having a processor via a network or storage medium. The invention can also be implemented in a circuit (eg, an ASIC) that implements one or more of the functions of the embodiments described above. Also, each unit of the image processing apparatus 100 or the imaging apparatus 102 may be implemented by hardware shown in FIG. 12, or may be implemented by software.

400 communication unit 401 detection area setting unit 402 operation reception unit 403 detection unit 404 estimation area determination unit 405 command management unit 406 attribute estimation unit

Claims (12)

a detecting means for detecting a plurality of persons included in an image photographed by the photographing means;
determining means for determining two or more areas including at least one of the plurality of persons detected by the detecting means;
a control means for outputting a control command for sequentially photographing each of the areas determined by the determining means by the photographing means;
estimating means for estimating attribute information of a person existing in the area from an image captured based on the control command output by the control means;
The control means is configured such that one of the areas determined by the determination means is included in the imaging range of the imaging means, and the attribute information of a person present in the one area can be estimated. An image processing apparatus, wherein the control command is output based on a condition. 2. The image processing apparatus according to claim 1, wherein said photographing condition is such that a human face area included in the area determined by said determining means has a predetermined number of pixels or more. the control command includes at least one of panning, tilting, and zooming of the photographing means;
The photographing condition is characterized in that the zoom magnification of the photographing means is equal to or greater than the minimum necessary zoom magnification for the estimation means to estimate the attribute information of a person existing in the area determined by the determination means. 3. The image processing apparatus according to claim 1 or 2, wherein 4. The control unit according to any one of claims 1 to 3, wherein said control unit outputs said control command for controlling said photographing unit such that the center of gravity of the photographing range of said photographing unit and the center of gravity of said area are substantially the same. 2. The image processing device according to item 1. 5. The image processing apparatus according to claim 1, wherein said attribute information includes at least one of information regarding a person's age and information regarding a person's sex. 6. The image processing according to any one of claims 1 to 5, further comprising setting means for setting a detection area, which is an area within the image from which the detection means detects the plurality of people. Device. The control means outputs the control command to increase the zoom magnification of the photographing means while the detection area is included in the photographing range of the photographing means,
7. The image processing apparatus according to claim 6, wherein said detection means detects said plurality of persons included in said detection area in an image taken by said photographing means controlled by said control means. 7. The image processing apparatus according to claim 6, wherein the detection area is an area within the image set by a user. The determining means determines the area including at least one of the plurality of persons detected by the detecting means, based on the number of pixels in the face area of the person detected by the detecting means. Item 9. The image processing apparatus according to any one of Items 1 to 8. 10. The method according to any one of claims 1 to 9, wherein said determining means determines at least one area including at least one of a plurality of persons whose attribute information could not be estimated by said estimating means. The described image processing device. a detection step of detecting a plurality of people included in the image captured by the imaging means;
a determining step of determining two or more regions containing at least one of the plurality of persons detected by the detecting step;
a control step of outputting a control command for sequentially photographing each of the regions determined by the determining step;
an estimating step of estimating attribute information of a person existing in the area from an image captured based on the control command output by the controlling step;
In the control step, one of the regions determined in the determination step is included in the imaging range of the imaging means, and the attribute information of a person present in the one region can be estimated. An image processing method, wherein the control command is output based on a condition. a detecting means for detecting a plurality of persons included in an image photographed by the photographing means;
determining means for determining two or more areas including at least one of the plurality of persons detected by the detecting means;
a control means for outputting a control command for sequentially photographing each of the areas determined by the determining means by the photographing means;
estimating means for estimating attribute information of a person existing in the area from an image captured based on the control command output by the control means;
The control means is configured such that one of the areas determined by the determination means is included in the imaging range of the imaging means, and the attribute information of a person present in the one area can be estimated. A program for causing a computer to function as control means for outputting the control command based on conditions.

Images