JP6914007B2 - Information processing device and information processing method - Google Patents

Description

The present invention relates to an information processing apparatus and an information processing method.

Conventionally, a method of counting the number of people in an image by analyzing an image captured by a surveillance camera has been known. For example, Patent Document 1 discloses a technique of analyzing a moving image to detect a person within a photographing range. Patent Document 2 discloses a technique for counting the number of people who have passed a predetermined area by tracking the detected person.

Japanese Unexamined Patent Publication No. 2015-070359 Japanese Unexamined Patent Publication No. 2008-217205

However, in the techniques disclosed in Patent Documents 1 and 2, the range of detecting a person is limited to the range of photographing.

The present invention has been made in view of these problems, and an object of the present invention is to provide a processing technique capable of counting the number of people in a wider range.

One aspect of the present invention relates to an information processing device. This information processing device is an information processing device that calculates an estimated value of the total number of specific objects in a target range from an image captured by the imaging device, and is a first information processing device included in a first range in the target range. A second image in which a specific object in the first imaging range is detected from an image captured in a first imaging range smaller than the range, and the position is different in the depth direction when viewed from the imaging device with respect to the first range in the target range. Based on the detection means for detecting a specific object in the second imaging range from the image captured in the second imaging range smaller than the second range included in the range of, and the result of detection by the detecting means, the first And in the first range, based on the number of specific objects in the first imaging range calculated by the first calculation means and the first calculation means, which calculate the number of specific objects in the second imaging range, respectively. An estimated value of the total number of specific objects is calculated, and an estimated value of the total number of specific objects in the second range is calculated based on the number of specific objects in the second imaging range calculated by the first calculation means. And at least the second calculation means for calculating the estimated value of the total number of specific objects in the target range based on each of the estimated values of the total number of specific objects in the first and second ranges. , Equipped with.

According to the present invention, the number of people can be counted for a wider range.

The schematic diagram of the information processing system including the information processing apparatus which concerns on embodiment. The block diagram which shows the function and structure of the network camera of FIG. The block diagram of the information processing apparatus of FIG. FIG. 3 is a block diagram showing a function and a configuration of the analysis processing unit of FIG. 3 (a). Data structure diagram of the table held in the recording unit. A representative screen view of the setting screen displayed on the display. The flowchart which shows the series of processing in the information processing apparatus of FIG. The schematic diagram for demonstrating the counting process using the object tracking. The schematic diagram for demonstrating the application example of the number of people counting processing. The schematic diagram for demonstrating another application example of the number of people counting processing. The schematic diagram for demonstrating still another application example of the number of people counting process.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments of the present invention are not limited to the following embodiments. The same or equivalent components, members, and processes shown in the drawings shall be designated by the same reference numerals, and redundant description will be omitted as appropriate. In addition, some of the members that are not important for explanation are omitted in each drawing.

The information processing device according to the embodiment executes human detection for images suitable for human detection among images such as moving images and still images obtained from an image pickup device such as a camera performing preset patrol. , Calculate the number of people. The information processing device estimates the number of people in the target range from the size of the range corresponding to the image obtained by detecting the person, the calculated number of people, and the size of the target range for which the number of people is to be counted. As a result, the existing preset patrol function provided for the purpose of monitoring or the like can be used together for counting the number of people.

(Embodiment)
FIG. 1 is a schematic diagram of an information processing system 100 including the information processing device 103 according to the embodiment. The information processing system 100 includes an information processing device 103 and at least one network camera 102. The information processing device 103 and the network camera 102 are connected via a network 101 such as the Internet.

The network camera 102 may be a surveillance camera installed in a theater, a movie theater, a stadium, or the like, and is configured to be controllable in pan-tilt-zoom (hereinafter referred to as "PTZ"). The network camera 102 has a pan head for pan and tilt control. The network camera 102 delivers the moving image to the network 101.

The information processing device 103 controls the pan head of the network camera 102, controls the image quality of the moving image from the network camera 102, and analyzes the moving image distributed from the network camera 102. A plurality of network cameras 102 and a plurality of information processing devices 103 may exist.

Hereinafter, the counting process of the number of persons by the information processing apparatus 103 according to the embodiment will be described. FIG. 2 is a block diagram showing the functions and configurations of the network camera 102 of FIG. The network camera 102 includes an imaging unit 201, a camera control unit 202, a processing unit 203, a communication unit 204, and a storage unit 205.

The imaging unit 201 generates a moving image by imaging an imaging target region in the real space. The image pickup unit 201 includes an image pickup element such as a CCD (Charge Coupled Device) or a CMOS (Complementary MOS) image sensor, a pan head for changing the image pickup direction, and a mechanism for changing the image pickup settings such as zoom, focus, and aperture. Includes a mechanism for performing masking processing and time superimposition processing on moving images, and an image processing mechanism for changing brightness and color tone.
The camera control unit 202 controls the image pickup unit 201 in response to a control command.

The processing unit 203 receives a request command from the outside via the communication unit 204, analyzes and executes the received request command. For example, if the request command is a command related to a control request of the network camera 102, the processing unit 203 converts the command into a control command and sends it to the camera control unit 202. The camera control unit 202 executes the received control command. If the request command is a command related to inquiring about the setting status of the network camera 102, the processing unit 203 acquires the corresponding status from the storage unit 205, converts it into a response format, and returns it.

The communication unit 204 communicates with another device. The communication unit 204 is connected to the network 101 and functions as an interface with the network 101.
The storage unit 205 stores model information, names, settings, and the like of the network camera 102.

FIG. 3A is a block diagram showing the functions and configurations of the information processing apparatus 103 of FIG. The information processing device 103 includes a communication unit 301, a camera state management unit 302, a command management unit 303, a moving image management unit 304, an analysis processing unit 305, a storage unit 306, a recording unit 307, and a reception unit 308. The input unit 309, the display control unit 310, and the display 311 are provided.

The communication unit 301 communicates with other devices. The communication unit 301 is connected to the network 101 and functions as an interface with the network 101.
The camera state management unit 302 acquires the state of the camera 102 from the network camera 102 to be analyzed, or receives a notification of the state change from the camera. The camera state management unit 302 grasps and manages the state of the pan head and the imaging mechanism of the network camera 102 based on the acquired state or the received notification. The camera state management unit 302 registers the state of the network camera 102 and the time in association with each other in the camera state table 502 (described later in FIG. 5B).

The command management unit 303 generates and transmits a request command for changing the pan head and the imaging mechanism of the network camera 102 to a desired state and a request command for acquiring a moving image from the network camera 102.

The moving image management unit 304 functions as an acquisition unit that acquires the moving image captured by the network camera 102 from the network camera 102 via the network 101. The moving image management unit 304 acquires and manages moving images distributed from the network camera 102. In the present embodiment, the command management unit 303 requests the network camera 102 for a moving image in a resolution (1280x720, 640x360, 320x180, etc.) and frame rate format prior to the counting process of the number of people. The moving image management unit 304 receives the moving image transmitted from the network camera 102 in response to the request. As the setting of the specified resolution, frame rate, format, etc., the setting to be used in another process such as the monitoring process performed in parallel with the number counting process may be used. Alternatively, the information processing apparatus 103 may apply the optimum setting for the counting process of the number of people.

The analysis processing unit 305 detects a person in the image and counts the detected person with respect to the moving image from the network camera 102.
The storage unit 306 is a RAM (Random Access Memory) or the like that temporarily stores programs and data, and temporarily stores the result of the analysis process executed by the analysis processing unit 305. The storage unit 306 also acquires the state of the network camera 102 at the time when the image subjected to the analysis process is captured from the camera state management unit 302 and temporarily stores the state.

The recording unit 307 is a hard disk, an optical disk, a memory card, or the like, and the settings in the network camera 102 are recorded.
The input unit 309 is a device operated by the user for input, such as a pointing device such as a mouse or a keyboard.
The reception unit 308 receives user input such as range designation from the user via the input unit 309.
The display 311 is a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), or the like. The display control unit 310 controls the display 311.
The display control unit 310 generates data corresponding to the screen to be displayed on the display 311 and provides the data to the display 311.

FIG. 3B is a hardware configuration diagram of the information processing device 103 of FIG. The CPU 320 mainly controls the operation of each component. The main memory 321 stores a control program executed by the CPU 320, and provides a work area when the program is executed by the CPU 320. The magnetic disk 322 is an example of an external storage device that stores an operating system (OS), device drivers of peripheral devices, and programs for realizing various application software including programs for performing processing described later. be. When the CPU 320 executes the program stored in the main memory 321 and the magnetic disk 322, the function (software) of the information processing apparatus 103 shown in FIG. 3A and the processing in the flowchart described later are realized. The program stored in the magnetic disk 322 is expanded in the main memory 321 as needed and executed by the CPU 320. The display memory 323 temporarily stores display data. The mouse 324 and keyboard 325 are as described above. Each of the above components is communicably connected to each other by a common bus 326.

In the example shown in FIG. 3B, only one CPU 320 is shown as hardware, but a plurality of CPUs may be provided. Alternatively, it may have a GPU. In any case, a plurality of units of arithmetic processing units (processors) such as a CPU and a GPU may be provided as hardware resources. Further, a plurality of magnetic disks 322 for storing the program may be provided. Alternatively, in another embodiment, at least some or all of the functions shown in FIG. 3A may be implemented as one or more dedicated hardware circuits.

FIG. 4 is a block diagram showing the functions and configurations of the analysis processing unit 305 of FIG. 3A. The analysis processing unit 305 includes a preset patrol setting unit 401, a target range setting unit 402, a state acquisition unit 403, a moving image acquisition unit 404, a determination unit 405, a detection unit 406, and an image number calculation unit 407. The total number calculation unit 408 and the like are included. 5A, 5B, and 5C are data structure diagrams showing an example of a preset table 501, a camera state table 502, and a count result table 503, respectively. These tables are held in the recording unit 307.

The preset patrol setting unit 401 sets the control of the network camera 102, for example, the preset patrol setting. A set of parameters (pan, tilt) that specifies the state of the pan head and parameters (zoom, focus, aperture) that specify the imaging settings is called a preset. In the preset patrol, a plurality of different presets are set. Furthermore, in the preset patrol, the order of each preset, the operating speed of the pan head when changing the preset, and the waiting time after the setting is transferred to each preset are set. These are collectively referred to as preset patrol settings. The network camera 102 is set so that a plurality of predetermined ranges are sequentially imaged by the preset patrol setting. Each of the plurality of predetermined ranges corresponds to each of the different presets included in the preset patrol setting. Hereinafter, each of the plurality of predetermined ranges will be referred to as a preset imaging range.

The preset patrol setting unit 401 receives the designation of the preset patrol setting from the user via the reception unit 308 and the input unit 309. The preset patrol setting unit 401 stores the designated preset patrol setting in the preset table 501 (FIG. 5A). The preset table 501 holds a preset ID that specifies a preset, a preset (pan, tilt, zoom, focus, aperture), an operation speed, and a standby time in association with each other. The preset ID specifies a preset and represents the order of the presets in the preset patrol. In the example of FIG. 5A, the preset patrol starts from the preset included in the entry of the preset ID “1”, and transitions in the order of the preset IDs “1”, “2”, “3”, .... Then, the preset patrol returns to the preset ID "1" when the standby time "rs" elapses in the preset of the preset ID "13". Explaining the operation of the network camera 102, the network camera 102 images the preset imaging range corresponding to the preset ID “1”, and then images the preset imaging range corresponding to the preset ID “2”. When the network camera 102 images the last preset imaging range corresponding to the preset ID "13" by imaging the preset IDs "3", "4", ... In order, the first preset imaging corresponding to the preset ID "1" Return to range imaging.

The target range setting unit 402 sets a target range for counting the number of people and stores it in the recording unit 307. The target range setting unit 402 causes the display 311 to display a setting screen for setting the target range via the display control unit 310. The target range setting unit 402 receives the target range specified by the user on the setting screen using the input unit 309 via the reception unit 308. The target range setting unit 402 stores the designated target range in the recording unit 307.

FIG. 6 is a representative screen view of the setting screen 601 displayed on the display 311. The setting screen 601 has an imageable range 602, a target range 603, and an image pickup range 604. The imageable range 602 is a range calculated from the movable range of the pan head of the target network camera 102 and the imaging angle of view. The imaging range 604 is an imaging range calculated from the current imaging state of the target network camera 102. The user draws the target range 603 on the setting screen 601 using a pointing device such as a mouse. The target range 603 shown in FIG. 6 is an example of the target range set by the user. The target range 603 is usually set larger than the imaging range 604. In this way, the setting screen 601 allows the user to specify the target range 603 with respect to the imageable range 602 of the network camera 102.

A panoramic image obtained by synthesizing the images captured by the network camera 102 may be displayed in the background of the imageable range 602. Further, the target range 603 may be set larger than the imageable range 602.

Returning to FIG. 4, when the state acquisition unit 403 is requested to change to a new preset in the preset patrol, the state of the network camera 102 set according to the presets up to that point is changed to the camera state table 502 (FIG. 5 (FIG. 5). b) Obtained from). The camera state table 502 holds the time and the state of the network camera 102 (pan, tilt, zoom, focus, aperture, resolution, frame rate, format) at that time in association with each other. The state acquisition unit 403 specifies the state to be obtained by using the time as a key in the camera state table 502, and acquires the specified state.

The moving image acquisition unit 404 acquires a moving image corresponding to the state acquired by the state acquisition unit 403 from the moving image management unit 304. That is, when a change to a new preset is requested, the moving image acquisition unit 404 acquires a moving image that captures the preset imaging range corresponding to the presets up to that point.

The determination unit 405 determines whether or not the moving image acquired by the moving image acquisition unit 404 satisfies the criteria for detecting a person. If it is determined that the number of people is satisfied, the number of people is counted, otherwise the number of people is not counted or skipped. For example, the determination unit 405 calculates an index of whether or not the subject is in focus, for example, the degree of focusing of the moving image, from the acquired moving image or the state of the network camera 102. The determination unit 405 determines that the criterion is satisfied when the calculated degree exceeds the degree required for detecting a person, and determines that the criterion is not satisfied otherwise. In another example, the determination unit 405 calculates the distance from the acquired state of the network camera 102 to the subject. The determination unit 405 calculates the number of pixels occupied by the preset image pickup range on the image sensor of the image pickup unit 201 from the calculated distance and the zoom state. The determination unit 405 determines that the criterion is satisfied when the calculated number of pixels exceeds the number of pixels required for detecting a person, and determines that the criterion is not satisfied otherwise. This corresponds to not performing human detection when the image quality required for human detection cannot be obtained due to the large digital zoom.

When the determination unit 405 determines that the detection unit 406 satisfies the condition, the detection unit 406 detects a person in the preset imaging range from the moving image acquired by the moving image acquisition unit 404. The human detection process in the detection unit 406 may be realized by using a known image recognition technique.

The imaging number calculation unit 407 calculates the number of people in the preset imaging range based on the detection result by the detection unit 406. The number-of-images calculation unit 407 registers the calculated number of people in the count result table 503 (FIG. 5 (c)) in association with the state acquired by the state acquisition unit 403. As a result, the number of people counted corresponding to each of the presets is registered in the count result table 503. The count result table 503 holds the state of the network camera 102 (pan, tilt, zoom, focus, aperture, resolution, frame rate, format) corresponding to the preset in association with the number of people counted. When the state of the network camera 102 to be registered is already registered in the count result table 503, the imaging number calculation unit 407 may update or overwrite the number of people corresponding to the state. It should be noted that such a situation occurs, for example, after the second lap of the preset patrol, and the number of people registered in the first lap is overwritten by the number of people counted in the same preset in the second lap.

The total number of people calculation unit 408 calculates the number of people in the target range based on the number of people calculated by the imaging number calculation unit 407, the size of the preset imaging range, and the size of the target range. Specifically, the total number of people calculation unit 408 refers to the count result table 503 and specifies a set of the available state and the count number of people. For example, when a change to the preset ID "5" is requested in the first lap of the preset patrol, a set of states and the number of people corresponding to each of the preset IDs "1", "2", "3", and "4". Will be available. From the second lap onward, depending on the determination result by the determination unit 405, a set corresponding to all preset IDs may be available. The total number calculation unit 408 calculates the size of the preset imaging range corresponding to each of the specified sets. The total number calculation unit 408 sums the calculated sizes of the preset imaging range. The total number calculation unit 408 totals the number of people in the specified group. The total number calculation unit 408 acquires the size of the target range from the recording unit 307. If there is no overlap between the preset imaging ranges, the total number of people calculation unit 408 divides the size of the acquired target range by the total size of the preset imaging ranges and multiplies the total number of people by the value obtained. , Calculate the number of people in the target range.

When there is an overlap, the total number of people calculation unit 408 obtains an effective size by further adding the total size of the overlapping portions to the total size of the preset imaging range. The total size of the overlapping parts represents the degree of overlapping. The total number of people calculation unit 408 calculates the number of people in the target range by multiplying the total number of people by the value obtained by dividing the acquired size of the target range by the effective size. If there is an overlap, the number of people detected in the overlap is counted multiple times, so the total number of people is greater than the total number of actual people. However, since the size of the overlapping portion is added to the size of the preset imaging range, the ratio value is not so different from the actual value. As a result, it is possible to obtain an estimated value of the number of people in the target range without actually counting the number of people in the overlapping portion. This is based on the standpoint that since the number of people calculated by the total number of people calculation unit 408 is an estimated value in the first place, it is not necessary to strictly perform only the overlapping processing to increase the processing load. Of course, the number of people in the target range may be estimated more accurately by counting the number of people in the overlapping portion.

The total number calculation unit 408 may calculate the number of people in the target range by using the number of people overwritten by the imaging number calculation unit 407. In this case, the number of people in the target range is calculated based on the number of people in the preset imaging range after the second lap instead of the number of people in the preset imaging range obtained in the first lap of the preset patrol.

The operation of the information processing apparatus 103 with the above configuration will be described.
FIG. 7A is a flowchart showing a series of processes in the information processing apparatus 103 of FIG. In S701, the analysis processing unit 305 acquires the setting of the target range from the recording unit 307. In S702, the command management unit 303 acquires the preset patrol setting from the preset table 501 held in the recording unit 307. In S703, the command management unit 303 determines whether or not the reception unit 308 has received the preset patrol end instruction from the user. If accepted (YES in S703), this process ends. If not accepted (NO in S703), the command management unit 303 determines in S704 whether or not the waiting period of the current preset has expired. The waiting period is a period having a length specified by the waiting time, starting from the time when the preset is set. If it has not expired (NO in S704), the command management unit 303 repeats S704. When it expires (YES in S704), the command management unit 303 refers to the preset patrol setting acquired in S702 in S705, and acquires it by specifying the next preset of the current preset. In S706, the command management unit 303 generates a request command for changing the pan head of the network camera 102 and the imaging mechanism to the state specified by the acquired preset, and transmits the request command to the network camera 102. In S707, the analysis processing unit 305 executes the counting process of the number of people, triggered by the transmission of the request command in S706. After that, the process returns to S703. If the current preset is the last preset of the preset patrol setting in S705, the first preset is acquired as the next preset.

FIG. 7B is a flowchart showing a series of processes performed in the counting process of the number of people in FIG. 7A. In S708, the state acquisition unit 403 acquires the state of the network camera 102. In S709, the moving image acquisition unit 404 acquires a moving image that captures the preset imaging range to be counted. In S710, the determination unit 405 determines whether or not the criteria necessary for detecting a person are satisfied. If not satisfied (NO in S710), the process proceeds to S714. If it is satisfied (YES in S710), the detection unit 406 detects a person from the acquired moving image in S711. In S712, the imaging number calculation unit 407 calculates the number of detected persons. In S713, the imaging number calculation unit 407 stores the calculated number of people in the count result table 503 in association with the state acquired in S708. In S714, the total number of people calculation unit 408 acquires the number of people calculated in the past from the count result table 503. In S715, the total number calculation unit 408 calculates an estimated value of the number of people in the target range based on the acquired past number of people.

According to the information processing apparatus 103 according to the present embodiment, it is possible to obtain an estimated value of the number of people in the entire target range from the result of the number of people count for the image obtained by capturing a part of the target range of the number of people count.

The configuration and operation of the information processing device 103 according to the embodiment have been described above. This embodiment is an example, and it will be understood by those skilled in the art that various modifications are possible for each component and combination of each process, and that such modifications are also within the scope of the present invention.

(Modification example)
In the present embodiment, the case where the counting process of the number of people is performed according to the preset patrol processing has been described, but the present invention is not limited to this, and the counting process of the number of people may be executed by itself. Alternatively, the number of people counting process may be executed together with the process other than the preset patrol process. More generally, when there is a target range for counting the number of a specific object such as a person and an image obtained by capturing an imaging range smaller than the target range, the technique according to the present embodiment. It is possible to infer the number in the target range by applying the idea.

In the present embodiment, the case where the user sets the target range and registers the setting in the recording unit 307 has been described, but the present invention is not limited to this, and for example, the target range may be a predetermined fixed one. ..

In the present embodiment, the case where the counting process of the number of people is performed every time the request for changing the preset is transmitted in the preset patrol has been described, but the present invention is not limited to this. For example, the number of people may be counted each time a new image is received from the network camera 102. Alternatively, the number of people may be counted for every other preset that changes in order, or the number of people may be counted for each cycle of preset patrols. In the present embodiment, the real-time property can be improved by updating the estimated value of the number of people in the target range more frequently than in the above-described modification.

Further, a moving image captured by the network camera 102 while the state of the network camera 102 changes from a state for capturing a preset imaging range to a state for capturing the next preset imaging range (hereinafter, "moving moving image"). The image) may be used for counting the number of people. The detection unit according to this modification detects a person by performing an object tracking process on a moving image while moving.

FIG. 8 is a schematic diagram for explaining the counting process using the object tracking. In FIG. 8, it is assumed that the state of the network camera 102 changes from the state for capturing the first preset imaging range 801 to the state for capturing the next second preset imaging range 802. The imaging range of the moving moving image captured during the change (hereinafter, referred to as “moving imaging range”) is the range indicated by diagonal lines in FIG. The moving moving image consists of many frames including the first frame 803, the second frame 804, and the third frame 805. In this variant, the detector uses object tracking from each frame to detect people while eliminating duplication. For example, the detection unit detects two people from the first frame 803, the first person 806 and the second person 807. Next, the detection unit detects the second person 807 and the third person 808 from the second frame 804, and also recognizes that the second person 807 was also detected in the first frame 803 by the object tracking process. .. The detection unit detects only the third person 808 from the result of the object tracking process. Similarly, the detection unit detects two people from the third frame 805, excluding the third person 808 detected in the second frame 804. The object tracking process itself may be realized by using a known image recognition technique. The detection unit detects people within the moving imaging range while removing duplication as described above, and the imaging number calculation unit calculates the number of detected people. The imaging number calculation unit registers the imaging conditions (pre- and post-presets, etc.) of the moving moving image and the calculated number of people in the count result table in association with each other. In the subsequent processing, by treating the moving imaging range in the same manner as the preset imaging range, the number of people in the target range can be estimated as described in the embodiment.

In the determination unit according to this modification, the speed at which the state of the network camera 102 changes from the state for imaging the first preset imaging range 801 to the state for imaging the next second preset imaging range 802 is a human. Determine if the detection criteria are met. The determination unit acquires the angle of view of imaging, the shutter speed, and the operating speed of the pan head from the state of the network camera 102. The determination unit calculates the moving speed with respect to the angle of view (not the operating speed of the pan head) from those acquired parameters. The determination unit determines that the criterion is satisfied when the calculated movement speed is lower than the movement speed at which an image with little or no blur required for detecting a person can be obtained, and determines that the criterion is not satisfied otherwise. .. If it is determined that the criteria are met, the detector detects a person in the moving moving image.

Alternatively, the determination unit calculates the overlap of the imaging ranges of the first frame 803 and the second frame 804 when detecting a person from the second frame 804, and determines whether the overlap is equal to or greater than the threshold value. .. If the overlap of the imaging ranges is smaller than the threshold value, a person is detected from the second frame 804 without tracking the object. This corresponds to the fact that when the overlap is small, the influence of the overlap on the estimated value is small. When the overlap of the imaging ranges is equal to or greater than the threshold value, a person is detected while tracking the object as described above.

In this modification, the detection unit may change the detection sensitivity according to the moving speed with respect to the angle of view. For example, when the moving speed is fast, that is, when the moving speed exceeds the threshold value, even one frame is detected, and when the moving speed is slow, that is, when the moving speed is slow, that is, when the moving speed is lower than the threshold value, a person hits several frames in a row. The person may be detected.

(Application example)
Hereinafter, an application example of counting the number of people by the information processing device according to the above-described embodiment and modification will be described. FIG. 9 is a schematic diagram for explaining an application example of the number of people counting process. In this application example, the network camera 102 is a surveillance camera mounted on the ceiling on the stage side of the theater, and images the audience seats 901 in order according to the preset patrol setting. In the application example of FIG. 9, there are a total of 18 preset imaging ranges PSR1 to 18, each of which is shown as a rectangle in FIG. 9 and is circulated in this order. The target range 902 is indicated by a broken line rectangle. When the information processing apparatus is requested to move from the preset imaging range PSR2 to the preset imaging range PSR3 in the first lap of the preset patrol, the information processing apparatus calculates the number of people in the preset imaging range PSR2 as "4". The information processing device 103 refers to the count result table and acquires the number of people "5" calculated for the preset imaging range PSR1. When the area of the preset imaging range PSR1 is A1, the area of the preset imaging range PSR2 is A2, and the area of the target range 902 is A3, the information processing apparatus calculates the target range 902 by calculating (4 + 5) and A3 / (A1 + A2). Get an estimate of the number of people in.

In the above calculation example, the number of people in the target range 902 is estimated assuming that the distribution density of people in the target range 902 is uniform. However, the calculation method is not limited to this, and for example, perspective in the target range 902 may be considered. In the example shown in FIG. 9, the preset imaging range PSR1 is a range closer to the network camera 102, and a person appears relatively large in the moving image obtained by imaging the range. On the other hand, the preset imaging range PSR 11 is a range farther from the network camera 102, and a person appears relatively small in a moving image in which the range is captured. When estimating the number of people in the target range 902, the information processing apparatus sets the target range 902 to the proximal stage consisting of PSR1 to PSR4, the middle stage consisting of PSR5 to PSR10, and the distal stage consisting of PSR11 to PSR18. It may be divided into three stages. The information processing device may estimate the number of people for each of the three stages, and the sum of the obtained three estimated values may be used as the estimated value of the number of people in the target range 902. The method of dividing the target range 902 is not limited to the above-mentioned three divisions, and may be appropriately divided according to the assumed distribution density of the count target.

FIG. 10 is a schematic diagram for explaining another application example of the number of people counting process. In the application example of FIG. 10, there are a total of 6 preset imaging ranges PSR1, PSR4, PSR5, PSR10, PSR11, and PSR18, which are patrolled in this order. The information processing device calculates the number of people in each preset imaging range and stores it in the count result table, and also calculates the number of people in the moving imaging range between the preset imaging ranges. However, the moving imaging range (indicated by diagonal lines in FIG. 10) when moving from the preset imaging range PSR5 to the preset imaging range PSR10 is not calculated because the moving speed with respect to the angle of view is too fast. The information processing device moves the total area of the six preset imaging ranges, the area of the moving imaging range when moving from the preset imaging range PSR1 to the preset imaging range PSR4, and the area of the moving imaging range from the preset imaging range PSR11 to the preset imaging range PSR18. Add the area of the imaging range while moving. The information processing apparatus obtains an estimated value of the number of people in the target range 902 by multiplying the total number of detected people by the value obtained by dividing the area of the target range 902 by the added area.

FIG. 11 is a schematic diagram for explaining still another application example of the number of people counting process. In the application example of FIG. 11, there are two moving imaging ranges 903 and 904, which overlap. The information processing apparatus calculates the number of people in each of the first moving imaging range 903 and the second moving imaging range 904 using the method according to the modified example described with reference to FIG. 8, and stores the number in the count result table. As described in the embodiment, the information processing apparatus calculates the number of people in the target range in consideration of the area of the overlapping portion 905 (indicated by the diagonal line in FIG. 11) of the two moving imaging ranges 903 and 904. do. As described above, the combination of the embodiment and the modified example is also possible.

(Other Examples)
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiment is supplied to the system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads the program. This is the process to be executed.

100 information processing system, 101 network, 102 network camera, 103 information processing device.

Claims (13)

An information processing device that calculates an estimated value of the total number of specific objects in the target range from images captured by the image pickup device.
The specific object in the first imaging range is detected from an image captured in a first imaging range smaller than the first imaging range included in the first range in the target range, and the first image in the target range is detected. In the second imaging range from an image obtained by imaging a second imaging range smaller than the second range included in the second range whose position is different in the depth direction when viewed from the imaging device with respect to the range 1. A detection means for detecting the specific object and
A first calculation means for calculating the number of the specific object in the first and second imaging ranges, respectively, based on the result of detection by the detection means.
Based on the number of the specific objects in the first imaging range calculated by the first calculation means, an estimated value of the total number of the specific objects in the first range is calculated, and the first calculation is performed. Based on the number of the specific objects in the second imaging range calculated by the means, an estimated value of the total number of the specific objects in the second range is calculated, and at least the first and second objects are estimated. A second calculation means for calculating an estimated value of the total number of the specific objects in the target range based on each of the estimated values of the total number of the specific objects in the range of.
An information processing device characterized by being equipped with. The second calculation means calculates an estimated value of the total number of the specific objects in the first range based on the size of the first imaging range and the size of the first range. The first aspect of the present invention is to calculate an estimated value of the total number of the specific objects in the second range based on the size of the second imaging range and the size of the second range. Information processing equipment. The second calculation means is
A numerical value obtained by dividing the size of the first range by the size of the first imaging range is multiplied by the number of the specific objects in the first imaging range calculated by the first calculation means. Therefore, the estimated value of the total number of the specific objects in the first range is calculated.
A numerical value obtained by dividing the size of the second range by the size of the second imaging range is multiplied by the number of the specific objects in the second imaging range calculated by the first calculation means. The information processing apparatus according to claim 2, wherein the estimated value of the total number of the specific objects in the second range is calculated. The imaging device is set so that at least a plurality of predetermined imaging ranges included in the first and second imaging ranges are sequentially imaged.
The second calculation means calculates an estimated value of the total number of the specific objects in the first and second ranges based on the number of the specific objects in each of the plurality of predetermined imaging ranges. The information processing apparatus according to any one of claims 1 to 3, wherein the information processing apparatus is characterized by the above. The imaging device is set to image the plurality of predetermined imaging ranges in order, capture the final imaging range, and then return to imaging of the first imaging range.
The information processing device according to claim 4, wherein the second calculation means updates an estimated value of the total number of the specific objects when the image pickup device images the last imaging range. The information processing device according to any one of claims 1 to 5, further comprising a receiving means for receiving a designation of the target range from the user with respect to the imageable range of the image pickup device. Further provided with a determination means for determining whether or not the image captured by the imaging device meets the criteria for detecting the specific object.
The information processing apparatus according to any one of claims 1 to 6, wherein the detection means detects the specific object from the image when it is determined by the determination means. The information processing apparatus according to claim 7, wherein the determination means determines that the criteria are satisfied when the degree of focusing exceeds the degree required for detecting the specific object. The information processing apparatus according to claim 7, wherein the determination means determines that the criteria are satisfied when the number of pixels exceeds the number of pixels required for detecting the specific object. The second calculation means further calculates an estimated value of the total number of the specific objects in the target range based on the degree of overlap between the first range and the second range. The information processing apparatus according to any one of claims 1 to 9. The information processing apparatus according to any one of claims 1 to 10, wherein the second range is a range corresponding to a position farther from the imaging position than the first range in the target range. .. It is an information processing method that calculates an estimated value of the total number of specific objects in the target range from the image captured by the image pickup device.
The specific object in the first imaging range is detected from an image captured in a first imaging range smaller than the first imaging range included in the first range in the target range, and the first image in the target range is detected. In the second imaging range from an image obtained by imaging a second imaging range smaller than the second range included in the second range whose position is different in the depth direction when viewed from the imaging device with respect to the range 1. To detect the specific object and
Based on the result of the detection, the number of the specific object in the first and second imaging ranges is calculated, respectively.
Based on the calculated number of the specific objects in the first imaging range, an estimated value of the total number of the specific objects in the first range is calculated, and the calculated second imaging is performed. Based on the number of the specific objects in the range, an estimate of the total number of the specific objects in the second range is calculated, and at least the total number of the specific objects in the first and second ranges. To calculate the estimated value of the total number of the specific objects in the target range based on each of the estimated values of
An information processing method characterized by including. A program for operating a computer as each means of the device according to any one of claims 1 to 11.

Images