JP7387080B1 - People flow measurement system, people flow measurement method, and people flow measurement program - Google Patents

Abstract

The people flow measurement system (90) includes a person detection section (120), a people measurement section (130), and a measured number of people correction section (140). The person detection unit (120) detects a target object, which is an object appearing in an input image that is a captured image of the target space, and is an object existing within the target space. The people counting section (130) measures the number of detected target objects. The measured number of people correction unit (140) uses a correction coefficient that is generated based on the three-dimensional structure of the target space and is used to correct the number of measured target objects. Correct the number of target objects.

Description

The present disclosure relates to a people flow measurement system, a people flow measurement method, and a people flow measurement program.

In conventional people flow measurement methods, in order to improve the accuracy of people measurement based on images, a correction coefficient is calculated based on the image analysis results and various conditions at the time of shooting, and the calculated correction coefficient is used. The number of people obtained using existing people counting methods was corrected (for example, Patent Document 1). The image analysis result is a result showing, as a specific example, a human body part detected from the image and a motion vector obtained from the image. Specific examples of various conditions at the time of photographing include the angle of view and the direction of photographing at the time of photographing the image.
In Patent Document 1, in order to realize this method, a detection means for detecting a human body from an image photographed by a photographing device, a counting means for counting the number of human bodies detected by the detection means, and an analysis result of the image and a The present invention discloses a technique including a correction means for correcting the number of human bodies counted by the counting means using a correction coefficient based on at least one of the photographing conditions at the time of photographing the image. Here, the correction coefficient is determined based on the part of the human body detected from the image, the motion vector obtained from the image, and the angle of view and shooting direction at the time of shooting.

Japanese Patent Application Publication No. 2018-151960

In the technique disclosed in Patent Document 1, the correction coefficient is calculated based on the appearance in the captured two-dimensional image without considering the three-dimensional structure of the object space that is the imaging target. Here, even if the appearance in the two-dimensional images is the same, the three-dimensional structure of the target space is not necessarily the same. Therefore, if the number of human bodies present in the target space is corrected using a correction coefficient calculated without considering the three-dimensional structure of the target space, the accuracy of the number of people correction may decrease. Therefore, Patent Document 1 has a problem in that the accuracy of the number of people correction decreases because the correction coefficient calculated based on the three-dimensional structure of the target space is not used.
The present disclosure provides a method for correcting the measured number of people in a target space using a correction coefficient calculated based on the three-dimensional structure of the target space in a people flow measurement method in order to correct the measured number of people in the target space with relatively high accuracy. purpose.

The people flow measurement system according to the present disclosure is
a person detection unit that detects a target object that is an object appearing in an input image that is a captured image of a target space and that is an object that exists within the target space;
a people counting unit that measures the number of detected target objects;
Correcting the number of measured target objects using a correction coefficient that is a coefficient generated based on the three-dimensional structure of the target space and used to correct the number of measured target objects. A first people flow measuring device is provided, which includes a measured number of people correction section.

According to the present disclosure, the measured number of people correction unit corrects the number of target objects using a correction coefficient generated based on the three-dimensional structure of the target space. Here, the number of target objects may be the number of people measured in the target space. Therefore, according to the present disclosure, in order to correct the number of people measured in a target space with relatively high accuracy in a people flow measurement method, the number of people measured using a correction coefficient calculated in consideration of the three-dimensional structure of the target space. can be corrected.

1 is a diagram illustrating a configuration example of a people flow measurement system 90 according to Embodiment 1. FIG. 1 is a diagram showing an example of a hardware configuration of a people flow measurement system 90 according to Embodiment 1. FIG. 5 is a flowchart showing the operation of the people flow measurement system 90 according to the first embodiment. FIG. 3 is a diagram illustrating correction coefficients according to the first embodiment. FIG. 3 is a diagram illustrating correction coefficients according to the first embodiment. FIG. 3 is a diagram illustrating a hardware configuration example of a people flow measurement system 90 according to a modification of the first embodiment.

In the description of the embodiments and the drawings, the same elements and corresponding elements are denoted by the same reference numerals. Descriptions of elements labeled with the same reference numerals will be omitted or simplified as appropriate. Arrows in the figure mainly indicate the flow of data or processing. Furthermore, "unit" may be read as "circuit," "process," "procedure," "process," or "circuitry" as appropriate.

Embodiment 1.
Hereinafter, this embodiment will be described in detail with reference to the drawings.

***Explanation of configuration***
FIG. 1 shows an example of a block diagram of a people flow measurement system 90 according to the first embodiment. The people flow measurement system 90 includes a camera 100 and a server 200, as shown in FIG. The camera 100 and the server 200 are communicably connected. The people flow measurement system 90 may include a plurality of cameras 100. Server 200 is also called a client server. A client server is also a computer.
The camera 100 includes an imaging section 110, a person detection section 120, a people measurement section 130, and a measured number of people correction section 140. The server 200 includes an image receiving section 210, a three-dimensional structure analyzing section 220, and a correction coefficient generating section 230. The camera 100 is also called a first people flow measuring device. The first people flow measurement device may include a computer. The server 200 is also called a second people flow measuring device. The first people flow measuring device and the second people flow measuring device may be appropriately configured integrally, or each part of each device may be provided with another device.

The imaging unit 110 photographs an image showing the target space. The target space is a three-dimensional space that is an object to be photographed by the camera 100.

The person detection unit 120 detects a target object. The target object is an object that appears in the input image and exists within the target space, and is, for example, a human body. The input image is an image of the target space. As a specific example, the person detection unit 120 detects a person included in the image indicated by the image data 2 output from the imaging unit 110. The image indicated by image data 2 is also an input image.
Image data 2 is data indicating an image captured by the imaging unit 110. The image data 2 may be data indicating a still image or may be data indicating a moving image.

The people counting section 130 measures the number of target objects detected by the person detecting section 120. As a specific example, the number of people measurement unit 130 measures the number of people in the target space based on the person detected by the person detection unit 120.

The measured number of people correction section 140 corrects the number of target objects measured by the number of people measurement section 130 using the correction coefficient. The correction coefficient is a coefficient generated based on the three-dimensional structure of the target space, and is used to correct the number of measured target objects. As a specific example, the measured number of people correction unit 140 uses the correction coefficient output from the correction coefficient generation unit 230 to correct the number of people measured by the number of people measurement unit 130.

The image receiving unit 210 receives image data 2 from the camera 100.

The three-dimensional structure analysis unit 220 analyzes the three-dimensional structure of the target space, which is expressed in the image output by the image reception unit 210.

The correction coefficient generation unit 230 generates correction coefficients based on the three-dimensional structure analyzed by the three-dimensional structure analysis unit 220. The correction coefficient is a coefficient used to correct the number of people measured by the number of people measuring section 130. The correction coefficient may be a single value, or may be prepared for each detected person, and may be a value set for each area in the target space, taking into account the number of people and population density in each area. It may be a value set as follows. Further, the correction coefficient may be a value set in consideration of environmental information indicating weather, events, etc. around the target space. The correction coefficient may be a value set in consideration of the presence or absence of an umbrella.
When an obstructing object is shown in the input image, the correction coefficient generation unit 230 generates a correction coefficient based on the number of target objects shown in the reference area, using the position of the camera 100 that captured the input image as a reference. A coefficient may be generated for use in estimating the number of target objects existing behind the shielding object when the target object is hidden. The reference area is an area within the input image that corresponds to the size of the obstruction in the input image. The shielding object is a three-dimensional structure that shields a part of the target space based on the position of the camera 100.

The corrected measured number of people 9 is data indicating the number of people corrected by the measured number of people correction unit 140, and is data indicating the number of people existing in the target space at the time when the image indicated by the image data 2 was captured.

The people flow measurement system 90 includes a processor 11, a memory 12, an image input interface 13, an image processing It includes hardware such as a processor 14 and a network interface 15. These pieces of hardware are connected as appropriate via signal lines. The people flow measurement system 90 may be composed of a general computer or a plurality of computers.

The processor 11 is an IC (Integrated Circuit) that performs arithmetic processing, and controls hardware included in the computer. The processor 11 is, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit).
The people flow measurement system 90 may include a plurality of processors replacing the processor 11. A plurality of processors share the role of the processor 11.

Memory 12 typically consists of volatile storage and non-volatile storage. A specific example of the volatile storage device is RAM (Random Access Memory). Specific examples of the nonvolatile storage device include a ROM (Read Only Memory), an HDD (Hard Disk Drive), or a flash memory.

The image input interface 13 is an interface that receives the image data 2, and a specific example is a USB (Universal Serial Bus) terminal.
The image processing processor 14 is a processor that has a function of executing image processing. The processor 11 and the image processor 14 may realize the functions of each part included in the people flow measurement system 90.
The network interface 15 is a receiver and a transmitter, and a specific example is a communication chip or NIC (Network Interface Card).

In this configuration, first, the image data 2 is input to the image receiving section 210 via the image input interface 13.
Next, the input image data 2 is input to the image processing processor 14. Image processing processor 14 detects a person in the image and stores person data indicating the detected person in memory 12 .
Next, the image processing processor 14 takes in the person data stored in the memory 12, increments a counter stored in the memory 12 that counts the number of people, and transfers the incremented counter back to the memory 12. Export as the result of counting the number of people.
The people flow measurement system 90 performs the above-described process on all human data detected from one image.

Further, the image processing processor 14 analyzes the three-dimensional structure indicated by the image data 2, and stores three-dimensional structure information indicating the analysis result in the memory 12.
Next, the processor 11 determines the three-dimensional position of the person detected in the above-mentioned image based on the three-dimensional structure information stored in the memory 12, and corrects the number of people to be measured based on the three-dimensional structure information. A process of creating a coefficient and storing the created correction coefficient in the memory 12 is executed. This process is executed for all detected persons.
Next, the image processing processor 14 takes in the number of people information stored in the memory 12 and the correction coefficient stored in the memory 12, executes a correction process using the taken in number of people information and the correction coefficient, The results of the correction processing are output to the memory 12. The measured number of people 9 on the memory 12 indicating the result of the correction process is sent to the network via the network interface 15.

The memory 12 stores a people flow measurement program. The people flow measurement program is a program that causes a computer to realize the functions of each part included in the people flow measurement system 90, and is a general term for programs executed in each device included in the people flow measurement system 90. The people flow measurement program is executed by the processor 11 and the image processing processor 14. The functions of each part included in the people flow measurement system 90 may be realized by software.

Data used when executing the people flow measurement program, data obtained by executing the people flow measurement program, etc. are appropriately stored in the storage device. Each part of the people flow measurement system 90 uses a storage device as appropriate. As a specific example, the storage device includes at least one of the memory 12, a register within the processor 11, and a cache memory within the processor 11. Note that the terms "data" and "information" may have the same meaning. The storage device may be independent of the computer.
The functions of memory 12 may be realized by other storage devices.

The people flow measurement program may be recorded on a computer-readable nonvolatile recording medium. Specific examples of the nonvolatile recording medium include an optical disk or a flash memory. The people flow measurement program may be provided as a program product.

***Operation explanation***
The operation procedure of the people flow measurement system 90 corresponds to a people flow measurement method. Further, a program that realizes the operation of the people flow measurement system 90 corresponds to a people flow measurement program.

FIG. 3 is a flowchart showing an example of the processing of the people flow measurement system 90 when the people flow measurement system 90 has the configuration shown in FIG. The processing of the people flow measurement system 90 will be explained using FIG. 3. Note that it is assumed that the target camera, which is the camera 100, executes the flowchart of camera processing.

(Step S101)
The imaging unit 110 captures an image showing the target space, and outputs image data 2 representing the captured image to the server 200.

(Step S102)
The measured number of people correction unit 140 receives the correction coefficient from the correction coefficient generation unit 230.

(Step S103)
The target camera performs the following processing on each image captured by the imaging unit 110.

(Step S104)
The imaging unit 110 captures an image showing the target space, and outputs image data 2 representing the captured image.

(Step S105)
The person detection unit 120 detects a person appearing in the image data 2 output by the imaging unit 110.

(Step S106)
The number of people measuring unit 130 measures the number of people in the image shown by the image data 2 based on the person detected by the person detection unit 120.

(Step S107)
The measured number of people correction unit 140 corrects the number of people measured by the number of people measurement unit 130 using a correction coefficient corresponding to the target camera, and outputs a corrected number of people measured 9 indicating the corrected number of people.

(Step S111)
The server 200 executes the following processing for each camera 100.

(Step S112)
The image receiving unit 210 receives image data 2 from the camera 100.

(Step S113)
The three-dimensional structure analysis unit 220 analyzes the three-dimensional structure in the target space shown in the image indicated by the received image data 2.

(Step S114)
The correction coefficient generation unit 230 calculates a correction coefficient based on the analysis result of the three-dimensional structure by the three-dimensional structure analysis unit 220, and transmits the calculated correction coefficient to the measured number of people correction unit 140. It is assumed that the transmitted correction coefficient is associated with data indicating the camera 100 corresponding to the correction coefficient.

With the above-described configuration and operation, the correction coefficient generation unit 230 accumulates correction coefficients based on the three-dimensional structure of the object to be photographed corresponding to each image data 2.

FIG. 4 is a diagram illustrating correction coefficients, and is a diagram showing a specific example of an input image and each frame provided for convenience of explanation. In FIG. 4, the white solid line frame and the black solid line frame are the three-dimensional structure analyzed by the three-dimensional structure analysis unit 220, and indicate the three-dimensional structure of the three-dimensional space that is the target space shown by the input image. There is. The white solid line frame is a frame corresponding to the upper part of the three-dimensional space, and the black solid line frame is a frame corresponding to the lower part of the three-dimensional space. Note that in FIG. 4, a white solid line frame and a black solid line frame are given to only part of the three-dimensional space for convenience of explanation, but in reality, the three-dimensional space extends over the entire input image. There is. Therefore, the three-dimensional structure analysis unit 220 analyzes the upper and lower parts of the three-dimensional space that extends over the entire input image.
The back area indicates an area that exists on the back side relative to the position of the camera 100 in the three-dimensional space. The near side area indicates an area that exists relatively to the near side with respect to the position of the camera 100 in the three-dimensional space.
The person detection unit 120 detects each person appearing in the input image. Generally, with technology for detecting people from images, it is possible to detect relatively large people with relatively high accuracy, and relatively small people can be detected with relatively high accuracy. can only be detected with low accuracy. Therefore, the correction coefficient generation unit 230 makes a relatively large correction for a person shown in the back area, and a relatively small correction for a person shown in the front area. When such a correction coefficient is calculated, the people flow measurement system 90 can measure the number of people in the target space with relatively high accuracy as a whole.

FIG. 5 shows that as a result of three-dimensional structure analysis by the three-dimensional structure analysis unit 220, a part of the area is shielded when viewed from the position of the camera 100 that captured the image data 2 in the three-dimensional space indicated by the input image. An example is shown in which a three-dimensional structure that is a shield is analyzed. In this example, there is a possibility that a person exists behind the three-dimensional structure, but the conventional human flow measurement method cannot detect the person behind the three-dimensional structure. Therefore, with conventional people flow measurement methods, it may not be possible to measure the number of people in a target space with relatively high accuracy.
In this example, the correction coefficient corresponding to the number of people behind the three-dimensional structure is set based on the three-dimensional size of the analyzed three-dimensional structure. It is desirable to do so. Therefore, as a specific example, the correction coefficient generation unit 230 generates an area where the number of people behind the three-dimensional structure is three-dimensionally equivalent to the area behind the three-dimensional structure (as a specific example, the broken line in FIG. 5 Calculate a correction coefficient that will be equivalent to the number of people measured in the area indicated by the frame). The area indicated by the broken line frame corresponds to the reference area. By using the correction coefficients calculated in this way, it is possible to calculate the number of people with a certain degree of accuracy even in areas blocked by shielding objects in the input image. Therefore, the people flow measurement system 90 can measure the number of people with relatively high accuracy as a whole.
Note that the correction coefficient may be data indicating an area to be referred to when estimating the number of people behind the shielding object.

***Explanation of effects of Embodiment 1***
As described above, according to the present embodiment, the number of people to be measured is corrected using a correction coefficient calculated in consideration of the three-dimensional structure of the target space. Therefore, according to this embodiment, the number of people can be measured more accurately using the people flow measurement method.

***Other configurations***
<Modification 1>
FIG. 6 shows an example of the hardware configuration of each device included in the people flow measurement system 90 according to this modification. FIG. 6 shows a server 200 as a representative of each device.
Each device included in the people flow measurement system 90 includes a processing circuit 18 in place of the processor 11, the processor 11 and the image processing processor 14, or the processor 11 and the memory 12.
The processing circuit 18 is hardware that implements at least a portion of each part included in the people flow measurement system 90.
Processing circuit 18 may be dedicated hardware or may be a processor that executes a program stored in memory 12.

When the processing circuit 18 is dedicated hardware, the processing circuit 18 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array) or a combination thereof.
The people flow measurement system 90 may include a plurality of processing circuits that replace the processing circuit 18. The plurality of processing circuits share the role of the processing circuit 18.

In the people flow measurement system 90, some functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

The processing circuit 18 is implemented, for example, by hardware, software, firmware, or a combination thereof.
The processor 11, memory 12, image processor 14, and processing circuit 18 are collectively referred to as a "processing circuitry." That is, the functions of each functional component of the people flow measurement system 90 are realized by processing circuitry.

***Other embodiments***
Although Embodiment 1 has been described, a plurality of parts of this embodiment may be implemented in combination. Alternatively, this embodiment may be partially implemented. In addition, this embodiment may be modified in various ways as necessary, and may be implemented as a whole or in part in any combination.
Note that the embodiments described above are essentially preferable examples, and are not intended to limit the present disclosure, its applications, and the scope of use. The procedures described using flowcharts and the like may be changed as appropriate.

2 image data, 9 number of people measured after correction, 11 processor, 12 memory, 13 image input interface, 14 image processing processor, 15 network interface, 18 processing circuit, 90 people flow measurement system, 100 camera, 110 imaging unit, 120 person detection unit , 130 people counting section, 140 measured number of people correction section, 200 server, 210 image receiving section, 220 three-dimensional structure analysis section, 230 correction coefficient generation section.

Claims (7)

a person detection unit that detects a target object that is an object appearing in an input image that is a captured image of a target space and that is an object that exists within the target space;
a people counting unit that measures the number of detected target objects;
Correcting the number of measured target objects using a correction coefficient that is a coefficient generated based on the three-dimensional structure of the target space and used to correct the number of measured target objects. A people flow measurement system comprising a first people flow measurement device comprising a measured number of people correction unit ,
The three-dimensional structure indicates the depth within the target space based on the position of the device that captured the input image. The people flow measurement system further includes:
a three-dimensional structure analysis unit that analyzes a three-dimensional structure of the target space based on the input image;
The people flow measurement system according to claim 1, further comprising a second people flow measurement device including a correction coefficient generation unit that generates the correction coefficient based on an analyzed three-dimensional structure. When the input image includes an obstruction, the correction coefficient generation unit is configured to calculate the correction coefficient within a reference area that is an area within the input image that corresponds to the size of the obstruction in the input image. Based on the number of target objects in the image, the correction coefficient is used to estimate the number of target objects that exist behind the shielding object when the position of the camera that captured the input image is used as a reference. The people flow measurement system according to claim 2, which generates a coefficient. The first people flow measuring device is a camera,
The people flow measuring system according to claim 2 or 3, wherein the second people flow measuring device is a computer. The human flow measuring system according to any one of claims 1 to 3 , wherein the target object is a human body. The computer detects a target object, which is an object appearing in an input image that is a captured image of a target space, and is an object existing within the target space,
the computer measures the number of detected target objects;
The computer calculates the measured target object using a correction coefficient that is a coefficient generated based on the three-dimensional structure of the target space and is used to correct the number of measured target objects. A method of measuring people flow that corrects the number of people ,
The three-dimensional structure indicates a depth within the target space based on the position of a device that captured the input image. A person detection process that detects a target object that is an object that is reflected in an input image that is an image of a target space and that is an object that exists within the target space;
A people counting process that measures the number of detected target objects;
Correcting the number of measured target objects using a correction coefficient that is a coefficient generated based on the three-dimensional structure of the target space and used to correct the number of measured target objects. A people flow measurement program that causes a computer included in a first people flow measurement device to execute a measured number of people correction process ,
The three-dimensional structure is a people flow measurement program that indicates the depth within the target space based on the position of the device that captured the input image.

Images