JP2020145511A - Camera device, person detection method, and person detection system - Google Patents

Abstract

To detect a person in a crowded imaging area with high accuracy.SOLUTION: A camera device includes an imaging unit that captures an imaging area, and a detection unit that detects a person appearing in the imaging area captured by the imaging unit by each of a plurality of detection objects, and determines congestion in the imaging area according to a comparison between the number of detected detection objects and a predetermined threshold value, and when the result of the congestion determination is non-congestion, the detection unit detects the person using a first detection object for detecting the person from the whole body of the person, and detects the person using a second detection object for detecting the person from a part of the person when the result of the congestion determination is congestion.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a camera device, a person detection method and a person detection system.

Conventionally, in order to appropriately protect the privacy of a person reflected in the imaging area, an accurate flow line analysis image in which the retention information or passage information of the person is superimposed on the background image updated at a predetermined timing is created and a simple operation is performed. A flow line analysis system and a flow line analysis method that can confirm the transition of the flow line analysis image are known.

For example, Patent Document 1 repeatedly creates a background image of an image captured in an imaging region captured by a camera device equipped with a fish-eye lens, and extracts flow line information regarding a retention position or a passing position of a moving body included in the image. It discloses a flow line analysis system and a flow line analysis method for creating a flow line analysis image in which the flow line information of a moving body is superimposed on the background image of the captured image. This flow line analysis system and the flow line analysis method have a second flow line created at the second timing with respect to the first flow line analysis image created at the first timing in response to the timing change operation. Switch to the analysis image and display it.

Japanese Unexamined Patent Publication No. 2016-206810

However, in the configuration of Patent Document 1, since a person is detected by the difference from the background image, for example, when the imaging area is crowded with many people, it becomes difficult to extract the background image and the person reflected in the imaging area. Could not be detected. Further, in the configuration of Patent Document 1, by imaging from directly above the imaging region with a fisheye lens, a person and a person can be distinguished by the difference in the crown portion, but the distortion increases as the distance from the center of the imaging region increases and the image quality becomes higher. There was a possibility that the person reflected in the imaging area could not be detected due to the deterioration of the image. Further, in a security camera device used in a facility such as a convenience store or a department store, the camera device is installed so that the lower side diagonally forward can be imaged at a fixed angle of view rather than the above-mentioned image capturing from directly above the imaging area. In many cases, such an installation makes it difficult to separate the person and the person reflected in the imaging area at the time of congestion, and there is a possibility that the person cannot be detected accurately.

The present disclosure has been devised in view of the above-mentioned conventional circumstances, and an object of the present disclosure is to provide a camera device, a person detection method, and a person detection system capable of detecting a person reflected in a congested imaging region with high accuracy.

In the present disclosure, an imaging unit that captures an imaging region and a person that appears in the imaging region imaged by the imaging unit are detected and detected by each of a plurality of detection objects that target a part or the whole body of the person. The detection unit includes a detection unit that determines congestion in the imaging region according to a comparison between the number of detected objects for detection and a predetermined threshold value, and the detection unit determines when the result of the congestion determination is non-congestion. The person is detected by using the first detection object for detecting the person from the whole body of the person, and when the result of the congestion determination is congestion, the person is detected from a part of the person. Provided is a camera device that detects the person using a second detection object for the purpose.

Further, the present disclosure is a person detection method for detecting a person reflected in an imaging region, in which the person is detected by each of a plurality of detection objects targeting a part or the whole body of the person, and the imaging region is described. The congestion determination of the imaging region is performed according to the comparison between the number of detected objects detected from the above and a predetermined threshold value, and when the result of the congestion determination is non-congestion, the person is detected from the whole body of the person. The person is detected by using the first detection object, and when the result of the congestion determination is congestion, the person is detected by using the second detection object that detects the person from a part of the person. Provide a method for detecting a person to be detected.

Further, the present disclosure is a person detection system including a camera device for capturing an imaging region and a display device, wherein the camera device detects a person in a part or the whole body of the person. The congestion of the imaging region is determined according to the comparison between the number of detected objects detected from the imaging region and a predetermined threshold value, and when the result of the congestion determination is non-congestion. , The person is detected by using the first detection object that detects the person from the whole body of the person, and when the result of the congestion determination is congestion, the person is detected from a part of the person. Provided is a person detection system that detects the person using the detection object of 2.

According to the present disclosure, a person reflected in a congested imaging region can be detected with high accuracy.

Explanatory drawing which shows an example of the surveillance system using the camera apparatus which concerns on Embodiment 1. The figure which shows an example of the system configuration of the camera device, the PC and the server device which concerns on Embodiment 1. Explanatory drawing which shows setting example (front) of foot coordinate calculation data Explanatory drawing which shows setting example (back) of foot coordinate calculation data Sequence diagram showing an example of the procedure for creating foot coordinate calculation data Diagram showing candidate candidates for multiple tracking objects Diagram showing an example of foot coordinate calculation data Flowchart showing an example of determining an object for tracking Flowchart showing an example of counting procedure for people reflected in the imaging area Diagram showing an example of detecting a person in a non-crowded state Diagram showing an example of detecting a person in a crowded state The figure which shows the operation procedure example of the number of people detection system using the camera apparatus which concerns on Embodiment 1. The figure which shows the example of the image captured by the camera device Diagram showing an example of count result data

(Background to the contents of the first embodiment)
Recently, a background image of an image captured in an imaging area captured by a camera device equipped with a fish-eye lens is repeatedly created to extract flow line information regarding a staying position or a passing position of a moving body included in the image, and a background image of the captured image. There are a flow line analysis system and a flow line analysis method for creating a flow line analysis image in which the flow line information of a moving body is superimposed. Similarly, a security camera device used in a facility such as a convenience store or a department store shall be installed so as to be able to image diagonally forward and lower sides with a fixed angle of view rather than imaging from directly above the imaging area. There are many. Therefore, when such a camera device is installed, it is difficult for the camera device equipped with the conventional fisheye lens to separate the person and the person reflected in the imaging area at the time of congestion, and there is a possibility that the person cannot be detected accurately. there were. In the above-mentioned Patent Document 1, a technique of detecting a person by a camera device installed so as to be able to image an obliquely front lower side with a fixed angle of view is not assumed.

Hereinafter, embodiments in which the configuration and operation of the camera device, the person detection system, and the person detection method according to the present disclosure are specifically disclosed will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of already well-known matters and duplicate explanations for substantially the same configuration may be omitted. This is to avoid unnecessary redundancy of the following description and to facilitate the understanding of those skilled in the art. It should be noted that the accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

(Embodiment 1)
FIG. 1 is an explanatory diagram showing an example of a person detection system 1000 using the camera device 100 according to the first embodiment. The person detection system 1000 shown in FIG. 1 includes a number counting system 500A, 500B, 500C, ... Installed in each of a plurality of facilities A, B, C, ..., And a PC (Personal Computer) 600 installed in the head office HO. And the cloud server 700. The plurality of facilities A, B, C, ... May be a facility such as a convenience store, a department store, a large commercial complex, a drug store, or one facility such as a department store or a large commercial complex. Further, the camera device according to the first embodiment is a camera device for crime prevention, and is installed so as to be able to image an obliquely front lower side with a fixed angle of view than an image taken from directly above the image pickup area.

The respective number counting systems 500A, 500B, 500C, ..., The PC 600 installed at the head office HO, and the cloud server 700 are connected to each other so as to be able to communicate with each other via the network NW1. The network NW1 is a wireless network or a wired network. Wireless networks include, for example, wireless LAN (Local Area Network), wireless WAN (Wide Area Network), 4G (4th generation mobile communication system), LTE (Long Term Evolution), LTE-Advanced, 5G (5th generation mobile communication system). ), Wi-fi (registered trademark), or WiGig (Wireless Gigabit). Wired networks are, for example, intranets or the Internet.

The number counting system 500A installed in the facility A shown in FIG. 1 includes a plurality of camera devices 100, 100A, 100B, 100C installed on the floor 1, a recorder 200, a server device 300, an input device 400, and a monitor. It is configured to include 450 and. The floor 2 has the same configuration as the floor 1 and a plurality of camera devices are installed, but the camera devices are not shown in FIG. Further, the internal configurations of the camera devices 100, 100A, 100B, 100C, ... Used on each floor of the facility A are the same.

The recorder 200 is configured by using, for example, a semiconductor memory or a hard disk device, and data of captured images captured by each of a plurality of camera devices 100, 100A, 100B, 100C, ... Installed in the facility A (hereinafter, imaging). Image data and notation) is stored. The captured image data stored in the recorder 200 is provided, for example, for crime prevention monitoring work, business management work, and the like.

The server device 300 as an example of the external terminal responds to an input operation of, for example, a user who operates the input device 400 (for example, an employee working at the facility A, a manager who manages the facility A, an employee of the head office HO, etc.). , Predetermined captured image data and data indicating the count result of the number of people count as an example of the measurement result described later (hereinafter, count result data) are displayed on the monitor 450.

The input device 400 is composed of, for example, a mouse, a keyboard, a touch panel, or a touch pad, and outputs a signal corresponding to a user's input operation to the camera devices 100, 100A, 100B, 100C, ..., Or the server device 300. In the input device 400 shown in FIG. 1, an arrow (broken line) is shown only for the camera device 100 in order to simplify the drawing, but an arrow is drawn between the input device 400 and the other camera devices 100A, 100B, 100C, ... It may be illustrated.

The monitor 450 as an example of the display device is configured by using, for example, an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence), and displays the count result data created by the server device 300. Although the monitor 450 is provided as an external terminal different from the server device 300, it may be included in the server device 300.

The PC600 installed in the head office HO is installed in each facility A, B, C, ... In response to an input operation by a user of the head office HO who operates the PC600 (for example, an employee or an officer who performs management work). The count result data created by the number of people counting system 500A, 500B, 500C, ... And the captured image data captured by the camera devices 100, 100A, 100B, 100C, ... Are acquired and displayed. In addition, the PC 600 holds various information (for example, number of visitors, event schedule information, maximum temperature information, minimum temperature information, weather information, sales information, etc.) necessary for creating count result data. Note that these various types of information may be stored in a server device (for example, a server device 300 in the case of facility A) provided for each facility A, B, C, .... The PC 600 may execute each process in the server device installed in the facilities A, B, C, .... As a result, the PC600 can aggregate the data of each facility A, B, C, ... To create the count result data, and the detailed data of one facility selected by the input operation to the PC600 of the head office HO (for example). , The count result data shown in FIG. 12 etc.), or the result of comparing the data of specific areas (for example, confectionery section, meat section, etc.) and facilities of each facility A, B, C, ... Can be displayed.

The cloud server 700 is used in the count result data created in the number counting systems 500A, 500B, 500C, ... Installed in each facility A, B, C, ..., And in the camera devices 100, 100A, 100B, 100C, .... The captured image data is stored. The cloud server 700 receives a predetermined process (for example, a specified date and a specified period) in response to an input operation of a user of the head office HO (for example, an employee or an officer who performs management work) who operates the PC 600 of the head office HO. Search and extract of the count result data), and send the processing result to the PC600 installed in the head office HO.

FIG. 2 is a diagram showing an example of a system configuration of the camera device 100, the PC 600, and the server device 300 according to the first embodiment. The camera device 100, the PC 600, and the server device 300 are connected to each other so as to be able to communicate with each other via the network NW1.

The network NW1 is a wireless network or a wired network. Wireless networks include, for example, wireless LAN (Local Area Network), wireless WAN (Wide Area Network), 4G (4th generation mobile communication system), LTE (Long Term Evolution), LTE-Advanced, 5G (5th generation mobile communication system). ), Wi-fi (registered trademark), or WiGig (Wireless Gigabit). Wired networks are, for example, intranets or the Internet.

The camera device 100 includes a control unit 10, an imaging unit 11, a memory 19, a receiving unit 110, and a transmitting unit 120. Further, the control unit 10 includes a detection unit 12, a tracking unit 13, a line passage determination unit 14, a number counting unit 15, and learning data 16. Further, the camera device 100 is installed at a position where an imaging region to be imaged can be imaged.

The control unit 10 is configured by using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array). The control unit 10 functions as a controller that controls the overall operation of the camera device 100, controls processing for controlling the operation of each part of the camera device 100 in cooperation with the memory 19, and between each part of the camera device 100. Data input / output processing, data calculation (calculation) processing, and data storage processing. The control unit 10 operates according to the program and data stored in the memory 19. The control unit 10 uses the memory 19 during operation, and temporarily stores the data or information created or acquired by the control unit 10 in the memory 19.

Further, the control unit 10 performs the latest mask area data, background image data, and foot coordinate calculation data by machine learning based on an captured image captured at a predetermined frame rate (for example, 10 fps (frame per second)) described later. (See FIG. 6) is created, and these data are set as a part of the training data 16.

The mask area data is stored in the memory 19, for example, and is data relating to an area in which a mirror image of a person such as a door or glass is reflected, and is set as an area to be excluded from the target of person detection. The mask area data is data for suppressing false detection of a part of a person and the whole body. When the camera device 100 is installed, the mask area data is set by the PC 600 or the server device 300 as initial setting data based on the first captured image. Further, the mask area data is created as the latest mask area data by machine learning based on the captured image captured at a predetermined frame rate, which will be described later, after the camera device 100 is installed. The latest background image data created is set as a part of the training data 16.

The background image data is stored in the memory 19, for example, and is created by excluding moving objects (for example, a part of the person and the whole body) included in the captured image, and detects a part or the whole body of the person reflected in the imaged area. It is the data of the background image to do. When the camera device 100 is installed, the background image data is set by the PC 600 or the server device 300 as initial setting data based on the first captured image. Further, the background image data is created as the latest background image data by machine learning based on the captured image captured at a predetermined frame rate, which will be described later, after the camera device 100 is installed. The latest background image data created is set as a part of the training data 16.

The foot coordinate calculation data is data for calculating the foot coordinates of the person based on a part of the detected person and the position of the person in the captured image. As will be described later, when the camera device 100 is installed, the foot coordinate calculation data is acquired from the server device 300 or the PC 600 based on the installation angle of the camera device 100, and is captured in the captured image and the imaging region. The camera device 100 itself may create and set the foot coordinate calculation data based on the person to be reflected. The foot coordinate calculation data created when the camera device 100 is set is stored in the memory 19 as initial setting data. Further, the foot coordinate calculation data may be created as the latest foot coordinate calculation data by machine learning based on the captured image captured at a predetermined frame rate, which will be described later, after the camera device 100 is installed. The latest created foot coordinate calculation data is set as a part of the learning data 16.

The memory 19 is, for example, a RAM (Random Access Memory) as a work memory used when executing each process of the control unit 10, and a ROM (Read Only Memory) for storing a program and data defining the operation of the control unit 10. And have. Data or information created or acquired by the control unit 10 is temporarily stored in the RAM. The ROM contains the operation of the control unit 10 (for example, a method in which the detection unit 12 detects a person and a part of the person from the captured image data captured by the image pickup unit 11, and a person whose tracking unit 13 is detected by the detection unit 12. A program is written that specifies how to track a part of the whole body and the whole body. The memory 19 stores data such as the installation position and installation angle of the camera device 100 when the camera device 100 is installed.

In addition, the memory 19 stores captured image data captured by the imaging unit 11, data on a part of the person detected by the detecting unit 12 and the whole body, data on the number of people counted by the number counting unit 15, and the like. Further, the memory 19 stores various data preset at the time of installation of the camera device 100. The various data are, for example, mask area data, background image data, foot coordinate calculation data, and the like. Various data are transmitted from the PC 600 or the server device 300 and received by the receiving unit 110 via the network NW1.

The image pickup unit 11 includes at least a lens and an image sensor. The imaging unit 11 images an imaging region at a predetermined frame rate (for example, 30 fps or 60 fps (frame per second)). Further, the image sensor is, for example, a CCD (Charged-Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) solid-state image sensor, and converts an optical image imaged on an imaging surface into an electric signal.

The detection unit 12 is configured by using, for example, a CPU, an MPU, or a DSP, and performs predetermined signal processing using an electric signal from the image pickup unit 11 to perform human-recognizable RGB (Red Green Blue) or YUV (Red Green Blue) or YUV ( Create captured image data defined by brightness / color difference). Based on the data of the captured image created, the detection unit 12 detects a part and the whole body of the person reflected in the imaging region by comparing with the mask area data and the background image data. The detection unit 12 detects a person appearing in the image pickup area excluding the area excluded from the person detection target based on the mask area data in the image pickup area. The detection unit 12 includes a first detection unit 12a, a second detection unit 12b, a third detection unit 12c, and a fourth detection unit 12d that detect each of a part of a person and the whole body reflected in the imaging region.

Each of the first detection unit 12a, the second detection unit 12b, the third detection unit 12c, and the fourth detection unit 12d tracks a part of the detected person and a part corresponding to the whole body by the tracking unit 13. Detected as a candidate for tracking object. The tracking object candidate is an example of a detection object. Each of the first detection unit 12a, the second detection unit 12b, the third detection unit 12c, and the fourth detection unit 12d detects each of a plurality of candidates for tracking objects indicating a part of a person. Further, each of the first detection unit 12a, the second detection unit 12b, the third detection unit 12c, and the fourth detection unit 12d is a person detection frame OB1, OB2, OB3 indicating the size of the detected object candidate for tracking. , OB4 (see FIG. 5). Each of the person detection frames OB1, OB2, OB3, and OB4 will be described later with reference to FIG.

Further, the detection unit 12 uses each of the plurality of tracking object candidates detected by each of the first detection unit 12a, the second detection unit 12b, the third detection unit 12c, and the fourth detection unit 12d as the detection result. get. The detection unit 12 compares the acquired detection result with a predetermined threshold value, and determines congestion in the imaging region. For example, the detection unit 12 compares the number of detections of a part of a plurality of persons included in the detection result with the whole body, and whether the imaging region is congested according to the comparison result and a predetermined threshold value set. Judge whether or not. The predetermined threshold value may be, for example, the ratio of the number of detected heads to the number of detected heads during imaging, or a value indicating the number of people. Based on the result of the congestion determination, the detection unit 12 sets the whole body of the person as an object for tracking in the case of non-congestion, and sets a part of the person as an object for tracking in the case of congestion. Output to the tracking unit 13. An example as the first detection object is a tracking object set in the case of non-congestion, and an example as the second detection object is a tracking object set in the case of congestion. Is.

The tracking unit 13 sets the data 13a of the tracking object for detecting and tracking each of a plurality of persons reflected in the imaging region based on the tracking object input from the detection unit 12. The tracking unit 13 creates the flow line information of a person moving in the imaging region by tracking by calculating the foot coordinates of the person using the tracking object and the foot coordinate calculation data. The flow line information is created for each of a plurality of persons reflected in the imaging region, and is information indicating at which position the person stayed or passed. The tracking unit 13 outputs the created flow line information to the line passage determination unit 14. The tracking unit 13 stores the input object for tracking until the tracking of the data 13a is completed.

The line passage determination unit 14 determines whether or not the person has passed the predetermined line based on the flow line information of the person input from the tracking unit 13 and the set predetermined line. The predetermined line is preset by the PC 600 with respect to an arbitrary position in the imaging region, and the number of lines may be plural. The line passage determination unit 14 can distinguish the direction of passing through a predetermined line based on the flow line information. Therefore, the line passage determination unit 14 can distinguish between the number of people entering the store and the number of people leaving the store for a predetermined line provided at the entrance of the facility, for example. As a result, the count result data created by the number counting systems 500A, 500B, 500C, ... Of each facility A, B, C, ... Is based on the flow line information of the person such as the number of people entering and leaving the store. Since the data that can be obtained is included, the employees of the head office HO can obtain more diverse information from the count result data. Further, when the line passage determination unit 14 determines that the person has passed the predetermined line, the line passage determination unit 14 outputs the determination result for each passing direction with respect to the predetermined line to the number counting unit 15.

The number of people counting unit 15 counts the number of people who have passed a predetermined line based on the determination result input from the line passing determination unit 14, and outputs the count result to the transmission unit 120.

When the camera device 100 is installed, the receiving unit 110 supplies the network NW1 with data necessary for detecting and counting a person such as mask area data, background image data, and foot coordinate calculation data from the PC 600 or the server device 300. Receive via. In addition, the receiving unit 110 counts a predetermined threshold value for determining congestion in the imaging region imaged by the camera device 100, or the number of people reflected in the imaging region, in response to an input operation of the user who operates the input device 400. When the information such as a predetermined line is changed, the information is received.

The transmission unit 120 includes learning data 16 (mask area data, background image data, foot coordinate calculation data, etc.) created at any time from the captured image captured by the imaging unit 11, movement line information regarding the retention or passage of a person, and The count result data of the person passing through the predetermined line is acquired and transmitted to the server device 300.

Next, the PC 600 shown in FIG. 2 will be described. The PC 600 as an example of the external terminal and the display device includes a control unit 20, a display unit 21, an input unit 22, a storage unit 28, a memory 29, a receiving unit 210, and a transmitting unit 220. To. In the following description, only the camera device 100 is described for the sake of simplicity, but other camera devices 100A, 100B, 100C, ... Which are communicably connected to the PC 600 may be used. ..

The control unit 20 is configured by using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array). The control unit 20 functions as a controller that controls the overall operation of the PC 600, controls processing for controlling the operation of each part of the PC 600 in cooperation with the memory 29, and data input / output processing with each part of the PC 600. , Performs data calculation processing and data storage processing. The control unit 20 operates according to the program and data stored in the memory 29. The control unit 20 uses the memory 29 during operation, and temporarily stores the data or information created or acquired by the control unit 20 in the memory 29.

The display unit 21 displays the count data created by the camera device 100. Further, the display unit 21 displays setting information such as a predetermined threshold value for determining congestion in the imaging region imaged by the camera device 100, or a predetermined line for counting the number of people reflected in the imaging region.

The input unit 22 displays setting information such as a predetermined threshold value for determining congestion in the imaging region to be imaged for each camera device, or a predetermined line for counting the number of people reflected in the imaging region. Further, the input unit 22 inputs data as initial settings (for example, mask area data, background image data, foot coordinate calculation data, etc.) necessary for installing the camera device 100 based on the user's input operation. The mask area data, the background image data, and the foot coordinate calculation data are based on the captured image first captured when the camera device 100 is installed, even if the data is set by the user and stored in the storage unit 28. It may be data set by the user. In addition, the input unit 22 can change the settings of existing settings (for example, a predetermined threshold value, a predetermined line, mask area data, foot coordinate calculation data, etc.) according to the input operation of the user.

The storage unit 28 stores the set value set for each camera device by the server device 300 or the PC 600. Further, the storage unit 38 has the count data acquired from the camera device 100 or the count data processed by the PC 600 (for example, the count data obtained by the other camera devices 100A, 100B, 100C, ... And the camera device 100, respectively. The data comparing the above, etc.) is stored. The storage unit 28 stores data as initial settings (for example, mask area data, background image data, foot coordinate calculation data, etc.) set by the user when installing the camera device for each camera device.

The memory 29 is, for example, a RAM (Random Access Memory) as a work memory used when executing each process of the control unit 20, and a ROM (Read Only Memory) for storing a program and data defining the operation of the control unit 10. And have. Data or information created or acquired by the control unit 20 is temporarily stored in the RAM. In the ROM, the operation of the control unit 20 (for example, a method of acquiring and processing each of the count data created for each camera device, and a method of setting a predetermined threshold value or a predetermined line for each camera device). Etc.) is written in the program. The memory 29 stores data such as the installation position and installation angle of the camera device 100 that are set when the camera device 100 is installed.

The receiving unit 210 receives the setting information set in the current camera device 100 or the count result data created by the camera device 100. Further, the transmission unit 220 transmits the setting information when the existing settings (for example, a predetermined threshold value, a predetermined line, mask area data, foot coordinate calculation data, etc.) are changed according to the input operation of the user. be able to.

Next, the server device 300 shown in FIG. 2 will be described. The server device 300 as an example of the external terminal includes a control unit 30, a storage unit 38, a memory 39, a reception unit 310, and a transmission unit 320. In the following description, only the camera device 100 is described for the sake of simplicity, but other camera devices 100A, 100B, 100C, ... That are communicably connected to the server device 300. May be good.

The server device 300 changes the settings of existing settings (for example, a predetermined threshold value, a predetermined line, mask area data, foot coordinate calculation data, etc.) according to an input operation of a user who operates the input device 400 shown in FIG. It can be performed.

The control unit 30 is configured by using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array). The control unit 30 functions as a controller that controls the overall operation of the server device 300, controls processing for controlling the operation of each part of the server device 300 in cooperation with the memory 39, and between each part of the server device 300. Performs data input / output processing, data arithmetic processing, and data storage processing. The control unit 30 operates according to the program and data stored in the memory 39. The control unit 30 uses the memory 39 during operation, and temporarily stores the data or information created or acquired by the control unit 30 in the memory 39.

The storage unit 38 stores the set value set for each camera device by the server device 300 or the PC 600. Further, the storage unit 38 stores the count data created by the camera device 100 or the count data processed by the PC 600.

The receiving unit 310 receives the count result data created by the camera device 100. The count result data received by the receiving unit 310 is stored in the storage unit 38. Further, when the existing settings (for example, a predetermined threshold value, a predetermined line, mask area data, foot coordinate calculation data, etc.) are changed according to the input operation of the user, the transmission unit 320 uses the setting information. Can be sent.

Next, a setting example of the foot coordinate calculation data will be described with reference to FIGS. 3A and 3B. FIG. 3A is an explanatory diagram showing a setting example (front surface) of the foot coordinate calculation data. FIG. 3B is an explanatory diagram showing a setting example (back surface) of the foot coordinate calculation data.

3A and 3B show how the camera device 100 captures the captured image SC1 in which the person HU1 appears at the time of installation and sets the foot coordinate calculation data. The foot coordinate calculation data is set based on the captured image SC1 captured by the camera device 100 and the person HU1 reflected in the captured image SC1. Further, the foot coordinate calculation data is set using the captured images SC1 on the front surface and the back surface of the person HU1. For the sake of simplicity, in FIGS. 3A and 3B, one captured image SC1 and one SC2 are shown for each of the front and back surfaces of the person, but the foot coordinates are obtained using a plurality of captured images. Calculation data may be set.

In addition, the captured image used for setting the foot coordinate calculation data is also stored as data for performing machine learning of the camera device 100. Therefore, the camera device 100 may detect a part or the whole body of the person by using each of the captured images showing the front and the back of the person, or may use either of them to detect a part of the person or a part of the person or The whole body may be detected. Further, since the feature amount (in other words, the difference with respect to the background image data) used for detection is significantly different in the captured image in which the back surface of the person is reflected in the camera device 100, in the captured image in which the back surface of the person is reflected, the head of the person is displayed. It becomes difficult to detect. For this reason, the camera device 100 can efficiently perform machine learning when the captured image used for setting the foot coordinate calculation data uses the captured image in which the head of the person is reflected on each of the front and back of the person. It is possible to detect the head of a person with higher accuracy during operation.

FIG. 3A shows a state in which the foot coordinate calculation data is set in a state where the person HU1 shown in the captured image SC1 is facing the front. On the captured image SC1, the line L1, the foot coordinates Pt1 of the person HU1, and the person detection frame OB11 are displayed. The camera device 100 detects the person HU1 reflected in the captured image SC1, detects and sets the person detection frame OB11 indicating the detection of the whole body of the person HU1 and the foot coordinates Pt1 of the person HU1. The line L1 is set based on the line setting preset in the PC 600.

Similarly, FIG. 3B shows how the foot coordinate calculation data is set in a state where the person HU2 shown in the captured image SC2 is facing the back surface. The person HU2 and the person HU1 may be the same person. On the captured image SC2, the line L2, the foot coordinates Pt2 of the person HU2, and the person detection frame OB12 are displayed. The camera device 100 detects the person HU2 reflected in the captured image SC2, detects and sets the person detection frame OB12 indicating the detection of the whole body of the person HU2 and the foot coordinates Pt2 of the person HU2. The line L2 is set based on the line setting preset in the PC 600. The line L2 is the same line as the line L1.

FIG. 4 is a sequence diagram showing an example of the setting procedure of the foot coordinate calculation data. The foot coordinate calculation data is set by the camera device 100 and the PC 600.

The camera device 100 captures a plurality of captured images of how a person reflected in the imaging region passes through different positions by the imaging unit 11 (T1).

The camera device 100 detects the whole body of a person as a setting object from each of the plurality of captured images by the first detection unit 12a (T2). The object for setting here is an object in which the whole body of a person is detected.

The camera device 100 sets each of the setting objects detected by the detection unit 12 to the foot coordinates of the detected setting object and the size of the setting object (in other words, the size of a person). Acquire in association (T3).

The camera device 100 transmits the detection results associated with the foot coordinates and the sizes of the plurality of objects for setting acquired from the transmission unit 120 to the PC 600 via the network NW1 (T4).

The PC 600 receives each of the foot coordinates and the size of the plurality of setting objects by the receiving unit 210 (T5).

The PC 600 compares each of the received foot coordinates and sizes of a plurality of objects for setting with the average height data of a person set in the PC 600 in advance, and is an object corresponding to a predetermined coordinate of the imaging region. The relative size of is calculated as set values, and foot coordinate calculation data is created based on these set values (T6).

The PC 600 stores the created foot coordinate calculation data in the storage unit 28 as the foot coordinate calculation data of the camera device 100, and transmits the created foot coordinate calculation data from the transmission unit 220 to the camera device 100 via the network NW1 (T7).

The camera device 100 receives the foot coordinate calculation data by the receiving unit 110 (T8).

The camera device 100 stores the received foot coordinate calculation data in the memory 19 and sets it as the foot coordinate calculation data referred to by the detection unit 12 (T9). As a result, the camera device 100 has the size of the candidate object for tracking (in other words, the person detection frames OB2, OB3, even if the detected object candidate for tracking is an object indicating a part of the person. OB4) and the size of the whole body of the person can be estimated according to the position of the candidate object for tracking, and further, the foot coordinates of the person can be estimated from the estimated size. Therefore, the camera device 100 can estimate the foot coordinates of the person reflected in the imaging region based on the created foot coordinate calculation data.

FIG. 5 is a diagram showing candidate candidates for a plurality of tracking objects. FIG. 5 is a captured image DE0 that captures a person HU0 reflected in the imaging region. Candidates for tracking objects are objects detected by each of the first detection unit 12a, the second detection unit 12b, the third detection unit 12c, and the fourth detection unit 12d. As candidates for the tracking object, each of the person detection frames OB1 to OB4 indicating the size of a part of the detected person and the whole body is set.

The detection unit 12 detects a part of the detected person HU0 and the whole body by machine learning using deep learning based on the captured image or by the difference from the background image. The first detection unit 12a detects the whole body of the person HU0, and sets the person detection frame OB1 in the region (size) where the whole body is detected. The second detection unit 12b detects the upper body which is a part of the person HU0, and sets the person detection frame OB2 in the area (size) where the upper body is detected. The third detection unit 12c detects the upper part of the scapula, which is a part of the person HU0, and sets the person detection frame OB3 in the detected region (size) on the scapula. The fourth detection unit 12d detects the head which is a part of the person HU0, and sets the person detection frame OB4 in the region (size) where the head is detected. Therefore, the person HU0 reflected in the imaging region is detected by the first detection unit 12a to the fourth detection unit 12d at each location of a part of the person and the whole body as candidates for a plurality of tracking objects. As a result, the camera device 100 can track the person HU01 in association with a plurality of tracking object candidates even when the camera device 100 is congested, so that the tracking accuracy can be improved.

As shown in FIG. 2, the first detection unit 12a to the fourth detection unit 12d are not limited to the four detection units, but may be increased depending on the installation angle of the camera device 100, the size of the imaging region, the setting of the detection target, and the like. Or you can reduce it. Further, the part and the whole body of the person detected by the first detection unit 12a to the fourth detection unit 12d are not limited to this, and may be a part of the face such as eyes and nose or arms and legs with large movement. .. As a result, the camera device 100 can set candidates for tracking objects according to various installation angles, installation environments, and detection targets, and can perform highly accurate person detection.

FIG. 6 is a diagram showing an example of foot coordinate calculation data. The foot coordinate calculation data shown in FIG. 6 is different from the foot coordinate calculation data described with reference to FIG. 4, and is data held in the memory 19 of the camera device 100, and is detected by the installation angle of the camera device 100. This is data for estimating the foot coordinates based on the detection position of a part of the person or the whole body.

The foot coordinate calculation data TB1 shown in FIG. 6 is configured to include the position of a person reflected in the imaging region corresponding to the installation angle of the camera device 100 and the size of the detected person. The camera device 100 compares the size of a part of the detected person with the detection sizes X1, X2, and X3 of the person, and the foot coordinates of the person are set to any one of the areas Ar1, Ar2, and Ar3. Estimate if it is located. Further, when the whole body of the person is detected, the camera device 100 compares the foot coordinates of the person with the detection sizes X1, X2, X3 of the person corresponding to the regions Ar1, Ar2, Ar3 in which the foot coordinates are located. By doing so, it is possible to create more accurate foot coordinate calculation data TB1. As shown in the imaging region model SC0, each of the regions Ar1, Ar2, and Ar3 is set as the region Ar1 in the foreground, the region Ar2 in the center, and the region Ar3 in the back in the imaging region. Further, each of the regions Ar1, Ar2, and Ar3 is not limited to the three regions as shown in FIG. 6, and may be further increased depending on the installation angle of the camera device 100, the size of the imaging region, and the detection accuracy required by the user. Area may be set. As a result, the camera device 100 can improve the estimation accuracy of the foot coordinates by estimating the foot coordinates of the person without requiring the setting by the user and further setting a large area.

FIG. 7 is a flowchart showing an example of determining an object for tracking. In FIG. 7, a plurality of persons reflected in the imaging area are detected by the detection unit 12, and an object suitable as an object for tracking the person reflected in the imaging area at the present time is set from the detected candidates for the plurality of tracking objects. It is a flowchart for doing.

The camera device 100 images the imaging region by the imaging unit 11 (St11). The camera device 100 detects a plurality of candidates for tracking objects for each of the plurality of persons appearing in the captured captured image by the first detection unit 12a to the fourth detection unit 12d (St12).

The camera device 100 includes the number of tracking object candidates indicating the whole body of the person detected by the first detection unit 12a and the second detection units 12b to 4th among the plurality of detected tracking object candidates. The number of candidates for the tracking object indicating a part of the person detected by the detection unit 12d is acquired (St13).

The camera device 100 compares the number of candidates for tracking objects indicating the whole body of the acquired person, the number of candidates for tracking objects indicating a part of the person, and a predetermined threshold value used for congestion determination. By doing so, the congestion is determined (St14). The predetermined threshold value is arbitrarily set by the user, and may be a value indicating a ratio or a value indicating the number of people.

Here, an example of a determination method for determining congestion will be described. In the camera device 100, for example, when the number of candidates for tracking objects showing the whole body of a person is 2, the number of candidates for tracking objects showing a part of a person (for example, the head) is 8. It is determined that, of the plurality of photographed persons, two persons can see the whole body of the person, and six persons can see only a part of the person. Here, when the predetermined threshold value is, for example, 50%, it is determined that the person is congested when the detection rate of the whole body of the person is 50% or less, and when the detection rate of the whole body of the person is larger than 50%. Judged as non-congested. The camera device 100 determines that the imaging region is congested because the detection rate of the whole body of a plurality of persons currently reflected in the imaging region is about 33%. When the predetermined threshold value is a value indicating the number of people, the determination can be made in the same manner. Further, the camera device 100 compares the candidates for the tracking object to be compared in the congestion determination based on the number of candidates for the tracking object in which, for example, the whole body and the upper body and the scapula and the head are detected, respectively. You may.

When the camera device 100 determines that the imaging region is congested as a result of the congestion determination in step St14 (St15, YES), the camera device 100 determines the object specified by the user as the tracking object (St16). .. The specified objects are all of the objects specified by the user, such as the upper body, the scapula, and the head, which represent a part of the person, but only on the scapula among the plurality of existing objects. May be good.

When the camera device 100 determines that the imaging region is not congested as a result of the congestion determination in step St14 (St15, NO), the camera device 100 determines an object indicating the whole body of the person as an object for tracking (St17). ..

The camera device 100 sets the determined tracking object (St18). As a result, the camera device 100 can easily track a plurality of people appearing in the captured captured image based on the determined tracking object. After the tracking object is determined in step St18, the camera device 100 shifts to the process of the flowchart showing an example of the counting procedure of the number of people reflected in the imaging region shown in FIG.

FIG. 8 is a flowchart showing an example of a procedure for counting a person reflected in the imaging region. FIG. 8 is a flowchart for tracking a person based on the tracking object after the tracking object is set by the processing of the flowchart shown in FIG. 7.

The camera device 100 determines whether or not the tracking object set in step St18 is an object indicating the whole body of the person (St21).

When the tracking object set in step St18 is an object indicating a part of a person (St21, NO), the camera device 100 steps from a plurality of tracking object candidates detected from the captured image. Only the tracking object set in St18 is extracted (selected) and determined (St22).

The camera device 100 refers to the foot coordinate calculation data based on the determined detection position of the tracking object and the detected size (St23). The foot coordinate calculation data to be referred to is new foot coordinate calculation data created by machine learning from the captured image captured by the camera device 100, even if the foot coordinate calculation data is set when the camera device 100 is installed. You may.

The camera device 100 estimates the foot coordinates of the person indicated by the tracking object based on the foot coordinate calculation data (St24).

In the camera device 100, when the tracking object set in step St18 is an object indicating the whole body of the person (St21, YES), or the tracking object set in step St18 is an object indicating a part of the person. Therefore, when the foot coordinates are estimated in step St24, the tracking unit 13 creates the flow line information for tracking the object for tracking. The camera device 100 determines whether or not the tracking object has passed the line set by the user based on the flow line information created by the line passage determination unit 14, and if so, the passage information (for example,). , Direction and time of passage, etc.) are detected (St25).

The camera device 100 counts the number of people corresponding to the number of tracking objects that have passed through the line based on the detected passage information by the number counting unit 15 (St26).

The camera device 100 creates count result data based on the count result of the number of people, transmits it to the PC 600 or the server device 300, and displays it (St27). After creating the count result data, the camera device 100 returns to the process of step St11 in FIG.

FIG. 9A is a diagram showing an example of detecting a person in a non-crowded state. FIG. 9B is a diagram showing an example of detecting a person in a crowded state. FIG. 9A is an example of detecting a person when the imaging region is determined to be non-congested as a result of the congestion determination, and the whole body of the person is detected. FIG. 9B is an example of detecting a person when the imaging region is determined to be congested as a result of the congestion determination, and a part of the person is detected. Note that FIG. 9A shows a state in which 0 persons and 1 person pass in the directions indicated by the passing direction icons LU and LD, respectively, and FIG. 9B shows 0 persons and 5 persons passing in the directions indicated by the passing direction icons LU and LD, respectively. Shows how a person passed by. Further, each of the captured images SC3 and SC4 is displayed on the display unit 21 of the PC600.

In FIG. 9A, the camera device 100 detects the whole body of a plurality of persons reflected in the captured image SC3. On the captured image SC3, each of the person detection frames OB13, OB14, OB15, OB16, OB17, and OB18, the predetermined lines L3, the passing direction icons LU and LD, and the count values C11 and C21 are displayed. Is displayed. Each of the person detection frames OB13 to OB18 indicates a detection range for each of the whole body of a plurality of persons. The passing direction icon LU indicates that the predetermined line L3 has been passed from the front to the back. The passing direction icon LD indicates that the predetermined line L3 has been passed from the back to the front. Each of the count values C11 and C21 indicates the number of persons who have passed the predetermined line L3 in the directions corresponding to the passing direction icons LU and LD, respectively.

In FIG. 9B, the camera device 100 detects a part of a plurality of people reflected in the captured image SC4. On the captured image SC4, each of the person detection frames OB21, OB22, and OB26 indicating the detection range of the upper body of the person and the person detection frames OB31, OB32, OB33, OB34, OB35 indicating the detection range on the scapula of the person, Each of OB36 and OB37, each of the person detection frames OB41, OB42, OB43, OB44, OB45, OB46, OB47 indicating the detection range of the person's head, each of the predetermined lines L4, and the passing direction icons LU and LD, respectively. And each of the count values C12 and C22 are displayed. The passing direction icon LU indicates that the predetermined line L4 has been passed from the front to the back. The passing direction icon LD indicates that the predetermined line L4 has been passed from the back to the front. Each of the count values C12 and C22 indicates the number of persons who have passed the predetermined line L4 in the directions corresponding to the passing direction icons LU and LD, respectively. In FIG. 9B, each of the person detection frames OB41 to OB47 showing the detection range of the person's head and a part of the predetermined line L4 are shown by broken lines, but these are broken lines for the purpose of making the drawing easier to see. It has been changed to.

FIG. 10 is a diagram showing an example of an operation procedure of the person detection system 1000 using the camera device 100 according to the first embodiment. In FIG. 10, unlike the processing of the flowcharts shown in FIGS. 7 and 8 described above, a more suitable tracking object can be automatically set.

The camera device 100 images the imaging region by the imaging unit 11 (St30). The camera device 100 detects a plurality of tracking objects for each of the plurality of persons reflected in the captured captured image by the first detection unit 12a to the fourth detection unit 12d (St31).

The camera device 100 includes the number of tracking objects indicating the whole body of the person detected by the first detection unit 12a and the second detection units 12b to the fourth detection unit 12d among the plurality of detected tracking objects. The number and the number of tracking objects indicating a part of the detected person are acquired (St32).

The camera device 100 compares the number of the acquired tracking objects indicating the whole body of the person, the number of the tracking objects indicating a part of the person, and a predetermined threshold value used for the congestion determination. Congestion determination is performed (St33). The predetermined threshold value is arbitrarily set by the user, and may be a value indicating a ratio or a value indicating the number of people.

The camera device 100 sets a plurality of tracking object candidates presumed to have been detected from one person among the plurality of tracking objects detected in step St32 in association with the one person. (St34). The camera device 100 associates each of the plurality of tracking objects with each other based on the estimated foot coordinates for each of the tracking objects.

The camera device 100 determines the object having the largest amount of features among the plurality of tracking objects detected for each person as the tracking object. The object for tracking is preferably an object that is detected with a larger feature amount or a larger area, but even if the object is specified by the user, the priority of the object set by the user (for example, the upper body having a large area). , On the scapula, then on the head).

Further, the tracking object may be determined based on the installation angle of the camera device 100. For example, the tracking object is determined to detect the upper body of the person because it is easy to detect the upper body of the person when the installation angle of the camera device 100 is 30 °. Further, when the installation angle of the camera device 100 is 45 °, it is more difficult for the tracking object to detect the upper body of the person than when the installation angle of the camera device 100 is 30 °. Since the detection area (in other words, the size of the frame of the person detection frame) when the head is detected becomes large, it may be determined to detect the head of the person.

The camera device 100 updates the data of the tracking object determined for each person (St36). The camera device 100 may determine a plurality of objects as tracking objects when there is a possibility that the detection accuracy may be lowered, such as when the determined tracking object contains a small amount of features. .. As a result, the camera device 100 does not have to frequently update the data of the tracking object, and the detection accuracy can be ensured.

The camera device 100 estimates the foot coordinates of each of the determined tracking objects using the set foot coordinate data, and creates the flow line information in which the tracking object is tracked by the tracking unit 13. To do. The camera device 100 determines whether or not the tracking object has passed the line set by the user based on the flow line information created by the line passage determination unit 14, and if so, the passage information (for example,). , Direction and time of passage, etc.) are detected (St37).

The camera device 100 counts the number of people corresponding to the number of tracking objects that have passed through the line based on the detected passage information by the number counting unit 15 (St38).

The camera device 100 creates count result data based on the count result of the number of people, transmits it to the PC 600 or the server device 300, and displays it (St39). After creating the count result data, the camera device 100 returns to the process of step St30.

FIG. 11 is a diagram showing an example of a captured image captured by the camera device 100. FIG. 11 is a captured image SC5 in which only the whole body of a person is detected, and shows a state in which four and five people pass in the directions indicated by the passing direction icons LU and LD, respectively. Further, the captured image SC5 is displayed on the display unit 21 of the PC600.

In FIG. 11, the camera device 100 detects the whole body of a plurality of persons reflected in the captured image SC5. A predetermined line L5, each of the passing direction icons LU and LD, and each of the count values C13 and C23 are displayed on the captured image SC5. The passing direction icon LU indicates that the predetermined line L5 has been passed from the front to the back. The passing direction icon LD indicates that the predetermined line L5 has been passed from the back to the front. Each of the count values C13 and C23 indicates the number of persons who have passed the predetermined line L5 in the directions corresponding to the passing direction icons LU and LD, respectively. As described above, the camera device 100 may set the person detection frame only on the whole body of the person based on the installation angle of the camera device 100 and the setting of the user, or a part of the person as described with reference to FIG. May be detected altogether. Further, the camera device 100 may display the flow line information of each person on the captured image SC5.

FIG. 12 shows an example of the count result data, and is a diagram showing an example of the transition of the number of people entering and leaving the store by time zone of the facility A by floor. The count result data is transmitted to the server device 300 or the PC 600 and displayed on the display unit 21 of the monitor 450 or the PC 600.

The horizontal axis of FIG. 12 indicates a time zone (for example, a time zone of every hour), and the vertical axis of FIG. 12 passes through a predetermined line detected by each of the camera devices 100, 100A, 100B, 100C, ... Shows the number of counts for each time zone of the person who did. The server device 300 or the PC 600 uses the count result data received from each of the camera devices 100, 100A, 100B, 100C, ... To graph the transition of the number of people by time zone, and the display unit 21 of the monitor 450 or the PC 600. Display on.

In FIG. 12, for example, the number of people entering the store (total), the number of people entering the store (total of the previous year), the number of people leaving the store (total), and the number of people leaving the store from 10:00, which is the opening time, to 21:00, which is the closing time, are shown. Number of people (total of the previous year), number of people entering the store (B2F: 2nd basement floor), number of people leaving the store (B2F: 2nd basement floor), number of people entering the store (B1F: 1st basement floor), number of people leaving the store (B1F: basement floor) 1st floor), the number of people entering the store (1F: 1st floor), the number of people leaving the store (1F: 1st floor), the number of people entering the store (2F: 2nd floor), and the number of people leaving the store (2F: 2nd floor) The transition of the count result for each hour is shown.

As a result, the user (for example, an employee of the head office or the facility) can visualize the movement of the person in each facility by a numerical value or a graph, which can be utilized for marketing analysis work. In addition, when the camera device images the inside of the store in the facility, the user can grasp the flow of customers, so it is used as reference information for examining the layout of each store in the facility. can do.

As described above, in the camera devices 100, 100A, 100B, 100C, ... According to the first embodiment, the image pickup unit 11 that images the image pickup area and the person reflected in the image pickup area imaged by the image pickup unit 11 are a part of the person. Alternatively, it is provided with a detection unit 12 that detects by each of a plurality of detection objects targeting the whole body and determines congestion in the imaging region according to the comparison between the number of detected detection objects and a predetermined threshold value. There is. When the result of the congestion determination is non-congestion, the detection unit 12 detects the person by using the first detection object for detecting the person from the whole body of the person, and the result of the congestion determination is the congestion. In that case, the person is detected by using a second detection object for detecting the person from a part of the person.

As a result, the camera devices 100, 100A, 100B, 100C, ... Are security camera devices used in facilities such as convenience stores and department stores, and have a fixed angle of view compared to imaging from directly above the imaging area. Even if it is installed so that the lower side diagonally forward can be imaged, the person reflected in the imaging area can be detected by a part of the person and the whole body at the time of congestion, so that the person can be detected with higher accuracy. Can be done.

Further, the camera devices 100, 100A, 100B, 100C, ... Further include a tracking unit 13 that tracks the movement of the person and creates the flow line information of the person. The tracking unit 13 sets a first detection object or a second detection object used based on the congestion determination as a tracking object for tracking a person and tracks the person. As a result, the camera devices 100, 100A, 100B, 100C, ... Can automatically set the object to be used for tracking according to the congestion state in the imaging region, and can track the movement of the person. Further, since the camera devices 100, 100A, 100B, 100C, ... Can obtain the flow line information of the person by tracking, information on the movement direction indicating from which direction the person has passed (moved) can be obtained. Obtainable.

Further, the camera devices 100, 100A, 100B, 100C, ... Are foot coordinate calculation data for calculating the foot coordinates of a person reflected in the imaging region imaged at the installation angle of the camera devices 100, 100A, 100B, 100C, ... The acquisition unit, which acquires the data, is further provided. As a result, the camera devices 100, 100A, 100B, 100C, ... Can calculate the foot coordinates of the person indicated by the tracking object even when only a part of the person is detected at the time of congestion. Further, the camera devices 100, 100A, 100B, 100C, ... Can track the movement of a person based on the calculated foot coordinates. The acquisition unit is an example of the reception unit 110 for receiving the foot coordinate calculation data transmitted from the PC 600.

Further, the camera devices 100, 100A, 100B, 100C, ... Calculate the installation angle at the time of installation of the camera devices 100, 100A, 100B, 100C, ... And the foot coordinates of the person reflected in the imaging area imaged at the installation angle. It is further provided with a setting unit for setting foot coordinate calculation data for the purpose. As a result, the camera devices 100, 100A, 100B, 100C, ... Can calculate the foot coordinates of the person indicated by the tracking object even when only a part of the person is detected at the time of congestion. Further, the camera devices 100, 100A, 100B, 100C, ... Can track the movement of a person based on the calculated foot coordinates. The setting unit is an example of the memory 19 that stores the foot coordinate calculation data.

Further, the camera devices 100, 100A, 100B, 100C, ... Are line passage determination units that determine whether or not a person has passed a predetermined line based on the foot coordinate calculation data and the second detection object. 14, further provided. As a result, the camera devices 100, 100A, 100B, 100C, ... Calculate the foot coordinates where the person is located even if the imaging area is congested and the detection object indicates a part of the person. Therefore, it is possible to determine with higher accuracy whether or not a predetermined line has been passed.

Further, the camera devices 100, 100A, 100B, 100C, ... Further include a number counting unit 15 for measuring the number of persons who have passed a predetermined line by the line passing determination unit 14. The number of people counting unit 15 creates a measurement result based on the number of people who have passed a predetermined line. As a result, the camera devices 100, 100A, 100B, 100C, ... Can count the number of people who have passed a predetermined line.

Further, the camera devices 100, 100A, 100B, 100C, ... Further include a number counting unit 15 for measuring the number of persons who have passed a predetermined line by the line passing determination unit 14. The number of people counting unit 15 creates a measurement result of measuring the number of people in each passing direction in which a person passes a predetermined line based on the flow line information. As a result, the camera devices 100, 100A, 100B, 100C, ... Can count the number of persons who have passed the predetermined line in each passing direction.

Further, the camera devices 100, 100A, 100B, 100C, ... Further include a transmission unit 120 for transmitting to the server device 300 or the PC 600 as an example of an external terminal for displaying the measurement result, as an example of the measurement result. The count result data of is transmitted to the server device 300 or the PC 600 communicably connected to the camera devices 100, 100A, 100B, 100C, .... As a result, the user can display the measurement results of the camera devices 100, 100A, 100B, 100C, ... Installed in the facility A on the display unit 21 of the monitor 450 or the PC 600 connected to the server device 300. ..

Further, the predetermined line is set based on the input of the server device 300 or the PC 600 as an example of the external terminal. As a result, the user can input the input operation for the setting related to the predetermined line by the input device 400 of the server device 300 or the input unit 22 of the PC 600, so that the predetermined line can be freely set.

Further, the predetermined threshold value is set based on the input of the server device 300 or the PC 600 as an example of the external terminal. As a result, the user can input an input operation for setting a predetermined threshold value by the input device 400 of the server device 300 or the input unit 22 of the PC 600, so that the predetermined threshold value can be freely set.

The second detection object is the person's upper body, scapula, or head. As a result, the camera devices 100, 100A, 100B, 100C, ... Can detect a part of the person as a detection object even when the imaging area is congested, so that a plurality of people are not separated. Even when it is reflected in the camera, it is possible to detect a plurality of persons with higher accuracy.

Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modification examples, modification examples, replacement examples, addition examples, deletion examples, and equal examples within the scope of claims. It is understood that it belongs to the technical scope of the present disclosure. In addition, each component in the various embodiments described above may be arbitrarily combined as long as the gist of the invention is not deviated.

The present disclosure is useful as a camera device, a person detection method, and a person detection system capable of detecting a person reflected in a crowded imaging area with high accuracy.

11 Imaging unit 12 Detection unit 12a 1st detection unit 12b 2nd detection unit 12c 3rd detection unit 12d 4th detection unit 13 Tracking unit 14 Line passage determination unit 15 Number of people counting unit 100 Camera device 300 Server device 400 Input device 450 Monitor 500A , 500B, 500C Number of people counting system 600 PC

Claims (13)

An imaging unit that captures the imaging area and
A person reflected in the imaging region imaged by the imaging unit is detected by each of a plurality of detection objects targeting a part or the whole body of the person, and the number of detected detection objects and a predetermined threshold value are determined. A detection unit that determines congestion in the imaging region according to comparison is provided.
The detection unit
When the result of the congestion determination is non-congestion, the person is detected by using the first detection object for detecting the person from the whole body of the person, and when the result of the congestion determination is congestion. Detects the person using a second detection object for detecting the person from a part of the person.
Camera device. A tracking unit that tracks the movement of the person and creates the flow line information of the person is further provided.
The tracking unit sets and tracks the first detection object or the second detection object used based on the congestion determination as a tracking object for tracking the person.
The camera device according to claim 1. Further, an acquisition unit for acquiring foot coordinate calculation data for calculating the foot coordinates of the person reflected in the imaging region imaged at the installation angle of the camera device is provided.
The camera device according to claim 2. The camera device is further provided with an installation angle at the time of installation and a setting unit for setting foot coordinate calculation data for calculating the foot coordinates of the person reflected in the imaging region imaged at the installation angle.
The camera device according to claim 2. A line passage determination unit for determining whether or not the person has passed the predetermined line based on the foot coordinate calculation data and the second detection object is further provided.
The camera device according to claim 4. A number counting unit for measuring the number of persons who have passed the predetermined line by the line passing determination unit is further provided.
The number of people counting unit creates a measurement result based on the number of people who have passed the predetermined line.
The camera device according to claim 5. A number counting unit for measuring the number of persons who have passed the predetermined line by the line passing determination unit is further provided.
The number counting unit creates a measurement result of measuring the number of people in each passing direction in which the person passes through the predetermined line based on the flow line information.
The camera device according to claim 5. Further provided with a transmitter for transmitting the measurement result to an external terminal.
The measurement result is transmitted to an external terminal communicably connected to the camera device.
The camera device according to claim 6 or 7. The predetermined line is set based on the input of the external terminal.
The camera device according to claim 8. The predetermined threshold value is set based on the input of the external terminal.
The camera device according to claim 8. The second detection object is the person's upper body, scapula or head.
The camera device according to any one of claims 1 to 10. It is a person detection method that detects a person reflected in the imaging area.
The person is detected by each of a plurality of detection objects targeting a part or the whole body of the person.
Congestion of the imaging region is determined according to the comparison between the number of detection objects detected from the imaging region and a predetermined threshold value.
When the result of the congestion determination is non-congestion, the person is detected by using the first detection object that detects the person from the whole body of the person, and when the result of the congestion determination is congestion, the person is detected. The person is detected by using a second detection object that detects the person from a part of the person.
Person detection method. A person detection system that includes a camera device that captures an imaging area and a display device.
The camera device is
A person is detected by each of a plurality of detection objects targeting a part or the whole body of the person.
Congestion of the imaging region is determined according to the comparison between the number of detection objects detected from the imaging region and a predetermined threshold value.
When the result of the congestion determination is non-congestion, the person is detected by using the first detection object that detects the person from the whole body of the person, and when the result of the congestion determination is congestion, the person is detected. The person is detected by using a second detection object that detects the person from a part of the person.
Person detection system.

Images