WO2016189785A1

WO2016189785A1 - Traffic line analysis system, camera device, and traffic line analysis method

Info

Publication number: WO2016189785A1
Application number: PCT/JP2016/001684
Authority: WO
Inventors: 高橋　秀明; 哲生田山; 隆恵小口; 晋平萩巣
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2015-05-22
Filing date: 2016-03-23
Publication date: 2016-12-01
Also published as: JP5909708B1; JP2016218856A

Abstract

Provided are a traffic line analysis system, a camera device, and a traffic line analysis method that are capable of ascertaining the congestion location and passage location of just the customers in a store where both customers and employees are present. A camera device (100) generates a background image from a captured image capturing a prescribed imaging region, extracts traffic line information pertaining to a congestion location or passage location in the imaging region of moving bodies including employees that are included in the captured image of the prescribed region, and on the basis of employee location information received by a BLE receiving terminal (160), acquires traffic line information pertaining to the congestion location or passage location in the imaging region of the employees included in the captured image, and transmits said information to a server device (300). The server device (300) receives the employee traffic line information, the traffic line information for the moving bodies including employees, and the background image from the camera device (100), and generates a traffic line analysis image by superimposing traffic line information, which is obtained by removing the employee traffic line information from the traffic line information for the moving bodies including employees, over the background image.

Description

Flow line analysis system, camera device, and flow line analysis method

The present disclosure relates to a flow line analysis system, a camera apparatus, and a flow line analysis method for generating a flow line analysis image in which a person's staying information or passage information is superimposed on an image obtained by imaging with a camera device.

For example, Patent Document 1 is known as a prior art for displaying an activity level of a person for each hour at a shooting site where a camera device is installed as a heat map image.

In Patent Literature 1, an activity level is calculated by analyzing a flow line of a person at a shooting site where a security camera connected via a network is installed, and a sensor detection result is superimposed on a floor plan of the shooting site. It is disclosed to generate a heat map image and display the heat map image on a browser screen corresponding to the security camera. Thereby, by browsing the heat map image displayed on the screen of the browser, it becomes possible to grasp the activity level of the person at the shooting site.

In addition, unlike the floor layout shown in Patent Document 1, a technique for generating and displaying a heat map image in which a person's flow line density and the number of people detection result are superimposed on an image captured by a camera device has also been proposed. (For example, refer nonpatent literature 1).

Here, when the detection result of the sensor is superimposed on the floor layout in Patent Document 1, it is necessary that the floor layout and the image at the shooting site of the security camera accurately match (match). In Reference 1, the floor layout is unchanged and unchanged, so the image and the floor layout are matched only when the layout of the shooting site is not changed from the time the floor layout that is the basis of the heat map image is created. To do.

JP 2009-134688 A

By the way, if you use the above-mentioned flow line analysis system in a store and try to grasp the staying position and passing position of the visitor in the store, not only the visitor but also the employee will be handled. There arises a problem that only the position and the passing position cannot be grasped.

In order to solve the above-described conventional problems, the present disclosure provides a flow line analysis system, a camera device, and a motion apparatus capable of grasping only a staying position and a passing position of a visitor in a store where visitors and employees are mixed. An object is to provide a line analysis method.

The present disclosure is a flow line analysis system including a position information transmission device, a position information reception device, a camera device, and a server device, the position information transmission device being possessed by an employee and the position of the employee The position information receiving device receives the position information of the employee transmitted from the position information transmitting device, the camera device picks up the imaging region, and repeatedly generates a background image of the captured image of the imaging region. Extracting the flow line information regarding the staying position or passing position in the imaging area of the moving object including the employee included in the captured image, and based on the output of the position information receiving device, the imaging area of the employee included in the captured image The flow line information related to the staying position or the passing position is acquired, and the generated background image, the extracted movement line information of the moving body, and the acquired flow line information of the employee are obtained every predetermined transmission cycle. server The server device generates a flow line analysis image in which the flow line information obtained by subtracting the flow line information of the employee from the flow line information of the moving body is superimposed on the background image of the captured image, and is generated. This is a flow line analysis system for displaying a generated flow line analysis image on a display unit.

In addition, the present disclosure includes a position information transmitting apparatus that is possessed by an employee and transmits the position information of the employee, a position information receiving apparatus that receives the position information of the employee transmitted from the position information transmitting apparatus, and a server apparatus And a camera device used in a flow line analysis system including: an imaging unit that captures an imaging region; and a background image of the captured image of the imaging region that is repeatedly generated and included in the captured image, including the employee Extracts flow line information related to the staying position or passing position in the body imaging area, and acquires flow line information related to the staying position or passing position in the imaging area of the employee included in the captured image based on the output of the position information receiving device. Then, for each predetermined transmission cycle, using the generated background image, the extracted flow line information of the moving body, and the acquired flow line information of the employee, the movement line information of the moving body is used as an employee. of A processor that generates a flow line analysis image in which the flow line information obtained by subtracting the line information is superimposed on the background image of the captured image, and a flow line analysis of the generated flow line analysis image for each predetermined transmission period A transmission unit that transmits the image to a server device that displays an image.

In addition, the present disclosure is a flow line analysis method in a flow line analysis system including a position information transmission device, a position information reception device, a camera device, and a server device, wherein the position information transmission device is The position information receiving apparatus receives the position information of the employee transmitted from the position information transmitting apparatus, images the imaging area in the camera apparatus, and takes a captured image of the imaging area. The background image is repeatedly generated, and the flow line information regarding the staying position or passing position in the imaging region of the moving object including the employee included in the captured image is extracted, and included in the captured image based on the output of the position information receiving device The flow line information related to the staying position or the passing position in the imaging region of the employee is acquired, and the generated background image and the extracted moving line information of the moving body are acquired every predetermined transmission cycle. To the server device, and in the server device, the flow line information obtained by subtracting the flow line information of the employee from the flow line information of the moving object is used as a background image of the captured image. This is a flow line analysis method for generating a superimposed flow line analysis image and displaying the generated flow line analysis image on a display unit.

According to the present disclosure, after obtaining the flow line information of the moving body other than the employee by subtracting the flow line information of the employee obtained by the position information receiving device from the flow line information of the moving body included in the captured image. Since the flow line analysis image in which the flow line information is superimposed on the background image is generated and displayed, only the staying position or the passing position in the imaging region of the moving body other than the employee can be grasped. For example, when used in a store where employees and visitors are mixed, only the staying position or passing position of the visitors can be grasped.

FIG. 1 is a block diagram showing a schematic configuration of a sales management system according to the present embodiment. FIG. 2 is a diagram illustrating an outline of a plurality of camera devices, a BLE receiving terminal, and a BLE transmitting terminal of the sales management system according to the present embodiment. FIG. 3 is a block diagram illustrating in detail the functional internal configuration of each of the camera device and the server device of the sales management system according to the present embodiment. FIG. 4 is a time chart for explaining the operation timing of each process of image input, background image generation, and flow line information analysis of the camera device of the sales management system according to the present embodiment. FIG. 5A is a diagram illustrating an example of a captured image input to the image input unit of the camera device of the sales management system according to the present embodiment. FIG. 5B is a diagram illustrating an example of the background image generated by the background image generation unit of the camera device of the sales management system according to the present embodiment. FIG. 6 is a flowchart for explaining a flow line analysis image generation process in the server device of the sales management system according to the present embodiment. FIG. 7 is a schematic diagram illustrating a flow line analysis image generation process in the server device of the sales management system according to the present embodiment. FIG. 8 is a block diagram showing in detail a functional internal configuration of a camera apparatus according to a modification of the present embodiment.

Hereinafter, embodiments of the flow line analysis system, the camera device, and the flow line analysis method according to the present disclosure (hereinafter referred to as “the present embodiment”) will be described with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of a sales management system 1000 according to the present embodiment. In the figure, a sales management system 1000 according to the present embodiment includes a flow

line analysis system

500A, 500B, 500C,... Installed in a plurality of stores A, B, C,. A smartphone 700, a cloud computer 800, and a setting terminal device 900 are included. Flow

line analysis systems

500A, 500B, 500C,... Installed in a plurality of stores A, B, C,... Mutually communicate with server device 600, smartphone 700, cloud computer 800, and setting terminal device 900 via network NW. It is connected.

The network NW is a wireless network or a wired network. The wireless network is, for example, a wireless LAN (Local Area Network), a wireless WAN (Wide Area Network), 3G, LTE (Long Term Evolution), or WiGig (Wireless Gigabit). The wired network is, for example, an intranet or the Internet.

A flow line analysis system 500A installed in the store A includes a plurality of

camera devices

100, 100A,..., 100N installed on the floor 1, a recorder 200, a server device 300, an input device 400, and a monitor 450. Have.

Camera devices

100, 100A,..., 100N, recorder 200, and server device 300 installed on floor 1 are connected to network NW. A plurality of

camera devices

100, 100A,..., 100N are also installed on the floor 2 as in the case of the floor 1, and illustration of the camera devices in the floor 2 is omitted. The internal configuration of each of the

camera devices

100, 100A,... 100N is the same, and details thereof will be described later with reference to FIG.

The flow line analysis system 500A installed in the store A further includes a plurality of (1-n, n is an integer) BLE transmission terminals 150 possessed by all employees of the store A (all store-related persons including the store manager). And a plurality of (1-m, m is an integer) BLE receiving terminals 160 installed in the store A. A plurality of BLE receiving terminals 160 are installed for each floor. The BLE transmission terminal 150 and the BLE reception terminal 160 have a communication function using BLE (Bluetooth (registered trademark) Low Energy), which is one of the extended uses of the short-range wireless communication technology Bluetooth (registered trademark). The BLE transmission terminal 150 transmits the position information of the employee who possesses this, and the BLE reception terminal 160 receives the position information transmitted from the BLE transmission terminal 150. The number of BLE receiving terminals 160 installed on each floor is determined in advance, but the number of BLE transmitting terminals 150 varies depending on the number of employees.

All the BLE receiving terminals 160 installed in the store A are connected to the network NW. In other stores B, C,..., All employees have a BLE transmission terminal 150, and a plurality of BLE reception terminals 160 are installed for each floor. FIG. 2 is a diagram showing an overview of the

camera devices

100, 100A,... And the BLE receiving terminal 160 installed on the floor 1 and the BLE transmitting terminal 150 possessed by the employee 151 on the floor 1, for example. As shown in the figure, each of a plurality of

camera devices

100, 100A,... And a plurality of BLE receiving terminals 160 are connected to a network NW. Each

camera device

100, 100 </ b> A,... Photographs the employee 151 and the visitor 152 on the floor 1. Each BLE transmission terminal 150 is possessed by the employee 151 on the floor 1 and transmits the position information of the employee 151.

Returning to FIG. 1, the recorder 200 is configured using, for example, a semiconductor memory or a hard disk device, and images (hereinafter referred to as camera devices) obtained by imaging the

camera devices

100, 100A,..., 100N installed in the store A. The image obtained by imaging is stored as “captured image”). The captured image data stored in the recorder 200 is used for monitoring work such as crime prevention. When the captured image obtained by the imaging of the camera device is stored in the recorder 200, the date and time and the identification numbers assigned to the

camera devices

100, 100A,.

The server device 300 is configured using, for example, a PC (Personal Computer) and operates a user who operates the input device 400 (for example, a user of a flow line analysis system and refers to an employee including a store manager of the store A, and so on). In response to the input operation, the camera device 100 is notified that a predetermined event (for example, a change in the layout of the sales floor on the floor 1 of the store A) has occurred. Although details will be described later, the server device 300 uses the data transmitted from the

camera devices

100, 100A,..., 100N installed in the store A, and the staying position of the visitor 152 in the store A Alternatively, a flow line analysis image (heat map image) in which the flow line information regarding the passing position is superimposed on the background image of the captured image is generated and displayed on the monitor 450. Details of the internal configuration of the server apparatus 300 will be described later with reference to FIG.

The input device 400 is configured using, for example, a mouse, a keyboard, a touch panel, or a touch pad, and outputs a signal corresponding to a user input operation to the camera device 100 or the server device 300. In FIG. 1, to simplify the drawing, an arrow is shown only between the input device 400 and the camera apparatus 100, but the input device 400 and other camera apparatuses (for example, the camera apparatuses 100 </ b> A and 100 </ b> N). An arrow may be shown between the two. The monitor 450 is configured using, for example, an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence-ence), and displays data of a flow line analysis image generated by the server device 300. The monitor 450 is provided as an external device different from the server device 300, but may be configured to be included in the server device 300.

The server device 600 of the operation headquarters is a flow line analysis system 500A installed in each store A, B, C,... According to an input operation of an employee (for example, an officer) of the operation headquarters who operates the server device 600. , 500B, 500C,... Browsing apparatus for acquiring and displaying the flow line analysis image generated. The smartphone 700 is a flow

line analysis system

500A, 500B, installed in each store A, B, C,... According to an input operation of an employee (for example, a sales representative) of the operation headquarters who operates the smartphone 700. It is a browsing apparatus for acquiring and displaying the flow line analysis image generated in 500C,. The cloud computer 800 is an online storage that stores data of flow line analysis images generated in the flow

line analysis systems

500A, 500B, 500C,... Installed in the stores A, B, C,.

The setting terminal device 900 is configured by using a PC, for example, and is dedicated for displaying a setting screen of the camera device of the flow

line analysis systems

500A, 500B, 500C,... Installed in each store A, B, C,. Browser software can be executed. The setting terminal device 900 displays the setting screen of the camera device (for example, CGI (Common Gateway Interface)) according to the input operation of the employee of the operation headquarters operating the device (for example, the system administrator of the sales management system 1000). The setting information of the camera device is edited (modified, added, deleted) and set.

(Camera device)
FIG. 3 is a block diagram illustrating in detail the functional internal configurations of the camera device 100 and the server device 300 of the present embodiment. In the sales management system 1000 shown in FIG. 1, the camera devices installed in the stores A, B, C,... Have the same configuration.

In FIG. 3, the camera device 100 includes an imaging unit 10, an image input unit 20, a background image generation unit 30, a flow line information analysis unit 40, a schedule management unit 50, a transmission unit 60, and an event information reception unit. 70, a background image storage unit 80, a passage / staying analysis information storage unit 90, an employee information acquisition unit 170, and an employee passage / staying analysis information storage unit 171.

The imaging unit 10 has at least a lens and an image sensor (not shown). The lens collects light (light rays) incident from the outside of the camera device 100 and forms an image on a predetermined imaging surface of the image sensor. As the lens, a fish-eye lens or a wide-angle lens capable of obtaining an angle of view of, for example, 140 degrees or more is used. The image sensor is, for example, a CCD (Charged-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) solid-state imaging device, and converts an optical image formed on the imaging surface into an electrical signal.

The image input unit 20 is configured using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor), and performs predetermined signal processing using an electrical signal from the imaging unit 10. The image data (frame) defined by RGB (Red Green Blue) or YUV (luminance / color difference) that can be recognized by humans is generated and output to the background image generation unit 30 and the flow line information analysis unit 40 To do.

The background image generation unit 30 includes an input image learning unit 31, a moving body separation unit 32, and a background image extraction unit 33, and repeatedly generates a background image of the captured image in the imaging region. The background image generation unit 30 is configured by using, for example, a CPU, MPU, or DSP, and each captured image data (frame) output from the image input unit 20 at a predetermined frame rate (for example, 30 fps (frame per second)). In addition, a background image from which the moving body included in the captured image is excluded is generated and stored in the background image storage unit 80. The background image generation processing in the background image generation unit 30 can use, for example, the method disclosed in the following reference patent document, but is not limited to the method disclosed in this reference patent document.

(Reference Patent Literature) JP 2012-203680 A Summary of the operation of the background image generation unit 30 will be briefly described with reference to FIGS. 4 and 5A and 5B. FIG. 4 is an explanatory diagram of an outline of the operation of the background image generation unit 30 of the camera device 100 of the present embodiment. FIG. 5A is a diagram illustrating an example of a captured image input to the image input unit 20. FIG. 5B is a diagram illustrating an example of the background image generated by the background image generation unit 30.

In FIG. 4, the generation results of the input image learning unit 31, the moving body separation unit 32, and the background image extraction unit 33 are schematically shown from the left side to the right side of the page orthogonal to the time axis from the upper side to the lower side of the page. It is shown that a store visitor carries one cardboard out of four cardboards containing beverages.

The input image learning unit 31 displays the distribution state of the pixel luminance and color difference values for each pixel in a plurality of captured image frames output from the image input unit 20 (for example, the respective frames FM1 to FM5 shown in FIG. 4). analyse. The moving body separation unit 32 uses the learning result of the input image learning unit 31 (that is, the analysis result of the luminance and color difference distribution state for each same pixel between a plurality of frames (for example, the time axis direction shown in FIG. 4)). In each of the frames FM1 to FM5 of the captured image, the information is separated into information on the moving body (for example, refer to frames FM1a to FM5a) and information other than the moving body (for example, background) (for example, refer to frames FM1b to FM5b). In the frames FM3 and FM4 of the picked-up images showing how a person who is a moving object carries a cardboard, the luminance and color difference values corresponding to the pixels of the cardboard carried by the person are along the time axis direction (for example, see FIG. 4). Since it changes, the moving body separating unit 32 regards the cardboard carried by the person as the moving body.

The background image extraction unit 33 uses the frames FM1b to FM5b in which information other than the moving body is reflected among the information separated by the moving body separation unit 32, and the background images of the frames FM1 to FM5 of the captured image output from the image input unit 20. The frames FM1c to FM5c are extracted and stored in the background image storage unit 80.

In the frame FM10a of the captured image shown in FIG. 5A, for example, a person who provides food in a cafeteria and a person who receives food in a tray are shown as moving bodies. In contrast to the frame FM10a of the captured image shown in FIG. 5A, in the background image frame FM10c generated by the background image generation unit 30 (see FIG. 5B), a person serving food in the same canteen as a moving object People who receive food are excluded so that they do not appear. The above is the outline of the operation of the background image generation unit 30.

3, the flow line information analysis unit 40 includes a target detection unit 41, a flow line information acquisition unit 42, and a passage / staying state analysis unit 43. The flow line information analysis unit 40 is configured using, for example, a CPU, MPU, or DSP, and takes a captured image for each data (frame) of the captured image output from the image input unit 20 at a predetermined frame rate (for example, 10 fps). The flow line information relating to the staying position or passing position of the moving body contained in the moving body is extracted and stored in the passing / staying analysis information accumulating unit 90.

The object detection unit 41 performs predetermined image processing (for example, person detection processing, face detection processing) on the frame of the captured image output from the image input unit 20, so that the moving object included in the frame of the captured image is detected. Detect the presence or absence. When the target detection unit 41 detects a moving body included in the frame of the captured image, the target detection unit 41 outputs information (for example, coordinate information of the frame) regarding the detection area of the moving body to the frame of the captured image to the flow line information acquisition unit 42. To do. In addition, the object detection part 41 outputs the information (for example, predetermined null information) regarding the detection area | region of a moving body to the flow line information acquisition part 42, when the moving body contained in the flame | frame of a captured image is not detected. .

The flow line information acquisition unit 42 is based on the information on the detection area of the moving object output from the target detection unit 41, and the information on the captured image output from the image input unit 20 and the information on the detection area of the past moving object. (For example, captured image information and coordinate information) are used to link information on the current and past detection areas, and pass / stay status as flow line information (for example, the amount of change in the coordinate information of the detection area of the moving object). The data is output to the analysis unit 43.

The passage / staying state analyzing unit 43 is based on the flow line information output from the flow line information acquiring unit 42 for a plurality of captured images, and the flow line information regarding the staying position or the passing position of the moving body in the frame of the captured image. (For example, “target position information”, “flow line information”, and “information about passage status or stay status”) are extracted and generated. Further, the passage / staying state analysis unit 43 uses a flow line analysis image (heat map) generated by the display image generation unit 350 of the server device 300 using the extraction result of the flow line information regarding the staying position or the passing position of the moving body. A visualized image of the color portion of (image) may be generated. The passage / staying state analysis unit 43 uses the flow line information for the frames of the plurality of captured images, so that the position where the moving body stays or the position where it has passed in the frames of the captured image output from the image input unit 20. It is possible to extract and generate the accurate flow line information regarding.

The employee information acquisition unit 170 is configured using, for example, a CPU, MPU, or DSP, and based on the output of the BLE receiving terminal 160, the movement related to the staying position or the passing position in the imaging area of the employee included in the captured image of the imaging area. The line information is detected and stored in the employee pass / stay analysis information storage unit 171. The schedule management unit 50 is configured using, for example, a CPU, MPU, or DSP, and the background image data stored in the background image storage unit 80 and the stay information of the moving object stored in the passage / stay analysis information storage unit 90 Alternatively, a predetermined transmission for periodically transmitting the data of the extraction result of the flow line information related to the passage information and the data of the employee information stored in the employee passage / staying analysis information storage unit 171 to the server device 300. A cycle is instructed to the transmission unit 60. The predetermined transmission period is, for example, 15 minutes, 1 hour, 12 hours, 24 hours, etc., but is not limited to these time intervals.

In response to an instruction from the schedule management unit 50 or the event information receiving unit 70, the transmission unit 60 transmits the background image data stored in the background image storage unit 80 and the movement stored in the passage / staying analysis information storage unit 90. Data on the extraction result of flow line information related to body stay information or passage information and employee information data stored in the employee passage / stay analysis information storage unit 171 are acquired and transmitted to the server device 300.

The event information receiving unit 70 receives (acquires) notification of detection of a predetermined event (for example, a change in the layout of the sales floor of the floor 1 of the store A) from the server device 300 or the input device 400, and notification of detection of the predetermined event The data of the background image stored in the background image storage unit 80 and the data of the result of extracting the flow line information related to the staying information or the passing information of the moving object stored in the passing / staying analysis information storing unit 90 And an instruction to transmit the employee information data stored in the employee passage / staying analysis information storage unit 171 to the server device 300 is output to the transmission unit 60.

The background image storage unit 80 is configured using, for example, a semiconductor memory such as an SD memory card or a hard disk device, and stores data (frames) of the background image generated by the background image generation unit 30. The passage / staying analysis information accumulating unit 90 is configured using, for example, a semiconductor memory or a hard disk device, and the flow line information extraction result (for example, the moving position staying or passing position of the moving object generated by the flow line information analyzing unit 40 (for example, Data of “target position information”, “flow line information”, and “information on passage status or stay status”) are stored. The employee passing / staying analysis information accumulating unit 171 is configured by using, for example, a semiconductor memory or a hard disk device, and the flow line information regarding the staying position or passing position in the employee imaging region acquired by the employee information acquiring unit 170. Store the data.

Note that the camera apparatus 100 illustrated in FIG. 3 may include a scene identification unit SD instead of the event information reception unit 70. The scene identification unit SD as an example of the image change detection unit detects whether there is a change in the captured image output from the image input unit 20 (for example, an event that the layout of the sales floor on the floor 1 of the store A has changed). When the scene identification unit SD detects a change in the captured image, the background image data stored in the background image storage unit 80 and the stay information of the moving object stored in the passage / stay analysis information storage unit 90 or Server apparatus 300 of the data of the extraction result of the flow line information related to the passing information and the data of the flow line information related to the staying position or the passing position in the employee imaging region stored in the employee passing / staying analysis information storage unit 171 A transmission instruction to the transmission unit 60 is output.

Further, the camera apparatus 100 shown in FIG. 3 may further include a person counting unit CT. The person counting unit CT as an example of the moving body detection unit performs predetermined image processing (for example, person detection processing) on the captured image output from the image input unit 20, so that the moving body included in the captured image is detected. Count the number of detections. The person count unit CT outputs information related to the number of detected moving bodies included in the captured image to the transmission unit 60.

(Server device)
3 includes an event information reception unit 310, a notification unit 320, a reception unit 330, a reception information storage unit 340, a display image generation unit 350, and a report generation output unit 360. is there. The event information receiving unit 310 receives, from the input device 400, information indicating that a predetermined event (for example, a change in the layout of the sales floor on the floor 1 of the store A) has occurred for each corresponding camera device (for example, the camera device 100). If so, a notification of detection of a predetermined event is received. The event information receiving unit 310 outputs to the notification unit 320 that a notification of detection of a predetermined event has been received. Note that the information indicating that a predetermined event has occurred includes an identification number of a camera device that captures an image of a location where the predetermined event has occurred.

The notification unit 320 transmits a notification of detection of a predetermined event output from the event information reception unit 310 to a corresponding camera device (for example, the camera device 100). The receiving unit 330 receives the data transmitted from the transmission unit 60 of the camera device 100 (that is, the background image data stored in the background image storage unit 80, the stay of the moving object stored in the passage / stay analysis information storage unit 90). Information or flow line information extraction result data related to passing information, and flow line information data related to staying position or passing position in the employee imaging area stored in the employee passing / staying analysis information storage unit 171) Then, the data is output to the reception information storage unit 340 and the display image generation unit 350.

The reception information storage unit 340 is configured using, for example, a semiconductor memory or a hard disk device, and receives data received by the reception unit 330 (that is, background image data stored in the background image storage unit 80, pass / stay analysis information storage unit) 90, data on the extraction result of the flow line information related to the staying information or passing information of the moving body, and the staying position or passing position in the imaging region of the employee stored in the employee passing / staying analysis information accumulating unit 171 Data of flow line information). The reception information storage unit 340 also stores information that can specify the captured image such as the date and time and the imaging location with respect to the installation position of each

camera device

100, 100A,.

The display image generation unit 350 is configured using, for example, a CPU, MPU, or DSP, and data acquired from the reception unit 330 or the reception information storage unit 340 (that is, background image data stored in the background image storage unit 80, passing through). / Data of the extraction result of the flow line information related to the stay information or passage information of the moving object stored in the stay analysis information storage unit 90, and the imaging region of the employee stored in the employee pass / stay analysis information storage unit 171 The flow line analysis image is generated using the flow line information data regarding the staying position or the passing position in FIG. That is, the display image generation unit 350 takes the flow line information obtained by subtracting the flow line information related to the stay position or the pass position in the imaging region of the employee from the flow line information related to the stay information or the pass information of the moving object. A flow line analysis image superimposed on the background image is generated. The display image generation unit 350 corresponds to an image generation unit and a display control unit.

The flow line analysis image is a background image that is excluded from the captured image captured by the camera device 100 so that a moving employee or a visitor is not reflected, and where the visitor is located in the imaging region corresponding to the captured image. The flow line information that visually indicates where the user has stayed or passed through is an image that is quantitatively visualized like a heat map. Further, the display image generation unit 350 displays the generated flow line analysis image on the monitor 450.

The report generation / output unit 360 is configured using, for example, a CPU, MPU, or DSP, and generates a flow line analysis report when a flow line analysis report generation instruction is input from the input device 400. The report generation / output unit 360 displays the generated flow line analysis report on the monitor 450.

FIG. 6 is a flowchart for explaining a flow line analysis image (heat map image) generation process in the server apparatus 300. FIG. 7 is a schematic diagram showing a flow line analysis image generation process. In FIG. 6, first, the user designates a place and period for acquiring the stay / passage position of the visitor (step S1). For example, floor 1 of store A is designated as the place, and January 1 to January 31 is designated as the period. When the place and period for acquiring the stay / passage position of the visitor are designated, the display image generation unit 350 displays the visitor and employee collected by the camera device (for example, the camera apparatus 100) at the designated place and period. The stay / passing position information [A] of all the detected persons including the received number is acquired from the reception information storage unit 340 (step S2). (A) of FIG. 7 is a figure which shows stay / passing position information [A] of all detected persons including a visitor, an employee, etc.

The display image generation unit 350 acquires the stay / passing position information [A] of all detected persons including visitors, employees, etc., and then collects the employees collected by the BLE receiving terminal 160 in the designated place and period. Only the stay / passing position information [B] of the detected number of persons is acquired from the reception information storage unit 340 (step S3). FIG. 7B is a diagram illustrating the stay / passing position information [B] of the detected number of employees only.

The display image generation unit 350 acquires the stay / passing position information [B] of the detected number of employees only, and then collects the stay / passing position information [A] collected by the camera device 100 at the BLE receiving terminal 160. The stay / passing position information [B] is subtracted (step S4). In other words, the staying / passing position information for only the visitors is obtained by subtracting the number of detected employees from the total number of detected persons. (C) of FIG. 7 is a figure which shows stay / passing position information [C] of the detected number of people only. As described above, the flow line information only for the visitors can be obtained.

The display image generation unit 350 acquires the flow line information of only the visitor, and then superimposes the flow line information on the background image of the captured image (stored in the reception information storage unit 340) to analyze the flow line. Generate an image. Then, the generated flow line analysis image is displayed on the monitor 450.

As described above, in the flow line analysis system 500A of the present embodiment, the BLE transmission terminal 150 is possessed by the employee, the BLE reception terminal 160 is installed in each place in the store, and the employee transmitted from the BLE transmission terminal 150 is transmitted. The position information is received by the BLE receiving terminal 160. The camera device 100 captures an image of a predetermined imaging area, repeatedly generates a background image of the captured image of the imaging area, and stays in the imaging area of the moving object including the employee included in the captured image of the predetermined imaging area. While extracting the flow line information related to the position or the passing position, the flow line information related to the staying position or the passing position in the imaging region of the employee included in the captured image based on the positional information of the employee received by the BLE receiving terminal 160. get. Then, for each predetermined transmission cycle, the generated background image, the flow line information of the moving object including the extracted employee, and the acquired flow line information of the employee are transmitted to the server device 300. The server device 300 receives the background image transmitted from the camera device 100, the flow line information of the moving object including the employee, and the flow line information of the employee, and the received flow line of the moving object including the employee. The flow line information of the employee was subtracted from the information, and the flow line information obtained by this (ie, the flow line information of the visitor) was superimposed on the background image of the captured image to generate a flow line analysis image. A flow line analysis image is displayed on the monitor 450.

This makes it possible to grasp only the staying position or passing position of a visitor in a store where employees and visitors are mixed. Further, since the employee and the visitor can be distinguished, the correlation between the employee's movement and the visitor's movement can be intuitively grasped. In addition, the store visit tendency can be grasped in consideration of the privacy of employees and visitors.

(Modification of this embodiment)
In the present embodiment described above, the flow line analysis image generation process is executed in the server apparatus 300, but the camera apparatus 100 may execute the flow line analysis image generation process. FIG. 8 is a block diagram illustrating in detail a functional internal configuration of a camera device 100S according to a modification of the present embodiment. The camera device 100S illustrated in FIG. 8 includes an imaging unit 10, an image input unit 20, a background image generation unit 30, a flow line information analysis unit 40, a schedule management unit 50, a transmission unit 60S, and an event information reception unit. 70, a background image storage unit 80, a passage / staying analysis information storage unit 90, and a display image generation unit 350S. In the description of each part of the camera device 100S illustrated in FIG. 8, the same components and operations as those of the camera device 100 illustrated in FIG. 3 are denoted by the same reference numerals, description thereof is omitted, and different contents will be described.

The display image generation unit 350S is stored in the background image data stored in the background image storage unit 80 and the passage / staying analysis information storage unit 90 in response to an instruction from the schedule management unit 50 or the event information reception unit 70. Data on the result of extraction of flow line information related to the staying information or passing information of the moving body, and flow line information related to the staying position or passing position in the imaging region of the employee stored in the employee passing / staying analysis information accumulating unit 171 Using this data, a flow line analysis image is generated by superimposing the flow line information obtained by subtracting the flow line information of the employee from the flow line information of the moving body including the employee on the background image of the captured image. To the transmitter 60S. The transmission unit 60S transmits the data of the flow line analysis image generated by the display image generation unit 350S to the server device 300.

As described above, in the modification of the present embodiment, the camera device 100S generates the background image of the captured image in the predetermined imaging area, and the staying position in the imaging area of the moving object including the employee included in the captured image in the imaging area Alternatively, the flow line information related to the passing position is extracted, and the flow line information related to the staying position or the passing position in the imaging region of the employee included in the captured image is acquired based on the output of the BLE receiving terminal 160. A flow line analysis image in which the flow line information obtained by subtracting the acquired employee flow line information from the flow line information is superimposed on the background image is generated.

Thereby, only the staying position or passing position of the visitor can be grasped in the camera device 100S. Further, since the employee and the visitor can be distinguished, the correlation between the employee's movement and the visitor's movement can be intuitively grasped. In addition, the store visit tendency can be grasped in consideration of the privacy of employees and visitors.

Further, since the camera apparatus 100S executes the process up to the generation process of the flow line analysis image and transmits the data of the flow line analysis image as the generation result to the server apparatus 300, for example, in a state where the processing load of the server apparatus 300 is considerably high. Since it is not necessary for the server apparatus 300 to execute the flow line analysis image generation process, an increase in the processing load of the server apparatus 300 can be suppressed.

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are of course within the technical scope of the present disclosure. Is done.

The present disclosure has an effect that it is possible to grasp the stay and passage of only the visitor in a store where the visitor and the employee are mixed, and the person's stay information or passage is added to the background image updated at a predetermined timing. The present invention is useful as a flow line analysis system, a camera device, and a flow line analysis method for generating an accurate flow line analysis image on which information is superimposed.

DESCRIPTION OF SYMBOLS 10 Imaging part 20 Image input part 30 Background image generation part 31 Input image learning part 32 Moving body separation part 33 Background image extraction part 40 Flow line information analysis part 41 Target detection part 42 Flow line information acquisition part 43 Passing / staying state analysis part 50 Schedule Management Unit 60 Transmission Unit 70 Event Information Receiving Unit 80 Background Image Storage Unit 90 Passing / Standing Analysis

Information Storage Unit

100, 100A, 100N, 100S Camera Device 150 BLE Transmission Terminal 151 Employee 152 Visitor 160 BLE Reception Terminal 170 Employee Employee information acquisition unit 171 Employee pass / stay analysis information storage unit 200

Recorder

300, 600 Server device 310 Event information reception unit 320 Notification unit 330 Reception unit 340 Reception information storage unit 350 Display image generation unit 360 Report generation output unit 400 Input Device 450

Monitor

500A, 500B, 500 Trail analysis system 700 smart phone 800 cloud computer 900 sets the terminal device 1000 sales management system CT people counting unit SD scene identification section

Claims

A flow line analysis system including a position information transmitting device, a position information receiving device, a camera device, and a server device,
The position information transmission device is possessed by an employee, transmits the employee's position information,
The location information receiving device receives the employee location information transmitted from the location information transmitting device,
The camera device is
Image the imaging area,
Repeatedly generating a background image of the captured image of the imaging region;
Extracting flow line information relating to a staying position or a passing position in the imaging area of the moving object including the employee included in the captured image,
Based on the output of the position information receiving device, to obtain the flow line information regarding the stay position or the passing position in the imaging area of the employee, which is included in the captured image,
For each predetermined transmission cycle, the generated background image, the extracted flow line information of the moving body, and the acquired flow line information of the employee is transmitted to the server device,
The server device
Generating a flow line analysis image in which the flow line information obtained by subtracting the flow line information of the employee from the flow line information of the moving body is superimposed on a background image of the captured image;
Displaying the generated flow line analysis image on a display unit;
Flow line analysis system.
The flow line analysis system according to claim 1,
The position information transmission device is a transmission terminal that conforms to the short-range wireless communication standard.
Flow line analysis system.
The flow line analysis system according to claim 2,
The position information receiving device is a receiving terminal compliant with the short-range wireless communication standard.
Flow line analysis system.
A position information transmitting apparatus that is possessed by an employee and transmits the position information of the employee; a position information receiving apparatus that receives the position information of the employee transmitted from the position information transmitting apparatus; and a server apparatus. A camera device used in a flow line analysis system including:
An imaging unit for imaging the imaging region;
Repeatedly generating a background image of the captured image of the imaging region;
Extracting flow line information relating to a staying position or a passing position in the imaging area of the moving object including the employee included in the captured image,
Based on the output of the position information receiving device, to obtain the flow line information regarding the stay position or the passing position in the imaging area of the employee, which is included in the captured image,
For each predetermined transmission cycle, using the generated background image, the extracted flow line information of the mobile body, and the acquired flow line information of the employee, the movement of the mobile body. A processor for generating a flow line analysis image in which the flow line information obtained by subtracting the flow line information of the employee from the line information is superimposed on a background image of the captured image;
A transmission unit that transmits the generated flow line analysis image to the server device that displays the flow line analysis image for each predetermined transmission period;
Camera device.
A flow line analysis method in a flow line analysis system including a position information transmission device, a position information reception device, a camera device, and a server device,
The position information transmission device is possessed by an employee, transmits the employee's position information,
The location information receiving device receives the employee location information transmitted from the location information transmitting device,
In the camera device,
Image the imaging area,
Repeatedly generating a background image of the captured image of the imaging region;
Extracting flow line information relating to a staying position or a passing position in the imaging area of the moving object including the employee included in the captured image,
Based on the output of the position information receiving device, to obtain the flow line information regarding the stay position or the passing position in the imaging area of the employee, which is included in the captured image,
For each predetermined transmission cycle, the generated background image, the extracted flow line information of the moving body, and the acquired flow line information of the employee is transmitted to the server device,
In the server device,
Generating a flow line analysis image in which the flow line information obtained by subtracting the flow line information of the employee from the flow line information of the moving body is superimposed on a background image of the captured image;
Displaying the generated flow line analysis image on a display unit;
Flow line analysis method.