JP4802285B2 - Flow line association method, apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To associate traffic lines with respect to the same traveling body from traffic lines recognized by each of a plurality of traffic line recognition systems. <P>SOLUTION: This traffic line association method includes: selecting the traffic line data of a traveling object located in a first image monitoring area from a first traffic line database; selecting camera image data on a point of time when the traveling object of the traffic line data is located in a first image monitoring area from a first image database; extracting a traveling object image from the camera image data; selecting the traffic line data of the traveling object located in a second image monitoring area from a second traffic line database; selecting the camera image data on a point of time when the traveling object of the traffic line data is located in a second image monitoring area from a second image database; extracting the traveling object image from the camera image data; determining whether or not the traveling objects of the respective images are the same by collating the both traveling object images; and associating, when it is determined the traveling objects are the same, the both traffic line data. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a technique for associating flow line data indicating a trajectory of a shopper who moves in a store, for example.

  Examples of a flow line recognition system that recognizes a trajectory of a shopper moving in a store such as a supermarket or a convenience store as a flow line include an image processing method using a camera and a wireless tag tracking method using a wireless tag reader. In the case of an image processing method, with the current image processing technology, it is extremely difficult to track the movement of a single shopper as a single flow line even if the entire area of the monitoring area is captured without blind spots by each camera. Have difficulty. For example, there may be a case where the flow line is interrupted or the flow line is switched when passing by another shopper.

  In Patent Document 1, flow line data for the same customer is selected from a plurality of flow line data generated by an image processing type flow line recognition system, and each is connected to generate one flow line data. A flow line editing system is disclosed.

  By the way, information processing necessary for flow line recognition is performed using a computer. However, since the information processing requires a large load, the number of cameras and wireless tag readers that can be connected to one computer is limited. For this reason, there is a limit to the size of a region where a flow line can be recognized by a single flow line recognition system. Thus, when recognizing a flow line over a vast area, it is conceivable to divide the area into a plurality of areas and construct a flow line recognition system for each. However, in that case, not only will it be very expensive to construct the system, but the subsequent maintenance costs will be enormous.

  In a store, in order to analyze a customer's behavior, it is not always necessary to track the travel route of the customer over the entire store, and it may be sufficient to be able to track the travel route of the same customer at a plurality of specific sales floors. Therefore, there is a need for a technique for associating a flow line for the same moving body from among the flow lines for each system recognized by a flow line recognition system constructed in each of a plurality of non-adjacent flow line recognition areas.

  The present invention has been made based on such circumstances, and the purpose of the present invention is to recognize each flow line even if a plurality of flow line recognition areas in which a flow line recognition system is constructed are not adjacent to each other. A flow line associating method and apparatus capable of associating flow lines for the same moving body from among the flow lines recognized by the system, and enabling a flow line analysis in a plurality of partial areas in a wide area. As well as a program.

  According to the flow line association method of the present invention, for each moving body that moves in the first flow line recognition area, the flow line data that indicates the trajectory of the moving body in the first flow line recognition area is stored. For each moving body that moves in a second flow line recognition area different from the first flow line recognition area, a flow line that indicates a trajectory in the second flow line recognition area of the moving body A second flow line database for storing data, a first for storing camera image data taken in the first image monitoring area, which is at least part of the first flow line recognition area, together with the shooting time And the second image database for storing camera image data taken in the second image monitoring area, which is at least a part of the second flow line recognition area, together with the shooting time. Against The computer capable of processing selects the flow line data of the moving body located in the first image monitoring area from the respective flow line data stored in the first flow line database (the first flow line selection). Step), selecting camera image data from the first image database when the moving object corresponding to the selected flow line data is located in the first image monitoring area (first image selection step); A moving body image is extracted from the selected camera image data (first moving body image extraction stage), and within the second image monitoring area from each flow line data stored in the second flow line database. Is selected (second flow line selection stage), and the camera image data at the time when the moving body corresponding to the flow line data is located in the second image monitoring area is selected as the second flow line data. Selected from the image database (Second image selection stage), a moving body image is extracted from the selected camera image data (second moving body image extraction stage), and the moving body image extracted by the first moving body image extraction stage And the moving body image extracted in the second moving body image extraction stage are compared to determine whether or not the moving bodies of each image are the same (moving body identification stage), and the moving bodies are determined to be the same. Then, by storing at least the identification information of the flow line data selected by the second flow line selection step in correspondence with at least the identification information of the flow line data selected by the first flow line selection step ( A flow line associating step), from the flow line data of each moving body stored in the first flow line database and the flow line data of each moving body stored in the second flow line database; Associate the flow line data It is a thing.

  The flow line recognition apparatus of the present invention can access the first and second flow line databases and the first and second image databases, and from among the flow line data stored in the first flow line database. The first flow line selecting means for selecting the flow line data of the moving body located in the first image monitoring area, and the moving body corresponding to the flow line data selected by the first flow line selecting means are the first First image selection means for selecting camera image data at a time point within one image monitoring area from the first image database, and a moving body image from the camera image data selected by the first image selection means. First moving body image extracting means for extracting and second moving line data for selecting a moving body located in the second image monitoring area from the respective flow line data stored in the second flow line database. The flow line selection means and this Second image selection means for selecting, from the second image database, camera image data at a time when the moving object corresponding to the flow line data selected by the second flow line selection means is located in the second image monitoring area; The second moving body image extracting means for extracting the moving body image from the camera image data selected by the second image selecting means, the moving body image extracted by the first moving body image extracting means, and the second The moving body identifying means for comparing the moving body images extracted by the moving body image extracting means to determine whether or not the moving bodies of the respective images are the same, and the moving body is identical by the moving body identifying means When the determination is made, the movement for storing at least the identification information of the flow line data selected by the second flow line selection means in correspondence with at least the identification information of the flow line data selected by the first flow line selection means. Line related Only those with a unit.

  The flow line recognition program according to the present invention stores the flow line data stored in the first flow line database in a computer that can access the first and second flow line databases and the first and second image databases. A first flow line selection function for selecting the flow line data of the moving body located in the first image monitoring area from the inside, and the moving body corresponding to the flow line data selected by the first flow line selection function A first image selection function for selecting from the first image database camera image data at the time when is located in the first image monitoring area, and a moving object from the camera image data selected by the first image selection function First moving body image extraction function for extracting an image, and selection of flow line data of a moving body located in the second image monitoring area from each flow line data stored in the second flow line database The second flow line selection function and the camera image data at the time when the moving object corresponding to the flow line data selected by the second flow line selection function is located in the second image monitoring area A second image selecting function for selecting from the image database; a second moving object image extracting function for extracting a moving object image from the camera image data selected by the second image selecting function; and a first moving object image A mobile object identification function for collating the mobile object image extracted by the extraction function and the mobile object image extracted by the second mobile object image extraction function to determine whether or not the mobile object of each image is the same; When it is determined by the moving body identification function that the moving bodies are the same, the moving body is selected by the second flow line selection function in correspondence with at least the identification information of the flow line data selected by the first flow line selection function. Flow line data It is intended for realizing a flow line association function to store at least identification information.

  According to the present invention in which such a measure is taken, even if the plurality of flow line recognition areas in which the flow line recognition systems are constructed are not adjacent to each other, the flow lines recognized by each of the flow line recognition systems are used. The flow lines for the same moving body can be associated with each other, and there is an effect that enables the flow line analysis in a plurality of partial areas in a wide area.

The block diagram which shows the principal part structure of the flow line correlation apparatus and flow line recognition system which are one Embodiment of this invention. 4 is a schematic diagram showing a structure of image data stored in an image database in the embodiment. FIG. The schematic diagram which shows the structure of the flow line data memorize | stored in the flow line database in the embodiment. Explanatory drawing of the store to which the same embodiment is applied. The schematic diagram which shows an example of the camera layout of the flow line recognition system in the embodiment. The schematic diagram which shows one data example of the setting table which the flow line correlation apparatus has in the embodiment. The schematic diagram which shows the data structure of the related flow line file which the flow line correlation apparatus has in the embodiment. The flowchart which shows the main process procedures which the control part of the flow line connection apparatus when the flow line correlation program starts in the embodiment. The flowchart which shows the specific procedure of the flow line P selection process in the process sequence shown in FIG. The flowchart which shows the specific procedure of the flow line Q selection process in the process sequence shown in FIG. The flowchart which shows the specific procedure of the flow line correlation process in the process sequence shown in FIG. The flowchart which shows the specific procedure of the flow line correlation process in the 2nd Embodiment of this invention. The schematic diagram which shows one layout example of the flow line correlation screen displayed on the display part of a flow line correlation process in the said 2nd Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. First, a first embodiment will be described with reference to FIGS.
In addition, this embodiment is a case where it applies to the store 1 provided with the four sales floor A, the sales floor B, the sales floor C, and the sales floor D which are divided into the shape of a rice field and a customer can come and go as shown in FIG. is there.

  As shown in FIG. 4, the sales floor A is adjacent to the sales floor B and the sales floor D, the sales floor B is adjacent to the sales floor A and the sales floor C, the sales floor C is adjacent to the sales floor B and the sales floor D, and the sales floor D is It is adjacent to sales floor C and sales floor A. Sales floor A and sales floor C and sales floor B and sales floor D are not adjacent. Passages are formed in the border areas of adjacent sales floors so that customers can come and go. In addition, the sales floor A and the sales floor B are provided with entrances G1 and G2 through which customers can enter and exit.

  Among the sales floors A to D, the sales floor A and the sales floor C each have a flow line recognition system SA, SC for recognizing the trajectory of a person (shopper) that is a moving body moving in the sales floor as a flow line. Has been built. That is, the sales floor A is a flow line recognition area of the flow line recognition system SA, and the sales floor C is a flow line recognition area of the flow line recognition system SC.

  Each of the flow line recognition systems SA and SC is an image processing system using a camera. As shown in FIG. 1, the image processing type flow line recognition system includes a plurality of cameras 2 and flow line recognition devices 3A and 3C mainly composed of computers.

  Each camera 2 is a wide-angle lens camera such as a camera using a fisheye lens or a camera with an omnidirectional mirror. These cameras 2 are used to track a trajectory of a shopper who moves in the corresponding sales floors A and C, that is, a person flow line, by the visual volume intersection method. The visual volume intersection method is, for example, a method in which a customer's head is photographed from a plurality of directions and a coordinate value in a three-dimensional coordinate system appropriately installed in the store space of the head is calculated from the position of the head of each image. is there. The number of cameras arranged in one monitoring area is not particularly limited, but at least three cameras are necessary in order to improve the position detection accuracy by the visual volume intersection method.

  Each camera 2 repeats a photographing operation at a constant cycle (for example, 1/15 seconds) by centralized control of the camera control unit 31 provided in the flow line recognition devices 3A and 3C, and the captured image (frame image) is a flow line recognition device. Send to 3A, 3C. In addition to the camera control unit 31, the flow line recognition device 3A includes a camera image database 32A, a flow line generation unit 33, and a flow line database 34A. In addition to the camera control unit 31, the flow line recognition device 3C includes a camera image database 32C, a flow line generation unit 33, and a flow line database 34C.

  The camera image databases 32A and 32C sequentially store frame image data taken by each camera 2. The structure of the frame image data stored in the camera image databases 32A and 32C is shown in the schematic diagram of FIG. As shown in FIG. 2, the frame image data is stored in the camera image databases 32A and 32C in association with the camera ID, the frame number, and the shooting time. The camera ID is an identification code of the camera 2 that has captured the image data. Each camera 2 has a unique camera ID set in advance. The frame number is a serial number from “1” that is counted up each time a frame image is captured by each camera 2. Under the control of the camera control unit 31, the frame numbers of the cameras 2 are synchronized. The shooting date and time is the date and time (year / month / day / hour / minute / second) when the image data was shot.

  The flow line generation unit 33 recognizes a shopper moving in the flow line recognition area (sale floor) by image processing of the frame image data stored in the camera image databases 32A and 32C, and the shopper is in the flow line recognition area. The flow line data indicating the trajectory in the flow line recognition area from the start point that is the entry point to the end point that is the exit point is created. The flow line generation unit 33 has a function of connecting the flow lines of the same shopper who has been interrupted into a plurality of flow lines, for example, by the technique described in Patent Document 1.

  The flow line databases 34 </ b> A and 34 </ b> C sequentially accumulate flow line data generated for each shopper by the flow line generation unit 33. The structure of the flow line data stored in the flow line databases 34A and 34C is shown in the schematic diagram of FIG. As shown in FIG. 3, the flow line data is stored in the flow line databases 34A and 34C in association with the flow line ID. The flow line ID is flow line specifying information for individually specifying the flow line data. Each time the flow line generation unit 33 generates the flow line data of one shopper, the flow line generation unit 33 attaches a unique flow line ID to the flow line data and stores them in the flow line databases 34A and 34C.

  The flow line data includes the frame number of the start point frame “1”, the frame number of the end point frame “n” (n> 1), and index data for each frame from the start point frame “1” to the end point frame “n”. It consists of and. The index data includes at least the three-dimensional coordinates (X, Y, H) in the store and the frame shooting time T. The start frame “1” number is the frame number of the frame image first recognized by the shopper of the flow line data, and the shooting time of this frame image is the frame shooting time T1 of the index data for the start frame “1”. Become. The end frame “n” number is the frame number of the frame image last recognized by the shopper, and the shooting time of this frame image is the frame shooting time Tn of the index data for the end frame “n”. The store coordinates for each frame number are obtained by specifying the position of the customer recognized from the frame image of the corresponding frame number in the three-dimensional world coordinate system (X, Y, H). In the present embodiment, in FIG. 4, the lower left corner is the origin O (0, 0) of the X and Y axes, and the floor is the origin of the H (Height) axis.

  The cameras 2 of the respective flow line recognition systems SA and SC are attached in a distributed manner to the ceilings of the corresponding sales floors A and C, respectively. In this embodiment, for convenience of explanation, it is assumed that six cameras 2 are attached to each of the sales floors A and C in the layout shown in # 1 to # 6 of FIG.

  The principal part structure of the flow line correlation apparatus 4 concerning this invention is shown with the block diagram of FIG. The flow line association apparatus 4 includes an input unit 41, a display unit 42, a database access unit 43, a program storage unit 44, a data storage unit 45, an association flow line file 46, and a control unit 47. The input unit 41 includes input devices such as a keyboard and a mouse. The display unit 42 includes a display device such as a liquid crystal display or a CRT display. By using a display with a touch panel, the input unit and the display unit may be combined.

  The database access unit 43 accesses the flow line databases 34A and 34C and the camera image databases 32A and 32C respectively included in the flow line recognition devices 3A and 3C of the flow line recognition systems SA and SC, and receives the flow line data and frame image data. Can read and write.

  The program storage unit 44 is composed of a ROM (Read Only Memory). The data storage unit 45 is composed of a RAM (Random Access Memory). The association flow line file 46 is composed of a recording medium such as a hard disk or a magneto-optical disk. The control unit 47 is configured mainly with a CPU (Central Processing Unit). The flow line association apparatus 4 having such a configuration is realized by a computer device such as a personal computer.

  The program storage unit 44 stores the flow line data recognized by the flow line recognition system SA of the sales floor A (flow line recognition area) and the flow line data recognized by the flow line recognition system SC of the sales floor C. A program for selecting and associating the flow line data of the same shopper who goes back and forth between the sales floors A and C, a so-called flow line association program is installed. The control unit 47 associates flow line data according to the flow line association program.

  Hereinafter, a procedure for associating the flow line data will be specifically described. First, the flow line data stored in the flow line databases 34A and 34C of the flow line recognition systems SA and SC will be described.

  In the flow line database 34A of the flow line recognition device 3A corresponding to the flow line recognition system SA, a trajectory in the sales floor A from the start point where the shopper enters the sales floor A to the end point where the shop exits is shown. Flow line data is accumulated. The shopper can enter and leave the sales floor A through the entrance / exit G1 or the passage in the boundary area AB between the sales floors A and B or the passage in the boundary area DA between the sales floors D and A. Therefore, the following nine types of flow line data are stored in the flow line database 34A.

1. Data of a shopper who enters from the entrance / exit G1 and goes around the sales floor A and exits from the entrance / exit G1.

2. Data of a shopper who enters the store from the entrance / exit G1 and moves to the store B through the passage in the boundary area AB.

3. Data of a shopper who enters the store from the entrance / exit G1 and moves to the store D through the passage in the boundary area DA.

4). Data of a shopper who enters the sales floor A from the sales floor B through the boundary area AB and returns to the sales floor B through the boundary area AB.

5. Data of a shopper who enters the sales floor A from the sales floor B through the passage in the boundary area AB and further moves to the sales floor D through the passage in the boundary area DA.

6). Data of a shopper who enters the sales floor A from the sales floor B through the passage in the boundary area AB and exits from the entrance / exit G1.

7). Data of a shopper who enters the sales floor A through the boundary area DA from the sales floor D and returns to the sales floor D through the boundary area DA again.

8). Data of a shopper who enters the sales floor A from the sales floor D through the passage in the boundary area DA and further moves to the sales floor B through the passage in the boundary area AB.

9. Data of a shopper who enters the sales floor A from the sales floor D through the boundary area DA and exits from the entrance G1.

  The flow line database 34C of the flow line recognition device 3SC corresponding to the flow line recognition system 2C shows a trajectory in the sales floor C from the start point where the shopper enters the sales floor C to the end point where the shopper exits. Flow line data is accumulated. The shopper can enter and leave the sales floor C through the boundary area BC or the boundary area CD. Therefore, the flow line data stored in the flow line database 34C are the following four types.

1. Data of a shopper who enters the sales floor C from the sales floor B through the boundary area BC and returns to the sales floor B through the boundary area BC again.

2. Data of a shopper who enters the sales floor C from the sales floor B through the passage in the boundary area BC and further moves to the sales floor D through the passage in the boundary area CD.

3. Data of a shopper who enters the sales floor C through the boundary area CD from the sales floor D and returns to the sales floor D through the boundary area CD again.

4). Data of a shopper who enters the sales floor C from the sales floor D through the passage of the boundary area CD and further moves to the sales floor B through the passage of the boundary area BC.

  Here, as a pattern for the shopper to go back and forth between the sales floor A and the sales floor C, the shop exits from the sales floor A through the passage in the boundary area AB, and enters the sales floor C through the passage in the boundary area BC through the sales floor B. The pattern ABC and its reverse pattern CBA, and the pattern ADC that leaves the sales floor A through the passage of the boundary area DA, enters the sales floor C from the passage of the boundary area CD through the sales floor D, and the reverse pattern CDA. is there. Therefore, the flow line data at the sales floor A of the same shopper and the flow line data at the sales floor C may be associated with each of the patterns ABC, CBA, ADC, and CDA.

  Therefore, the flow line association device 4 selects the flow line data of the shopper located in the first image monitoring area from the respective flow line data stored in the first flow line database 34A (the first flow line database 34A). Flow line selection stage). Then, the camera image data at the time when the shopper corresponding to the selected flow line data is located in the first image monitoring area is selected from the first image database 32A (first image selection stage). Then, a shopper's image is extracted from the selected camera image data (first moving body image extraction stage). Next, the flow line data of the shopper located in the second image monitoring area is selected from the flow line data stored in the second flow line database 34C (second flow line selection stage). Then, the camera image data at the time when the shopper corresponding to the flow line data is located in the second image monitoring area is selected from the second image database 32C (second image selection stage). Then, a shopper's image is extracted from the selected camera image data (second moving body image extraction stage). The shopper image extracted in the first moving body image extraction step and the shopper image extracted in the second moving body image extraction step are collated to determine whether the shopper of each image is the same person or not. (Moving object identification stage). When it is determined that the shopper is the same person, the flow line data selected by the second flow line selection step in correspondence with at least the identification information of the flow line data selected by the first flow line selection step. At least identification information is stored (flow line association step). Thus, the flow line of the same shopper is obtained from the flow line data of each shopper stored in the first flow line database 34A and the flow line data of each shopper stored in the second flow line database 34C. Associate data with each other.

  In order to realize such a flow line association process, the flow line association apparatus 4 includes a data structure setting table 5 shown in FIG. The setting table 5 may be stored in a memory built in the control unit 47 or may be stored in the data storage unit 45. The setting table 5 in FIG. 6 corresponds to the camera layout shown in FIG.

  In the case of FIG. 5, there are two cameras 2 # 3 and # 6 of the flow line recognition system SA that can photograph the rectangular boundary area AB between the sales floor A and the sales floor B. There are three cameras 2, # 4, # 5, and # 6 of the flow line recognition system SC that can photograph the rectangular boundary region BC between the sales floor B and the sales floor C. The cameras 2 capable of photographing the rectangular boundary area CD between the sales floor C and the sales floor D are the two cameras # 1 and # 4 of the flow line recognition system SC. The cameras 2 that can photograph the rectangular boundary area DA between the sales floor D and the sales floor A are the three cameras # 1, # 2, and # 3 of the flow line recognition system SA.

  In the setting table 5, the two-dimensional coordinates of the upper left corner of the area (item “upper left coordinate”) and the two-dimensional coordinates of the lower right corner (item “item“ Lower right coordinates "), identification information (item" target system ") of a flow line recognition system having a camera capable of photographing the boundary area, and specific information of the camera (item" judgment camera ") are set. Yes.

  Further, the flow line association device 4 stores the associated flow line data in the association flow line file 46. As shown in FIG. 7, the association flow line file 46 corresponds to a unique association flow line ID (item “association flow line ID”), and a flow line ID for specifying a flow line of the flow line recognition system SA. (Item “SA flow line ID”), flow line ID (item “SA flow line ID”) for specifying the flow line of the flow line recognition system SA associated with this flow line ID, sales floor A and sales floor C It is possible to record time information (item “time required for movement”) required for movement between the two.

  For example, when the start of a flow line association job is instructed by an operation input of the input unit 41, the flow line association program is activated. Thereby, the control part 47 controls each part in the procedure shown in the flowchart of FIG.

  First, as ST (step) 1, the control unit 47 determines the association direction. For example, when the flow line association process of the shopper who moved from the sales floor A to the sales floor C is designated by the operation input of the input unit 41, the processes of ST11 to ST18 are executed. When the flow line association processing of the shopper who has moved from the sales floor C to the sales floor A is designated, the processing of ST21 to ST28 is executed.

  The patterns of the shopper who moves from the sales floor A to the sales floor C include the pattern ABC and the pattern ADC. Therefore, first, a process of associating the flow line data at the sales floor A of the shopper with the pattern ABC and the flow line data at the sales floor C is executed.

  That is, the control unit 47 selects the flow line data P of the customer who has left the sales floor A through the passage of the boundary area AB from the flow line database 34A of the flow line recognition system SA as ST11 (flow line P selection process). Further, as ST12, the flow line database 34C of the flow line recognition system SC selects the flow line data Q of the customer who has entered the sales floor C through the passage of the boundary region BC (flow line Q selection process). In ST13, it is determined whether or not at least one of the flow line data P and the flow line data Q has been selected. If it can be selected (YES in ST13), the controller 47 associates the flow line data P with the flow line data Q in ST14 (flow line association process). If it cannot be selected (NO in ST13), the flow line association process in ST14 is not performed.

  Next, a process of associating the flow line data at the sales floor A of the customer with the pattern ADC and the flow line data at the sales floor C is executed.

  That is, the control unit 47 selects the flow line data P of the customer who has left the sales floor A through the passage of the boundary area DA from the flow line database 34A of the flow line recognition system SA in ST15 (flow line P selection process). In ST16, the flow line data Q of the customer who has entered the sales floor C through the boundary area CD is selected from the flow line database 34C of the flow line recognition system SC (flow line Q selection process). In ST17, it is determined whether or not at least one of the flow line data P and the flow line data Q has been selected. If it can be selected (YES in ST17), the controller 47 associates the flow line data P with the flow line data Q in ST18 (flow line association process). If it cannot be selected (NO in ST17), the flow line association process in ST18 is not performed. This is the end of the flow line association program.

  The patterns of the shopper who moves from the sales floor C to the sales floor A include the pattern CBA and the pattern CDA. Therefore, first, a process of associating the flow line data at the sales floor C of the shopper with the pattern CBA and the flow line data at the sales floor A is executed.

  That is, the control unit 47 selects the flow line data P of the customer who has left the sales floor C through the passage of the boundary area BC from the flow line database 34C of the flow line recognition system SC as ST21 (flow line P selection process). In ST22, the flow line data Q of the customer who has entered the sales floor A through the passage in the boundary area AB is selected from the flow line database 34A of the flow line recognition system SA (flow line Q selection process). In ST23, it is determined whether or not at least one of the flow line data P and the flow line data Q has been selected. If it can be selected (YES in ST23), the controller 47 associates the flow line data P with the flow line data Q in ST24 (flow line association process). If it cannot be selected (NO in ST23), the flow line association process in ST24 is not performed.

  Next, a process of associating the flow line data at the sales floor C of the shopper with the pattern CDA and the flow line data at the sales floor A is executed.

  That is, the control unit 47 selects the flow line data P of the customer who has left the sales floor C through the passage of the boundary area CD from the flow line database 34C of the flow line recognition system SC as ST25 (flow line P selection process). In ST26, the flow line data Q of the customer who has entered the sales floor A through the boundary area DA is selected from the flow line database 34A of the flow line recognition system SA (flow line Q selection process). In ST27, it is determined whether or not at least one of the flow line data P and the flow line data Q has been selected. If it can be selected (YES in ST27), the controller 47 associates the flow line data P with the flow line data Q in ST28 (flow line association process). If it cannot be selected (NO in ST27), the flow line association process in ST28 is not performed. This is the end of the flow line association program.

  FIG. 9 is a flowchart specifically showing the flow line P selection process in ST11, ST15, ST21 and ST25. Since the process of each step is the same except that the target flow line recognition system is different, it will be described collectively with reference to FIG.

  When the flow line P selection process is started, the control unit 47 first resets the data number counter m1 to “0” as ST31. In ST32, the pointer p is reset to “0”. Next, in ST33, the pointer p is increased by "1". Then, as ST34, the pointer p is changed to the flow line database (the flow line database 34A in ST11 and ST15, the flow line recognition system SA in ST11 and ST15, and the flow line recognition system SC in ST21 and ST25). In ST21 and ST25, it is determined whether or not the total number of flow line data stored in the flow line database 34C) has been exceeded.

  At this time, since it has not exceeded yet, the control unit 47 sends the flow line data indicated by the pointer p from the flow line database 34A or 34C of the target system SA or SC as ST35 (hereinafter referred to as p-th flow line data). Capture. In ST36, the two-dimensional coordinates (Xn, Yn) of the end point index information are detected from the p-th flow line data.

  Next, the control unit 47 sets in ST37 corresponding to the comparison target boundary area (the boundary area AB in ST11, the boundary area DA in ST15, the boundary area BC in ST21, and the boundary area CD in ST25). The upper left coordinate and lower right coordinate data are read out. Then, these coordinate data are compared with the two-dimensional coordinates (Xn, Yn) of the end point index information detected in the process of ST36, and the position indicated by the two-dimensional coordinates (Xn, Yn) It is determined whether or not the boundary area is AB, DA, BC or CD (first flow line selection means).

  For example, when the boundary area to be compared is the area AB, the upper left coordinates are (X1, Y1) and the lower right coordinates are (X2, Y2). Therefore, when “Xn” of the end point two-dimensional coordinates (Xn, Yn) has a relationship of X1 ≦ Xn ≦ X2 and “Yn” has a relationship of Y1 ≧ Yn ≧ Y2, the end point two-dimensional coordinates (Xn, Yn) is determined to be inside the boundary area AB. Similarly, in the case of the boundary area DA, when “Xn” of the end point two-dimensional coordinates (Xn, Yn) has a relationship of X7 ≦ Xn ≦ X8 and “Yn” has a relationship of Y7 ≧ Yn ≧ Y8. , The end point two-dimensional coordinates (Xn, Yn) are determined to be inside the boundary area DA. In the case of the boundary area BC, the relationship “Xn” of the end point two-dimensional coordinates (Xn, Yn) satisfies X3 ≦ Xn ≦ X4. And “Yn” is in the relationship of Y3 ≧ Yn ≧ Y4, the end point two-dimensional coordinates (Xn, Yn) are determined to be inside the boundary region BC. When “Xn” of the dimensional coordinates (Xn, Yn) has a relationship of X5 ≦ Xn ≦ X6 and “Yn” has a relationship of Y5 ≧ Yn ≧ Y6, the end point two-dimensional coordinate (Xn, Yn) is a boundary. It is determined that it is inside the area DA.

  If it is determined that the position indicated by the end point two-dimensional coordinates (Xn, Yn) is not inside the boundary region AB, DA, BC or CD to be compared, the control unit 47 returns to the process of ST33. Then, the pointer p is further increased by “1”, and the processes after ST34 are executed again.

  On the other hand, when it is determined that the position indicated by the end point two-dimensional coordinates (Xn, Yn) is inside the boundary region AB, DA, BC or CD to be compared, the control unit 47 sets p38 as ST38. The end frame n number is obtained from the second flow line data. Next, in ST39, the information of the target system and the information of the determination camera set corresponding to the boundary region to be compared are acquired from the setting table 5. Then, from the image database of the target system (the image database 32A in ST11 and ST15, and the image database 32C in ST21 and ST25), the end point frame detected in the process of ST38 among the frame image data photographed by the determination camera. The image data assigned with the n number and stored is taken in (first image selection means).

  Next, the control unit 47 analyzes the frame image data of the end frame n number fetched from the image database 32A or 32C in ST40, and extracts a person image from these image data (first moving body image extracting means) ). If the person image can be extracted, the control unit 47 associates the flow line ID of the p-th flow line data and the frame n index information indicating the end point with the extracted human image data in ST41. One record is generated and stored in the first flow line memory on the data storage unit 45. In ST42, the data number counter m1 is incremented by "1". Thereafter, the control unit 47 returns to the process of ST33. Then, the pointer p is further increased by “1”, and the processes after ST34 are executed again.

  Thus, the control unit 47 performs the processes of ST35 to ST42 every time the pointer p is moved up until the pointer p exceeds the total number of the flow line data stored in the flow line database 34A or 34C of the target flow line recognition system. Execute repeatedly. When the pointer p exceeds the total number of flow line data, the flow line P selection process ends.

  By executing the flow line P selection process of this procedure, in ST11, the flow line ID and end point index information of the flow line data having the boundary area AB as the end point and the person image data are stored in the first flow line memory. The The human image data is obtained from images taken by the cameras identified by # 3 and # 6 among the cameras 2 of the flow line recognition system SA when the end point of the flow line data reaches the boundary area AB. Extracted. In ST15, the flow line ID and end point index information of the flow line data having the boundary area DA as the end point, and the person image data are stored in the first flow line memory. The human image data is taken by the cameras identified by # 1, # 2 and # 3 among the cameras 2 of the flow line recognition system SA when the end point of the flow line data reaches the boundary area DA. Extracted from images. In ST21, the flow line ID and end point index information of the flow line data having the boundary region BC as the end point and the person image data are stored in the first flow line memory. The human image data is captured by the cameras identified by # 4, # 5, and # 6 among the cameras 2 of the flow line recognition system SC when the end point of the flow line data reaches the boundary region BC. Extracted from images. In ST25, the flow line ID and end point index information of the flow line data having the boundary region CD as the end point and the person image data are stored in the first flow line memory. The human image data is obtained from images taken by the cameras identified by # 1 and # 4 among the cameras 2 of the flow line recognition system SC when the end point of the flow line data reaches the boundary region CD. Extracted.

  FIG. 10 is a flowchart specifically showing the procedure of the flow line Q selection process in ST12, ST16, ST22 and ST26. Since the procedure of each step is the same except that the target flow line recognition system is different, it will be described collectively with reference to FIG.

  When the flow line Q selection process is started, the control unit 47 first resets the data number counter m2 to “0” in ST51. In ST52, the pointer q is reset to “0”. Next, in ST53, the pointer q is incremented by “1”. Then, as ST54, the pointer q becomes the flow line database (the flow line database 34C in ST12 and ST16) of the target flow line recognition system (the flow line recognition system SC in ST12 and ST16, and the flow line recognition system SA in ST22 and ST26). In ST22 and ST26, it is determined whether or not the total number of flow line data stored in the flow line database 34A) has been exceeded.

  At this point, since it has not yet exceeded, the control unit 47 sends the flow line data indicated by the pointer q from the flow line database 34C or 34A of the target system SC or SA as ST55 (hereinafter referred to as qth flow line data). Capture. In ST56, the two-dimensional coordinates (X1, Y1) of the starting point index information are detected from the q-th flow line data.

  Next, the control unit 47 sets, in ST57, from the setting table 5 corresponding to the boundary area to be compared (the boundary area BC in ST12, the boundary area CD in ST16, the boundary area AB in ST22, and the boundary area DA in ST26). The upper left coordinate and lower right coordinate data are read out. Similar to the process of ST36, these coordinate data and the two-dimensional coordinates (X1, Y1) of the starting point index information detected by the process of ST56 are collated and indicated by the two-dimensional coordinates (X1, Y1). It is determined whether the position to be detected is inside the boundary region BC, CD, AB or DA to be compared (second flow line selecting means).

  When it is determined that the position indicated by the start point two-dimensional coordinates (X1, Y1) is not inside the boundary region BC, CD, AB or DA to be compared, the control unit 47 returns to the process of ST53. Then, the pointer q is further increased by “1”, and the processing after ST54 is executed again.

  On the other hand, when it is determined that the position indicated by the start point two-dimensional coordinates (X1, Y1) is inside the boundary region BC, CD, AB or DA to be compared, the control unit 47 sets q as ST58. The starting point frame 1 number is obtained from the second flow line data. Next, in ST59, information on the target system and information on the determination camera set corresponding to the boundary region to be compared are acquired from the setting table 5. Then, from the image database of the target system (the image database 32C in ST12 and ST16, the image database 32A in ST22 and ST26), the start frame detected in the process of ST58 in the frame image data photographed by the determination camera. The image data assigned with the number 1 and stored is taken in (second image selection means).

  Next, the control unit 47 analyzes the frame image data of the starting point frame 1 number fetched from the image database 32C or 32A as ST60, and extracts a person image from these image data (second moving body image extracting means). ). If the person image can be extracted, the control unit 47 associates the flow line ID of the qth flow line data and the frame 1 index information indicating the start point with the extracted human image data in ST61. One record is generated and stored in the second flow line memory on the data storage unit 45. In ST62, the data number counter m2 is incremented by "1". Thereafter, the control unit 47 returns to the process of ST53. Then, the pointer q is further increased by “1”, and the processing after ST54 is executed again.

  In this way, the control unit 47 performs the processing of ST55 to ST62 every time the pointer q is raised until the pointer q exceeds the total number of flow line data stored in the flow line database 34C or 34A of the target flow line recognition system. Execute repeatedly. When the pointer q exceeds the total number of flow line data, the flow line Q selection process ends.

  By executing the flow line Q selection process of such a procedure, in ST12, the flow line ID and start point index information of the flow line data starting from the boundary area BC and the data of the person image are stored in the second flow line memory. The The human image data is obtained by the cameras identified by # 4, # 5, and # 6 among the cameras 2 of the flow line recognition system SC at the time when the start point of the flow line data is detected in the boundary region BC. Extracted from the captured image. In ST16, the flow line ID and start point index information of the flow line data starting from the boundary region CD and the data of the person image are stored in the second flow line memory. The human image data is an image taken by the camera identified by # 1 and # 4 among the cameras 2 of the flow line recognition system SC at the time when the start point of the flow line data is detected in the boundary region CD. Extracted from In ST22, the flow line ID and start point index information of the flow line data starting from the boundary area AB and the person image data are stored in the second flow line memory. The human image data is an image taken by the camera identified by # 3 and # 6 among the cameras 2 of the flow line recognition system SA when the start point of the flow line data is detected in the boundary area AB. Extracted from In ST26, the flow line ID and start point index information of the flow line data starting from the boundary area DA and the data of the person image are stored in the second flow line memory. The human image data is captured by the cameras identified by # 1, # 2, and # 3 among the cameras 2 of the flow line recognition system SA when the start point of the flow line data is detected in the boundary area DA. Extracted from the processed image.

  FIG. 11 is a flowchart specifically showing the flow line association processing procedure of ST14, ST18, ST24 and ST28. Since the processing procedure of each step is the same, it will be described collectively with reference to FIG.

  When the flow line association process is started, the control unit 47 first resets the pointer p to “0” in ST71. Next, in ST72, the pointer p is increased by "1". Then, in ST73, it is determined whether or not the pointer p has exceeded the data number counter m1. The data number counter m1 is the number of data stored in the first flow line memory.

  When the pointer p does not exceed the data number counter m1 (NO in ST73), the control unit 47 reads data indicated by the pointer p (hereinafter referred to as p-th data) from the first flow line memory as ST74.

  Next, the control unit 47 resets the pointer q to “0” in ST75. Next, in ST76, the pointer q is increased by "1". Then, in ST77, it is determined whether or not the pointer q exceeds the data number counter m2. The data number counter m2 is the number of data stored in the second flow line memory.

  When the pointer q does not exceed the data number counter m2 (NO in ST77), the control unit 47 reads data indicated by the pointer q (hereinafter referred to as qth data) from the second flow line memory as ST78.

  Thereafter, the control unit 47 determines whether or not the person specified by both person image data is the same person from the person image data included in the pth data and the person image data included in the qth data in ST79. A person identification process for determination is executed (moving body identification means).

  As a result of the person identification process, when it is determined that they are not the same person (NO in ST80), the control unit 47 returns to the process of ST76. Then, the pointer q is further increased by “1” (ST76), and if it is confirmed that the pointer q has not reached the data number counter m2 (NO in ST77), the processing after ST78 is executed again.

  On the other hand, when it is determined that they are the same person (YES in ST80), the control unit 47 generates a new association flow line ID as ST81. Moreover, the control part 47 calculates the elapsed time from the end point arrival time Tn of the end point index information included in the pth data to the start point arrival time T1 of the start point index information included in the qth data as ST82. This elapsed time is the required travel time ΔT (ΔT = T1−Tn) between the target sales floors (Sales floor A → Sales floor C in ST14 and ST18, Sales floor C → Sales floor B in ST24 and ST28). ).

  Then, the control unit 47 generates one record from each data of the new association flow line ID, the p-th data flow line ID, the q-th data flow line ID, and the movement required time ΔT in ST83, and the association flow line file 46 (Flow line association means).

Thereafter, the controller 47 deletes the qth data from the second flow line memory in ST84. In ST85, the data number counter m2 is counted down by “1”. Thereafter, the process returns to ST72.
Thus, by deleting the associated qth data from the second flow line memory, the processing procedure required for searching for the qth data associated with the next p + 1th data is reduced.

  If the pointer q exceeds the data number counter m2 in ST77, the control unit 47 returns to the process of ST72 without executing the processes after ST78.

  Thus, every time the pointer p is incremented by “1”, the control unit 47 executes the processes after ST74. If the pointer p exceeds the data number counter m1 (YES in ST73), the control unit 47 clears the first flow line memory and the second flow line memory in ST86. This is the end of the flow line association process.

  By executing the flow line association process of such a procedure, the flow line data identified by the flow line ID of the p-th data determined to be the same person and the flow line ID of the q-th data. Is associated with the flow line data identified by. That is, in ST14, the flow line data at the sales floor A and the flow line data at the sales floor C of the shopper who has moved from the sales floor A to the sales floor C through the sales floor B are associated with each other. In ST18, the flow line data at the sales floor A and the flow line data at the sales floor C of the shopper who has moved from the sales floor A to the sales floor C through the sales floor D are associated with each other. In ST24, the flow line data at the sales floor C of the shopper who has moved from the sales floor C to the sales floor A through the sales floor B is associated with the flow line data at the sales floor A. In ST28, the flow line data at the sales floor C of the shopper who has moved from the sales floor C to the sales floor A through the sales floor D is associated with the flow line data at the sales floor A.

  In ST40 of FIG. 9 and ST60 of FIG. 10, a technique for extracting a human image without distortion from the camera image data uses, for example, a well-known technique disclosed in Patent Document 2, Patent Document 3, and the like. Similarly, in ST79 of FIG. 11, the technique of person identification processing for determining whether or not the extracted two person images are the same person by comparing (pattern recognition, similarity between color histograms, etc.) is, for example, a patent. A known technique disclosed in Document 4 or Patent Document 3 is used.

  As described above, according to the present embodiment, a shopper who goes back and forth between the sales floor A and the sales floor C that are not adjacent to each other without building a flow line recognition system in each of the sales floors A, B, C, and D of the store. Can be associated. Therefore, the store side's desire to analyze the behavior of the shopper who goes to and from a specific sales floor can be realized at low cost.

  Further, according to the present embodiment, there is an advantage that it is possible to manage even the time required for the movement of the shopper who goes back and forth between specific sales floors.

  Next, a second embodiment of the present invention will be described with reference to FIGS. This second embodiment is also applied to the store 1 having the layout shown in FIG. 4, and the image processing type flow line recognition systems SA and SC using cameras are constructed in the sales floor A and the sales floor C. is doing. The second embodiment is different from the first embodiment only in part of the procedure of the flow line association process. Accordingly, FIGS. 1 to 10 are also used, and the same portions are denoted by the same reference numerals for description.

  FIG. 12 is a flowchart specifically showing the procedure of the flow line association processing in the second embodiment. When the flow line association process is started, the control unit 47 first displays the flow line association screen 6 on the display unit 42 as ST91.

  An example of the flow line association screen 6 is shown in FIG. As illustrated, the flow line association screen 6 includes a flow line area 61, a first flow line ID area 62, a first image area 63, a second flow line ID area 64, and a second image area 65, and “YES”. "Button 66 and" NO "button 67 are displayed. Each button image can be input by the input unit 41.

  Next, the control part 47 performs the process same as ST71-ST78 of the flow line correlation process in 1st Embodiment as ST92-ST99. That is, the p-th data is read from the first flow line memory, and the q-th flow line data is read from the second flow line memory.

  Thereafter, the control unit 47 displays the flow line ID included in the p-th data in ST100 in the first flow line ID area 62 and displays the person image of the person image data in the first image area 63 (ST1). Camera image display means). Further, the flow line ID included in the qth data is displayed in the second flow line ID area 64, and the person image of the person image data is displayed in the second image area 65 (second camera image display means). Further, two flow lines identified by both flow line IDs are displayed in the flow line area 61. Then, the control unit 47 waits for one of the “YES” button 66 and the “NO” button 67 to be operated and input as ST101 (association declaration receiving means).

  Here, when the “NO” button 67 is input through the input unit 41 (NO in ST101), the control unit 47 returns to the process in ST97. Then, the pointer q is further increased by “1” (ST98), and if it is confirmed that the pointer q has not reached the data number counter m2 (NO in ST99), the processing after ST100 is executed again.

  On the other hand, when the “YES” button 66 is input via the input unit 41 (YES in ST101), the control unit 47 performs ST102 to ST106 of the flow line association process in the first embodiment as ST102 to ST106. The same processing as ST85 is executed. That is, one record is generated from each data of the newly associated flow line ID, the p-th data flow line ID, the q-th data flow line ID, and the movement required time ΔT, and is stored in the associated flow line file 46 (flow line association). means).

  Thus, every time the pointer p is incremented by “1”, the control unit 47 executes the processes after ST95. If the pointer p exceeds the data number counter m1 (YES in ST94), the control unit 47 clears the first flow line memory and the second flow line memory in ST107. This is the end of the flow line association process.

  In the second embodiment having such a configuration, a p-th person image is displayed in the first image area 63 of the flow line association screen 6 and a q-th person image is displayed in the second image area 65. The Therefore, the operator compares the person images displayed in both the image areas 63 and 65 to determine whether or not they are the same person.

  If they are not the same person, the operator operates the “NO” button 67. Then, the display of the second image area 63 is updated to the next q-th person image. In this way, the “NO” button 67 is repeatedly operated until an image of the same person as the image displayed in the first image area 63 is displayed in the second image area 65, and the image of the same person is displayed as the second image. If displayed in the area 63, the operator operates the “YES” button 66 as an association declaration means. Then, the flow line ID of the p-th data is associated with the flow line ID of the q-th data.

  Also in the second embodiment having such a configuration, the flow line data of the shopper who goes back and forth between the sales floor A and the sales floor C that are not adjacent to each other can be associated as in the first embodiment. Moreover, since the operator determines whether or not they are the same person by visual observation using the camera image, it is more reliable.

  Note that the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the spirit of the invention in the implementation stage.

  For example, in the above-described embodiment, the flow line P selection process is performed first and then the flow line Q selection process is performed in the process of FIG. 8, but the flow line Q selection process is performed first, and then the flow line P selection is performed. Processing may be executed.

  In the above embodiment, the image monitoring area is a boundary area of each sales floor. However, when there is a passage through which a large number of customers pass in the sales floor, the area of the passage may be used as the image monitoring area. That is, the image monitoring area may be a place where a person image corresponding to the flow line data can be extracted from the camera image data.

  In the embodiment, the flow line recognition systems SA and SC of each sales floor adopt an image processing method using a camera. However, other methods such as a wireless tag tracking method may be adopted. In this case, the sales floor A is provided with a camera for photographing the boundary area AB and the boundary area DA, and the sales floor C is provided with a camera for photographing the boundary area BC and the boundary area CD. Then, the frame image data of each camera is stored in the image database in association with the time information. Then, by extracting a person image from the frame image data at the start point detection time or the end point arrival time of the flow line data having each boundary region as the start point or the end point, and executing the person identification process, the same effect as the above embodiment Can be played.

  However, as in this embodiment, by adopting an image processing method using a camera, a camera for motion line recognition can be used as an image data acquisition camera for determining the association of a motion line as it is, The system cost can be reduced.

  Moreover, in the said embodiment, each sales floor is divided in the center of the sales floor divided into the shape of a rice field. For example, the shopper in the sales floor A goes to the sales floor B or the sales floor D but goes to the sales floor C. Although the present invention is applied to a store that does not go, the store form to which the present invention is applicable is not limited to this. For example, each sales floor does not have to be partitioned at the center of the sales floor divided into a square shape. Further, for example, the case of associating the flow line data of a customer moving on the sales floor on the first floor with the flow line data of a customer moving on the sales floor on the third floor can be implemented in the same manner as in the above embodiment. Furthermore, it is possible to associate the flow line data recognized by each of three or more flow line recognition systems.

  In addition, the present invention is not limited to an apparatus that associates customer flow lines for each sales floor, and flow line data of a moving body (a person, an animal, a vehicle, etc.) that moves in the first flow line recognition area; The present invention can be similarly applied to an apparatus that associates flow line data of a moving body that moves in a second flow line recognition area different from the first flow line recognition area.

  In the above embodiment, the present invention has been described on the assumption that the flow line connection program is stored in advance in the program storage unit 44. However, this program may be downloaded from the network to the flow line association apparatus 4. What is stored in the recording medium may be installed in the flow line association apparatus 4. The recording medium may be any form as long as it can store a program such as a CD-ROM and can be read by a computer. Further, the function obtained by installing or downloading in advance may be realized in cooperation with an OS (operating system) or the like inside the computer.

  In addition, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, the constituent elements over different embodiments may be combined.

  SA, SC ... flow line recognition system, 3A, 3C ... flow line recognition device, 4 ... flow line association device, 5 ... setting table, 6 ... flow line association screen, 32SA, 32SB, 32SC, 32SD ... image database, 34SA, 32A, 32C ... image database, 34A, 34C ... flow line database, 43 ... database access unit, 44 ... program storage unit, 46 ... association flow line file, 47 ... control unit.

JP 2009-009394 A JP 2001-209795 A JP 2009-059042 A JP 2002-305717 A

Claims (12)

A first flow line database for storing, for each moving body moving in the first flow line recognition area, flow line data indicating a trajectory of the moving body in the first flow line recognition area; For storing, for each moving body moving in a second flow line recognition area different from one flow line recognition area, flow line data indicating a trajectory of the moving body in the second flow line recognition area. A second flow line database, a first image database for storing camera image data taken in the first image monitoring area, which is at least a part of the first flow line recognition area, together with a shooting time; and Access is made to each database of the second image database for storing camera image data taken in the second image monitoring area, which is at least a part of the second flow line recognition area, together with the photographing time. OK From the flow line data of each moving body stored in the first flow line database and the flow line data of each moving body stored in the second flow line database, the computer uses the same moving body. A flow line association method for associating flow line data,
The computer
A first flow line selection step of selecting flow line data of a moving body located in the first image monitoring area from each flow line data stored in the first flow line database;
The camera image data at the time when the moving body corresponding to the flow line data selected in the first flow line selection stage is located in the first image monitoring area is selected from the first image database. 1 image selection stage;
A first moving body image extraction step of extracting a moving body image from the camera image data selected by the first image selection step;
A second flow line selection step of selecting flow line data of a moving body located in the second image monitoring area from each flow line data stored in the second flow line database;
The camera image data is selected from the second image database when the moving body corresponding to the flow line data selected in the second flow line selection step is located in the second image monitoring area. Two image selection stages;
A second moving body image extraction step of extracting a moving body image from the camera image data selected by the second image selection step;
The mobile body image extracted in the first mobile body image extraction stage and the mobile body image extracted in the second mobile body image extraction stage are collated to determine whether the mobile bodies of the images are the same. A mobile object identification stage,
If it is determined in this moving body identification step that the moving bodies are the same, the second flow line selection step corresponds to at least identification information of the flow line data selected in the first flow line selection step. A flow line associating step for storing at least identification information of the flow line data selected by:
A flow line associating method characterized by comprising: The time required for the moving body determined to be the same in the moving body identification step to move from the first flow line recognition area to the second flow line recognition area is expressed as the first time. A required time calculation step for calculating from information on the end point of the flow line data selected in the flow line selection step and information on the start point of the flow line data selected in the second flow line selection step,
In the flow line association step, at least identification information of the flow line data selected in the second flow line selection step in correspondence with at least identification information of the flow line data selected in the first flow line selection step. 2. The flow line associating method according to claim 1, further comprising: storing a required time calculated by the required time calculation step. A first flow line database for storing, for each moving body moving in the first flow line recognition area, flow line data indicating a trajectory of the moving body in the first flow line recognition area; For storing, for each moving body moving in a second flow line recognition area different from one flow line recognition area, flow line data indicating a trajectory of the moving body in the second flow line recognition area. A second flow line database, a first image database for storing camera image data taken in the first image monitoring area, which is at least a part of the first flow line recognition area, together with a shooting time; and Access is made to each database of the second image database for storing camera image data taken in the second image monitoring area, which is at least a part of the second flow line recognition area, together with the photographing time. OK From the flow line data of each moving body stored in the first flow line database and the flow line data of each moving body stored in the second flow line database, the computer uses the same moving body. A flow line association method for associating flow line data,
The computer
A first flow line selection step of selecting flow line data of a moving body located in the first image monitoring area from each flow line data stored in the first flow line database;
The camera image data of a period in which the moving body corresponding to the flow line data selected in the first flow line selection stage is located in the first image monitoring area is acquired from the first image database. A first camera image display stage for displaying the image on the display unit;
A second flow line selection step of selecting flow line data of a moving body located in the second image monitoring area from each flow line data stored in the second flow line database;
The camera image data of a period in which the moving body corresponding to the flow line data selected in the second flow line selection step is located in the second image monitoring area is acquired from the second image database. A second camera image display stage for displaying the image on the display unit;
An association declaration receiving step for receiving an association declaration between the flow line data selected by the first flow line selection step and the flow line data selected by the second flow line selection step;
When the association declaration is received, at least identification information of the flow line data selected by the second flow line selection step in correspondence with at least identification information of the flow line data selected by the first flow line selection step. A flow line association stage for storing
A flow line associating method characterized by comprising: In response to accepting the association declaration in the association declaration receiving step, a moving object corresponding to the flow line data selected in the first flow line selection step is moved from the first flow line recognition area to the second. The time required for moving to the flow line recognition area is information on the end point of the flow line data selected in the first flow line selection stage and the flow line data selected in the second flow line selection stage. A required time calculation stage calculated based on information on the starting point of
In the flow line association step, at least identification information of the flow line data selected in the second flow line selection step in correspondence with at least identification information of the flow line data selected in the first flow line selection step. 4. The flow line associating method according to claim 3, wherein the required time calculated by the required time calculation step is stored. A first flow line database for storing, for each moving body moving in the first flow line recognition area, flow line data indicating a trajectory of the moving body in the first flow line recognition area; For storing, for each moving body moving in a second flow line recognition area different from one flow line recognition area, flow line data indicating a trajectory of the moving body in the second flow line recognition area. A second flow line database, a first image database for storing camera image data taken in the first image monitoring area, which is at least a part of the first flow line recognition area, together with a shooting time; and Access is made to each database of the second image database for storing camera image data taken in the second image monitoring area, which is at least a part of the second flow line recognition area, together with the photographing time. OK It is in,
First flow line selection means for selecting flow line data of a moving body located in the first image monitoring area from each flow line data stored in the first flow line database;
The camera image data at the time when the moving body corresponding to the flow line data selected by the first flow line selection means is located in the first image monitoring area is selected from the first image database. 1 image selection means;
First moving body image extracting means for extracting a moving body image from the camera image data selected by the first image selecting means;
Second flow line selection means for selecting flow line data of a moving body located in the second image monitoring area from each flow line data stored in the second flow line database;
The camera image data at the time when the moving object corresponding to the flow line data selected by the second flow line selection means is located in the second image monitoring area is selected from the second image database. Two image selection means;
Second moving body image extraction means for extracting a moving body image from the camera image data selected by the second image selection means;
The mobile body image extracted by the first mobile body image extraction means and the mobile body image extracted by the second mobile body image extraction means are collated to determine whether the mobile bodies of the images are the same. Mobile object identification means for
When it is determined by the moving body identification means that the moving bodies are the same, the second flow line selection means is associated with at least identification information of the flow line data selected by the first flow line selection means. Flow line association means for storing at least identification information of the flow line data selected by:
A flow line associating device comprising: The time required for the moving body determined to be the same by the moving body identifying means to move from the first flow line recognition area to the second flow line recognition area is expressed as the first time. Required time calculating means for calculating from information related to the end point of the flow line data selected by the flow line selecting means and information related to the start point of the flow line data selected by the second flow line selecting means,
The flow line association means corresponds to at least identification information of the flow line data selected by the first flow line selection means, and at least identification information of the flow line data selected by the second flow line selection means. 6. The flow line associating device according to claim 5, wherein the required time calculated by said required time calculating means is stored. A first flow line database for storing, for each moving body moving in the first flow line recognition area, flow line data indicating a trajectory of the moving body in the first flow line recognition area; For storing, for each moving body moving in a second flow line recognition area different from one flow line recognition area, flow line data indicating a trajectory of the moving body in the second flow line recognition area. A second flow line database, a first image database for storing camera image data taken in the first image monitoring area, which is at least a part of the first flow line recognition area, together with a shooting time; and Access is made to each database of the second image database for storing camera image data taken in the second image monitoring area, which is at least a part of the second flow line recognition area, together with the photographing time. OK It is in,
First flow line selection means for selecting flow line data of a moving body located in the first image monitoring area from each flow line data stored in the first flow line database;
The camera image data during a period in which the moving body corresponding to the flow line data selected by the first flow line selection unit is located in the first image monitoring area is acquired from the first image database. First camera image display means for displaying the image on the display unit;
Second flow line selection means for selecting flow line data of a moving body located in the second image monitoring area from each flow line data stored in the second flow line database;
The camera image data of a period in which the moving body corresponding to the flow line data selected by the second flow line selection means is located in the second image monitoring area is acquired from the second image database. Second camera image display means for displaying the image on the display unit;
Association declaration receiving means for receiving an association declaration between the flow line data selected by the first flow line selection means and the flow line data selected by the second flow line selection means;
When the association declaration is received, at least identification information of the flow line data selected by the second flow line selection means in correspondence with at least identification information of the flow line data selected by the first flow line selection means. A flow line association means for storing
A flow line associating device comprising: In response to receiving the association declaration by the association declaration receiving unit, a moving body corresponding to the flow line data selected by the first flow line selecting unit is moved from the first flow line recognition area to the second line. The time required for moving to the flow line recognition area is information on the end point of the flow line data selected by the first flow line selection means and the flow line data selected by the second flow line selection means. And a required time calculation means for calculating based on information on the starting point of
The flow line association means corresponds to at least identification information of the flow line data selected by the first flow line selection means, and at least identification information of the flow line data selected by the second flow line selection means. 8. The flow line associating device according to claim 7, wherein the required time calculated by said required time calculating means is stored. A first flow line database for storing, for each moving body moving in the first flow line recognition area, flow line data indicating a trajectory of the moving body in the first flow line recognition area; For storing, for each moving body moving in a second flow line recognition area different from one flow line recognition area, flow line data indicating a trajectory of the moving body in the second flow line recognition area. A second flow line database, a first image database for storing camera image data taken in the first image monitoring area, which is at least a part of the first flow line recognition area, together with a shooting time; and Access is made to each database of the second image database for storing camera image data taken in the second image monitoring area, which is at least a part of the second flow line recognition area, together with the photographing time. OK To a computer,
A first flow line selection function for selecting flow line data of a moving body located in the first image monitoring area from each flow line data stored in the first flow line database;
The camera image data when the moving body corresponding to the flow line data selected by the first flow line selection function is located within the first image monitoring area is selected from the first image database. 1 image selection function,
A first moving body image extraction function for extracting a moving body image from the camera image data selected by the first image selection function;
A second flow line selection function for selecting flow line data of a moving body located in the second image monitoring area from each flow line data stored in the second flow line database;
The second image database selects the camera image data at the time when the moving body corresponding to the flow line data selected by the second flow line selection function is located in the second image monitoring area. 2 image selection functions;
A second moving body image extraction function for extracting a moving body image from the camera image data selected by the second image selection function;
The mobile body image extracted by the first mobile body image extraction function and the mobile body image extracted by the second mobile body image extraction function are collated to determine whether the mobile bodies of the images are the same. Mobile object identification function
When it is determined by the moving body identification function that the moving bodies are the same, the second flow line selection function is associated with at least identification information of the flow line data selected by the first flow line selection function. A flow line association function for storing at least identification information of the flow line data selected by:
A flow line association program for realizing In the computer,
The time required for the moving bodies determined to be the same by the moving body identification function to move from the first flow line recognition area to the second flow line recognition area is expressed as the first time. Further realizing a required time calculation function that is calculated based on the information on the end point of the flow line data selected by the flow line selection function and the information on the start point of the flow line data selected by the second flow line selection function. And
The flow line association function corresponds to at least identification information of the flow line data selected by the first flow line selection function, and at least identification information of the flow line data selected by the second flow line selection function. And a required time calculated by the required time calculation function. A first flow line database for storing, for each moving body moving in the first flow line recognition area, flow line data indicating a trajectory of the moving body in the first flow line recognition area; For storing, for each moving body moving in a second flow line recognition area different from one flow line recognition area, flow line data indicating a trajectory of the moving body in the second flow line recognition area. A second flow line database, a first image database for storing camera image data taken in the first image monitoring area, which is at least a part of the first flow line recognition area, together with a shooting time; and Access is made to each database of the second image database for storing camera image data taken in the second image monitoring area, which is at least a part of the second flow line recognition area, together with the photographing time. OK To a computer,
A first flow line selection function for selecting flow line data of a moving body located in the first image monitoring area from each flow line data stored in the first flow line database;
The camera image data of a period in which the moving body corresponding to the flow line data selected by the first flow line selection function is located in the first image monitoring area is acquired from the first image database. A first camera image display function for displaying the image on the display unit;
A second flow line selection function for selecting flow line data of a moving body located in the second image monitoring area from each flow line data stored in the second flow line database;
The camera image data of a period in which the moving body corresponding to the flow line data selected by the second flow line selection function is located in the second image monitoring area is acquired from the second image database. A second camera image display function for displaying the image on the display unit;
An association declaration acceptance function for accepting an association declaration between the flow line data selected by the first flow line selection function and the flow line data selected by the second flow line selection function;
When the association declaration is received, at least identification information of the flow line data selected by the second flow line selection function in correspondence with at least identification information of the flow line data selected by the first flow line selection function. A flow line association function for storing
A flow line association program for realizing In the computer,
In response to receiving the association declaration by the association declaration receiving function, a moving body corresponding to the flow line data selected by the first flow line selection function is moved from the first flow line recognition area to the second. The time required for moving to the flow line recognition area is information on the end point of the flow line data selected by the first flow line selection function and the flow line data selected by the second flow line selection function. The required time calculation function that is calculated based on the information about the start point of the
The flow line association function corresponds to at least identification information of the flow line data selected by the first flow line selection function, and at least identification information of the flow line data selected by the second flow line selection function. And a required time calculated by the required time calculation function.

Images