JP6814673B2 - Movement route prediction device and movement route prediction method - Google Patents

Description

The present invention relates to a movement route prediction device and a movement route prediction method.

Patent Document 1 states, "There is no monitoring system capable of predicting anomalous behavior before it occurs. Such systems report them after such infringement or accident occurs. Instead, it can prevent various security breaches and save lives. "," The present invention is a method for predicting the future location of a moving object being observed by a surveillance device. Includes the step of receiving motion data corresponding to the moving object from a monitoring device and generating metadata based on the received data, wherein the metadata is the time stamp, location, and size of the moving object. In addition, the method further comprises performing at least one wavelet transformation on the data structure based on the metadata and generating a key based on the at least one wavelet transformation. , Including the step of querying the predictive model database using that key, in which the predictive model database stores predictive models based on trained data, in which the predictive models are each. , A model key based on the first part of the trained locus, a model prediction based on the second part of the trained locus, and a count corresponding to the correlation between the model key and the model prediction. , Which includes the step of receiving at least one predictive model from the predictive model database, and receives it based on the similarity between the key and the model key of the predictive model received. Further, this method receives the received model key and receives. It includes the step of generating at least one future location for the moving object based on the count. "

Japanese Patent Publication No. 2012-518846

Patent Document 1 describes a method of predicting abnormal behavior in a video surveillance system based on surveillance data such as video data. Here, for example, since the future movement route differs between an abnormal person such as a criminal and a normal person, a person such as an action taken by the person when accurately predicting the movement route of the person based on monitoring data such as video data. You also need to consider information about. In Patent Document 1, the future location of a moving object is predicted based on a prediction model generated from monitoring data, but no consideration is given to information about the person such as actions taken by the person.

The present invention has been made in view of such a background, and an object of the present invention is to provide a movement route prediction device capable of accurately predicting a movement route of a person, and a movement route prediction method.

One of the present inventions for achieving the above object is a movement path prediction device, which is a sensor information acquisition unit that acquires sensor information, which is information acquired by a sensor, and movement of a person based on the sensor information. A movement history information generation unit that generates movement history information that is information about history , an information storage unit that stores behavior information that is information about a person's behavior and attribute information of a person, and the movement history information, the behavior information, And a prediction model generation unit that generates a prediction model including information that associates the action taken by the person with the prediction movement route that is the movement route that the person is predicted to move based on the attribute information, and tracking. It includes a movement route prediction unit that acquires the predicted movement route corresponding to the behavior information of the target person from the prediction model, and a prediction result output unit that outputs information indicating the acquired predicted movement route.

In addition, the problems disclosed in the present application and the solutions thereof will be clarified by the column of the form for carrying out the invention and the drawings.

According to the present invention, the movement path of a person can be accurately predicted.

It is a figure which shows the structure of the movement route prediction system shown as 1st Embodiment. This is an example of the hardware of the movement route prediction device. It is a block diagram of the movement route prediction apparatus of 1st Embodiment. It is a flowchart explaining the movement route prediction processing. It is a flowchart explaining the movement locus information generation processing. This is an example of movement locus information. It is a flowchart explaining behavior information generation processing. This is an example of operation model information. This is an example of behavioral information. This is an example of a prediction model. This is an example of a predicted movement route. This is an example of a prediction result display screen. It is a flowchart explaining the prediction model generation process. It is a figure which shows the structure of the movement route prediction system of 2nd Embodiment. It is a block diagram of the movement route prediction apparatus of 2nd Embodiment. It is a figure which shows the structure of the movement route prediction system of 3rd Embodiment. This is an example of a prediction result display screen displayed on a mobile information terminal.

Hereinafter, embodiments will be described. In the following description, duplicate description may be omitted by using the same reference numerals for the same or similar configurations.

[First Embodiment]
FIG. 1 shows the configuration of the movement route prediction system 1 shown as the first embodiment. As shown in the figure, the movement route prediction system 1 includes one or more image acquisition devices 2 and a movement route prediction device 100. The movement route prediction system 1 is based on the information provided by the image acquisition device 2, and is referred to as information on the movement locus of the person (hereinafter referred to as movement locus information) and information on the behavior of the person (hereinafter referred to as behavior information). ) Is generated, the future movement route of the person (tracking target person 3) is predicted based on the generated movement trajectory information and action information, and the predicted result is provided (displayed, etc.) to the user. The movement route prediction system 1 is applied to, for example, behavior monitoring of suspicious persons and children in stores such as shopping centers and convenience stores, and behavior monitoring of persons with disabilities and elderly people in welfare facilities.

The image acquisition device 2 and the movement route prediction device 100 communicate via, for example, a wired or wireless communication means (LAN (Local Area Network), wireless LAN (Wi-Fi, etc.), USB (Universal Serial Bus Interface), etc.). It is connected as possible. The image acquisition device 2 is an example of a sensor for acquiring movement locus information and behavior information of a person. For example, when an image acquisition device 2 such as a surveillance camera is installed in advance at the site, the image acquisition device 2 can be used as a sensor in the movement route prediction system 1. In that case, the system introduction cost Can be reduced.

The image acquisition device 2 acquires a time-series image (hereinafter, referred to as a time-series image) of the monitored area. A time-series image is a collection of images (still images) acquired in time series. Information (for example, a time stamp) indicating the date and time when each image was acquired (photographed) is attached to each image constituting the time series image. The time-series image may be, for example, one obtained from a video (moving image).

The plurality of image acquisition devices 2 constituting the movement path prediction system 1 are set with an installation position, a shooting direction, a lens magnification, and the like so that the shooting ranges are different from each other. A part of the shooting range may overlap between the plurality of image acquisition devices 2. Further, the image acquisition device 2 may be configured so that the shooting direction automatically faces the direction in which the person exists in cooperation with a known mechanism such as motion detection.

As the image acquisition device 2, for example, a video camera such as a surveillance camera is assumed, but the configuration and type of the image acquisition device 2 are not necessarily limited as long as they can acquire time-series images. The image acquisition device 2 may be, for example, a night-vision camera such as an infrared camera capable of shooting at night, or a camera capable of acquiring a distance image (Time Of Flight (TOF) camera, stereo camera, etc.). .. The image acquisition device 2 includes a wired or wireless communication means (LAN interface (Local Area Network Interface), USB interface (Universal Serial Bus Interface), wireless communication module, etc.) for communicating with the movement route prediction device 100. ..

The movement route prediction device 100 acquires the movement trajectory information and action information of the person based on the time-series image acquired by the image acquisition device 2, and based on the acquired information, determines the future movement route of the person (tracking target person 3). Predict and provide (display, etc.) the predicted result to the user. The movement route prediction device 100 acquires the movement trajectory information based on the time-series image acquired by the image acquisition device 2. Further, the movement route prediction device 100 acquires the behavior information of a person by performing pattern matching with, for example, a motion model stored in advance for the time-series image acquired by the image acquisition device 2.

FIG. 2 shows an example of the hardware of the movement route prediction device 100. As shown in the figure, the movement route prediction device 100 has a processor 51, a main storage device 52, an auxiliary storage device 53, an input device 54, an output device 55, and a communication device 56. They are connected so as to be able to communicate with each other via the communication means of The movement route prediction device 100 can function as an information processing device.

The processor 51 is configured by using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), and the like. When the processor 51 reads and executes the program stored in the main storage device 52, all or a part of the functions of the movement route prediction device 100 are realized. The main storage device 52 is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a non-volatile semiconductor memory (NVRAM (Non Volatile RAM)), or the like, and stores programs and data.

The auxiliary storage device 53 is, for example, a hard disk drive or an SSD (Solid State Drive).
), Optical storage devices (CD (Compact Disc), DVD (Digital Versatile Disc), etc.), storage systems, IC cards, SD memory cards, optical recording media, and other recording media readers / writers. Programs and data stored in the auxiliary storage device 53 are loaded into the main storage device 52 at any time. The auxiliary storage device 53 may have a configuration independent of the movement route prediction device 100, such as network storage.

The input device 54 is a user interface that accepts external input, and is, for example, a keyboard, a mouse, a touch panel, or the like. In addition, for example, the movement route prediction device 100 may be configured to receive an external input via the communication device 56.

The output device 55 is a user interface that provides various information such as processing progress and processing results to the outside, and is, for example, a screen display device (liquid crystal display (Liquid Crystal Display), projector, graphic card, etc.), a printing device, or the like. .. In addition, for example, the movement route prediction device 100 may be configured to provide various information such as the processing progress and the processing result to the outside via the communication device 56.

The communication device 56 is a wired or wireless communication interface that realizes communication with other devices or elements such as the image acquisition device 2, and is, for example, a NIC (Network Interface Card), a wireless communication module, or the like. ..

FIG. 3 is a block diagram showing a function included in the movement route prediction device 100 and information (data) stored in the movement route prediction device 100. As shown in the figure, the movement route prediction device 100 includes an image acquisition unit 311, a movement locus information generation unit 312, an action information generation unit 313, a movement route prediction unit 314, a prediction result output unit 316, and a prediction model generation unit 315. It also has each function of the information storage unit 317. These functions are realized, for example, by the processor 51 of the movement route prediction device 100 reading and executing a program stored in the main storage device 52 and the auxiliary storage device 53. Further, these functions are realized by, for example, hardware (FPGA (Field-Programmable Gate Array), ASIC (Application Specific Integrated Circuit), etc.) included in the movement route prediction device 100. In addition to these functions, the movement route prediction device 100 may include, for example, an operating system, a device driver, a DBMS (DataBase Management System), and the like.

The information storage unit 317 uses various information used to realize the function of the movement route prediction device 100, specifically, time-series image 351 and movement locus information 352, action information 353, motion model information 354, prediction model 355, and prediction. The result 356 is stored. The information storage unit 317 manages such information by, for example, a file system or a DBMS.

Among the above functions, the image acquisition unit 311 acquires the time-series image 351 sent from the image acquisition device 2. The movement locus information generation unit 312 generates the movement locus information 352 based on the time-series image. The action information generation unit 313 generates the action information 353 based on the time-series image. The movement route prediction unit 314 predicts the movement route of the person (tracking target person 3) based on the movement locus information 352 and the action information 353. The prediction model generation unit 315 generates a prediction model 355 used for predicting the movement path of a person based on the movement locus information 352 and the action information 353. The prediction result output unit 316 generates information (prediction result 356) indicating the prediction result predicted by the movement route prediction unit 314 and outputs it to the output device 55.

FIG. 4 is a flowchart illustrating a process (hereinafter, referred to as a movement route prediction process S400) performed by the movement route prediction device 100 when predicting a movement route of a person. Hereinafter, the movement route prediction process S400 will be described with reference to the figure.

As shown in the figure, first, the image acquisition unit 311 of the movement route prediction device 100 acquires (receives) the time-series image 351 sent from the image acquisition device 2 (S411). It is desirable that the image acquisition unit 311 receives the time-series image 351 from the image acquisition device 2 in real time, but the time-series image 351 has a time lag as long as it does not interfere with the prediction of the movement path of the person described later. May be received from the image acquisition device 2.

Subsequently, the movement locus information generation unit 312 of the movement route prediction device 100 performs a process of generating the movement locus information 352 of the person based on the time-series image 351 (hereinafter, referred to as the movement locus information generation process S412). Further, the action information generation unit 313 of the movement route prediction device 100 performs a process of generating the action information 353 (hereinafter, referred to as an action information generation process S413). The movement route prediction device 100 may perform the movement locus information generation process S412 and the action information generation process S413 as parallel processing, or may be performed as sequential processing in which one is completed and the other is started.

FIG. 5 is a flowchart illustrating the details of the movement locus information generation process S412. Hereinafter, the movement locus information generation process S412 will be described with reference to the figure.

As shown in the figure, the movement locus information generation unit 312 first acquires the movement locus of a person based on the time-series image 351 acquired in S411 of FIG. 4 (S511). The movement locus information generation unit 312 uses, for example, a method of tracking a feature point of a person, a method of capturing the movement of a person using optical flow and clustering, a method of tracking a person using a color histogram, and the like. To get. To acquire the position coordinates of each point constituting the movement locus, for example, the position estimation algorithm, the image analysis algorithm, and the video analysis known for the time-series image are used as parameters such as the installation position of the image acquisition device 2, the shooting direction, and the lens magnification. This is done by applying an algorithm or the like.

Subsequently, the movement locus information generation unit 312 connects the loci about the same person acquired from the time-series image 351 of each image acquisition device 2. First, the movement locus information generation unit 312 identifies an area in which a person is captured (hereinafter, referred to as a person area) from the time-series image 351 acquired from each image acquisition device 2 (S512). The movement locus information generation unit 312 specifies a person area by, for example, a method in which the image acquisition device 2 cuts out a predetermined range with reference to a feature point of the person, a method using a known person detection algorithm, or the like.

Subsequently, the movement locus information generation unit 312 obtains a feature amount (hereinafter, referred to as a person feature amount) for each of the specified person areas (S513). The movement locus information generation unit 312 uses, for example, a color histogram, an edge feature amount, a feature amount obtained by a machine learning method such as deep learning, and the like as a person feature amount. For example, information (metadata) such as the orientation of a person and the posture of a person may be used as a person feature amount. Further, for example, a feature amount for each part of a person such as a person's hand or foot may be used as a person feature amount.

Subsequently, the movement locus information generation unit 312 obtains the degree of similarity between the person feature quantities (S514). The movement locus information generation unit 312 uses, for example, a method based on vector distances such as Euclidean distance and cosine distance, and a method of adaptively changing the distance according to a situation by using a machine learning method such as distance learning (Metric Learning). The degree of similarity between personal features is obtained by such means.

Subsequently, the movement locus information generation unit 312 connects the movement loci of the person by combining the movement loci based on the time-series images having a high degree of similarity between the person feature quantities (the degree of similarity exceeds a preset threshold value). S515). Regarding the range not shown in any of the image acquisition devices 2, for example, the start point and end point of the movement locus of each time-series image to be connected, the time of the start point and the time of the end point, and the person can move. Complement the movement trajectory with information about the range. For example, the movement locus information generation unit 312 arranges the movement loci in chronological order, and connects the end point of the movement locus before the time and the start point of the locus after the time with the shortest distance for the adjacent movement loci. Complement what is as a movement trajectory.

The information storage unit 317 stores the movement locus obtained as described above as the movement locus information 352 (S516).

According to the above processing, it is possible to acquire the movement locus of the person (tracking target person 3) over a wide range of the monitoring target area based on the time-series images acquired from the plurality of image acquisition devices 2. Therefore, for example, it is possible to easily realize a mechanism for predicting the movement route of a person in the entire facility to be monitored.

FIG. 6 shows an example of the movement locus information 352 generated by the movement locus information generation unit 312. As shown in the figure, the movement locus information 352 is composed of one or more records having each item of locus ID 611, image acquisition device ID 612, local coordinates 613, world coordinates 614, and acquisition date and time 615. Each record corresponds to the place where the person existed (each point constituting the movement locus) identified from the time series image.

Among the above items, the locus ID 611 is set with an identifier (hereinafter, referred to as a locus ID) assigned to each locus obtained by the movement route prediction device 100. The same locus ID is assigned to the movement locus of the same person. The entire movement locus is represented by connecting the points represented by the records with the same locus ID.

In the image acquisition device ID 612, an identifier (hereinafter, referred to as an image acquisition device ID) of the image acquisition device 2 that provides the time-series image that is the acquisition source of the contents of the record is set.

In the local coordinates 613, the position coordinates of the points represented by the record are set in the coordinate system (hereinafter, referred to as local coordinates) individually set for each image acquisition device 2.

In the world coordinates 614, the position coordinates of the points represented by the record are set in the coordinate system (hereinafter referred to as world coordinates) set uniformly (commonly) for the plurality of image acquisition devices 2. Will be done. In the present embodiment, the local coordinates and the world coordinates are set as two-dimensional coordinates (plane coordinates), but the local coordinates and the world coordinates may be three-dimensional coordinates. The conversion between the local coordinates and the world coordinates can be performed by, for example, a transformation determinant (conversion map) generated based on the installation position, shooting direction, lens magnification, etc. of each of the plurality of image acquisition devices 2.

The acquisition date and time 615 is set to the date and time when the time-series image that is the basis of the record is acquired by the image acquisition device 2 (the date and time acquired from the time stamp attached to the time-series image).

For example, in a period in which the movement locus changes drastically (a period in which the number of changes per unit time is large), a record of movement locus information 352 is generated at short time intervals, and a period unit time in which the time change of the locus is small is generated. If the records of the movement locus information 352 are generated at long time intervals during the period (the period during which the number of occurrences of the change is small), the increase in the data size of the movement locus information 352 can be suppressed.

FIG. 7 is a flowchart illustrating the details of the action information generation process S413 shown in FIG. Hereinafter, the behavior information generation process S413 will be described with reference to the figure.

First, the action information generation unit 313 of the movement route prediction device 100 uses the same method as S512 in FIG. 5 described above to capture a person from the time-series images of each image acquisition device 2 (hereinafter referred to as a person area). ) Is specified (S711).

Subsequently, the action information generation unit 313 detects information on the movement of the person for each of the specified person areas (S712). For example, the action information generation unit 313 extracts skeleton information (skeleton information) by performing image processing on a person area, and "lifts an object" from information related to joint movement acquired from the extracted skeleton information. Is detected. Further, for example, the action information generation unit 313 detects the motion of "prowling" because it "reciprocates in a narrow area".

Subsequently, the action information generation unit 313 identifies the action of the person for each person area based on the action detected for each person area (S713). The action information generation unit 313 identifies the action by comparing a series of actions or combinations of actions of the person with the action model information 354.

FIG. 8 shows an example of the operation model information 354. As shown in the figure, the motion model information 354 is composed of one or more records in which the motion or combination of motions of a person (motion 811) and the action 812 are associated with each other. The content of the operation model information 354 is set by the user, for example, via a user interface configured by using the input device 54 and the output device 55. Further, for example, the movement route prediction device 100 automatically generates the content of the motion model information 354 by using an image that performs an motion until reaching a certain action as learning data and automatically specifying the action based on the learning data. You may do so.

When the action information generation unit 313 detects both the "lifting" motion and the "bagging" motion from the person area based on the motion model information 354, the action of "shoplifting" based on the motion model information 354, for example. Identify as. Further, when the action information generation unit 313 detects an action of "reciprocating in a narrow area", it identifies the action as a "lost child" based on the action model information 354.

Returning to FIG. 7, the information storage unit 317 subsequently stores the action specified by the action information generation unit 313 as the action information 353. As a result, the action information generation process S413 of FIG. 4 is completed.

FIG. 9 shows an example of action information 353. As shown in the figure, the action information 353 is composed of one or more records having each item of the locus ID 911, the action 912, the occurrence coordinate 913, and the occurrence date and time 914.

Among the above items, the above-mentioned locus ID is set in the locus ID 911. The locus ID associates the action with the movement locus of the person who caused the action (a record in which the same locus ID as the locus ID 611 of the movement locus information 352 is set).

The action specified by the above-mentioned action information generation process S413 is set in the action 912. In this example, "shoplifting", "lost child", and "invasion of the exclusion zone" are set in the action 912.

Position coordinates represented by world coordinates indicating the place where the action of the record is performed are set in the generation coordinates 913.

The date and time when the action is performed (date and time obtained from the time stamp attached to the time-series image) is set in the occurrence date and time 914.

Returning to FIG. 4, the movement route prediction unit 314 of the movement route prediction device 100 subsequently compares the contents of the movement locus information 352 and the behavior information 353 with the contents of the prediction model 355, thereby causing a person (tracking target person). Predict the movement route of 3) (S414).

FIG. 10 shows an example of the prediction model 355. The method of generating the prediction model 355 will be described later.

As shown in the figure, the prediction model 355 is composed of one or more records having each item of prediction ID 1011, action 1012, occurrence coordinate 1013, occurrence date and time 1014, prediction movement route 1015, and probability 1016.

Among the above items, the prediction ID 1011 is set with an identifier of the prediction result (each record of the prediction model 355).

Each item of the action 1012, the occurrence coordinate 1013, and the occurrence date and time 1014 is set with the same contents as the action 912, the occurrence coordinate 913, and the occurrence date and time 914 in the action information 353. The action 1012, the occurrence coordinate 1013, and the occurrence date and time 1014 specify the predicted movement route 1015 and the probability 1016 based on the behavior (contents of the action information 353) of the person (tracking target person 3) who intends to predict the movement route. It becomes a key item at the time.

Information indicating a predicted movement route (hereinafter, referred to as a predicted movement route) is set in the predicted movement route 1015 (information for identifying the predicted movement route 1015 in FIG. 11 in this example (route ID described later)). To.

FIG. 11 shows an example of the information set in the predicted movement route 1015. As shown in the figure, the predicted movement route 1015 is set with information consisting of one or more records including each item of the route ID 1111, the coordinates 1112, and the order 1113. Of these, the route ID 1111 is set with an identifier of the predicted movement route (hereinafter, referred to as a route ID) (the same route ID as the predicted movement route 1015 of the prediction model 355 is set). The position coordinates of the points on the predicted movement path are set in the coordinates 1112. Information indicating the order in which the points are connected is set in the order 1113. By arranging the points represented by each record according to the order 1113, the entire predicted movement route is represented.

Returning to FIG. 10, the probability 1016 is set with information indicating the probability that the person (tracking target person 3) will pass the predicted movement route. The prediction model is generated by generating records of the prediction model 355 at short time intervals during a period in which the movement locus changes drastically, and by generating records of the prediction model 355 at long time intervals in a period in which the time change of the locus is small. It is possible to suppress an increase in the data size of 355.

Returning to FIG. 4, in S414, the movement route prediction unit 314 contrasts the action information 353 with the prediction model 355, and records the action taken by the person (tracked person 3) (for example, the action of the action information 353). (Records in which the combination of 912, occurrence coordinates 913, and occurrence date and time 914 match) are extracted from the prediction model 355.

For example, when the locus ID 911 of the action information 353 in FIG. 9 is a record of "1", the movement route prediction unit 314 predicts a record in which "shoplifting" set in the action 912 is set in the action 1012. Extract from. In this example, the records of the first row and the second row (the record having the prediction ID 1011 of "1" and the record having the prediction ID 1011 being "2") are extracted from the records shown in FIG.

The occurrence coordinates 913 and the occurrence date and time 914 do not necessarily have to completely match the occurrence coordinates 1013 and the occurrence date and time 1014 of the prediction model 355. For example, the match / mismatch is determined with a predetermined width (allowable range). You can. By using the date and time of occurrence for the determination of match / mismatch, it is possible to appropriately predict the movement locus according to the situation at the site such as the degree of congestion that changes depending on the time zone.

Returning to FIG. 4, in the following S415, the prediction result output unit 316 of the movement route prediction device 100 is a person (tracking target person 3) based on the contents of the record of the prediction model 355 extracted by the movement route prediction unit 314 in S414. Information indicating the prediction result of the movement route of the above is generated as the prediction result 356, and the generated prediction result 356 is output to the output device 55.

FIG. 12 is an example of the prediction result 356 output to the output device 55 (hereinafter, referred to as a prediction result display screen 1200). As shown in the figure, the prediction result display screen 1200 shows the prediction result 356 superimposed on the map information (overhead map in the store, etc.) around the area where the person (tracking target person 3) moves. In the figure, reference numeral 1201 indicates the position where the action (shoplifting) was performed, reference numeral 1202 indicates the current position of the person (tracking target person 3), and reference numeral 1203 indicates the movement trajectory of the person (tracking target person 3). ing. Further, the broken lines indicated by reference numerals 1204 and 1205 indicate the predicted movement paths of the persons (tracked persons 3), respectively. The broken line indicated by reference numeral 1204 is the predicted movement path 1015 of the record whose prediction ID 1011 in FIG. 10 is "1", and "80%" which is the content of the probability 1016 is also displayed. Further, the broken line indicated by the reference numeral 1205 is the predicted movement path 1015 of the record whose prediction ID 1011 is “2” in FIG. 10, and “20%” which is the content of the probability 1016 is also displayed.

Based on the prediction result display screen 1200 shown in the figure, a user such as a monitor can know that a person (tracking target person 3) is likely to move along a movement route indicated by reference numeral 1205 in the future. it can. Therefore, for example, it can be determined that it is better to monitor the line along the movement route.

[Generation of prediction model]
FIG. 13 is a flowchart illustrating a process of generating the above-mentioned prediction model 355 (hereinafter, referred to as a prediction model generation process S1300). Hereinafter, the prediction model generation process S1300 will be described with reference to the figure.

The prediction model generation unit 315 of the movement route prediction device 100 acquires the movement loci for a plurality of persons from the movement locus information 352 and the action information for a plurality of persons from the action information 353 (S1311, S1312). Subsequently, the prediction model generation unit 315 associates the acquired movement locus with the action information using the locus ID as a key (S1313). Subsequently, the prediction model generation unit 315 learns about the associated data. Learning is performed by discovering statistical properties such as the date and time of occurrence of an action, the coordinates of occurrence, and the subsequent movement path of a person (S1314). Subsequently, the prediction model generation unit 315 stores the learning result as the prediction model 355 (S1315). This completes the prediction model generation process S1300.

By the way, in the first embodiment described above, the behavior information 353 is used as the information about the person, but the attribute information (adult, male, female, child, young person, old man, etc.) may be used as the information about the person. .. By generating the prediction model 355 using the attribute information as the information about the person (for example, specifying the "wandering" of the "child" (attribute information) as the "lost child"), the prediction accuracy of the movement locus can be improved. .. In the movement route prediction system 1 described above, an image acquisition device 2 (video camera or the like) for acquiring a time series image 351 is used as a sensor for acquiring the movement trajectory information 352 and the action information 353. By utilizing this, for example, the movement route prediction device 100 may be provided with a function (attribute information generation unit) for generating attribute information such as the age and gender of a person based on the time series image 351.

[Second Embodiment]
FIG. 14 shows the configuration of the movement route prediction system 1 of the second embodiment. As shown in the figure, the movement route prediction system 1 of the second embodiment includes one or more mobile information terminals 4, one or more relay devices 6, and a movement route prediction device 100. In the movement route prediction system 1 of the first embodiment, an image acquisition device 2 such as a video camera is used as an example of a sensor for acquiring the movement trajectory information and the behavior information of the person (tracking target person 3), but the second embodiment Then, the portable information terminal 4 is used as the sensor.

The mobile information terminal 4 is, for example, a smartphone, a mobile phone, or a PDA (Personal Digital).
Assistant) etc. In this embodiment, it is assumed that a person carries the mobile information terminal 4 with him / her. The mobile information terminal 4 has a function of outputting information indicating its own position (position measurement function by wireless LAN (Wi-Fi or the like), GPS (Global Positioning System) or the like). The relay device 6 is a device for communicably connecting the mobile information terminal 4 and the movement route prediction device 100, and is, for example, a router for a wireless LAN. Since the hardware of the movement route prediction device 100 is the same as that of the first embodiment, the description thereof will be omitted.

FIG. 15 is a block diagram showing a function included in the movement route prediction device 100 of the second embodiment and information (data) stored in the movement route prediction device 100 of the second embodiment. Unlike the first embodiment, the movement route prediction device 100 of the second embodiment includes a terminal position information acquisition unit 321 instead of the image acquisition unit 311. The terminal position information acquisition unit 321 acquires the information indicating the current position of the mobile information terminal 4 sent from the mobile information terminal 4. Unlike the first embodiment, the information storage unit 317 stores information indicating the current position of the mobile information terminal 4 acquired by the terminal position information acquisition unit 321 as time-series position information 361.

The movement locus information generation unit 312 of the second embodiment generates the movement locus information 352 based on the time series position information 361. The action information generation unit 313 of the second embodiment generates the action information 353 by inputting a setting from the user, for example. Other configurations of the movement route prediction device 100 of the second embodiment are basically the same as those of the movement route prediction device 100 of the first embodiment.

The movement route prediction device 100 of the second embodiment identifies each mobile information terminal 4 by, for example, an identifier (MAC address, IP address) given to each mobile information terminal 4. When using an IP address, for example, the IP address may change if it goes out of the communicable area of the relay device 6, so for example, the area to be monitored should be within the communicable area. ..

As described above, even when the mobile information terminal 4 is used as the sensor, it is possible to realize a configuration that predicts the movement route of the person (tracking target person 3). Further, in the movement route prediction system 1 of the second embodiment, the movement route prediction device 100 does not need to perform image processing, and the load of the movement route prediction device 100 is reduced as compared with the movement route prediction system 1 of the first embodiment. be able to.

[Third Embodiment]
FIG. 16 shows the configuration of the movement route prediction system 1 of the third embodiment. As shown in the figure, the movement route prediction system 1 of the third embodiment has a configuration for providing the prediction result 356 to the mobile information terminal 4 in addition to the configuration provided in the movement route prediction system 1 of the first embodiment. Including.

The prediction result output unit 316 of the movement route prediction device 100 of the third embodiment generates a screen in a format that can be displayed on the mobile information terminal 4 (hereinafter, prediction result display screen 1700) as a screen for displaying the prediction result 356. .. The communication device 56 of the movement route prediction device 100 of the third embodiment wirelessly communicates with the mobile information terminal 4 via a communication means (mobile phone network, Internet, wireless LAN, etc.), and displays the prediction result display screen 1700 on the mobile information terminal. Send to 4. Other configurations of the movement route prediction device 100 of the third embodiment are basically the same as those of the movement route prediction device 100 of the first embodiment.

FIG. 17 shows an example of the prediction result display screen 1700 output by the prediction result output unit 316 of the movement route prediction device 100 of the third embodiment. In the figure, reference numeral 1701 indicates the position where the action (shoplifting) was performed, reference numeral 1702 indicates the current position of the person (tracking target person 3), and reference numeral 1703 indicates the movement trajectory of the person (tracking target person 3). ing. Further, the broken lines indicated by reference numerals 1704 and 1705 indicate the predicted movement paths of the persons (tracked persons 3), respectively.

According to the movement route prediction system 1 of the third embodiment, for example, a user such as an observer efficiently picks up a person (tracking target person 3) while checking the prediction result display screen 1700 displayed on the mobile information terminal 4. Can be tracked. Further, the movement route prediction device 100 updates the contents of the prediction result display screen 1700 in real time based on, for example, a time-series image sent from the image acquisition device 2, so that the user can use a person (a person () based on the latest situation at the site. The tracked person 3) can be tracked.

By the way, it goes without saying that the present invention is not limited to the above embodiment and can be variously modified without departing from the gist thereof. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. Further, it is possible to add / delete / replace a part of the configuration of the above embodiment with another configuration.

Further, each of the above configurations, functional units, processing units, processing means and the like may be realized by hardware by designing a part or all of them by, for example, an integrated circuit. Further, each of the above configurations, functions, etc. may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files that realize each function can be placed in a memory, a hard disk, a recording device such as an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

Further, in each of the above figures, the control lines and information lines are shown as necessary for explanation, and not all the control lines and information lines in the implementation are necessarily shown. For example, in practice almost all configurations may be considered interconnected.

Further, the arrangement form of various functional units, various processing units, and various databases of the movement route prediction device 100 described above is only an example. The arrangement form of the various functional units, the various processing units, and the various databases can be changed to the optimum arrangement form from the viewpoints of the performance, processing efficiency, communication efficiency, and the like of the hardware and software included in the movement route prediction device 100.

Further, the above-mentioned database configuration (schema, etc.) can be flexibly changed from the viewpoints of efficient use of resources, improvement of processing efficiency, improvement of access efficiency, improvement of search efficiency, and the like.

1 Movement route prediction system, 2 Image acquisition device, 3 Tracked person, 4 Mobile information terminal, 6
Relay device, 100 movement route prediction device, 311 image acquisition unit, 312 movement trajectory information generation unit, 313 action information generation unit, 314 movement route prediction unit, 315 prediction model generation unit, 316 prediction result output unit, 317 information storage unit, 351 time series image, 352 movement locus information, 353 behavior information, 354 motion model information, 355 prediction model, 356 prediction result, S400 movement route prediction processing, S412 movement locus information generation processing, S413 behavior information generation processing, 1200 prediction result display Screen, S1300 Predictive model generation process, 170
0 Prediction result display screen

Claims (15)

A sensor information acquisition unit that acquires sensor information, which is information acquired by the sensor,
A movement history information generation unit that generates movement history information, which is information related to the movement history of a person, based on the sensor information.
An information storage unit that stores behavioral information and attribute information of a person, which is information about the behavior of a person ,
Based on the movement history information, the action information, and the attribute information, a prediction model including information that associates an action taken by a person with a predicted movement route that is a movement route in which the person is predicted to move is provided. a prediction model generation unit to be generated,
A movement route prediction unit that acquires the predicted movement route corresponding to the behavior information of the tracked person from the prediction model, and
A prediction result output unit that outputs the acquired information indicating the predicted movement route, and
A movement route prediction device. The movement route prediction device according to claim 1.
The prediction model generation unit includes an action taken by a person based on the movement history information, the action information, and the attribute information, a predicted movement route which is a movement route in which the person is predicted to move, and the person. A prediction model including information associated with the probability that the movement route predicted to move is the predicted movement route is generated.
The prediction result output unit outputs the acquired information indicating the predicted movement route and the probability of the predicted movement route.
Movement route prediction device. The movement route prediction device according to claim 1 or 2 .
It is provided with an action information generation unit that generates the action information based on the sensor information.
The movement route prediction unit acquires the predicted movement route corresponding to the behavior information of the tracking target person generated based on the sensor information from the prediction model.
Movement route prediction device. The movement route prediction device according to claim 1 or 2 .
An attribute information generation unit that generates the attribute information based on the sensor information is further provided.
Movement route prediction device. The movement route prediction device according to any one of claims 1 or 2 .
The sensor is one or more image acquisition devices.
The sensor information acquisition unit acquires time-series images from the sensor as the sensor information.
The movement history information generation unit generates the movement history information based on the time-series image.
Movement route prediction device. The movement route prediction device according to claim 4.
The sensor is one or more image acquisition devices.
The sensor information acquisition unit acquires a time-series image from the image acquisition device as the sensor information by communicating with the sensor.
The attribute information generation unit generates the attribute information based on the time-series image.
Movement route prediction device. The movement route prediction device according to claim 1 or 2 .
The sensor is one or more personal digital assistants having a function of acquiring the current position.
The sensor information acquisition unit acquires information indicating the current position of the mobile information terminal from the mobile information terminal as the sensor information.
Movement route prediction device. The movement route prediction device according to claim 2 .
Equipped with a communication device that communicates with mobile information terminals
The prediction result output unit generates a screen in a format that can be displayed on the mobile information terminal, which describes information indicating the predicted movement route and the probability of the predicted movement route, and displays the screen on the mobile information terminal. Send,
Movement route prediction device. Information processing device
Steps to acquire sensor information, which is information acquired by the sensor,
A step of generating movement history information, which is information on the movement history of a person, based on the sensor information.
Steps to memorize behavioral information and attribute information of a person, which is information about the behavior of a person ,
Based on the movement history information, the action information, and the attribute information, a prediction model including information that associates an action taken by a person with a predicted movement route that is a movement route in which the person is predicted to move is provided. Steps to generate ,
A step of acquiring the predicted movement route corresponding to the behavior information of the tracked person from the predicted model,
A step of outputting information indicating the acquired predicted movement route,
A movement route prediction method that executes. The movement route prediction method according to claim 9.
The information processing device
Based on the movement history information, the action information, and the attribute information, the action taken by the person, the predicted movement route which is the movement route where the person is predicted to move, and the predicted movement route where the person is predicted to move are predicted. A step of generating a prediction model including information associated with the probability that the movement route is the predicted movement route, and
A step of outputting the acquired information indicating the predicted movement route and the probability of the predicted movement route, and
A movement route prediction method that further executes. The movement route prediction method according to claim 9 or 10 .
The information processing device
A step of generating the action information based on the sensor information,
A step of acquiring the predicted movement route corresponding to the behavior information of the tracking target person generated based on the sensor information from the prediction model.
A movement route prediction method that executes. The movement route prediction method according to claim 9 or 10 .
The information processing device
A step of generating the attribute information based on the sensor information,
A movement route prediction method that further executes. The movement route prediction method according to any one of claims 9 or 10 .
The sensor is one or more image acquisition devices.
The information processing device
A step of acquiring a time-series image from the sensor as the sensor information,
A step of generating the movement history information based on the time-series image,
A movement route prediction method that further executes. The movement route prediction method according to claim 12.
The sensor is one or more image acquisition devices.
The information processing device
A step of acquiring a time-series image as the sensor information from the image acquisition device by communicating with the sensor.
A step of generating the attribute information based on the time-series image,
A movement route prediction method that further executes. The movement route prediction method according to claim 9 or 10 .
The sensor is one or more personal digital assistants having a function of acquiring the current position.
A step in which the information processing device acquires information indicating the current position of the mobile information terminal from the mobile information terminal as the sensor information.
A movement route prediction method that further executes.