JP2017182761A - Position identification device, position identification program, route specification device, and route specification program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To identify an image position which is common among a plurality of photography parts.SOLUTION: A movement trajectory of a user of facilities is stored in a storage device from photographic images of a first camera and a second camera (S102, S104). For the respective first camera 14 and second camera 16, a map of a probability showing a degree of likelihood of photographing of a user, having been photographed by one of the first camera 14 and second camera 16, by the other is generated and stored in the storage device (S110-S118). The probability map of the first camera which is stored and a probability map of the second camera are used to perform matching processing between the first camera 14 and second camera 16 (S120). Consequently, the need for detailed initial settings of setting relative position relation between the first camera and second camera and so on is eliminated.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a position identification device, a position identification program, a route identification device, and a route identification program.

  In Patent Document 1, the degree of similarity of a user corresponding area to a processing target image is obtained using each of images captured in different shooting ranges by a plurality of shooting units, and the user corresponding area is determined as a processing target image. Is disclosed that switches the processing target image to the switching destination image when the similarity of the user corresponding region in the switching destination image exceeds the similarity of the user corresponding region to the processing target image. ing.

  In Patent Literature 2, when detecting a detection target image such as a human face image from the processing target image captured by the imaging unit, a map indicating the existence probability distribution of the detection target image using the luminance of the processing target image is provided. A technique for generating a detection target image from a processing target image using the map generated is disclosed.

JP 2013-196199 A JP 2014-89626 A

  The present invention reduces the processing load for identifying the image positional relationship of a plurality of imaging units compared to the case of identifying the same object moving across a plurality of imaging units using the feature amount of the captured image. An object of the present invention is to provide a position identification device, a position identification program, a route identification device, and a route identification program that can be used.

  According to a first aspect of the present invention, a position identification device includes a plurality of photographing units that respectively photograph a predetermined photographing range, and movement trajectory information of a person included in a photographed image photographed by each of the plurality of photographing units in time series. A person is photographed at the image position for each image position of a photographed image by any one of the plurality of photographing units using the accumulation unit to be accumulated and the movement trajectory information accumulated in the accumulation unit. A derivation unit for deriving the probability that a person is photographed at each image position of a photographed image photographed by another photographing unit at the same time as the recorded time, and a plurality of photographing units based on the probability derived by the derivation unit And an identification unit that identifies an image position common to the captured images of each of the above.

  The position identification device according to a second aspect is the position identification device according to the first aspect, wherein the image position is grid information indicating a position of a grid obtained by dividing the photographed image in a predetermined shape in length and width.

  According to a third aspect of the present invention, there is provided the position identification device according to the first or second aspect, further comprising a storage unit that stores the probability derived by the deriving unit.

  The position identification program according to claim 4 accumulates, in a time series, movement trajectory information of a person included in a captured image captured by each of a plurality of imaging units that respectively capture a predetermined imaging range, Using the movement trajectory information stored in the storage unit, for each image position of the image captured by any one of the plurality of imaging units, at the same time as the time when the person was captured at the image position Deriving the probability that a person is photographed at each image position of the photographed image taken by another photographing unit, and identifying the common image position among the photographed images by each of the plurality of photographing units from the derived probability To cause the computer to execute a process including this.

  The route specifying device according to claim 5 is the same person for each person included in the captured image captured by each of the plurality of capturing units based on the probability derived by the deriving unit in the position identifying device. A calculation unit that calculates a certain probability for each person, and a movement locus of an imaging unit that has captured a captured image including a person whose probability of being the same person exceeds a predetermined threshold based on the probability calculated by the calculation unit A specifying unit that links information and specifies a movement path of a person who has moved across the plurality of imaging units.

  The route specifying device according to a sixth aspect is the route specifying device according to the fifth aspect, wherein the calculation unit has a common movement trajectory information of each person included in a photographed image photographed by each of the plurality of photographing units. The probability of being the same person is calculated with respect to the time to perform.

  The route specifying program according to claim 7 uses movement trajectory information in which movement trajectory information of a person included in a captured image captured by each of a plurality of imaging units that respectively capture a predetermined imaging range is accumulated in time series. In addition, for each image position of a photographed image by any one of the plurality of photographing units, a photographed image photographed by another photographing unit at the same time as the person was photographed at the image position. Based on the probability derived by the deriving unit for deriving the probability that a person is photographed at each image position, each person included in the photographed image photographed by each of the plurality of photographing units is the same person Probability is calculated for each person, and based on the calculated probability of being the same person, the movement trajectory information of the imaging unit that has taken a captured image including a person whose probability of being the same person exceeds a predetermined threshold is linked. Before To execute the processing including identifying the movement path of the person has moved across a plurality of the imaging unit to the computer.

  According to the position identification device of claim 1, the photographing positional relationship of the plurality of photographing units is compared with the case where the same object moving across the plurality of photographing units is identified using the feature amount of the photographed image. The load of the process to identify can be reduced.

  According to the position identification device of claim 2, the load required for the process of identifying the shooting positions of a plurality of shooting units is reduced as compared with the case where grid information indicating the position of the grid obtained by dividing the shot image is not used. Can do.

  According to the position identification device of the third aspect, it is possible to reduce the load required for the process of identifying the photographing positions of the plurality of photographing units as compared with the case where the derived probability is not stored in the storage unit.

  According to the position identification program of claim 4, as compared with the case where the same object moving across the plurality of imaging units is identified using the feature amount of the captured image, the imaging positional relationship of the plurality of imaging units It is possible to reduce the load of processing for identifying.

  According to the route specifying device of the fifth aspect, compared to the case where the same object moving across the plurality of imaging units is specified using the feature amount of the captured image, the movement is performed across the plurality of imaging units. It is possible to reduce the processing load for specifying the movement path of the same object.

  According to the route specifying device of the sixth aspect, the movement path of the same person between the plurality of photographing units is compared with the case where the probability of being the same person is not calculated at a common time when the person is photographed by the plurality of photographing units. The load required for the processing to be identified can be reduced.

  According to the route specifying program of the seventh aspect, compared to the case where the same object moving across the plurality of imaging units is specified using the feature amount of the captured image, the movement is performed across the plurality of imaging units. It is possible to reduce the processing load for specifying the movement path of the same object.

It is an image figure which shows an example of the environment which can use the position identification apparatus which concerns on 1st Embodiment. It is a block diagram which shows an example of the computer which can implement | achieve the control apparatus which concerns on 1st Embodiment. It is a flowchart which shows an example of the flow of the position identification process which concerns on 1st Embodiment. It is an image figure which shows an example of the grid frame set with respect to the picked-up image of the 1st camera which concerns on 1st Embodiment. It is an image figure which shows an example of the grid frame set with respect to the picked-up image of the 1st camera which concerns on 1st Embodiment. It is an image figure which shows an example of the tracking data accumulate | stored in the storage device which concerns on 1st Embodiment. It is a flowchart which shows an example of the flow of the pre-processing for producing | generating the probability map which concerns on 1st Embodiment. It is an image figure which shows an example of the movement locus | trajectory by the tracking data in a 1st camera. It is an image figure which shows an example of the movement locus | trajectory by the tracking data in a 2nd camera. It is an image figure which shows the process in which the movement locus | trajectory of each user in a 1st camera and a 2nd camera is connected. It is a flowchart which shows an example of the flow of the path | route identification process which concerns on 3rd Embodiment. It is an image figure which shows the process in which the detailed movement locus | trajectory of each user in a 1st camera and a 2nd camera is connected. It is a flowchart which shows an example of the flow of the path | route identification process which concerns on 4th Embodiment. It is an image figure which shows an example of a grid area | region with high correspondence with the grid of a 2nd camera and a 3rd camera in the grid frame set with respect to the picked-up image of a 1st camera.

  Hereinafter, an example of a position identification device according to an embodiment of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 shows a schematic diagram of an example of an environment in which a location identification device 10 according to the first embodiment can be used to photograph a facility such as a store with a plurality of photographing units as seen from the front side. In the first embodiment, a user who moves in a facility is tracked from a photographed image, and a photographing position common to a plurality of photographing units is identified.

  As shown in FIG. 1, the position identification device 10 includes a control device 12 to which a first camera 14 and a second camera 16 are connected. FIG. 1 shows an example of an environment in which the first camera 14 and the second camera 16 are installed on a ceiling in a facility 20 such as a store. In the first embodiment, the first camera 14 and the second camera 16 will be described using a photographing lens such as a fish eye based on the equidistant projection method so that the facility 20 can be photographed in a wide range. . However, the photographing lenses of the first camera 14 and the second camera 16 are not limited to photographing lenses such as fish eyes, and wide-angle lenses and standard lenses may be used.

  It is assumed that the shooting range of the first camera 14 and the shooting range of the second camera 16 are at least partially overlapped. However, the first camera 14 and the second camera 16 are installed with the shooting direction roughly determined without performing detailed initial setting of the camera such as setting processing of the angle of view or shooting range. For example, as shown in FIG. 1, when the second camera 16 is set, the field of view at the time of installation varies depending on the field angle θa or the field angle θb. Although the details will be described later, the detailed initial setting of the first camera 14 and the second camera 16 is not required by performing the matching process of the first camera 14 and the second camera 16.

  In the first embodiment, a case where two cameras, the first camera 14 and the second camera 16, are used will be described. However, the number of cameras is not limited to two, and may be three or more. In addition, the installation of the camera is not limited to the ceiling in the facility 20, and may be installed on a wall surface or may be installed on the floor surface in a self-supporting manner.

  FIG. 2 shows an example of a computer that can implement the control device 12 as a computer 30. The computer 30 includes a CPU 32, a RAM 34, a ROM 36, and an input / output port (I / O) 40, which are connected to each other via a bus 42. The I / O 40 is connected to a storage device 44, a display 46, an input device 48 such as a keyboard, and a timer 50 for notifying the current time. A first camera 14 and a second camera 16 are also connected to the I / O 40.

  The storage device 44, an HDD (Hard Disk Drive), a nonvolatile flash memory, or the like can be used. As will be described in detail later, a map storage unit 44M and a tracking data storage unit 44T are set in the storage device 44. The map storage unit 44M is an area for storing a probability map indicating the probability that the position in the captured image is a common position between the first camera 14 and the second camera 16. The tracking data storage unit 44T is an area for storing time-series data (coordinates and time information on the captured image) indicating the position of the user in the captured image. The ROM 36 also stores a control program 38 for causing the computer 30 to function as the control device 12. The CPU 32 reads the control program 38 from the ROM 36 and develops it in the RAM 34 to execute processing. Thereby, the computer 30 which executed the control program 38 operates as the control device 12 shown in FIG.

  Next, the operation of the computer 30 functioning as the control device 12 will be described.

  FIG. 3 shows an example of the flow of the position identification process of the computer 30 that functions as the control device 12 when the control program 38 is executed by the CPU 32. The CPU 32 executes the processing routine shown in FIG. 3 when instructed by the user or periodically.

  First, in step S <b> 100, the CPU 32 sets a grid frame for each of the first camera 14 and the second camera 16. Step S100 is executed to reduce the amount of calculation in the position identification process. In addition, when performing a position identification process in detail, the setting of a grid frame is unnecessary.

  FIG. 4 shows an example of the grid frame 15 having a plurality of grids G in the vertical and horizontal directions set for the captured image of the first camera 14. An image captured by the first camera 14 captured by a photographing lens such as a fish eye is substantially circular. The grid frame 15 set for the captured image of the first camera 14 is based on design data such as the lens distortion coefficient of the first camera 14 so that the areas of the grids G are the same on the actual plane. Determined. In this embodiment, in order to further reduce the amount of calculation in the position identification process, the grid frame 15 set for the captured image of the first camera 14 is a main region (predetermined main region ( A grid frame 15M set for the area in the thick line shown in FIG. 4 is used. FIG. 5 shows a grid frame 15 </ b> M set for the captured image of the first camera 14. In the following description, the grid frame set for the captured image of the second camera 16 will be described as the grid frame 17M.

  Next, in step S102 shown in FIG. 3, the captured images of the first camera 14 and the second camera 16 are acquired. In the next step S104, a user who newly appears in each captured image is detected by known image processing, a user ID is assigned, and tracking is started for the user of the assigned user ID. To do. This tracking is a process of acquiring a state in which the position on the captured image changes with the movement of the user as the movement locus of the user. Next, in step S <b> 106, the movement trajectory of the user present in the captured image is accumulated in the accumulation device 44.

  In this embodiment, as a user's movement trajectory, representative position coordinates where a predetermined part (for example, the head) of the user is present in the captured image are acquired in time series, and information indicating the movement trajectory is obtained. Tracking data is stored in the storage device 44. The tracking data is stored by associating coordinates on the captured image indicating the position of the user's head existing in the acquired captured image with the time information at which the captured image was acquired. That is, in step S106, the current time is acquired from the timer 50, and the current time is associated with the position coordinates of the user tracked by each of the first camera 14 and the second camera 16, and the storage device 44 is associated. To accumulate. Specifically, the tracking data is stored in the tracking data storage unit 44T set in the storage device 44. Accordingly, tracking data indicating the position of the user in the captured image in time series is accumulated.

  FIG. 6 shows an example of tracking data stored in the storage device 44. FIG. 6A shows tracking data in the first camera 14, and FIG. 6B shows tracking data in the second camera 16. FIG. 6 shows an example in which coordinates on the captured image indicating the position of the user's head are accumulated. Instead of the coordinates on the captured image, the grid G in the grid frame including the coordinates is shown. Information indicating the position of the image may be accumulated. 6A, the position of the grid G in the first camera 14 is indicated by “Ai”, and in FIG. 6B, the position of the grid G in the second camera 16 is indicated by “Bj”.

  FIG. 8 shows an example of the movement locus of the user (ID: A-001) based on the tracking data in the first camera 14, and FIG. 9 shows the user (ID: B-) based on the tracking data in the second camera 16. An example of the movement locus of (001) is shown. In FIGS. 8 and 9, the result of coordinate conversion from the design data such as the ratio of the vertical and horizontal equal ratio, that is, the grid frame, the distortion coefficient of the photographic lens, and the like so that each grid G is a quadrangle on the actual plane. The image which set the grid frame to the picked-up image of is shown. Moreover, in FIG. 8, the position of the user's head is indicated by the nodes Da1 to Da4. Moreover, in FIG. 9, the position of the user's head is indicated by nodes Db1 to Db4. In the example shown in FIGS. 8 and 9, the movement trajectory of the user (ID: A-001) based on the tracking data in the first camera 14 and the user (ID: B-001) based on the tracking data in the second camera 16 are used. A correlation with the movement trajectory cannot be found.

  Next, as shown in FIG. 3, in step S108, the CPU 32 determines whether the current state meets a predetermined condition. In the present embodiment, as a predetermined condition, a period in which tracking data is accumulated in the accumulation device 44, for example, a time period such as one week is defined. That is, when the final time tm (FIG. 6) has elapsed from the time t1 at which the tracking data is stored in the storage device 44, affirmative determination is made in step S108. If a negative determination is made in step S108, the process returns to step S102, and the tracking data accumulation is repeated.

  When an affirmative determination is made in step S108, a user who has taken one of the first camera 14 and the second camera 16 is compared with each of the first camera 14 and the second camera 16 in steps S110 to S118. On the other hand, a map showing the certainty of being photographed is created.

  First, in step S110, the probability that the user observed by the first camera 14 is observed by the second camera 16 is calculated for each grid unit of the grid frame 15M set in the captured image of the first camera 14. Specifically, when a user is observed on a specific grid Ai set in the captured image of the first camera 14 using the tracking data stored in the storage device 44, the captured image of the second camera 16 is displayed. The probability P (Ai, Bj) that the user was observed in the set specific grid Bj is calculated. The calculation of the probability P is performed for each of the plurality of grids set in the captured image of the first camera 14, that is, the process of step S110 is repeated until an affirmative determination is made in step S112.

  By doing in this way, the correspondence of the grids contained in the imaging area which the 1st camera 14 and the 2nd camera 16 overlap can be identified. That is, when a user stands at a certain position in the overlapping shooting area, the user has a certain grid (grid Ai) of the first camera 14 and a corresponding grid (grid) in the captured image of the second camera 16. Bj). That is, when a person is photographed on the grid Ai of the first camera 14, the same person must be photographed on the grid Bj in the photographed image of the second camera 16 at the same timing. Therefore, when the object is photographed at a certain position in the image photographed by one camera, when the probability that the object is photographed is determined for each divided region of the photographed image of the other camera, At the corresponding position, the probability that the object is photographed is detected with high probability. By detecting such a probability, it becomes possible to detect corresponding positions as the same position in images taken by a plurality of cameras.

  An example of the calculation of the probability P will be described. First, an arbitrary grid Ai set in the captured image of the first camera 14 is specified. In the identified grid Ai, the time when the user is observed is identified by the tracking data of the first camera 14. A grid in which the user is observed by the second camera 16 is derived from the tracking data of the second camera 16 that is the same time as the identified observation time. A value obtained by using the derived total number of grids by the second camera 16 as a denominator and using the derived number of grids as a numerator is defined as a value of the probability P (Ai, Bj). The above processing is performed for each of the plurality of grids set in the captured image of the first camera 14. Thereby, when a user is observed on the grid Ai in the first camera 14, a probability P (Ai, Bj) indicating the degree of probability that the user is observed in the grid Bj in the second camera 16 can be calculated.

  Next, the probability P (Bj, Ai) of the grid in the first camera 14 with respect to the grid in the second camera 16 is calculated. That is, as in step S110 and step S112, the user observed by the second camera 16 in the grid unit of the grid frame 17M set in the captured image of the second camera 16 is the first camera 14 in step S114. The process of calculating the observed probability is repeated until an affirmative determination is made in step S116.

  In the next step S118, the grid unit probability P (Ai, Bj) calculated in step S110 is used as the probability map of the first camera 14, and the grid unit probability P (Bj, Ai) calculated in step S114 is used as the second camera. 16 probability maps are generated, and each probability map is stored in the storage device 44. That is, a probability map indicating the probability that the position in the captured image is the same position between the first camera 14 and the second camera 16 is stored in the map storage unit 44M set in the storage device 44. Specifically, the probability of each of the all grids of the second camera 16 with respect to each grid of the first camera 14 is set as a probability map of the first camera 14, and all the grids of the first camera 14 with respect to each grid of the second camera 16. Is stored as a probability map of the second camera 16 in the map storage unit 44M.

  In the next step S120, after performing the matching process of the first camera 14 and the second camera 16 using the probability map of the first camera 14 and the probability map of the second camera 16 stored in the storage device 44, This processing routine ends.

  The matching process of the first camera 14 and the second camera 16 is a process of identifying a common positional relationship on the captured images of the first camera 14 and the second camera 16. That is, the grid pair common to the first camera 14 and the second camera 16 is identified. Specifically, in the probability map, the grid pair of the grid set for the first camera 14 and the grid set for the second camera 16 with the highest probability is identified as a grid pair in a common positional relationship. In this case, the processing may be performed by using the probability map of the first camera or the probability map of the second camera 16, but using both the probability map of the first camera and the probability map of the second camera 16, Can be specified. For example, a first group consisting of a plurality of grid pairs with a high probability according to the probability map of the first camera and a second group consisting of a plurality of grid pairs with a high probability according to the probability map of the second camera 16 are obtained, What is necessary is just to identify a grid pair with the highest probability among a group and a 2nd group as a grid pair in a common positional relationship.

  Through the above processing, the common area between the area of the captured image captured by the first camera 14 and the area of the captured image captured by the second camera 16 is determined based on the degree of certainty that the user has been observed. Regions can be identified. Therefore, it is not necessary to perform detailed initial settings such as setting the relative positional relationship between each of the first camera 14 and the second camera 16.

  The probability map of the first camera 14 and the probability map of the second camera 16 obtained as described above tend to have a higher probability with respect to the grid corresponding to an arbitrary region where the user stays. For example, in a facility such as a store, it is conceivable that a user or a store clerk who performs a checkout process stays in a place where the checkout is performed. For example, it is assumed that the first camera 14 is photographing a certain area in the store, and the second camera 16 is photographing an area overlapping therewith. At this time, it is assumed that the clearing place (the position where the so-called cash register is installed) is included in the shooting area of the second camera 16 but is outside the shooting area of the first camera 14. In such a case, if there is a person in the overlapping area of the first camera 14 and the second camera 16, the person is photographed at that position in the photographed image of the first camera 14. In the images taken by the second camera 16 at the same timing, people are detected not only at the corresponding positions in the overlapping area but also around the clearinghouse. In this way, the position where the person constantly stays may cause erroneous detection such that the simultaneous existence probability is detected even though it is not an overlapping region. Therefore, it is preferable to set the probability P to be low for the region where the user is expected to stay. In other words, it is assumed that the object moves across a plurality of cameras, and an object (such as a customer in a store in the case of this embodiment) that is a target for detecting such movement is being captured by a plurality of cameras. The position between the cameras is identified by the position detected at the same time, and erroneous probability detection by an object (such as a store clerk in this embodiment) that is supposed to stay in a narrow range is excluded. In the embodiment, it is possible to further improve the accuracy of position identification.

Also, in tracking data that is the user's movement trajectory, if the time-series user's position is extremely far away, it is highly likely that the image processing has been affected by noise, etc. It is preferable to exclude the movement trajectory including the position from the probability calculation.
Also, in tracking data that is the user's movement trajectory, even when the user's position has not changed for a long time, it is highly likely that the user's position has been affected by noise or the like in image processing, and is excluded from the probability calculation. It is preferable.

  Therefore, in the first embodiment, before generating a probability map, a process of lowering the probability P for an area where a user is expected to stay and a movement trajectory including the possibility of noise in tracking data are generated. Preprocessing is performed as processing to be excluded from the probability calculation.

  FIG. 7 shows an example of the flow of preprocessing for generating a probability map. The processing routine shown in FIG. 7 executes processing between step S108 and step S110 shown in FIG.

  In the preprocessing shown in FIG. 7, the CPU 32 sets a provision coefficient W for the probability calculation in step S130. The assigning coefficient W is a variable indicating the variance U of the probability P with respect to the grid, and is given when the probability P is calculated. In the next step S132, tracking data is acquired, and in the next step S134, a distance L between coordinates in the acquired tracking data is calculated. In the next step S136, data of position coordinates where the distance L exceeds a predetermined threshold and data of position coordinates indicating that the position of the user has not changed for a long time (L is almost zero) are excluded. This processing routine is finished.

  As a result, the probability P can be lowered with respect to the region where the user is expected to stay, and the movement trajectory including the possibility of noise in the tracking data can be excluded from the probability calculation.

[Second Embodiment]
Next, a second embodiment will be described. Since the second embodiment has the same configuration as that of the first embodiment, the same parts are denoted by the same reference numerals and detailed description thereof is omitted. In the second embodiment, it is assumed that the distance between the grids, that is, the actual distance between the grids set in the captured image is known in advance.

  In the second embodiment, the distance between the first camera 14 and the second camera 16 is derived in the process of step S120 shown in FIG. Specifically, a plurality of grid pairs identified by the matching process of the first camera 14 and the second camera 16 and common to the first camera 14 and the second camera 16 are extracted. For the plurality of extracted grid pairs, the actual distance between the known grids is used to calculate the distance between the first camera 14 and the second camera 16 by a known triangulation calculation described in JP 2013-168089 A or the like. Deriving the distance.

  Thereby, the process of measuring the distance between the first camera 14 and the second camera 16 can be omitted when the first camera 14 and the second camera 16 are installed.

[Third Embodiment]
Next, a third embodiment will be described. Since the third embodiment has the same configuration as that of the first embodiment, the same portions are denoted by the same reference numerals and detailed description thereof is omitted.

  The movement of the user by identifying the common area between the area of the captured image captured by the first camera 14 and the area of the captured image captured by the second camera 16 described in the above embodiment. Trajectories (tracking) can be connected. That is, using the probability map derived in the above embodiment, the movement trajectory in the captured image captured by the first camera 14 and the movement trajectory in the captured image captured by the second camera 16 can be connected.

  FIG. 10 shows a process of connecting the movement trajectory (tracking) of the user. 10A shows the movement trajectory of the user included in the captured image captured by the first camera 14, and FIG. 10B shows the movement of the user included in the captured image captured by the second camera 16. Show the trajectory. FIG. 10C shows a concept in which the movement locus (tracking) is connected when the user of the movement locus by the first camera 14 and the user of the movement locus by the second camera 16 are the same user. showed that.

  As shown in FIG. 10, the movement trajectory of the same user is connected across a plurality of images captured by each of a plurality of cameras by superimposing the captured images with the grid pairs identified from the probability map. can do. Accordingly, it is possible to specify the movement path of the person who has moved across the first camera 14 and the second camera 16.

  Therefore, in the present embodiment, the movement route of the person who has moved across the first camera 14 and the second camera 16 is specified using the probability map derived in the first embodiment. Hereinafter, a route specifying device for specifying a movement route of a person will be described in detail.

  Next, the operation of the computer 30 functioning as the route specifying device according to the present embodiment will be described.

  FIG. 11 shows an example of the flow of the route specifying process of the computer 30 in which the control program 38 is executed by the CPU 32 and the control device 12 functions as the route specifying device. The CPU 32 executes the processing routine shown in FIG. 11 when instructed by the user or periodically.

  First, in step S140, the CPU 32 stores the probability maps of the first camera 14 and the second camera 16 derived by executing the same processes as in steps S100 to S118 shown in FIG. That is, the probability of each grid of the second camera 16 with respect to each grid of the first camera 14 is defined as a probability map of the first camera 14, and each grid of the first camera 14 with respect to each grid of the second camera 16 is used. The probability is stored in the map storage unit 44M as a probability map of the second camera 16. Note that the process of step S140 is a process of deriving a probability map used for the route specifying process. If the probability map has been derived in advance, the process of step S140 is not necessary.

  In the next step S142, each captured image of the first camera 14 and the second camera 16 is acquired in the same manner as in step S104 and step S106 in FIG. 44 (stored in the tracking data storage unit 44T). Note that the process of step S142 is a process of continuously accumulating the movement trajectory as tracking data when a movement route including a user moving in real time is specified, and movement that has been accumulated (stored) in advance. When the user is specified from the trajectory, it is only necessary to acquire data from the tracking data storage unit 44T, and the process of step S142 is unnecessary.

  In the next step S144, one user R among the users observed by the first camera 14 is set. That is, in step S144, the tracking data of the first camera 14 is acquired from the tracking data storage unit 44T, and any of the acquired tracking data (total number Rmax) is extracted as the tracking data of the user R in the first camera 14. In step S144, the observation time in the tracking data of the user R is set to the observation time of the first camera 14.

  In the next step S146, one user V among the users observed by the second camera 16 is set. That is, in step S146, tracking data of the second camera 16 is acquired from the tracking data storage unit 44T, and any of the acquired tracking data (total number Vmax) is extracted as tracking data of the user V in the second camera 16. In step S144, the observation time in the tracking data of the user R is set to the observation time of the first camera 14.

  Next, in step S148, the tracking time of the user R in the first camera 14 set in step S144 and the tracking data of the user V in the second camera 16 set in step S146 have a common observation time. A range Tth is obtained. The time range Tth is a time range in which the user V is observed by the second camera 16 at the time when the user R is observed by the first camera 14, and the user R and the user V are the first camera 14 and There is a high possibility that the second camera 16 was commonly observed. Therefore, by calculating the certainty that the user R and the user V are commonly observed by the first camera 14 and the second camera 16 for the time range Tth, as compared to the case of obtaining all the observation times of the tracking data. Processing load can be suppressed.

Therefore, in the next step S150, using each probability map, the identity Mrv indicating the probability that the user R and the user V are the same person observed in common with the first camera 14 and the second camera 16 is obtained. Ask. The identity Mrv can be obtained, for example, by identity calculation using the following equations (1) and (2).

  In equation (1), the probability Pt indicating the probability that the user exists in common at the image position (grid Ai) by the first camera 14 and the image position (grid Bj) by the second camera 16 at time t is obtained. it can. Specifically, the node t in the tracking data is surely present in the grid Bj of the second camera 16 with respect to the user existing in the grid Ai of the first camera 14 obtained from the probability map of the first camera 14. The probability P (Ai, Bj) of the first term indicating the presence and the user existing in the grid Bi of the second camera 16 obtained from the probability map of the second camera 16 exists in the grid Aj of the first camera 14 The probability Pt is obtained by multiplying the probability P (Bj, Ai) of the second term indicating the certainty to be performed. Time t is the time within the time range Tth obtained in step S148.

  In the equation (2), it is probable that the user R observed on the movement path based on the tracking data of the first camera 14 and the user V observed on the movement path based on the tracking data of the second camera 16 are the same person. Identity Mrv can be determined. Specifically, the direct product of the probabilities Pt at each time t within the time range Tth obtained by the equation (1) is calculated. The obtained identity Mrv corresponds to the combination of the user R (tracking data) of the first camera 14 set in step S144 and the user V (tracking data) of the second camera 16 set in step S146. In addition, it is stored in the storage device 44.

  In the next step S152, it is determined whether or not the processes in steps S146 to S150 have been completed for all the users (tracking data) observed by the second camera 16. If a negative determination is made in step S152, the process returns to step S146, and the processes of steps S146 to S150 are executed for the user observed by the unprocessed second camera 16. On the other hand, if an affirmative determination is made in step S152, the process proceeds to step S154, and it is determined whether or not the value of identity Mrv exceeds a predetermined threshold Mo (Mrv> Mo). The threshold value Mo is a value obtained in advance by experiments as a criterion value for determining that the user R of the first camera 14 and the user V of the second camera 16 are the same person.

  If an affirmative determination is made in step S154 (Mrv> Mo), it is determined in step S156 that user R and user V are the same person, and the process proceeds to step S160. On the other hand, if a negative determination is made in step S154 (Mrv ≦ Mo), it is determined in step S158 that user R and user V are different persons, and the process proceeds to step S160. In step S156, the determination result is stored in the storage device 44 in association with the combination of the tracking data of the user R of the first camera 14 and the tracking data of the user V of the second camera 16.

  By the way, it is preferable that the user V observed by the second camera 16 as a user common to the user R observed by the first camera 14 is one user. That is, it is desirable that the user R and the user V have a one-to-one correspondence. Therefore, when it is determined in step 154 that a plurality of combinations of the user R and the user V are the same person, a combination having a high probability of being the same person is set from the plurality of combinations. For example, what is necessary is just to add the process which sets the combination of the user R and the user V from whom the value of identity Mrv becomes the maximum value to step S154.

  In the next step S160, it is determined whether or not the processing in steps S144 to S158 has been completed for all the users (tracking data) observed by the first camera 14. If a negative determination is made in step S160, the process returns to step S144, and the processes in steps S144 to S158 are executed for the user observed by the unprocessed first camera 14. On the other hand, if an affirmative determination is made in step S160, the process proceeds to step S162, and the tracking data of the user R of the first camera 14 determined to be the same person, the tracking data of the user V of the second camera 16, and To complete the present processing routine.

  FIG. 12 shows a process of specifying the user's movement path by connecting the movement trajectory (tracking) of the user. FIG. 12 is an example showing FIG. 10 in more detail, and FIG. 12A shows the movement trajectory TRR of the user R included in the captured image captured by the first camera 14, and FIG. Indicates the movement trajectory TRV of the user V included in the photographed image photographed by the second camera 16. FIG. 12C shows the movement path TR of the person who has moved over the shooting range of the first camera 14 and the second camera 16 by connecting the movement trajectories of the user R and the user V.

  As shown in FIG. 12A, the user R has moved from the node (for example, the position of the head) Da01 to the node Da07 in units of grids. Further, as shown in FIG. 12B, the user V has moved from the node (for example, the position of the head) Db01 to the node Db09 in units of grids. In the example shown in FIG. 12, the movement trajectory TRR based on the tracking data of the user R in the first camera 14 and the movement trajectory TRV based on the tracking data of the user V in the second camera 16 each including the common time range Tth of the observation times. showed that. In FIG. 12, the movement trajectory TRR of the user R of the first camera 14 in the time range Tth is shown in the range from the node Da04 to the node Da07, and the movement trajectory TRV of the user V of the second camera 16 is represented by the node Da04. It was shown in the range of ~ node Da04. A common area is identified between the area of the captured image captured by the first camera 14 and the area of the captured image captured by the second camera 16 from the degree of certainty that the user has observed. can do. This common area is shown as area Area.

  Here, the movement trajectory TRR included in the area Area shown in FIG. 12A and the movement trajectory TRV included in the area Area shown in FIG. 12B are used because the value of the identity Mrv exceeds the threshold Mo. It is determined that the person R and the user V are the same person. Therefore, the movement path TR of the user who is determined to be the same person can be formed by connecting the movement trajectory TRR and the movement trajectory TRV by overlapping the overlapping portions. This movement path TR can be specified as a movement path of a person who has moved across the imaging range of the first camera 14 and the second camera 16.

  As described above, in the present embodiment, the movement trajectory of the user R by the first camera 14 and the second camera from the degree of probability (probability map) that the user is commonly observed by a plurality of cameras. 16 and the movement trajectory of the user V can determine the same common user. Since the same user is determined using the probability map in this way, the processing load for identifying the moving route of the user moving across the imaging range of the first camera 14 and the second camera 16 is reduced. be able to.

  In the present embodiment, the probability map is stored in the storage device 44 and the tracking data is stored in the storage device 44 and then the tracking data is connected. However, the present invention is not limited to this. Absent. For example, the probability map and tracking data may be stored in advance in a storage device such as an HDD or a non-volatile flash memory, and the tracking data may be connected using the probability map and tracking data read from the storage device. In this case, step S140 shown in FIG. 11 can be omitted, and in each of steps S144 to S150, a process of reading the probability map or the tracking data from the storage device may be executed.

[Fourth Embodiment]
Next, a fourth embodiment will be described. Since the fourth embodiment has the same configuration as that of the third embodiment, the same parts are denoted by the same reference numerals and detailed description thereof is omitted.

  In the third embodiment, using the probability map, the movement trajectory in the captured image captured by the first camera 14 and the movement trajectory in the captured image captured by the second camera 16 are connected to each other, and the first camera 14 is connected. And the movement path | route of the user who moves across the 2nd camera 16 was specified. In this case, the identity Mrv using each probability map is obtained for all the cameras that photograph the inside of the facility, and the same common user is determined, so that the amount of calculation increases as the total number of cameras increases. To do. However, there may be a low probability that there is the same user that is common between cameras that photograph the inside of the facility. Therefore, in the present embodiment, the amount of calculation is reduced by suppressing the calculation processing when the probability that the same user common to the cameras is present is low.

  In the present embodiment, a case will be described in which the inside of the facility 20 is photographed by three cameras, the first camera 14, the second camera 16, and a third camera (not shown). In the following description, a third camera (not shown) may be described with reference to the third camera 18.

  Next, the operation of the computer 30 functioning as the route specifying device according to the present embodiment will be described.

FIG. 13 shows an example of the flow of the route specifying process according to the present embodiment.
First, in step S170, as in step S140 of FIG. 11, the CPU 32 stores probability maps of the first camera 14, the second camera 16, and the third camera 18, respectively. In the next step S172, as in step S142 of FIG. 11, the captured images of the first camera 14, the second camera 16, and the third camera 18 are acquired, and the movement trajectory of the user existing in the captured image is obtained. The data is accumulated in the accumulating device 44 (stored in the tracking data storage unit 44T).

  In the next step S174, using each probability map, a grid area having a high correspondence with the grid of the captured image of the other camera is set for each camera in each grid of the captured image of the own camera. For example, when obtaining the correspondence between the second camera 16 and the third camera 18 for the first camera 14, first, the probability P is equal to or greater than a predetermined threshold value using a probability map of the first camera 14 to the second camera 16. Find the grid. A region including a grid equal to or greater than the obtained threshold is set as a grid region 17GR having a high correspondence with the second camera 16 of the first camera 14. Next, using a probability map of the first camera 14 for the third camera 18, a grid having a probability P equal to or higher than a predetermined threshold is obtained. A region including a grid that is equal to or greater than the obtained threshold is set to a grid region 19GR having a high correspondence with the third camera 18 of the first camera 14.

FIG. 14 shows an example of a grid region having a high correspondence relationship with the grids of the second camera 16 and the third camera 18 in the grid frame 15M set for the captured image of the first camera 14.
As shown in FIG. 14, in each grid G of the first camera 14, a grid area 17GR having a high correspondence with the grid of the second camera 16, a grid area 19GR having a high correspondence with the grid of the third camera 18, The grid G that is not included in the grid G has a very low probability of including the movement path of the user moving across the first camera 14 and the second camera 16 or the second camera 16 and the third camera 18. Therefore, for example, when obtaining the identity Mrv of the user observed in common with the first camera 14 and other cameras, only when the nodes of the tracking data of the first camera 14 include the grid regions 17GR and 19GR, By calculating the identity Mrv, the amount of calculation can be suppressed.

  Next, in step S176 shown in FIG. 13, as in step S144 of FIG. 11, one user R is set out of the users observed by the first camera 14, and tracking data of the user R is extracted. Further, the observation time in the tracking data of the user R is set as the observation time of the first camera 14.

  In the next step S178, a camera having the possibility of co-occurrence in the tracking data of the user R is determined. That is, using the grid area set in step S174, grid area determination is performed in which a camera having a grid having a high correspondence with the grid included in the tracking data of the user R by the first camera 14 is executed. If the determination result of step S178 is the second camera 16, an affirmative determination is made in step S180, and the process proceeds to step S182. If the determination result of step S178 is the third camera 18, a negative determination is made in step S180, an affirmative determination is made in step S181, and the process proceeds to step S182. On the other hand, if the determination result of step S178 indicates that the correspondence between the second camera 16 and the third camera 18 is low, a negative determination is made in step S180 and step S181, and the process proceeds to step S195. In step S195, it is determined that there is a low possibility that the same user exists between the cameras, and the process proceeds to step S196.

  Next, in step S182, as in step S146 of FIG. 11, one user V is set among the users observed using the second camera 16 or the third camera 18 determined in step S178 as the corresponding camera. In the next step 184, as in step S148 of FIG. 11, the tracking data of the user R in the first camera 14 and the tracking data of the user V in the corresponding camera (the second camera 16 or the third camera 18). A time range Tth having a common observation time is obtained. Then, in the next step S186, the user R and the user V use the first camera 14 and the second camera using the respective probability maps and the equations (1) and (2), similarly to the step S150 in FIG. 16, the identity Mrv indicating the probability of being the same person observed in common is obtained.

  Next, in step S188, as in step S152 of FIG. 11, it is determined whether or not the processing of all users observed by the corresponding camera has been completed. If a negative determination is made, the process returns to step S182. On the other hand, if an affirmative determination is made in step S188, the process proceeds to step S190, and whether or not the value of identity Mrv exceeds a predetermined threshold value Mo (Mrv> Mo) as in step S154 of FIG. Judging. When an affirmative determination is made in step S190 (Mrv> Mo), similarly to step S192de and step S156 of FIG. 11, it is determined that user R and user V are the same person, and the process proceeds to step S196. On the other hand, if a negative determination is made in step S190 (Mrv ≦ Mo), it is determined in step S194 that the user R and the user V are different persons as in step S158 in FIG. 11, and the process proceeds to step S196.

  In step S196, as in step S160 of FIG. 11, it is determined whether or not the processing of all users observed by the first camera 14 has been completed. If a negative determination is made, the process returns to step S176. On the other hand, if an affirmative determination is made in step S196, the process proceeds to step S198, and the tracking data of the user R of the first camera 14 determined to be the same person as in step S162 in FIG. The processing data is terminated by connecting the tracking data of the user V.

  As described above, in the present embodiment, when determining the same user common to a plurality of cameras, an area having a high correspondence relationship with other cameras in advance with respect to the probability map or the grid frame of the camera. By predetermining, it is possible to reduce the processing load for specifying the movement route of the user who moves across a plurality of cameras.

  In this embodiment, the case where the present invention is applied to three cameras has been described as an example of a plurality of cameras. However, the number of cameras is not limited to three, and can be applied to four or more cameras. Needless to say.

  In the above-described embodiment, the case where a part of the shooting range of the camera overlaps has been described as an example, but the present invention may be applied to the case where the shooting range of the camera is included. That is, for example, the present invention can be applied to the case where the first camera 14 is used to take an image of the inside of a facility and the second camera 16 is used to take a detailed image of a part of the facility within the shooting range of the first camera 14.

  Although the present invention has been described in detail with respect to specific embodiments, the present invention is not limited to such embodiments, and various other embodiments can be taken within the scope of the present invention. is there.

  In each of the above embodiments, the processing performed by executing the program stored in the ROM 36 has been described. However, the processing of the program may be realized by hardware.

  Furthermore, the processing in each of the above embodiments may be stored and distributed as a program in a storage medium such as an optical disk.

10 Position identification device (an example of a position identification device)
12 controller 14 first camera 15 grid frame 15M grid frame 16 second camera 17M grid frame 20 facility 30 computer 32 CPU
34 RAM
36 ROM
38 control program 40 input / output port 42 bus 44 storage device 46 display 48 input device 50 timer

Claims (7)

A plurality of photographing units each for photographing a predetermined photographing range;
An accumulator that accumulates time-sequential movement trajectory information of a person included in a captured image captured by each of the plurality of imaging units;
Using the movement trajectory information stored in the storage unit, for each image position of a captured image by any one of the plurality of imaging units, the same time as the person was captured at the image position A derivation unit for deriving a probability that a person is photographed at each image position of a photographed image photographed by another photographing unit;
From the probability derived by the derivation unit, an identification unit that identifies an image position common among the captured images by each of the plurality of imaging units,
A position identification device comprising: The position identification device according to claim 1, wherein the image position is grid information indicating a position of a grid obtained by dividing the captured image in a predetermined shape in the vertical and horizontal directions. The position identification device according to claim 1, further comprising a storage unit that stores the probability derived by the deriving unit. Accumulating movement trajectory information of a person included in a captured image captured by each of a plurality of imaging units each capturing a predetermined imaging range in a time series,
Using the movement trajectory information stored in the storage unit, for each image position of a captured image by any one of the plurality of imaging units, the same time as the person was captured at the image position In addition, a probability that a person is photographed at each image position of a photographed image photographed by another photographing unit is derived,
From the derived probability, identify the common image position among the captured images by each of the plurality of imaging units,
A position identification program for causing a computer to execute processing including the above. Each person included in a photographed image photographed by each of the plurality of photographing units based on the probability derived by the deriving unit in the position identification device according to any one of claims 1 to 3. A calculation unit for calculating the probability of being the same person for each person;
Based on the probability calculated by the calculation unit, the movement trajectory information of the imaging unit that captured a captured image including a person whose probability of being the same person exceeds a predetermined threshold is connected to the plurality of imaging units. A specifying unit for specifying a movement path of a person who has moved across;
A path specifying device comprising: The route specifying according to claim 5, wherein the calculation unit calculates the probability of being the same person for a time that is common in movement trajectory information of each person included in the captured image captured by each of the plurality of imaging units. apparatus. Using the movement trajectory information obtained by chronologically accumulating the movement trajectory information of a person included in a captured image captured by each of a plurality of imaging units that respectively capture a predetermined imaging range, For each image position of a photographed image taken by any one photographing unit, a person is photographed at each image position of a photographed image photographed by another photographing unit at the same time as the person was photographed at the image position. Based on the probability derived by the deriving unit for deriving the probability of being, for each person included in the captured image captured by each of the plurality of imaging units, to calculate the probability of being the same person for each person,
Based on the calculated probability of being the same person, the movement trajectory information of the imaging unit that has captured the captured image including the person whose probability of being the same person exceeds a predetermined threshold is connected to the plurality of imaging units. A route specifying program for causing a computer to execute processing including specifying a moving route of a person who has moved across.