JP2013253779A - Article position estimation system, article position estimation method, and article position estimation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an article position estimation system capable of determining an entry position, a current position or an exit position of a person having no tag in a case where there exist plurality of such persons.SOLUTION: An article position estimation system includes: a camera 101 and a tag reader 102 as observation devices for obtaining an observation value by detecting an observation-target article existing in an environment; a positional likelihood calculation part 104 for calculating a positional likelihood of each observation value being the article; an article position estimation part 105 for estimating the position of the article; a correlation value calculation part 107 for calculating a correlation value between an observation value and the article; an entry/exit article candidate detection part 92 for detecting an article candidate that has entered into or exited from an observation zone for each observation device; and an entry/exit article determination part 110 for determining an article that has entered into or exited from the environment.

Description

  The present invention relates to an object position estimation system, an object position estimation method, and an object position estimation program that estimate the position of an object such as a person or an animal that exists within an observation range of an observation device such as a sensor.

  Conventionally, as a technology for detecting the position of an object, the positioning data of a tag reader (RFID positioning device) and a camera (security camera) are compared (overlapped), and a tag is attached to a location where both the positioning data of the tag reader and the camera are aligned. There is a person who possesses a person and judges that there is a suspicious person who does not have a tag at a place where only the positioning data of the camera is present (see Patent Document 1).

JP 2006-311111 A

  However, in Patent Document 1, since the camera does not have a function of tracking a detected person and an ID identification function, when there are a plurality of suspicious persons who do not have a tag, from which position the suspicious person enters, There was a problem that it was impossible to determine what kind of trajectory was traced and from which position the car left.

  Therefore, the present invention solves the above-described problem, and even when there are a plurality of persons who do not have a tag, the position of each person or the current position (when the object position estimation process is performed) An object of the present invention is to provide an object position estimation system, an object position estimation method, and an object position estimation program capable of determining a leaving position.

  In order to achieve the above object, the present invention is configured as follows.

According to the first aspect of the present invention, the observation is performed by observing a plurality of objects existing in the environment, and outputting the observation position of each of the plurality of objects and the likelihood information of the ID for each object as an observation value. Equipment,
A position likelihood calculating unit that calculates likelihood information of the position of each observation value with respect to each object based on the estimated position and the observation position of each object;
A correlation value calculation unit that calculates a correlation value between the observed value and the estimated position of each object based on the likelihood information of the ID and the likelihood information of the position;
Based on the correlation value, an entry / exit object candidate detection unit that detects a candidate for an object that has entered the observation range of the observation apparatus or an object that has left the observation range;
Based on the output result of the entry / exit object candidate detection unit, an entry / exit object determination unit that determines an object that has entered the observation range of the observation device or an object that has left the observation range of the observation device;
Based on the determination result of the entry / exit object determination unit, based on the estimated position of each object, the likelihood information of the ID, and the likelihood information of the position, the presence at the position of each object at the time when the observed value was obtained An object position estimating unit that calculates a probability as an estimated position of each object;
An object position estimation system is provided.

According to the tenth aspect of the present invention, a plurality of objects existing in the environment are respectively observed by the observation device, and the observation position of each of the plurality of objects and the likelihood information of the ID for each object are observed values. After getting as
Based on the estimated position and the observed position of each object by the position likelihood calculating unit, the position likelihood information for each object of each observed value is calculated,
A correlation value calculation unit calculates a correlation value between the observed value and the estimated position of each object based on the likelihood information of the ID and the likelihood information of the position,
Based on the correlation value, the entry / exit object candidate detection unit detects an object that has entered the observation range of the observation apparatus or an object candidate that has left the observation range,
Based on the output result of the entry / exit object candidate detection unit, the entry / exit object determination unit determines an object that has entered the observation range of the observation device or an object that has left the observation range of the observation device,
The object position estimation unit, based on the determination result of the entry / exit object determination unit, the estimated position of each object and the likelihood information of the ID and the likelihood information of the position at the time when the observed value was obtained An object position estimation method is provided, wherein the existence probability at the position of each object is calculated as an estimated position of each object.

According to an eleventh aspect of the present invention, a computer
After observing a plurality of objects existing in the environment with an observation device and obtaining the observation position of each of the plurality of objects and ID likelihood information for each object as observation values,
A function for calculating position likelihood information of each observation value for each object based on the estimated position and observation position of each object by a position likelihood calculating unit;
A function for calculating a correlation value between the observed value and the estimated position of each object based on the likelihood information of the ID and the likelihood information of the position by a correlation value calculating unit;
Based on the correlation value, an entry / exit object candidate detection unit detects a candidate for an object that has entered the observation range of the observation device or an object that has left the observation range;
Function for determining an object that has entered the observation range of the observation device or an object that has left the observation range of the observation device based on the output result of the entry / exit object candidate detection unit by the entry / exit object determination unit When,
The object position estimation unit, based on the determination result of the entry / exit object determination unit, the estimated position of each object and the likelihood information of the ID and the likelihood information of the position at the time when the observed value was obtained A function of calculating the existence probability at the position of each object as the estimated position of each object;
An object position estimation program for realizing is provided.

  According to this configuration, even when there are a plurality of persons who do not have a tag, it is possible to estimate the entry position, the current position, or the exit position of each person with an ID.

  According to the object position estimation system, the object position estimation method, and the object position estimation program of the present invention, regardless of whether the tag is possessed or not possessed, a plurality of persons' approach positions or current positions (when the object position estimation process is performed) Alternatively, the leaving position can be estimated.

The block diagram which shows the structure of the object position estimation system which concerns on 1st Embodiment of this invention. The block diagram which shows the structure of the camera of the object position estimation system which concerns on 1st Embodiment of this invention. The block diagram which shows the structure of the approaching object candidate calculation part of the object position estimation system which concerns on 1st Embodiment of this invention. The block diagram which shows the structure of the leaving object candidate calculation part of the object position estimation system which concerns on 1st Embodiment of this invention. The block diagram which shows the structure of the entrance / exit determination part of the object position estimation system which concerns on 1st Embodiment of this invention. The figure explaining the observation condition in the room as a living space which is the environment where the observation target exists in the object position estimation system according to the first embodiment of the present invention. The flowchart which shows the process of the object position estimation system which concerns on 1st Embodiment of this invention. The figure of the color feature-value table for calculating ID likelihood with the camera which is an example of the observation apparatus of the object position estimation system which concerns on 1st Embodiment of this invention. The figure of a person ID conversion table required in order to acquire person ID from tag ID with the tag reader which is an example of the observation apparatus of the object position estimation system which concerns on 1st Embodiment of this invention. The flowchart which shows the calculation process of ID likelihood with the camera which is an example of the observation apparatus of the object position estimation system which concerns on 1st Embodiment of this invention. The figure which shows the observation condition of the camera which is an example of the observation apparatus of the object position estimation system which concerns on 1st Embodiment of this invention. The figure which shows the calculation condition of the data association value of the object position estimation system which concerns on 1st Embodiment of this invention. The figure which shows an example of the observation history database of the object position estimation system which concerns on 1st Embodiment of this invention. The flowchart which shows the approach determination process in the sensing range by the tag reader which is an example of the observation apparatus of the object position estimation system which concerns on 1st Embodiment of this invention. The flowchart which shows the leaving determination process outside the sensing range by the tag reader which is an example of the observation apparatus of the object position estimation system which concerns on 1st Embodiment of this invention. The figure which shows an example of the position estimation log | history database of the object position estimation system which concerns on 1st Embodiment of this invention. The figure which shows an example of the position estimation log | history database of the object position estimation system which concerns on 1st Embodiment of this invention. The figure which shows an example of the position estimation log | history database of the object position estimation system which concerns on 1st Embodiment of this invention. The figure which shows an example of the calculation condition of the correlation value of the object position estimation system which concerns on 1st Embodiment of this invention. The figure which shows an example of the calculation condition of the correlation value of the object position estimation system which concerns on 1st Embodiment of this invention. The figure which shows the example of the object position estimation by the Kalman filter of the object position estimation system which concerns on 1st Embodiment of this invention. The figure which shows the example of the movement locus | trajectory of the person which the display apparatus of the monitoring system using the object position estimation system concerning 1st Embodiment of this invention projected. The figure which shows an example of the observation history database in which the observation value only of a camera is recorded in the object position estimation system concerning 1st Embodiment of this invention.

  Embodiments according to the present invention will be described below in detail with reference to the drawings.

  DESCRIPTION OF EMBODIMENTS Hereinafter, various embodiments of the present invention will be described before detailed description of embodiments of the present invention with reference to the drawings.

According to the first aspect of the present invention, the observation is performed by observing a plurality of objects existing in the environment, and outputting the observation position of each of the plurality of objects and the likelihood information of the ID for each object as an observation value. Equipment,
A position likelihood calculating unit that calculates likelihood information of the position of each observation value with respect to each object based on the estimated position and the observation position of each object;
A correlation value calculation unit that calculates a correlation value between the observed value and the estimated position of each object based on the likelihood information of the ID and the likelihood information of the position;
Based on the correlation value, an entry / exit object candidate detection unit that detects a candidate for an object that has entered the observation range of the observation apparatus or an object that has left the observation range;
Based on the output result of the entry / exit object candidate detection unit, an entry / exit object determination unit that determines an object that has entered the observation range of the observation device or an object that has left the observation range of the observation device;
Based on the determination result of the entry / exit object determination unit, based on the estimated position of each object, the likelihood information of the ID, and the likelihood information of the position, the presence at the position of each object at the time when the observed value was obtained An object position estimating unit that calculates a probability as an estimated position of each object;
An object position estimation system is provided.

According to the second aspect of the present invention, the entry / exit object candidate detection unit includes:
Based on the correlation value, an approaching object candidate detection unit that detects a candidate for an object that has entered the observation range of the observation device;
Based on the correlation value, a retreated object candidate detection unit that detects a candidate for a retreated object from within the observation range of the observation device;
An object position estimation system according to the first aspect is provided.

According to the third aspect of the present invention, an observation history database that records the observation values together with the time at which each object was observed;
A position estimation history database that records the calculation result of the estimated position of each object together with the time when each object was observed;
The object position estimation system according to the first or second aspect is further provided.

According to a fourth aspect of the present invention, a plurality of the observation devices are provided,
The approaching object candidate detection unit is
A first estimated position number comparison unit that compares the number of observed values for each observation device with the estimated number of positions of an object;
When the number of observed values is larger than the number of estimated positions of the object based on the comparison result in the first estimated position number comparing unit, the observation device determines the difference number between the number of observed values and the estimated number of positions of the object. An approaching object candidate number calculating unit that calculates the number of detected approaching object candidates;
For each observation value, an approaching object that determines that an approaching object has been detected at the observation position of the observation value by the number of approaching object candidate counts calculated by the approaching object candidate number calculation unit in order of increasing correlation value. A determination unit, and
The leaving object candidate detection unit
A second estimated position number comparison unit that compares the number of observation values for each observation device with the estimated number of positions of the object;
When the number of estimated positions of the object is larger than the number of observed values based on the comparison result in the second estimated position number comparing unit, the observation device calculates the difference number between the estimated position number of the object and the number of observed values. A retreat object candidate number calculation unit that calculates the number of detected retirement object candidates;
For each estimated position of each object, a retired object determining unit that determines that the object has retreated by the number of retired object candidates calculated by the retired object candidate number calculating unit in descending order of the maximum correlation value. An object position estimation system according to the second aspect is provided.

According to the fifth aspect of the present invention, at least one observation device is constituted by a camera,
At least one other observation device is composed of a tag reader,
Further, it includes an entry / exit tag determination unit that determines entry / exit of a tagged object based on the observation value of the tag reader and the estimated position of each object, and records the determination result in the position estimation history database,
The correlation value calculation unit, the entry / exit object candidate detection unit detection unit, and the entry / exit object determination unit estimate the observation value output by the camera and the object previously determined to have entered by the entry / exit object determination unit. An object position estimation system according to any one of the first to fourth aspects, wherein only the position is used.

According to the sixth aspect of the present invention, the entry / exit object determination unit comprises:
An entering object number calculating unit for calculating an average value or a maximum value of the number of entering object candidates output by the entry / exit object candidate detecting unit for each observation device;
A first clustering unit that clusters all the entering objects detected by the entering object candidate detecting unit with respect to positions such that the number of clusters is the number of entering objects calculated by the entering object number calculating unit;
An ID likelihood sum calculating unit that calculates a sum for each ID for the ID likelihood included in the observed values of the approaching objects belonging to the cluster clustered by the first clustering unit;
It is determined that the ID object having the highest ID likelihood has entered the average position of the cluster among the sums of ID likelihoods for the object that is calculated by the ID likelihood sum calculation unit and has no estimated position. An object entry determination unit;
A moving object number calculating unit for calculating an average value or a maximum value of the number of moving object candidates output for each observation device by the moving object candidate detecting unit;
A second clustering unit that clusters all the leaving objects detected by the leaving object candidate detection unit with respect to positions such that the number of clusters is the number of leaving objects calculated by the leaving object number calculation unit;
The object position estimation according to any one of the first to third aspects, further comprising: a retreated object presence determination unit that determines that a retired object exists at an average position of each cluster clustered by the second clustering unit. Provide a system.

According to the seventh aspect of the present invention, at least one observation device is constituted by a camera,
The likelihood information of the ID for each object output as the observation value from the camera is not included in the observation history in addition to the object included in the observation history recorded in the observation history database. The object position estimation system according to the third aspect is characterized in that likelihood information of an object ID is included.

  According to an eighth aspect of the present invention, the entry / exit object candidate detection unit determines that an entry object has been detected at the observation position of each observation value when no estimated position of the object exists. The object position estimation system according to any one of the first to seventh aspects is provided.

  According to the ninth aspect of the present invention, the entry / exit object candidate detection unit, when no observation value is acquired as a result of observation by the observation device, all objects are within the observation range of the observation device. The object position estimation system according to any one of the first to eighth aspects is characterized in that it is determined that the user has left.

According to the tenth aspect of the present invention, a plurality of objects existing in the environment are respectively observed by the observation device, and the observation position of each of the plurality of objects and the likelihood information of the ID for each object are observed values. After getting as
Based on the estimated position and the observed position of each object by the position likelihood calculating unit, the position likelihood information for each object of each observed value is calculated,
A correlation value calculation unit calculates a correlation value between the observed value and the estimated position of each object based on the likelihood information of the ID and the likelihood information of the position,
Based on the correlation value, the entry / exit object candidate detection unit detects an object that has entered the observation range of the observation apparatus or an object candidate that has left the observation range,
Based on the output result of the entry / exit object candidate detection unit, the entry / exit object determination unit determines an object that has entered the observation range of the observation device or an object that has left the observation range of the observation device,
The object position estimation unit, based on the determination result of the entry / exit object determination unit, the estimated position of each object and the likelihood information of the ID and the likelihood information of the position at the time when the observed value was obtained An object position estimation method is provided, wherein the existence probability at the position of each object is calculated as an estimated position of each object.

According to an eleventh aspect of the present invention, a computer
After observing a plurality of objects existing in the environment with an observation device and obtaining the observation position of each of the plurality of objects and ID likelihood information for each object as observation values,
A function for calculating position likelihood information of each observation value for each object based on the estimated position and observation position of each object by a position likelihood calculating unit;
A function for calculating a correlation value between the observed value and the estimated position of each object based on the likelihood information of the ID and the likelihood information of the position by a correlation value calculating unit;
Based on the correlation value, an entry / exit object candidate detection unit detects a candidate for an object that has entered the observation range of the observation device or an object that has left the observation range;
Function for determining an object that has entered the observation range of the observation device or an object that has left the observation range of the observation device based on the output result of the entry / exit object candidate detection unit by the entry / exit object determination unit When,
The object position estimation unit, based on the determination result of the entry / exit object determination unit, the estimated position of each object and the likelihood information of the ID and the likelihood information of the position at the time when the observed value was obtained A function of calculating the existence probability at the position of each object as the estimated position of each object;
An object position estimation program for realizing is provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
In the object position estimation system (apparatus) 90 according to the first embodiment of the present invention, a camera and a UWB (Ultra Wide Band) tag reader (hereinafter simply referred to as a “closed environment”) are used. By installing a tag reader), even when there are a plurality of persons as an example of an object that does not have a tag, the approach position or the current (when performing the object position estimation process) of each person. The position or the leaving position is estimated with an ID.

  FIG. 1A is a diagram showing a configuration of an object position estimation apparatus 90 according to the first embodiment of the present invention.

  The object position estimation apparatus 90 according to the first embodiment of the present invention includes a camera 101 as an example of the observation apparatus 100, a tag reader 102 as another example of the observation apparatus 100, and an object position estimation processing unit 91. The object position estimation processing unit 91 includes an entry / exit tag determination unit 104, an object position estimation unit 105, a position likelihood calculation unit 107, a correlation value calculation unit 108, and an entry / exit object candidate detection unit 92 (entrance object candidate detection). Unit 109, a leaving object candidate detection unit 110), and an entry / exit object determination unit 111. Further, the object position estimation device 90 may include an observation history database 103 and a position estimation history database 106. Of course, the object position estimation apparatus 90 does not include the observation history database 103 and the position estimation history database 106, and can acquire necessary information from a database outside the object position estimation apparatus 90 by means of communication or the like. You may comprise. The necessary information may be acquired from the external database, and the necessary information may be held by an internal storage unit of each member, if necessary.

  The camera 101 and the tag reader 102, which are examples of the observation apparatus 100, each observe the inside of the observation area in the room every predetermined observation period (for example, 100 ms). Then, the camera 101 and the tag reader 102 detect a person existing in the observation area, and detect the detected position (observation position) of the person and the ID likelihood (ID likelihood information) of the person as a detection result. Is recorded in the observation history database 103.

  The entry / exit tag determination unit 104 determines entry / exit of the person carrying the tag into the observation area based on information recorded in the observation history database 103 and the position estimation history database 106.

  The position likelihood calculating unit 107 performs position likelihood for each person estimated to currently exist (at the time of position likelihood calculation) in the observation area with respect to each observation value recorded in the observation history database 103. Is calculated.

  The correlation value calculation unit 108 estimates that each observation value recorded in the observation history database 103 currently exists (when calculating the correlation value) in the observation area based on the information in the position estimation history database 106. A correlation value is calculated for each person.

  The entry / exit object candidate detection unit 92 may be configured by one unit, but may be configured by being divided into an entry object candidate detection unit 109 and a leaving object candidate detection unit 110. As an example, in the following description, an example in which an approaching object candidate detection unit 109 and a leaving object candidate detection unit 110 are divided into two parts will be described.

  The approaching object candidate calculation unit 109 calculates a candidate for a person who has entered the observation area based on the calculation result of the correlation value calculation unit 108. The leaving object candidate calculation unit 110 calculates a candidate for a person who has left the observation area based on the calculation result of the correlation value calculation unit 108.

  The entry / exit determination unit 111 records the results output by the approaching object candidate calculation unit 109 and the leaving object candidate calculation unit 110 in the position estimation history database 106.

  The object position estimation unit 105 estimates the position of a person existing in the observation area based on the information recorded in the position estimation history database 106.

  FIG. 2 shows a room 201 as a specific example of a living environment that is an example of a closed environment. The room 201 includes a camera 101 as an example of an observation apparatus 100 that is a component of the object position estimation system 90 according to the first embodiment of the present invention and a tag reader 102 as another example of the observation apparatus 100. Yes. Near the center of the quadrangular ceiling of the rectangular parallelepiped room 201, a camera 101 and a tag reader 102 are installed. It is assumed that a person 202A who possesses the tag 203A, a person 202B who possesses the tag 203B, and a person 203C who does not possess the tag 203 enter and exit the room 201 in the room 201. In addition, there is no restriction | limiting in the number of persons entering / exiting the room 201, and when showing arbitrary one, it demonstrates as the person 202 typically. Similarly, when an arbitrary tag is indicated, the description is given assuming that the arbitrary tag is the tag 203. Further, as an example of an entrance to the room 201 that is a closed environment, a door 204A, a door 204B, and a door 204C are arranged. Hereinafter, in the case where an arbitrary door is shown, the door 204 is described as a representative.

  Hereinafter, each component will be described using FIG. 1A, FIG. 1B, and FIG. 2 while corresponding to the flowchart of the object position estimation process of the object position estimation system 90 of FIG. FIG. 3 is a flowchart showing the overall processing of the object position estimation system 90.

<Explanation of observation equipment>
In the process of step S301, the observation apparatus 100 observes the inside of the room 201 every predetermined observation period (for example, 100 ms). Then, the observation apparatus 100 detects the person 202 existing in the room 201, and records the detection position of the person 202 and the ID likelihood of the person 202, which are detection results, in the observation history database 103 as observation values.

  The ID likelihood is a probability representation of which ID the detected object (for example, the person 202 in this case) is likely to be. For example, since the tag reader 102 can reliably identify the ID, when the tag reader 102 detects the tag of the object A, the ID likelihood is 1 for the object A and 0 for the other objects. . On the other hand, in the case of the camera 101, even if the ID identification result is the object A, since the possibility of an ID identification error by image processing is included, it is ensured that the actually detected object is A. It cannot be guaranteed. For example, even if the identification result of the camera 101 is the object A, there is a possibility that an object other than the object A (object B, object C, etc.) has been detected. In such a case, the ID likelihood is a probability value assigned to all objects that can be identified by the camera 101. For example, the ID likelihood is determined such that the probability of being an object A is 0.8, the probability of being an object B is 0.1, and the probability of being an object C is 0.1. This is an example of determining the ID likelihood, and the present invention is not limited to this.

<Explanation of camera>
Hereinafter, a specific method for calculating the detection position of the person 202 and the ID likelihood when the camera 101 is used as the observation apparatus 100 will be described. Here, as shown in FIG. 1B, the camera 101 includes an observation unit (imaging unit) 101a, an image processing unit 101b, an ID identification processing unit 101c, and an internal storage unit (storage unit) 101S.

  In order to detect the person 202 using the camera 101, the image processing unit 101c of the camera 101 needs to perform image processing on the image data acquired by the camera 101. As the method, for example, a background difference method can be used. The method is as follows. The image processing unit 101b compares the background image data prepared by capturing in advance with the camera 101 and the current image data (at the time of human detection) captured by the camera 101. The above-mentioned background image data is data of an environment when the person 202 is not present, for example, the background image of the room 201. Thereafter, the image processing unit 101b extracts areas having different pixel values as difference areas. Then, the image processing unit 101b detects the difference area as a person 202. However, since there is a possibility that the image data is mixed with noise, when the image processing unit 101b can determine that the difference area is sufficiently small with respect to the person 202, the image processing unit 101b It may be determined that the difference area is not the person 202. Here, the case where the difference area is sufficiently small with respect to the person 302 may be a case where the number of pixels in the difference area is equal to or less than a preset threshold value based on the minimum number of pixels that can be recognized as the person 202.

  Assuming that the camera 101 is installed so as to look down on the ground from the ceiling to the ceiling surface, the image processing unit 101b processes the barycentric position of the difference area acquired by the camera 101 as the detection position of the person 202. Note that the conversion from the pixel position in the camera coordinate system to the XY coordinate system in the room 201 in the image processing unit 101b can be realized by performing camera calibration in advance.

<Camera ID likelihood calculation: Color>
Further, the ID identification processing unit 101c of the camera 101 obtains the ID likelihood of the detected person 202 by performing ID identification processing on the difference area. As shown in detail in FIG. 1B, the ID identification processing unit 101c includes a comparison unit 101cb, a calculation unit 101cd, a tracking success probability acquisition unit 101ce, a data association value acquisition unit 101cf, an ID likelihood allocation unit 101cg, It is comprised with the addition part 101ch. As a method for obtaining the ID likelihood, for example, there is a method of matching color feature amounts. Specifically, the color storage amount of the person 202 to be identified is stored in advance in the internal storage unit 101S. Then, the comparison unit 101cb of the ID identification processing unit 101c compares the ratio of the color feature components included in the difference area. FIG. 4 shows an example of a color feature amount table in which color feature amount ratios of three persons who can enter and leave the room 201 (each ID is TAG_01, TAG_02, and TAG_03) are recorded. This color feature amount table is stored in the internal storage unit 101S. According to the color feature table of FIG. 4, for example, a person of TAG_01 has an R (red) component of 0.65, a G (green) component of 0.2, and a B (blue) component of 0.15. is there. Here, it is assumed that the ratio of the color feature amount in the difference area is 0.12 for the R component, 0.2 for the G component, and 0.68 for the B component. At this time, the comparison unit 101cb of the ID identification processing unit 101c compares the ratio of the color feature amount component of each person recorded in the color feature amount table with the ratio of the color feature amount component of the difference area. Then, the calculation unit 101cd of the ID identification processing unit 101c calculates the matching rate. For the person of TAG_01, the R component is matched at 0.12, the G component is matched at 0.2, the B component is matched at 0.15, and the matching is 0.47 in total. Similarly, the person with TAG — 02 matches with a total of 0.9, and the person with TAG — 03 matches with a total of 0.42. For example, the matching rate calculated by the calculation unit 101cd of the ID identification processing unit 101c described above can be used as the ID likelihood. That is, for the ID of the person 202 detected in the difference area, the ID likelihood for TAG_01 is 0.47, the ID likelihood for TAG_02 is 0.9, and the ID likelihood for TAG_03 is 0.42. This ID likelihood calculation processing is performed by the ID identification processing unit 101c for all detected people 202. It is assumed that the color feature amount table is recorded in the internal storage unit 101S of the camera 101.

<Camera ID likelihood calculation: tracking>
However, there may be a case where it is difficult to create the color feature amount table in advance, for example, the clothes worn by the person 202 are different every day. In such a case, the ID identification processing unit 101c may determine the ID likelihood according to the tracking state of the person 202. A specific description will be given while corresponding to the flowchart of FIG. 6, the camera observation state of FIG. 7, and the data association state of FIG. 8.

  In the process of step S601, the camera 101 observes the inside of the room 201 and detects N observation values (difference areas). However, N> 0 and N is an integer. Looking at the example of the camera observation state in FIG. 7, at time 2010: 3: 3: 14: 23: 28: 00, the camera 101 detects two observation values (observation values CAM_01 and CAM_02). See N) = 2. The method for detecting the difference area is as described above.

  In the process of step S602, the ID identification processing unit 101c of the camera 101 refers to the position estimation history database 106, and the person 202 who is currently estimated to exist in the room 201 (when performing the process of step S602). (M). However, M> 0 and M is an integer. Details of the position estimation history database 106 will be described later. The description using the flowchart of FIG. 6 is performed with M = 2.

  Thereafter, the process is performed on the observed values obtained in the process of step S601. Note that “A” in loop 1 represents an index number for each observation value, and the number is incremented from 0 in the order in which the observation values were obtained within the observation values obtained at the same time. Allocated to

  Next, in the process of step S603, the comparison unit 101cb of the ID identification processing unit 101c obtains the ratio of the color feature amount of the observed value detected by the camera 101. Looking at the example of the camera observation state in FIG. 7, the observed value CAM_01 has a ratio of R component: G component: B component = 0.3: 0.6: 0.1.

  In the process of step S604, the calculation unit 101cd of the ID identification processing unit 101c determines whether or not each of the observed values is an observed value being tracked. Note that the determination by the calculation unit 101 cd of the ID identification processing unit 101 c as to whether tracking is in progress is performed by matching the feature amount of the color acquired by the calculation unit 101 cd of the ID identification processing unit 101 c in step S602. The description of the matching processing method is omitted because it is equivalent to the ID likelihood calculation method using the color feature amount. For example, in the case of the observed value CAM_01, since the observed value was not obtained before the time 2010: 3: 3: 14: 23: 28: 00, the ID identification processing unit indicates that the observed value CAM_01 is not the observed value being tracked. It is determined by the calculation unit 101cd of 101c (proceeding to step S605). In the case of the observed value CAM_03, first, the calculation unit 101cd of the ID identification processing unit 101c determines whether there is an observed value that matches the color feature amount of the previous observed value (CAM_01, CAM_02). Specifically, when the calculation unit 101cd of the ID identification processing unit 101c determines that there is a previous observed value that is equal to or higher than the threshold and has the highest matching rate, the ID value is identified as the observed value CAM_03 being the observed value being tracked. This is determined by the calculation unit 101cd of the processing unit 101c. When the threshold value is set to 0.8, the observed value CAM_03 is determined by the calculation unit 101cd of the ID identification processing unit 101c that the observed value CAM_01 is being tracked (proceeds to step S606). When the threshold is set to 0.95, the observation value CAM_03 is determined not to be an observation value being tracked by the calculation unit 101cd of the ID identification processing unit 101c because there is no previous observation value that meets the condition. (Proceed to step S605). The threshold value is set empirically.

  In the process of step S605, the ID likelihood of all the persons 202 currently detected by the tag is added to the ID likelihood of the observed value determined by the calculation unit 101cd of the ID identification processing unit 101c as not being tracked in the process of step S604. Degrees are assigned. More specifically, as described above, the calculation unit 101cd of the ID identification processing unit 101c determines whether the observed value is being tracked. Then, the calculation unit 101cd of the ID identification processing unit 101c assigns 1 / M of the number of people 202 detected by the current tag as the ID likelihood of each of the people 202 detected by the current tag.

  In the process of step S606, the tracking success probability acquisition unit 101ce of the ID identification processing unit 101c acquires the tracking success probability (P) of the observation value of the camera 101. However, P> 0. As the tracking success probability, a probability obtained in advance by conducting a prior experiment may be used. In the process of step S604, the matching rate with the previous observation value determined to be tracking may be used as the tracking success probability.

  Next, in the process of step S607, the data association value acquisition unit 101cf of the ID identification processing unit 101c acquires the data association value assigned to the observation value that is the tracking source. The data association value is a value calculated by the data association value acquisition unit 101cf of the ID identification processing unit 101c during the process of the object position estimation unit 105. Further, the observed value is a value that stochastically indicates which person 202 is the observed value obtained by detecting. The object position estimation unit 105 will be described later. According to FIG. 7, the observed value of the observed source of the observed value CAM_03 is CAM_01 (determined by the color feature amount as described above). Further, according to FIG. 8, the data association value for TAG_01 of the observed value CAM_01 is 0.3, and the data association value for TAG_02 is 0.7.

  Next, in the process of step S608, (P / data association value) is assigned to the ID likelihood of each person 202 for the observation value determined to be tracked in the process of step S604, and the ID likelihood allocation of the ID identification processing unit 101c. The part 101cg is assigned. Assuming that the tracking success probability (P) is 0.8, in the case of the observed value CAM_03, (0.3 / 0.8) is the ID likelihood allocation unit of the ID identification processing unit 101c in the ID likelihood for TAG_01. The ID likelihood for TAG_02 is assigned (0.7 / 0.8) by the ID likelihood allocating unit 101cg of the ID identification processing unit 101c.

  Next, in the process of step S609, ((1-P) / M) is added to the ID likelihood for each person 202 allocated in the process of step S608 by the adder 101ch of the ID identification processor 101c. In the case of the observed value CAM_03, ((1-0.8) / 2) is added to the ID likelihood for TAG_01 and TAG_02 by the adder 101ch of the ID identification processor 101c.

<Description of tag reader>
Hereinafter, a specific calculation method of the detection position of the person 202 and the ID likelihood when the tag reader 102 is used in the observation apparatus 100 will be described. The tag reader 102 is configured to include an internal storage unit 102S.

  When the tag reader 102 is used as an example in FIG. 2, the detection position of the person 202 can be determined using, for example, the principle of three-point surveying. Specifically, the installation position of each antenna as a component of the tag reader 102 is stored in advance in the internal storage unit 102S of the tag reader 102 or the like. Then, a sphere having a radius of the distance to the tag 203 detected by each antenna is drawn with the installation position of each antenna as the center. At this time, a position where the spherical surfaces drawn by the antennas overlap most is a position where the tag 203 owned by the person 202 exists. Then, the tag reader 102 determines that this position is a position where the person 202 exists.

As described above, the tag reader 102 can directly read the ID of the person 202 recorded on the tag 203 from the tag 203. Thereby, the probability that the ID likelihood of the person 202 is the ID of the person 202 can be 1. When the ID of the person 202 is not recorded in the tag 203, for example, a person ID conversion table that can determine the ID likelihood of the person 202 from the tag ID may be used. An example of the person ID conversion table is shown in FIG. In the person ID conversion table of FIG. 5, for example, when a tag with tag ID = T4 is detected, it indicates that a person with person ID = TAG_01 has been detected. The person ID conversion table may be recorded in the internal storage unit 102S of the tag reader 102. Further, it is recorded in an external database (not shown) or the like, and the tag reader 102 acquires necessary information from the external database and determines the ID likelihood of the person 202 as necessary. Also good.

<Explanation of observation history database>
FIG. 9 shows an example of the observation history database 106. The observation history database 106 shown in FIG. 9 can record the observation ID, the observation time, the detection position, and the ID likelihood. The observation ID is an ID for identifying which observation device 100 detects the observed value at what time. The observation time is the time when the observation value is acquired. Note that each observation apparatus 100 includes a timer (not shown) therein. The detection position is a position where each observation device detects the person 202, and is a position in the XY coordinate system of the room 201 in FIG. The ID likelihood is a probabilistic representation of which ID the detected person 202 is likely to be.

  For example, observation ID = TAGR_01 is an observation value acquired at coordinates (280, 720) at time 2010: 3: 3: 14: 23: 27: 900, and the ID likelihood of TAG_01 is 1 from the detected tag. It is shown that. In FIG. 9, “−” is recorded in the detection position and ID likelihood of observation ID = TAGR_08, CAM_09. This indicates that the person 202 could not be detected as a result of observation. ing.

  Note that the camera 101 and the tag reader 102 are observing at an interval of 100 msec as an example, and the observed values are recorded in the observation history database 106 every 100 msec as an example. Further, “...” In FIG. 9 means that the observation value is omitted (actually recorded in the observation history database 106).

<Entry / exit tag determination unit>
In the process of step S302, the entry / exit tag determination unit 104 determines whether the person 202 carrying the tag 203 enters and leaves the room 201 based on the information recorded in the observation history database 103 and the position estimation history database 106. judge. As described above, the tag reader 102 does not make an identification error of the person 202 (the tag 203 possessed by the person 202). Therefore, the entry / exit tag determination unit 104 determines that the person with the ID acquired by the tag reader 102 exists in the room 201. Further, the entry / exit tag determination unit 104 determines that there is no person in the room 201 with an ID that the tag reader 102 could not acquire. Hereinafter, the process of step S302 will be described in detail.

  With reference to the observation history database 106 shown in FIG. 9 and the flowchart shown in FIG. 10, the entry tag determination by the entry / exit tag determination unit 104 will be described.

  In the process of step S1001, the tag reader 102 acquires N observation values. From the observation history database 103 shown in FIG. 9, it can be seen that at time 2010: 3: 3: 14: 23: 27: 900, the tag reader 102 has acquired N = 2 observation values.

  Next, in the process of step S1002, the entry / exit tag determination unit 104 determines the ID of the person who is currently estimated to exist in the room 201 (when performing the entry tag determination process, for example, the process of step S1002), It is acquired by referring to the position estimation history database 106.

  Thereafter, the process is performed on each observation value obtained in the process of step S1001. Note that “A” in loop 1 represents an index number for each observation value, and the number is incremented from 0 in the order in which the observation values were obtained within the observation values obtained at the same time. Allocated to

  Next, in the process of step S1003, based on the information acquired by the entrance / exit tag determination unit 104 in steps S1001 and S1002, it is estimated that the ID of the person 202 indicated by the detected tag 203 exists in the room 201. It is determined whether or not. Specifically, at the time immediately before the time 2010: 3: 3: 14: 23: 27: 900 at which the tag 203 is detected (the observation time of the tag reader 102 as an example of the previous observation device 100), the position estimation history database. It is determined by the entry / exit tag determination unit 104 whether or not 106 has information on the estimated position of the person with the ID. If the entry / exit tag determination unit 104 determines that the person with the ID does not exist, the process proceeds to step S1004. If the entry / exit tag determination unit 104 determines that a person with the ID exists, the process of step S1003 is performed on the next observation value.

  After loop 1, the process ends.

  On the other hand, in the process of step S1004, the entry / exit tag determination unit 104 records the ID of the person 202 indicated by the detected tag 203 and the information on the position where the tag 203 is detected in the position estimation history database 106. FIG. 12 shows the result of entry / exit tag determination unit 104 performing entry tag determination using the observation values of observation ID = TAGR_01 and TAGR_02 in FIG. This result shows an example of data stored in the position estimation history database 106. In the position estimation history database 106 of FIG. 12, the time recorded in the observation ID (the time when the tag 203 is detected) and the estimated position where the tag 203 exists are recorded. The estimated position is determined based on the position error characteristic of the tag reader 102. In the case of the tag reader 102 shown in the example of FIG. 12, it is indicated that there is a probability that a person indicated by the tag 203 conforming to a Gaussian distribution with a variance of 20 cm exists with the position where the tag 203 is detected as an average position. It is assumed that the position error characteristic is obtained in advance by conducting a preliminary experiment or the like. In FIG. 12, N (0, Q) represents a Gaussian distribution with an average of 0 and a variance Q. Then, go to loop 1.

  Next, the exit tag determination by the entrance / exit tag determination unit 104 will be described using the flowchart shown in FIG.

  In the process of step S1101, the tag reader 102 acquires an observation value.

  In the process of step S1102, the entry / exit tag determination unit 104 currently has the tag of the person 202 possessing the tag that is estimated to be present in the room 201 (when performing the process of exit tag determination, for example, the process of step S1102). The number of IDs is acquired by referring to the position estimation history database 106.

  Thereafter, the process is performed on each observation value obtained in the process of step S1001. Note that “A” in loop 1 represents an index number for each observation value, and the number is incremented from 0 in the order in which the observation values were obtained within the observation values obtained at the same time. Allocated to

  Next, in the process of step S1103, the entrance / exit tag determination unit 104 is currently in the room 201 (when performing the process of exit tag determination, for example, the process of step S1103) based on the information acquired in steps S1101 and S1102. It is determined whether or not the tag reader 102 has detected the tag 203 indicating the ID of the person 202 that is estimated to exist within the tag. Specifically, the entry / exit tag determination unit 104 is recorded in the position estimation history database 106 at a time immediately before the time when the tag 203 is detected (an observation time of the tag reader 102 which is an example of the previous observation apparatus 100). It is determined whether or not the tag 203 indicating the existing tag ID is detected by the tag reader 102. If the entry / exit tag determination unit 104 determines that the tag 203 indicating the tag ID has not been detected by the tag reader 102, the process proceeds to step S1104. If the tag reader 102 detects that the tag 203 indicating the tag ID is detected by the entry / exit tag determination unit 104, the process of step S1003 is performed on the next observation value.

  After loop 1, the process ends.

  On the other hand, in the process of step S1104, in the process of step S1103, the entry / exit tag determination unit 104 determines that the person 202 indicated by the tag 203 determined not to be detected by the tag reader 102 has left the room 201. The fact that the person 202 has left and does not exist in the room 201 may be recorded in the position estimation history database 106 so as to be expressed using a symbol such as “−”. FIG. 13 shows the position estimation history database 106 that represents a situation in which the person 202 is not present in the room 201.

  If there is a possibility that the tag reader 102 may miss the tag 203, for example, the process of step S1103 is NO for N times consecutively and the person 202 determined by the entry / exit tag determination unit 104 has left. The determination may be made by the entry / exit tag determination unit 104. Here, N> 0 and N is an integer.

<Description of Position Likelihood Calculation Unit>
In the process of step S303 in FIG. 3, the position likelihood calculating unit 107 calculates the current (object position estimation process, for example, step) for each observation value in the observation history database 103 based on the information in the position estimation history database 106. When the process of S303 is performed), the position likelihood is calculated for each person 202 estimated to exist in the room 201.

  The position likelihood is the position (observation) of the newly observed person 202 based on the estimated position of each person 202 in the position estimation history database 106 at present (the time when the observation device 100 acquires the observation value). In regard to (position), it is a probability representation of which ID 202 the person 202 is likely to be. For example, it is estimated that the person A exists at the position “10” on the one-dimensional coordinate, the person B is estimated at the position “20”, and the person C is estimated at the position “40”. To do. In this situation, it is assumed that a person is detected at the position “0”. The position likelihood at this time is obtained by, for example, obtaining the reciprocal of the distance from the estimated position of each person A, B, C in the position estimation history database 106 by the position likelihood calculating unit 107 and performing normalization. It can be calculated by the likelihood calculating unit 107. At this time, the position likelihood calculation unit 107 calculates the probability that the person A is 0.58, the probability that the person B is 0.28, and the probability that the person C is 0.14. This is an example of determining the position likelihood, and the present invention is not limited to this. For example, the position likelihood calculating unit 107 may obtain the position likelihood based on (Equation 1) in consideration of the variance-covariance matrix (in the predicted distribution) of the person 202 being estimated. POS_LL is position likelihood. x is an observation position, and μ is an estimated position of the person 202. Σ is a variance in the predicted distribution of the person 202. Details of the predicted distribution will be described in the object position estimation unit 105.

<Description of Correlation Value Calculation Unit>
Next, in the process of step S304, the correlation value calculation unit 108 calculates the current (object position estimation process, for example, for each observation value recorded in the observation history database 103 based on the information in the position estimation history database 106. When the process of step S304 is performed), a correlation value is calculated for each person 202 estimated to be present in the room 201.

  The correlation value is an observation value obtained by detecting which person 202 among each person 202 currently estimated (the time when the observation apparatus 100 acquires the observation value) for each observation value. It is a probability representation of whether or not there is. Specifically, the correlation value is a value obtained by multiplying the likelihood of ID by the likelihood of position, and the correlation value calculation unit 108 calculates correlation values for (the number of observed values × the number of persons). FIG. 15 shows an example of the correlation value between the observation value of the camera 101 and the person 202 acquired at time 2010: 3: 3: 14: 23: 28: 00.

<Description of the approaching object candidate calculation unit>
In the process of step S <b> 305, the approaching object candidate calculation unit 109 calculates a candidate for the person 202 who has entered the room 201 based on the calculation result of the correlation value calculation unit 108. As shown in FIG. 1C, the approaching object candidate calculation unit 109 includes a first estimated position number comparison unit 109a, an approaching object candidate number calculation unit 109b, and an approaching object determination unit 109c. The first estimated position number comparison unit 109a compares the number of observation values for each observation device with the estimated number of positions of the object. If the number of observed values is larger than the estimated number of positions of the object based on the comparison result in the first estimated position number comparing section 109a, the approaching object candidate number calculating section 109b is the difference between the observed value number and the estimated position number of the object. The number is calculated as the number of approaching object candidates detected by the observation apparatus 100. The approaching object determination unit 109c detects the approaching object at the observation position of the observation value by the number of approaching object candidates calculated by the approaching object candidate number calculation unit 109b in order of the maximum correlation value for each observation value. Judge. Here, the number of estimated positions of the object means the number of objects whose estimated positions are recorded in the position estimation history database 106 at the time when the observed value is obtained. Since the position estimation result at the time when the observed value was obtained has not yet been recorded in the position estimation history database 106 (because the position estimation is step S308 of the last process), the time when the previous observed value was obtained This represents the number of objects with the estimation result at. Therefore, in a specific example, the number of objects recorded as “-” in the position estimation history database 106 is not included in the number.

  Specific processing contents will be described using the observation history database 103 in FIG. 9, the position estimation history database 106 in FIG. 14, and the correlation values in FIG.

  First, the first estimated position number comparison unit 109 a of the approaching object candidate calculation unit 109 acquires the number of observation values of the camera 101 from the observation history database 103 via the correlation value calculation unit 108. For example, as shown in the observation history database 103 in FIG. 9, at the time 2010: 3: 3: 14: 23: 28: 00, the camera 101 acquires three observation values (CAM_03, CAM_04, CAM_05). Yes. The first estimated position number comparison unit 109a acquires this from the observation history database 103 via the correlation value calculation unit 108.

  Next, at the present time (when performing the object position estimation process, for example, the process of step S305), the first estimated position number comparison unit 109a uses the correlation value calculation unit 108 from the position estimation history database 106 to estimate the person being estimated. Get the number 202. For example, as shown in the position estimation history database 106 in FIG. 14, at time 2010: 3: 3: 14: 23: 27: 900, it is estimated that two persons (TAG — 01, TAG — 02) are in the room 201. (The latest position estimation result when the observed value at time 2010: 3: 3: 14: 23: 28: 00 is obtained is that at time 2010: 3: 3: 14: 23: 27: 900. Meaning).

  At this time, based on the information acquired by the first estimated position number comparison unit 109a, the approaching object candidate number calculation unit 109b of the approaching object candidate calculation unit 109 has one more observation value than the number of people 202. Calculate that. Then, the approaching object candidate number calculation unit 109b can determine that one person 202 has entered the room 201.

  When the approaching object candidate number calculation unit 109b determines that the person 202 has entered, the approaching object determination unit 109c of the approaching object candidate calculation unit 109 detects the observation value having the highest correlation value for each person 202. To associate. In the example of the correlation value in FIG. 15, TAG_01 is associated with the observation value CAM_03 by the approaching object determination unit 109c. TAG_02 is associated with the observed value CAM_04 by the approaching object determination unit 109c. The observed value CAM_05 is not associated with any person 202 by the approaching object determination unit 109c.

  Thereafter, the approaching object determination unit 109c determines that the observation value (CAM_05) that is not associated with any person 202 is an observation value obtained by observing the person 202 that has newly entered the room 201. . Then, the approaching object judging unit 109c of the approaching object candidate calculating unit 109 notifies the entering / leaving object judging unit 111 that there is a possibility that there is a person 202 who does not have the tag 203 at the position of the observed value CAM_05.

  Here, a case where a person 202 enters the room 201 for the first time will be described. In this case, it is estimated that no person 202 exists in the room 201. For this reason, even if an observed value is obtained, the estimated value of the person 202 does not exist, so the correlation value calculation unit 108 cannot calculate the correlation value. Therefore, when the observation value of the camera 101 is obtained only when no person 202 exists in the room 201, the entry / exit object determination unit 111 determines that the observation values are all in the room 201. It is determined that the person 202 has newly entered. FIG. 19 shows an example of the observation history database 103 in which observation values of only the camera 101 are recorded. This indicates that the tag reader 102 is not functioning in the observation history database 103 of FIG. In the situation where the observed value (CAM_11) and the observed value (CAM_12) are obtained at time 2010: 3: 3: 14: 23: 27: 900, the estimated position of the person 202 does not exist, so the observed value (CAM_11 ) And the observed value (CAM_12) can be determined by the entry / exit object determination unit 111 if there are persons 202 (NONTAG_01 and NONTAG_02) that have entered the room 201 at two positions (observation positions) at which the observed value (CAM_12) is obtained. Here, “NONTAG — 01” and “NONTAG — 02)” mean the ID of the person 202 who does not have the tag 203, respectively. Although details will be described later, it is the entry / exit object determination unit 111 that actually determines the addition of NONTAG — 01 and NONTAG — 02. Since the estimated position of the person 202 exists at time 2010: 3: 3: 14: 23: 28: 00, the association is performed based on the correlation value with the estimated position of each person 202 as described above. The entry / exit object determination unit 111 can determine that the observation value that has not been obtained is an observation value obtained by observing the person 202 who has newly entered the room 201.

  From the above, the operation is possible even in the situation where the observation value of the tag reader 102 does not exist. That is, the object position estimation apparatus 90 can operate even in a situation where the tag reader 102 and the entrance / exit tag determination unit 104 in the configuration diagram of FIG. 1A are omitted.

<Description of the leaving object candidate calculation unit>
In the process of step S306, the leaving object candidate calculation unit 110 calculates a candidate for the person 202 who has left the room 201 based on the calculation result of the correlation value calculation unit 108. Specific processing contents will be described using the observation history database 103 in FIG. 9, the position estimation history database 106 in FIG. 14, and the correlation values in FIG. As shown in FIG. 1D, the leaving object candidate calculation unit 110 includes a second estimated position number comparison unit 110a, a leaving object candidate number calculation unit 110b, and a leaving object determination unit 110c. The second estimated position number comparison unit 110a compares the number of observation values for each observation apparatus with the estimated number of positions of the object. When the estimated number of positions of the object is larger than the number of observed values based on the comparison result in the second estimated position number comparing unit 110a, the number of objects to be moved away calculating unit 110b is the difference number between the estimated number of positions of the object and the number of observed values Is calculated as the number of retreat object candidates detected by the observation apparatus 100. The withdrawing object determination unit 110c determines that the object has moved out by the number of withdrawing object candidates calculated by the withdrawing object candidate number calculation unit 110b in descending order of the maximum correlation value for the estimated position of each object.

  First, the number of observation values of the camera 101 is acquired from the observation history database 103 via the correlation value calculation unit 108 by the second estimated position number comparison unit 110a of the leaving object candidate calculation unit 110. For example, as shown in the observation history database 103 in FIG. 9, at the time 2010: 3: 3: 14: 23: 29: 900, the camera 101 acquires two observation values (CAM_01, CAM_02). The second estimated position number comparison unit 110a acquires this from the observation history database 103 via the correlation value calculation unit 108.

  Next, the number of persons 202 currently being estimated (when the object position estimation process, for example, the process of step S306 is performed) is compared with the second estimated position number from the position estimation history database 106 via the correlation value calculation unit 108. Obtained by the unit 110a. For example, as shown in the position estimation history database 106 in FIG. 14, it is estimated that three persons (TAG — 01, TAG — 02, NONTAG — 01) are in the room 201 at time 2010: 3: 3: 14: 23: 28: 00. Has been. That is, here, two people 202 possessing the tag 203 (their IDs are “TAG — 01” and “” TAG — 02) and one person 202 not possessing the tag 203 (the ID is “NONTAG — 01”) Are estimated to be in the room 201.

  At this time, the leaving object candidate number calculation unit 110b of the leaving object candidate calculation unit 110 calculates that the number of observation values is one less than the number of people 202. Then, the leaving object candidate number calculation unit 110b determines that one person 202 has left the room 201.

  When the moving object candidate number calculation unit 110b determines that the person 202 has moved, the moving object determination unit 110c of the moving object candidate calculation unit 110 detects the person 202 having the highest correlation value for each observation value. To associate. In the example of the correlation value of FIG. 16, the observed value CAM_06 is associated with TAG_01 and the leaving object determining unit 110c, and the observed value CAM_07 is associated with TAG_02 and the leaving object determining unit 110c. The person 202 (NONTAG_01) is not associated with any observation value in the retreated object determination unit 110c.

  Thereafter, the leaving object determination unit 110c determines that the person 202 (NONTAG — 01) that has not been associated with any observation value is the person 202 who has left the room 201. Then, the leaving object determination unit 110c of the leaving object candidate calculation unit 110 notifies the entrance / exit determination unit 111 that the person 202 (NONTAG — 01) may have left the room 201.

  Here, a case immediately after all the people 202 have left the room 201 will be described. In this case, since all the people 202 leave the room 201 and no observation value is obtained, the correlation value calculation unit 108 cannot calculate the correlation value. Therefore, the entry / exit object determination unit 111 determines that all the persons 202 are persons 202 who have left the room 201 only when no observation value has been acquired.

<Description of entry / exit object determination unit>
In the process of step S307, the entry / exit determination unit 111 records the results output by the approaching object candidate calculation unit 109 and the leaving object candidate calculation unit 110 in the position estimation history database 106.

  For example, at time 2010: 3: 3: 14: 23: 28: 00 in FIG. 9, information on the observation value CAM_005 determined to be an observation value obtained by observing a person 202 newly entering the room 201 is obtained as follows: Assume that the entry / exit determination unit 111 receives the entry object candidate calculation unit 109.

  Then, the entry / exit determination unit 111 determines an ID representing a person 202 who has not yet been recorded in the position estimation history database 106 and does not have the tag 203. For example, the entrance / exit determination unit 111 refers to the position estimation history database 106, and the entrance / exit determination unit 111 calculates and determines an ID that is not yet used. As shown in FIG. 9, at time 2010: 3: 3: 14: 23: 27: 900, the ID representing the person 202 who does not have the tag 203 is not recorded. It is determined that the ID representing the person 202 who has newly entered 201 is NONTAG_01. After the next observation, the entry / exit determination unit 111 can determine the IDs representing the person 202 who does not have the tag 203 as NONTAG — 02, NONTAG — 03,.

  When the ID of the person 202 who has entered the room 201 is determined by the entry / exit determination unit 111, the person 202 with the ID exists according to a Gaussian distribution with a variance of 15 cm, with the position where the observed value CAM_005 is observed as the average position. there's a possibility that. Therefore, the entry / exit determination unit 111 records the person 202 with the ID in the position estimation history database 106 (see FIG. 14).

  As shown in FIG. 9, at the next time 2010: 3: 3: 14: 23: 29: 900, the entry / exit determination unit 111 calculates the information of the person 202 who has left the room 201 (NONTAG — 01). Suppose that it is received from the unit 110.

  Then, the entry / exit determination unit 111 records in the position estimation history database 106 that the person 202 indicated by NONTAG — 01 has left the room 201 (for example, using a symbol “−” as shown in FIG. 14). Can indicate that the person 202 is not present in the room 201).

  Here, the camera 101 has a detection error of the person 202 and an overdetection problem that detects an object other than the person 202 as the person 202. The tag reader 102 also has a possibility of a detection error of the tag 203. In such a case, only when the same observation value (observation value that can be determined to have been obtained by observing the same person by color feature amount, etc.) has been listed as an entry candidate twice consecutively from the approaching object candidate detection unit 109. The observation value can be used for the processing of the entry / exit object determination unit 111. Similarly, only when the same person 202 (for example, NONTAG — 01) has been listed as a leaving candidate twice consecutively from the leaving object candidate detection unit 110, the person 202 is included in the processing of the entering / leaving object determination unit 111. It may be used.

  Here, a case where two cameras 101 are installed and they have observation ranges overlapping each other will be described. In this case, in the above method, when both the first camera and the second camera observe the person 202 entering the room 201, the number of entering objects is two based on the observation value of each camera. turn into. Therefore, when the cameras 101 have overlapping observation ranges, as shown in FIG. 1E, the entrance / exit determination unit 111 has an approaching object number calculation unit 111a, a first clustering unit 111b, and an ID likelihood. A sum calculation unit 111c and an object entry determination unit 111d are provided.

  The approaching object number calculation unit 111a calculates the average value or the maximum value of the approaching object candidate numbers output by the entry / exit object candidate detection unit 92 for each observation device as the approaching object number.

  The first clustering unit 111b clusters all the entering objects detected by the entering object candidate detecting unit 109 with respect to the positions so that the number of clusters becomes the number of entering objects calculated by the entering object number calculating unit 111a.

  The ID likelihood sum calculation unit 111c calculates the sum for each ID for the ID likelihood included in the observed values of the approaching objects belonging to the cluster clustered by the first clustering unit 111b.

  The object approach determination unit 111d calculates the average position of the cluster with the ID having the highest ID likelihood among the sums of the ID likelihoods calculated for the object with the estimated position calculated by the ID likelihood sum calculation unit 111c. It is determined that the vehicle has entered.

  Hereinafter, the operation of these components will be described.

  For example, consider a case in which the first camera 101A has an observation range from coordinates 1 to 10 and the second camera 101B has an observation range from coordinates 6 to 15 (in order to simplify the description, the description will be given in one dimension). . That is, the range (1-5) observed only by the first camera 101A, the observation range (6-10) where the first camera 101A and the second camera 101B overlap each other, and only the second camera 101B observed Range (11-15) is present.

  At this time, first, the correlation value is calculated only between the observed value observed in each observed range (observed range) and the person 202 estimated to exist in the corresponding observed range. Calculation is performed by the calculation unit 108. Then, based on the correlation value calculated by the correlation value calculation unit 108, the approaching object candidate calculation unit 109 and the leaving object candidate calculation unit 110 determine the approaching object to the room 201 and the leaving object from the room 201, respectively. To do.

  A method for determining an object entering the room 201 in the overlapping observation range will be described.

  The average number of approaching object candidates determined based on the observation values acquired by the cameras 101A and 101B with respect to the observation values acquired in the overlapping observation range is the number of entering objects of the entrance / exit determination unit 111. Calculation is performed by the calculation unit 111a. For example, when two approaching object candidates are output according to the observation value of the first camera 101A and four approaching object candidates are output according to the observation value of the second camera 101B, the approaching object number calculation unit 111a causes the approaching object The average value of the number of candidates is 3 (= (2 + 4) / 2). At this time, there are a total of six (= (2 + 4)) observation values determined to be obtained by observing the approaching object in the overlapping observation range. Clustering is performed by the first clustering unit 111b of the entrance / exit determination unit 111 so that the six observation values become three clusters with respect to the position. Then, after the end of clustering in the first clustering unit 111b, the center of gravity position of the observation values belonging to each cluster is calculated as the position where the approaching object exists by the ID likelihood sum calculation unit 111c of the entry / exit determination unit 111, and the entry / exit is calculated. The object approach determination unit 111d of the determination unit 111 makes the determination. Here, the reason why the average value of the number of entering object candidates is used is that the possibility of detection error and overdetection of the first (second) camera 101A (101B) is taken into consideration.

  A method for determining an object that has left the room 201 in the overlapping observation range will be described. When the leaving object candidate calculation unit 110 outputs an ID indicating the same person 202 as a leaving candidate object in both observation values of the first camera 101A and the second camera 101B, the person 202 indicated by the ID is a leaving object. Judge. In addition, when three or more cameras 101 are installed in the room 201, the retreated object candidate calculation unit 110 outputs an ID indicating the same person 202 as a retirement candidate object in a majority of the camera observation values. The person 202 indicated by the ID may be determined to be a leaving object.

  The method for determining an object entering the room 201 outside the overlapping observation range and a method for determining an object leaving the room 201 may be the same as the method described in the case of one camera.

<Description of the object position estimation unit>
In the process of step S308, the object position estimating unit 105 estimates the position of the person 202 existing in the room 201 based on the information recorded in the position estimation history database 106.

  Specifically, the object position estimator 105 solves the association problem that determines which person 202 is the observed value, and the object position estimator 105 uses the observed value used to update the position of the person 202. decide. Thereafter, the object position estimation unit 105 performs estimation (update) of the position of the person 202 by performing the Bayes filtering process represented by the Kalman filter or the like in the object position estimation unit 105.

  As a method for solving the association problem, first, an evidence value is calculated by the object position estimation unit 105. The evidence value is equivalent to the correlation value described above (for convenience of explanation, only terms are used properly). Next, the calculated evidence value is normalized (so that the sum of the evidence values for all IDs is 1 for each observed value), and the association value is obtained by the object position estimation unit 105. Then, for each person 202, only the observation value with the maximum association value is used in the object position estimation unit 105, and the object position estimation unit 105 estimates the position using a Kalman filter or the like. This is an example of determining an observation value used for position estimation, and the present invention is not limited to this. For example, the observation value to be used may be an observation value whose association value is equal to or greater than a threshold value. Alternatively, all of the observation values may be used, and the position update amount may be weighted with the association value when the position is estimated. The weighting method using the association value will be described later.

  Hereinafter, the Kalman filter will be described.

  The Kalman filter is based on the assumption that both the information of the state of the object position estimation system (for example, the position of the object in the first embodiment of the present invention) and the observation value of the camera 101 include noise. The object position estimation unit 105 estimates the state of the likely object position estimation system 90.

  FIG. 17 shows an example in which a Kalman filter is used for object position estimation processing. The vertical axis represents the probability, and the horizontal axis represents the position of the object.

When the object moves as represented by (Expression 2), the camera 101 can obtain the observed value 1703 obtained by (Expression 3). Here, A represents the motion model of the object, x represents the object position of the object, and v represents the process noise generated during movement. Further, y represents an observation value, H represents an observation model that associates the object position x with the observation value y, w represents observation noise, and t represents time. Accordingly, the object of the motion model in _{A t} the time _t, x _t, the object position in the _{x t-1,} respectively time t and the time t-1, _{v t} is the time t, the process noise occurring during movement Represents. Y _t is an observation value at time t, H _t is an observation model that associates the object position x _t with the observation value y _t at time t, and w _t is an observation noise at time t.

  Here, when the process noise v and the observation noise w are white Gaussian noises, p (w) is expressed as (Equation 4) and p (v) is expressed as (Equation 5). . N (0, Q) represents a Gaussian distribution with an average of 0 and a variance of Q. N (0, R) similarly represents a Gaussian distribution with an average of 0 and a variance of R.

When the observation value 1703 is obtained, the object position estimation unit 105 updates the prior probability distribution 1701 (hereinafter referred to as “prior distribution”) regarding the position of the obtained object at the present time (when the object position estimation process is performed). The object position estimation unit 105 creates a prediction probability distribution 1702 (hereinafter referred to as “prediction distribution”). The average (position) of the prediction distribution 1702 can be obtained by the object position estimation unit 105 using (Expression 6), and the variance of the prediction distribution 902 can be obtained by the object position estimation unit 105 using (Expression 7). Note that x _{a | b} represents an estimated value of x at time a based on information at time b. For example, “x _{t | t−1} ” in (Expression 6) represents an estimated value of the object position x at time t based on the information at time t−1, and “Pd _{t | t} ” in (Expression 7). " _-1 " represents an estimated value of Pd at time t based on information at time t-1. Further, “Pd _{t−1 | t−1} ” represents an estimated value of Pd at time t−1 based on the information at time t−1. Here, Pd represents the distribution of the distribution. Q _t represents the variance of time t.

When the predicted distribution 1702 is obtained by the object position estimating unit 105, the posterior distribution 1704 is obtained by the object position estimating unit 105 from the observed value 1703 and the predicted distribution 1702. The average (position) of the posterior distribution can be obtained by the object position estimation unit 105 using (Equation 8), and the variance of the posterior distribution can be obtained by the object position estimation unit 105 using (Expression 9). Here, K is a value called Kalman gain, and is obtained by the object position estimation unit 105 in (Equation 10). The Kalman gain is a value that determines the update amount. When the accuracy of the observed value is good (dispersion R is very small), the value of the Kalman gain increases to increase the update amount. Conversely, when the accuracy of the prior distribution is good (dispersion Pd is very small), the value of the Kalman gain is small in order to reduce the update amount. K _{t in} (Equation 8) represents the Kalman gain at time t. “X _{t | t} ” represents an estimated value of the object position x at time t based on the information at time t. “Pd _{t | t} ” in (Expression 9) represents an estimated value of the variance Pd of the distribution at time t based on the information at time t. R _t represents the variance R at time t.

When weighting the update amount of the position of the person 202 with the association value, (Equation 10) may be replaced with the following (Equation 11) by the object position estimation unit 105. D _{person 202} represents an association value for person 202.

  In order to update the position of the person 202 using all the observation values, it is necessary to repeat the above process by the number of observation values.

  As described above, the estimation result of the position of the person 202 obtained by the object position estimation unit 105 is recorded in the position estimation history database 106. FIG. 14 shows an example of the estimation result of the object position estimation unit 105.

  FIG. 18 shows an example of the movement trajectory of the person 202 displayed on the display device (for example, a display such as a monitor) 95 of the monitoring system using the object position estimation device 90 according to the embodiment of the present invention. An ID (that is, TAG — 01, TAG — 02, NONTAG — 01, NONTAG — 02) indicating the person 202 is assigned to each movement locus. Further, when the person 202 indicated by the ID (that is, TAG — 01, TAG — 02, NONTAG — 01, NONTAG — 02) enters from which position within the observation range of the observation apparatus 100, follows what trajectory, and performs the current (object position estimation process) ), It is possible to visually confirm where it exists. The trajectory to be displayed may be only the trajectory of the person 202 that is currently estimated to exist within the observation range (when the object position estimation process is performed), or only the trajectory of the ID specified by the user. Furthermore, the trajectory of the person 202 who existed in the past observation range may be displayed by specifying the time.

  In particular, in FIG. 18, a person 202 who has IDs of NONTAG — 01 and NONTAG — 02 and does not have a tag, in other words, a person who can be regarded as a suspicious person enters from which position within the observation range of the observation apparatus 100. It is possible to visually confirm what kind of trajectory is traced and at which position it is present (when performing object position estimation processing). Thereby, detection of a suspicious person (person who does not possess a tag), estimation of the escape route, etc. can be performed easily.

  Of course, the present invention may include an output device 96 that can output information on the object position estimation result to a necessary device instead of or in addition to the display device 95. .

  In the embodiment or the modified example, the object for which the object position is to be estimated is not limited to a person but may be an animal or an article.

  It is to be noted that, by appropriately combining any of the various embodiments, the effects possessed by them can be produced.

  The object position estimation system, the object position estimation method, and the object position estimation program according to the present invention can estimate until a person who does not have a tag enters, tracks, and leaves a sensing range. Therefore, it is difficult to give a tag to an ingress person in advance, the flow management of a customer in the store, or the detection of a suspicious person (person who does not have a tag) in an office, etc. and its escape route This is particularly useful for estimation.

90 object position estimation device 91 object position estimation processing unit 92 entry / exit object candidate detection unit 95 display device 96 output device 100 observation device 101 camera 101b image processing unit 101c ID identification processing unit 101cb comparison unit 101cd calculation unit 101ce tracking success probability acquisition unit 101cf Data association value acquisition unit 101cg ID likelihood allocation unit 101ch Addition unit 101S Storage unit (internal storage unit of camera)
102 Ultra Wide Band Tag Reader 103 Observation History Database 104 Entry / Exit Tag Determination Unit 105 Object Position Estimation Unit 106 Position Estimation History Database 107 Position Likelihood Calculation Unit 108 Correlation Value Calculation Unit 109 Entry Object Candidate Detection Unit 110 Departure Object Candidate Detection Unit 111 Entry Retreated object determination unit 201 room 202 people 202A people (tag possessing person 1)
202B (person with tag 2)
202C people (tag not possessed person 1)
203 Tag 203A Tag (1 person carrying the tag)
203B tag (with 2 tag holders)
204A Door (1)
204B Door (2)
204C Door (3)
1701 Prior distribution 1702 Predicted distribution 1703 Observed value 1704 Posterior distribution

Claims (11)

An observation device that observes a plurality of objects existing in the environment, and outputs the observation position of each of the plurality of objects and the likelihood information of the ID for each object as an observation value;
A position likelihood calculating unit that calculates likelihood information of the position of each observation value with respect to each object based on the estimated position and the observation position of each object;
A correlation value calculation unit that calculates a correlation value between the observed value and the estimated position of each object based on the likelihood information of the ID and the likelihood information of the position;
Based on the correlation value, an entry / exit object candidate detection unit that detects a candidate for an object that has entered the observation range of the observation apparatus or an object that has left the observation range;
Based on the output result of the entry / exit object candidate detection unit, an entry / exit object determination unit that determines an object that has entered the observation range of the observation device or an object that has left the observation range of the observation device;
Based on the determination result of the entry / exit object determination unit, based on the estimated position of each object, the likelihood information of the ID, and the likelihood information of the position, the presence at the position of each object at the time when the observed value was obtained An object position estimating unit that calculates a probability as an estimated position of each object;
An object position estimation system comprising: The entry / exit object candidate detection unit includes:
Based on the correlation value, an approaching object candidate detection unit that detects a candidate for an object that has entered the observation range of the observation device;
Based on the correlation value, a retreated object candidate detection unit that detects a candidate for a retreated object from within the observation range of the observation device;
The object position estimation system according to claim 1, comprising: An observation history database that records the observed values together with the time at which each object was observed;
A position estimation history database that records the calculation result of the estimated position of each object together with the time when each object was observed;
The object position estimation system according to claim 1 or 2, further comprising: A plurality of the observation devices are provided,
The approaching object candidate detection unit is
A first estimated position number comparison unit that compares the number of observed values for each observation device with the estimated number of positions of an object;
When the number of observed values is larger than the number of estimated positions of the object based on the comparison result in the first estimated position number comparing unit, the observation device determines the difference number between the number of observed values and the estimated number of positions of the object. An approaching object candidate number calculating unit that calculates the number of detected approaching object candidates;
For each observation value, an approaching object that determines that an approaching object has been detected at the observation position of the observation value by the number of approaching object candidate counts calculated by the approaching object candidate number calculation unit in order of increasing correlation value. A determination unit, and
The leaving object candidate detection unit
A second estimated position number comparison unit that compares the number of observation values for each observation device with the estimated number of positions of the object;
When the number of estimated positions of the object is larger than the number of observed values based on the comparison result in the second estimated position number comparing unit, the observation device calculates the difference number between the estimated position number of the object and the number of observed values. A retreat object candidate number calculation unit that calculates the number of detected retirement object candidates;
For each estimated position of each object, a retired object determining unit that determines that the object has retreated by the number of retired object candidates calculated by the retired object candidate number calculating unit in descending order of the maximum correlation value. The object position estimation system according to claim 2. At least one observation device is composed of a camera,
At least one other observation device is composed of a tag reader,
Further, it includes an entry / exit tag determination unit that determines entry / exit of a tagged object based on the observation value of the tag reader and the estimated position of each object, and records the determination result in the position estimation history database,
The correlation value calculation unit, the entry / exit object candidate detection unit detection unit, and the entry / exit object determination unit estimate the observation value output by the camera and the object previously determined to have entered by the entry / exit object determination unit. The object position estimation system according to any one of claims 1 to 4, wherein only the position is used. The entry / exit object determination unit includes:
An entering object number calculating unit for calculating an average value or a maximum value of the number of entering object candidates output by the entry / exit object candidate detecting unit for each observation device;
A first clustering unit that clusters all the entering objects detected by the entering object candidate detecting unit with respect to positions such that the number of clusters is the number of entering objects calculated by the entering object number calculating unit;
An ID likelihood sum calculating unit that calculates a sum for each ID for the ID likelihood included in the observed values of the approaching objects belonging to the cluster clustered by the first clustering unit;
It is determined that the ID object having the highest ID likelihood has entered the average position of the cluster among the sums of ID likelihoods for the object that is calculated by the ID likelihood sum calculation unit and has no estimated position. An object entry determination unit;
A moving object number calculating unit for calculating an average value or a maximum value of the number of moving object candidates output for each observation device by the moving object candidate detecting unit;
A second clustering unit that clusters all the leaving objects detected by the leaving object candidate detection unit with respect to positions such that the number of clusters is the number of leaving objects calculated by the leaving object number calculation unit;
The object position estimation according to any one of claims 1 to 3, further comprising a leaving object presence determination unit that determines that a leaving object exists at an average position of each cluster clustered by the second clustering unit. system. At least one observation device is composed of a camera,
The likelihood information of the ID for each object output as the observation value from the camera is not included in the observation history in addition to the object included in the observation history recorded in the observation history database. The object position estimation system according to claim 3, wherein likelihood information of an object ID is included.   The entry / exit object candidate detection unit determines that an entry object has been detected at an observation position of each observation value when there is no estimated position of the object. The object position estimation system as described in one.   The entry / exit object candidate detection unit determines that all objects have moved out of the observation range of the observation apparatus when no observation value is acquired as a result of observation by the observation apparatus. The object position estimation system according to any one of claims 1 to 8. After observing a plurality of objects existing in the environment with an observation device and obtaining the observation position of each of the plurality of objects and ID likelihood information for each object as observation values,
Based on the estimated position and the observed position of each object by the position likelihood calculating unit, the position likelihood information for each object of each observed value is calculated,
A correlation value calculation unit calculates a correlation value between the observed value and the estimated position of each object based on the likelihood information of the ID and the likelihood information of the position,
Based on the correlation value, the entry / exit object candidate detection unit detects an object that has entered the observation range of the observation apparatus or an object candidate that has left the observation range,
Based on the output result of the entry / exit object candidate detection unit, the entry / exit object determination unit determines an object that has entered the observation range of the observation device or an object that has left the observation range of the observation device,
The object position estimation unit, based on the determination result of the entry / exit object determination unit, the estimated position of each object and the likelihood information of the ID and the likelihood information of the position at the time when the observed value was obtained An object position estimation method, wherein the existence probability at the position of each object is calculated as an estimated position of each object. On the computer,
After observing a plurality of objects existing in the environment with an observation device and obtaining the observation position of each of the plurality of objects and ID likelihood information for each object as observation values,
A function for calculating position likelihood information of each observation value for each object based on the estimated position and observation position of each object by a position likelihood calculating unit;
A function for calculating a correlation value between the observed value and the estimated position of each object based on the likelihood information of the ID and the likelihood information of the position by a correlation value calculating unit;
Based on the correlation value, an entry / exit object candidate detection unit detects a candidate for an object that has entered the observation range of the observation device or an object that has left the observation range;
Function for determining an object that has entered the observation range of the observation device or an object that has left the observation range of the observation device based on the output result of the entry / exit object candidate detection unit by the entry / exit object determination unit When,
The object position estimation unit, based on the determination result of the entry / exit object determination unit, the estimated position of each object and the likelihood information of the ID and the likelihood information of the position at the time when the observed value was obtained A function of calculating the existence probability at the position of each object as the estimated position of each object;
Object position estimation program to realize.

Images