JP6340862B2 - Indoor positioning server, indoor positioning method, program, and indoor positioning system - Google Patents

Description

  TECHNICAL FIELD The present invention relates to indoor position measurement technology, and in particular, a position capable of accurately obtaining indoor position information of a pedestrian carrying a portable terminal having a PDR (Pedestrian Dead Reckoning) function. Related to measurement technology.

  In recent years, application software that obtains location information using GPS (Global Positioning System) has been installed in portable terminals and the like, and it is possible to obtain current location information.

However, since the GPS function cannot be used indoors, indoor positioning technology using technologies such as Wi-Fi, sound waves, and visible light communication has been studied.
However, there is a problem that installing these communication devices in the entire indoor area is not practical because it is costly and troublesome.

Therefore, in recent years, the development of PDR technology for calculating the current position indoors by combining an acceleration sensor, a gyro sensor, and the like mounted on a portable terminal has been progressing.
By installing PDR application software in a mobile terminal, a user of the mobile terminal can obtain his current position information.

However, the PDR technique has a problem that the measurement error of each sensor mounted on the mobile terminal is accumulated, and the displacement of the calculated position information becomes large.
Therefore, a method for correcting the positional information shift has been proposed (for example, Patent Documents 1 and 2).

  Japanese Patent Laid-Open No. 2004-133867 compares the movement (velocity) vector of a moving object photographed by a monitoring camera with the movement vector obtained from a sensor group mounted on a mobile terminal. A mobile positioning device that sends a correction signal for the obtained position to a portable terminal is disclosed.

  Further, Patent Document 2 transmits a two-dimensional map of a moving object and a stationary object included in a captured image from a camera to a mobile terminal, and the mobile terminal obtains behavior information and two-dimensional information obtained from a sensor group mounted on the mobile terminal. A method has been proposed in which maps are collated, a matching moving body is determined as a moving body having the mobile terminal, and the position of the moving body is obtained based on the position information of the stationary body.

International Publication No. 2011/068184 Japanese Patent No. 4761307

However, the positioning methods shown in Patent Document 1 and Patent Document 2 are not assumed to be used in a wide space such as an underground plaza such as a station or an underground mall, and the shooting range of the camera is narrow. There is a problem that it may not be detected.
In addition, it is conceivable to install a plurality of cameras in order to cover a wide space, but there is a problem that costs are increased.

  Further, the method of Patent Document 1 has a problem that it requires software for creating a two-dimensional map on the camera side and transmitting it to the mobile terminal, which is expensive.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide an indoor positioning device that can cover a wide range and efficiently position a user indoors at a low installation cost. Etc. is to provide.

In order to achieve the above-described object, the first invention is an indoor positioning system connected via a network to a camera that performs imaging for a predetermined time according to a control signal and a portable terminal that calculates a current position from sensor measurement information. A server that receives notification from the portable terminal that the camera has been approached, sends the control signal to the camera, causes the camera to perform shooting, and based on an image received from the camera Then, motion extraction means for extracting the motion of the moving object photographed in the image, and motion calculation for calculating the motion of the mobile terminal based on the measurement information and time information of the sensor received from the mobile terminal Means, collating means for collating the movement of the mobile object and the movement of the portable terminal, and the collating means matches the movement of the mobile object and the movement of the portable terminal. Indoor positioning, characterized in that a when it is determined, obtains the current position of the mobile terminal based on the position of the moving body before SL in the image, comprises a, a position notification means for notifying to the mobile terminal It is a server.

The control signal is preferably at least one of a rotation angle of the camera and a zoom magnification.
The camera control means receives a notification that the camera is approached, acquires the current position of the mobile terminal from the mobile terminal, and calculates the control signal for adjusting the focus of the camera to the current position. It is desirable to transmit to the camera.

According to the first invention, when a pedestrian carrying a mobile terminal approaches the vicinity of the camera, the server is notified of this, and the server sends a signal for rotating the camera in the direction of the pedestrian and zooming. It is possible to take an image that matches the direction.
This makes it possible to cover a wider range of pedestrians.
Further, only when there is a pedestrian carrying the mobile terminal, the camera performs shooting, and a system with high processing efficiency can be constructed.

  In addition, the information processing apparatus further includes storage means for storing a plurality of beacon positions arranged in the vicinity of the camera in association with identification information, and the camera control means stores the identification information of the beacon detected from the mobile terminal. The control signal may be received, and the control signal for adjusting the focus of the camera to the position of the beacon corresponding to the identification information may be calculated and transmitted to the camera.

  The beacon with low installation cost enables the mobile device to detect that it is close to the camera regardless of the current position due to the pedestrian dead reckoning of the mobile device, and to detect and measure a wide range of pedestrians more efficiently. Is possible.

Further, the camera control means counts the number of the portable terminals that transmit the identification information of the beacon for each beacon for a predetermined time, and focuses the camera on the identification with the largest number of the portable terminals. The control signal that matches the position of the beacon corresponding to the information may be calculated and transmitted to the camera.
Thereby, it becomes possible to measure the position of many people more efficiently.

In addition, the camera control means is presumed that a cumulative error in calculating the current position by the mobile terminal is large among the mobile terminals that transmit the identification information of the beacon for each beacon for a predetermined time. Counting the number of mobile terminals, calculating the control signal to adjust the focus of the camera to the position of the beacon corresponding to the identification information with the largest number of mobile terminals estimated to have a large cumulative error, You may make it transmit to a camera.
As a result, it is possible to more efficiently execute position correction on a mobile terminal having a large accumulated error.

A second invention is an indoor positioning method executed by an indoor positioning server connected via a network to a camera that performs shooting for a predetermined time in accordance with a control signal and a portable terminal that calculates a current position from sensor measurement information. And receiving a notification that the camera is approaching from the mobile terminal, sending a control signal to the camera and causing the camera to perform shooting, and based on an image received from the camera, A motion extraction step of extracting a motion of a moving object photographed in the image; and a motion calculation step of calculating a motion of the mobile terminal based on the measurement information and time information of the sensor received from the mobile terminal; A collation step for collating the movement of the mobile object and the movement of the portable terminal, and the movement of the mobile object and the portable terminal by the collation step If the motion is determined to match, obtains the current position of the mobile terminal based on the position of the moving body before SL in the image, characterized in that it comprises a position notification step of notifying the mobile terminal This is an indoor positioning method.
According to the second invention, when a pedestrian carrying a mobile terminal approaches the vicinity of the camera, the server is notified of this, and the server sends a signal for rotating the camera in the direction of the pedestrian and zooming. It is possible to take an image that matches the direction.
This makes it possible to cover a wider range of pedestrians.
Further, only when there is a pedestrian carrying the mobile terminal, the camera performs shooting, and a system with high processing efficiency can be constructed.

According to a third aspect of the present invention, there is provided a computer for causing a computer to function as an indoor positioning server connected via a network to a camera that performs photographing for a predetermined time in accordance with a control signal and a portable terminal that calculates a current position from sensor measurement information. The program is received from the camera, camera control means for receiving notification that the computer has approached the camera from the portable terminal, sending the control signal to the camera, and causing the camera to perform shooting. Based on the image, the movement extracting means for extracting the movement of the moving object photographed in the image, the movement of the portable terminal is calculated based on the measurement information and time information of the sensor received from the portable terminal. Motion calculating means, collating means for collating the movement of the mobile object and the movement of the mobile terminal, and movement of the mobile object by the collating means If the movement of the fine said mobile terminal is determined to match, it obtains the current position of the mobile terminal based on the position of the moving body before SL in the image, to function as a position notifying means for notifying to the mobile terminal It is a program for.
By installing the program according to the third invention in a general-purpose computer, the indoor positioning server according to the first invention can be obtained and the indoor positioning method according to the second invention can be executed.

According to a fourth aspect of the present invention, there is provided an indoor positioning system in which a camera, a portable terminal, and a server are connected via a network, wherein the camera performs photographing for a predetermined time in accordance with a control signal sent from the server. And an image transmission means for transmitting an image photographed by the photographing means to the server, and the portable terminal is close to the camera, a position calculating means for calculating a current position from sensor measurement information An approach notification means for notifying the server of the notification, and a transmission means for transmitting the measurement information and time information of the sensor to the server, wherein the server has approached the camera from the portable terminal. A camera control unit that receives the notification, sends the control signal to the camera, and causes the camera to execute shooting; and based on the image received from the camera, the image Motion extraction means for extracting the movement of the moving object being photographed in the camera, and motion calculation means for calculating the movement of the mobile terminal based on the measurement information and the time information of the sensor received from the mobile terminal, if the motion of the moving body and collating means for collating the movement of the mobile terminal, the motion of the motion and the mobile terminal of the mobile by the collation means is determined to match, the moving body before SL in the image An indoor positioning system comprising: position notification means for obtaining a current position of the mobile terminal based on the position of the mobile terminal and notifying the mobile terminal.

  As described above, the indoor positioning system and the like of the present invention can cover a wide area and efficiently position a user indoors at a low installation cost.

The block diagram which shows the structural example of the indoor positioning system 1a which concerns on the 1st Embodiment of this invention. The figure explaining the indoor positioning system 1a which concerns on 1st Embodiment The figure which shows the hardware structural example of the movable camera 2 The figure which shows the hardware structural example of the portable terminal 3 with a PDR function The figure which shows the hardware structural example of the server 4 The flowchart which shows the flow of a process of the indoor positioning system 1a which concerns on 1st implementation mobile The figure which shows the structural example of the indoor positioning system 1b which concerns on 2nd Embodiment. The figure explaining the indoor positioning system 1b which concerns on 2nd Embodiment. The flowchart which shows the flow of a process of the indoor positioning system 1b which concerns on 2nd Embodiment (the 1) The flowchart (the 2) which shows the flow of a process of the indoor positioning system 1b which concerns on 2nd Embodiment. The figure explaining another method of the indoor positioning system 1b which concerns on 2nd Embodiment. The flowchart which shows the flow of a process of another method of the indoor positioning system 1b which concerns on 2nd Embodiment (the 1) The flowchart which shows the flow of a process of another method of the indoor positioning system 1b which concerns on 2nd Embodiment (the 2)

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing a configuration example of the indoor positioning system 1a according to the first embodiment of the present invention.
The indoor positioning system 1a includes a movable camera 2, a portable terminal 3 with a PDR function, and a server 4. The movable camera 2 is connected to the server 4 via a network 5a, and the portable terminal 3 with a PDR function is a server. 4 and the network 5b.

The movable camera 2 is a network camera that can control swinging and zooming with an external control signal.
The mobile terminal 3 with the PDR function is, for example, a smartphone equipped with sensors such as an acceleration sensor and a gyro sensor, and PDR application software that calculates the position of the mobile terminal 3 from sensor data by the acceleration sensor and the gyro sensor is provided. It is installed.

The server 4 is equipped with software for measuring the position of the mobile terminal 3 with the PDR function, and is connected to the movable camera 2 and the mobile terminal 3 with the PDR function via the networks 5a and 5b, respectively.
The network 5a is a communication network regardless of wired or wireless.
The network 5b is a wireless communication network such as Wi-Fi.

FIG. 2 is a diagram illustrating the indoor positioning system 1a according to the first embodiment.
The movable camera 2 of the indoor positioning system 1a is installed in one corner of an indoor space such as an underground shopping center or an underground plaza having a passage 12 or the like.

The movable camera 2 can swing, for example, within the range of the movable range 15, and the rotation angle that is the swing angle can be controlled from an external server 4 (not shown).
The shooting range 13 is an example of a shooting possible range at the rotation angle of the movable camera 2 shown in the figure.

A pedestrian 11 carrying the mobile terminal 3 with the PDR function moves through the passage 12.
Although the walking route of the pedestrian 11 cannot be predicted, for example, walking routes a, b, and c (14a, 14b, and 14c) can be considered.
In the case of walking routes a and b (14a and 14b), even a conventional fixed camera that does not rotate the rotation angle of the camera passes through the photographing range 13. However, in the case of the walking route c (14c), the pedestrian 11 is not photographed unless the camera direction is changed by the movable camera 2.
In the indoor positioning system 1a according to the first embodiment, the pedestrian can be photographed by rotating the rotation angle of the movable camera 2 in the direction of the pedestrian 11 carrying the mobile terminal 3 with the PDR function.

FIG. 3 is a diagram illustrating a hardware configuration example of the movable camera 2.
The movable camera 2 has a configuration in which, for example, a camera unit 21 capable of zooming and rotating in the horizontal direction, a control unit 23, a communication control unit 25, and the like are connected by a bus 27.

The control unit 23 is composed of, for example, a PLC (Programmable Logic Controller) and the like, and performs rotation control and zoom control of the camera unit 21, shooting start / end operation control, and the like.
The communication control unit 25 includes a communication control device, a communication port, and the like, and controls communication with the server 4 via the network 5a. The network 5a may be wired or wireless.

FIG. 4 is a diagram illustrating a hardware configuration example of the mobile terminal 3 with the PDR function.
The mobile terminal 3 with a PDR function is, for example, a smartphone, and has a configuration in which a control unit 31, a storage unit 32, a communication control unit 34, an input unit 35, a display unit 36, a sensor unit 37, and the like are connected by a bus 38. .

  The control unit 31 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

The CPU calls and executes a program stored in the ROM, the storage unit 32, and the like in a work memory area on the RAM, drives and controls each device connected via the bus 38, and realizes processing performed by the computer.
The ROM is a non-volatile memory and permanently holds a computer boot program, a program such as BIOS, data, and the like.
The RAM is a volatile memory, and temporarily stores a program, data, and the like loaded from the storage unit 32, ROM, storage medium, and the like, and includes a work area used by the control unit 31 for performing various processes.

  The storage unit 32 stores a program executed by the control unit 31, data necessary for program execution, an OS (Operating System), and the like. The storage unit 32 stores, for example, PDR application programs and data.

  The communication control unit 34 includes a communication control device, a communication port, and the like, and controls communication with the server 4 via the network 5b. The network 5b is configured by a Wi-Fi network, for example.

The input unit 35 includes a touch panel for operating the mobile terminal 3 or the like.
The display unit 36 is a display device such as a liquid crystal panel. The display unit 36 displays position information and the like that are measured by the indoor positioning system 1a according to the first embodiment and sent from the server 4, for example. The

  The sensor unit 37 is composed of a sensor group such as an acceleration sensor and a gyro sensor, and measures acceleration and angular velocity, respectively. The measured data is stored in the storage unit 32, and is read out and used by the control unit 31 when the PDR application program is executed.

FIG. 5 is a diagram illustrating a hardware configuration example of the server 4.
The server 4 is based on the image sent from the movable camera 2 and the sensor group data of the portable terminal 3 sent from the portable terminal 3 carried by the pedestrian 11 in the indoor positioning system 1a of the first embodiment. It is a computer for performing the process which calculates | requires the positional information on the pedestrian 11. FIG.

  As shown in FIG. 5, the server 4 includes, for example, a control unit 41, a storage unit 42, a media input / output unit 43, a communication control unit 44, an input unit 45, a display unit 46, and a peripheral device I / F (interface) unit 47. Etc. are connected by the bus 24.

The control unit 41 includes a CPU, a ROM, a RAM, and the like.
The CPU calls and executes a program stored in the ROM, the storage unit 42, and the like in a work memory area on the RAM, drives and controls each device connected via the bus 48, and realizes processing performed by the computer.
The ROM permanently holds a computer boot program, a program such as BIOS, data, and the like.
The RAM temporarily stores programs, data, and the like loaded from the storage unit 42, ROM, storage medium, and the like, and includes a work area used by the control unit 41 to perform various processes.

  The storage unit 42 stores a program executed by the control unit 41, data necessary for program execution, an OS, and the like. The storage unit 42 obtains a program for obtaining an image by controlling the movable camera 2 in the indoor positioning system 1 of the first embodiment, and obtains measurement data of the sensor of the portable terminal 3 from the portable terminal 3 to obtain a camera image. A program and data for checking the position information of the mobile terminal 3 by collating with the movement of the mobile body inside are stored.

The media input / output unit 43 (drive device) inputs / outputs data, for example, media such as a CD drive (-ROM, -R, -RW, etc.), DVD drive (-ROM, -R, -RW, etc.) Has input / output devices.
The communication control unit 44 includes a communication control device, a communication port, and the like, and controls communication with other devices, for example, the movable camera 2 and the mobile terminal 3 with the PDR function, via the network. The network 5a that communicates with the movable camera 2 may be wired or wireless. A network 5b that communicates with the mobile terminal 3 with the PDR function is connected to the server 4 via a wireless access point such as Wi-Fi.

The input unit 45 inputs data and includes, for example, a pointing device such as a keyboard, a mouse, and a touch panel, and an input device such as a numeric keypad.
The display unit 46 is a display device such as a CRT monitor or a liquid crystal panel.

The peripheral device I / F (interface) unit 47 is a port for connecting peripheral devices, and is configured by USB, IEEE1394, RS-232C, or the like, and the connection form may be wired or wireless.
The bus 48 is a path that mediates transmission / reception of control signals and data signals between the devices.

  FIG. 6 is a flowchart showing a process flow of the indoor positioning system 1a according to the first embodiment.

The control unit 31 of the mobile terminal 3 with the PDR function operates the PDR application program stored in the storage unit 32 at all times or by the operation of the user of the mobile terminal 3.
For example, although the mobile terminal 3 with the PDR function is normally equipped with a GPS function, the PDR application program may be activated when positioning using the GPS function is not possible.

The control unit 31 of the mobile terminal 3 with the PDR function calculates current location information based on sensor measurement data by the sensor unit 37 (step 101).
The calculation of the current location information in step 101 is obtained by integrating the position information of a predetermined point obtained by a user instruction, a GPS function, etc., and measurement data of an acceleration sensor or gyro sensor using a known technique of PDR. Based on the movement information, the position information of the current location is calculated.

Next, the control part 31 of the portable terminal 3 with a PDR function determines whether the movable camera 2 exists in the vicinity (step 102).
For example, the position information of the movable camera 2 is stored in the storage unit 32 of the mobile terminal 3 with the PDR function, and the distance from the current location information obtained in step 101 is within a predetermined distance (for example, 50 m). It is determined whether or not exists.
When the movable camera 2 does not exist within the predetermined distance (No in Step 102), the process returns to Step 101.

  When the movable camera 2 exists within the predetermined distance (Yes in Step 102), the control unit 31 of the mobile terminal 3 with the PDR function transmits the current location information to the server 4 (Step 103).

When receiving the current location information of the mobile terminal 3 with the PDR function via the communication control unit 44, the control unit 41 of the server 4 identifies the movable camera 2 in the vicinity of the current location information, and identifies the movable camera 2 as the current location. A control signal for rotating and zooming in the direction of information is transmitted to the movable camera 2 (step 105).
The control signal is, for example, a rotation angle and a zoom magnification.

The control unit 23 of the movable camera 2 receives a control signal from the server 4 via the communication control unit 25, and the control unit 23 controls the rotation angle and zoom magnification of the camera unit 21 according to the received control signal, and shoots. Is started (step 106).
Shooting may be performed for a predetermined time, for example, 20 seconds.
The control unit 23 of the movable camera 2 transmits an image photographed by controlling the camera unit 21 to the server 4 through the communication control unit 25 together with the photographing time (step 107).

The control unit 41 of the server 4 receives a captured image from the movable camera 2 via the communication control unit 44, analyzes the image using a known technique, and determines whether a moving person (moving body) is captured ( Step 108).
For example, there are 30 captured images per second, and a moving object can be extracted by taking a difference between adjacent screens in time series.
If the moving person (moving body) has not been photographed (No in step 108), the process ends.

On the other hand, when a moving person (moving body) is photographed (Yes in step 108), the control unit 41 of the server 4 moves from a change in coordinates of a predetermined point of the extracted moving body on the image. A movement vector indicating the movement of the body (photographed person) is obtained (step 109). The predetermined points are, for example, the center of gravity, the highest point, the lowest point, and the like.
Since the movable camera 2 rotates and zooms, when obtaining the movement vector, correction is performed in consideration of the control signal (rotation angle, zoom magnification) and the like transmitted to the movable camera 2.

  On the other hand, after step 103, the control unit 31 of the portable terminal 3 with the PDR function includes data such as acceleration and angular acceleration for a certain time (for example, 20 seconds) measured by the sensor unit 37 together with time data, and a communication control unit. 25 to the server 4 (step 104).

  The control unit 41 of the server 4 obtains a movement vector indicating the movement of the mobile terminal 3 with the PDR function from the information of the measurement data (acceleration, angular acceleration, etc.) received from the mobile terminal 3 with the PDR function (step 110).

Next, the control unit 41 of the server 4 indicates the movement vector indicating the movement of the moving object (photographed person) calculated from the camera image in Step 109 and the movement of the mobile terminal 3 with the PDR function calculated in Step 110. A movement vector matching process is performed (step 111).
The collation process determines that the moving body (photographed person) on the camera image is the pedestrian 11 carrying the mobile terminal 3 with the PDR function if the two movement vectors in the same time zone are approximate.
If the two movement vectors are not approximated in the matching process (No in step 111), the process ends.

  When it is determined that the moving body (photographed person) on the camera image is the pedestrian 11 carrying the mobile terminal 3 with the PDR function (Yes in step 111), the control unit 41 of the server 4 moves on the camera image. The corrected position coordinates of the body (photographed person) are obtained and transmitted to the portable terminal 3 with the PDR function via the communication control unit 44 as the position information of the pedestrian 11 of the portable terminal 3 with the PDR function together with the time information of the image. (Step 112).

The control unit 31 of the mobile terminal 3 with the PDR function receives the position information and the time information from the server 4 and stores them in the storage unit 32 as the position information of the time (step 113).
Through the above processing, accurate position information at the time is set in the mobile terminal 3 with the PDR function. As a result, the subsequent current location information can be obtained based on the measurement data of the sensor unit 37 by executing the PDR application program of the mobile terminal 3 with the PDR function.

  If the PDR application program of the mobile terminal 3 with the PDR function is always operated, the above processing is executed every time it passes the vicinity of the movable camera 2 of the indoor positioning system 1a, and the indoor position information is accurately obtained. It becomes possible to update.

In steps 102 to 103 of the processing flow of the indoor positioning system 1a according to the first embodiment described above, the presence or absence of the nearby movable camera 2 is determined in the portable terminal 3 with the PDR function. This processing may be performed on the server 3 side.
That is, the current location information is calculated by the PDR function in step 101, and the obtained current location information is transmitted to the server 4. On the server 4 side, the position information of the movable camera 2 stored in advance in the storage unit 42 is searched. The movable camera 2 in the vicinity of the current location information of the mobile terminal 3 with the PDR function may be extracted.
Then, if the process after step 105 is performed similarly, it is possible to obtain the positional information of the portable terminal 3 with a PDR function.

(Second Embodiment)
Next, an indoor positioning system 1b according to the second embodiment of the present invention will be described.
FIG. 7 is a block diagram illustrating a configuration example of the indoor positioning system 1b according to the second embodiment.
The difference from the first embodiment is that a BLE (Bluetooth (registered trademark) Low Energy) beacon 6 is used in order to detect that a pedestrian carrying the mobile terminal 3 with the PDR function is in the vicinity of the movable camera 2. The other points are the same as the indoor positioning system 1a according to the first embodiment.

The movable camera 2 has the same configuration as that described in FIG. 3 in the indoor positioning system 1a of the first embodiment.
The configuration of the mobile terminal 3 with the PDR function is the same as that described with reference to FIG. 4, but the communication control unit 34 can receive the radio wave of the BLE beacon 6.
The configuration of the server 4 is the same as that described with reference to FIG. 5, but the storage unit 42 stores in advance the positional information of each BLE beacon 6 in association with the identification information.

The BLE beacon 6 transmits a signal radio wave with low power consumption according to the BLE standard in a range of approximately 10 m.
The mobile terminal 3 with the PDR function receives the signal radio wave of the BLE beacon 6 by the communication control unit 34. The signal radio wave includes identification information of the BLE beacon 6.

FIG. 8 is a diagram for explaining the indoor positioning system 1b according to the second embodiment.
A movable camera 2 and a plurality of BLE beacons 6 of the indoor positioning system 1b are installed at one corner of an indoor space such as an underground shopping center or an underground plaza having a passage 12 and the like.
The movable camera 2 can swing, and a rotation angle that is a swing angle can be controlled from an external server 4 (not shown).
The shooting range 13 is an example of a shooting possible range at the rotation angle of the movable camera 2 shown in the figure.

The plurality of BLE beacons 6 </ b> A to 6 </ b> C are appropriately installed around the passage 12 or the like where the pedestrian 11 carrying the mobile terminal 3 with the PDR function moves.
For example, it is considered that the pedestrian 11b moving from the upper direction of FIG. 8 toward the camera 2 enters the beacon radio wave reachable range 16A of the BLE beacon A (6A). Further, it is considered that the pedestrian 11c moving in the direction of the camera 2 from the lower passage in FIG. 8 enters the beacon radio wave reachable range 16C of the BLE beacon C (6C). Further, it is considered that the pedestrian 11a moving in the direction of the camera 2 from the right passage in FIG. 8 enters the beacon radio wave reachable range 16A or 16B of the BLE beacon A (6A) or the BLE beacon B (6b).

In the indoor positioning system 1b according to the second embodiment, the rotation angle of the movable camera 2 is rotated in the direction of the BLE beacon 6 detected by the portable terminal 3 with the PDR function carried by the pedestrian 11. Can be taken.
For example, when the mobile terminal 3 with the PDR function of the pedestrian 11c detects the radio wave of the BLE beacon 6C, the server 4 rotates the movable camera 2 in the direction of the BLE beacon 6C (rotation direction 17) and walks. The person 11c is photographed.

9 and 10 are flowcharts showing the flow of processing of the indoor positioning system 1b according to the second embodiment.
Each BLE beacon 6 (6A to 6C) continues to transmit radio waves (step 201).

When the pedestrian 11 carrying the portable terminal 3 with the PDR function enters the beacon radio wave reachable range 16 of any of the BLE beacons 6, the communication control unit 34 of the portable terminal 3 with the PDR function receives the signal radio wave, and the control unit 31 Acquires beacon identification information (beacon ID) (step 202).
The control unit 31 of the mobile terminal 3 with the PDR function transmits the acquired identification information (beacon ID) of the BLE beacon 6 to the server 4 via the communication control unit 34 (step 203). At this time, the identification information of the portable terminal 3 with the PDR function is also transmitted.

The server 4 receives the identification information (beacon ID) of the BLE beacon 6 and the identification information of the portable terminal 3 with the PDR function transmitted from the portable terminal 3 with the PDR function via the communication control unit 44.
The control unit 41 searches the position information of the BLE beacon 6 stored in the storage unit 42 based on the received identification information (beacon ID) of the BLE beacon 6 and rotates to face the BLE beacon 6. Then, a control signal for zooming is transmitted to the movable camera 2 (step 204).
The control signal is, for example, a rotation angle and a zoom magnification.

The processing flow after step 204 is the same as the processing after step 106 in the processing flow of the indoor positioning system 1a shown in FIG.
That is, the control unit 23 of the movable camera 2 receives a control signal from the server 4 via the communication control unit 25, and the control unit 23 controls the rotation angle and zoom magnification of the camera unit 21 by the received control signal. Then, shooting is started (step 205).
Shooting may be performed for a predetermined time, for example, 20 seconds.
The control unit 23 of the movable camera 2 transmits an image photographed by controlling the camera unit 21 to the server 4 through the communication control unit 25 together with the photographing time (step 206).

Turning to FIG.
The control unit 41 of the server 4 receives a captured image from the movable camera 2 via the communication control unit 44, analyzes the image using a known technique, and determines whether a moving person (moving body) is captured ( Step 207).
If the moving person (moving body) has not been photographed (No in step 207), the process ends.

On the other hand, when a moving person (moving body) has been photographed (Yes in step 207), the control unit 41 of the server 4 moves from a change in coordinates of a predetermined point of the moving body extracted on the image. A movement vector indicating the motion of the body (photographed person) is obtained (step 208). The predetermined points are, for example, the center of gravity, the highest point, the lowest point, and the like.
Since the movable camera 2 rotates and zooms, when obtaining the movement vector, correction is performed in consideration of the control signal (rotation angle, zoom magnification) and the like transmitted to the movable camera 2.

  On the other hand, after step 203, the control unit 31 of the mobile terminal 3 with the PDR function includes data such as acceleration and angular acceleration for a certain time (for example, 20 seconds) measured by the sensor unit 37 together with time data, and a communication control unit. 25 to the server 4 (step 221 in FIG. 10).

  The control unit 41 of the server 4 obtains a movement vector indicating the movement of the mobile terminal 3 with the PDR function from the information of the measurement data (acceleration, angular acceleration, etc.) received from the mobile terminal 3 with the PDR function (step 209).

Next, the control unit 41 of the server 4 indicates the movement vector indicating the movement of the moving object (photographed person) calculated from the camera image in Step 208 and the movement of the mobile terminal 3 with the PDR function calculated in Step 209. A movement vector matching process is performed (step 210).
The collation process determines that the moving body (photographed person) on the camera image is the pedestrian 11 carrying the mobile terminal 3 with the PDR function if the two movement vectors in the same time zone are approximate.
If the two movement vectors are not approximated in the matching process (No in step 210), the process ends.

  When it is determined that the moving body (photographed person) on the camera image is the pedestrian 11 carrying the mobile terminal 3 with the PDR function (Yes in Step 210), the control unit 41 of the server 4 moves on the camera image. The corrected position coordinates of the body (photographed person) are obtained and transmitted to the portable terminal 3 with the PDR function via the communication control unit 44 as the position information of the pedestrian 11 of the portable terminal 3 with the PDR function together with the time information of the image. (Step 211).

The control unit 31 of the mobile terminal 3 with the PDR function receives the position information and the time information from the server 4, and stores them in the storage unit 32 as the position information at the time (step 212).
Through the above processing, accurate position information at the time is set in the mobile terminal 3 with the PDR function. As a result, the subsequent current location information can be obtained based on the measurement data of the sensor unit 37 by executing the PDR application program of the mobile terminal 3 with the PDR function.

  The BLE beacon 6 is an inexpensive device, and the indoor positioning system 1b according to the second embodiment is an inexpensive indoor positioning system that efficiently detects a pedestrian 11 carrying the mobile terminal 3 with the PDR function over a wide range. It becomes possible to build on.

(Third embodiment)
Next, another method (referred to as the third embodiment) of the indoor positioning system 1b according to the second embodiment will be described.
FIG. 11 is a diagram illustrating an indoor positioning system 1b according to the third embodiment.

The movable camera 2 and the plurality of BLE beacons 6 of the indoor positioning system 1b are installed in one corner of an indoor space such as an underground shopping center or an underground plaza having a passage 12 and the like as in the second embodiment.
The movable camera 2 can swing, and a rotation angle that is a swing angle can be controlled from an external server 4 (not shown).
The shooting range 13 is an example of a shooting possible range at the rotation angle of the movable camera 2 shown in the figure.

The plurality of BLE beacons 6 </ b> A to 6 </ b> C are appropriately installed around the passage 12 or the like where the pedestrian 11 carrying the mobile terminal 3 with the PDR function moves.
It is considered that there are a plurality of pedestrians 11 carrying the portable terminal 3 with the PDR function that detects the radio signal of the same BLE beacon 6 in a place where there is a lot of traffic.
The indoor positioning system 1b according to the third embodiment realizes processing corresponding to such a case.

In FIG. 11, for example, there are three pedestrians 11 within the beacon radio wave reach 16A of the BLE beacon A (6A) and one radio wave detected by the BLE beacon B (6B), and one BLE beacon C (6C). Suppose there are two people.
In that case, the indoor positioning system 1b according to the third embodiment shoots the movable camera 2 in the direction of the BLE beacon 6 having the largest number of detected pedestrians 11, and provides position information to each pedestrian 11. Perform the provided process.
That is, in the example of FIG. 11, the rotation angle and zoom magnification are controlled so that the movable camera 2 is directed in the direction (rotation direction 18) of the BLE beacon A (6A) in which the three pedestrians 11 detect the radio waves. Shoot and process.

FIG.12 and FIG.13 is a flowchart which shows the flow of a process of the indoor positioning system 1b which concerns on 3rd Embodiment.
Each BLE beacon 6 (6A to 6C) continues to transmit radio waves (step 301).

When the pedestrian 11 carrying the portable terminal 3 with the PDR function enters the beacon radio wave reachable range 16 of any of the BLE beacons 6, the communication control unit 34 of the portable terminal 3 with the PDR function receives the signal radio wave, and the control unit 31 Acquires beacon identification information (beacon ID) (step 302).
The control unit 31 of the mobile terminal 3 with the PDR function transmits the acquired identification information (beacon ID) of the BLE beacon 6 to the server 4 via the communication control unit 34 (step 303).

The server 4 receives the identification information (beacon ID) of the BLE beacon 6 transmitted from the mobile terminal 3 with the PDR function via the communication control unit 44.
The server 4 receives the information on the identification information (beacon ID) of the BLE beacon 6 detected by each of the mobile terminals 3 with the PDR function from the plurality of mobile terminals 3 with the PDR function.

The control unit 41 of the server 4 counts the number of mobile terminals 3 that are notified that radio waves have been detected for each BLE beacon 6 at regular intervals (step 304).
That is, in the example of FIG. 11, the counting results are 3 for BLE beacon A (6A), 1 for B (6B), and 2 for C (6C).

The control unit 41 of the server 4 is stored in the storage unit 42 based on the identification information (beacon ID) of the BLE beacon 6 (BLE beacon A (6A) in the example of FIG. 11) having the maximum number of detection results among the counting results. The position information of the current BLE beacon 6 is searched, and the control signal for rotating and zooming to face the direction of the BLE beacon 6 is transmitted to the movable camera 2 (step 305).
The control signal is, for example, a rotation angle and a zoom magnification.

The processing flow after step 306 is substantially the same as the processing after step 205 in the processing flow of the indoor positioning system 1b according to the second embodiment shown in FIG. 9 and FIG.
The indoor positioning system 1b according to the third embodiment is different in that the position information of a plurality of mobile terminals 3 with a PDR function that detect the radio waves of the BLE beacon 6 is obtained.

The control unit 23 of the movable camera 2 receives a control signal from the server 4 via the communication control unit 25, and the control unit 23 controls the rotation angle and zoom magnification of the camera unit 21 according to the received control signal, and shoots. Is started (step 306).
Shooting may be performed for a predetermined time, for example, 20 seconds.
The control unit 23 of the movable camera 2 transmits an image photographed by controlling the camera unit 21 to the server 4 through the communication control unit 25 together with the photographing time (step 307).

Turning to FIG.
The control unit 41 of the server 4 receives a captured image from the movable camera 2 via the communication control unit 44, analyzes the image using a known technique, and determines whether a moving person (moving body) is captured ( Step 308).
If the moving person (moving body) has not been photographed (No in step 308), the process ends.

On the other hand, when a moving person (moving body) is photographed (Yes in step 308), the control unit 41 of the server 4 moves from a change in coordinates of a predetermined point of the moving body extracted on the image. A movement vector indicating the motion of the body (photographed person) is obtained (step 309). The predetermined points are, for example, the center of gravity, the highest point, the lowest point, and the like.
In step 309, since it is considered that a plurality of persons (moving bodies) are captured in the received camera image, a movement vector for each moving body (capturing person) is obtained.
Since the movable camera 2 rotates and zooms, when obtaining the movement vector, correction is performed in consideration of the control signal (rotation angle, zoom magnification) and the like transmitted to the movable camera 2.

  On the other hand, the control unit 31 of each mobile terminal 3 with the PDR function that detects the radio wave of the BLE beacon 6 performs acceleration, angular acceleration, and the like for a predetermined time (for example, 20 seconds) measured by the sensor unit 37 after step 303. The data is transmitted together with the time data to the server 4 via the communication control unit 25 (step 321). At this time, the identification information of the portable terminal 3 with the PDR function is also transmitted.

  The control unit 41 of the server 4 obtains a movement vector indicating the movement of each mobile terminal 3 with a PDR function from information of measurement data (acceleration, angular acceleration, etc.) received from at least one mobile terminal 3 with a PDR function ( Step 310).

Next, the control unit 41 of the server 4 includes at least one movement vector indicating the movement of the moving object (photographed person) calculated from the camera image in step 309 and each mobile terminal 3 with the PDR function calculated in step 209. The movement vector indicating the movement of the movement is collated (step 311).
The collation process determines that the moving body (photographed person) on the camera image is the pedestrian 11 carrying the mobile terminal 3 with the PDR function if the two movement vectors in the same time zone are approximate.
If there is no approximation that approximates the two movement vectors in the verification process (No in step 311), the process ends.

When there is a movement vector determined that the moving body (photographed person) on the camera image is the pedestrian 11 carrying the mobile terminal 3 with the PDR function (Yes in step 311), the control unit 41 of the server 4 The corrected position coordinates of the moving body (photographed person) on the camera image are obtained, and the position information of the pedestrian 11 of the collated portable terminal 3 with the PDR function is given to the portable terminal 3 with the PDR function having the corresponding identification number. The time information of the image is transmitted via the communication control unit 44 (step 312).
The control unit 31 of the mobile terminal 3 with the PDR function receives the position information and the time information from the server 4 and stores them in the storage unit 32 as the position information of the time (step 313).

  The server 4 executes step 312 in parallel or sequentially, obtains the position information of the plurality of mobile terminals 3 with the PDR function, and transmits the position information to each mobile terminal 3 with the PDR function.

  Through the above processing, accurate position information at the time is set in each mobile terminal 3 with the PDR function. As a result, the subsequent current location information can be obtained based on the measurement data of the sensor unit 37 by executing the PDR application program of the mobile terminal 3 with the PDR function.

  The indoor positioning system 1b according to the third embodiment efficiently detects a pedestrian 11 carrying a large number of mobile terminals 3 with a PDR function over a wide range and provides accurate position information. A positioning system can be constructed at low cost.

  In the indoor positioning system 1b of the third embodiment described above, for each BLE beacon 6, the number of mobile terminals 3 with a PDR function that detected beacon radio waves is counted. 3, the number of mobile terminals 3 estimated to have a large accumulated error in position calculation by the PDR function may be counted.

  For example, the mobile terminal 3 with the PDR function that is estimated to have a large accumulated error has not been subjected to position correction by the server 4 for a long time, or has been performing position calculation by the PDR function over a long distance. Terminal 3.

  In order to extract the mobile terminal 3 for which the position correction by the server 4 has not been performed for a long time, for example, the server 4 stores the previous position correction date and time of each mobile terminal 3 with the PDR function, and the PDR function When a notification that the BLE beacon 6 is detected is received from the attached mobile terminal 3 (steps 303 and 304 in FIG. 12), the previous position correction date and time of the mobile terminal 3 is referred to for a predetermined time (for example, 1 When time has passed since the previous position correction date and time, it is estimated that the accumulated error is large and the coefficient is used.

  Instead of the server 4 storing the previous position correction date and time, the mobile terminal 3 with the PDR function stores the previous position correction date and time, and in step 303 of FIG. You may make it transmit to.

In order to extract the mobile terminal 3 that has performed position calculation by the PDR function over a long distance, for example, the mobile terminal 3 with the PDR function has moved from the position information that was initially set or the previous position correction. May be stored and transmitted to the server 4 together with the beacon ID and the user ID in step 303 of FIG.
In this case, the server 4 receives information on the travel distance, and estimates that the accumulated error is large if the travel distance is equal to or greater than a predetermined distance.

  For each BLE beacon 6, the server 4 counts the mobile terminals 3 with the PDR function that are estimated to have a large cumulative error for each BLE beacon 6 (step 304), and the direction of the BLE beacon 6 with the largest number A control signal is sent to point the movable camera 2 at (step 305), and thereafter, the processing of step 306 to step 313 and step 321 of FIG. 12 and FIG. 13 is executed as in the description of the second embodiment. You can do it.

By the above processing, the mobile terminal 3 with the PDR function that is estimated to have a large accumulated error is counted for each BLE beacon 6, and the mobile terminal that is estimated to have a large accumulated error in the vicinity of the BLE beacon 6 that has a large number. Position information can be provided to each pedestrian 11 carrying 3.
This makes it possible to more efficiently execute the position correction of the mobile terminal 3 with the PDR function that requires position correction.

Although the BLE beacon 6 is used in the indoor positioning system 1b of the second and third embodiments described above, the transmitter is not limited to the BLE beacon 6 and may be a transmitter such as Wi-Fi.
In the indoor positioning systems 1a and 1b according to the first to third embodiments, the movable camera 2 is used. However, a camera having a fisheye lens may be used instead of the movable camera.
When a fisheye lens is used, a process for calibrating the distortion at the periphery of the lens is necessary.

  As described above, the indoor positioning system 1 of the present invention can provide accurate location information to the portable terminal 3 efficiently and inexpensively in a wide range of places where GPS does not function, such as underground plazas and underground shopping streets. become.

  As mentioned above, although embodiment of this invention was described referring an accompanying drawing, the technical scope of this invention is not influenced by embodiment mentioned above. It is obvious for those skilled in the art that various modifications or modifications can be conceived within the scope of the technical idea described in the claims, and these are naturally within the technical scope of the present invention. It is understood that it belongs.

1a, 1b .... Indoor positioning system 2 ... Movable camera 3 ... Mobile terminal 3 with PDR function
4 ... Server 5a, 5b ... Network 6 ... BLE beacon

Claims (9)

An indoor positioning server connected via a network to a camera that shoots for a predetermined time according to a control signal and a mobile terminal that calculates a current position from sensor measurement information,
Camera control means for receiving notification from the portable terminal that the camera is approached, sending the control signal to the camera, and causing the camera to perform shooting;
Based on the image received from the camera, a motion extraction means for extracting the motion of the moving object photographed in the image;
Motion calculating means for calculating the motion of the mobile terminal based on the measurement information and time information of the sensor received from the mobile terminal;
Collating means for collating the movement of the mobile body and the movement of the mobile terminal;
If the movement of the movement and the mobile terminal of said mobile is determined to coincide with the collating unit obtains a current position of the mobile terminal based on the position of the moving body before SL in the image, to the mobile terminal A position notification means for notification;
An indoor positioning server characterized by comprising:   The indoor positioning server according to claim 1, wherein the control signal is at least one of a rotation angle and a zoom magnification of the camera.   The camera control means receives a notification that the camera is approached, acquires the current position of the mobile terminal from the mobile terminal, and calculates the control signal for adjusting the focus of the camera to the current position. The indoor positioning server according to claim 1, wherein the indoor positioning server is transmitted to the camera. Storage means for storing a plurality of beacon positions arranged in the vicinity of the camera in association with identification information;
The camera control means receives the identification information of the beacon detected from the mobile terminal, calculates the control signal for adjusting the focus of the camera to the position of the beacon corresponding to the identification information, and sends the control signal to the camera. The indoor positioning server according to claim 1, wherein the indoor positioning server is transmitted.   The camera control means counts the number of the portable terminals transmitting the identification information of the beacon for each beacon for a predetermined time, and focuses the camera on the identification information having the largest number of portable terminals. The indoor positioning server according to claim 4, wherein the control signal that matches the position of the corresponding beacon is calculated and transmitted to the camera.   The mobile terminal is estimated to have a large accumulated error in calculating the current position by the mobile terminal among the mobile terminals that transmit the identification information of the beacon for each beacon for a predetermined time. And calculating the control signal for adjusting the focus of the camera to the position of the beacon corresponding to the identification information with the largest number of mobile terminals that are estimated to have a large accumulated error, The indoor positioning server according to claim 4, wherein the indoor positioning server transmits. An indoor positioning method executed by an indoor positioning server connected via a network to a camera that shoots for a predetermined time according to a control signal and a mobile terminal that calculates a current position from sensor measurement information,
A camera control step of receiving a notification that the camera is approaching from the portable terminal, sending the control signal to the camera, and causing the camera to perform shooting;
Based on the image received from the camera, a motion extraction step of extracting the motion of the moving object photographed in the image;
A motion calculating step for calculating a motion of the mobile terminal based on the measurement information and time information of the sensor received from the mobile terminal;
A collation step of collating the movement of the mobile object and the movement of the mobile terminal;
When the movement and the motion of the mobile terminal of said mobile is determined to coincide with the verification step, it obtains the current position of the mobile terminal based on the position of the moving body before SL in the image, to the mobile terminal A position notification step for notification;
The indoor positioning method characterized by including. A program for causing a computer to function as an indoor positioning server connected via a network to a camera that shoots for a predetermined time according to a control signal and a mobile terminal that calculates a current position from measurement information of a sensor,
The computer,
Camera control means for receiving notification from the portable terminal that the camera is approached, sending the control signal to the camera, and causing the camera to perform shooting;
A motion extracting means for extracting a motion of a moving object photographed in the image based on an image received from the camera;
Movement calculating means for calculating movement of the portable terminal based on the measurement information and time information of the sensor received from the portable terminal;
Collating means for collating the movement of the mobile body and the movement of the mobile terminal;
If the movement of the movement and the mobile terminal of said mobile is determined to coincide with the collating unit obtains a current position of the mobile terminal based on the position of the moving body before SL in the image, to the mobile terminal Position notification means for notification,
Program to function as. An indoor positioning system in which a camera, a mobile terminal, and a server are connected via a network,
The camera
Photographing means for photographing for a predetermined time according to a control signal sent from the server;
Image transmission means for transmitting an image captured by the imaging means to the server;
Comprising
The portable terminal is
Position calculating means for calculating the current position from the measurement information of the sensor;
Approach notifying means for notifying the server that the camera has been approached;
Transmitting means for transmitting measurement information and time information of the sensor to the server;
Comprising
The server
Camera control means for receiving notification from the portable terminal that the camera is approached, sending the control signal to the camera, and causing the camera to perform shooting;
Based on the image received from the camera, a motion extraction means for extracting the motion of the moving object photographed in the image;
Motion calculating means for calculating the movement of the mobile terminal based on the measurement information and the time information of the sensor received from the mobile terminal;
Collating means for collating the movement of the mobile body and the movement of the mobile terminal;
If the movement of the movement and the mobile terminal of said mobile is determined to coincide with the collating unit obtains a current position of the mobile terminal based on the position of the moving body before SL in the image, to the mobile terminal A position notification means for notification;
An indoor positioning system comprising:

Images