JP2013068482A - Azimuth direction correction system, terminal device, server device, azimuth direction correction method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an azimuth direction correction system, a terminal device, a server device, an azimuth direction correction method and a program capable of correcting an azimuth direction sensor with a small processing amount.SOLUTION: An azimuth direction correction system 100 comprises: photographing means 101 which acquires photographic image of a landscape including a road; azimuth direction detection means 102 which detects a photographing azimuth of the photographic image; position detection means 103 which detects a photographing position of the photographic image; map storage means 104 which stores map information including an arrangement state of the road; and error detection means 105. The error detection means 105: compares an arrangement direction of the road, on the photographic image, identified based on the photographic image as well as the photographing azimuth with the arrangement direction of the road, on a map at the photographing position, identified based on the map information as well as the photographing position; and calculates an error of the azimuth direction detection means.

Description

  The present invention relates to an azimuth correction system, a terminal device, a server device, an azimuth correction method, and a program, and more particularly to a technique for correcting an azimuth sensor using a camera.

  Conventionally, as a method of calibrating a direction sensor using geomagnetism such as an electronic compass, a method for giving a predetermined motion to a terminal on which the direction sensor is mounted, for example, on two planes among an XY plane, a YZ plane, and a ZX plane A method such as rotating several times has been proposed.

  However, even if such a calibration is performed, the direction of the actual geomagnetism and the direction of the geomagnetism measured by the direction sensor are shifted due to the influence of local peripheral magnetism, etc., and the direction sensor shows the correct direction. May not be possible.

  Such a phenomenon may have a serious influence on an application using the direction sensor. For example, in recent years, an application using so-called air tag type augmented reality has appeared that superimposes information associated with a location on a camera image and makes the information appear as if it were attached to real space. ing. In such an application, if the azimuth information by the azimuth sensor cannot be acquired correctly, there arises a problem that the air tag is displayed at a position deviated from the position to be actually displayed.

  In order to solve such a problem, Patent Document 1 determines a predetermined fixed object included in an outside scene image that is sequentially captured by the in-vehicle camera as a target, sequentially calculates the turning direction and turning angle of the vehicle from the movement angle of the target, The structure which correct | amends a direction sensor using the value of these turning directions and angles is disclosed.

JP 2005-114565 A

  However, the technique disclosed in Patent Document 1 requires a large amount of arithmetic processing in order to sequentially identify and track a target in an image, calculate a turning direction and angle, and correct a direction sensor. In order to realize such arithmetic processing, it usually requires hardware with high processing capability and a large amount of power consumption. Therefore, it is difficult to say that the above method is optimal for a portable terminal or the like that has a limit in available processing capacity and power.

  The present invention has been made to solve such problems, and provides an azimuth correction system, a terminal device, a server device, an azimuth correction method, and a program capable of correcting an azimuth sensor with a small amount of processing. The purpose is to do.

  An azimuth correction system according to the present invention includes an imaging unit that acquires a captured image of a landscape including a road, an orientation detection unit that detects an imaging direction of the captured image, and a position detection unit that detects an imaging position of the captured image. Map storage means for storing map information including road arrangement status, road arrangement direction on the photographed image specified based on the photographed image and the photographing orientation, the map information and the photographing position, And an error detection means for calculating an error of the azimuth detection means by comparing the road arrangement direction on the map at the photographing position specified based on

  Further, the terminal device according to the present invention includes a communication unit for communicating with the server device, a shooting unit for acquiring a shot image of a landscape including a road, a direction detection unit for detecting a shooting direction of the shot image, A position detection unit that detects a shooting position of the shot image; and a map storage unit that stores map information including a road arrangement state. The communication unit is configured to display the shot image, the shooting direction, and the shooting direction. While transmitting to the said server apparatus, the information concerning the error of the said direction detection means is received from the said server apparatus.

  The server device according to the present invention includes a communication unit that receives a captured image of a landscape including a road, a captured position, and a captured orientation from a terminal device, and the captured image specified on the basis of the captured image and the captured orientation. An error detection unit that compares an arrangement direction of the road and an arrangement direction of the road on the map at the shooting position specified based on the map information and the shooting position, and calculates an error of the direction detection unit The communication means transmits information related to the error to the terminal device.

  The azimuth correction method according to the present invention includes a step of acquiring a photographic image of a landscape including a road, a step of detecting a photographic direction of the photographic image, a step of detecting a photographic position of the photographic image, and the photographic image. The road arrangement direction of the road on the photographed image specified based on the photographing direction, the map information including the road arrangement state, and the road on the map at the photographing position specified based on the photographing position. And comparing the arrangement direction and calculating an error of the azimuth detecting means.

  Another azimuth correction method according to the present invention includes a step of acquiring a shot image of a landscape including a road, a step of detecting a shooting direction of the shot image, a step of detecting a shooting position of the shot image, and the shooting And transmitting the image, the shooting direction and the shooting direction to the server device, and receiving from the server device information related to an error of the direction detection means.

  Another azimuth correction method according to the present invention includes: a step of receiving a photographed image of a landscape including a road, a photographing position and a photographing direction from a terminal device; and the photographing specified based on the photographed image and the photographing azimuth. Comparing the road layout direction on the image with the road layout direction on the map at the shooting position specified on the basis of the shooting position and the map information including the road layout status; The method includes a step of calculating an error and a step of transmitting information related to the error to the terminal device.

  A program according to the present invention is a program for causing a computer to execute a predetermined operation, the step of acquiring a photographed image of a landscape including a road, the step of detecting a photographing direction of the photographed image, and the photographed image And a step of transmitting the photographed image, the photographing azimuth and the photographing azimuth to the server device and receiving information relating to an error of the azimuth detecting means from the server device. It is.

  Another program according to the present invention is a program for causing a computer to execute a predetermined operation, the step of acquiring a photographed image of a landscape including a road, the step of detecting the photographing orientation of the photographed image, A step of detecting a shooting position of a shot image; and a step of transmitting the shot image, the shooting direction and the shooting direction to the server device, and receiving information relating to an error of the direction detection means from the server device. It is what you have.

  Another program according to the present invention is a program for causing a computer to execute a predetermined operation, the step of receiving a photographed image of a landscape including a road, a photographing position and a photographing direction from a terminal device, and the photographed image The road on the map at the shooting position specified based on the shooting direction and the road layout direction on the shot image specified based on the shooting direction, the map information including the road arrangement status, and the shooting position Are compared with each other and calculating an error of the azimuth detecting means, and transmitting information related to the error to the terminal device.

  According to the present invention, it is possible to provide an azimuth correction system, a terminal device, a server device, an azimuth correction method, and a program that can correct an azimuth sensor with a small amount of processing.

It is a figure which shows the structure of Embodiment 1 of this invention. It is a figure which shows the structure of Embodiment 1 of this invention. It is a figure which shows the processing flow of Embodiment 1 of this invention. It is a figure which shows the processing flow of Embodiment 1 of this invention. It is a figure which shows the processing content of Embodiment 1 of this invention. It is a figure which shows the processing content of Embodiment 1 of this invention. It is a figure which shows the processing content of Embodiment 1 of this invention. It is a figure which shows the processing flow of Embodiment 1 of this invention.

  First, the configuration of the azimuth correction system 100 according to the first embodiment of the present invention will be described with reference to FIG.

  The azimuth correction system 100 according to the present embodiment includes an imaging unit 101, an azimuth detection unit 102, a position detection unit 103, a map storage unit 104, and an error detection unit 105.

  The azimuth correction system 100 may be, for example, a stand-alone system in which these means are all mounted on the same device, or these means are distributed to the terminal device 110 and the server device 120 as shown in FIG. The distributed system may be implemented. In the case of a distributed system as shown in FIG. 2, it is preferable that at least the photographing unit 101, the direction detection unit 102, and the position detection unit 103 are mounted on the terminal device 110. Here, the present embodiment will be described using the distributed system shown in FIG. 2 as an example.

  The terminal device 110 is typically a mobile phone terminal, a car navigation terminal, or the like. In the present embodiment, the terminal device 110 includes at least an imaging unit 101, an orientation detection unit 102, and a position detection unit 103. In addition, the terminal device 110 has an information processing function for performing various arithmetic processing according to a predetermined program, a communication function for performing data communication with the server device 120 on the network via a packet communication network and the Internet, etc. In addition, a display function for displaying predetermined information, for example, a photographing result by the photographing unit 101 or a processing result by the information processing function, an input function for accepting an operation by a user, and the like may be provided.

  The photographing unit 101 has a function of outputting a photographed image as electronic data. In the present embodiment, a camera sensor that converts visible light collected by a photographic lens into electronic data using an imaging device is used as the imaging unit 101.

  The direction detection unit 102 is a unit that detects geomagnetism and outputs direction information. The azimuth detecting means 102 typically first calculates the north-south direction by comparing the geomagnetic strengths in the three-axis (XYZ) directions output from a sensor such as an electronic compass. By comparing and calculating the strength of the geomagnetism in the three axis directions, the azimuth in the direction in which the sensor faces is detected. In the present embodiment, the azimuth detecting means 102 is set to detect the azimuth in the direction (the optical axis direction of the photographic lens) in which the photographing means 101 has taken a picture.

  The position detection unit 103 has a function of acquiring position information with a GPS sensor and outputting the latitude and longitude of the current location. In the present embodiment, the position detection unit 103 is set to detect the position where the shooting unit 101 has shot.

  The server device 120 is typically a server computer. In the present embodiment, the server device 120 includes at least a map storage unit 104 and an error detection unit 105. In addition, the server device 120 has an information processing function for performing various arithmetic processes according to a predetermined program, and a communication function for performing data communication and the like with the terminal device 110 on the network via a packet communication network and the Internet. Also good.

  The server device 120 generally performs relatively advanced information processing with less restrictions in terms of processing capability, power consumption, and the like compared to the terminal device 110 that is assumed to be carried on a daily basis by the user. It is suitable for. Therefore, in the present embodiment, the map storage unit 104 that requires a large amount of storage capacity and the error detection unit 105 that requires a high speed and a large amount of information processing capability are mounted on the server device 120 as described later. Yes.

  The map storage means 104 is means for storing and managing map information including road arrangement information. The map storage means 104 in the present embodiment includes a map database that stores map information and a function that acquires and outputs map information in the vicinity of a specified position from the map database.

  The error detection unit 105 uses the captured image, the azimuth information and the position information output from the terminal device 110, the map information output from the map storage unit 104, and the like to the azimuth information output from the azimuth detection unit 102. It is a means for detecting the included error. More specifically, the error detection unit 105 analyzes the captured image output from the imaging unit 101 to detect a road included in the captured image, and uses the orientation information output from the orientation detection unit 102 or the like. A function for specifying the road arrangement direction, a map storage means 104 for acquiring map information of a point indicated by the position information output by the position detection means 103, a function for specifying the road arrangement direction at the point, and the captured image And a function of detecting an error in the azimuth information by comparing the road arrangement direction specified by the map information with the road arrangement direction specified by the map information.

  Next, processing of the azimuth correction system 100 according to the first exemplary embodiment of the present invention will be described with reference to FIGS. 3 and 4.

  First, the operation of the terminal device 110 will be described with reference to FIG.

  The terminal device 110 acquires a photographed image of the landscape by the photographing unit 101. In the present embodiment, it is assumed that a road is included in the landscape photographed by the photographing means. At the same time, the azimuth information of the photographing direction is acquired from the azimuth detection means 102 as the azimuth information. Further, latitude / longitude information at the time of shooting is acquired as position information from the position detection means 103 (SA1).

  For example, the terminal device 110 can start the processing related to SA1 when the user uses an application launcher as the input function and activates the augmented reality application.

  FIG. 5 shows an example of the augmented reality application screen. The augmented reality application uses a display function provided in the terminal device 110 to superimpose and display a captured image acquired by the imaging unit 101 and air tag information that is previously associated with latitude and longitude information. Application. First, the augmented reality application detects the photographing position and photographing direction of the photographed image by the position detecting means 103 and the azimuth detecting means 102, and determines the geographical range (photographing range) included in the photographed image as a latitude. And by longitude. Subsequently, the augmented reality application acquires air tag information associated in advance with the latitude and longitude within the imaging range from the air tag information stored in advance in a storage means (not shown). The augmented reality application displays the acquired air tag information superimposed on an appropriate position of the captured image. As the air tag information, for example, the names of landmarks such as buildings, parks, and libraries are used. Thereby, the user can easily know the names of the landmarks included in the captured image.

  Here, if the azimuth detecting means 102 is affected by the magnetism of the surrounding environment and an error occurs in the azimuth information, the air tag information is not a position to be originally displayed (position indicated as A in FIG. 5), It is displayed at a position shifted by the amount of error (position indicated by B in FIG. 5). In order to prevent such a phenomenon, the terminal device 110 executes the following processing following the processing of SA1.

  The terminal device 110 transmits the acquired captured image, azimuth information, and position information to the server device 120 by a communication function (SA2).

  Next, the terminal apparatus 110 receives, from the server apparatus 120, azimuth correction amount data for correcting the error of the azimuth detecting means 102 as a response (SA3). The terminal device 110 corrects the error of the azimuth detecting means 102 using the azimuth correction amount data (SA4). The correction of the azimuth detecting means 102 is typically performed by the following processing. First, the correction amount received from the server device 120 is input to a driver or middleware that calculates the direction information provided in the direction detection unit 102. Next, the driver or middleware corrects the correction amount for the direction output by a sensor such as an electronic compass. Finally, the driver or middleware outputs the corrected orientation as an output value of the orientation detection means 102.

  Next, the operation of the server device 120 will be described with reference to FIG.

  The server device 120 receives the captured image, the azimuth information, and the position information from the terminal device 110 by the communication function (SB1).

  Next, the error detection unit 105 of the server device 120 analyzes the captured image and identifies the arrangement direction of the road included in the captured image (SB2). For example, a gray continuous object having a certain width or more is detected from a camera image, and a linear object whose brightness is significantly different from the gray object such as a white line or a yellow line, and these objects are detected. The longitudinal direction of the edge can be the road arrangement direction. Alternatively, it is also possible to recognize the road arrangement direction by using the fact that an object located farther becomes smaller on the captured image. More specifically, for example, a plurality of divided images obtained by slicing a captured image in a predetermined direction such as a horizontal direction or a vertical direction and a predetermined interval, or a plurality of divided images generated by dividing into a mesh shape are generated. The adjacent divided images are compared with each other. At this time, if objects having similar colors can be detected in each adjacent slice image and it can be confirmed that the area of those objects changes, the direction in which those objects are reduced becomes the road direction. be able to.

  Further, the error detection unit 105 calculates the road arrangement direction using the road arrangement direction obtained by the above-described processing and the direction information acquired from the terminal device 110 (SB2). An example of a method for calculating the road arrangement direction will be described with reference to FIG. As described above, in this embodiment, the azimuth information indicates the azimuth in the optical axis direction of the lens of the photographing unit 101. Therefore, in the captured image shown in FIG. 3, the direction of the vector indicated by the arrow A is the direction of the azimuth information. Here, it is assumed that the arrangement direction of the road obtained by the above-described processing is the direction of the vector indicated by the arrow B. At this time, the azimuth indicated by the arrow B is calculated by first calculating the angle formed by the arrow A and the arrow B, and then correcting the angle using the angle of view of the lens of the photographing unit 101 to thereby determine the azimuth of the arrow A and the arrow B. It can be calculated by obtaining a difference in corners and finally adding this difference to the azimuth information.

  For example, in the captured image of FIG. 3, it is assumed that the azimuth information indicated by the arrow A heading from the front to the front indicates north. On the other hand, the road arrangement direction is indicated by an arrow B from the lower right to the upper left. At this time, if the difference in azimuth between the arrow A and the arrow B is −45 degrees, the azimuth of the arrow B is northwest.

  Subsequently, the error detection unit 105 acquires map information from the map storage unit 104 using the position information acquired from the terminal device 110 as a key (SB3). At this time, the map storage unit 104 acquires map information in the vicinity of the position information given from the error detection unit 105 from the map database and outputs the map information to the error detection unit 105. Here, it is assumed that the map database stores map information including road arrangement status, that is, map information including information on road coordinates and arrangement directions at the coordinates, in association with position information including latitude and longitude.

  The error detection means 105 extracts a road existing at the latitude and longitude indicated by the position information from the map information, and specifies the arrangement direction of the road. Further, the error detection unit 105 compares the road arrangement direction calculated based on the map information with the road arrangement direction calculated based on the photographed image in SB2. As a result, if there is a difference between the two, the difference is calculated (SB4). This difference indicates an error included in the road arrangement direction calculated based on the photographed image, assuming that the road arrangement direction in the map information is accurate. Here, since the road arrangement direction calculated based on the photographed image is calculated based on the direction information generated by the direction detection unit 102 of the terminal device 110, the difference is, in the end, the direction detection unit. It can be recognized that the error 102 has.

  The concept of the process related to SB4 will be described with reference to FIG. In FIG. 7, a solid arrow indicates a road arrangement direction based on map information. On the other hand, a broken arrow indicates a road arrangement direction calculated based on a captured image. Here, when the azimuth angle of the difference between the solid line arrow and the broken line arrow is α, the error detection unit 105 recognizes that the error of the direction detection unit 102 is α degrees.

  After the above processing, the error detection unit 105 transmits the error α calculated in SB4 as a correction amount for the direction detection unit 102 to the terminal device 110 by the communication unit (SB5).

  The sequence of the above-described processing executed in the terminal device 110 and the server device 120 will be described with reference to FIG.

  First, the terminal device 110 acquires a captured image, direction information, and position information from each unit of the terminal device 110, and transmits the acquired image to the server device 120 (SC1).

  Next, the error detection means 105 of the server device 120 gives the position information received from the terminal device 110 to the map database of the map storage means 104 as a key (SC2).

  The map database of the map storage means 104 returns map data including road arrangement status to the error detection means 105 based on the obtained position information (SC3).

  Finally, the error detection unit 105 compares the road arrangement direction specified from the captured image and the direction information received from the terminal device 110 with the road arrangement direction specified from the map data, and detects the direction of the terminal device 110. The azimuth correction amount necessary for the device 102 to output an accurate azimuth is calculated and transmitted to the terminal device 110 (SC4).

  In this embodiment, since the orientation detection unit 102 is corrected based on the captured image acquired by the imaging unit 101 that is not easily affected by magnetism, the electronic compass or the like was affected by the magnetism due to a change in the surrounding environment. However, it is possible to appropriately correct the orientation information.

  In the present embodiment, the server device 120 performs processing that requires relatively high processing capability, such as analysis processing of captured images, analysis processing of map data, and calculation processing of azimuth correction amounts. The amount of information processing in the terminal device 110 can be reduced, and the power consumption of the terminal device 110 can be suppressed.

  Further, in the present embodiment, on the basis of data that can be detected easily from a still image of a certain temporary point, which is a road arrangement direction, which is a detection target in terms of image processing technology, which is relatively easy to detect. Since the azimuth correction amount is calculated, the correction amount of the azimuth detecting means 102 can be obtained with a relatively smaller processing amount than the background art.

  Furthermore, since the data related to the road arrangement status used for the azimuth correction according to the present embodiment is information that is widely installed in devices such as navigation systems today, the present invention can be applied to these devices. Integration is technically easy.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  For example, in the above-described embodiment, the road arrangement direction is used to calculate the correction amount of the azimuth detecting means 102. However, the road need not necessarily be used, and can be detected from a photographed image such as a building or a mountain. It may be any fixed object that is included in the map information.

  In the above-described embodiment, the hardware configuration has been described. However, the present invention is not limited to this, and arbitrary processing may be realized by causing a CPU (Central Processing Unit) to execute a computer program. Is possible. In this case, the computer program can be stored and provided to the computer using various types of non-transitory computer readable media. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)). The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

DESCRIPTION OF SYMBOLS 100 Direction correction system 101 Imaging | photography means 102 Direction detection means 103 Position detection means 104 Map storage means 105 Error detection means 110 Terminal device 120 Server apparatus

Claims (10)

Photographing means for acquiring a photographed image of a landscape including a road;
Azimuth detecting means for detecting the shooting direction of the shot image;
Position detecting means for detecting the shooting position of the shot image;
Map storage means for storing map information including road arrangement status;
Road arrangement direction on the map at the shooting position specified on the basis of the direction of road arrangement on the shot image specified based on the shot image and the shooting direction, and the map information and the shooting position. An azimuth correction system comprising error detection means for comparing the direction and calculating an error of the azimuth detection means. The azimuth correction system has a server device and a terminal device that are communicably connected,
The terminal device includes the photographing unit, the direction detection unit, and the position detection unit,
The azimuth correction system according to claim 1, wherein the server device includes the map storage unit and the error detection unit. A communication means for communicating with the server device;
Photographing means for acquiring a photographed image of a landscape including a road;
Azimuth detecting means for detecting the shooting direction of the shot image;
Position detecting means for detecting the shooting position of the shot image;
Map storage means for storing map information including road arrangement status,
The communication device transmits the captured image, the imaging orientation, and the imaging orientation to the server device, and receives information related to an error of the orientation detection unit from the server device. A communication means for receiving a photographed image of a landscape including a road, a photographing position and a photographing orientation from a terminal device;
Road arrangement direction on the map at the shooting position specified on the basis of the direction of road arrangement on the shot image specified based on the shot image and the shooting direction, and the map information and the shooting position. Error detection means for comparing the direction and calculating the error of the direction detection means,
The communication device is a server device that transmits information related to the error to the terminal device. Acquiring a captured image of a landscape including a road;
Detecting a shooting direction of the shot image;
Detecting a shooting position of the shot image;
The map at the shooting position specified based on the direction of road arrangement on the captured image specified based on the captured image and the shooting direction, map information including road arrangement status, and the shooting position. Comparing with an arrangement direction of the upper road, and calculating an error of the azimuth detecting means. Acquiring a captured image of a landscape including a road;
Detecting a shooting direction of the shot image;
Detecting a shooting position of the shot image;
A method of transmitting the photographed image, the photographing azimuth, and the photographing azimuth to the server device, and receiving from the server device information related to an error of the azimuth detecting means. Receiving a captured image of a landscape including a road, a shooting position and a shooting direction from a terminal device;
The map at the shooting position specified based on the direction of road arrangement on the captured image specified based on the captured image and the shooting direction, map information including road arrangement status, and the shooting position. Comparing the direction of the road on the upper side and calculating an error of the direction detection means;
Transmitting the information related to the error to the terminal device. A program for causing a computer to execute a predetermined operation,
Acquiring a captured image of a landscape including a road;
Detecting a shooting direction of the shot image;
Detecting a shooting position of the shot image;
And a step of transmitting the photographed image, the photographing azimuth and the photographing azimuth to the server device, and receiving from the server device information related to an error of the azimuth detecting means. A program for causing a computer to execute a predetermined operation,
Acquiring a captured image of a landscape including a road;
Detecting a shooting direction of the shot image;
Detecting a shooting position of the shot image;
And a step of transmitting the photographed image, the photographing azimuth and the photographing azimuth to the server device, and receiving from the server device information related to an error of the azimuth detecting means. A program for causing a computer to execute a predetermined operation,
Receiving a captured image of a landscape including a road, a shooting position and a shooting direction from a terminal device;
The map at the shooting position specified based on the direction of road arrangement on the captured image specified based on the captured image and the shooting direction, map information including road arrangement status, and the shooting position. Comparing the direction of the road on the upper side and calculating an error of the direction detection means;
And a step of transmitting information relating to the error to the terminal device.