JP4433385B2 - Destination guide device and portable terminal device - Google Patents

Description

  The present invention relates to a destination guidance device and a portable terminal device that guide a user to a destination (navigation) on a display screen of a portable terminal device such as a mobile phone, and more particularly to a destination guidance device using a camera-captured video. And a portable terminal device.

  2. Description of the Related Art Conventionally, navigation devices that use an image (video) taken with a digital camera have been proposed as navigation devices for guiding a vehicle or a pedestrian.

  For example, the apparatus described in Patent Document 1 stores in advance data or a real image that models a space in which a vehicle travels, and collates such an estimated image with an actually captured image (scene image). (Pattern matching) is performed to embed instruction information (an arrow indicating the traveling direction) in the scene image.

The device described in Patent Document 2 displays a traveling direction to a destination based on the user as an arrow on the screen of the mobile phone when the user of the mobile phone moves to the destination.
JP 2003-14477 A JP 2003-83762 A

  However, the apparatus described in Patent Document 1 needs to create data that is modeled in advance for a place that is subject to road guidance, or store an actual image, even if it is only for a main motorway. Preparing such data and actual images is expensive both in terms of cost and work. In addition, when performing road guidance for pedestrians using a mobile terminal, the road on which a person walks becomes a problem, and the direction and height of a captured image taken by the mobile terminal varies from the captured image when the vehicle is running. Therefore, it is considered that it is difficult to model and match an image that is contrasted with a captured image.

  On the other hand, since the apparatus described in Patent Document 2 does not use a captured image, there is no such problem, but simply displaying an arrow indicating the direction of the destination on the display screen reveals the relationship with the actual landscape at hand. There was a problem that it was difficult.

  The present invention has been made in such a background, and the purpose of the present invention is to provide a destination guide that can superimpose an instruction display on a captured image with a relatively simple configuration without preparing an image to be collated with the captured image in advance. It is providing a device and a portable terminal device.

  A destination guide apparatus according to the present invention includes a digital camera, a display unit that displays an image captured by the digital camera, an operation unit that inputs a destination, a current position detection unit that detects a current position, and the current position. A screen demarcating means for demarcating the screen of the digital camera on the basis of the above, a destination direction coordinate corresponding to the destination, and a destination direction superimposed on a photographed image of the digital camera on the display screen of the display unit And a control means for displaying instructions with respect to the coordinates.

  In this configuration, the current position detecting means detects the current position, and the screen defining means defines the screen of the digital camera with reference to the current position. The control means determines a destination direction coordinate corresponding to the destination based on the current position and the destination, and superimposes the captured image of the digital camera on the display screen of the display unit to display an instruction for the destination direction coordinate. I do.

  The photographed image is preferably a through image of the digital camera. Thereby, the photographed image can follow the change of the landscape almost in real time according to the movement of the user.

  The screen demarcating means includes an azimuth detecting means for detecting the azimuth facing the destination guide device, a camera direction detecting means for detecting a direction in a vertical plane to which the lens of the digital camera is facing, and the digital camera. The screen can be defined based on the output of both detection means and the imaging magnification.

  More specifically, the screen is defined by a range of horizontal and vertical angles of view around the current position, and the destination direction coordinate is the direction vector of the direction vector extending from the current position to the destination. It is determined by the relative angle ratio within the range of the angle of view and the size of the screen.

  The camera direction detecting means can be constituted by means for detecting a current direction in which the destination guidance apparatus is facing and means for detecting an inclination angle of the destination guidance apparatus. In this case, when the camera unit is configured to rotate with respect to the destination guide main body, the direction in the vertical plane to which the lens faces can be detected in consideration of the rotation angle.

  The control means may suppress at least updating of the indication display when the amount of change is smaller than a predetermined threshold in both the direction vector from the current position to the destination and the current position. As a result, the processing load of the apparatus is reduced without significantly impairing the destination guidance function.

  The present invention can also be understood as a mobile terminal device equipped with a digital camera.

  The present invention can also be understood as a computer program installed in a terminal device or a recording medium in which the computer program is recorded so as to be readable in order to realize the operation of the destination guide device in the terminal device.

  ADVANTAGE OF THE INVENTION According to this invention, the destination guidance apparatus which can superimpose an instruction | indication display on a picked-up image with a comparatively simple structure can be provided, without preparing the image which should be collated with a picked-up image beforehand. The present invention is particularly suitable for guiding a pedestrian by applying it to a portable terminal device such as a mobile phone with a camera.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the drawings. In this embodiment, an example in which a mobile terminal such as a mobile phone is used as a destination guidance device will be described.

  FIG. 1 shows a front view (a) and a rear view (b) showing the external appearance of the portable terminal 401. As can be seen from the front view, this portable terminal has a display (display unit) 201 and a key operation unit 202 on the front side, and a speaker 302 on the back side. In addition, a camera (digital camera) 200 is provided on the top of the portable terminal. Although the lens 301 of the camera is directed to the back side, in the case of a rotary type, the photographing direction can be set regardless of the inclination of the mobile terminal body by having an axis with respect to the horizontal direction of the mobile terminal body. . In addition, here, a so-called stick-type portable terminal is shown as an example, but a two-fold type portable terminal may be used.

  FIG. 2 is a block diagram illustrating a schematic configuration of the mobile terminal 401. This mobile terminal includes an acceleration sensor 100, a geomagnetic sensor 101, a camera 102, a GPS receiver 103, a central processing unit 104 that performs functional processing of the mobile terminal itself, a memory 106, a microphone / speaker 107, a communication unit 108, a display 201, A key operation unit 202 and the like are provided. When the camera 102 is a rotation type, a means (not shown) for detecting the rotation angle is also provided. Any known means can be used as this means.

  From the acceleration sensor 100, the acceleration of the mobile terminal 401 is detected, and the current speed and the angle 404 (FIG. 3) of the mobile terminal itself are detected. An angle 404 is a terminal tilt angle formed by a direction perpendicular to the back surface of the terminal 401 and a vertical line. The geomagnetic sensor 101 detects the azimuth of geomagnetism, and recognizes the azimuth that the mobile terminal (and thus the user) faces based on the change in the detected azimuth. The camera 102 is used to capture an image in the user traveling direction. The GPS receiver 103 detects the current position (latitude and longitude) of the mobile terminal. Information on these sensors is processed by the central processing unit 104 and displayed on the display 201, stored in the memory 106, or transmitted to the user as sound through a speaker. The communication unit 108 is a part that performs wireless voice and data communication with a base station (not shown) via an antenna. As will be described later, the communication unit 108 accesses a server that provides navigation information (such as map information) on the communication network.

  FIG. 4 shows the relationship between various positions and angles around the user's position. Here, the user coordinates α (αx, αy, αz) as the current position coordinates are set as a reference position, and the direction in which the camera lens faces is set as a center direction vector 711. Here, αx, αy, and αz are longitude, latitude, and altitude, respectively. However, if the altitude is known, it can be used, but even if it is not used, the error is considered to be within a predetermined range, so the altitude need not necessarily be used. Further, the angle of view of the camera is determined with the center direction vector 711 as the center. Here, the angle of view is determined by the horizontal opening angle θ and the vertical opening angle φ of a virtual screen (corresponding to a display screen) 702 captured from the position of the user coordinate α. The screen 702 becomes a part of a spherical surface with the user coordinate α as the origin, and substantially corresponds to the display screen of the display. The direction vector 713 is obtained by projecting the center direction vector 711 onto the horizontal plane including the user coordinates α. An angle from the reference direction (for example, east) to the traveling direction vector 713 is defined as an angle ε. In addition, an angle from the traveling direction vector 713 to the central direction vector 711 is defined as an angle δ. A point where the destination direction vector 712 intersects the screen 702 is defined as an intersection point 707.

  Next, the operation of the present embodiment will be described with reference to the flowchart of FIG. This process is realized by the central processing unit 104 executing a predetermined program.

  First, a destination is set according to a user's operation (S11). Specifically, the destination is set by the user selecting a preset building, place, or the like, or inputting an address (a place name, an address, etc.).

  Next, the current position of the user (mobile terminal) is measured (S12). Specifically, the user coordinate α (αx, αy, αz) is measured using the GPS receiver 103. Further, the current orientation of the mobile terminal body is measured (S13). This direction is regarded as the user's direction of travel. For example, the angle ε from the reference direction (for example, east) 710 to the traveling direction vector 713 is measured with respect to the horizontal direction by the geomagnetic sensor 101. Next, the terminal tilt angle 404 is measured by the acceleration sensor 100 (S14).

  Therefore, it is checked whether the camera is a rotary type (S15). If it is a fixed type, the screen angle δ is obtained from the terminal tilt angle 404 (S17). If the rotation type, the camera rotation angle is confirmed (S16), and the terminal tilt angle 404 is corrected by this to obtain the screen angle δ (S17). A center direction vector 711 is determined by the angle ε and the angle δ. Next, the angle of view (θ, φ) is calculated from the currently set optical magnification and the enlargement ratio by digital zoom (S18).

  Next, a destination direction vector 712 is obtained from the destination coordinates β (βx, βy, βz) and the user coordinates α (αx, αy, αz) (S19). The destination coordinates β (βx, βy, βz) can be recognized by the server and received from the server based on the destination input by the user in step S11. It is checked whether or not this destination direction vector 712 falls within the screen 702 (S20). This can be done by comparing the horizontal and vertical angles of the destination direction vector 712 with the angle range of the screen 702. If within the screen 702, the relative position information within the screen is calculated (S21). The coordinates of the intersection 707 are not absolute coordinates but are obtained as relative positions on the screen (the distance from the user coordinates to the screen position is arbitrary).

  FIG. 6A shows the relationship between the vectors in the horizontal plane including the traveling direction vector 713. The angle of view θ is determined with the traveling direction vector 713 as the center. The coefficient a is determined by the position of the destination direction vector 712 within the angle of view θ. FIG. 6B shows the relationship between the vectors on the vertical plane including the traveling direction vector 713. A vector forming an angle δ on the vertical plane with the traveling direction vector 713 is a central direction vector 711, and the angle of view φ is determined with the central direction vector 711 as a center. The coefficient b is determined by the position of the destination direction vector 712 within the angle of view φ. These coefficients a and b substantially function as relative positions.

  Therefore, as shown in FIG. 7, the relative position on the screen displaying the camera-through image is converted into destination direction coordinates 1001 (S22). This is because ratios a and b of the horizontal and vertical angles of the destination direction vector 712 with respect to the respective angle of view when the horizontal and vertical angles of view are respectively assigned to the horizontal and vertical sizes of the screen. Can be obtained by proportionally allocating both sizes.

  “Camera-through” refers to an operation of continuously displaying images obtained by the image sensor in real time in order to select a subject before pressing the shutter of the camera. The guidance display is updated by displaying the arrow pointing to the coordinate 1001 so as to overlap the camera-through image (S23). As a result, the user can see the instruction display indicating the updated destination direction on the camera through image that changes as the user walks.

  The processes in steps S12 to S23 are repeatedly executed until the destination is reached (S24). When the user arrives at the destination, the user may be notified on the screen and notified by voice or sound.

  The following can be considered as the guidance display in step S23.

(1) Method with Reference Position of Arrow on Screen Lower Side For example, as shown in FIG. 8, an arrow 1004 extending from the screen lower side center 1002 to the destination direction coordinate 1001 is displayed as one form of instruction display. FIG. 9 shows an example in which an arrow 1004 is superimposed and displayed on an actual camera through image. According to these display methods, since the arrow 1004 indicating the destination on the screen is the direction in which the user should proceed, there is an advantage that the user can easily understand. As shown in FIG. 10, the distance to the destination may be displayed together with the arrow 1004. The distance here is assumed to be a linear distance between coordinates, but if available from the navigation system, a distance display 1003 along the route may be added. The distance from the current position to the destination may be calculated as a straight line distance at the terminal, or the location information of the destination and the current position is sent to a predetermined server via the communication network, and the server follows the shortest path. The distance may be obtained and the result may be received.

  When the destination direction coordinate B is outside the screen 702, the arrow 1004 disappears from the screen 702, so that the user knows that the destination direction is out of the screen range.

When the destination direction coordinate B is outside the screen 702, a part of the spherical surface centered on the user coordinate α is the screen 702. Therefore, even if the destination direction coordinate is outside the screen, the coordinates on the screen are it is possible to find the destination direction coordinates of the system. Therefore, instead of erasing the arrow, as shown in FIG. 11, the user draws a line from the lower center 1002 of the screen to the destination direction coordinate 1005 and draws an arrow 1006 on the line, so that the user recognizes the direction to proceed. be able to.

  The display by the arrow in the present embodiment indicates the direction of the destination from the current position, and does not necessarily guide the route (way) on the way, but the direction of the destination on the actual shooting screen is It is useful because it understands. Further, the route guidance may be displayed by switching to a conventional guidance screen based on map display, or the screen may be divided and both displays may be displayed together in the divided area.

(2) Method of having the reference position of the arrow at the center of the screen The center of the screen may be used as the reference position of the arrow instead of the lower side of the screen. For example, as shown in FIG. 12, when the destination direction coordinate A1202 is in the screen 702, an arrow A1204 indicating the direction with respect to the destination direction coordinate A1202 in the screen obtained from the screen center 1201 is displayed. To do. When the destination direction coordinate B is outside the screen 702, as in the case of FIG. 11, even if the destination direction coordinate is outside the screen, the screen center 1201 with respect to the destination direction coordinate B1203 in the coordinate system on the screen. By drawing a line from the line and drawing an arrow B1205 on the line, the user can recognize the direction to go.

(3) Modifications In any of the above (1) and (2), when the destination direction coordinates are not in the screen, the arrow or the like for indicating the direction is different from the case where the destination direction is on the screen. By displaying in a display form (shape / color, blinking, etc.), the user may recognize that the destination direction coordinates are outside the screen.

  It should be noted that the direction indication arrow moves violently even when the user coordinate α slightly changes when the destination direction deviates greatly from the traveling direction. In this case, the text display 1300 may indicate that the moving direction of the arrow is not an arrow but is greatly shifted as shown in FIG. 13 without displaying the intense movement of the arrow.

  FIG. 14 is a flowchart for explaining the operation in the second embodiment of the present invention. The apparatus configuration is the same as that of the first embodiment. 14, the same processing steps as those in FIG. 5 are denoted by the same reference numerals, and detailed description thereof is omitted. The difference from the process of FIG. 5 is that a new step S19a is added between steps S19 and S20. In step S19a, when the amount of change is small in both the direction vector 712 to the destination and the current position (smaller than the respective predetermined threshold values), in-screen discrimination (S20), relative position calculation (S21), coordinates The conversion (S22) and the guidance display update (S23) are omitted, and the process proceeds to step S24. As a result, it is possible to prevent the guidance display from being updated more frequently than necessary, to reduce the processing load on the mobile terminal, and to reduce the amount of current consumption. Further, the update of the camera-through image itself may be similarly suppressed.

  Although the preferred embodiments of the present invention have been described in detail above, various modifications and changes other than those mentioned above can be made. For example, the distance to the destination is not an information display that requires character interpretation such as the distance display, but the size and color of an arrow or the like for direction indication may be changed according to the distance to the destination. Good.

It is the front view (a) and back view (b) which show the external appearance of the portable terminal in embodiment of this invention. It is a block diagram which shows schematic structure of the portable terminal shown in FIG. It is a figure which shows the relationship between a user and a portable terminal. It is a figure which shows the relationship of the various positions and angle centering on a user's position. It is a flowchart for demonstrating the operation | movement of this Embodiment. It is a figure which shows the relationship of each vector in a horizontal surface and a perpendicular surface containing a advancing direction vector. It is explanatory drawing of how to obtain | require the destination direction coordinate on a screen in embodiment of this invention. It is a figure which shows one form of the instruction | indication display on the basis of the screen lower side center in embodiment of this invention. It is a figure which shows the example which overlapped and displayed the arrow on the actual camera through image. It is a figure which shows the example which displayed the distance to the destination with the arrow shown in FIG. It is a figure which shows the example of a display of the arrow in case the destination direction coordinate is outside a screen in embodiment of this invention. It is a figure which shows the other form of the instruction | indication display on the basis of the screen center in embodiment of this invention. It is a figure which shows the text display instead of the arrow in embodiment of this invention. It is a flowchart for demonstrating the operation | movement in the 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Acceleration sensor, 101 ... Magnetic sensor, 102 ... Camera, 103 ... GPS receiver, 104 ... Central processing unit, 106 ... Memory, 107 ... Microphone / speaker, 108 ... Communication part, 201 ... Display, 202 ... Key operation part , 301 lens, 302 speaker, 401 mobile terminal, 404 tilt angle, 702 screen, 707 intersection, 711 center direction vector, 712 destination direction vector, 713 travel direction vector, 1001 purpose Ground direction coordinates, 1002 ... Center of screen bottom, 1004 ... Arrow, 1005 ... Destination direction coordinates, 1006 ... Arrow, 1201 ... Screen center, 1300 ... Text display, α ... User coordinates, β ... Destination coordinates

Claims (6)

A destination guidance device for guiding a user to a destination on a display screen,
A digital camera,
A display unit for displaying images taken by the digital camera;
An operation unit for inputting a destination;
Current position detecting means for detecting the current position;
Direction detection means for detecting the direction to which the destination guidance device is directed, camera direction detection means for detecting the direction in the vertical plane to which the lens of the digital camera is facing, and the imaging magnification of the digital camera are set Screen defining means for defining a screen of the digital camera in a range of horizontal and vertical angles of view centered on the current position based on outputs of both detection means and the imaging magnification .
A destination direction coordinate corresponding to the destination is determined by a ratio of a relative angle within a range of the angle of view of a direction vector extending from the current position to the destination, and the size of the screen, and the display unit superimposed on the captured image of the digital camera on the display screen and a control means for instructing the display for the destination direction coordinates,
The control means displays an arrow on the line connecting the reference position corresponding to the current position in the screen and the destination direction coordinates as the instruction display .   The destination guide device according to claim 1, wherein the photographed image is a through image of the digital camera. The camera direction detecting means, the purpose of claim 1, wherein further comprising means for detecting a current orientation facing the destination guidance device, and means for detecting the inclination angle of the destination guide device Ground guidance device. When the camera unit is configured to rotate with respect to the destination guide main body, the camera direction detection unit detects a direction in a vertical plane to which the lens is facing in consideration of the rotation angle. The destination guide device according to claim 3 .   2. The control unit according to claim 1, wherein, when the amount of change is smaller than a predetermined threshold value in both the direction vector from the current position to the destination and the current position, the control unit suppresses at least updating of the indication display. Destination guide device. A mobile terminal device that guides a user to a destination on a display screen,
A digital camera,
A display unit for displaying images taken by the digital camera;
An operation unit for inputting a destination;
Current position detecting means for detecting the current position;
Direction detection means for detecting the direction to which the destination guidance device is directed, camera direction detection means for detecting the direction in the vertical plane to which the lens of the digital camera is facing, and the imaging magnification of the digital camera are set Screen defining means for defining a screen of the digital camera in a range of horizontal and vertical angles of view centered on the current position based on outputs of both detection means and the imaging magnification .
Communication means for performing at least data communication;
A destination direction coordinate corresponding to the destination is determined by a ratio of a relative angle within a range of the angle of view of a direction vector extending from the current position to the destination, and the size of the screen, and the display unit superimposed on the captured image of the digital camera on the display screen and a control means for instructing the display for the destination direction coordinates,
The control means displays an arrow on a line connecting a reference position corresponding to a current position in the screen and the destination direction coordinates as the instruction display .