JP2005274150A - Mobile information terminal and azimuth information utilizing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To acquire necessary azimuth information regardless of a used state of a terminal and effectively utilize it. <P>SOLUTION: The mobile information terminals can measure an azimuth by a magnetic sensor 7. If execution of azimuth is directed with an azimuth measurement lock button 8a, a CPU1 records the azimuth information measured with the magnetic sensor 7 in a memory 2. Then, when a camera module 9 executes image photographing, the CPU1 produces letters on the basis of the azimuth information recorded in the memory 2, and composes the letters on the photographed image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a portable information terminal having a sensor capable of measuring an azimuth and a method of using azimuth information in the terminal.

  In recent years, geomagnetic sensors capable of measuring orientation are installed in portable information terminals such as automobile navigation systems, mobile phones and PDAs (personal digital assistants), and the orientation information measured by these geomagnetic sensors is used for various processes. (For example, Patent Document 1).

The geomagnetic sensor is basically installed on a horizontal plane at an observation point, and is configured to detect a biaxial component of a geomagnetic vector on the horizontal plane. The magnetic direction can be calculated from the biaxial components detected by the geomagnetic sensor and used as the direction information.
JP 2002-196055 A

  By the way, an increasing number of portable information terminals such as mobile phones and PDAs are equipped with an imaging device (camera) capable of capturing an image. In addition, there is a device that can display a current position on a map using a GPS (Global Positioning System).

  In such a conventional portable information terminal, when using orientation information measured by a magnetic sensor, when the display surface of the display device and the optical axis of the camera are configured to be orthogonal, the display surface and the magnetic sensor are mounted. Since the planes are mounted so as to be parallel, it has been difficult to obtain accuracy in both azimuth measurement during map display and azimuth measurement during camera shooting.

  Also, for example, if you want to add direction information that the camera is facing to the captured image (in the case of a landscape image, etc.) and if you want to add the opposite direction information (in the direction of the surface of a building, etc.) It was difficult to use properly.

  The present invention has been made paying attention to the above circumstances, and the purpose thereof is a portable information terminal capable of acquiring and effectively using information on a required direction regardless of the use state of the terminal It is to provide a direction information utilization method.

  In order to achieve the above object, the present invention provides a measuring means for measuring an azimuth by a magnetic sensor, an instruction input means for inputting an instruction to perform an azimuth measurement by the measuring means, and an instruction input by the instruction input means. The azimuth information recording means for recording the azimuth information measured by the measuring means, the image photographing means for photographing an image, and the character generation for generating the character based on the azimuth information recorded by the azimuth information recording means And recording means for recording the image photographed by the image photographing means and the character generated by the character generating means in association with each other.

  In addition, the measuring means for measuring the azimuth in two axes by one magnetic sensor, the map information displaying means for displaying the map information, and the information on the azimuth measured by the measuring means are displayed by the map information displaying means. Direction information adding means for adding to the map information.

  According to the present invention, it is possible to acquire information on a required direction regardless of the usage state of the terminal and effectively use it.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing the configuration of the portable information terminal in this embodiment. As shown in FIG. 1, the portable information terminal includes a CPU 1, a memory 2, an LCD controller 3, an LCD 4, an I / O controller 5, an A / D converter 6, a biaxial geomagnetic sensor 7 (hereinafter simply referred to as a magnetic sensor 7). , Button 8 and camera module 9 are connected via an address bus 11 and a data bus 12.

  In the portable information terminal, the CPU 1 operates in accordance with a program incorporated in the memory 2 and displays necessary information on the LCD 4 in accordance with an instruction input from the button 8 by the user. In the present embodiment, it is assumed that a program and data for displaying map information using position information acquired by the GPS module 10 and displaying images taken by the camera module 9 are recorded in the memory 2. Furthermore, in the portable information terminal in the present embodiment, the azimuth information acquired by the magnetic sensor 7 can be used for map display and image display (details will be described later).

  Information on the operation of the button 8 by the user is transmitted to the CPU 1 via the I / O controller 5. The display on the LCD 4 is performed through the LCD controller 3. Similarly, the A / D converter 6 connected to the I / O controller 5 is connected to a magnetic sensor 7 capable of measuring the orientation in the horizontal plane direction so that the CPU 1 can read the magnetic information. The camera module 9 is also connected to the CPU 1 and performs an imaging operation in response to an instruction from the CPU 1. The photographed image information is stored in the memory 2 and processed. A GPS (Global Positioning System) module 10 makes it possible to acquire latitude and longitude information in this apparatus using satellite position information. The map information can be displayed by a program stored in the memory 2 to indicate the current position, and the direction can be displayed based on the direction information detected by the magnetic sensor 7. The CPU 1, the LCD controller 3, the I / O controller 5, the memory 2, and the camera module 9 are connected via an address bus 11 and a data bus 12.

  FIG. 2 shows an external configuration of the portable information terminal. 2A shows the front side where the LCD 4 is provided, and FIG. 2B shows the back side.

  In addition to the LCD 4, a plurality of buttons 8 are provided on the front side of the portable information terminal. The plurality of buttons 8 include an orientation measurement lock button 8a that is operated to instruct execution of orientation measurement by the magnetic sensor 7, and a shutter button 8b that is operated to instruct execution of image shooting.

  An imaging optical system 9a for the camera module 9 is provided on the back side of the portable information terminal. The imaging optical system 9a is normally covered by the lens cover 14, and is exposed when the lens cover 14 is slid as shown in FIG. In this embodiment, the magnetic sensor 7 is attached to the imaging optical system 9a. A detailed mounting structure will be described later (FIGS. 3, 4, 5, 6, and 7). By mounting the magnetic sensor 7 on the imaging optical system 9a, it is easy to grasp the relationship between the optical axis direction of the imaging optical system 9a (image shooting direction) and the measurement surface of the orientation information by the magnetic sensor 7.

  First, the mounting structure of the magnetic sensor 7 in this embodiment will be described.

  FIG. 3A is a side sectional view of the portable information terminal, and FIG. 3B is a diagram showing the mounting position of the magnetic sensor 7 as viewed from the optical axis direction of the imaging optical system 9a.

  In FIG. 3, the LCD 4 is arranged in parallel to the main board 20 on which main circuit components of the portable information terminal are mounted. An imaging optical system 9a is mounted on the main substrate 20 so that the optical axis direction is perpendicular to the main substrate 20 (perpendicular to the display screen of the LCD 4). The imaging optical system 9a has a substantially cylindrical shape, and a sub-substrate support housing 22 is mounted on the outer periphery of the imaging optical system 9a so as to be rotatable (in the arrow direction shown in FIG. 3B). In the sub-substrate support housing 22, the sub-substrate 26 on which the magnetic sensor 7 is mounted can be rotated by a rotation axis orthogonal to the rotation axis of the sub-substrate support housing 22 (the optical axis of the imaging optical system 9a). Is provided with a hinge mechanism 24 (in the direction of the arrow shown in FIG. 3A). That is, the magnetic sensor 7 is rotatable with respect to the imaging optical system 9a by the rotation axis (first axis) of the sub-substrate support housing 22, and the axis of the hinge mechanism 24 (second axis) orthogonal to the axis. ). For this reason, the magnetic sensor 7 can make the orientation measurement plane (two axes) parallel to the ground surface, no matter how the portable information terminal is tilted. That is, it is possible to obtain a direction in which magnetism can be measured with high accuracy, and accurate azimuth information can be acquired.

  In the portable information terminal of this embodiment, for example, the orientation of the magnetic sensor 7 can be adjusted manually. That is, it is assumed that the sub-substrate 26 can be rotated and the sub-substrate 26 can be rotated by the hinge mechanism 24 after the state shown in FIG.

  In addition, it is possible to provide a mechanism capable of rotational drive control with respect to the sub-substrate support housing 22 and the hinge mechanism 24 so that the adjustment can be performed according to an instruction from the CPU 1.

  FIG. 3 shows the position of the magnetic sensor 7 suitable for the map display application. When displaying a map, the portable information terminal is used in a state where the display surface of the LCD 4 is parallel to the ground, for example. In this state, the sub-board 26 on which the magnetic sensor 7 is mounted is parallel to the LCD 4 so that it is parallel to the ground surface, and the geomagnetism can be detected accurately in the horizontal direction (two axes).

  When the portable information terminal displays a map with the map display function, the portable information terminal displays a character string or a symbol representing the direction together with the map based on the direction information acquired by the magnetic sensor 7, or the direction of the portable information terminal The map can be rotated for display and used for other processing.

  FIG. 4 shows the position of the magnetic sensor 7 suitable for photographing with a vertically long composition (called a portrait position) by the camera module 9. The portable information terminal is used in a state where the display surface of the LCD 4 is perpendicular to the ground as shown in FIG. 2, for example, when taking an image at the portrait position. In this case, the sub-board 26 is rotated with respect to the sub-board support housing 22 by the hinge mechanism 24 so as to be perpendicular to the LCD 4, and the sub-board support housing 22 is rotated to turn the sub-board 26 and the ground surface. Are parallel to each other (parallel to the horizontal side of the display screen shown in FIG. 2A). Thus, by making the sub-board 26 parallel to the ground surface, the geomagnetism can be accurately detected in the horizontal direction (two axes) when the camera module 9 captures an image.

  FIG. 5 shows the position of the magnetic sensor 7 suitable for shooting with a horizontally long composition (referred to as a landscape position) by the camera module 9. When taking an image at a landscape position, the portable information terminal is used, for example, in a state rotated by 90 ° from the state shown in FIG. 2 (the display surface of the LCD 4 is perpendicular to the ground). In this case, the sub-board 26 is rotated with respect to the sub-board support housing 22 by the hinge mechanism 24 so as to be perpendicular to the LCD 4, and the sub-board support housing 22 is rotated to turn the sub-board 26 and the ground surface. Are parallel to each other (parallel to the vertical side of the display screen shown in FIG. 2A). Thus, by making the sub-board 26 parallel to the ground surface, the geomagnetism can be accurately detected in the horizontal direction (two axes) when the camera module 9 captures an image.

  When the portable information terminal takes an image with the camera module 9, the portable information terminal can be used to add the azimuth information acquired by the magnetic sensor 7 to the taken image. Details of the use of the direction information at the time of image shooting will be described later (FIGS. 8 and 9).

  In this way, the sub-board 26 on which the magnetic sensor 7 is mounted is mounted so as to be rotatable about two axes with respect to the main board 20 (imaging optical system 9a). In addition, regardless of various usage styles such as portrait position and landscape position, the magnetic sensor 7 can be adjusted to a sensitive direction suitable for geomagnetism measurement, and direction information can be obtained with high accuracy.

  Next, a mounting form of the magnetic sensor 7 different from the structure in which the sub-substrate support housing 22 and the hinge mechanism 24 can be rotated by two axes will be described (FIGS. 6 and 7).

  FIG. 6 shows an example in which the sub board 26 on which the magnetic sensor 7 is mounted is fixedly attached to the sub board support housing 22.

  FIG. 6 shows attachment positions specialized for two typical positions (portrait position, landscape position) when an image is captured by the camera module 9. The sub-board 26 (magnetic sensor 7) shown in FIG. 6 has a structure in which the sub-board support housing 22 is fixed at a position rotated (approximately 45 °) to an intermediate position between FIGS. 4B and 5B. is there. The sub board 26 is rotatable with respect to the sub board support housing 22 at this position. With this structure, both of the two orthogonal sides of the display screen of the LCD 4 are at an angle of approximately 45 degrees in both the portrait position and the landscape position. Accordingly, the geomagnetic component force can be measured regardless of the position of the portable information terminal when shooting, and the orientation information can be acquired without taking the trouble of rotating the sub-board support housing 22. Can do.

  FIG. 7 shows an example in which the sub-board 26 on which the magnetic sensor 7 is mounted is fixedly mounted.

  FIG. 7 shows a method of fixedly attaching the sub-board 26 when the user designates either a map display or a portrait position or a landscape position and uses direction information.

  In FIG. 7, the virtual cube shown in FIG. 7B, that is, a cube having a plane parallel to the display surface of the LCD 4 (X direction) and a side parallel to the optical axis (Y direction) of the camera module 9 is shown. Assuming that the sub-board 26 is aligned with the two axes (orientation measurement surfaces) of the magnetic sensor 7 in parallel with a plane (triangle in FIG. 7B) formed by connecting the three apexes that are not adjacent to each other in the cube. Attach.

  Thereby, at any of the three positions, the magnetic sensor 7 can detect the geomagnetic force and acquire the azimuth information.

  In this way, by limiting the positions where the apparatus is used, it is possible to obtain the azimuth information without the trouble of adjusting the magnetic sensor 7 (sub-board 26). In addition, the structure for mounting the magnetic sensor 7 is simplified, and it is possible to increase the accuracy of azimuth measurement while configuring the sensor at a low cost.

  Next, processing (image capturing application) using azimuth information measured by the magnetic sensor 7 in the present embodiment will be described.

  In the portable information terminal according to the present embodiment, a character (or character string) generated based on the orientation information acquired by the magnetic sensor 7 is synthesized with an image (photograph) taken using the camera module 9. The function (direction information addition function) that can be done is installed.

  With this direction information addition function, various settings can be made through the setting window shown in FIG. The CPU 1 displays a setting window as shown in FIG. 8 when a setting related to the azimuth information adding function is requested by a user instruction from the button 8.

  In the setting window, a check box is provided that allows the user to select whether or not to add orientation information to a captured image (item “attach orientation information”). When a check for adding azimuth information is input, the azimuth information acquired by the magnetic sensor 7 is used as it is, that is, azimuth information indicating the direction in which the imaging optical system 9a is facing is added, or the magnetic sensor 7 It is possible to select whether to add direction information in the opposite direction indicated by the acquired azimuth information, that is, to add information indicating the direction opposite to the direction in which the imaging optical system 9a is facing.

  When shooting a landscape or the like, the direction that matches the direction of the camera module 9 is written in the image, and when shooting a building, the direction that is opposite to the direction of the camera module 9 is the north side. Is allowed to be written on the image. Thereby, the azimuth information for explaining the direction (shooting direction) of the camera module 9 and the information for explaining the direction in which the object to be photographed faces can be easily switched and added to the image.

  Further, it is possible to arbitrarily select a character string (designated character string) to be added to characters representing azimuth information (for example, “east”, “west”, “south”, “north”). In the portable information terminal, a plurality of designated character strings that can be added to the orientation information are prepared. For example, it is assumed that “direction”, “side”, etc. are prepared in addition to “direction” and “side surface” shown in FIG. The user can select and set an arbitrary designated character string (or character) in advance from among the plurality.

  Next, the processing operation by the direction information addition function will be described with reference to the flowchart shown in FIG.

  First, the CPU 1 monitors whether there is an orientation measurement lock instruction by operating the orientation measurement lock button 8a (step A1). When an orientation measurement lock instruction is input here (step A1, Yes), the CPU 1 acquires orientation information measured by the magnetic sensor 7 and records it in the memory 2 (steps A2 and A3).

  That is, the user obtains the azimuth information to be added to the photographed image, and the azimuth measurement lock button 8a is set after the portable information terminal is in a state where the magnetic sensor 7 is horizontal with respect to the ground plane. By operating the azimuth information can be acquired and held with high accuracy.

  Next, it is assumed that the photographing application is started and the execution of image photographing is instructed by operating the shutter button 8b (step A4, Yes). The CPU 1 causes the camera module 9 to take an image and records the image data obtained by this shooting in the memory 2 (step A5).

  Here, the CPU 1 refers to the contents previously set by the setting window, and records characters representing the direction generated based on the direction information according to the setting in association with the image data (steps A6 to A11). In the following description, it is assumed that “add direction information” is set.

  First, it is determined which of the camera orientation information and the reverse orientation information is set to be added to the image (step A6).

  If the setting for adding camera orientation information is made (step A6, camera orientation), the CPU 1 determines the orientation characters (east, west, south, north, and south) indicated by the orientation information acquired by the magnetic sensor 7 recorded in the memory 2. ) And a preset designated character string are generated (step A10).

  For example, if “north” is obtained as the character based on the direction information, and “direction” is selected as the designated character string as shown in FIG. 8, a composite character string “north direction” is generated. .

  The CPU 1 records the image data recorded by photographing and the composite character string in association with each other (step A11). Here, for example, image data of a composite character string is combined with captured image data and recorded (step A11).

  Separately from the captured image data, data of a composite character string may be recorded. In this case, when display of a captured image with orientation information added is requested, a composite character string representing the orientation is added to the captured image based on the composite character string data and displayed.

  On the other hand, when the setting to add the reverse orientation information is made (step A6, reverse orientation), the CPU 1 reverses the orientation indicated by the orientation information acquired by the magnetic sensor 7 recorded in the memory 2 (180 °). Rotation) (step A7).

  Then, the CPU 1 generates a composite character string in which the characters (east, west, south, and north) in the direction indicated by the reverse direction obtained from the direction information acquired by the magnetic sensor 7 are combined with a preset designated character string (step). A8).

  For example, when “South” is obtained as the character based on the reverse orientation, and “side” is selected as the designated character string as shown in FIG. 8, a composite character string of “South Side” is generated. .

  The CPU 1 records the image data recorded by shooting and the composite character string in association with each other (step A9). Here, for example, image data of a composite character string is combined with captured image data and recorded. Also. As described above, the composite character string data may be recorded separately from the captured image data.

  In this way, it is possible to perform azimuth measurement by operating the azimuth measurement lock button 8a, hold the azimuth information, and add it to an image captured thereafter. Since the measurement information is updated again by operating the direction measurement lock button 8a, the state of the portable information terminal (magnetic sensor 7) is arbitrarily changed, and the direction information in that state can be easily obtained. Can be retained.

  Therefore, for example, even in the situation where the mobile information terminal needs to be photographed with a large tilt in image shooting, it is possible to accurately capture the image by changing the state of the mobile information terminal after correctly measuring the orientation beforehand. Azimuth information can be easily associated with a captured image.

  In the above description, the magnetic sensor 7 is described as being mounted on the imaging optical system 9a via the sub-board 26 and the sub-board support housing 22, but it is mounted at other positions of the portable information terminal. Of course it is possible. For example, it can be provided near the top of the casing of the portable information terminal. In this case, for example, as shown in FIG. 3, it is assumed that the magnetic sensor 7 is attached to the casing of the portable information terminal via a member that allows the magnetic sensor 7 to rotate about two axes.

The block diagram which shows the structure of the portable information terminal in this embodiment. The figure which shows the external appearance structure of the portable information terminal in this embodiment. The figure for demonstrating the attachment structure of the magnetic sensor 7 in this embodiment. The figure for demonstrating the attachment structure of the magnetic sensor 7 in this embodiment. The figure for demonstrating the attachment structure of the magnetic sensor 7 in this embodiment. The figure which shows the example which fixed and mounted | wore the sub board | substrate 26 with which the magnetic sensor 7 was mounted with respect to the sub board | substrate support housing 22. FIG. The figure which shows the example which fixed and mounted | wore the sub-board | substrate 26 with which the magnetic sensor 7 was mounted. The figure which shows an example of the setting window used with the direction information addition function in this embodiment. The flowchart for demonstrating the processing operation by the direction information addition function in this embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... CPU, 2 ... Memory, 3 ... LCD controller, 4 ... LCD, 5 ... I / O controller, 6 ... A / D converter, 7 ... Magnetic sensor, 8 ... Button, 8a ... Direction measurement lock button, 8b ... Shutter Buttons, 9 ... Camera module, 10 ... GPS module, 11 ... Address bus, 12 ... Data bus, 14 ... Lens cover 14, 20 ... Main board, 22 ... Sub-board support housing, 24 ... Hinge mechanism, 26 ... Sub-board.

Claims (8)

Measuring means for measuring the direction by a magnetic sensor;
An instruction input means for inputting an instruction to perform azimuth measurement by the measuring means;
In accordance with an instruction input by the instruction input means, an azimuth information recording means for recording azimuth information measured by the measuring means;
Image photographing means for photographing an image;
Character generating means for generating a character based on the direction information recorded by the direction information recording means;
A portable information terminal comprising: a recording unit that records an image captured by the image capturing unit and a character generated by the character generating unit in association with each other. The character generating means
Setting means for setting which of the character representing the direction measured by the measuring unit and the character representing the direction opposite to the direction is generated;
The portable information terminal according to claim 1, wherein a character representing a direction is generated according to the setting by the setting unit. Character setting means for setting an additional character to be added to the character generated by the generating means;
The portable information terminal according to claim 1, further comprising an adding unit that adds the character set by the character setting unit to a character representing a direction generated by the generating unit. Measuring means for measuring the azimuth in two axial directions by one magnetic sensor;
Map information display means for displaying map information;
A portable information terminal comprising azimuth information adding means for adding azimuth information measured by the measuring means to map information displayed by the map information display means.   The magnetic sensor is mounted by means of a member that makes the magnetic sensor rotatable about a first axis and rotatable about a second axis orthogonal to the first axis. Item 5. A portable information terminal according to item 4. A display device having a rectangular display screen;
The portable information terminal according to claim 1 or 4, wherein the mounting surface of the magnetic sensor has an angle in the vicinity of 45 degrees on each of two orthogonal sides of the display screen of the display device. A display device having a display screen, and an imaging device having an optical axis in a direction orthogonal to the display screen,
The mounting surface of the magnetic sensor is parallel to a plane formed by connecting three non-adjacent vertices of a cube having a plane parallel to the display screen of the display device and a side parallel to the optical axis of the imaging device. The portable information terminal according to claim 1 or 4, wherein A direction information utilization method in a portable information terminal having a measuring unit for measuring a direction and an image capturing unit for capturing an image,
Enter the direction to execute the direction measurement,
In response to this instruction, azimuth measurement is performed to record azimuth information,
A method of using azimuth information, characterized in that a character based on recorded azimuth information is generated and combined with a photographed image.

Images