JP2004164559A - Portable terminal device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a portable terminal device which has improved operability of movement of an indication symbol on a display picture without damaging portability of the device. <P>SOLUTION: An imaging function of a camera incorporated in the main body 1 of a portable telephone 100 is utilized to pick up an image of a subject, and the indication symbol displayed on a picture of a display part 20 is moved on the basis of displacement of the image which is caused by movement of the main body 1. An imaging lens 11 is installed in a position opposed to an operation table 5 so that an image of an object on the surface of the operation table 5 can be imaged when the main body 1 is placed on the operation table 5, and the focal length of the imaging lens 11 is enabled to be set to a short distance so that an image of the subject (placed on the surface of the operation table) approximated to the main body 1 can be imaged, and thus the indication symbol can be moved by operating the main body 1 on the surface of the operation table like a mouse. At this point, light for imaging may be complemented by turning a light emitting part of an auxiliary light emitting device 30 for automatic focus detection toward the lens 11 to radiate an area to be imaged on the operation surface with auxiliary light L. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a mobile terminal device, and more particularly to position control of instruction symbols such as a pointer and a cursor on a display screen of the mobile terminal device.
[0002]
[Prior art]
In recent years, mobile terminal devices such as mobile phones and mobile computers have been used. These portable terminal devices are usually provided with a display unit, and an image, various information, an instruction symbol indicating a processing target, and the like are displayed on the screen of the display unit. In order to move the instruction symbol, some means that can be operated by the user is required. As such a means, a key provided on the apparatus main body (for example, see Patent Document 1), a joystick or a trackball is used. Have been proposed (for example, see Patent Document 2), and those using an operation unit removed from the main body as a mouse (for example, see Patent Document 3).
[0003]
[Patent Document 1]
JP 2002-149314 A
[0004]
[Patent Document 2]
JP 2001-16635 A
[0005]
[Patent Document 3]
JP 2001-36619 A
[0006]
[Problems to be solved by the invention]
However, those using keys, joysticks, and trackballs are difficult to operate instructing the intended position instantaneously, and the operability is not so good. A mouse that uses an operation unit that has been removed from the main body has good operability, but requires a mechanism for attaching and detaching the mouse part, and secures a space for housing the components that make up the mouse, such as a trackball. As a result, the size and weight of the apparatus are increased, resulting in a problem that portability is impaired.
[0007]
In view of the above problems, an object of the present invention is to provide a portable terminal device that has better operability when moving an instruction symbol on a display screen without impairing the portability of the device. Is.
[0008]
[Means for Solving the Problems]
A mobile terminal device according to the present invention is a mobile terminal device that includes an imaging unit that captures an image of a subject and a display unit that displays a screen including a moving instruction symbol on the main body. A first mode setting means for setting one of a mode and a pointing mode for controlling the display position of the instruction symbol; and in the pointing mode, the movement of the image picked up by the image pickup means is interlocked with the displacement caused by moving the main body. And pointing means for moving the instruction symbol on the screen.
[0009]
Here, the “instruction symbol” is a symbol indicating a coordinate position on the display screen, and examples thereof include a pointer or cursor on a character input screen or an item selection screen, and a character or target mark on a game screen.
[0010]
The “camera mode” is a mode in which an image of a subject is captured by an imaging unit, as in general imaging using a digital camera. In this camera mode, for example, the display screen is captured by the imaging unit. An image of a subject is displayed almost in real time, and an image picked up at that time is stored in a memory or a medium as image data by an input instructing image pickup from the user.
[0011]
On the other hand, the “pointing mode” is a mode in which an instruction symbol such as a pointer on the display screen is moved in conjunction with the movement of the apparatus main body. In this pointing mode, for example, an image of a subject imaged by an imaging unit Is monitored in near real time, the change in the position or angle of the device body is obtained from the movement (displacement) of the image, and the indication symbol is displayed by a distance proportional to the amount of change in the direction in which the user moves or tilts the body. Move on the screen.
[0012]
In the portable terminal device according to the present invention, the imaging unit has an imaging lens so that the imaging unit can capture an image of the operation table on which the main body is placed in response to the setting of the pointing mode by the first mode setting unit. The focal length may be shortened.
[0013]
Here, the “operation table” is a table on which the main body can be placed and the main body can be moved on the surface, and examples thereof include an office desk, a table, a floor, and a wall. The surface of the “operation console” does not necessarily have to be a flat surface, and does not have to be parallel to the ground.
[0014]
Further, the apparatus further includes auxiliary light emitting means for emitting auxiliary light for automatic focus detection toward the subject when the subject is imaged, and the auxiliary light emitting means responds to the setting of the pointing mode by the first mode setting means. Then, the auxiliary light may be irradiated on the operation table surface.
[0015]
The auxiliary light emitting means controls the irradiation direction so as to irradiate the auxiliary light toward a specific area on the operation table surface imaged by the imaging means in response to the setting of the pointing mode by the first mode setting means. It may be done. The control of the irradiation direction may be performed by rotating the light emitting portion of the auxiliary light and rotating the light emitting portion, or by providing a reflector in the light emitting optical path of the auxiliary light and reflecting the light. You may carry out by changing the angle of a board.
[0016]
In the portable terminal device according to the present invention, in the pointing mode, the second mode is set by switching between the non-mouse mode in which the main body is moved away from the operating table and the mouse mode in which the main body is moved on the operating table. The imaging unit further includes a setting unit, and the imaging unit sets the focal length of the imaging lens included in the imaging unit so that an image of the operation table on which the main body is placed can be captured in response to the setting of the mouse mode by the second mode setting unit. It may be shortened.
[0017]
Here, the “non-mouse mode” is a mode in which the main body is moved away from the operation console. In this non-mouse mode, for example, a subject that is away from the main body while the main body is held in the hand. An image of (person or landscape) is taken, and the instruction symbol is moved based on the displacement of the image.
[0018]
On the other hand, the “mouse mode” is a mode in which the main body is placed and moved on the operation table surface. In this mouse mode, for example, in order to capture an image of a subject close to the main body such as the operation table surface, The imaging lens is switched to a configuration with a short focal length, and the instruction symbol is moved based on the displacement of the captured image of the operation table. In other words, the main body itself is used like a mouse.
[0019]
Further, the apparatus further includes auxiliary light emitting means for emitting auxiliary light for automatic focus detection toward the subject when imaging the subject, and the auxiliary light emitting means responds to the setting of the mouse mode by the second mode setting means. Then, the auxiliary light may be irradiated on the operation table surface.
[0020]
In addition, the auxiliary light emitting means controls the irradiation direction so as to irradiate the auxiliary light toward a specific area on the operation table imaged by the imaging means in response to the setting of the mouse mode by the second mode setting means. It may be done. The control of the irradiation direction may be performed by rotating the light emitting portion of the auxiliary light and rotating the light emitting portion, or by providing a reflector in the light emitting optical path of the auxiliary light and reflecting the light. You may carry out by changing the angle of a board.
[0021]
Furthermore, in the portable terminal device according to the present invention, the optical opening of the imaging unit and the light emitting unit of the auxiliary light emitting unit are formed so as to be blocked by the operation table surface when the main body is placed on the operation table surface. It may be installed inside the recess of the main body.
[0022]
In the portable terminal device according to the present invention, the imaging means has an imaging means so that an image of the operation table on which the main body is placed can be taken in response to the setting of the camera mode by the first mode setting means. While changing the focal length of the imaging lens to be short, the captured image may be enlarged by a predetermined magnification.
[0023]
In addition, the main body has one or more predetermined functions related to the instruction symbols assigned to positions on the main body that can be operated with the index finger or middle finger of the user's one hand holding the main body placed on the operation table surface. It may be further provided with a button.
[0024]
Examples of the “predetermined function related to the instruction symbol” include a function for moving the cursor to a position indicated by the pointer by clicking a button, a function for selecting a portion traced with the pointer while pressing the button, and the like. Can be considered.
[0025]
【The invention's effect】
According to the mobile terminal device of the present invention, as described above, in the mobile terminal device provided on the main body with the imaging means for capturing the image of the subject and the display means for displaying the screen including the moving instruction symbol, A first mode setting means for setting one of a camera mode for picking up an image of the subject by the means and a pointing mode for controlling the display position of the instruction symbol; and moving the main body of the image picked up by the image pickup means in the pointing mode The camera is equipped with a pointing means that moves the instruction symbol on the screen in conjunction with the displacement caused by this, so the camera mode and pointing mode can be set. The instruction symbol is linked with the displacement caused by moving the main body of the image taken by the means. It can be moved on the display screen. Accordingly, the instruction symbol can be controlled by simply moving or tilting the main body of the mobile terminal device up, down, left, right, front, back, left, or right without adding components that increase the size of the main body. Since the instruction symbol can be moved in conjunction with the movement, the instruction symbol can be moved on the display screen with good operability with a configuration that does not impair the portability of the apparatus.
[0026]
In the portable terminal device according to the present invention, the imaging unit includes an imaging lens included in the imaging unit so that the imaging unit can capture an image of the operation table on which the main body is placed in response to the setting of the pointing mode by the first mode setting unit. If the focal length is to be shortened, an image on the surface of the operation table can be taken, and the instruction symbols on the display screen can be controlled from the displacement of the image. By moving it, the instruction symbol can be moved, and a more stable operation is possible.
[0027]
Further, it further comprises auxiliary light emitting means for emitting auxiliary light for automatic focus detection toward the subject when the subject is imaged, and the auxiliary light emitting means responds to the setting of the pointing mode by the first mode setting means. If the operation table surface is irradiated with auxiliary light, even when operating in a relatively dark place such as indoors, or when the main unit itself blocks the light to the operation table surface, Light necessary for imaging can be supplemented, and operation can be performed in a more preferable situation in a dark place.
[0028]
Further, in response to the setting of the pointing mode by the first mode setting means, the irradiation direction of the auxiliary light emitting means is controlled so as to irradiate the auxiliary light toward a specific area on the operation table surface imaged by the imaging means. If it is to be performed, the auxiliary light can be directly applied to the area to be imaged, so that the operation can be performed in a more preferable situation in a dark place.
[0029]
In the portable terminal device according to the present invention, in the pointing mode, the second mode setting is set by switching between the non-mouse mode in which the main body is moved away from the operation table and the mouse mode in which the main body is placed on the operation table surface and moved. Means for shortening the focal length of the imaging lens of the imaging means so that the imaging means can take an image of the operation table on which the main body is placed in response to the setting of the mouse mode by the second mode setting means. In non-mouse mode, you can take an image of a subject away from the main unit even in a place where there is no operating console, and control the indication symbols on the display screen from the displacement of the image. In mode, it is possible to take an image on the surface of the operation console and control the indication symbols on the display screen from the displacement of the image. In the control mode, it is possible to control the indication symbol.
[0030]
Further, the apparatus further includes auxiliary light emitting means for emitting auxiliary light for automatic focus detection toward the subject when imaging the subject, and the auxiliary light emitting means responds to the setting of the mouse mode by the second mode setting means. If the operation table surface is irradiated with auxiliary light, even when operating in a relatively dark place such as indoors, or when the main unit itself blocks the light to the operation table surface, Light necessary for imaging can be supplemented, and operation can be performed in a more preferable situation in a dark place.
[0031]
Further, in response to the setting of the mouse mode by the second mode setting unit, the irradiation direction of the auxiliary light emitting unit is controlled so as to irradiate the auxiliary light toward a specific area on the operation table surface imaged by the imaging unit. If it is to be performed, the auxiliary light can be directly applied to the area to be imaged, so that the operation can be performed in a more preferable situation in a dark place.
[0032]
Further, in the portable terminal device according to the present invention, the optical opening of the imaging unit and the light emitting unit of the auxiliary light emitting unit are formed so as to be blocked by the operation table surface when the main body is placed on the operation table surface. If it is installed inside the dent of the main body, the imaging means and the auxiliary light emitting means do not protrude from the main body, and the shape becomes easier to hold.
[0033]
In the portable terminal device according to the present invention, the imaging unit has the imaging unit so that an image of the operation table on which the main body is placed can be captured in response to the setting of the camera mode by the first mode setting unit. If the focal length of the imaging lens can be changed and the captured image is enlarged by a predetermined magnification, the image on the surface of the operation table is captured and the captured image is enlarged. It can be displayed on the display means. For this reason, if a magazine or newspaper is placed on the operation table and imaged, a small character of the magazine or newspaper can be enlarged and displayed on the display means, whereby the portable terminal device according to the present invention can be used as an electronic loupe. it can.
[0034]
Also, one or more predetermined functions related to the instruction symbols are assigned to positions where the main body can be operated with the index finger or middle finger of the user's one hand holding the main body placed on the operation table surface on the main body. If the button is further provided, various operations can be performed as if it were a mouse.
[0035]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of a mobile terminal device of the present invention will be described with reference to the drawings. FIG. 1 is a schematic diagram showing a configuration of a mobile phone 100 with a digital camera (hereinafter referred to as a camera-equipped phone) as an embodiment of a mobile terminal device according to the present invention, and FIG. 2 is an overview thereof. 2A, 2B, and 2C respectively show a front view, a back view, and a side view of the camera-equipped mobile phone 100 according to the present embodiment.
[0036]
The camera-equipped mobile phone 100 according to the present embodiment includes an imaging device 10 that captures an image of a subject, and a display unit 20 that includes a captured image and various processing screens, a liquid crystal panel for displaying a pointer that indicates a processing target, an organic EL, and the like. An AF auxiliary light emitting device 30 that emits auxiliary light L for automatic focus detection (AF) toward a subject, an operation unit 40 in which buttons for various operations are arranged, A control unit 50 that performs mode setting, display control of the display unit 20, other various calculations and control of each unit, and a memory 60 that stores various data are provided.
[0037]
The imaging apparatus 10 further collects reflected light from the subject and forms an image of the subject on an imaging element 12 described later, and the subject image formed by the imaging lens 11 as an electrical signal. The image sensor 12 to output and the converter 13 which converts the electric signal from the image sensor 12 into image data.
[0038]
The AF auxiliary light emitting device 30 further includes a light emitting unit 31 having a light emitting surface, and a driving unit 32 for controlling the L light amount and the irradiation direction of the auxiliary light. The light emitting unit 31 has a structure that can be rotated to change the irradiation direction.
[0039]
The operation unit 40 further includes a menu button 41 for switching functions, click buttons 42a and 42b to which a predetermined function related to the pointer S is assigned, a power button for turning on / off the power, a dial button, and the like. It consists of buttons.
[0040]
The display unit 20 is on the inner surface of the main body lid 1b, the imaging lens 11 and the auxiliary light emitting unit 31 are in a recess on the back surface of the main body main portion 1a, and the operation unit 40 is on the front surface of the main body main portion 1a. (Especially, the click buttons 41a and 41b are located at positions where the user can operate with the index finger and middle finger when the user grips the main body 1 placed on the operation table 5 with one hand). Each is installed inside 1b.
[0041]
The mode set for the camera-equipped mobile phone 100 includes a camera mode and a pointing mode, and the control unit 50 automatically sets these modes according to the function to be used. The camera mode is a mode that is set when capturing an image of a subject, as in the case of capturing a subject using a normal digital camera. The pointing mode is a process that is performed on the display unit 20 together with a processing screen. When a pointer S for indicating a target is displayed, the main body 1 is placed on an appropriate operation table 5 such as a table and moved in a plane to operate the main body 1 itself like a mouse. This mode is set when the display position is controlled.
[0042]
The control of the display position of the pointer S uses a method of moving the coordinate position on the display screen from the movement of the image picked up by the image pickup device, which is disclosed in “Mouse Scanner” of Japanese Patent Application Laid-Open No. 2000-20230. Can be realized. That is, an image on the operation surface of the operation console 5 is continuously imaged in real time through the imaging lens 11, the image sensor 12, and the converter 13, and the movement direction of the main body and The amount of movement is detected, and the pointer S is moved based on the detection result.
[0043]
When the camera mode is set, the imaging lens 11 is switched to a setting with a long focal length under the control of the control unit 50, and when the pointing mode is set, the imaging lens 11 is switched to a setting with a short focal length. As a result, in the camera mode, an image of a subject away from the main body 1 can be captured, and in the pointing mode, a subject close to the main body 1, that is, an image of the operation table on which the main body 1 is placed can be captured.
[0044]
The image sensor 12 outputs an image of the subject formed by the imaging lens 11 as an electrical signal.
[0045]
When receiving the image data read command from the control unit 50, the converter 13 reads the electrical signal from the image sensor 12 and converts it into image data.
[0046]
The AF auxiliary light emitting device 30 has a structure that rotates so that the light emitting unit 31 can face relatively far from the imaging lens 11 and in the vicinity of the imaging lens 11 within the angle of view of the imaging lens 11. Yes. When the camera mode is set, the drive unit 32 controls the rotation of the light emitting unit 31 so that the auxiliary light L for automatic focus detection is emitted toward the subject under control of the control unit 50 to emit light. The surface is directed far away from the imaging lens 11 and set to an appropriate amount of light. Further, when the pointing mode is set, driving is performed so that the light emitting surface is directed toward the vicinity of the imaging lens 11 and the auxiliary light L for assisting imaging on the operation surface of the operation table 5 is irradiated toward the operation table surface. The unit 32 receives control from the control unit 50 and controls the rotation of the light emitting unit 31 so that the light emitting surface is directed to the vicinity of the imaging lens 11 and is set to an appropriate amount of light.
[0047]
The click buttons 42a and 42b are buttons to which a function related to the pointer S is assigned. As the function, for example, when a screen for inputting characters is displayed on the display unit 20, the click button 42a is used. When one click is performed, the cursor position is moved to the position pointed to by the pointer S at that time, or when a screen for selecting an item is displayed on the display unit 20, double clicking the click button 42a. A function can be considered in which the item pointed to by the pointer S at that time is selected and executed. Furthermore, a function is also conceivable in which a submenu is displayed when the click button 42b is clicked once. By providing such a click button, operation can be performed with a feeling close to that of a mouse as an accessory of a computer.
[0048]
The control unit 50 displays an image, a processing screen, and a pointer S on the display unit 20, and when the pointing mode is set, in order to irradiate the subject with the appropriate auxiliary light L, the AF auxiliary light In order to control the drive unit 32 of the light emitting device 30 and to move the display position of the pointer S based on the moving direction and moving amount of the main body 1, images are continuously captured in real time on the image sensor 12 and the converter 13. The image data is stored in the memory 14. Among the stored image data, the latest two image data are read and compared, the relative position of the overlapping image pattern is detected between the two images, and the movement direction and movement amount of this pattern are determined. To detect. The movement direction of the main body 1 is opposite to the movement direction of the pattern, and the movement amount of the main body 1 is proportional to the movement amount of the pattern. Therefore, the main body of the camera-equipped mobile phone 100 is determined from the movement direction and movement amount of the pattern. 1 can be obtained, and the pointer S is controlled to move in the same direction as the movement direction of the main body 1 and with a width corresponding to the movement amount of the main body 1.
[0049]
The memory 60 stores the image data converted by the converter 13 and the set mode. In the camera mode, each image / converted image data is stored. In the pointing mode, the image data is continuously recorded in real time. Image data representing at least the latest two images among the images picked up automatically is stored. As described above, the latest two images are used by the control unit 50 to detect the moving direction and moving amount of the main body 1 of the camera-equipped mobile phone 100.
[0050]
A description of the telephone function of the mobile phone with a digital camera according to the present embodiment is omitted.
[0051]
Next, the operation of the camera-equipped mobile phone 100 according to the present embodiment will be described. FIG. 3 is a flowchart showing a processing flow of the camera-equipped mobile phone 100.
[0052]
First, the user selects a function that the mobile phone with camera 100 wants to use. When the user presses the menu button 41 of the operation unit 40, the control unit 50 displays a menu screen representing a function list on the display unit 20, and the user selects a desired function from the list (step S1). ).
[0053]
The control unit 50 sets a mode according to the selected function (step S2). For example, the camera mode is set when the camera imaging function is selected, and the pointing mode is set when the address book management function is selected. Then, processing is performed for each set mode (step S3).
[0054]
When the camera mode is set, as shown in FIG. 4, the control unit 50 controls the drive unit 32 of the AF auxiliary light emitting device 30 so that the light emitting surface of the light emitting unit 31 is oriented for camera imaging, that is, imaging. The lens 11 is rotated so as to face far within the angle of view, and the light amount of the auxiliary light L is also set to the light amount for camera imaging (step S4). Then, the presence / absence of an imaging instruction by the user is determined (step S5), and if there is an imaging instruction, an imaging process is performed (step S6). Next, it is confirmed whether or not a function switching operation to be used has occurred (step S7). If there is a function switching operation, the process returns to step S1, and if not, the process returns to step S5 to wait for an imaging instruction from the user. . The imaging process is performed according to the following procedure. First, auxiliary light L is emitted toward the subject from the light emitting unit 31 of the AF auxiliary light emitting device 30 as necessary, automatic focus detection and focusing are performed, and the image of the subject is imaged by the imaging lens 11. 12 is imaged. The converter 13 converts the electrical signal corresponding to the subject image output from the image sensor 12 into image data, and the converted image data is stored and stored in the memory 14.
[0055]
On the other hand, when the pointing mode is set, as shown in FIG. 5, the control unit 50 causes the imaging lens 11 to have a short focal length so that the operation table surface can be imaged when the main body 1 is placed on the operation table 5. Switching to the setting and further controlling the driving unit 31 of the AF auxiliary light emitting device 30 so that the light emitting surface of the light emitting unit 31 is in the direction for controlling the display position of the pointer S, that is, on the operation surface of the operation console 5 for imaging. Then, the imaging lens 11 is rotated so as to face the vicinity of the imaging lens 11 within the angle of view, and the auxiliary light L with an appropriate amount of light is irradiated (step S8). Then, as shown in FIG. 6, when the user places the main body on the operation table 5 and moves it back and forth and right and left, an image on the surface of the operation table 5 passes through the imaging lens 11, the image sensor 12, and the converter 13. Images are continuously captured as data, and at least the latest two images are always stored in the memory 60. The control unit 50 reads out the latest two image data from the memory 60 and compares them to detect temporal displacement of the image (step S9). Then, it is determined whether or not there is a displacement (step S10). If there is a displacement, the movement direction and the movement amount of the main body 1 are obtained from the displacement, and the pointer S is moved by an amount corresponding to these (step S10). S11). Next, it is confirmed whether or not a function switching operation to be used has occurred (step S12). If there is a function switching operation, the process returns to step S1, and if not, the process returns to step S9 to detect the displacement of the image again. .
[0056]
In the pointing mode, pressing the click buttons 42a and 42b at the position of the index finger and middle finger of one hand holding the main body 1 moves the cursor to the position indicated by the pointer S, or the pointer S indicates The selected item can be selected and a submenu can be displayed.
[0057]
According to the camera-equipped mobile phone 100 of the present embodiment, the camera mode and the pointing mode can be set, and when the pointing mode is set, the image is picked up by the imaging device 10 already provided in the main body 1 of the camera-equipped mobile phone 100. The pointer S can be moved on the display screen in conjunction with the displacement of the captured image caused by moving the main body 1. Accordingly, the pointer S can be controlled by merely moving the main body 1 back and forth and right and left without adding parts that increase the size of the main body 1, and the pointer is linked to the movement of the user's hand. Since S can be moved, the pointer S can be moved on the display screen with good operability with a configuration that does not impair the portability of the apparatus.
[0058]
Further, when the pointing mode is set, the focal length of the imaging lens 11 is shortened so that when the main body 1 of the mobile phone 100 with a camera is placed on the surface of the operation table 5, the surface of the operation table 5 can be imaged. Therefore, the pointer S can be moved by moving the surface while placing the main body 1 on the operation table 5, and stable operation is possible.
[0059]
When the pointing mode is set, the AF auxiliary light emitting device 30 receives the control from the control unit 50 and rotates the light emitting unit 31 so that the light emitting surface of the light emitting unit 31 faces the imaging lens 11 side. Since the auxiliary light L can be irradiated to a specific region to be imaged on the surface of the operation table 5, it is possible to supplement the light necessary for imaging the surface of the operation table 5 as a subject. Can be operated with.
[0060]
Further, the inside of a dent formed so that the light emitting unit 31 of the AF auxiliary light emitting device 30 and the imaging lens 11 of the imaging device 10 are closed by the operation table surface when the main body 1 is placed on the operation table 5 surface. Therefore, the imaging device 10 and the AF auxiliary light emitting device 30 do not stick out of the main body, and the user can easily hold them. Furthermore, a predetermined function related to the pointer S is provided on the main body 1 of the camera-equipped mobile phone 100 at a position where it can be operated with the index finger or middle finger of the user's one hand holding the main body 1 placed on the surface of the operation table 5. Since the assigned click buttons 42a and 42b are provided, various operations can be performed with a sense closer to that of a mouse.
[0061]
By the way, when the main body 1 of the camera-equipped mobile phone 100 is operated on the operation console 5 to move the pointer S, the user has moved the position of the main body 1 on the operation console 5 to a position where it is difficult to operate. At that time, it is expected that the position will be corrected. However, if the pointer S is moved while always seeing the displacement of the image, the position of the main body 1 is returned to the original position, and the position of the pointer S is also returned to the original position. Therefore, if a movement stop button for stopping the movement of the pointer S only while the button is pressed is further provided on the main body 1 and the user changes the position of the main body 1 while pressing this button, the pointer S is moved unnecessarily. The position of the main body 1 can be corrected without any problem. On the contrary, a movement permission button that allows the pointer S to move only while being pressed may be provided, and the user may operate the main body 1 while pressing this button. As another method, the focus of the imaging lens 11 is fixed on the surface of the operation table 5 and the degree of defocusing of the captured image is detected. While the degree of defocusing exceeds a predetermined threshold value, The movement of the pointer S may be stopped. In this way, the image is out of focus while the user lifts and moves the main body 1 of the camera-equipped mobile phone 100 to a desired position, and the pointer S also moves during that time. First, the position of the main body 1 can be corrected.
[0062]
In the pointing mode, in addition to the mode in which the main body 1 of the camera-equipped mobile phone 100 is placed on the operation table 5 and moved (hereinafter referred to as mouse mode), the main body 1 is further moved. It may be possible to switch and set a mode (hereinafter referred to as a non-mouse mode) that moves away from the operation console 5. In the non-mouse mode, when the pointer S for instructing the processing target is displayed together with the processing screen on the display unit 20, the main body 1 is not placed on the operation console 5 or the like, but the main body 1 is moved by hand. This is a mode in which the display position of the pointer S is controlled by gripping and moving or tilting forward, backward, left and right in the air. In this non-mouse mode, as shown in FIG. 7, an image of a subject away from the main body 1 is captured by the imaging device 10 with the focal length of the imaging lens 11 set to be as long as in the camera mode. The display position of the pointer S is controlled from the displacement.
[0063]
As described above, if the non-mouse mode and the mouse mode can be further set in the pointing mode, the non-mouse mode is set in a place where the operation console 5 is not provided, and the camera-equipped mobile phone 100 is separated from the main body 1. An image of the subject can be captured, and the pointer S on the display screen can be controlled based on the displacement of the image. When the operation console 5 is present, the mouse mode is set and the image on the operation console 5 is captured. Since the pointer S on the display screen can be controlled from the displacement of the image, the pointer S can be controlled with an operation mode suitable for the place.
[0064]
In the non-mouse mode, as in the mouse mode, when the position and angle of the main body 1 of the camera-equipped mobile phone 100 are restored, the position of the pointer S is also restored. It is desirable to provide a movement permission button on the main body 1.
[0065]
In the present embodiment, the mode setting is automatically set by the control unit 50 in accordance with the function to be used. Of course, the user may set the mode manually. Absent.
[0066]
In the present embodiment, when the camera mode is set, the imaging lens 11 is switched to a setting with a short focal length so that an object close to the main body 1, that is, an image of the operation table on which the main body 1 is placed can be captured. Also good. At this time, the control unit 50 enlarges the image captured by the imaging device 10 with a predetermined magnification and displays the image on the display unit 20. As a result, as shown in FIG. 8, characters placed on the operation table can be enlarged and displayed on the display unit 20, and the camera-equipped mobile phone 100 of the present embodiment can be used as an electronic loupe.
[0067]
In this case, the control unit 50 determines whether the image captured by the imaging device 10 is too bright or too dark, and adjusts the light amount of the light emitting unit 31 so that the image displayed on the display unit 20 has appropriate brightness. May be.
[0068]
Further, the controller 50 determines whether the contrast is large or small by comparing the contrast of the image captured by the imaging device 10 (that is, the difference in brightness between the character and the background) with a threshold value, and determines that the contrast is small. In such a case, image processing may be performed on the image captured by the imaging device 10 so that the image displayed on the display unit 20 has an appropriate contrast. Specifically, when it is determined that the contrast is small, the image captured by the imaging device 10 is binarized and separated into a high-luminance image and a low-luminance image. What is necessary is just to display an image on the display part 20 by each of two types of brightness | luminance values which have a predetermined appropriate contrast.
[0069]
In addition, the size of characters to be displayed on the display unit 20 may be set in advance, and characters of the same size may be displayed on the display unit 20 regardless of the size of characters to be photographed. Alternatively, the magnification of an image to be displayed may be increased by clicking the click button 42a, and the magnification may be decreased by clicking the click button 42b.
[Brief description of the drawings]
FIG. 1 is a schematic diagram showing a configuration of a mobile phone with a digital camera according to an embodiment of the present invention.
FIG. 2 is an overview of a mobile phone with a digital camera according to an embodiment of the present invention.
FIG. 3 is a flowchart showing a processing flow of the mobile phone with a digital camera according to the present embodiment.
FIG. 4 is a diagram showing a mobile phone with a digital camera when shooting an object in camera mode.
FIG. 5 is a diagram showing a mobile phone with a digital camera when the pointer is moved in the pointing mode.
FIG. 6 is a diagram showing a state where the pointer is moved by operating the main body of the mobile phone with a digital camera.
FIG. 7 is a diagram showing how the pointer is moved by operating the main body of the mobile phone with a digital camera in the non-mouse mode.
FIG. 8 is a diagram showing a state where characters on the operation table are photographed in the camera mode.
[Explanation of symbols]
1 Mobile phone with digital camera
1a Main part of mobile phone with digital camera
1b Mobile phone cover with digital camera
5 Operation table
10 Imaging device
11 Imaging lens
12 Image sensor
13 Converter
20 display
30 AF auxiliary light emitting device
31 Light emitting part
32 Drive unit
40 Operation unit
41 Menu button
42a, 42b Click button
50 Control unit
60 memory
100 Mobile phone with digital camera
L Auxiliary light
S pointer

Claims (10)

In a mobile terminal device provided on a main body with an imaging unit that captures an image of a subject and a display unit that displays a screen including a moving instruction symbol.
First mode setting means for setting one of a camera mode for picking up an image of a subject by the image pickup means and a pointing mode for controlling a display position of the instruction symbol;
And a pointing means for moving the instruction symbol on the screen in conjunction with a displacement caused by moving the main body of the image picked up by the image pickup means in the pointing mode. Terminal device. In response to the pointing mode setting by the first mode setting means, the imaging means shortens the focal length of the imaging lens of the imaging means so that the image of the operation table on which the main body is placed can be taken. The mobile terminal device according to claim 1, wherein the mobile terminal device is a device. An auxiliary light emitting unit that emits auxiliary light for automatic focus detection toward the subject when the subject is imaged;
3. The portable terminal according to claim 2, wherein the auxiliary light emitting means irradiates the auxiliary light on the operation table surface in response to the setting of the pointing mode by the first mode setting means. apparatus. Irradiating the auxiliary light emitting means to irradiate the auxiliary light toward a specific region on the operation table surface imaged by the imaging means in response to the setting of the pointing mode by the first mode setting means. 4. The portable terminal device according to claim 3, wherein the direction is controlled. In the pointing mode, further comprising a second mode setting means for switching between a non-mouse mode in which the main body is moved away from the operation table and a mouse mode in which the main body is placed on the operation table surface and moved.
In order to capture the image of the operation surface on which the main body is placed in response to the setting of the mouse mode by the second mode setting unit, the imaging unit shortens the focal length of the imaging lens included in the imaging unit. The portable terminal device according to claim 1, wherein: An auxiliary light emitting unit that emits auxiliary light for automatic focus detection toward the subject when the subject is imaged;
6. The portable terminal according to claim 5, wherein the auxiliary light emitting means irradiates the auxiliary light on the operation table surface in response to the setting of the mouse mode by the second mode setting means. apparatus. Irradiating so that the auxiliary light emitting means emits the auxiliary light toward a specific area on the operation table surface imaged by the imaging means in response to the setting of the mouse mode by the second mode setting means. The portable terminal device according to claim 6, wherein the direction is controlled. The optical aperture of the imaging means and the light emitting portion of the auxiliary light emitting means are inside a recess of the main body formed so as to be blocked by the operation table surface when the main body is placed on the operation table surface. The portable terminal device according to claim 3, 4, 6, or 7, wherein the portable terminal device is installed. The focal length of the imaging lens of the imaging unit is shortened so that the imaging unit can capture an image of the operation table on which the main body is placed in response to the setting of the camera mode by the first mode setting unit. The portable terminal device according to any one of claims 1 to 8, wherein the portable terminal device is capable of being changed in such a manner that a captured image is enlarged at a predetermined magnification. A predetermined function related to the instruction symbol is assigned to a position where the main body can be operated with the index finger or middle finger of one hand of a user holding the main body placed on the operation table surface on the main body. The mobile terminal device according to claim 2, further comprising two or more buttons.

Images