JP2005234881A - Tactile force indicating portable terminal and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a tactile force indicating portable terminal capable of indicating a temperature feeling or an unevenness feeling of a display image through human skin in a pseudo state. <P>SOLUTION: A working memory 3 stores unevenness information showing unevenness of the image displayed on a display part 5 of a cellphone 11, a tactile force area detection part 6 detects a tactile force area indicating the unevenness information inside the image of the display part 5 according to movement of the cellphone, a tactile force pin 9 indicates the unevenness feeling of the image, and a tactile force pin control part 8 vertically operates or vibrationally operates the tactile force pin corresponding to the tactile force area on the basis of the unevenness information of the image acquired from the working memory. Temperature information about the image is acquired from the working memory 3, and a cooling and heating device 13 is operated by a temperature control part 12 to cool or heat the tactile force pin 9. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention is applied to mobile terminals such as mobile phones, PHS (Personal Handyphone System), PDA (Personal Digital Assistants: personal digital assistants), etc., and presents pseudo-tactile sensation and temperature of an image displayed on the screen. The present invention relates to a tactile pressure sense presenting portable terminal equipped with a tactile pressure sense presenting function.

  Conventionally, in a portable terminal such as a mobile phone, PHS, PDA, etc., an image with a feeling of unevenness and an image with a feeling of temperature displayed on the screen are only referred to, and the unevenness is touched through the human skin using the portable terminal. It was impossible.

  In addition, as a technology related to the braille display, a predetermined number of actuators are arranged in a matrix on the screen, and each actuator can be projected and retracted from the screen. There has been proposed a braille display device having a display screen in which a plurality of display units configured to display braille or a graph in accordance with a retreating state are arranged in a matrix (see, for example, Patent Document 1).

Alternatively, as a technique related to a user interface using the sense of touch of a user's hand, a displacement generation unit including a plurality of displacement units that protrude from the housing of the information processing terminal and transmit the displacement to the touched user's hand, and processing Corresponding to the above, there has been proposed an information processing terminal including control means for controlling individual timings of a plurality of displacement portions of the displacement generating means (see, for example, Patent Document 2).
JP 58-118687 A JP 2002-135378 A

  However, the conventional technique only displays the braille or graph by the pin-shaped actuator protruding and retracting with respect to the screen, or only transmitting the displacement to the user's hand by the displacement portion protruding from the housing. It has been impossible to present to the user the unevenness and temperature of the display image as tactile pressure information.

  Therefore, an object of the present invention is to provide a tactile pressure sense presenting portable terminal and a method capable of presenting a feeling of unevenness and temperature of a display image in a pseudo manner through human skin.

  In order to solve the above-described problems, the present invention provides a portable terminal including a display unit that displays an image, a concavo-convex information storage unit that stores concavo-convex information indicating a concavo-convex of an image displayed on the display unit, and the mobile terminal. Corresponding to the tactile pressure sense region, the tactile pressure sense means for presenting the sense of unevenness of the image, and the tactile pressure sense region. The tactile pressure sense means comprises tactile pressure sense control means for moving up and down and / or vibrating based on the unevenness information of the image acquired from the unevenness information storage means.

  Also, temperature information storage means for storing temperature information indicating the temperature of the image displayed on the display means, cooling / heating means for cooling or heating the tactile pressure sense means, and the tactile pressure sense corresponding to the tactile pressure sense region The apparatus further includes temperature control means for performing cooling or heating control of the cooling / heating means based on the temperature information of the image acquired from the temperature information storage means.

  With the configuration described above, it is possible to simulate the impression of unevenness and the temperature of an image through human skin.

  According to the present invention, not only an image having unevenness information and temperature information is displayed to a user of a mobile terminal, but also an effect that a sense of unevenness and temperature of the image can be presented in a pseudo manner by the tactile sense means. is there. For example, it becomes possible to reproduce the hand feeling of clothes displayed on the display means in a pseudo manner by the tactile pressure sense means, or to present the braille displayed on the display means by the tactile pressure sense means.

  In addition, based on the temperature information of the image, the warmth or coldness of the object displayed on the display means is simulated by the tactile pressure sense means, or the temperature or the like of the virtual reality image displayed on the display means is also detected by the tactile pressure sense. It is also possible to present it in a pseudo manner by means.

  In addition, since the space detection unit can measure the spatial coordinates according to the movement of the mobile terminal placed on the desk or the like, it is possible to increase inputs from other than the operation unit of the mobile terminal.

  Next, the best mode of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a main part of a circuit configuration of a mobile phone as a mobile terminal according to the present invention. The cellular phone 11 includes a CPU (central processing unit) 1, a ROM 2, a work memory 3, a display control unit 4, a display unit 5, a tactile pressure sensing area detection unit 6, a trackball (or infrared measurement unit) 7, and a tactile pressure sense pin. The controller 8 includes a tactile pressure pin 9, a bus 10, a temperature controller 12, and a cooling and heating device 13. The same applies to PHS, PDA, and the like.

  2A is a front view of the mobile phone, and FIG. 2B is a rear view of the mobile phone. The mobile phone 11 includes an upper part of the casing and a lower part of the casing, and is foldable. As shown in FIG. 2A, the display unit 5 is disposed on the front side of the upper part of the casing of the mobile phone 11, and the operation unit 14 and the plurality of tactile pressure pins 9 are provided on the front side of the lower part of the casing. Is arranged. Further, as shown in FIG. 2B, a trackball (or infrared measurement unit) 7 is disposed on the back side of the lower part of the casing of the mobile phone 11. The trackball (or infrared measurement unit) 7 uses a known mouse principle.

  In FIG. 1, the CPU 1 is connected to each part in the mobile phone via a bus 10. The CPU 1 performs the tactile pressure sense presenting process of the flowchart of FIG. 4 by executing the control program. The ROM 2 is a read-only memory that stores various control programs to be executed by the CPU 1 and fixed data such as image unevenness information and temperature information. The work memory 3 is constituted by a RAM (Random Access Memory), and stores data temporarily required for the CPU 1 to execute the control program. The work memory 3 also stores image information displayed on the display unit 5 and image unevenness information and temperature information presented to the user by the tactile pressure pin 9.

  The trackball (or infrared measurement unit) 7 places the back side of the lower part of the casing of the mobile phone 11 on a desk, for example, and measures the movement distance when the lower part of the casing is moved in any direction on the desk. Is possible. The tactile sensation area detection unit 6 detects data measured by the trackball (or infrared measurement unit) 7 and passes the measurement data (X coordinate value, Y coordinate value) to the display control unit 4. The tactile sensation pin control unit 8 obtains the unevenness information held by the image data corresponding to the image displayed on the display unit 5 from the work memory 3, and based on the unevenness information, the top and bottom of the plurality of tactile pressure sense pins 9 is detected. Control the motion (or vertical motion and vibration motion).

  The tactile sensation pin 9 is composed of a plurality of pins (about 5 × 5) arranged in a matrix as shown in FIG. 2A, and is arranged below the operation unit 14 (keypad unit). One of the tactile pressure sense pins 9 corresponds to one pixel of the tactile pressure sense region 5C (FIG. 5) displayed on the display unit. Further, the tactile pressure sense pin 9 is set such that the interval between the pins is 1 mm to several mm based on the theory of two-point discrimination threshold.

  FIG. 3A is a side view showing an arrangement state of the tactile sensation pins 9 that are not covered with the tactile sensation membrane on the mobile phone 11, and the height of the tactile sensation pins 9 from the housing surface of the mobile phone 11 is, for example, It is set to 3 mm to 5 mm. FIG. 3B is a side view showing an arrangement state of the tactile sensation pin 9 covered with the tactile sensation film 15 on the mobile phone 11, and the tactile sensation film 15 covers the upper part of the tactile pressure sensation pin 9. It is arranged.

  Each of the tactile sensation pins 9 is driven in the vertical direction by the tactile sensation pin control unit 8 based on the unevenness information. As shown in FIG. 3B, the haptic pin 9 moves up and down, for example, about 0 mm to 5 mm (or moves up and down and vibrates), thereby causing unevenness to the user of the mobile phone 11. Present. The tactile sensation film 15 can be made of various materials, but it is difficult to cut and is made of a resilient material. By operating the tactile sensation pin 9 with the tactile sensation film 15 covering the tactile sensation pin 9, a pseudo-smooth image is presented to the user.

  The temperature control unit 12 acquires temperature information held by image data corresponding to the image displayed on the display unit 5 from the work memory 3 and controls the cooling and heating device 13. The cooling and heating device 13 cools and heats the tactile pressure pin 9. As specific cooling and heating methods in the cooling and heating apparatus 13, various methods such as joule heating by electricity, cooling or heating by Peltier elements, cooling by cooling water, cooling by cooling gas, and the like can be considered. Then, detailed explanation is omitted.

  In the present embodiment, the tactile pressure sense information of the tactile pressure sense region 5C (FIG. 5) in the image displayed on the display unit 5 of the mobile phone 11 is detected by the tactile pressure pin control unit 8, and the tactile pressure sense pin 9 is operated. Further, when the back side of the lower part of the casing of the mobile phone 11 is placed on a desk or the like and moved, the distance on the display unit 5 is determined based on the movement distance measured by the trackball (or infrared measurement unit) 7. The tactile pressure sense area 5C is moved up, down, left and right on the screen.

  Next, the operation of this embodiment will be described with reference to the flowchart of FIG. When a user of the mobile phone 11 refers to an image from a web page or a Java (registered trademark of US Sun Microsystems, Inc.) application or the like, the image displayed on the display unit 5 of the mobile phone 11 is uneven information or temperature. When the information is held (YES in step S1), the tactile pressure sense mode is added to the submenu displayed on the display unit 5 (step S2). When the user selects the tactile pressure sense mode on the display unit 5 (YES in step S3), the tactile pressure sense of the image is presented by the tactile pressure pin 9.

  Whether or not temperature information is presented by the tactile pressure pin 9 when the image displayed on the display unit 5 holds the temperature information (YES in step S4) before the tactile pressure sense of the image is presented by the tactile pressure pin 9 A key corresponding to the submenu to be selected is assigned (step S5). On the other hand, when the image displayed on the display unit 5 does not hold the temperature information (NO in step S4), the above assignment is not performed.

  5A and 5B are diagrams showing a screen in which a tactile sensation area is being presented, in which FIG. 5A shows a display screen of clothes having unevenness information and temperature information, and FIG. 5B shows a Braille display screen having unevenness information. . In the tactile pressure sense mode, the tactile pressure sense region 5C is displayed on the display unit 5, and the user moves the mobile phone 11 up and down, left and right on a horizontal surface such as a desk, or performs key input such as a cross key. The pressure-sensitive area 5C can be moved. This operation is performed by operating the trackball (or infrared measurement unit) 7 and detecting the data measured by the trackball (or infrared measurement unit) 7 by the tactile pressure sensing region detection unit 6.

  Corresponding to the unevenness in the tactile sensation region 5C, the tactile sensation pin 9 operates in the vertical direction (or operates in the vertical direction and vibrates) with respect to the housing surface of the mobile phone 11. On the screen 5A that holds the temperature information, the temperature control unit 12 controls the cooling and heating device 13 by associating ON (ON) / OFF (OFF) of the temperature information with the key corresponding to the submenu (step S5). . In this case, it is desirable to make temperature information correspond to each tactile pressure pin 9, but if it is difficult, temperature control is performed by adopting average data or most data from temperature information in the tactile pressure sense region 5 </ b> C. To do. If necessary, the haptic pin 9 is automatically or manually covered with the haptic film 15 (see FIG. 3B), and the smoothness is presented to the user (step S6).

  When ending the tactile sensation mode, pressing the key corresponding to the portion displayed on the screen of the display unit 5 (YES in step S7) ends the tactile sensation mode and the screen (WEB page or Java application).

  On the other hand, if the image in the WEB page or Java application or the like does not hold unevenness information or temperature information (NO in step S1), the tactile pressure mode is not added to the submenu and the current screen remains unchanged (WEB page or Java Application). In addition, even if the image on the WEB page or Java application holds unevenness information or temperature information, but the user does not select the tactile pressure mode (NO in step S3), the current screen remains the same. (WEB page or Java application).

  When the user is using the mobile phone 11 in a mode other than the tactile pressure mode, the tactile pressure pin control unit 8 causes the vertical movement range of the tactile pressure pin 9 to be viewed from the side of the mobile phone 11 − Control to be several mm to 0 mm.

  According to the present embodiment, not only the image having unevenness information and temperature information is displayed on the display unit 5 to the user of the mobile phone 11 but also the unevenness and temperature feeling of the image are simulated by the tactile pressure pin 9. There is an effect that can be presented. For example, the touch feeling of the clothes displayed on the screen 5A (FIG. 5A) of the display unit 5 is simulated by the tactile pressure pin 9, or the braille displayed on the screen 5B (FIG. 5B). Can be presented by the tactile pressure pin 9.

  In addition, the warmth and coldness of the object displayed on the display unit 5 are simulated by the tactile pressure pin 9 based on the temperature information of the image, and the foreign temperature that is a virtual reality is also simulated by the tactile pressure pin 9. Can also be presented.

  In addition, since the spatial coordinates corresponding to the movement of the mobile phone 11 placed on a desk or the like can be measured by the trackball (or infrared measurement unit) 7, input from other than the operation unit 14 of the mobile phone 11 is possible. Can be increased.

  In the first embodiment described above, a case has been described in which a tactile pressure sense of an image having unevenness information and temperature information is presented on a mobile phone in a pseudo manner, but the distance from an object to be imaged is measured for an image captured by a camera. Thus, the unevenness information can be given to the captured image, and the tactile pressure sense of the captured image can be presented in a pseudo manner.

  In the first embodiment described above, the operation only in the tactile pressure mode has been described. However, the tactile pressure pin can be applied to other operations (for example, pressing a pot).

It is a block diagram which shows the circuit structure of the mobile telephone by this invention. (A) is a front view of a mobile phone, and (b) is a rear view of the mobile phone. (A) is a side view of a mobile phone provided with a tactile pressure pin, and (b) is a side view of the mobile phone provided with a tactile pin and a tactile membrane. It is a flowchart which shows a tactile sensation presentation process. (A) is a figure which shows the display screen of the clothes which have uneven | corrugated information and temperature information, (b) is a figure which shows the braille display screen which has uneven | corrugated information.

Explanation of symbols

1 CPU
3 working memory 5 display unit 6 tactile pressure detection unit 7 trackball (or infrared measurement unit)
8 Tactile Pressure Sensing Pin Control Unit 9 Tactile Pressure Sensing Pin 10 Bus 11 Mobile Phone 12 Temperature Control Unit 13 Cooling and Heating Device

Claims (6)

In a portable terminal having a display means for displaying an image,
Concavo-convex information storage means for storing concavo-convex information indicating the unevenness of the image displayed on the display means;
An area detecting means for detecting a tactile sensation area that presents unevenness information in the screen of the display means according to the movement of the mobile terminal;
Tactile sensation means for presenting the unevenness of the image;
And a tactile pressure control means for causing the tactile pressure sense means corresponding to the tactile pressure sense area to move up and down and / or vibrate based on the unevenness information of the image acquired from the unevenness information storage means. A tactile sense presentation mobile terminal. Temperature information storage means for storing temperature information indicating the temperature of the image displayed on the display means;
Cooling and heating means for cooling or heating the tactile pressure sense means;
A temperature control unit that controls cooling or heating of the cooling / heating unit based on the temperature information of the image acquired from the temperature information storage unit with respect to the tactile pressure unit corresponding to the tactile pressure sense region; The tactile pressure sense presenting portable terminal according to claim 1. The tactile pressure sense means is composed of a plurality of pins,
The tactile pressure sense presenting portable terminal according to claim 1, wherein unevenness information for each pixel of the image is associated with each of the plurality of pins. A method for presenting a tactile sensation of a portable terminal provided with a display means for displaying an image,
Obtaining unevenness information of the image from unevenness information storage means for storing unevenness information indicating unevenness of the image displayed on the display means;
Detecting a tactile sensation region presenting unevenness information in the screen of the display means according to the movement of the mobile terminal;
A tactile sensation means that presents a feeling of unevenness in the image corresponding to the tactile pressure sense region, and a step of moving up and down and / or vibrating based on the acquired unevenness information of the image. Pressure sense presentation method. Acquiring temperature information of the image from temperature information storage means for storing temperature information indicating the temperature of the image displayed on the display means;
5. The tactile sensation presentation method according to claim 4, further comprising a step of cooling or heating the tactile sensation means corresponding to the tactile sensation area based on the acquired temperature information of the image. The tactile sensation means is composed of a plurality of pins,
5. The tactile pressure sense presentation method according to claim 4, wherein unevenness information for each pixel of the image is associated with each of the plurality of pins.

Images