JP2016095832A - Electronic equipment, operation control method for electronic equipment, and operation control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide electronic equipment that even a visually impaired person can easily use by using a tactile sense (texture feeling).SOLUTION: Electronic equipment includes: a visual display part 20 for displaying an operation screen; a touch coordinate detection part 30 for detecting operation content performed to a display object; and a tactile sense presentation part 40 for presenting a tactile sense. The electronic equipment includes: tactile sense presentation control means 13 for allowing a user to present a tactile sense corresponding to the display object via the tactile sense presentation part 40 on the operation screen; operation content acquisition means 14 for detecting whether or not the temporal change of a touch position to be detected on the display object by the touch coordinate detection part 30 is equal to or less than a preliminarily applied threshold, and the state continues in a predetermined time or more; and sensory stimulation information output control means 15 for outputting sensory stimulation information corresponding to the display object to the outside when the state continues in the predetermined time or more.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an electronic device, an operation control method for the electronic device, and a program, and more particularly, to an electronic device that can be easily operated even by a visually impaired person.

  Conventionally, a display device equipped with a touch panel that can be input with a finger has been incorporated into a system that controls display contents and device operations in response to input, thereby contributing to the realization of easy-to-use interactive operability. Yes. For this reason, electronic devices incorporating touch panels, such as smartphones and tablet terminals, are rapidly spreading.

  In addition, among electronic devices of this type, many devices having a highly public purpose on the premise that they are operated by an unspecified number are provided with such a display device with a touch panel. More specifically, a ticket kiosk at a station, an ATM (Automated Teller Machine) at a bank, a multimedia kiosk terminal device at a convenience store, etc. Registered trademark), FamilyMart's "Famiport (registered trademark)", or vending machine for drinking water.

  On the other hand, the surface of a display device with a touch panel is uniformly hard, and has the same tactile sensation regardless of which part is displayed on the screen. For this reason, it is virtually impossible to perceive without relying on the visual sense (without looking at the panel) where the touch panel can be touched for effective input and whether or not valid input has been made. is there.

  If a person with visual disabilities is a device with independent operation keys, such as a remote control of a television receiver, a conventional mobile phone terminal (feature phone), or a keyboard of a personal computer, the operation keys It is possible to operate as long as you remember the position and location. However, it is almost impossible for a visually impaired person to operate an electronic device provided with a display device with a touch panel. In particular, in devices such as the above-described ticket vending machines and ATMs, measures are required to make it easier for visually impaired people to use these devices.

  In Patent Document 1, an electrode is embedded at a specific position on the front surface of a display device with a touch panel, and a voltage signal having a specific frequency is applied to the electrode to electrically vibrate the portion where the electrode is embedded. Describes a technique of presenting a tactile sensation (texture feeling, rough feeling) to a user's finger touching the surface of the apparatus using the above.

  However, with this technique, a texture is given even to the wiring for transmitting a voltage signal to the electrodes. In particular, when many electrodes are embedded in order to give a texture to many places, complicated wiring is required, and a texture is often given to places that are not necessary.

  Furthermore, with this technique, it is possible to present a textured feeling only at the locations where the electrodes are previously embedded. It is practically impossible to change the position and number of textures to be displayed corresponding to the display object because it involves rearrangement of electrodes.

  In addition, there are each of the following as technical documents related to this. Patent Document 2 describes a technique for reducing the size and power consumption of the touch panel by providing feedback to a user's fingertip that is operated by vibrating the touch panel. Patent Document 3 describes a technique in which the content of the key touched by the user when the touch panel vibrates and the finger is released is used as the input content.

  In Patent Document 4, when a setting operation of the device is performed according to the position and time the user touches the touch panel in a so-called multi-function machine having functions such as a printer and a copy, the setting contents are read out by voice, and A technique of vibrating the touch panel according to a pattern according to the setting is described.

JP 2011-248884 A JP 2009-169612 A Japanese Patent Laid-Open No. 2003-016502 JP 2014-131133 A WO 2011/052484 Specification US Patent Application Publication No. 2008/0204420

  According to the display device with a touch panel provided with the tactile presentation device described in the prior art described in Patent Document 1 described above, the location of the display object that is the target of the touch input can be determined only by the tactile sense without the user relying on vision. You can find it.

  However, in order to actually operate the device without relying on vision, it is necessary to store in advance the arrangement of each display object and the operation content corresponding to each display object. This is because only the “location of the display object” can be known through tactile sense, and there is no way to know on the spot what type of operation each display object corresponds to without relying on vision. is there.

  As mentioned above, devices that are used for highly public purposes such as ticket vending machines and ATMs are required to make it easier for visually impaired people to use these devices. The arrangement of display objects and the operation contents corresponding to the display objects are different for each device or each scene where the input operation is performed on the same device. It is not practically possible to have a common format for the operation system in all such devices and in all scenes where input operations are performed, such as the so-called QWERTY layout of a keyboard of a personal computer. .

  Therefore, it is extremely difficult for the user to store in advance the arrangement of display objects and operation contents that differ for each device. This is still insufficient as a measure for facilitating the use of the device by visually impaired persons. Further, the prior art described in Patent Document 1 gives a texture to an unnecessary part as described above. There is also a problem that “change of position and number is virtually impossible”. The technology for solving the above points is not described in any of the remaining Patent Documents 2 to 4.

  An object of the present invention is to provide an electronic device that can be easily used by a visually impaired person using a tactile sensation (texture feeling) presented at a position corresponding to a display object. It is to provide a method and program.

  In order to achieve the above object, an electronic apparatus according to the present invention performs a visual display unit that displays an operation screen including an operation target range by a user, and a display object displayed in the operation target range. An electronic device including a touch coordinate detection unit that detects the content of the touch operation and a tactile sense presentation unit that presents a tactile sensation perceivable by the user on the operation screen, and corresponds to a display object on the operation screen Tactile presentation control means for presenting a tactile sensation to the user via the tactile presentation unit, and the temporal change in the touch position detected on the display object by the touch coordinate detection unit is less than or equal to a preset threshold value and the state is preset An operation content acquisition unit that detects whether or not the display object has continued for a predetermined time, and a feeling corresponding to the display object when the state continues for a preset time or longer. Having a sensory stimulation information output control means for externally output via the sensory stimulation information output unit provided in advance stimulation information, and wherein.

  In order to achieve the above object, an operation control method for an electronic device according to the present invention is a method for controlling the operation of an electronic device including a visual display unit that displays an operation screen including a range to be operated by a user. The operation screen display control means creates an operation screen which is the content obtained by executing the processing by the means and displays it on the visual display unit. The operation screen is perceivable by the user within the operation target range on the operation screen. The tactile sense presentation control means presents the corresponding tactile sense via the tactile sense presenting unit, and the touch coordinate detecting unit detects the content of the touch operation performed by the user on the display object displayed in the operation target range, The operation content acquisition means determines whether the temporal change in the touch position detected on the display object is equal to or less than a preset threshold value and whether the state has continued for a preset time. And outputting the sensory stimulus information corresponding to the display object externally through the sensory stimulus information output unit provided in advance with the sensory stimulus information output control means when the state continues for a preset time or longer. Features.

  In order to achieve the above object, an operation control program for an electronic device according to the present invention provides a visual display unit that displays an operation screen including an operation target range by a user, and a display object displayed within the operation target range. A processor provided in an electronic device, comprising: a touch coordinate detection unit that detects a content of a touch operation performed by the user; and a tactile sense presentation unit that presents a tactile sensation perceivable by the user on the operation screen. In addition, a procedure for creating an operation screen that is the content obtained by executing the process and displaying it on the visual display unit, and a tactile sense corresponding to the operation screen that can be perceived by the user within the operation target range on the operation screen. Procedures to be presented via the presentation unit, procedures to detect the content of touch operations performed by the user on display objects displayed within the operation target range, display objects A procedure for detecting whether the temporal change in the touch position detected on the screen is less than or equal to a preset threshold value and whether or not the state has continued for a preset time, and when the state has continued for a preset time or more Further, it is characterized in that a procedure for outputting sensory stimulus information corresponding to the display object to the outside via a sensory stimulus information output unit in which sensory stimulus information output control means is provided in advance is executed.

  As described above, the present invention is configured to present a tactile sensation corresponding to a display object, and output a sound corresponding to the display object when a change in the touch position with time is below a threshold. Even if the user does not previously store the arrangement of the display object and the operation content corresponding to the display object, the user can perform the operation by sensing the sense of touch. Thus, it is possible to provide an electronic device, an operation control method for the electronic device, and a program that can be easily used by a visually impaired person.

On the other hand, since the present invention is configured to output a sound corresponding to a display object when a change with time of the touch position continues below a threshold value, a person who is not visually impaired can respond to the display object. No sound is output when tapping or other operation is performed. For this reason, for those who have no visual impairment, the input operation can be performed in a state where originally unnecessary sound is not output, so that it can be used comfortably.
In this manner, the present invention can provide an electronic device, an operation control method for the electronic device, and a program that can be used easily or comfortably by a person who is visually impaired or who is not visually impaired.

It is explanatory drawing shown about the structure of the electronic device which concerns on 1st Embodiment. FIG. 2 is an explanatory diagram conceptually showing a more detailed configuration of an overall control unit in the electronic device shown in FIG. 1. FIG. 2 is an explanatory diagram conceptually showing a more detailed configuration of a tactile sense providing unit in the electronic device shown in FIG. 1. It is explanatory drawing shown about the drive method of the tactile sense presentation part shown in FIG. It is explanatory drawing shown about the cross-sectional model of the tactile sense presentation part shown in FIG. 3 and FIG. 5 is a graph obtained by measuring the relationship between threshold values of amplitudes of voltage signals necessary for a user to perceive a change in tactile sensation with respect to a frequency of an attractive force acting on a finger in the tactile sense presenting device illustrated in FIGS. 3 is a flowchart illustrating an operation performed by an overall control unit (particularly, an operation content acquisition unit) in the electronic device illustrated in FIGS. It is explanatory drawing shown about the operating method of the electronic device shown to FIGS. FIG. 8A shows a state where the operation screen display control means displays the operation screen on the visual display unit (operation target range), and FIG. 8B shows that the user places a finger on the operation target range and performs a search. FIG. 8C shows a state where the user is pressing and holding a specific position on the operation target range, and FIG. 8D shows a state where the user is tapping a specific position on the operation target range. Each is shown. It is a flowchart which shows operation | movement of the electronic device which concerns on related technology. It is the apparatus used for the measurement of the speed of a finger, and a user's hand photograph. FIG. 10A is a photograph when the user is not searching for the operation target area, and FIG. 10B is a photograph when the user is searching for the target area. It is a graph which shows the result of having measured the time change of the speed of a fingertip. It is a figure explaining the shift | offset | difference of the position of the display object and the position of a tactile sense in the electronic device which concerns on related technology. FIG. 12A is a diagram showing the position of the display object when the finger is moved in the right direction and the position where the finger is present when a tactile sensation occurs, and FIG. It is the figure which showed the position of the display object at the time of moving, and the position where a finger exists when a tactile sensation occurs. It is a flowchart which shows operation | movement of the electronic device which concerns on 1st Embodiment. It is a figure explaining the gap with the position of the display object in the electronic device concerning a 1st embodiment, and the position of tactile sense. FIG. 14A is a diagram showing the position of the display object when the finger is moved in the right direction and the position where the tactile sensation is generated. FIG. 14B is a display object when the finger is moved in the left-right direction. It is the figure which showed the position of and the position where a tactile sensation occurs. It is explanatory drawing shown about the structure of the electronic device which concerns on the 2nd Embodiment of this invention. FIG. 16 is an explanatory diagram conceptually showing a more detailed configuration of the overall control unit in the electronic device shown in FIG. 15. 17 is a flowchart illustrating an operation performed by a general control unit (particularly, operation content acquisition unit) in the electronic device illustrated in FIGS. It is explanatory drawing shown about the structure of the electronic device which concerns on the 3rd Embodiment of this invention. It is a figure explaining the mobile body carrying the electronic device shown in FIG.

(First embodiment)
Hereinafter, the structure of the 1st Embodiment of this invention is demonstrated based on FIG.
The electronic device 100 according to the first embodiment of the present invention includes a visual display unit 20 that displays an operation screen including an operation target range 61 by a user, and display objects 62a, 62b,... Displayed in the operation target range. The electronic device includes a touch coordinate detection unit 30 that detects the content of the touch operation performed by the user and a tactile sense presentation unit 40 that presents a tactile sensation perceivable by the user on the operation screen. Here, the electronic device 100 includes a tactile sense presentation control unit 13 that presents a tactile sense corresponding to the display object on the operation screen to the user via the tactile sense presenting unit, and a touch position detected on the display object by the touch coordinate detecting unit. An operation content acquisition means 14 for detecting whether or not the temporal change is less than or equal to a preset threshold and the state has continued for a preset time or more, and a sense provided with sensory stimulus information corresponding to the display object in advance It has sensory stimulus information output control means (sound output control means 15) that outputs externally via a stimulus information output part (sound output part 50). This sensory stimulus information can be configured as sound.

  Here, the tactile sense presentation unit 40 is configured to be able to present different tactile sensations simultaneously in the first region and the second region on the operation screen. Further, the overall control unit 10 presents a tactile sensation that can be perceived by the user at a time before the user performs a touch operation on the position corresponding to the display object on the created operation screen. The tactile sense presentation control means 13 for driving the tactile sense presenting unit 40 is provided.

  Then, when an operation of hitting the operation screen at a position corresponding to the display object is detected, the operation content acquisition unit 14 executes a predetermined function associated with the display object 62a, 62b,. Be controlled.

With the above configuration, the electronic device 100 can be easily used by a person with visual impairment using the presented tactile sensation (texture feeling).
Hereinafter, this will be described in more detail.

  FIG. 1 is an explanatory diagram showing the configuration of the electronic device 100 according to the first embodiment of the present invention. The electronic device 100 includes a visual display unit 20, a touch coordinate detection unit 30, a tactile sense presentation unit 40, and an overall control unit 10 that performs overall control of the overall operation. The visual display unit 20 is typically a liquid crystal display device, and displays an operation screen as a result of processing performed by the overall control unit 10 to the user.

  The touch coordinate detection unit 30 is an optical touch sensor that is disposed in a frame portion of the visual display unit 20 that is typically a liquid crystal display device. A user can operate the operation screen displayed on the visual display unit 20. An operation such as a touch to be performed is received, and a point where the operation is performed on the operation screen is input to the overall control unit 10 as coordinate input information.

  The tactile sense presenting unit 40 is a tactile sense presenting panel attached to the surface of the visual display unit 20. The visual display unit 20, the touch coordinate detection unit 30, and the tactile sense presentation unit 40 have a common operation target range 61. The operation target range 61 is a range in which the visual display unit 20 displays an operation screen, and the touch coordinate detection unit 30 is also a range in which an operation performed by the user is accepted. In FIG. 1, the visual display unit 20, the tactile sense presentation unit 40, and the touch coordinate detection unit 30 are stacked in this order from the bottom, but the operation target range 61 may function in common. For example, the order is not particularly limited.

  Although a more specific configuration will be described later, the tactile sense providing unit 40 has a tactile sense (texture feeling) at a position corresponding to each of the specific display objects 62a, 62b,. And the user can determine the position of the display object only by touch.

  The overall control unit 10 is typically a processor as a main body that executes a computer program, and functions as each operation means described in the next stage when the operation control program is executed. The overall control unit 10 is also provided with a sound output unit 50. The sound output unit 50 is typically a speaker or an earphone, and outputs a sound as a result of processing performed by the overall control unit 10 to the user. The overall control unit 10 does not have to be in the same housing as the visual display unit 20, the touch coordinate detection unit 30, and the tactile sense presentation unit 40.

  FIG. 2 is an explanatory diagram conceptually showing a more detailed configuration of the overall control unit 10 in the electronic device 100 shown in FIG. The overall control unit 10 functions as an application execution unit 11, an operation screen display control unit 12, a tactile presentation control unit 13, an operation content acquisition unit 14, and a sound output control unit 15 by executing an operation control program. To do. The overall control unit 10 is also provided with temporary storage means 16 for temporarily storing various data (time parameters, touch position vectors, etc. described later) necessary for operation of the operation control program.

  The application execution unit 11 performs a general operation corresponding to a user input operation performed by the visual display unit 20 and the touch coordinate detection unit 30, and causes the visual display unit 20 to display an operation screen as a result. Further, an operation command is also given to the tactile sense presentation unit 40 according to the content of the operation screen displayed on the visual display unit 20. Further, the sound output control means 15 is designated with a sound to be reproduced, and a command is given to reproduce the sound.

  Here, the general operation is, for example, sales of an item selected by the user when the electronic device 100 is a ticket vending machine or a vending machine. Moreover, when the electronic device 100 is a bank ATM, it is processing, such as deposit / withdrawal, transfer, and transfer according to a user's operation. Of course, the electronic device 100 is also provided with a unique mechanism or the like for performing these operations, but these are not particularly shown because they do not need to be described in detail here.

  The operation screen display control unit 12 causes the visual display unit 20 to display an operation screen corresponding to the operation performed by the application execution unit 11. The tactile sensation presentation control means 13 controls the tactile sensation presentation unit 40 so as to present a tactile sensation (texture feeling) at a position corresponding to a specific display object 62a, 62b,.

  The operation content acquisition means 14 detects the operation content intended by the user from the coordinate input information which is the content of the operation input by the user to the touch coordinate detection unit 30 on the displayed operation screen, and the detected operation content To the application execution means 11. The sound output control unit 15 receives the output from the application execution unit 11 and outputs the sound designated by the application execution unit 11 via the sound output unit 50.

(1st Embodiment: About a tactile sense presentation part)
FIG. 3 is an explanatory diagram conceptually showing a more detailed configuration of the tactile sense presentation unit 40 in the electronic device 100 shown in FIG. The tactile sense presentation unit 40 includes a plurality of X electrodes 42 extending in the x direction on the planar support substrate 41, and a plurality of Y electrodes 43 extending in the y direction orthogonal to the X electrodes 42 on the support substrate 41. The X electrode drive circuit 44 connected to each of the X electrodes 42, the Y electrode drive circuit 45 connected to each of the Y electrodes 43, and each of the X electrode drive circuit 44 and the Y electrode drive circuit 45 connected to each other It is comprised by the control part 46. FIG.

  The X electrode 42 and the Y electrode 43 intersect with each other through an insulating film at the intersecting portion, and electrical insulation between the two is maintained. In addition, an insulating film is formed on the X electrode 42 and the Y electrode 43, and when the user touches the surface of the electronic device 100 with a finger from above, the gap between the X electrode 42 and the finger, and between the Y electrode 43 and the finger. Is electrically insulated.

  The control unit 46 controls the X electrode drive circuit 44 and the Y electrode drive circuit 45 based on the information about the target area to be presented with the texture feeling input from the tactile sense presentation control means 13. The X electrode drive circuit 44 and the Y electrode drive circuit 45 send a voltage signal of a necessary frequency to the electrodes in a necessary range of the X electrode 42 or the Y electrode 43 according to the control information input from the control unit 46. Apply.

FIG. 4 is an explanatory diagram showing a driving method of the tactile sense presentation unit 40 shown in FIG. Here, the X electrode 42 and the Y electrode 43 are distinguished from each other by different symbols. That is, in the example shown in FIG. 4, but on the support board 41 28 present X electrodes 42 and the 46 pieces of Y electrodes 43 are formed, X ₀₀ ~ upward the X electrode 42 of their respective from below called X _27, each of the Y electrodes 43 from right to left of _Y 03 _{to Y 48.}

In addition, a region where a texture feeling is to be presented is a target region 47. The target region 47 is in the range of X _{11 to} X _{14 for} the X electrode 42 and in the range of Y _{24 to} Y _{27 for} the Y electrode 43. The controller 46 gives control signals to the X electrode drive circuit 44 and the Y electrode drive circuit 45 based on the information of the target region 47 given from the outside.

In response to this control signal, the X electrode drive circuit 44 applies a voltage signal of frequency f ₁ = 1000 Hz to X _{11 to} X ₁₄ , and the Y electrode drive circuit 45 applies a voltage of frequency f ₂ = 1240 Hz to Y ₂₄ to Y _27. Apply a signal. Here, the X electrode drive circuit 44 and the Y electrode drive circuit 45 are shown in FIG. 4 in order to prevent voltage induced by capacitive coupling between the electrodes for the X electrode 42 and the Y electrode 43 that do not correspond to them. In the example, it is grounded. Alternatively, a DC voltage or a voltage signal having a frequency of 2240 Hz or more may be applied instead of grounding (the reason will be described later).

When the above signals are applied to the X electrode 42 and the Y electrode 43 and the surface of the tactile sense presentation unit 40 is traced with a finger, a texture is perceived only in the target region 47 where X _{11 to} X ₁₄ and Y ₂₄ to Y ₂₇ intersect. The By arbitrarily selecting an electrode to which this voltage signal is applied, it is possible to present a texture to an arbitrary predetermined region. Further, by selecting all the X electrodes and all the Y electrodes, it is possible to present a texture to the entire region including all the intersections between the X electrodes and the Y electrodes.

We have _{experimentally,} X 11 _{to X} in the region excluding the region on the electrode target area 47 of the ₁₄ are not presented feeling texture, also excluding the target area 47 from the area on the _Y 24 _{to Y 27} electrodes It was confirmed that the texture was not presented even in the area. That is, the inventors have confirmed that human fingers do not perceive a texture when the frequency of the voltage signal applied to the electrodes is 1000 Hz or 1240 Hz.

On the other hand, in the target region 47, the X electrode to which the voltage signal of f ₁ = 1000 Hz is applied and the Y electrode to which the voltage signal of f ₂ = 1240 Hz is applied are adjacent to each other. Has occurred. Hereinafter, a mechanism for presenting a feeling of texture by beat will be described.

  FIG. 5 is an explanatory diagram illustrating a cross-sectional model of the tactile sense presentation unit 40 illustrated in FIGS. 3 and 4. As described above, the plurality of X electrodes 42 and the plurality of Y electrodes 43 are arranged adjacent to each other on a planar support substrate 41 (not shown in FIG. 5). Here, of the X electrode 42 and the Y electrode 43, one electrode 48 that models a finger is arranged at a position facing the two X electrodes 42 and the two Y electrodes 43 arranged in the target region 47. Has been. Since the human body has a grounding effect, the electrode 48 can be modeled as being grounded through a resistor 49 having a resistance value R.

Now, a voltage signal V ₁ represented by V ₁ = Acos (2πf ₁ t) is applied to the X electrode 42 in the target region 47. The amplitude of the voltage signal _{V 1} was A, the frequency is _{f 1,} t represents time. In addition, a voltage signal V ₂ represented by V ₂ = Acos (2πf ₂ t) is applied to the Y electrode 43 in the target region 47. The amplitude of the voltage signal _{V 2} is the amplitude equal to A, the frequency of the voltage signal _{V 1} was a _{f 2.}

  Between the electrode 48 and each of the X electrodes 42 in the target region 47 can be modeled as a parallel plate capacitor having a capacitance C, and between the electrode 48 and each of the Y electrodes 43 in the target region 47. Can be modeled as a parallel plate capacitor having capacitance C.

At this time, the voltage V _P appearing at the electrode 48 is V _P = (V ₁ + V ₂ ) / 2 when the resistance value R is sufficiently high.

The electrostatic force acting between one X electrode 42 and the electrode 48 modeling a finger is expressed as F _e1 as shown in FIG. F _e1 is obtained as follows when a formula known as a force acting between electrodes of a parallel plate capacitor is applied. Here, ε is the dielectric constant of the insulating film on the X electrode, and S is the electrode area of the parallel plate capacitor.

Similarly, F _e2 when expressed as F _e2 to an electrostatic force shown in FIG. 3 acting between the one Y electrode 43 and the finger electrode 48 that models the is determined as follows.

If the distance between the electrodes is so fine that the electrostatic force F _e1 and the electrostatic force F _e2 cannot be distinguished by a human fingertip, the total force of F _e1 and F _e2 will act on the finger macroscopically. Can be considered. Since the sum F of all forces acting on the electrode 48 modeling the finger is F = 2 (F _e1 + F _e2 ) from FIG. 3, the values of V ₁ , V ₂ , Formula 1 and Formula 2 are used. Thus, it can be obtained as follows.

From [Equation 3], the total F of all forces acting on the modeled electrode 48 is a periodic function having a value range of [0, A ² C ² / (εS)] and a frequency of (f ₁ + f ₂ ). It can be seen that the value range is [0, 2] and the frequency is obtained by multiplication by a periodic function that is an absolute value of (f ₁ −f ₂ ). The frequency of the envelope is an absolute value of (f ₁ −f ₂ ).

In this basic form, since the frequency f ₁ = 1000 Hz and the frequency f ₂ = 1240 Hz, the absolute value of the difference is 240 Hz. Therefore, the attractive force F acting on the finger changes at 240 Hz as shown in [Equation 3]. Therefore, when a person traces the surface of the tactile sense presentation unit 40 with a finger, a change in frictional force occurs at a frequency of 240 Hz. Since 240 Hz is a frequency at which the human skin mechanoreceptor is sensitive, texture perception can be provided.

  Furthermore, the inventors have confirmed through experiments whether or not a texture is perceived with respect to the frequency of the voltage signal. When the same voltage signal is applied to all the X electrodes 42 and Y electrodes 43 on the support substrate 41 and the presence or absence of texture is confirmed, the texture is found when the frequency of the voltage signal is in the range of more than 5 Hz and less than 500 Hz. It was confirmed that the texture was not perceived when the feeling was perceived and the frequency of the voltage signal was not within this range.

In addition, a voltage signal having the frequency f ₁ is applied to all the X electrodes 42 on the support substrate 41, a voltage signal having the frequency f ₂ is applied to all the Y electrodes 43, and the absolute value of the difference between f ₁ and f ₂ is applied. We experimentally confirmed the presence of texture perception. As a result, when the absolute value of the difference between f ₁ and f ₂ is greater than 10 Hz and less than 1000 Hz, the texture is perceived, and when the absolute value of the difference between f ₁ and f ₂ is 10 Hz or less or 1000 Hz or more, the texture is It was confirmed that it was not perceived.

From these results, when the frequency of the voltage signal applied to the X electrode is f ₁ and the frequency of the voltage signal applied to the Y electrode is f ₂ , f ₁ and f ₂ are both 500 Hz or more, and f ₁ and f If f ₁ and f ₂ are set so that the absolute value of the difference from ₂ is greater than 10 Hz and less than 1000 Hz, an X electrode that gives a voltage signal of frequency f ₁ and a Y electrode that gives a voltage signal of frequency f ₂ It is possible to realize a tactile sensation presentation unit 40 that presents a texture feeling in a region where and intersect, and does not present a texture feeling in other regions.

  Furthermore, from the matter described in [Equation 3] and its consideration, it was considered that the frequency of the attractive force acting on the finger was acting on the perception of the texture. An experiment was conducted to confirm the relationship with perception. FIG. 6 is a graph obtained by measuring the relationship between the thresholds of the amplitudes of the voltage signals necessary for the user to perceive a change in tactile sensation with respect to the frequency of the attractive force acting on the finger in the tactile sense providing unit 40 shown in FIGS. .

  In the graph of FIG. 6, the same voltage signal is applied to all the X electrodes 42 and all the Y electrodes 43 on the support substrate 41 while changing the frequency, and the threshold value of the amplitude necessary to perceive a tactile change is shown. It is the result of measurement. The lower axis shows the frequency of the voltage signal applied to all the X electrodes 42 and all the Y electrodes 43, and the left axis shows the threshold value of the amplitude of the voltage signal necessary to perceive a tactile change.

In this experiment, the frequency of the attractive force acting on the operator's finger is twice the frequency f ₁ of the applied voltage signal. In order to derive this relationship, the resistance value of the resistor R49 shown in FIG. 5 is set to a finite value excluding infinity, extremely zero, and the frequency of the voltage signal applied to the X electrode 42 and the Y electrode 43 is set. Are both set as f ₁ and the electrostatic force F may be obtained. In FIG. 6, the frequency of the attractive force acting on the finger is shown on the upper axis. That is, the relationship between the frequency of the attractive force acting on the finger and the amplitude threshold necessary for perception is represented by the upper axis and the left axis in FIG.

  From the graph of FIG. 6, it can be seen that the threshold value is minimum when the frequency of the attractive force acting on the finger is around 200 Hz. That is, it can be said that the human skin receptor perceives the texture with the highest sensitivity when the frequency of the attractive force acting on the finger is around 200 Hz. From the graph of FIG. 6, the bottom of the graph of the relationship between the threshold and the frequency is not only that the frequency of the attractive force acting on the finger is around 200 Hz, but the beginning and end of the trough of the graph are around 10 Hz and 1000 Hz, respectively. I understand that there is.

  That is, the texture is perceived when the frequency of the attractive force is within a range of 10 to 1000 Hz. At frequencies outside this range, texture is not perceived and friction is perceived.

The effect | action in the above experiment can be demonstrated as follows. When a voltage signal having a frequency f ₁ is applied to the predetermined X electrode 42 on the support substrate 41 and a voltage signal having a frequency f ₂ different from the frequency f ₁ is applied to the predetermined Y electrode 43, the X electrode and the Y electrode In the target region 47 including the intersection, an attractive force having an absolute value of the frequency (f ₁ −f ₂ ) acts on the finger.

For this reason, if the absolute value of the frequency (f ₁ −f ₂ ) is greater than 10 Hz and less than 1000 Hz, a texture is presented to the target region 47 including the intersection of the predetermined X electrode 42 and the predetermined Y electrode 43. can do.

In the region on the X electrode excluding the target region 47 including the intersection, an attractive force having a frequency twice the frequency f ₁ acts on the finger from the formula of the force acting between the electrodes of the parallel plate capacitor. In the region on the Y electrode excluding the target region 47 including the intersection, an attractive force having a frequency twice as _{high as} the frequency f2 acts on the finger.

Therefore, by setting both f ₁ and f ₂ to be 500 Hz or more, a region on a predetermined X electrode excluding a target region 47 including an intersection of a predetermined X electrode and a predetermined Y electrode or a predetermined Y electrode In both areas, an attractive force of 1000 Hz or more acts on the finger, and the texture is not presented. Therefore, if the target area 47 is referred to as the first area and the other areas are referred to as the second area, the tactile sense presentation unit can simultaneously present different tactile sensations in the first area and the second area. It can also be configured.

  In the existing tactile sense presentation device (the above-mentioned Patent Document 1), a space for routing a plurality of independent wires for each electrode for presenting a texture feeling is required, and as a result, a texture feeling is presented. There is a problem that the distance between the electrodes is wide and the spatial resolution of the tactile sense presentation device is low. In this respect, in the tactile sense presenting unit 40, the electrode for presenting the sense of texture also serves as the wiring, so that the spatial resolution can be increased.

  Further, according to this embodiment, since the shape of the electrode is difficult to be visually recognized, it is possible to suppress deterioration of the display quality inherent to the display device even when the electrode is overlapped with the visual display unit 20. The existing tactile sensation presentation apparatus has a problem that an originally unnecessary texture is presented in the area where the wiring is routed. However, according to this embodiment, this problem is also solved.

  In the example described above, there is only one target region 47. However, this example can be easily expanded even when the target region has a plurality of locations. That is, the absolute value of the difference in frequency applied to each of the X electrode and the Y electrode is within a range of 10 to 1000 Hz (preferably near 200 Hz) in each target region, and is outside that range outside the target region. Thus, the electrode number to which the voltage signal is applied and its frequency may be determined.

  As described above, the tactile sense presentation unit 40 can present a tactile sensation different from other regions in an arbitrary region on the tactile sense presentation surface above the X electrode 42 and the Y electrode 43. The position of the region or presence / absence of tactile sensation can be freely controlled by voltage signals applied to the X electrode 42 and the Y electrode 43. Various tactile sensations can be presented by changing the waveform and amplitude of the voltage signal.

  The tactile sense presentation unit used in the first embodiment is composed of a tactile sense presentation device having the following characteristics. That is, the support substrate 41, a plurality of parallel X electrodes 42 extending in the first direction on the support substrate, and extending in the second direction on the support substrate and insulated from the X electrodes. A plurality of Y electrodes 43 parallel to each other, a voltage signal of a first frequency is applied to the X electrode corresponding to the target region 47 input from the outside among the X electrodes, Among them, a drive circuit that applies a voltage signal of the second frequency to the Y electrode corresponding to the target region is provided, and the first and second frequencies are both 500 [Hz] or more, and the first and second This is a tactile sensation presentation device having a period in which the absolute value of the frequency difference is greater than 10 Hz and less than 1000 Hz.

(First embodiment: About operation content acquisition means)
FIG. 7 is a flowchart illustrating an operation performed by the overall control unit 10 (particularly, the operation content acquisition unit 14) in the electronic device 100 illustrated in FIGS. In the operation described here, an input operation detected by the operation content acquisition unit 14 and detection conditions for the operation will be described first.

  First, in the electronic device 100, the visual display unit 20 is a range in which an operation screen is displayed, and the touch coordinate detection unit 30 is also a range in which an operation performed by a user is received. A range that is also a target to present a tactile sensation (texture feeling) at a position corresponding to a specific display object 62a, 62b,...

  Then, touching the operation target range 61 with the user's finger is referred to as “touchdown”. During the touchdown, the coordinate input information is detected by the touch coordinate detection unit 30. Further, the user's finger moving away from the operation target range 61 is referred to as “touch-up”.

  The operation in which the user's finger taps the operation target range 61 is referred to as “tap”. The generation condition of the tap is that “the touch-up was performed without a change in the touch position within 1 second after the occurrence of the touch-down”. Here, in order to suppress the influence of fluctuations in the touch position due to an error in operation performed by a person with his / her finger, the change in the touch position at each occurrence of touchdown and touchup is determined by the tap detection threshold S. If it is smaller, it is determined that “the touch position has not changed”.

The operation in which the user's finger continues to move while touching the operation target range 61, that is, searches for the display objects 62a, 62b,... The search continuation condition is “the touch position is continuously changed without being stationary for 1 second or longer”. Again, in order to suppress the influence of fluctuations in the touch position caused by an operation error or the like, the movement of the touch position in one cycle (typically 1/120 seconds) in which the position detection process is performed is the stationary detection threshold v. If it is smaller than _m , it is determined to be “still”.

The tap detection threshold value S is a threshold value for determining “change in touch position for 1 second”, whereas the static detection threshold value v _m is determined as “change in touch position for 1 cycle (1/120 seconds)”. There is a difference in that it is a threshold for Of course, the time conditions such as “1 second” and “1/120 seconds” are design items that can be changed as necessary.

  An operation in which the user's finger stops at a specific position on the operation target range 61 for one second or longer is referred to as “long press”. The condition for generating the long press is that “the state of touching the same position has continued for one second or longer”. Here again, it is determined whether or not the touch position is stationary under the same conditions as in the “search” determination.

  When the application execution unit 11 is in a state of accepting an input operation by the user, the operation screen display control unit 12 displays an operation screen in a state of accepting the input operation on the visual display unit 20 (operation target range 61) (step S101). Further, the tactile sense presentation control means 13 controls the tactile sense presenting unit 40 so as to present a tactile sense (texture feeling) at a position corresponding to the specific display object 62a, 62b,. S102).

Then, the operation content acquisition unit 14 initializes the time parameter k = 0 and the touch position vector r ₀ = (0, 0) and stores them in the temporary storage unit 16 (step S103), and touches within the operation target range 61. A state of waiting for the occurrence of down is entered (step S104). Of course, the initial value of this r ₀ is just one example of setting.

If no touchdown occurs (No in step S104), the standby state is continued. If a touchdown occurs (Yes in step S104), the operation content acquisition unit 14 initializes the time parameter n = 0 (step S105), increments the value of n (increments by one), and temporarily stores the unit. 16 (step S106), the touch position vector rn which is the touch position at the current value of the time parameter _n is acquired, and at the same time, this is stored in the temporary storage means 16 (step S107).

Then, the operation content acquisition unit 14 touches the touch position vector r _{n at} the current time parameter (n) and the touch position vector r at the time parameter (n−1) immediately before the current time stored in the temporary storage unit 16. the distance between the _n-1 is still detection threshold _{v m} whether the difference is less than, i.e. _| r _{n -r} n-1 | determines _{<v m} or not (step S108).

Since Typically the process of the position detection to be performed at a frequency of 120 times per second, r _n-1 is the touch position in 1/120 seconds before time than r _n In other words, | r _n It can be considered that −r _n−1 | is the moving speed of the touch position in 1/120 seconds. The fact that this determines whether the still detection threshold v or less than _m, while included in account the variation of the touch position humans is due to such an error of the operation carried out in their fingers, still without touch position changes This means that it is determined whether or not it is in a state of being performed.

  If it is determined that it is not stationary (No at Step S108), the operation content acquisition unit 14 resets the time parameter k = 0 and stores it in the temporary storage unit 16 (Step S109), and waits for the occurrence of touch-up. (Step S110). If touch-up does not occur (No in step S110), the processing from step S106 is repeated.

If the touch-up occurs (Step S110 is YES), the operation content acquisition unit 14, and r _n is a touch position vector of the point of touch-up occurs, the touchdown stored in the temporary storage means 16 has occurred It is determined whether or not the distance from the current touch position vector r ₁ is smaller than the tap detection threshold S, that is, whether or not | r _n −r ₁ | <S (step S111). If this is YES, the operation content acquisition unit 14 continues to determine whether or not the time parameter k <120 (step S112). If either of steps S111 and S112 is NO, the process returns to step S103, and the process by the operation content acquisition unit 14 is performed again from the initial setting.

  The determination of whether or not the change in the touch position at the time of occurrence of each of the touchdown and the touchup is smaller than the tap detection threshold S means that the change in the touch position due to the above-described operation error or the like. This means that it is determined whether or not the touch position has not changed between the touchdown and the touchup. The determination of whether or not the time parameter k <120 means that the position detection process is performed at a frequency of 120 times per second, so whether the time from touchdown to touchup is within 1 second. It means to judge whether or not.

  If both steps S111 and S112 are yes, the “tap generation condition” that “the touch-up was performed within one second after the occurrence of the touch-down without any change in the touch position” was satisfied. Therefore, the operation content acquisition unit 14 notifies the application execution unit 11 that a tap operation has occurred at the coordinate position (step S113), and returns the process to step S103.

  If step S108 is YES, that is, it is determined that the touch is continued in a stationary state, the operation content acquisition unit 14 increments the value of the time parameter k and stores it in the temporary storage unit 16 (step S114). , K> 120, that is, whether or not the touch is continued for 1 second or more is determined (step S115). If this is NO, the process proceeds to step S110 and waits for the occurrence of touch-up.

  The state in which Step S115 is No, that is, a state in which the user touches the operation target range 61 and is searching for a place where a tactile sensation (texture feeling) is presented by the processing in Step S102, that is, “1 second or longer” The “continuation condition of search” is satisfied, “the touch position is continuously changed without being stationary”. Therefore, in this state, the time parameter k is not reset.

  If step S115 is YES, since the “long press occurrence condition” that “the state of touching the same touch position has continued for 1 second or longer” is satisfied, the operation content acquisition unit 14 The fact that the long press occurrence condition is satisfied and the touch coordinates are transmitted. The application execution means 11 instructs the sound output control means 15 to output a sound corresponding to the display object based on the touch coordinates (step S116), resets the time parameter k to 0 (step S117), The processing from step S106 is repeated. Upon receiving the instruction, the sound output control means 15 outputs the sound via the sound output unit 50. Thereafter, this operation is repeated until the electronic device 100 finishes the operation.

(First embodiment: specific operation method)
An operation method of electronic device 100 realized by the operation described above will be described. FIG. 8 is an explanatory diagram showing an operation method of the electronic device 100 shown in FIGS. 8A shows a state in which the operation screen display control means 12 displays the operation screen on the visual display unit 20 (operation target range 61), and FIG. 8B shows a state in which the user places a finger on the operation target range 61. FIG. 8C shows a state in which a search is being performed, FIG. 8C shows a state in which the user presses and holds a specific position on the operation target range 61, and FIG. 8D shows a specific position on the operation target range 61. Each tapped state is shown.

  A plurality of display objects 62a, 62b,... Are displayed on the operation screen displayed on the visual display unit 20 (operation target range 61) by the operation screen display control means 12. The display object may be single or plural, and items to be displayed and their arrangement and positional relationship can be arbitrarily determined. For example, it may be a numeric keypad for inputting numbers from 0 to 9, alphabets or kana characters, or “OK” or “Cancel”.

  However, the position where each display object 62a, 62b,... Is displayed is a position where the touch coordinate detection unit 30 simultaneously receives an input operation corresponding to the display item, and the tactile sense presenting unit 40 presents a tactile sense (texture feeling). It will also be a position to do. In the example shown in FIG. 8A, numeric keys are displayed as the display objects 62a, 62b,.

  In the example shown in FIG. 8B, the user searches the display objects 62a, 62b,... That are presented with a sense of touch (texture feeling) while touching the operation target range 61 with the finger 63. It is in a state of being. The user at this time can find the location of the display objects 62a, 62b,... By tactile sense, but does not know which display object corresponds to which input content (such as characters).

  Therefore, as shown in FIG. 8C, the user stops the movement of the finger 63 at a position where one of the display objects 62a, 62b,... Is found, and rests at the same touch position for 1 second or longer. Then, the “long press” shown in steps S115 to 117 in FIG. 7 is detected, and the sound output control means 15 outputs the sound having the content corresponding to the display object. For example, if this corresponds to a key for inputting “1” among the numeric keys, a sound “1” is output from the sound output unit 50.

  If the output sound is intended by the user, as shown in FIG. 8D, the user taps the position, that is, performs a “tap” operation. Then, the content, that is, the operation content of inputting “1” in this example, is passed to the software operating by the application execution means 11. If “1” is not the content intended by the user, the user continues the finger movement, that is, the search shown in FIG.

  As described above, in the electronic device according to the first embodiment, when the operation content acquisition unit 14 detects an operation of hitting the operation screen at a position corresponding to the display object, the electronic device via the application execution unit 11 Then, control is performed to execute a predetermined function (in this example, an operation of inputting “1” to the operating software) associated with the display object.

  In the above example, an explanation has been given of an example in which a sound of “1” is output when a button display object with a number “1” written thereon is pressed and held, but the output sound is similar to a sound emitted by a human through a vocal organ. The sound is not limited to For example, a pattern composed of sound generation and stop, such as Morse code “· −−−−”, may be output. Alternatively, when the function of the button of the display object is assigned to play a specific music piece, a part or all of the music piece may be output as a sound.

  In the above example, an example in which a sound of “1” is output when the display object of the button on which the number “1” is written is long pressed is described. However, the output information is not limited to sound. However, information that stimulates all sensory organs (sensory stimulus information), that is, information that stimulates hearing, touch, taste, sight, and smell can be used. For example, if the user's degree of visual impairment is such that characters cannot be read but light and dark can be recognized, the user understands the meaning of the display object by controlling the light and dark of the screen according to Morse code be able to.

  Alternatively, even if the screen is vibrated according to the Morse code, the user can understand the meaning of the display object. Alternatively, the user can understand the meaning of the display object by providing the taste presentation means and presenting the taste corresponding to the display object. Alternatively, the user may be able to understand the meaning of the display object by providing olfactory presentation means and presenting an odor corresponding to the display object.

Accordingly, the electronic device of the present invention is expressed as follows.
That is, a visual display unit that displays an operation screen including an operation target range by the user, and a touch coordinate detection unit that detects the content of the touch operation performed by the user on the display object displayed in the operation target range And a tactile sense providing unit that presents a tactile sense perceivable by the user on the operation screen,
Tactile sensation control means for causing a user to present a haptic corresponding to the display object on the operation screen via the tactile sensation presentation unit;
Operation content acquisition means for detecting whether a temporal change in the touch position detected on the display object by the touch coordinate detection unit is equal to or less than a predetermined threshold value and whether the state has continued for a predetermined time;
Sensory stimulus information output control means for externally outputting sensory stimulus information corresponding to the display object via the sensory stimulus information output unit provided in advance when the state continues for the predetermined time or longer. It is a feature.

(Overall operation and effect of the first embodiment)
Next, the overall operation of the first embodiment will be described.
The operation control method for an electronic device according to the present embodiment is a method for controlling the operation of the electronic device 100 including the visual display unit 20 that displays an operation screen including the operation target range 61 by the user. An operation screen, which is the content obtained by executing the process, is created by the operation screen display control means 12 and displayed on the visual display unit 20 (FIG. 7: step S101). The tactile sensation presentation control means 13 presents the tactile sense corresponding to the operation screen perceivable by the user via the tactile sense presenting unit 40 (FIG. 7: step S102), and the display objects 62a and 62b displayed in the operation target range 61. ,... Is detected by the touch coordinate detection unit 30 (FIG. 7: step S107) and detected on the display object. The operation content acquisition means 14 detects whether or not the temporal change of the touched position is less than or equal to a preset threshold and the state has continued for a preset time or more (FIG. 7: Steps S108 and 115). If the sensory stimulus information output control means (sound output control means 15) is preliminarily provided with sensory stimulus information output control means (sound output control means 15), the sensory stimulus information output control means (sound output part 50) is provided. To the outside (FIG. 7: step S116). This sensory stimulus information can be configured as sound.

  Here, the step of presenting the tactile sensation by the tactile sense presenting unit 40 presents a tactile sensation perceived by the user at a position corresponding to the display objects 62a, 62b, ... on the operation screen. (FIG. 7: Step S102). Further, it is detected whether or not the operation content acquisition means 14 has moved on the operation screen at a position corresponding to the display objects 62a, 62b,... (FIG. 7: Steps S110 to 112). The operation content acquisition unit 14 causes the application execution unit 11 to execute the operation content corresponding to the display object (FIG. 7: Step S113).

Here, each of the above-described operation steps may be programmed to be executable by a computer, and may be executed by a processor included in the overall control unit 10 that directly executes each of the steps. The program may be recorded on a non-temporary recording medium, such as a DVD, a CD, or a flash memory. In this case, the program is read from the recording medium by a computer and executed.
By this operation, this embodiment has the following effects.

  As described above, if the tactile sense presentation device described in the prior art is applied to an electronic device, the user can find the location of the display object that is the target of tap input without relying on vision. However, in order to actually operate the device without relying on vision, it is necessary to store in advance the arrangement of each display object and the operation content corresponding to each display object. In particular, this is still insufficient as a measure for making it easier for visually impaired people to use the electronic device.

  In this embodiment, when the user finds the location of a display object for which a tactile sensation is presented in advance without using touch information at a position corresponding to the operation key, the operation content corresponding to the display object is read out by sound. It is done. Thus, even a user who does not store the arrangement of each display object and the operation content can easily operate the electronic device without relying on vision. In addition, even when the user touches the operation target range with a plurality of fingers at the same time, the operation can be performed in the same manner as in this example, so that the operability can be improved in this respect.

(Effect common to 1st Embodiment and 2nd Embodiment mentioned later)
In addition, since the present invention is configured to output a sound corresponding to the display object when a change in the touch position with time continues to be equal to or less than a threshold value, a user who is not visually impaired can respond to the display object. No sound is output when tapping or other operation is performed. For this reason, it is possible for a user who has no visual impairment to comfortably use an input operation in a state where originally unnecessary sound is not output.
As described above, the present invention does not require a change in device settings between a user who is visually impaired and a user who is not visually impaired, and it is easy or comfortable for both users. An electronic device that can be used, an operation control method for the electronic device, and a program can be provided.

(Effects peculiar to the first embodiment)
Further, in this embodiment, since the texture is localized and presented at the position of the display object based on the position information of the display object, the position shift caused by the time lag, which is one of the problems of the prior art. The problem is solved. For example, Patent Document 4 described above describes a technique of performing voice output and panel vibration “depending on the content of an input operation”, which determines an input operation performed via a touch panel, The panel is vibrated according to the result. That is, there has been a problem that a deviation occurs between the position of the operation key and the position where the tactile sensation occurs due to the time lag resulting from the determination.

  On the other hand, in the present embodiment, information on an input operation performed via the touch panel to present a tactile sensation is not required. The tactile sense presentation unit of the first embodiment is configured so that different tactile sensations can be presented simultaneously in the first area and the second area on the operation screen, so that the user performs a touch operation on the display object. The tactile sensation presentation control means for driving the tactile sensation presentation unit is provided so that the tactile sensation perceived by the user can be presented at the position of the display object at a time before the tactile sense. Can be presented. For this reason, the problem of misalignment does not occur. Therefore, even when the user searches quickly, the position of the display object and the position of the texture feeling always coincide with each other, and the operability is remarkably improved.

(Addition of effects of the first embodiment)
The problem that a deviation occurs between the position of the display object and the position where the tactile sensation occurs is a problem found through the inventors' thoughts and experiments. Details are described below.

  FIG. 9 is a flowchart created by the present inventor based on the description in Patent Document 3. This flowchart is described in Patent Document 3, “When a transaction selection screen is displayed on the display unit and the position of the customer's finger touching the touch panel enters the key area displayed on the display unit, the vibration generating unit Represents the process of vibrating the touch panel. The process goes through the following steps in order.

A transaction selection screen including a display object corresponding to the operation key (key) is displayed on the visual display unit (display unit) (S301).
The user (customer) touches the screen (touch panel surface) with a finger (S302).
-A touch coordinate detection part detects a touch, and detects a touch coordinate (S303).
The touch coordinates are compared with the position (area) of the display object (S304).
If the touch coordinates are within the area of the display object, a signal for driving the actuator (vibration generating unit) is generated. (S305)
The actuator vibrates the touch panel surface (S306).

  Since the above steps are sequentially performed, when the user searches for the operation key by searching, a predetermined time elapses from when the user's finger enters the range of the display object until the touch surface vibrates. This time is called delay or time lag. This time lag (that is, the time from the end of the process in step S302 until the actual touch surface starts to vibrate in step S306 after the process in steps S303 to S305) is referred to the response time of the current smartphone. , 55 milliseconds or more.

  On the other hand, when the user searches for the position of the operation key, it has been found through experiments by the present inventors that the speed of the finger of the user exceeds 700 mm / second. The product of the time lag and the finger speed is 38.5 mm, and this value represents the difference between the position of the display object and the position where the finger is present when a tactile sensation occurs.

  The measurement of the user's finger speed will be described with reference to FIGS. 10 and 11. FIG. 10 shows an electronic apparatus according to the first embodiment, which has a visual display unit and a tactile sense presentation unit having a diagonal size of 10.4 inches. On this electronic device, nine square buttons were displayed in the form of 3 rows and 3 columns as display objects. Tactile sensations were presented at the positions of these display objects using a tactile sensation presentation unit. This state is shown in FIG. 10A when the user is not searching.

  Using this electronic device, the user was given the task of searching (searching) for the position of the button by tactile sense, and the motion of the finger at that time was photographed as a moving image to determine the speed of the fingertip. A photograph taken during this task is shown in FIG. 10B as when the user is searching.

  FIG. 11 is a diagram illustrating a time change of the measured fingertip speed. The maximum fingertip speed measured exceeds 700 mm / sec. The user moved the scanning position in the horizontal direction gradually in the vertical direction of the screen while scanning the finger in the horizontal direction of the screen and searching. When the user's fingertip reaches the left and right edges of the screen, the fingertip speed takes a minimum value, and when the user's finger is near the center of the screen, the fingertip speed takes a maximum value.

  Based on these results, the difference between the position of the display object and the position where the finger is present when the sense of touch is generated is visualized as shown in FIG. FIG. 12A shows the position of the display object 621 when a search is performed by moving the finger in the right direction (movement direction 631) in the electronic device which has been written down by the present inventor based on the related technology and sequentially goes through steps S301 to S306. And a position where a finger is present when a tactile sensation is generated. The position of the display object 621 perceived by vision is indicated by the label “sight”. As an example of the screen, when the display object 621 is displayed as a black square and the background color is displayed as white, the pulse waveform indicated by the visual label indicates that the horizontal axis corresponds to the position and the vertical axis indicates the display object. This corresponds to a luminance of 621. The position perceived by tactile sensation is indicated by the label “tactile”. In the pulse waveform indicated by the tactile label, the horizontal axis corresponds to the position, and the vertical axis corresponds to the perceived vibration intensity. The display object 621 in this example is a button used for tap input.

  As can be understood from FIG. 12A, the position where the finger is present when the tactile sensation is shifted to the right by 38.5 mm with respect to the vision where the display object 621 is present. When the horizontal size of the display object 621 that is the target of tap input is smaller than 38.5 mm, there is a graphic operation between the two in relation to the position of the button of the tap input perceived by tactile sense and the position of the display object 621. Since there is no such product, there is a problem that even if the position of a button perceived by tactile sensation is tapped, the function of the button is not executed.

  FIG. 12B shows the position of the display object 622 when searching by moving the finger in the right direction (movement direction 631) and the left direction (movement direction 632), and the position where the finger exists when a tactile sensation occurs. It is shown relatively. The position where the tactile sensation is perceived by the finger when searching in the right direction is indicated by the label “tactile sensation (1)”. When tactile sensation is perceived in the figure, the intensity of vibration on the vertical axis increases, and when tactile sensation is not perceived, the intensity of vibration decreases. A pulse-like change is shown. Similarly, the position where the tactile sensation is perceived when searching in the left direction is indicated by the label “tactile (2)”.

  As can be understood from FIG. 12B, when a search is performed by moving the finger to the right, the position where the finger is present is shifted to the right by 38.5 mm with respect to the visual sense. On the other hand, when a search is performed by moving the finger to the left, the position where the finger exists when the tactile sensation occurs is shifted to the left by 38.5 mm. When the horizontal size of the display object 622 that is the target of tap input is smaller than 77.0 mm, the position of the tap input button perceived by searching in the right direction and the tap input perceived by searching in the left direction There is no product between the two in terms of graphic operations. That is, the position of the button perceived by searching in the right direction and the position of the button perceived by searching in the left direction are greatly different, and the user is confused and the tap input is hindered.

  As described above, when displaying a display object and presenting a tactile sensation using a related technique such as Patent Document 3, there is a time lag (delay) between the display and the tactile sensation. For this reason, as described above, when searching for the tactile sense presentation position, a large positional deviation occurs between the display position and the position where vibration is actually generated and the tactile sensation is felt. This misalignment significantly reduced usability.

  On the other hand, FIG. 13 shows a flowchart in the electronic apparatus according to the present embodiment. The electronic apparatus of the present embodiment has a structure and operation using FIGS. 1 to 8 and includes a process of sequentially performing the following steps.

The display object corresponding to the operation key is displayed on the visual display unit 20 (S311).
The tactile sense display panel (tactile sense presenting unit 40) is driven so as to present a tactile sense (texture feeling) localized at a position corresponding to the display object (S312).
The user touches the operation target range 61 (S313).

  The electronic device 100 of the present embodiment includes a tactile sense presentation unit 40 that can simultaneously present different tactile sensations on a surface. For this reason, as shown in the flowchart, the tactile sense providing unit 40 can be driven from a time point before the user touches the operation target range 61. As a result, it is possible to solve the problem of misalignment caused by the time lag, which is one of the problems of the related art.

  FIG. 14 is a diagram for explaining the difference between the position of the display object and the position of the tactile sensation in the electronic device 100 of the present embodiment, which goes through the above-described steps S311 to S313 in order. FIG. 14A is a diagram showing the position of the display object 621 and the position where a tactile sensation occurs when the finger is moved in the right direction (movement direction 631), and FIG. It is the figure which showed the position of the display object 621 at the time of moving to the left direction (movement direction 632) and the left direction (movement direction 632), and the position where a tactile sensation occurs. As in FIG. 12, in FIG. 14, the position of the display object 621 perceived by vision is indicated by a “visual” label, the horizontal axis corresponding to the position, and the vertical axis corresponding to the luminance of the display object 621. The position perceived by tactile sense is indicated by a label “tactile sense”, the horizontal axis corresponding to the position, and the vertical axis corresponding to the perceived strength of texture. In the present embodiment, since a textured texture localized at the position of the display object is presented, a pulse-like change is shown with respect to the position.

  As described above, the tactile sense presenting unit 40 of the present embodiment is configured to be able to present different tactile sensations simultaneously in the first region and the second region on the operation screen. The tactile sense providing unit 40 and the tactile sense that drives the tactile sense presenting unit 40 so that the tactile sense perceivable by the user is presented at the position of the display object before the user touches the display object. By providing the presentation control means 13, it is possible to present a tactile sensation different from others in advance at the position of the display object. For this reason, the problem of misalignment does not occur.

  As can be understood from these drawings, in the electronic device 100 of the present embodiment, there is no deviation between the position of the display object and the position where the sense of touch is generated. The reason is that the area of the display object of the tactile sense presentation unit 40 has been driven from a time before the user touches the operation target range 61. Alternatively, the comparison operation between the touch position and the position of the display object is unnecessary. Therefore, even when the user searches quickly, the position of the display object coincides with the position where the tactile sensation occurs, and the operability is remarkably improved.

  In the operation according to the present embodiment described above, the contents of the operation performed by the user on the device are all output via the sound output unit. For example, it may be inconvenient if someone asks about the details of the operation, such as when entering a PIN at an ATM such as a bank. In such a case, use an earphone to output sound. You can also prevent others from hearing you.

The electronic device of the present invention can be said to be an electronic device having the following characteristics from another aspect. That is, the electronic device of the present invention is
A tactile presentation unit that can change the tactile sensation of a partial area of the surface, in other words, a tactile presentation unit that can simultaneously present different tactile sensations on the surface;
A touch detection unit that detects a user's contact with the tactile sense presentation unit;
A tactile object presented by changing the tactile sensation of a predetermined area of the tactile sense presenting unit;
With
A tactile sense presentation control unit that drives the tactile sense presenting unit to cause the tactile object to be presented before the user touches the tactile object;
-When a predetermined touch motion is detected with respect to the tactile object, a motion associated with the tactile object is executed.
Thus, in the present invention, the visual presentation unit can be omitted. In this case, “display object” is replaced with “tactile object”.

(Second Embodiment)
In the electronic device 200 according to the second embodiment of the present invention, instead of the configuration of the first embodiment described above, the overall control unit 210 and the operation content acquisition unit 214 touch the display objects 62a, 62b,. When the position is detected, the vibration presenting control unit 213 for controlling the operation of the vibration presenting unit 240 provided in advance is provided instead of the tactile sense presenting unit so as to directly vibrate the entire operation target range 61.

According to this configuration, it is possible to obtain the same effect as that of the first embodiment at a lower cost, and it is possible to obtain the effect of existing devices only by adding and replacing software.
Hereinafter, this will be described in more detail.

  FIG. 15 is an explanatory diagram showing a configuration of an electronic device 200 according to the second embodiment of the present invention. Since the electronic device 200 includes many configurations and contents common to the electronic device 100 according to the first embodiment described above, the common points are denoted by the same names and reference numerals as those of the first embodiment. The electronic device 200 includes a visual display unit 20 that is common to the first embodiment, a vibration presentation unit 240 that is different from the first embodiment, and a touch coordinate detection unit 230 that is different from the first embodiment. The overall control unit 210 is different from that of the first embodiment.

  The touch coordinate detection unit 230 is a touch panel disposed on the surface of the visual display unit 20 that is typically a liquid crystal display device. By attaching a mechanical vibrator around the touch panel, the touch panel was given a vibration presentation function. For this reason, the electronic device 200 has a configuration in which the touch panel also serves as a tactile sense presentation unit.

  The tactile sense presenting unit 40 in the first embodiment presents a tactile sense (texture feeling) at a position corresponding to each of the specific display objects 62a, 62b,. On the other hand, the vibration presentation unit 240 directly vibrates the entire operation target range 61, that is, the entire surface that the user touches for operation, and presents a tactile sensation to the user. Specifically, it can be realized by using a piezoelectric vibrator or a small motor used in a vibrator for a mobile phone terminal.

  16 is an explanatory diagram conceptually showing a more detailed configuration of the overall control unit 210 in the electronic device 200 shown in FIG. The overall control unit 210 functions as each of the application execution unit 11, the operation screen display control unit 12, the vibration presentation control unit 213, the operation content acquisition unit 214, and the sound output control unit 15 by executing the operation control program. To do. Except for the vibration presentation control unit 213 and the operation content acquisition unit 214, they are the same as those in the first embodiment. The overall control unit 210 is also provided with a sound output unit 50 and a temporary storage unit 16 that are common to the first embodiment.

  The vibration presentation control unit 213 controls the vibration presentation unit 240 according to an operation instruction from the application execution unit 11 to vibrate the entire operation target range 61 and presents vibration to the user.

  FIG. 17 is a flowchart illustrating an operation performed by the overall control unit 210 (particularly, the operation content acquisition unit 214) in the electronic device 200 illustrated in FIGS. Since FIG. 17 also includes many operations common to the first embodiment shown in FIG. 7, the common operations are denoted by the same reference numerals. The definition of each operation of “touch down”, “touch up”, “long press”, “search”, and “tap” detected by the operation content acquisition unit 214 and the detection conditions thereof are also the same as in the first embodiment. .

  When the application execution unit 11 is in a state of accepting an input operation by the user, the operation content acquisition unit 214 performs the same operation as step S101 to step S104 in FIG.

If a touchdown occurs (Yes in Step S104), the operation content acquisition unit 214 performs the same operation as Steps S105 to S107 in FIG. Operation details obtaining means 214 transmits the touch position vector r _n touchdown has acquired the information and the step S107 that occurs in the application execution unit. Application execution means, the touch position vector _{r n} of coordinates acquired in step S107 the display object 62a, 62b, judges whether a position corresponding to ... (step S201).

_If the coordinates of rn correspond to the display object (Yes in step S201), the application execution means 11 instructs the vibration presentation control means 213 to generate vibration (step S202), and the processing after step S108 is performed. move on. Upon receiving the instruction, the vibration presentation control unit 213 vibrates the entire operation target range 61 via the vibration presentation unit 240.

_If the coordinates of rn do not correspond to the display object (No in step S201), the process proceeds to step S108 and subsequent steps. The processing after step S108 is the same as the operation of the first embodiment described in FIG.

  According to the electronic device 200 according to the present embodiment described above, the same effect as that of the electronic device 100 according to the first embodiment described above is obtained, and a tactile sensation (texture feeling) is provided at a position corresponding to the position of each display object. It can be obtained even in a device that does not include a tactile sense presentation unit to present. That is, since it is sufficient to provide a piezo vibrator or a vibrator for a mobile phone terminal, the effect of the present invention can be obtained at a lower cost. In addition, it is possible to implement the present embodiment on existing devices only by adding and replacing software and obtain the effects.

(Third embodiment)
The electronic device according to the third embodiment of the present invention is mainly different from the first embodiment described above in that the visual display unit and the tactile sense presentation unit are spatially separated. Since the visual display unit and the tactile sense presentation unit are spatially separated, there is no need to match the size of the visual object representing the operation target with the size of the tactile object, and the relative display object displayed on the visual display unit It is only necessary to control the actual position and the relative position of the tactile object presented on the tactile sense presenting unit to substantially match. This embodiment is an example in which the present invention is applied to an interface device for a driver in an automobile assuming use by a user who has no visual impairment.

  Hereinafter, with reference to FIG. 18 and FIG. 19, the interface device will be described in detail using a temperature setting operation of the air conditioner as an example.

  FIG. 18 is an explanatory diagram illustrating a configuration of an electronic device 300 according to the third embodiment, and FIG. 19 is a mobile body on which the electronic device 300 is mounted. Since the electronic device 300 includes many configurations and contents common to the electronic device 100 according to the first embodiment described above, the common points are denoted by the same names and reference numerals as those in the first embodiment.

  The electronic device 300 includes a tactile sense presentation unit 40 and a touch coordinate detection unit 30 that are common to the first embodiment in the operation target range 61. On the other hand, the visual display unit 320 is not integrated with the tactile sense presentation unit 40 or the touch coordinate detection unit 30 and is configured to be arranged at an arbitrary place. The visual display unit 320 is a projector-type display device that includes a screen transparent to visible light and a projection optical system. Since the visual display unit 320 can be arranged independently of the operation target range 61, each of the visual display unit 320 and the operation target range 61 can be arranged at a position that is most convenient for the user. As a result, an effect of improving usability can be obtained.

  As shown in FIG. 19, the visual display unit 320 is disposed on the dashboard in front of the driver's seat and functions as a head-up display for the user. On the other hand, the operation target range 61 including the tactile sensation presentation unit 40 and the touch coordinate detection unit 30 is arranged near the place where the finger of the right hand is positioned when the user places the right arm on the armrest.

  The tactile sense presenting unit 40 presents tactile sense objects 70 a and 70 b corresponding to the display objects 362 a and 362 b displayed on the visual display unit 320. The display object 362a is displayed in the lower half area and the left half area of the screen of the visual display unit 320. The corresponding haptic object 70a is displayed in the lower half region and the left half region of the operation target range 61. The display object 362b is displayed in the lower half and right half of the screen of the visual display unit 320. The corresponding haptic object 70b is displayed in the lower half and right half of the operation target range 61.

  Unlike the first embodiment, the visual display unit 320 is not integrated with the tactile sensation presentation unit 40, and thus the concept of matching the edge of the display object 362a and the edge of the tactile object 70a does not exist. Therefore, in the present embodiment, the relative positional relationship between the display objects 362a and 362b displayed on the visual display unit 320 and the relative positional relationship between the haptic objects 70a and 70b presented on the haptic presentation unit 40 are as follows. It is controlled and driven by the overall control unit 310 so as to roughly match. Alternatively, the graphic obtained by planarly viewing the visual display unit 320 and the display objects 362a and 362b and the graphic obtained by planarly viewing the operation target range 61 and the tactile objects 70a and 70b are substantially similar to each other. The overall control unit 310 controls and drives.

  The overall control unit 310 includes operation content acquisition means as in the first embodiment, but its function is different from that in the first embodiment. That is, the operation content acquisition unit according to the present embodiment is associated with the tactile objects 70a and 70b when an action (tap operation) of hitting the operation target range 61 at a position corresponding to the tactile objects 70a and 70b is detected. Control is performed to execute a predetermined function.

  In the area of the tactile objects 70a and 70b, as in the first embodiment, when a user receives an input operation, a texture is presented locally. That is, the tactile sense providing unit 40 is driven so as to present a sense of texture to the areas of the tactile objects 70a and 70b before the user touches the areas of the tactile objects 70a and 70b.

  An air conditioner 370 is connected to the overall control unit 310. The visual display unit 320 is provided with a temperature display region 362c.

  Next, a case where the user selects and inputs the display object 362a written “Down” shown in FIG. 18 will be described. In this embodiment, the user visually recognizes the visual display unit 320, recognizes the positions and functions of the display objects 362a and 362b, and operates the operation target range 61. Since the tactile objects 70a and 70b are presented in the operation target range 61 at positions corresponding to the display objects 362a and 362b, it is possible to know the position where the tap operation should be performed without viewing the operation target range 61.

  A user who intends to lower the set temperature visually recognizes that the display object 362a for lowering the set temperature is in the lower left and the display object 362b for raising the set temperature is in the lower right by visually recognizing the visual display unit 320. . Thereafter, the user searches the operation target range 61 to perceive the tactile objects 70a and 70b arranged side by side, and performs a tap operation at the position of the tactile object 70a. By this operation, the set temperature is lowered, for example, by one scale.

  In the present embodiment, the control of the set temperature of the air conditioner 370 has been described, but the present invention is not limited to this, and can be applied to operate and control various auxiliary devices.

(Effect of the third embodiment)
The effects of the electronic device 300 according to the third embodiment and a mobile object equipped with the electronic device 300 will be described. One effect is that the usability is improved by arranging the visual display unit 320 and the operation target range 61 at positions where the user can use them most easily.

  The user is obliged to watch forward while driving. In the example of the moving body shown in FIG. 19, the visual display unit 320 is provided very close to the line of sight that the user pays attention to, and the set temperature is lowered or raised without visually recognizing the operation target range. You can. Therefore, the user can operate the air conditioner 370 while devoting himself to fulfilling the duty of forward caution, and can continue safe driving.

  Another effect is that when the user searches for the haptic objects 70a and 70b by searching, the position of the haptic objects 70a and 70b perceived by the user is constant regardless of the finger speed, and the user should perform a tap operation. It is to be able to know the position correctly.

  This is because, as in the first embodiment, the haptic presentation unit 40 presents a texture to the areas of the haptic objects 70a, 70b before the user touches the areas of the haptic objects 70a, 70b. It is because it is driven to. Therefore, there is no deviation between the position of the haptic objects 70a and 70b and the position of the haptic objects 70a and 70b perceived by the user.

  On the other hand, when the related technology is applied, the touch coordinates of the user's finger are detected, and whether or not the detected coordinates are within a predetermined area (area to accept a tap operation) is compared. If it is within the region, a signal for driving the actuator is generated and the touch surface is vibrated, so that there is a time lag between the user's finger entering the region and the touch surface vibrating. For this reason, the problem which the said area | region and the position where a finger | toe exists when a tactile sense generate | occur | produces arises according to the speed of a finger | toe. That is, there arises a problem that the position of the area that accepts the tap operation perceived by the user is different from the position of the area that actually accepts the tap operation.

  The tactile sense presentation unit 40 that localizes and presents the tactile sense is not limited to the configuration described in the first embodiment, that is, the configuration in which the XY electrodes are arranged in a matrix. For example, a configuration described in Patent Document 1 in which a plurality of segment electrodes are arranged on a substrate can be used. Alternatively, a configuration described in Patent Document 5 in which a plurality of shape memory alloys are arranged on a substrate can be used. Alternatively, a configuration described in Patent Document 6 in which tactile pins are arranged in a matrix can be used. Alternatively, a configuration that can be inferred from this document, for example, a configuration in which a plurality of piezoelectric vibrators are arranged in a matrix on the surface can be used.

(Extended embodiment)
The first to third embodiments described above can be variously expanded without changing the gist thereof. This extension will be described below.

  In each of the embodiments described above, when the user performs a “tap” operation, the operation content is passed to the software operating by the application execution unit 11. In both the first and second embodiments, instead of “tapping”, a force to further push the finger that has been stationary at the same touch position by the operation of “push”, that is, “long push”, in the stationary state as it is. It can be replaced with an operation of strengthening.

  In this case, if a load sensor is further added to the operation target range (the visual display unit 20, the touch coordinate detection unit 30, and the tactile sense presentation unit 40) and it is detected that the pressing force has increased, “push” occurs. You may make it determine. Further, as a configuration in which the operation content acquisition unit 14 (or 214) can detect the area of the finger that is in contact with the operation target range, it is determined that “push” has occurred when the area is detected to have increased. You may do it. In the electronic apparatus of this example, when the operation content acquisition unit 14 detects an operation of pushing the operation screen at a position corresponding to the display object, the application execution unit 11 is used to perform a predetermined process associated with the display object. Controlled to perform function.

  Although the present invention has been described with the specific embodiments shown in the drawings, the present invention is not limited to the embodiments shown in the drawings, and as long as the effects of the present invention are obtained, Any configuration known so far can be employed.

  The present invention is applicable to all electronic devices equipped with a touch panel. Used by visually impaired people not only on ticket machines, ATMs and multimedia kiosk terminal devices as mentioned above, but also on smartphones, tablet terminals, notebook personal computers, etc. that are assumed to be owned and used by individuals Since measures for making it easy to perform may be required, it is meaningful to apply the present invention.

100, 200, 300 Electronic device 10, 210, 310 Overall control unit 11 Application execution means 12 Operation screen display control means 13 Tactile sense presentation control means 14, 214 Operation content acquisition means 15 Sound output control means 16 Temporary storage means 20, 320 Visual Display unit 30, 230 Touch coordinate detection unit 40 Tactile sense presentation unit 41 Support substrate 42 X electrode 43 Y electrode 44 X electrode drive circuit 45 Y electrode drive circuit 46 Control unit 47 Target region 48 Electrode 49 Resistance 50 Sound output unit 61 Operation target range 62a, 62b, 362a, 362b Display object 362c Temperature display area 63 Finger 70a, 70b Tactile object 213 Vibration presentation control means 240 Vibration presentation unit 370 Air conditioner 621, 622 Display object 631, 632 Movement direction

Claims (18)

A visual display unit that displays an operation screen including an operation target range by the user, a touch coordinate detection unit that detects the content of the touch operation performed by the user on the display object displayed in the operation target range, An electronic device including a tactile sensation providing unit that presents a tactile sensation perceivable by the user on the operation screen,
Tactile sense presentation control means for presenting a tactile sense corresponding to the display object on the operation screen to the user via the tactile sense providing unit;
Operation content acquisition means for detecting whether a temporal change in a touch position detected on the display object by the touch coordinate detection unit is equal to or less than a preset threshold and the state continues for a preset time;
Sensory stimulus information output control means for externally outputting sensory stimulus information corresponding to the display object via a sensory stimulus information output unit provided in advance when the state continues for the preset time or longer, Electronic equipment characterized by   The electronic device according to claim 1, wherein the sensory stimulus information is a sound.   The electronic device according to claim 1, wherein the tactile sense providing unit is configured to be able to present different tactile sensations simultaneously in the first area and the second area on the operation screen.   Based on the display position of the display object, a tactile sensation that drives the tactile sensation providing unit to present a tactile sensation perceived by the user at a time before the user performs a touch operation on the position. The electronic apparatus according to claim 3, further comprising a presentation control unit. The tactile sense presentation unit
A support substrate; a plurality of parallel X electrodes extending in a first direction on the support substrate; and extending in a second direction on the support substrate and insulated from the X electrode. A plurality of parallel Y electrodes,
A voltage signal of a first frequency is applied to an X electrode corresponding to a target region input from the outside among the X electrodes, and a second one is applied to a Y electrode corresponding to the target region among the Y electrodes. A drive circuit for applying a voltage signal having a frequency of 2;
The first and second frequencies are both 500 [Hz] or higher,
5. The electronic device according to claim 4, wherein the electronic device has a period in which an absolute value of a difference between the first and second frequencies is greater than 10 [Hz] and less than 1000 [Hz].   When the operation content acquisition unit detects the touch position on the display object, an operation of the vibration presenting unit provided in advance instead of the tactile sense presenting unit is performed so as to directly vibrate the entire operation target range. The electronic device according to claim 1, further comprising vibration presentation control means for controlling. The operation content acquisition means is
2. The control according to claim 1, wherein when a motion of hitting the operation screen at a position corresponding to the display object is detected, a predetermined function associated with the display object is controlled. The electronic device described. The operation content acquisition means is
2. The apparatus according to claim 1, wherein when an operation of pushing the operation screen at a position corresponding to the display object is detected, control is performed to execute a predetermined function associated with the display object. The electronic device described. A tactile sense presentation unit capable of simultaneously presenting different tactile sensations on the surface;
A touch detection unit for detecting a user's contact with the tactile sense presentation unit;
A tactile object presented by changing the tactile sensation of a predetermined area of the tactile sense presentation unit;
Tactile sense presentation control means for driving the tactile sense presenting unit to cause the tactile sense object to be presented at a time prior to the user touching the tactile sense object,
An electronic device characterized in that when a predetermined touch motion is detected with respect to the tactile object, a motion associated with the tactile object is executed. A visual display unit spatially separated from the tactile sense presentation unit;
A control unit that controls the relative position of the display object displayed on the visual display unit and the relative position of the tactile object presented on the tactile presentation unit to substantially match;
The electronic apparatus according to claim 9, further comprising: A method of operating and controlling an electronic device including a visual display unit that displays an operation screen including an operation target range by a user,
The operation screen display control means creates and displays the operation screen, which is the content obtained by executing the process by the application execution means, on the visual display unit,
The tactile sensation presentation control means presents a tactile sense corresponding to the operation screen perceivable by the user within the operation target range on the operation screen via the tactile sense presenting unit,
The touch coordinate detection unit detects the content of the touch operation performed by the user on the display object displayed in the operation target range,
The operation content acquisition means detects whether the temporal change of the touch position detected on the display object is less than or equal to a preset threshold and the state has continued for a preset time or more,
When the state continues for the preset time or longer, sensory stimulus information corresponding to the display object is externally output via a sensory stimulus information output unit provided in advance with a sensory stimulus information output control means. Operation control method.   The operation control method according to claim 11, wherein the sensory stimulus information is a sound. The step of presenting a tactile sensation by the tactile sensation providing unit includes:
The operation control according to claim 11, wherein, based on a display position of the display object on the operation screen, the tactile sense providing unit presents a tactile sense perceivable by the user at the position. Method. Detecting whether or not the operation content acquisition means has hit the operation screen at a position corresponding to the display object;
The operation control method according to claim 11, wherein when the hitting motion is detected, the operation content acquisition unit causes the application execution unit to execute the operation content corresponding to the display object. A visual display unit that displays an operation screen including an operation target range by the user, a touch coordinate detection unit that detects the content of the touch operation performed by the user on the display object displayed in the operation target range, An electronic device including a tactile sensation providing unit that presents a tactile sensation perceivable by the user on the operation screen,
In the processor provided in the electronic device,
A procedure for creating the operation screen, which is a content obtained by executing the process, and displaying the operation screen on the visual display unit;
A procedure for presenting a tactile sense corresponding to the operation screen perceivable by the user within the operation target range on the operation screen via the tactile sense providing unit;
A procedure for detecting the content of a touch operation performed by the user on the display object displayed in the operation target range;
A procedure for detecting whether a temporal change in the touch position detected on the display object is equal to or less than a preset threshold and whether the state has continued for a preset time;
And, when the state continues for the preset time or longer, executes a procedure of outputting sensory stimulus information corresponding to the display object to the outside via a sensory stimulus information output unit provided in advance with sensory stimulus information output control means An operation control program.   The operation control program according to claim 15, wherein the sensory stimulus information is a sound. The procedure for presenting a tactile sensation to the tactile sense presenting unit is as follows:
The operation control program according to claim 15, wherein a tactile sensation perceived by the user is presented at a position corresponding to the display object on the operation screen. Detecting whether or not there was an action of hitting the operation screen at a position corresponding to the display object,
The operation control program according to claim 15, further comprising executing a procedure for executing the operation content corresponding to the display object when the hitting operation is detected.

Images