JP7317494B2 - ELECTRONIC DEVICE AND METHOD OF CONTROLLING ELECTRONIC DEVICE - Google Patents

Description

TECHNICAL FIELD This disclosure relates to electronic devices that provide feedback regarding objects included in a user interface.

In recent years, electronic devices equipped with touch panels, such as smart phones and car navigation systems, have become widespread.

When a user operates an object such as an icon included in a user interface displayed via a touch panel, the electronic device activates a function corresponding to the object.

Since the surface of the touch panel is uniformly hard, the user has the same tactile sensation regardless of which part of the touch panel is touched. Therefore, it is desirable to provide feedback to the user that perceives the presence of the object without activating the function corresponding to the object. Also, when a function corresponding to an object is activated, it is desirable to provide the user with feedback that allows the user to perceive that an operation has been received to activate the function.

A technique described in Patent Document 1 is known as a technique for solving the above problem. US Pat. , is disclosed to provide haptic feedback 302 . (2) User 2 then presses GUI object 310 harder to increase the pressure value above actuation threshold 321 . It is described that GUI object 310 is thereby actuated and that haptic feedback 304 and visual feedback 305 are provided to notify user 2 that GUI object 310 has been selected and actuated.

JP-A-2008-33739 JP 2016-95832 A

The technique described in Patent Document 1 has the following problems.

(Problem 1) In the device described in Patent Document 1, the feedback for perceiving the existence of an object and the feedback for perceiving that an object has been selected are provided as mechanical vibrations. Therefore, when the user presses the object after the vibration that notifies the presence of the object is provided, there is a possibility that the pressing of the object cannot be detected due to the vibration mixed as noise.

In order to eliminate the mixing of noise described above, it is conceivable to reduce mechanical vibration. However, if the mechanical vibration is reduced, the user may not be able to clearly perceive the presence of the button.

(Problem 2) The device described in Patent Document 1 detects the position of the user's finger on the touch panel, and provides feedback when it is confirmed that the finger is placed on the object. When the finger moves fast, the display position of the object and the position of the object perceived by the feedback are deviated. This is because a delay occurs due to the execution of processing for position detection.

FIG. 9 is a diagram showing the deviation between the display position of an object 901 on a conventional touch panel and the position of the object 901 perceived by feedback. Here, it is assumed that the finger movement speed takes a minimum value at the edge of the screen and a maximum value at the center of the object 901 . The local maximum shall be greater than 700 mm/s. Also, the size of the object 901 is assumed to be 40.0 mm.

In the graph shown in FIG. 9 , the display position of the object 901 perceived by visual feedback is indicated by the “visual” label, and the position of the object 901 perceived by haptic feedback is indicated by the “tactile” label. The pulse waveform labeled "visual" represents the brightness of the object 901 at the position on the display screen. The pulse waveform labeled "tactile" represents the strength of the stimulus (vibration) at the position on the display screen.

When the moving speed of the finger when passing the object 901 is the maximum value of 700 mm/sec, as shown in FIG. A gap of 38.5 mm occurred between the positions.

Due to this deviation, even if the user who receives the haptic feedback performs an operation such as pressing, the pressing operation may be performed outside the area of the object 901 . As a result, there is a problem that the pressing operation by the user is not recognized as an operation on the object 901 .

The present disclosure provides an electronic device and a control method thereof for solving the above problems.

A representative example of the invention disclosed in the present application is as follows. An electronic device comprising a display screen for displaying a user interface including at least one object, the display mechanism for displaying the user interface, a coordinate detection mechanism for detecting the coordinates of a user's contact point on the display screen, and any a first feedback presentation mechanism that presents a local tactile sensation for causing the user to perceive the object of the first feedback presentation mechanism, a measurement mechanism that measures the force caused by the user's pressing operation on the display screen, and an actuation opportunity that is detected based on the force a second feedback presentation mechanism that generates mechanical vibration; and an update mechanism that updates a baseline that is a reference value of the force, wherein the update mechanism updates the force value measured by the measurement mechanism. A force value stored in a buffer and stored in the buffer when the user's contact with the display mechanism is detected by the coordinate detection mechanism, or when the user's pressing operation is detected by the measurement mechanism. is discarded, and if the accumulated number of force values stored in the buffer is greater than a certain number, then the force value is used to update the baseline.

According to the present disclosure, it is possible to realize an electronic device having a feedback providing means capable of solving (Problem 1) and (Problem 2).

It is a figure showing an example of composition of electronic equipment of a 1st embodiment. 4A and 4B are diagrams for explaining the structure and operation of the electrostatic touch panel of the first embodiment; FIG. 4A and 4B are diagrams for explaining the structure and operation of the electrostatic touch panel of the first embodiment; FIG. 4 is a flowchart illustrating processing executed by the electronic device of the first embodiment; 7 is a flowchart for explaining touch detection processing executed by a press detection unit according to the first embodiment; 9 is a flowchart for explaining press detection processing executed by a press detection unit according to the first embodiment; FIG. 4 is a diagram showing an example of presentation of tactile sensation accompanying operation of the display screen of the electronic device of the first embodiment; FIG. 4 is a diagram showing a deviation between the display position of an object and the position of the object perceived by feedback in the electronic device of the first embodiment; It is a figure which shows an example of a structure of the tactile sense presentation apparatus of 2nd Embodiment. It is a figure which shows an example of a structure of the tactile sense presentation apparatus of 2nd Embodiment. FIG. 14 is a flowchart for explaining baseline update processing executed by the control device of the fourth embodiment; FIG. FIG. 10 is a diagram showing the deviation between the display position of an object on a conventional touch panel and the position of the object perceived by feedback.

Embodiments of the present disclosure will be described below with reference to the drawings. In the configurations of the invention described below, the same or similar configurations or functions are denoted by the same reference numerals, and overlapping descriptions are omitted.

(First embodiment)
FIG. 1 is a diagram showing an example of the configuration of an electronic device 10 according to the first embodiment.

The electronic device 10 includes a tactile presentation device 100 , a control device 110 and a storage device 120 . The tactile sense presentation device 100 and the control device 110 are connected via a connection line, and the control device 110 and the storage device 120 are connected to each other via a connection line.

The tactile presentation device 100 presents a user with a UI (user interface) including at least one object, and receives an operation via the UI. The tactile sense presentation device 100 also provides feedback for perceiving an object included in the UI and feedback for perceiving that an operation on the object has been received.

A tactile sense presentation device 100 is composed of an electrostatic tactile panel/touch panel 101 , a force sensor 102 , a liquid crystal display (LCD) 103 , a carrier 104 , a base 105 , a lateral actuator 106 and a leaf spring 107 . Note that each component of the tactile sense presentation device 100 is stored in an arbitrary housing. The electrostatic tactile panel/touch panel 101, the force sensor 102, and the liquid crystal display 103 are configured to realize a display screen for displaying the UI, and the carrier 104, the lateral actuator 106, and the leaf spring 107 are configured to realize mechanical vibration. be.

The base 105 is a base member of the tactile presentation device 100 . A lateral actuator 106 and a leaf spring 107 are installed on the base 105 . Further, on the base 105, a carrier 104 that vibrates due to the action of a lateral actuator 106 and leaf springs 107 is provided. Carrier 104 vibrates in a particular direction with respect to base 105 . Vibration of the carrier 104 realized by the lateral actuator 106 and leaf spring 107 is also referred to as mechanical vibration.

The lateral actuator 106 is a device that generates movement in a direction parallel to the display screen (eg, x-axis direction or y-axis direction). The leaf spring 107 is used as a mechanism for generating vibrations in accordance with the movement of the lateral actuator 106 .

The carrier 104 is a member serving as a base on which the members forming the display screen are laminated. A liquid crystal display 103 , a force sensor 102 , and an electrostatic tactile panel/touch panel 101 are provided on the carrier 104 .

In the first embodiment, it is configured to vibrate in a direction substantially parallel to the horizontal direction of the display screen of the display device.

The liquid crystal display 103 , the force sensor 102 , and the electrostatic tactile panel/touch panel 101 are installed so as to be substantially parallel to the base 105 .

The electrostatic tactile panel/touch panel 101 is composed of an electrostatic tactile panel 200 (see FIG. 2) and a touch panel. The capacitive tactile panel 200 uses electrostatic force to give the texture of an object when the user drags the UI. The touch panel detects the position of the user's finger on the display screen. The structure and operation of the electrostatic touch panel 200 will be described with reference to FIGS. 2A and 2B.

The force sensor 102 measures the force applied by the user's finger to the display screen. In the first embodiment, the force in the direction substantially parallel to the vertical direction of the display screen is measured. That is, the force sensor 102 measures force in the pressing direction. The liquid crystal display 103 displays images corresponding to the UI. The force sensor 102 outputs the difference (relative value) between the baseline, which is the force reference value (zero point), and the measured value (measurement value). In the following description, "force measurement" means calculating a relative value, and "measured force value" means the aforementioned relative value.

The control device 110 is an arithmetic device that executes programs. Arithmetic devices are processors, GPUs (Graphics Processing Units), FPGAs (Field Programmable Gate Arrays), and the like. Storage device 120 stores programs and data used by control device 110 . Storage device 120 is, for example, a memory. Storage device 120 contains a work area used by the program. Note that the program may be installed in the electronic device 10 in advance, or may be installed from a storage medium in which the program is recorded.

The control device 110 operates as a functional unit (module) for controlling the tactile presentation device 100 . Specifically, the control device 110 operates as a display control unit 111 , a layout analysis unit 112 , an electrostatic tactile control unit 113 , a press detection unit 114 and a mechanical vibration tactile control unit 115 .

The display control unit 111 controls display of the UI on the display screen. Specifically, the display control unit 111 acquires UI setting information from the storage device 120, and controls the liquid crystal display 103 based on the information so that a UI including at least one object is displayed. The UI setting information includes object layouts, object attributes, and the like. Also, the attributes of the object include information indicating whether or not to provide feedback.

The layout analysis unit 112 analyzes the setting information of the UI and identifies the layout of objects included in the UI and the attributes of the objects. The layout analysis unit 112 outputs the layout of the objects and the attributes of the objects to the electrostatic haptic control unit 113 .

The electrostatic tactile sense control unit 113 controls the electrostatic tactile sense panel/touch panel 101 so as to give a texture feeling to the position of a specific object on the display screen.

The press detection unit 114 detects contact of the user's finger on the display screen based on the output from the electrostatic tactile panel/touch panel 101, and also specifies the position (contact position) of the user's finger on the display screen. . The press detection unit 114 also detects an object press event based on the force measured by the force sensor 102, the position of the user's finger on the display screen, and UI setting information. An object pressing event indicates an operation of pressing an object by the user. When the press detection unit 114 detects an object press event, the press detection unit 114 outputs a mechanical vibration generation request to the mechanical vibration type tactile sense control unit 115 so that the user perceives that the operation of the object has been received.

The mechanical vibration type haptic controller 115 controls the lateral actuator 106 to generate mechanical vibrations.

As for each functional unit of the control device 110, a plurality of functional units may be combined into one functional unit, or one functional unit may be divided into a plurality of functional units for each function.

The electronic device 10 of the first embodiment includes a display mechanism that displays a UI, a first feedback presentation mechanism that presents a local tactile sensation for causing the user to perceive an object, and a user on the display screen, which are realized by the above configuration. A coordinate detection mechanism that detects the coordinates of the contact point, a measurement mechanism that measures the force of the user's pressing operation on the display screen, and a second feedback presentation that generates mechanical vibrations to make the user perceive that the operation on the object has been received. It is characterized by having a mechanism.

A configuration that realizes the display mechanism is, for example, a liquid crystal display (LCD) 103 and a display control unit 111 . A configuration that realizes the first feedback presentation mechanism is, for example, the electrostatic tactile panel/touch panel 101 and the electrostatic tactile control unit 113 . A configuration that realizes the coordinate detection mechanism is, for example, the electrostatic tactile panel/touch panel 101 and the press detection unit 114 . A configuration that realizes the measurement mechanism is, for example, the force sensor 102 . A configuration that realizes the second feedback presentation mechanism is, for example, the carrier 104, the base 105, the lateral actuator 106, the leaf spring 107, and the mechanical vibration type tactile control section 115. FIG.

2A and 2B are diagrams for explaining the structure and operation of the electrostatic touch panel 200 of the first embodiment. In a first embodiment, the electrostatic tactile panel 200 described in Patent Document 2 is used as a device that provides feedback.

The electrostatic touch panel 200 is composed of a substrate on which a plurality of electrodes 210 and 211 are patterned, and a circuit (not shown) that supplies voltage to each of the plurality of electrodes 210 and 211 . The substrate comprises two layers, an insulating layer 201 and an electrode layer 202 .

Electrodes 210 and electrodes 211 are arranged in a pattern orthogonal to each other. Further, the intersecting portions of the electrodes 210 and 211 intersect with an insulating film interposed therebetween. An insulating film is formed on the electrodes 210 and 211 . This electrically insulates between the finger and the electrodes 210 and 211 when the user touches the display screen.

Here, the operation of the electrostatic tactile panel 200 when giving a texture feeling to the object 220 displayed at the position shown in FIG. 2A on the liquid crystal display 103 will be described.

The electrostatic tactile sense controller 113 identifies the electrodes 210 and 211 on which the object 220 overlaps, and outputs control signals for applying voltage to the identified electrodes 210 and 211 to the electrostatic tactile panel 200 . The circuit of the electrostatic tactile panel 200 outputs voltage signals of required frequencies to the specified electrodes 210 and 211 based on the control signal. This makes it possible to give a texture feeling to a portion of the object 220, that is, to a local area of the display screen. In this way, since the texture can be presented locally, when the UI is dragged with a plurality of fingers at the same time, the texture is presented to the finger dragging the object 220, and the texture is presented to the finger dragging other than the object 220. does not present texture. Also, when a plurality of users drag the UI at the same time, the user whose finger is dragging the object 220 is presented with a texture feeling, and the user whose finger is dragging something other than the object 220 is not presented with a texture feeling.

The electrodes 210 and 211 to which no voltage is applied are grounded to prevent voltage from being induced by capacitive coupling between the electrodes. A DC voltage or a voltage of a specific frequency or higher may be applied instead of grounding.

The texture given by applying voltages to the electrodes 210 and 211 on which the object 220 overlaps will be described with reference to FIG. 2B.

When a user operates the electrostatic tactile panel/touch panel 101 located on the top layer of the display screen, a frictional force Fr is generated in the horizontal direction of the display screen, and a force Fn is generated in the vertical direction of the display screen.

When the user performs an operation of tracing the object 220 with a finger (drag operation), since the frictional force on the object 220 is different from that on other parts, the texture of the object 220 can be given to the user. For a drag operation, the force Fn is sufficiently small.

When the finger presses the object 220 on the electrostatic tactile panel/touch panel 101, the frictional force Fr decreases and the force Fn increases.

Note that the plurality of electrodes 210 and the plurality of electrodes 211 may be used to detect contact with an object such as a finger. In this case, by time-division or space-division of the plurality of electrodes 210 and the plurality of electrodes 211, the electrodes for presenting the electrostatic tactile sensation and the electrodes for detecting the contact of the object can be separated. Thereby, the electrostatic tactile panel/touch panel 101 in which the electrostatic tactile panel and the touch panel are integrated can be realized. The press detection unit 114 can detect finger contact and identify the contact position based on the capacitance generated between the electrodes 210 and 211 by the operation on the touch panel.

By attaching an optical touch sensor to the electrostatic tactile panel, the electrostatic tactile panel/touch panel 101 in which the electrostatic tactile panel and the touch panel are integrated can be realized.

Next, processing executed by the electronic device 10 will be described. FIG. 3 is a flow chart illustrating processing executed by the electronic device 10 of the first embodiment.

The display control unit 111 selects a UI to be displayed on the display screen based on an operation state or the like, and updates the UI by controlling the liquid crystal display 103 based on setting information of the selected UI (step S101).

Next, the layout analysis unit 112 acquires the setting information of the selected UI from the storage device 120 (step S102), and analyzes the layout of the objects in the UI (step S103).

Next, the layout analysis unit 112 selects electrostatic electrode pattern data for applying voltages to the electrodes 210 and 211 of the electrostatic touch panel 200 based on the analysis results (step S104). The electrostatic electrode pattern data defines the voltage magnitude and frequency. The layout analysis unit 112 outputs the electrostatic electrode pattern data to the electrostatic touch control unit 113 together with the analysis result.

Based on the analysis result and the electrostatic electrode pattern data, the electrostatic tactile controller 113 presents the electrostatic tactile sensation by applying a voltage to the predetermined electrodes 210 and 211 of the electrostatic tactile panel 200 (step S105).

Thereafter, the press detection unit 114 starts detection processing (step S106). In the detection process, a touch detection process for detecting contact of the user's finger on the electrostatic tactile panel/touch panel 101 and a press detection process for detecting a pressing operation on the electrostatic tactile panel/touch panel 101 are executed.

The display control unit 111 returns to step S<b>101 when it detects that the UI switching timing has occurred due to the user's operation or the like.

FIG. 4A is a flowchart illustrating touch detection processing executed by the press detection unit 114 according to the first embodiment.

The press detection unit 114 detects a signal indicating contact of the user's finger on the electrostatic tactile panel/touch panel 101 (step S201). The timing of detecting the signal indicating the touch of the user's finger is arbitrarily set.

For example, the press detection unit 114 detects a change in capacitance of the electrostatic tactile panel/touch panel 101 as a signal. A change in the value of the force sensor 102 may be detected as a signal.

Next, the press detection unit 114 calculates coordinates indicating the contact position of the user's finger on the electrostatic tactile panel/touch panel 101 (step S202). For example, the coordinates are calculated based on the positions of the electrodes 210 and 211 where changes in capacitance are detected.

Next, the press detection unit 114 stores the calculated coordinates in the storage device 120 (step S203). Thereafter, the press detection unit 114 returns to step S201.

The previously calculated coordinates may be deleted by overwriting the coordinates calculated this time, or may be managed as history data.

FIG. 4B is a flowchart for explaining press detection processing executed by the press detection unit 114 according to the first embodiment.

The press detection unit 114 acquires the force value measured by the force sensor 102 (step S211). The timing at which the force sensor 102 measures force can be set arbitrarily. The execution cycles of the touch detection process and the press detection process may be the same or different.

At this time, the press detection unit 114 acquires the coordinates of the contact position (press position) from the storage device 120 .

The pressing detection unit 114 determines whether or not an object pressing event has occurred based on the comparison result of the force value and the threshold value and the coordinates of the contact position (step S212). For example, the press detection unit 114 determines whether the force value is greater than a threshold and the coordinates of the contact position match the coordinates of the object. If the above condition is satisfied, the press detection unit 114 determines that an object press event has occurred. Determination processing using only force values may provide feedback due to misdetection of press events caused by external vibrations or the like. In the present disclosure, erroneous detection of a press event can be suppressed by executing determination processing using the contact state and the force value.

If it is determined that an object pressing event has not occurred, the pressing detection unit 114 returns to step S211.

When it is determined that an object pressing event has occurred, the pressing detection unit 114 instructs the mechanical vibration type tactile sense control unit 115 to provide feedback by mechanical vibration (step S213). Thereafter, the press detection unit 114 returns to step S211.

When the mechanical vibration type tactile sense control unit 115 receives an instruction from the press detection unit 114 , the mechanical vibration is generated by controlling the lateral actuator 106 . This causes carrier 104 to vibrate in a particular direction.

Note that the touch detection process and the press detection process are executed in parallel as separate threads. Generally, on the electrostatic tactile panel/touch panel 101, a drag operation is performed more frequently than a press operation. Therefore, when the touch detection process and the press detection process are executed in the same thread, the touch detection process cannot be executed until the press detection process ends. Therefore, operational efficiency is reduced. However, by running the two sensing processes as separate threads, operational efficiency can be improved.

FIG. 5 is a diagram showing an example of presentation of tactile sensation accompanying operation of the display screen of the electronic device 10 of the first embodiment.

As shown in FIG. 5, two objects 220 are displayed on the electrostatic tactile panel/touch panel 101 located in the upper layer of the display screen. It is assumed that the user's finger transitions from state (A) to state (D) as shown.

State (A) indicates a state in which the user starts a drag operation by tracing the electrostatic tactile panel/touch panel 101 with a finger in order to search for the object 220 displayed on the display screen. At this point, the electrostatic tactile sense is presented to each object 220 by the electrostatic tactile sense control unit 113 . Therefore, when the user's finger passes over the object 220, the user can perceive the presence of the object 220 as texture.

State (B) shows a state in which the finger is moved to a specific object 220 by a drag operation. State (C) indicates a state in which the user has pressed the object 220 . Note that in the state (C), the finger is in a stopped state, so the stimulus due to the electrostatic tactile sensation is small. In this case, the press detection unit 114 detects the occurrence of an object press event.

A state (D) indicates a state in which a mechanical vibration is generated to make the user perceive that the operation of the object 220 has been accepted in accordance with the detection of the occurrence of the pressing event.

With the UI of the present disclosure, the user can perform a series of operations from searching for an object to pressing it without removing the finger from the display screen. That is, a seamless UI can be provided.

Next, effects of the electronic device 10 described in the first embodiment will be described.

(Effect 1) The user can perceive the position of the object 220 by being provided with electrostatic tactile feedback. Therefore, the user can grasp the position of the object 220 without relying on vision.

(Effect 2) The electrostatic tactile sensation for perceiving the position of the object 220 is presented in a state localized at the position of the object 220 before the user's operation. FIG. 6 is a diagram showing the deviation between the display position of the object 220 and the position of the object 220 perceived by feedback in the electronic device 10 of the first embodiment. As shown in FIG. 6, in the electronic device 10 of the first embodiment, no deviation occurs between the position of the object 220 perceived by visual feedback and the position of the object 220 perceived by haptic feedback. Therefore, (problem 2) can be solved.

(Effect 3) The electronic device 10 presents an electrostatic tactile sensation for perceiving the object 220 and presents a tactile sensation due to mechanical vibration for perceiving manipulation of the object 220 . Since the two haptic sensations are provided by different mechanisms, the user can easily distinguish between the two haptic sensations.

(Effect 4) Since the mechanical vibration vibrates in a direction substantially parallel to the horizontal direction of the display screen, it is possible to reduce the influence on the measurement of force in the vertical direction. As a result, the pressing event of the object 220 can be appropriately detected while the object 220 is being perceived. Therefore, (Problem 1) can be solved.

(Effect 5) Large electric power is required to generate mechanical vibration. Therefore, conventional devices have the problem that the power required to provide feedback is large. On the other hand, since the electronic device 10 of the present disclosure also uses electrostatic tactile feedback, power consumption can be reduced as compared with conventional devices.

(Effect 6) In the conventional device, mechanical vibration was used for both the feedback for perceiving the position of the object 220 and the feedback for perceiving that the operation of the object 220 was accepted. Mechanical vibrations are used only for feedback to perceive that the operation of 220 has been accepted. Therefore, there is an effect that the life of the structure for generating mechanical vibration is extended.

(Second embodiment)
In the second embodiment, the configuration of the tactile presentation device 100 is different. While the display-type force sensor 102 was used in the first embodiment, a small force sensor is used in the second embodiment.

7A and 7B are diagrams showing an example of the configuration of the tactile presentation device 100 of the second embodiment.

In FIGS. 7A and 7B, it is assumed that the liquid crystal display 103 is laminated on the electrostatic tactile panel/touch panel 101 .

A plate spring 107 is installed on the base 105 so that the carrier 104 vibrates in a direction substantially parallel to the horizontal direction of the display screen. In addition, a lateral actuator 106 is installed on the base 105 so that the carrier 104 applies a force in a direction substantially parallel to the horizontal direction of the display screen.

Beams (beams) 108 are installed on the carrier 104 in a cantilever configuration so as to be substantially parallel to the horizontal direction of the display screen, strain gauges are installed at predetermined positions of the beams 108, and these are used as components. A small force sensor 102 is installed. The beam 108 bends in the vertical direction of the display screen as the user presses the display screen.

The frame on which the electrostatic tactile panel/touch panel 101 is installed is provided with pillars 109 for fixing to the carrier 104 .

As shown in FIG. 7B, the carrier 104 vibrates in a direction substantially parallel to the horizontal direction of the display screen by the lateral actuator 106 and the leaf spring 107 .

Note that the control method of the electronic device 10 of the second embodiment is the same as the control method of the electronic device 10 of the first embodiment, so the description is omitted.

By configuring the tactile sense presentation device 100 using the small force sensor 102 instead of the display-type force sensor 102, the display quality of the UI displayed on the display screen can be improved.

7A and 7B, the vibration generated by the lateral actuator 106 is efficiently transmitted to the electrostatic tactile panel/touch panel 101, so that the electrostatic tactile panel/touch panel 101 can generate a large acceleration. can.

The structural features are as follows. The lateral actuator 106 is strongly coupled to the base 105, the linkage is strongly coupled to the lateral actuator 106, and the linkage and the carrier 104 are strongly coupled. Here, "strongly coupled" means that the coupling portion is coupled without play in the vibration direction. In this embodiment, each element is strongly fixed to each other with screws. Further, the carrier 104 and the pedestal 130 are strongly coupled, the pedestal 130 and the beam 108 are strongly coupled, the beam 108 and the pillar 109 are strongly coupled, and the pillar and the capacitive touch panel/touch panel 101 are strongly coupled.

As described above, since the portion from the lateral actuator 106 to the electrostatic touch panel/touch panel 101 is coupled without play in the vibration direction, the vibration generated by the lateral actuator 106 can be efficiently generated without attenuation. It becomes possible to transmit to the tactile panel/touch panel 101 .

(Third embodiment)
The third embodiment differs from the first embodiment in the method of providing feedback for perceiving the position of an object. Specifically, the electrostatic touch panel 200 is changed to another member.

One possibility is to employ a temperature display instead of the electrostatic tactile panel 200 . A temperature display is a display that can change the temperature of a surface. An arrangement of small temperature displays to form a display screen is conceivable as a configuration for providing perceptual feedback on the position of an object. By controlling the temperature of the temperature display corresponding to the display position of the object to be different than the temperature of other temperature displays, it is possible to provide perceptual feedback of the position of the object.

One candidate is to replace the capacitive tactile panel 200 with a display that uses magneto-rheological and electromagnet arrays to actuate fluids to provide feedback at any location. By providing controls that affect the local viscosity of the fluid corresponding to the display position of the object, feedback can be provided to perceive the position of the object.

As one candidate, instead of the electrostatic touch panel 200, a display having an uneven surface is adopted. In this display, a transparent polymer layer is placed on a layer having micropores, and irregularities are formed on the surface by injecting a fluid through the micropores. By controlling the injection of fluid into the pores corresponding to the displayed position of the object, it is possible to provide perceptual feedback of the position of the object.

Note that the control method of the electronic device 10 of the third embodiment is the same as the control method of the electronic device 10 of the first embodiment except for control processing for providing feedback for perceiving the object 220. is omitted.

(Fourth embodiment)
In the fourth embodiment, controller 110 updates the baseline. Since the magnitude of the force applied to the force sensor 102 changes depending on the environment such as temperature and atmospheric pressure, the frequency of use of the electronic device 10, etc., it is necessary to periodically update the baseline.

In order to update the baseline, control device 110 needs to accumulate a certain number of values (measured values) measured by force sensor 102 when no force is applied to force sensor 102 . Therefore, the control device 110 needs to correctly determine the state in which no force is applied to the force sensor 102 and accumulate the measured values. If the baseline is updated using the measurement value measured while force is applied to the force sensor 102, it is not possible to correctly detect a pressing event or the like.

FIG. 8 is a flowchart illustrating baseline update processing executed by the control device 110 of the fourth embodiment.

Here, it is assumed that the press detection unit 114 executes baseline update processing. Note that the control device 110 may be configured to include a baseline update unit that executes baseline update processing.

Note that the execution cycle of the baseline update process may be the same as or different from the execution cycle of the detection process.

The press detection unit 114 acquires the measurement value of the force sensor 102 and stores it in the buffer (step S301). It is assumed that the buffer is included in the storage device 120 .

The press detection unit 114 determines whether or not the electrostatic tactile panel/touch panel 101 is pressed based on the measured force value (step S302). For example, the press detection unit 114 determines whether the force value is greater than a threshold. The threshold may be the same as the threshold used in step S212, or may be different.

When it is determined that the electrostatic tactile panel/touch panel 101 is pressed, the press detection unit 114 clears the buffer (step S304). Thereafter, the press detection unit 114 returns to step S301.

When it is determined that the electrostatic tactile panel/touch panel 101 is not pressed, the press detection unit 114 determines whether or not the user's finger is in contact with the electrostatic tactile panel/touch panel 101 (step S303). For example, the press detection unit 114 determines whether or not a signal is detected. Further, the press detection unit 114 may execute touch detection processing and make a determination based on the execution result.

When it is determined that the user's finger is in contact with the electrostatic tactile panel/touch panel 101, the press detection unit 114 clears the buffer (step S304). Thereafter, the press detection unit 114 returns to step S301.

When it is determined that the user's finger is not in contact with the electrostatic tactile panel/touch panel 101, the press detection unit 114 determines whether or not the number of accumulated measurement values is greater than the threshold (step S305). . That is, it is determined whether or not the number of measurement values required to update the baseline has been accumulated.

If it is determined that the number of accumulated measurement values is equal to or less than the threshold, the press detection unit 114 returns to step S301.

If it is determined that the number of accumulated measurement values is greater than the threshold, the press detection unit 114 calculates a baseline using the measurement values accumulated in the buffer, and sets the calculated baseline (step S306). Thereafter, the press detection unit 114 returns to step S301. For example, the press detection unit 114 calculates the average value of the measured values as the baseline.

Through the above processing, the update timing of the baseline can be appropriately controlled. This makes it possible to correctly detect a press event or the like.

Although the embodiments of the present application have been described above, the present disclosure is not limited to the above embodiments. A person skilled in the art can easily change, add, or convert each element of the above-described embodiments within the scope of the present disclosure. A part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment.

Further, each of the configurations, functions, processing units, processing means, etc. described above may be partially or wholly realized by hardware.

10 Electronic device 100 Tactile presentation device 101 Electrostatic tactile panel/touch panel 102 Force sensor 103 Liquid crystal display 104 Carrier 105 Base 106 Lateral actuator 107 Leaf spring 108 Beam 109 Column 110 Control device 111 Display control unit 112 Layout analysis unit 113 Electrostatic tactile sense Control unit 114 Press detection unit 115 Mechanical vibration type tactile control unit 120 Storage device 130 Base 200 Electrostatic tactile panel 201 Insulating layer 202 Electrode layers 210, 211 Electrodes 220, 901 Object

Claims (7)

An electronic device comprising a display screen displaying a user interface including at least one object,
a display mechanism for displaying the user interface;
a coordinate detection mechanism for detecting the coordinates of the contact point of the user on the display screen;
a first feedback presentation mechanism that presents a localized haptic sensation to cause a user to perceive an arbitrary object;
a measuring mechanism for measuring a force caused by a user's pressing operation on the display screen;
a second feedback presentation mechanism that detects an actuation trigger based on the force and generates mechanical vibration;
and an update mechanism that updates the baseline that is the reference value of the force,
The update mechanism is
storing the force value measured by the measuring mechanism in a buffer;
When the coordinate detection mechanism detects the user's contact with the display mechanism, or when the measurement mechanism detects the user's pressing operation, discarding the force value stored in the buffer;
An electronic device, wherein the baseline is updated using the force value when the accumulated number of the force values stored in the buffer is larger than a predetermined number. The electronic device according to claim 1,
The second feedback presentation mechanism includes:
detecting as the actuation trigger when the force at the position of the object is greater than a threshold;
An electronic device that generates the mechanical vibration in order to perceive that an operation on the object has been received. The electronic device according to claim 2,
The first feedback presentation mechanism includes:
Consists of a substrate on which a plurality of electrodes are patterned, and a circuit that applies a voltage to the electrodes,
An electronic device that presents an electrostatic tactile sensation by applying a voltage to an electrode corresponding to the position of a specific object on the display screen. The electronic device according to claim 2,
The electronic device, wherein the second feedback presentation mechanism generates a mechanical vibration that vibrates in a direction substantially parallel to a direction perpendicular to the direction of the force. The electronic device according to claim 1,
The electronic device, wherein the second feedback presentation mechanism and the display mechanism are connected by a connection mechanism that reduces damping of the mechanical vibration. The electronic device according to claim 1,
The electronic device, wherein the coordinate detection mechanism and the second feedback presentation mechanism operate independently. A control method for an electronic device having a display screen displaying a user interface including at least one object,
The electronic device
a display mechanism for displaying the user interface;
a coordinate detection mechanism for detecting the coordinates of the contact point of the user on the display screen;
a first feedback presentation mechanism that presents a localized haptic sensation to cause a user to perceive an arbitrary object;
a measuring mechanism for measuring a force caused by a user's pressing operation on the display screen;
a second feedback presentation mechanism that detects an actuation trigger based on the force and generates mechanical vibration;
an update mechanism for updating the baseline that is the reference value of the force,
The method for controlling the electronic device includes:
the first feedback presentation mechanism identifying an object to present a haptic sensation based on the setting information of the user interface;
the first feedback presentation mechanism presenting a haptic sensation at the location of the identified object on the display screen;
When the force at the position of the specified object is greater than a threshold value, the second feedback presentation mechanism detects it as the actuation opportunity, and causes the mechanical vibration to make the user perceive that the operation on the specified object has been received. and
the updating mechanism storing the force values measured by the measuring mechanism in a buffer;
The force value stored in the buffer when the update mechanism detects a user's contact with the display mechanism by the coordinate detection mechanism or when the user's pressing operation is detected by the measurement mechanism. and discarding
the update mechanism updating the baseline with the force value if the cumulative number of force values stored in the buffer is greater than a certain number;
A control method for an electronic device, comprising:

Images