JP2017174210A - Electronic equipment and driving method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide electronic equipment capable of determining a display area of a character or number by a voice and vibration.SOLUTION: The electronic equipment operates a prescribed electrical appliance, and comprises: a character recognition part for recognizing characters included in an image of an operation part which is imaged by an imaging part; a storage part for storing control data which associates a prescribed word or number indicating the operation content of the electrical appliance, the operation content indicated by the prescribed word or number, and a voice indicating the prescribed word or number or the operation content; a collation part for collating the characters recognized by the character recognition part, and the word or number included in the control data stored in the storage part; a display control part for displaying a GUI component which indicates the word or number for which collation is established; a driving control part for, when a position of an operation input enters in a display area of the GUI component, driving a vibration element with a driving signal so that, intensity of natural vibration is changed according to a degree of time-variant change of the position of the operation input; and a voice guide control part for, when the position of the operation input enters the display area of the GUI component, outputting a voice guide.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an electronic device and a method for driving the electronic device.

  Conventionally, a reading unit that reads a document image on a document and outputs image data; an extraction unit that analyzes the image data read by the reading unit and extracts a rectangular region of an arbitrary size from the document image; There is provided a data processing apparatus having an adding means for adding different attribute data based on the form of each rectangular area extracted by the extracting means.

  Recognizing means for recognizing character information by analyzing image data in each rectangular area to which attribute data is added by the adding means, and adding character information and the attribute data recognized by the recognizing means to each area Storage means for storing the image data together with the image data in a memory resource.

  Also, output means for outputting character information as sound, first designation means for designating attributes of areas to be searched for image data stored in the memory resource, and designation by the first designation means. And a first determination means for determining whether an area based on the attribute is selected from the memory resource. In addition, the image forming apparatus further includes a first control unit that controls reading by the output unit based on a determination result of the first determination unit (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 11-085457

  A conventional data processing apparatus reads and analyzes a document image on a document, recognizes character information using a rectangular area, and stores it in a memory resource. Then, when a predetermined condition is satisfied, the speech reading by the output means for outputting the character information as speech is controlled.

  Therefore, the conventional data processing apparatus can make the operator confirm the contents of the document image by voice within a short time without displaying the image of the entire document image.

  By the way, the conventional data processing apparatus can confirm the content of the document image by voice, but cannot recognize the position of the character in the image of character information such as a word.

  For this reason, the conventional data processing apparatus cannot be used for applications based on the premise that the position of characters or the like in the image is recognized.

  An object of the present invention is to provide an electronic device that can determine a display area of characters or numbers by voice and vibration, and a driving method of the electronic device.

An electronic apparatus according to an embodiment of the present invention is an electronic apparatus that operates a predetermined electrical appliance, and includes a display unit, a top panel that is disposed on a display surface side of the display unit and has an operation surface, and the operation A position detection unit that detects a position of an operation input performed on the surface, and a vibration element that generates the natural vibration on the operation surface by being driven by a drive signal that generates the natural vibration of an ultrasonic band on the operation surface A transmission unit that transmits an operation command to the electrical appliance, an audio output unit, an imaging unit that captures an image of the operation unit of the electrical appliance,
A character recognition unit for recognizing characters included in the image of the operation unit captured by the imaging unit; a predetermined word or number representing an operation content of an appliance; an operation content represented by the predetermined word or number; A first storage unit that stores control data associated with a predetermined word or number or voice guidance representing the operation content, a character recognized by the character recognition unit, and the control stored in the first storage unit A collation unit that collates words or numbers included in the data; a display control unit that displays a GUI component representing the word or number for which the collation is established; and the operation detected by the position detection unit. When the input position enters the display area of the GUI part, the vibration element is driven by the drive signal so that the natural vibration intensity changes according to the degree of temporal change of the operation input position. That includes a drive control unit, the position of the operation input detected by the position detection unit enters the GUI component in the display region, and a voice guidance control unit for outputting the voice guidance to the audio output unit.

  It is possible to provide an electronic device that can determine a display area of characters or numbers by voice and vibration, and a driving method of the electronic device.

1 is a perspective view showing an electronic device 100 according to an embodiment and a washing machine 10 operated by the electronic device 100. FIG. It is a top view which shows the electronic device of embodiment. It is a figure which shows the AA arrow cross section of the electronic device shown in FIG. It is a figure which shows the wave front formed in parallel with the short side of the top panel 120 among the standing waves which arise in a top panel by the natural vibration of an ultrasonic band. It is a figure explaining a mode that the dynamic friction force applied to the fingertip which performs operation input changes with the natural vibration of the ultrasonic band produced on the top panel of an electronic device. It is a figure which shows the structure of the electronic device 100 of embodiment. It is a figure explaining the utilization scene of the electronic device. It is a figure explaining the utilization scene of the electronic device. It is a figure explaining the utilization scene of the electronic device. It is a figure explaining the utilization scene of the electronic device. It is a figure explaining the utilization scene of the electronic device. It is a figure which shows the structure of the control data which the electronic device 100 utilizes. 4 is a flowchart illustrating processing executed by an application processor 220 of the electronic device 100. It is a figure which shows the electronic device 100 by the modification of embodiment. It is a figure which shows the screen of the electronic device 100 called by the function demonstrated in FIG. 14, and the washing machine 10. FIG. It is a figure which shows the electronic device 100 by the modification of embodiment. It is a figure explaining the acquisition method of the image by the modification of embodiment. It is a figure which shows the electronic device 100 by the modification of embodiment. It is a figure which shows the electronic device 100 by the modification of embodiment. It is a figure explaining the utilization method of the electronic device 100 by the modification of embodiment. It is a figure explaining the utilization method of the electronic device 100 by the modification of embodiment.

  Embodiments to which an electronic device and a driving method of the electronic device of the present invention are applied will be described below.

<Embodiment>
FIG. 1 is a perspective view showing an electronic device 100 and a washing machine 10 operated by the electronic device 100 according to the embodiment.

  The washing machine 10 includes an opening / closing part 10 </ b> A and an operation part 11. FIG. 1 shows the washing machine 10 as viewed from above. The opening / closing part 10 </ b> A is a part provided with a lid for taking laundry in and out of the washing tub.

  The operation unit 11 is provided with buttons 11A to 11H. The button 11A is an “ON” button for turning on the power, and the button 11B is an “OFF” button for turning off the power. The button 11C is a “water level” button for adjusting the water level, and the button 11D is a “process” button for selecting a process.

  The button 11E is a “course” button for selecting a course, and the button 11F is a “reservation” button for setting a reserved operation. The button 11G is a “start” button for starting, and the button 11H is a “stop” button for stopping (pausing).

  Here, a mode in which buttons representing the operation contents of the buttons 11A to 11H are surrounded by a closed curve will be described.

  As an example, the electronic device 100 is a smartphone terminal, a tablet computer, or a portable information terminal using a touch panel as an input operation unit. The input operation unit 101 of the electronic device 100 is provided with a display panel below the touch panel, and various buttons 102 and the like (hereinafter referred to as GUI operation unit 102) using a GUI (Graphic User Interface) are displayed on the display panel. Is done. The GUI operation unit 102 may be a slide type switch (slider).

  A user of the electronic device 100 usually touches the input operation unit 101 with a fingertip in order to operate the GUI operation unit 102. In addition, the electronic device 100 includes a speaker 103 and a camera 104. The speaker 103 is an example of an audio output unit. The camera 104 is an example of an imaging unit. 1 shows the camera 104 on the display panel side of the electronic device 100, the camera 104 may be on the opposite side of the display panel.

  Next, a specific configuration of the electronic device 100 will be described with reference to FIG.

  FIG. 2 is a plan view showing the electronic device 100 according to the embodiment, and FIG. 3 is a view showing a cross section taken along the line AA of the electronic device 100 shown in FIG. 2 and 3, an XYZ coordinate system that is an orthogonal coordinate system is defined as shown.

  The electronic device 100 includes a housing 110, a top panel 120, a double-sided tape 130, a vibration element 140, a touch panel 150, a display panel 160, and a substrate 170.

  The housing 110 is made of, for example, resin, and as shown in FIG. 3, the substrate 170, the display panel 160, and the touch panel 150 are disposed in the recess 110 </ b> A, and the top panel 120 is bonded by the double-sided tape 130. .

  The top panel 120 is a thin flat plate member that is rectangular in plan view, and is made of transparent glass or a reinforced plastic such as polycarbonate. The surface of the top panel 120 (the surface on the Z-axis positive direction side) is an example of an operation surface on which the user of the electronic device 100 performs operation input.

  The top panel 120 has a vibrating element 140 (hereinafter, referred to as 140) bonded to the surface in the negative Z-axis direction, and four sides in a plan view are bonded to the housing 110 by a double-sided tape 130. The double-sided tape 130 only needs to be able to bond the four sides of the top panel 120 to the housing 110, and does not have to be a rectangular ring as shown in FIG.

  A touch panel 150 is disposed on the Z-axis negative direction side of the top panel 120. The top panel 120 is provided to protect the surface of the touch panel 150. Further, another panel or a protective film may be provided on the surface of the top panel 120.

  The top panel 120 vibrates when the vibration element 140 is driven in a state where the vibration element 140 is bonded to the surface in the negative Z-axis direction. In the embodiment, the top panel 120 is vibrated at the natural vibration frequency of the top panel 120 to generate a standing wave in the top panel 120. However, since the vibration element 140 is bonded to the top panel 120, it is actually preferable to determine the natural vibration frequency in consideration of the weight of the vibration element 140 and the like.

  The vibration element 140 is bonded to the surface on the Z-axis negative direction side of the top panel 120 along the long side extending in the Y-axis direction on the X-axis negative direction side. The vibration element 140 may be an element that can generate vibrations in an ultrasonic band. For example, an element including a piezoelectric element such as a piezoelectric element can be used.

  Here, any one of the vibration elements 140 is an example of a first vibration element, and any one of the vibration elements 140 is an example of a second vibration element.

  The vibration element 140 is driven by a drive signal output from a drive control unit described later. The amplitude (intensity) and frequency of vibration generated by the vibration element 140 are set by the drive signal. Further, on / off of the vibration element 140 is controlled by a drive signal. The vibration elements 140 are turned on / off independently of each other.

  In addition, an ultrasonic band means a frequency band about 20 kHz or more, for example. In the electronic device 100 according to the embodiment, the frequency at which the vibration element 140 vibrates is equal to the frequency of the top panel 120. Therefore, the vibration element 140 is driven by a drive signal so as to vibrate at the natural frequency of the top panel 120. Is done. This is the same in the case where all the vibration elements 140 are driven, the case where any two are driven, and the case where any one is driven.

  The touch panel 150 is disposed on the display panel 160 (Z-axis positive direction side) and below the top panel 120 (Z-axis negative direction side). The touch panel 150 is an example of a position detection unit that detects a position where the user of the electronic device 100 touches the top panel 120 (hereinafter referred to as an operation input position).

  On the display panel 160 below the touch panel 150, various buttons and the like (hereinafter referred to as GUI operation unit) by GUI are displayed. For this reason, the user of the electronic device 100 usually touches the top panel 120 with a fingertip in order to operate the GUI operation unit.

  The touch panel 150 may be a position detection unit that can detect the position of an operation input to the user's top panel 120, and may be, for example, a capacitance type or resistance film type position detection unit. Here, a mode in which the touch panel 150 is a capacitance type position detection unit will be described. Even if there is a gap between the touch panel 150 and the top panel 120, the capacitive touch panel 150 can detect an operation input to the top panel 120.

  In addition, here, a form in which the top panel 120 is disposed on the input surface side of the touch panel 150 will be described, but the top panel 120 may be integrated with the touch panel 150. In this case, the surface of the touch panel 150 becomes the surface of the top panel 120 shown in FIGS. 2 and 3, and an operation surface is constructed. Moreover, the structure which excluded the top panel 120 shown in FIG.2 and FIG.3 may be sufficient. Also in this case, the surface of the touch panel 150 constructs the operation surface. In this case, the member having the operation surface may be vibrated by the natural vibration of the member.

  In the case where the touch panel 150 is a capacitive type, the touch panel 150 may be disposed on the top panel 120. Also in this case, the surface of the touch panel 150 constructs the operation surface. Moreover, when the touch panel 150 is a capacitance type, the structure which excluded the top panel 120 shown in FIG.2 and FIG.3 may be sufficient. Also in this case, the surface of the touch panel 150 constructs the operation surface. In this case, the member having the operation surface may be vibrated by the natural vibration of the member.

  The display panel 160 may be a display unit that can display an image, such as a liquid crystal display panel or an organic EL (Electroluminescence) panel. The display panel 160 is installed on the substrate 170 (Z-axis positive direction side) by a holder or the like (not shown) inside the recess 110A of the housing 110.

  The display panel 160 is driven and controlled by a driver IC (Integrated Circuit), which will be described later, and displays a GUI operation unit, images, characters, symbols, graphics, and the like according to the operation status of the electronic device 100.

  The substrate 170 is disposed inside the recess 110 </ b> A of the housing 110. A display panel 160 and a touch panel 150 are disposed on the substrate 170. The display panel 160 and the touch panel 150 are fixed to the substrate 170 and the housing 110 by a holder or the like (not shown).

  In addition to the drive control device described later, various circuits necessary for driving the electronic device 100 are mounted on the substrate 170.

  In the electronic device 100 configured as described above, when the user's finger contacts the top panel 120 and the movement of the fingertip is detected, the drive control unit mounted on the substrate 170 activates at least one of the vibration elements 140. Driven to vibrate the top panel 120 at an ultrasonic frequency. The frequency of this ultrasonic band is a resonance frequency of a resonance system including the top panel 120 and the vibration element 140 and causes the top panel 120 to generate a standing wave.

  The electronic device 100 provides a tactile sensation to the user through the top panel 120 by generating a standing wave in the ultrasonic band.

  Next, standing waves generated in the top panel 120 will be described with reference to FIG. In electronic device 100 according to the embodiment, vibration element 140 is driven in accordance with the position of an operation input to top panel 120. As described above, the vibration element 140 is disposed at the end of the top panel 120 along the long side on the back surface of the top panel 120.

  FIG. 4 is a diagram showing a wave front formed in parallel to the short side of the top panel 120 among standing waves generated in the top panel 120 due to the natural vibration of the ultrasonic band, and FIG. 4A is a side view. (B) is a perspective view. 4A and 4B, XYZ coordinates similar to those in FIGS. 2 and 3 are defined. In FIGS. 4A and 4B, the amplitude of the standing wave is exaggerated for ease of understanding. 4A and 4B, a standing wave generated when one vibration element 140 similar to the vibration element 140 is disposed along one short side of the top panel 120 will be described. The short side of the top panel 120 is a side extending in the X-axis direction in FIG. 2, and one vibration element 140 </ b> C is disposed at the end of the top panel 120 along the short side on the Y-axis positive direction side. The generated standing wave will be described.

  When the Young's modulus E, density ρ, Poisson's ratio δ, long side dimension l, thickness t of the top panel 120 and the standing wave period k existing in the long side direction are used, the natural frequency of the top panel 120 is obtained. (Resonance frequency) f is expressed by the following equations (1) and (2). Since the standing wave has the same waveform in units of ½ period, the number of periods k takes values in increments of 0.5, which are 0.5, 1, 1.5, 2.

Note that the coefficient α in Expression (2) collectively represents coefficients other than k ² in Expression (1).

  The standing waves shown in FIGS. 4A and 4B are waveforms when the number of periods k is 10, as an example. For example, when the Gorilla (registered trademark) glass having a long side length l of 140 mm, a short side length of 80 mm, and a thickness t of 0.7 mm is used as the top panel 120, the period number k is 10. In this case, the natural frequency f is 33.5 [kHz]. In this case, a drive signal having a frequency of 33.5 [kHz] may be used.

  The top panel 120 is a flat plate member. When the vibration element 140 is driven to generate the natural vibration of the ultrasonic band, the top panel 120 is bent as shown in FIGS. 4A and 4B. A standing wave is generated on the surface.

  Note that, here, a description will be given of a mode in which one vibration element 140 is bonded along the short side extending in the X-axis direction on the Y-axis positive direction side on the surface of the top panel 120 on the Z-axis negative direction side. Two vibration elements 140 may be used. When two vibration elements 140 are used, the other vibration element 140 is bonded to the surface of the top panel 120 on the Z-axis negative direction side along the short side extending in the X-axis direction on the Y-axis negative direction side. That's fine. In this case, the two vibration elements 140 may be arranged so as to be axially symmetric with respect to a center line parallel to the two short sides of the top panel 120 as a symmetry axis.

  In addition, when the two vibrating elements 140 are driven, they may be driven in the same phase when the number of periods k is an integer, and in the opposite phase when the number of periods k is a decimal (a number including an integer part and a decimal part). What is necessary is just to drive.

  Next, the natural vibration of the ultrasonic band generated in the top panel 120 of the electronic device 100 will be described with reference to FIG.

  FIG. 5 is a diagram illustrating a state in which the dynamic friction force applied to the fingertip that performs the operation input changes due to the natural vibration of the ultrasonic band generated in the top panel 120 of the electronic device 100. 5A and 5B, the user performs an operation input to move the finger along the arrow from the back side of the top panel 120 to the near side while touching the top panel 120 with the fingertip. The vibration is turned on / off by turning the vibration element 140 on / off.

  5A and 5B, in the depth direction of the top panel 120, the range in which the finger touches while the vibration is off is shown in gray, and the range in which the finger touches while the vibration is on is shown in white. .

  The natural vibration of the ultrasonic band occurs in the entire top panel 120 as shown in FIG. 4, but in FIGS. 5A and 5B, the user's finger is on the front side from the back side of the top panel 120. The operation pattern which switches on / off of a vibration during moving to is shown.

  In the operation pattern shown in FIG. 5A, the vibration is turned off when the user's finger is on the back side of the top panel 120, and the vibration is turned on in the middle of moving the finger to the near side.

  On the other hand, in the operation pattern shown in FIG. 5B, the vibration is turned on when the user's finger is on the back side of the top panel 120, and the vibration is turned off in the middle of moving the finger to the near side. Yes.

  Here, when the natural vibration of the ultrasonic band is generated in the top panel 120, an air layer due to the squeeze effect is interposed between the surface of the top panel 120 and the finger, and the surface of the top panel 120 is traced with the finger. The coefficient of dynamic friction decreases.

  Accordingly, in FIG. 5A, the dynamic friction force applied to the fingertip is large in the range indicated in gray on the back side of the top panel 120, and the dynamic friction force applied to the fingertip is small in the range indicated in white on the near side of the top panel 120. Become.

  For this reason, as shown in FIG. 5A, the user who performs an operation input to the top panel 120 senses a decrease in the dynamic friction force applied to the fingertip and perceives the ease of slipping of the fingertip when the vibration is turned on. It will be. At this time, the user feels that a concave portion exists on the surface of the top panel 120 when the dynamic friction force decreases due to the surface of the top panel 120 becoming smoother.

  On the other hand, in FIG. 5B, the dynamic friction force applied to the fingertip is small in the range shown in white on the front side of the top panel 120, and the dynamic friction force applied to the fingertip is large in the range shown in gray on the front side of the top panel 120. Become.

  For this reason, as shown in FIG. 5B, the user who performs an operation input to the top panel 120 senses an increase in the dynamic friction force applied to the fingertip when the vibration is turned off, You will perceive the feeling of being caught. And when a dynamic friction force becomes high because it becomes difficult to slip a fingertip, it will feel like a convex part exists in the surface of the top panel 120. FIG.

  From the above, in the case of (A) and (B) in FIG. 5, the user can feel unevenness with the fingertip. For example, humans perceive unevenness in this way, for example, “Printed Transfer Method and Sticky-band Illusion for Tactile Design” (Proceedings of the 11th SICE System Integration Division Annual Conference (SI2010, Sendai) ___ 174 -177, 2010-12). It is also described in “Fishbone Tactile Illusion” (The 10th Annual Conference of the Virtual Reality Society of Japan (September 2005)).

  Here, the change in the dynamic friction force when switching on / off the vibration has been described, but this is the same when the amplitude (intensity) of the vibration element 140 is changed.

  Next, the configuration of the electronic device 100 according to the embodiment will be described with reference to FIG.

  FIG. 6 is a diagram illustrating a configuration of the electronic device 100 according to the embodiment.

  The electronic device 100 includes a vibration element 140, an amplifier 141, a touch panel 150, a driver IC (Integrated Circuit) 150A, a display panel 160, a driver IC 160A, an amplifier 180A, a speaker 103, a camera 104, a control device 200, a sine wave generator 310, and An amplitude modulator 320 is included.

  The control device 200 includes an application processor 220, a communication processor 230, a drive control unit 240, and a memory 250. The control device 200 is realized by an IC chip, for example.

  In addition, the drive control unit 240, the sine wave generator 310, and the amplitude modulator 320 constitute the drive control device 300.

  In FIG. 6, the casing 110, the top panel 120, the double-sided tape 130, and the substrate 170 (see FIG. 2) are omitted. Here, the amplifier 141, the driver IC 150A, the driver IC 160A, the drive control unit 240, the memory 250, the sine wave generator 310, and the amplitude modulator 320 will be described.

  Each of the amplifiers 141 is disposed between the drive control device 300 and the vibration element 140, and amplifies the drive signal output from the drive control device 300 to drive the vibration element 140.

  The driver IC 150 </ b> A is connected to the touch panel 150, detects position data indicating a position where an operation input to the touch panel 150 has been performed, and outputs the position data to the control device 200. As a result, the position data is input to the application processor 220 and the drive control unit 240. Note that inputting position data to the drive control unit 240 is equivalent to inputting position data to the drive control apparatus 300.

  The driver IC 160A is connected to the display panel 160, inputs drawing data output from the drive control device 300 to the display panel 160, and causes the display panel 160 to display an image based on the drawing data. As a result, a GUI operation unit or an image based on the drawing data is displayed on the display panel 160.

  The amplifier 180A is connected to the application processor 220, amplifies the audio signal input from the application processor 220, and outputs the amplified audio signal to the speaker 103. The speaker 103 outputs the audio signal input from the amplifier 180A as audio.

  The application processor 220 reads audio data stored in the memory 250 in response to an operation input performed on the top panel 120, and outputs the audio data to the amplifier 180A. As a result, sound corresponding to the operation input performed on the top panel 120 is output from the speaker 103.

  The application processor 220 performs processing for executing various applications of the electronic device 100. The application processor 220 includes a main control unit 221, a character recognition unit 222, a collation unit 223, a display control unit 224, a drive control unit 240, and a voice guidance control unit 225.

  The main control unit 221 is a part that supervises the control processing of the application processor 220, and supervises the processes of the character recognition unit 222, the collation unit 223, the display control unit 224, the drive control unit 240, and the voice guidance control unit 225. The main control unit 221 is an example of a command generation unit, a request unit, a reading unit, and an overall processing unit.

  The character recognition unit 222 recognizes characters included in an image captured by the camera 104. The character recognition unit 222 uses an OCR (Optical Character Reader) to extract a pattern included in the image by processing such as binarization or edge processing, and collate with the built-in dictionary data (pattern matching). To recognize characters included in the image. A known character recognition technique can be used for such a character recognition unit 222.

  The collating unit 223 collates the word whose character is recognized by the character recognizing unit 222 with the word included in the control data stored in the memory 250, and includes the word included in the control data by the character recognizing unit 222. It is determined whether there is a word identical to the word whose character is recognized.

  When the collation unit 223 determines that the same word as the word whose character is recognized by the character recognition unit 222 is in the word included in the control data, the collation is established. If the same word as the word whose character is recognized by the character recognition unit 222 does not exist in the control data, the collation is not established.

  The display control unit 224 reads data in an area corresponding to the word that has been verified by the verification unit 223 in the control data, and generates data representing a GUI button that represents the word that has been verified by the verification unit 223. The display control unit 224 outputs data representing the GUI button to the driver IC 160A, and causes the display panel 160 to display the GUI button.

  Note that the GUI button is a kind of GUI component 102 shown in FIG. 1 and is a GUI component 102 that can be operated so as to be pressed.

  The drive controller 240 is generated by the display controller 224 and the position of the user's fingertip (position of operation input) detected by the touch panel 150 enters the display area of the GUI button displayed on the display panel 160, and When the position of the operation input is moved, the vibration element 140 is driven. At this time, the drive control unit 240 drives the vibration element 140 with a drive pattern corresponding to the word displayed on the GUI button in the control data.

  The drive control unit 240 outputs amplitude data to the amplitude modulator 320 in order to drive the vibration element 140. The amplitude data is data representing an amplitude value for adjusting the strength of the drive signal used for driving the vibration element 140. The amplitude value is set according to the temporal change degree of the position data. Here, as the temporal change degree of the position data, a speed at which the user's fingertip moves along the surface of the top panel 120 is used. The moving speed of the user's fingertip is calculated by the drive control unit 240 based on the temporal change degree of the position data input from the driver IC 150A.

  The voice guidance control unit 225 generates the position of the user's fingertip (position of operation input) detected by the touch panel 150 in the display area of the GUI button generated by the display control unit 224 and displayed on the display panel 160. The voice guidance is performed based on the voice data corresponding to the word displayed on the GUI button in the control data.

  The voice guidance control unit 225 outputs the content of the voice guidance represented by the voice data to the amplifier 180A. As a result, the voice representing the voice guidance is output from the speaker 103.

  The voice guidance control process and the drive control of the vibration element 140 performed in accordance with the voice guidance control process will be described later with reference to FIGS.

  The communication processor 230 executes processing necessary for the electronic device 100 to perform communication such as 3G (Generation), 4G (Generation), LTE (Long Term Evolution), and WiFi. The communication processor 230 may be capable of supporting infrared communication or communication with an RFID (Radio Frequency Identification) tag.

  As an example, the drive control apparatus 300 according to the embodiment reduces the amplitude value as the moving speed increases, so that the user senses the touch feeling from the fingertip regardless of the moving speed of the fingertip. The lower the value, the larger the amplitude value.

  Data representing the relationship between the amplitude data representing the amplitude value and the moving speed is stored in the memory 250.

  Here, a mode in which the amplitude value corresponding to the moving speed is set using the data stored in the memory 250 will be described, but the amplitude value A may be calculated using the following equation (3). The amplitude value A calculated by Equation (3) decreases as the moving speed increases, and increases as the moving speed decreases.

  Here, A0 is the amplitude reference value, V is the moving speed of the fingertip, and a is a predetermined constant. When the amplitude value A is calculated using the equation (3), the data representing the equation (3) and the data representing the amplitude reference value A0 and the predetermined constant a may be stored in the memory 250.

  Further, the drive control device 300 according to the embodiment vibrates the top panel 120 in order to change the dynamic friction force applied to the fingertip when the user's fingertip moves along the surface of the top panel 120. Since the dynamic friction force is generated when the fingertip is moving, the drive control unit 240 vibrates the vibration element 140 when the moving speed becomes equal to or higher than a predetermined threshold speed.

  Therefore, the amplitude value represented by the amplitude data output by the drive control unit 240 is zero when the moving speed is less than the predetermined threshold speed, and when the moving speed is equal to or higher than the predetermined threshold speed, the amplitude value is determined according to the moving speed. Set to the amplitude value. When the moving speed is equal to or higher than a predetermined threshold speed, the higher the moving speed, the smaller the amplitude value is set, and the lower the moving speed is, the larger the amplitude value is set.

  In addition, the drive control device 300 according to the embodiment outputs amplitude data to the amplitude modulator 320 when the position of the fingertip where the operation input is performed is within a predetermined region where vibration is to be generated.

  Whether or not the position of the fingertip that performs the operation input is within a predetermined region where the vibration is to be generated is based on whether or not the position of the fingertip that performs the operation input is within the predetermined region where the vibration is to be generated. Determined.

  Here, the position on the display panel 160 such as a GUI operation unit to be displayed on the display panel 160, a region for displaying an image, or a region representing the entire page is specified by region data representing the region. In all applications, the area data exists for all GUI operation units displayed on the display panel 160, areas for displaying images, or areas representing the entire page.

  The memory 250 includes data representing the relationship between the amplitude data representing the amplitude value and the moving speed, data representing the type of application, area data representing the GUI operation unit where the operation input is performed, and the pattern representing the vibration pattern. Stores data associated with data. The memory 250 is an example of a first storage unit and a second storage unit.

  The memory 250 stores control data necessary for performing drive control of the vibration element 140 and voice guidance. The control data includes words, operation details, areas, vibration patterns, audio data, and essential function flags.

The control data includes a word indicating the operation content in the operation unit of various electrical appliances, a display area of a GUI button representing the word, and a vibration that drives the vibration element 140 when an operation input is performed on the GUI button. This is data in which a pattern is associated with voice data for voice guidance performed when an operation input is made to the GUI button. The control data will be described later with reference to FIG.
In addition to the washing machine 10, the electrical appliance refers to various electrical products such as a microwave oven, a television, a video recorder, and an audio. The electrical appliance may include a computer, a tablet computer, a smartphone, a game machine, and the like.

  In addition, the memory 250 stores data and programs necessary for the application processor 220 to execute an application, data and programs necessary for the communication processor 230 for communication processing, and the like.

  The sine wave generator 310 generates a sine wave necessary for generating a drive signal for vibrating the top panel 120 at a natural frequency. For example, when the top panel 120 is vibrated at a natural frequency f of 33.5 [kHz], the frequency of the sine wave is 33.5 [kHz]. The sine wave generator 310 inputs an ultrasonic band sine wave signal to the amplitude modulator 320.

  Here, a mode in which the sine wave generator 310 is used will be described, but a clock generator that generates a clock may be used instead of the sine wave generator 310. For example, the rising and falling waveforms of the clock can be blunted by setting the slew rate of the clock generated by the clock generator small. In this way, a clock with a small slew rate may be used instead of the sine wave generated by the sine wave generator 310. That is, instead of a sine wave, a waveform signal whose amplitude periodically changes may be used.

  The amplitude modulator 320 generates a drive signal by modulating the amplitude of the sine wave signal input from the sine wave generator 310 using the amplitude data input from the drive control unit 240. The amplitude modulator 320 modulates only the amplitude of the sine wave signal in the ultrasonic band input from the sine wave generator 310, and generates the drive signal without modulating the frequency and phase.

  Therefore, the drive signal output from the amplitude modulator 320 is an ultrasonic band sine wave signal obtained by modulating only the amplitude of the ultrasonic band sine wave signal input from the sine wave generator 310. Note that when the amplitude data is zero, the amplitude of the drive signal is zero. This is equivalent to the amplitude modulator 320 not outputting a drive signal.

  FIG. 7 to FIG. 11 are diagrams for explaining usage scenes of the electronic device 100.

  First, the user takes a picture of the operation unit 11 of the washing machine 10 with the electronic device 100. As a result, the image of the operation unit 11 is displayed on the display panel 160 as shown in FIG. The operation unit 11 is provided with buttons 11A to 11H. A circular border line surrounding the buttons 11A to 11H is an example of a closed curve.

  Next, when the user causes the electronic device 100 to perform processing such as image processing, character recognition, and collation processing, the GUI buttons 161A to 161H and the peripheral area 162 are displayed on the display panel 160 as shown in FIG. Is done. The peripheral area 162 is a rectangular area surrounding the GUI buttons 161A to 161H, and other information other than the GUI buttons 161A to 161H is also displayed.

  Other information includes various settings (water level (22L, 34L, 45L), process (washing, rinsing, water injection, dehydration), course (standard, speedy, dry, blanket) selected by operating the GUI buttons 161A to 161F. Information such as letters or numbers of reservation time (1 hour, 3 hours), power supply). Such other information is a character or a number recognized outside the region surrounded by the closed curve in step S5.

  The GUI buttons 161A to 161H, the peripheral area 162, and other information are generated when the electronic device 100 executes processing such as image processing, character recognition, and collation processing on the image of the operation unit 11 illustrated in FIG. And displayed on the display panel 160. The GUI buttons 161A to 161H correspond to the buttons 11A to 11H of the operation unit 11 of the washing machine 10 shown in FIG.

  The GUI buttons 161A to 161H are all surrounded by a circular line (border). Hereinafter, the area on the line surrounding the GUI buttons 161A to 161H and the area inside the line are referred to as display areas of the GUI buttons 161A to 161H.

  The GUI button 161A is an “ON” button for turning on the power, and the GUI button 161B is an “OFF” button for turning off the power. The GUI button 161C is a “water level” button for adjusting the water level, and the GUI button 161D is a “process” button for selecting a process.

  The GUI button 161E is a “course” button for selecting a course, and the GUI button 161F is a “reservation” button for setting a reserved operation. The GUI button 161G is a “start” button for starting, and the GUI button 161H is a “stop” button for stopping (temporarily stopping).

  The GUI buttons 161A to 161H are displayed on the display panel 160 as GUI components, and the electronic device 100 can transmit commands represented by the GUI buttons 161A to 161H to the washing machine 10 through wireless communication.

  For this reason, when the user touches each GUI button 161A to 161H (operation input) with a fingertip or the like, the washing machine 10 operates in the same manner as when the buttons 11A to 11H of the operation unit 11 are pressed.

  The display panel 160 includes other information such as the amount of water level (22L, 34L, 45L), specific contents of the process (washing, rinsing, water injection, dehydration), and specific contents of the course (standard, speedy). , Dry, blanket), reservation time (1 hour, 3 hours) is displayed.

  These pieces of information can be sequentially selected by pressing GUI buttons 161C, 161D, 161E, and 161F. The selected content can be identified, for example, by being displayed in a color different from the content that has not been selected.

  Specifically, for example, 22L can be selected by pressing the GUI button 161C once, 34L can be selected by pressing the GUI button 161C twice, and 45L can be selected by pressing the GUI button 161C three times. be able to. Then, when the GUI button 161C is pressed once again (fourth press), 22L can be selected. The same applies to the GUI buttons 161D, 161E, and 161F.

  Next, as shown in FIG. 9, when the user presses the GUI button 161G (start button) and moves the fingertip within the display area of the GUI button 161G, the electronic device 100 drives the vibration element 140, A natural vibration of the ultrasonic band is generated on the top panel 120 and a voice guidance “start button” is output from the speaker 103. Electronic device 100 may output such voice guidance even if the fingertip is not moving as long as the fingertip is within the display area of GUI button 161G.

  Here, the electronic device 100 drives the vibration element 140 with a drive pattern as shown in FIG. In FIG. 10, the horizontal axis represents time, and the vertical axis represents the amplitude of the drive signal that drives the vibration element 140.

  When the user's fingertip touches a portion other than the display area of the GUI buttons 161A to 161H on the top panel 120 at time t1, the electronic device 100 drives the vibration element 140 with a drive signal having an amplitude A3. Since the amplitude of the drive signal at this time is constant at A3, a smooth and smooth tactile sensation is provided to the user's fingertip.

  At time t2, when the index finger enters the GUI button 161G, the vibration element 140 is driven by a drive signal whose amplitude periodically varies with time, and a voice guidance “start button” is received from the speaker 103. Is output.

  Since the amplitude of this drive signal periodically fluctuates in a sine wave shape between A1 and A2, a rough tactile sensation is provided to the user's fingertip. In addition, the amplitudes A1 and A2 are both smaller than the amplitude A3. For this reason, when the index finger enters the GUI button 161G, a change in tactile sensation applied to the user's fingertip is perceived. Also, the voice guidance notifies that the current fingertip position is within the display area of the “start button” (button 161G).

  For this reason, the user can recognize that the current fingertip position is in the GUI button 161G of the “start button” without visually recognizing the tactile sensation and the voice guidance.

  When the user further moves the index finger and goes out of the display area of the GUI button 161G at time t3, the vibration element 140 is driven by the drive signal having the amplitude A3.

  Accordingly, the user can recognize without visually confirming that the current fingertip position is out of the “start button” (GUI button 161G).

  When the user stops the movement of the index finger at time t4, the amplitude of the drive signal becomes zero and the drive of the vibration element 140 is stopped. The same applies when the user lifts the index finger from the top panel 120 at time t4.

  In this way, the user's fingertip is guided to the GUI button 161G using the tactile sensation utilizing the squeeze film effect and the voice guidance. Therefore, the user can accurately recognize the position of the GUI button 161G without visual inspection.

  As shown in FIG. 11, when the user strongly presses the GUI button 161 </ b> G, the electronic device 100 determines that the operation is confirmed and transmits a command to start washing to the washing machine 10. Thereby, the washing machine 10 starts washing.

  In this way, the user can execute the operation (start) of the washing machine 11 associated with the GUI button 161G by performing a confirmation operation on the electronic device 100.

  For example, the confirmation operation is a strong pressing of the top panel 120, and the electronic device 100 receives an input by performing the confirmation operation. The function of accepting such a confirmation operation is realized by the OS of the application processor 220 (see FIG. 6), for example. The confirmation operation is not limited to pressing the top panel 120 strongly, and may be accepted based on an increase in the area of a fingertip or the like that is touching the top panel 120 (operating input).

  Although a scene where the “start button” (GUI button 161G) is pressed has been described here, the same applies to the other GUI buttons 161A to 161F and 161H.

  FIG. 12 is a diagram illustrating a structure of control data used by the electronic device 100. Control data is stored in the memory 250. The data shown in FIG. 12 is data for realizing the operations shown in FIGS. 7 to 11 as an example.

  The control data includes words, operation details, areas, vibration patterns, audio data, and essential function flags.

  The control data consists of the words "start", "start", "stop", "stop", "water level", "level", "water volume", "process", "step", "work", "course" , “Automatic”. Here, a word representing the operation content of the washing machine is shown as a word included in the control data, but the control data also includes a word representing the operation content of the electrical appliance other than the washing machine. The control data includes a word list (word list) representing the operation contents of various electrical appliances.

  The operation content is assigned to each word. The operation content “start washing” is assigned to the words “start” and “start”. The two words “start” and “start” are associated with one operation content “start washing” as a button for realizing the operation content “start washing”. This is because there is a possibility that a button on which any word of “start” is written is provided in the washing machine 10. For this reason, in addition to the two words “start” and “start”, a word “START” may be further added.

  The words “stop” and “stop” are assigned the operation content “stop washing”. The two words “stop” and “stop” are associated with one operation content “stop washing” because “stop” or “stop” is used as a button for realizing the operation “stop washing”. This is because there is a possibility that the washing machine 10 is provided with a button on which any word of “stop” is written. For this reason, in addition to the two words “stop” and “stop”, the word “STOP” may be further added.

  The words “water level”, “level”, and “water volume” are assigned the operation content “setting water level”. Three words of “water level”, “level”, and “water amount” are associated with one operation of “water level setting”. This is because there is a possibility that a button on which any one of the three words is written is provided in the washing machine 10.

  The operation content “process setting” is assigned to the words “process”, “step”, and “work”. Three words “process”, “step”, and “work” are associated with one operation “process setting”. This is because there is a possibility that a button on which any one of the three words is written is provided in the washing machine 10.

  The operation content “course setting” is assigned to the words “course” and “automatic”. Two words of “course” and “automatic” are associated with one operation of “course setting”. This is because there is a possibility that a button on which any one of the two words is written is provided in the washing machine 10.

  The area represents an area for displaying a GUI button for displaying each word. A GUI button for displaying the words “start” and “start” is displayed in the region f1 (X, Y). The region f1 (X, Y) represents coordinates in the entire display region of the display panel 160.

  The coordinates represented by the region f1 (X, Y) are obtained by pressing the “start” or “start” GUI button displayed on the display panel 160 when the collation by the collation unit 223 is established for the word “start” or “start”. Represents the display area to be displayed. This display area is, for example, a circular area.

  In the present embodiment, a circle outline representing a boundary line is displayed at the boundary of the circular area represented by the area f1 (X, Y), and the characters “start” or “start” are displayed in the circle outline. As a result, a “start” or “start” GUI button is generated.

  This also applies to the other areas f2 (X, Y) to f5 (X, Y),... Assigned for words other than “start” and “start”.

  A GUI button for displaying the words “stop” and “stop” is displayed in the region f2 (X, Y). A GUI button for displaying the words “water level”, “level”, and “water volume” is displayed in the region f3 (X, Y).

  A GUI button for displaying the words “process”, “step”, and “work” is displayed in the region f4 (X, Y). The GUI button for displaying the words “course” and “automatic” is displayed in the region f5 (X, Y).

  Note that the regions f1 (X, Y) to F5 (X, Y) are set so as not to overlap each other and to be slightly opened.

  The vibration pattern represents a drive pattern that drives the vibration element 140 when the position of the operation input moves within the display area of the GUI button that displays each word.

  The words “start” and “start” are associated with the vibration pattern of the waveform A. When the position of the operation input moves in the display area of the GUI button that displays the word “start” or “start”, the vibration element 140 is driven with a vibration pattern based on the waveform A.

  The words “stop” and “stop” are associated with the vibration pattern of the waveform B. A vibration pattern based on the waveform C is associated with the words “water level”, “level”, and “water volume”. A vibration pattern based on the waveform D is associated with the words “process”, “step”, and “work”. A vibration pattern based on the waveform E is associated with the words “course” and “automatic”.

  The voice data is data representing voice guidance output from the speaker 103 when the operation input position is within the display area of the GUI button for displaying each word.

  The words “start” and “start” are associated with voice data “start button” and “start button”, respectively. The words “stop” and “stop” are associated with audio data “is a stop button” and “is a stop button”, respectively.

  The words “water level”, “level”, and “water volume” are associated with audio data “water level button”, “level button”, and “water volume button”, respectively. The words “process”, “step”, and “work” are associated with audio data “process button”, “step button”, and “work button”, respectively. The words “course” and “automatic” are associated with audio data “course button” and “automatic button”, respectively.

  The essential function flag is a flag indicating whether the word is an essential word as a GUI button. If the value of the essential function flag is “1”, it indicates that it is an essential function, and if the value of the essential function flag is “0”, it indicates that it is an essential function.

  Here, as an example, the value of the essential function flag of the words “stop” and “stop” is “1”, and it is essential that the GUI buttons of the words “stop” and “stop” are generated. Yes. This is because if the washing machine 10 cannot be stopped, use may be hindered.

  Although not shown in FIG. 12, the value of the essential function flag may be set to “1” for the word “OFF” representing the “OFF” button for turning off the power. This is because if there is no GUI button with the word “OFF”, use may be hindered.

  When there is a word whose essential function flag is “1” in the control data, the electronic device 100 can be used when the word is recognized and collation is established.

  Note that the bottom line of FIG. 12 shows words, operation details, areas, vibration patterns, audio data, and essential function flags for portions (peripheral areas) other than the GUI button display area. For the part other than the display area of the GUI button, there is no word, the operation content is “none”, the area is f9 (X, Y), the vibration pattern is waveform I, and the voice data is “none”. Is.

  The region f9 (X, Y) represents, for example, the peripheral region 162 shown in FIG. The region f9 (X, Y) may be regarded as a region obtained by excluding the display region of the GUI button for the word that has been verified from the entire display region of the display panel 160.

  The electronic device 100 performs character recognition on the image captured by the user and collates the recognized character with the word of the control data. Then, a GUI button is generated for the word for which collation is established, and a GUI button representing the word is displayed in an area associated with the word in the control data.

  At this time, if there is a word whose essential function flag is “1” in the control data, the electronic device 100 can be used when the word is recognized and collation is established. .

  Then, when the user moves the fingertip that is performing the operation input within the display area of the GUI button of the word, the vibration element 140 is driven with the vibration pattern associated with the control data, and the control data Voice guidance is output from the speaker 103 based on the voice data associated therewith.

  By using the control data as shown in FIG. 12, it is possible to generate the operation and drive signals as shown in FIGS.

  FIG. 13 is a flowchart illustrating processing executed by the application processor 220 of the electronic device 100.

  First, the application processor 220 starts a flow when an application for remote-controlling an electrical appliance is washed (start).

  The application processor 220 determines whether or not an operation for reading registration data has been performed (step S1). Registered data refers to data in which a GUI button generated for a word that has been verified is stored in the memory 250 by performing verification on characters obtained by character recognition from an image acquired by a user. An example of the registration data is data including GUI buttons 161A to 161H, a peripheral area 162, and other information shown in FIG.

  If the application processor 220 determines that the operation of reading the registration data has not been performed (S1: NO), the application processor 220 requests acquisition of an image (step S2). Specifically, the user is requested to acquire an image of an operation unit of an electrical appliance that the user wants to perform a remote control operation. At this time, the user may activate the camera 104 to acquire an image, or may acquire a desired photograph from the photograph data stored in the memory 250. In step S2, a message “Please select an image” may be displayed on the display panel 160. The process of step S2 is executed by the main control unit 221.

  The application processor 220 displays the image acquired in step S2 on the display panel 160 (step S3). The process of step S3 is executed by the main control unit 221 outputting the image acquired in step S2 to the display control unit 224, and the display control unit 224 displaying the image input from the main control unit 221 on the display panel 160. Is done.

  The application processor 220 extracts a closed curve by performing image processing such as edge processing on the image displayed on the display panel 160 (step S4). The closed curve is, for example, a closed curve surrounding words included in the buttons 11A to 11H provided in the operation unit 11 of the electric appliance such as the washing machine 10 illustrated in FIG. The process of step S4 is executed by the main control unit 221.

  The application processor 220 performs character recognition on the area surrounded by the closed curve extracted in step S4 (step S5). In the process of step S5, the character recognition unit 222 uses OCR to extract a pattern included in the image by a process such as binarization or edge processing, and collates with the built-in dictionary data (pattern matching). ) To recognize characters included in the image.

  In step S5, character recognition is basically performed on the area surrounded by the closed curve. Here, however, characters or numbers included outside the area surrounded by the closed curve are also recognized and surrounded by the closed curve. Data shall be held as characters or numbers recognized outside the area.

  The application processor 220 collates the word whose character is recognized by the character recognition unit 222 with the word included in the control data stored in the memory 250, and the character recognition unit 222 uses the word included in the control data. It is determined whether there is a word identical to the word whose character is recognized (step S6). The process of step S6 is executed by the collation unit 223.

  If the same word as the word whose character is recognized in step S5 is in the word included in the control data, the collation is established. If the same word as the word whose character is recognized does not exist in the control data, the collation is not established.

  If the application processor 220 determines that collation is established (S6: YES), the application processor 220 generates a GUI button and displays it on the display panel 160 (step S7). The process in step S7 is executed by the display control unit 224.

  In addition to the GUI button, the display control unit 224 generates a peripheral area of the GUI button and other information to be displayed in the peripheral area, and causes the display panel 160 to display them together.

  The display control unit 224 reads data in an area corresponding to the word for which collation is established in the control data, and generates data representing a GUI button representing the word for which collation by the collation unit 223 is established. Then, the display control unit 224 outputs data representing the GUI button to the driver IC 160A, and causes the display panel 160 to display the GUI button.

  The GUI button is a GUI button having a circle outline representing a boundary line at the boundary of a circular area represented by the area and a word for which matching has been established in the circle outline. The color of the display area and the color of the word may be set in advance.

  The application processor 220 stores the data representing the GUI button, the peripheral area, and other information generated in step S7 in the memory 250 as registration data (step S8). The process of step S is executed by the main control unit 221.

  The application processor 220 determines whether an operation input has been performed (step S9). The process of step S9 is executed by the main control unit 221.

  The application processor 220 determines whether an operation input is performed by determining whether the touch panel 150 has detected coordinates. The application processor 220 detects the coordinates of the position of the operation input in step S9.

  If the application processor 220 determines that an operation input has been performed (S9: YES), the application processor 220 determines whether the position of the operation input has moved (step S10). The process of step S10 is executed by the main control unit 221.

  The application processor 220 may determine whether the position of the operation input has moved by determining whether the coordinates detected by the touch panel 150 have changed.

  If the application processor 220 determines that the position of the operation input is moving (S10: YES), the application processor 220 determines whether the position of the operation input is within the display area of the GUI button (step S11). The process of step S11 is executed by the main control unit 221.

  Since the display area of the GUI button is generated in the process of step S7, the application processor 220 determines whether the coordinates of the position of the operation input are included in the display area of any GUI button. It may be determined whether the position of the operation input is within the display area of the GUI button.

  If the application processor 220 determines that the position of the operation input is within the display area of the GUI button (S11: YES), the application processor 220 vibrates according to the vibration pattern corresponding to the GUI button including the position of the operation input in the control data. The element 140 is driven (step S12). The process of step S12 is executed by the drive control unit 240.

  The application processor 220 performs voice guidance according to the voice data corresponding to the GUI button including the position of the operation input in the control data (step S13). The process of step S13 is executed by the voice guidance control unit 225.

  The application processor 220 determines whether a confirmation operation has been performed (step S14). The process of step S14 is executed by the main control unit 221.

  For example, the confirmation operation is a strong pressing of the top panel 120, and the electronic device 100 receives an input by performing the confirmation operation. The function of accepting such a confirmation operation is realized by the OS of the application processor 220 (see FIG. 6), for example. The confirmation operation is not limited to pressing the top panel 120 strongly, and may be accepted based on an increase in the area of a fingertip or the like that is touching the top panel 120 (operating input).

  If the application processor 220 determines that the confirmation operation has been performed (S14: YES), the application processor 220 transmits data representing the operation content corresponding to the GUI button to which the confirmation operation has been performed in the control data to the electrical appliance (step S15). ). The process of step S15 is executed by the main control unit 221.

  The application processor 220 determines whether or not an operation input has been performed for a predetermined time or more since the previous operation input was performed (step S16). The process of step S16 is executed by the main control unit 221.

  If the application processor 220 determines that the operation input has not been performed for a predetermined time or longer (S16: YES), the application processor 220 ends the series of processes (end). This is because it is considered that the operation of the electric appliance by the electronic device 100 is finished.

  If the application processor 220 determines in step S1 that an operation for reading registration data has been performed (S1: YES), the application processor 220 reads the registration data from the memory 250 and advances the flow to step S9. This is because it is not necessary to perform steps S2 to S8 because the user has selected to operate the appliance based on the registration data previously stored in the memory 250.

  If the application processor 220 determines in step S6 that collation is not established (S6: NO), the flow returns to step S2. In the case where collation is not established, for example, there are a case where a character recognized as a character does not match a word included in the control data and a case where a character is not recognized correctly.

  In these cases, since it is not possible to operate the electrical appliance with the electronic device 100, the flow is returned to step S2 to request acquisition of an image again.

  If the application processor 220 determines in step S9 that no operation input has been performed (S9: NO), the application processor 220 repeatedly executes the process in step S9 until an operation input is performed. This is to prepare for subsequent user operation inputs.

  If the application processor 220 determines in step S10 that the position of the operation input has not moved (S10: NO), the flow returns to step S9. This is to prepare for subsequent user operation inputs.

  If the application processor 220 determines in step S11 that the position of the operation input is not within the display area of the GUI button (S11: NO), the application processor 220 changes the vibration pattern in the peripheral area corresponding to the GUI button in the control data. Accordingly, the vibration element 140 is driven (step S17). The process of step S17 is executed by the drive control unit 240.

  When the application processor 220 drives the vibration element 140 in step S17, the flow returns to step S9. This is to prepare for subsequent user operation inputs.

  If the application processor 220 determines in step S14 that the confirmation operation has not been performed (S14: NO), the flow returns to step S9. This is to prepare for subsequent user operation inputs.

  If the application processor 220 determines in step S16 that the operation input has not been performed for a predetermined time or longer (S16: NO), the flow returns to step S9. This is to prepare for subsequent user operation inputs.

  By executing the processing as described above, it is possible to operate the electrical appliance with the electronic device 100 based on the image of the operation unit of the electrical appliance acquired by the user. Further, when the user operates the electrical appliance with the electronic device 100, the electronic device 100 drives the vibration element 140 when the operation input moves within the GUI button, so that the fingertip is inside the GUI button. This can be transmitted to the user by touch.

  For example, as shown in FIG. 10, when the vibration pattern when the user's fingertip touches the surrounding area from time t1 to time t2 enters the display area of the GUI button 161G at time t2, the driving pattern changes to a different driving pattern. By changing, this corresponds to notifying the user that the fingertip is inside the GUI button through the tactile sensation.

  When the operation input enters the GUI button, a voice guidance indicating the operation content of the GUI button is output from the speaker 103, so that the user can execute the GUI button that he / she touches with only touch and voice. Possible operation contents can be grasped.

  As described above, according to the embodiment, it is possible to provide the electronic device 100 that can discriminate the region by vibration and sound, and the driving method of the electronic device 100.

  Therefore, the user can grasp the GUI button and the operation content of the GUI button by vibration and voice without visual observation, and can use the electronic device 100 as a remote controller without visual observation. Can be operated.

  For this reason, various electrical appliances can be operated with the electronic device 100. For example, in addition to the washing machine 10, various electric appliances such as a microwave oven, a television, a video recorder, and an audio can be operated by recognizing the GUI button by vibration and voice with one electronic device 100.

  In the above description, the GUI button is used as an example of the GUI component. However, the level of the operation direction in addition to the voice guidance of the slider operation content using the slider-type GUI component is described. It may be voice-guided whether or not the information changes.

  In the above description, the electronic device 100 includes the display panel 160. However, the electronic device 100 may not include the display panel 160. This is because the operation content can be recognized by vibration and voice.

  In the above description, the vibration pattern is changed between the peripheral area and the GUI button display area. However, the vibration element 140 may not be driven in the peripheral area. In this case, when the vibration starts, it can be recognized that the fingertip has entered the display area of the GUI button.

  In the above description, the buttons representing the operation details of the buttons 11A to 11H have been described with a closed curve. However, the buttons representing the operation details of the buttons 11A to 11H may not be surrounded with the closed curve.

  In such a case, character recognition is performed without extracting a closed curve, and a switch using a GUI button may be generated for a word that is matched with a word included in the word list of the control data. The characters for which collation has not been established may be displayed around the GUI button as other information.

  In the above description, the control data includes the word list, and the form of collating the character recognized character with the word has been described. However, the control data may include a list of numbers in addition to the word list. Then, GUI buttons may be generated for words and numbers that have been verified.

  FIG. 14 is a diagram illustrating an electronic device 100 according to a modification of the embodiment. The electronic device 100 shown in FIG. 14 can call registration data registered in the past. The display panel 160 has a GUI button 164A for selecting a function started at XX month x day △, a GUI button 164B for selecting a function started at XX month △ day, and stopped at △ month x day x time. A GUI button 164C for selecting a function is displayed.

  Also, voice data for voice guidance of contents displayed by the GUI buttons 164A, 164B, 164C is associated.

  For this reason, for example, when the user operates the GUI button 164 </ b> B, the voice guidance “call the function started at time of × month Δ day ○” is output from the speaker 103.

  FIG. 15 is a diagram showing the screen of the electronic device 100 called by the function described in FIG. 14 and the washing machine 10. GUI buttons 161A to 161H based on the called registration data are displayed on the display panel 160 of the electronic device 100, and the user operates the washing machine 10 using the called registration data using the electronic device 100. Can do.

  The contents of the voice guidance may be recorded by the user. The user's voice, favorite music, or a sound combining these may be recorded, and the contents displayed by the GUI buttons 164A, 164B, 164C may be voice-guided.

  FIG. 16 is a diagram illustrating an electronic device 100 according to a modification of the embodiment. The GUI buttons 163A to 163H may be displayed on the display panel 160 in an arrangement different from the buttons 11A to 11H of the operation unit 11 of the washing machine 10, such as GUI buttons 163A to 163H shown in FIG. The GUI buttons 163A to 163H correspond to the GUI buttons 161A to 161H shown in FIG. Moreover, you may display in the shape different from the buttons 11A-11H of the operation part 11 of the washing machine 10 like GUI buttons 163A-163H.

  FIG. 17 is a diagram illustrating an image acquisition method according to a modification of the embodiment.

  When the image of the operation unit 11 of the washing machine 10 illustrated in FIG. 17A is acquired by the electronic device 100, if the operation unit 11 does not fit in one image, for example, as illustrated in FIG. In addition, the operation unit 11 may be captured in a plurality of images. In FIG. 17B, as an example, the operation unit 11 is imaged by dividing it into three or more images.

  Then, by combining a plurality of images by image processing, the entire image of the imaging unit 11 may be processed into one photograph as shown in FIG. If the processing shown in FIG. 13 is performed based on the photograph thus obtained, the electronic device 100 can generate registration data.

  FIG. 18 is a diagram illustrating an electronic device 100 according to a modification of the embodiment. In the electronic device 100 shown in FIG. 18, the GUI button 161H as a stop button cannot be acquired. This may occur when the character “stop” cannot be correctly recognized in character recognition and collation is not established, or when the word “stop” is not included in the word list of the control data. It is. In such a case, the function of the stop button cannot be used.

  In FIG. 18, the word of the GUI button 161A is “person” instead of “on”. This is a case where the character “ON” in the operation unit 11 of the washing machine 10 is mistakenly recognized as “person” in character recognition, and collation is established for the word “person”. Since such a “person” GUI button 161A is not assigned a function of turning on the power, the washing machine 10 cannot be turned on.

  In such a case, the electronic device 100 outputs a voice guidance “please re-shoot” from the speaker 103 to prompt the user to re-acquire the photo.

  FIG. 19 is a diagram illustrating an electronic device 100 according to a modification of the embodiment. In the electronic device 100 shown in FIG. 19, the GUI button 161 </ b> B as the “OFF” button cannot be acquired. This is because the word “OFF” representing the GUI button 161B as the “OFF” button has the value of the essential function flag set to “1”.

  In such a case, the electronic device 100 outputs a voice guidance “please re-shoot” from the speaker 103 to prompt the user to re-acquire the photo.

  FIG. 20 is a diagram illustrating a method of using electronic device 100 according to a modification of the embodiment.

  In FIG. 20, the electronic device 100 can download one or a plurality of registration data from the server 50 and store it in the memory 250. In this case, registration data generated from an image by the electronic device 100 may also be stored in the memory 250.

  The registration data held by the server 50 is allocated for each type of electrical appliance. In FIG. 20, the registration data of the electrical appliance of the type AAA-111 and the electrical appliance of BBB-222 is shown.

  The user can access the server 50, input the type of appliance to be used by the user, download registration data from the server 50, and use the electronic device 100. For this reason, the electronic device 100 can be easily used as a remote controller for various electrical appliances.

  FIG. 21 is a diagram illustrating a method of using electronic device 100 according to a modification of the embodiment. The washing machine 10 shown in FIG. 21 has a built-in RFID tag 15. In the IC chip of the RFID tag 15, data representing the types and operation contents of the switches 11 </ b> A to 11 </ b> H of the operation unit 11 is registered.

  The user reads the data registered in the IC chip of the RFID tag 15 by short-range wireless communication, performs a collation process on the read data, and generates GUI buttons 161A to 161H.

  Even with such a technique, the electronic device 100 can be easily used as a remote controller for an electrical appliance. In this case, it is not necessary to acquire a photograph of the operation unit 11 with the camera 104, and it is not necessary to perform character recognition.

  Further, data indicating the type of the washing machine 10 is registered in the IC chip of the RFID tag 15, and the data indicating the type read by the electronic device 100 from the RFID tag 15 is used to change the model from the server 50 shown in FIG. 20. Corresponding registration data may be downloaded.

  In addition, the washing machine 10 may hold registration data instead of the RFID tag 15, and the electronic device 100 may download the registration data from the washing machine 10 by wireless communication using a wireless LAN (Local Area Network) or the like. .

Although the electronic device and the driving method of the electronic device according to the exemplary embodiments of the present invention have been described above, the present invention is not limited to the specifically disclosed embodiments, and is claimed. Various modifications and changes can be made without departing from the scope.
Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)
An electronic device for operating a predetermined electrical appliance,
A display unit;
A top panel disposed on the display surface side of the display unit and having an operation surface;
A position detection unit for detecting a position of an operation input performed on the operation surface;
A vibration element that generates the natural vibration on the operation surface by being driven by a drive signal that generates the natural vibration of an ultrasonic band on the operation surface;
A transmission unit for transmitting an operation command to the electrical appliance;
An audio output unit;
An imaging unit that captures an image of the operation unit of the appliance;
A character recognition unit that recognizes characters included in the image of the operation unit captured by the imaging unit;
Control data associating a predetermined word or number representing the operation content of the electrical appliance, the operation content represented by the predetermined word or number, and the voice guidance representing the predetermined word or number or the operation content is stored. 1 storage unit;
A collation unit that collates characters recognized by the character recognition unit with words or numbers included in the control data stored in the first storage unit;
A display control unit for displaying a GUI part representing a word or a number for which the collation is established on the display unit;
When the position of the operation input detected by the position detection unit enters the display area of the GUI component, the intensity of the natural vibration changes according to the temporal change degree of the position of the operation input. A drive controller for driving the vibration element with the drive signal;
An electronic device comprising: a voice guidance control unit that outputs the voice guidance to the voice output unit when a position of the operation input detected by the position detection unit falls within a display area of the GUI component.
(Appendix 2)
When a confirmation operation for confirming the operation of the GUI component is performed, the command generation unit further includes a command generation unit that generates the operation command representing the operation content corresponding to the word or the number for which the collation is established, and transmits the operation command to the transmission unit. The electronic device according to appendix 1.
(Appendix 3)
The electronic device according to claim 1 or 2, wherein the character recognition unit recognizes a character surrounded by a closed curve in the image of the operation unit captured by the imaging unit.
(Appendix 4)
The first storage unit stores, as the control data, the predetermined word or number, the operation content, the voice guidance, and the vibration pattern of the drive signal in association with each other.
When the position of the operation input detected by the position detection unit enters the display area of the GUI component, the drive control unit determines the intensity of the natural vibration according to the temporal change degree of the position of the operation input. The electronic device according to any one of appendices 1 to 3, wherein the vibration element is driven by the drive signal based on the vibration pattern corresponding to the word or number for which the collation is established so as to change.
(Appendix 5)
When the character included in the image is not recognized or when the collation is not established, the drive control unit does not drive the vibration element, and the voice guidance control unit outputs the voice guidance to the voice output unit. The electronic device according to any one of supplementary notes 1 to 4, which is not allowed to be performed.
(Appendix 6)
When a character included in the image is not recognized, or when the verification is not established, further includes a request unit that requests re-imaging of the image of the operation unit,
The character recognition unit recognizes a character included in the image re-imaged by the imaging unit;
The collation unit collates characters recognized from the image re-captured by the character recognition unit and words or numbers included in the control data stored in the first storage unit,
The display control unit displays on the display unit a GUI component representing a word or a number for which the collation is established for characters recognized from the re-imaged image,
When the character included in the image is recognized and the collation is established, the drive control unit determines the time of the position of the operation input when the position of the operation input falls within the display area of the GUI part. Appendix 5 wherein the vibration element is driven by the drive signal so that the intensity of the natural vibration changes according to the degree of change, and the voice guidance control unit causes the voice output unit to output the voice guidance. Electronic equipment.
(Appendix 7)
The predetermined word or number associated by the control data includes a word or number that is essential to establish the collation,
If the collation for the essential word or number is not established, the drive control unit does not drive the vibration element, and the voice guidance control unit does not cause the voice output unit to output the voice guidance. The electronic device according to claim 6.
(Appendix 8)
The drive control unit drives the vibration element and the voice guidance control unit causes the voice output unit to output the voice guidance when the collation for the essential word or number is established. Electronics.
(Appendix 9)
A second storage unit that stores, as registration data, a word or a number that has been verified;
A reading unit that reads a predetermined word or number from the registration data based on the operation input;
A GUI component representing the word or number read by the reading unit is displayed on the display unit on the display control unit, and the position of the operation input detected by the position detection unit is the read word or number. Is entered into the display area of the GUI part representing the above, the drive control unit is supplied with the drive signal so that the intensity of the natural vibration changes according to the degree of temporal change in the position of the operation input. The electronic device according to any one of appendices 1 to 8, further comprising: an overall processing unit that is driven and causes the voice guidance control unit to output the voice guidance to the voice output unit.
(Appendix 10)
An electronic device for operating a predetermined electrical appliance,
A top panel having an operation surface;
A position detection unit for detecting a position of an operation input performed on the operation surface;
A vibration element that generates the natural vibration on the operation surface by being driven by a drive signal that generates the natural vibration of an ultrasonic band on the operation surface;
A transmission unit for transmitting an operation command to the electrical appliance;
An audio output unit;
An imaging unit that captures an image of the operation unit of the appliance;
A character recognition unit that recognizes characters included in the image of the operation unit captured by the imaging unit;
Control data associating a predetermined word or number representing the operation content of the electrical appliance, the operation content represented by the predetermined word or number, and the voice guidance representing the predetermined word or number or the operation content is stored. 1 storage unit;
A collation unit that collates characters recognized by the character recognition unit with words or numbers included in the control data stored in the first storage unit;
An assigning unit that assigns a predetermined area of the top panel to the word or number for which the matching has been established;
When the position of the operation input detected by the position detection unit enters the predetermined region, the vibration element is changed so that the intensity of the natural vibration changes according to the degree of temporal change in the position of the operation input. A drive control unit for driving the drive signal with the drive signal;
An electronic device comprising: a voice guidance control unit that outputs the voice guidance to the voice output unit when the position of the operation input detected by the position detection unit falls within the predetermined area.
(Appendix 11)
An electronic device for operating a predetermined electrical appliance,
A display unit;
A top panel disposed on the display surface side of the display unit and having an operation surface;
A position detection unit for detecting a position of an operation input performed on the operation surface;
A vibration element that generates the natural vibration on the operation surface by being driven by a drive signal that generates the natural vibration of an ultrasonic band on the operation surface;
A transmission unit for transmitting the electrical appliance and the operation command;
An audio output unit;
A reading unit for reading an identifier of the appliance;
Control data associating the identifier, a predetermined word or number representing the operation content of the electrical appliance, the operation content represented by the predetermined word or number, and the voice guidance representing the predetermined word or number or the operation content A storage unit for storing
A display control unit that displays a GUI component representing the predetermined word or number associated in the control data with the identifier read by the reading unit on the display unit;
When the position of the operation input detected by the position detection unit enters the display area of the GUI component, the intensity of the natural vibration changes according to the temporal change degree of the position of the operation input. A drive controller for driving the vibration element with the drive signal;
A voice guidance control unit for outputting the voice guidance to the voice output unit;
Including electronic equipment.
(Appendix 12)
A method of driving an electronic device that operates a predetermined electrical appliance,
The electronic device is
A display unit;
A top panel disposed on the display surface side of the display unit and having an operation surface;
A position detection unit for detecting a position of an operation input performed on the operation surface;
A vibration element that generates the natural vibration on the operation surface by being driven by a drive signal that generates the natural vibration of an ultrasonic band on the operation surface;
A transmission unit for transmitting an operation command to the electrical appliance;
An audio output unit;
An imaging unit that captures an image of the operation unit of the appliance;
A character recognition unit that recognizes characters included in the image of the operation unit captured by the imaging unit;
Control data associating a predetermined word or number representing the operation content of the electrical appliance, the operation content represented by the predetermined word or number, and the voice guidance representing the predetermined word or number or the operation content is stored. 1 storage unit;
A collation unit that collates characters recognized by the character recognition unit with words or numbers included in the control data stored in the first storage unit;
A display control unit for displaying a GUI part representing a word or a number for which the collation is established on the display unit;
When the position of the operation input detected by the position detection unit enters the display area of the GUI component, the intensity of the natural vibration changes according to the temporal change degree of the position of the operation input. A method for driving an electronic device, wherein a vibration element is driven by the drive signal, and the voice guidance is output to the voice output unit.

DESCRIPTION OF SYMBOLS 100 Electronic device 110 Case 120 Top panel 130 Double-sided tape 140 Vibration element 150 Touch panel 160 Display panel 170 Board | substrate 200 Control apparatus 220 Application processor 221 Main control part 222 Character recognition part 223 Verification part 224 Display control part 225 Voice guidance control part 230 Communication Processor 240 Drive controller 250 Memory 300 Drive controller 310 Sine wave generator 320 Amplitude modulator

Claims (12)

An electronic device for operating a predetermined electrical appliance,
A display unit;
A top panel disposed on the display surface side of the display unit and having an operation surface;
A position detection unit for detecting a position of an operation input performed on the operation surface;
A vibration element that generates the natural vibration on the operation surface by being driven by a drive signal that generates the natural vibration of an ultrasonic band on the operation surface;
A transmission unit for transmitting an operation command to the electrical appliance;
An audio output unit;
An imaging unit that captures an image of the operation unit of the appliance;
A character recognition unit that recognizes characters included in the image of the operation unit captured by the imaging unit;
Control data associating a predetermined word or number representing the operation content of the electrical appliance, the operation content represented by the predetermined word or number, and the voice guidance representing the predetermined word or number or the operation content is stored. 1 storage unit;
A collation unit that collates characters recognized by the character recognition unit with words or numbers included in the control data stored in the first storage unit;
A display control unit for displaying a GUI part representing a word or a number for which the collation is established on the display unit;
When the position of the operation input detected by the position detection unit enters the display area of the GUI component, the intensity of the natural vibration changes according to the temporal change degree of the position of the operation input. A drive controller for driving the vibration element with the drive signal;
An electronic device comprising: a voice guidance control unit that outputs the voice guidance to the voice output unit when a position of the operation input detected by the position detection unit falls within a display area of the GUI component.   When a confirmation operation for confirming the operation of the GUI component is performed, the command generation unit further includes a command generation unit that generates the operation command representing the operation content corresponding to the word or the number for which the verification is established, and transmits the operation command to the transmission unit The electronic device according to claim 1.   The electronic device according to claim 1, wherein the character recognition unit recognizes a character surrounded by a closed curve in the image of the operation unit captured by the imaging unit. The first storage unit stores, as the control data, the predetermined word or number, the operation content, the voice guidance, and the vibration pattern of the drive signal in association with each other.
When the position of the operation input detected by the position detection unit enters the display area of the GUI component, the drive control unit determines the intensity of the natural vibration according to the temporal change degree of the position of the operation input. 4. The electronic device according to claim 1, wherein the vibration element is driven by the drive signal based on the vibration pattern corresponding to the word or the number for which the collation is established so as to change. 5.   When the character included in the image is not recognized or when the collation is not established, the drive control unit does not drive the vibration element, and the voice guidance control unit outputs the voice guidance to the voice output unit. The electronic device according to any one of claims 1 to 4, wherein the electronic device is not allowed to be used. When a character included in the image is not recognized, or when the verification is not established, further includes a request unit that requests re-imaging of the image of the operation unit,
The character recognition unit recognizes a character included in the image re-imaged by the imaging unit;
The collation unit collates characters recognized from the image re-captured by the character recognition unit and words or numbers included in the control data stored in the first storage unit,
The display control unit displays on the display unit a GUI component representing a word or a number for which the collation is established for characters recognized from the re-imaged image,
When the character included in the image is recognized and the collation is established, the drive control unit determines the time of the position of the operation input when the position of the operation input falls within the display area of the GUI part. The said vibration element is driven with the said drive signal so that the intensity | strength of the said natural vibration changes according to a degree of change, The said voice guidance control part outputs the said voice guidance to the said voice output part. The electronic device described. The predetermined word or number associated by the control data includes a word or number that is essential to establish the collation,
The said drive control part does not drive the said vibration element, and the said voice guidance control part does not make the said voice guidance output the said voice guidance when the collation about the said essential word or number is not materialized. The electronic device as described in any one of thru | or 6.   The said drive control part drives the said vibration element when the said collation about the said essential word or number is materialized, The said voice guidance control part makes the said voice output part output the said voice guidance. Electronic equipment. A second storage unit that stores, as registration data, a word or a number that has been verified;
A reading unit that reads a predetermined word or number from the registration data based on the operation input;
A GUI component representing the word or number read by the reading unit is displayed on the display unit on the display control unit, and the position of the operation input detected by the position detection unit is the read word or number. Is entered into the display area of the GUI part representing the above, the drive control unit is supplied with the drive signal so that the intensity of the natural vibration changes according to the degree of temporal change in the position of the operation input. The electronic device according to any one of claims 1 to 8, further comprising: an overall processing unit that is driven and causes the voice guidance control unit to output the voice guidance to the voice output unit. An electronic device for operating a predetermined electrical appliance,
A top panel having an operation surface;
A position detection unit for detecting a position of an operation input performed on the operation surface;
A vibration element that generates the natural vibration on the operation surface by being driven by a drive signal that generates the natural vibration of an ultrasonic band on the operation surface;
A transmission unit for transmitting an operation command to the electrical appliance;
An audio output unit;
An imaging unit that captures an image of the operation unit of the appliance;
A character recognition unit that recognizes characters included in the image of the operation unit captured by the imaging unit;
Control data associating a predetermined word or number representing the operation content of the electrical appliance, the operation content represented by the predetermined word or number, and the voice guidance representing the predetermined word or number or the operation content is stored. 1 storage unit;
A collation unit that collates characters recognized by the character recognition unit with words or numbers included in the control data stored in the first storage unit;
An assigning unit that assigns a predetermined area of the top panel to the word or number for which the matching has been established;
When the position of the operation input detected by the position detection unit enters the predetermined region, the vibration element is changed so that the intensity of the natural vibration changes according to the degree of temporal change in the position of the operation input. A drive control unit for driving the drive signal with the drive signal;
An electronic device comprising: a voice guidance control unit that outputs the voice guidance to the voice output unit when the position of the operation input detected by the position detection unit falls within the predetermined area. An electronic device for operating a predetermined electrical appliance,
A display unit;
A top panel disposed on the display surface side of the display unit and having an operation surface;
A position detection unit for detecting a position of an operation input performed on the operation surface;
A vibration element that generates the natural vibration on the operation surface by being driven by a drive signal that generates the natural vibration of an ultrasonic band on the operation surface;
A transmission unit for transmitting the electrical appliance and the operation command;
An audio output unit;
A reading unit for reading an identifier of the appliance;
Control data associating the identifier, a predetermined word or number representing the operation content of the electrical appliance, the operation content represented by the predetermined word or number, and the voice guidance representing the predetermined word or number or the operation content A storage unit for storing
A display control unit that displays a GUI component representing the predetermined word or number associated in the control data with the identifier read by the reading unit on the display unit;
When the position of the operation input detected by the position detection unit enters the display area of the GUI component, the intensity of the natural vibration changes according to the temporal change degree of the position of the operation input. A drive controller for driving the vibration element with the drive signal;
A voice guidance control unit for outputting the voice guidance to the voice output unit;
Including electronic equipment. A method of driving an electronic device that operates a predetermined electrical appliance,
The electronic device is
A display unit;
A top panel disposed on the display surface side of the display unit and having an operation surface;
A position detection unit for detecting a position of an operation input performed on the operation surface;
A vibration element that generates the natural vibration on the operation surface by being driven by a drive signal that generates the natural vibration of an ultrasonic band on the operation surface;
A transmission unit for transmitting an operation command to the electrical appliance;
An audio output unit;
An imaging unit that captures an image of the operation unit of the appliance;
A character recognition unit that recognizes characters included in the image of the operation unit captured by the imaging unit;
Control data associating a predetermined word or number representing the operation content of the electrical appliance, the operation content represented by the predetermined word or number, and the voice guidance representing the predetermined word or number or the operation content is stored. 1 storage unit;
A collation unit that collates characters recognized by the character recognition unit with words or numbers included in the control data stored in the first storage unit;
A display control unit for displaying a GUI part representing a word or a number for which the collation is established on the display unit;
When the position of the operation input detected by the position detection unit enters the display area of the GUI component, the intensity of the natural vibration changes according to the temporal change degree of the position of the operation input. A method for driving an electronic device, wherein a vibration element is driven by the drive signal and the voice guidance is output to the voice output unit.

Images