JP2013236136A - Electronic apparatus, control method, and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a user to easily recognize a specific sound from among multiple output sounds.SOLUTION: In one embodiment, an electronic apparatus (for example, a portable phone 1A) includes: a piezoelectric element; a sound generation part that is vibrated by the piezoelectric element and generates air conduction sounds and vibration sounds transmitted by vibrating a part of a human body; a display on which a screen of image contents including sounds is reproduced and displayed; and a detection part detecting a contact position of the part of the human body with the sound generation part. The sound generation part generates sounds that are included in the image contents and is assigned to a position on the screen corresponding to the contact position in a case where the contact of the part of the human body with the sound generation part is detected when the image contents are reproduced on the display.

Description

  The present application relates to an electronic device, a control method, and a control program.

  Patent Document 1 describes an electronic device that transmits air conduction sound and vibration sound to a user. Patent Document 1 describes that when a voltage is applied to a piezoelectric element of a vibrating body arranged on the outer surface of a housing of an electronic device, the vibrating body flexes and vibrates due to expansion and contraction of the piezoelectric element. Further, Patent Document 1 describes that when a user brings a vibrating body that flexes and vibrates into contact with the auricle, air conduction sound and vibration sound are transmitted to the user. According to Patent Document 1, air conduction sound is sound that is transmitted to the eardrum through the ear canal through the vibration of the air due to the vibration of the object, and is transmitted to the user's auditory nerve by the vibration of the eardrum. According to Patent Document 1, the vibration sound is a sound transmitted to the user's auditory nerve through a part of the user's body (for example, cartilage of the outer ear) that contacts the vibrating object.

JP 2005-348193 A

  By the way, in general, in an electronic device, when playing back video such as moving image content, if a plurality of (multiple channels) sounds are output in accordance with the video to be played back, a user is output. There is a need to listen to a specific sound from multiple sounds.

  An electronic apparatus according to one aspect includes a piezoelectric element, a sound generating unit that generates vibration sound transmitted by vibrating a part of a human body by vibrating by the piezoelectric element, and image content including sound. A display on which the screen is reproduced and displayed, and a detection unit that detects a contact position of a part of the human body with respect to the sound generation unit, and the sound generation unit when the image content is reproduced on the display When a partial contact of the human body is detected, the sound generation unit generates a sound that is included in the image content and that is associated with a position on the screen that corresponds to the contact position Let

  A control method according to one aspect includes a piezoelectric element, a sound generating unit that generates vibration sound transmitted by vibrating a part of a human body by vibrating by the piezoelectric element, and image content including sound. A control method executed by an electronic device comprising a display on which a screen of the above is reproduced and a detection unit that detects a contact position of a part of the human body with respect to the sound generation unit, wherein the image content is displayed on the display. If a partial contact of the human body with the sound generation unit is detected during reproduction, the sound is included in the image content and is associated with a position on the screen corresponding to the contact position. Generating a sound generated by the sound generator.

  A control program according to one aspect includes a piezoelectric element, a sound generator that vibrates by the piezoelectric element, generates a sound that is transmitted by vibrating a part of a human body, and image content including sound. When the image content is being reproduced on the display, the electronic device includes a display on which the screen is reproduced and displayed and a detection unit that detects a contact position of a part of the human body with respect to the sound generation unit. When a contact of a part of the human body with the generation unit is detected, a sound that is included in the image content and that is associated with a position on the screen corresponding to the contact position is detected by the sound generation unit. The step to be generated is executed.

FIG. 1 is a front view of a mobile phone according to an embodiment. FIG. 2 is a rear view of the mobile phone. FIG. 3 is a cross-sectional view of the mobile phone according to the embodiment. FIG. 4 is a diagram illustrating an example of the shape of the panel. FIG. 5 is a diagram showing an example of panel vibration. FIG. 6 is a block diagram of the mobile phone according to the embodiment. FIG. 7 is a diagram illustrating an example of a state in which a moving image being shot by the out camera is displayed on the display. FIG. 8 is a diagram illustrating an example of coordinate values stored in the setting data. FIG. 9 shows the sound associated with the vicinity of the position on the screen corresponding to the position of the ear in contact with the panel when the contact of the user's ear with the panel is detected while playing the moving image content. It is a figure which shows the outline | summary of the process of the mobile telephone which emphasizes and outputs. FIG. 10 is a diagram illustrating an example of a processing procedure for storing coordinate values on a screen corresponding to positions on a moving image that can be regarded as transmission points of a plurality of input sounds during moving image shooting. FIG. 11 shows the sound associated with the vicinity of the position on the screen corresponding to the position of the ear in contact with the panel when the contact of the user's ear with the panel is detected while playing the moving image content. It is a figure which shows an example of the process sequence which emphasizes and outputs. FIG. 12 is a diagram illustrating an outline of processing of a mobile phone that emphasizes and outputs a sound having a sound source in the vicinity of the contact position of the ear according to the magnitude of the pressure when the user's ear contacts the panel. is there. FIG. 13 is a diagram illustrating an example of a processing procedure for emphasizing and outputting a sound having a sound source in the vicinity of the contact position of the ear according to the amount of pressing when the user's ear contacts the panel. FIG. 14 is a diagram illustrating an example of a positional relationship between the ear and the panel when the user's ear and the panel are in contact with each other. FIG. 15 is a diagram showing an area on the moving image content where sound is output when the ear and the panel are in the positional relationship shown in FIG. FIG. 16 is a front view of the mobile phone according to the fifth embodiment. 17 is a cross-sectional view taken along line cc of the mobile phone shown in FIG. FIG. 18 is a front view of a mobile phone according to the sixth embodiment. 19 is a cross-sectional view of the mobile phone shown in FIG. 18 taken along the line dd. FIG. 20 is a diagram illustrating a change example of the frequency characteristic due to the reinforcing member.

  Embodiments for carrying out the present invention will be described in detail with reference to the drawings. Hereinafter, a mobile phone will be described as an example of an electronic device that transmits air conduction sound and vibration sound to a user.

(Embodiment 1)
With reference to FIGS. 1, 2, and 3, the overall configuration of the mobile phone 1 </ b> A according to the embodiment will be described. FIG. 1 is a front view of the mobile phone 1A. FIG. 2 is a rear view of the mobile phone 1A. FIG. 3 is a cross-sectional view schematically showing an aa cross section of the mobile phone 1A. As shown in FIGS. 1, 2, and 3, the mobile phone 1A includes a display 2, a button 3, an illuminance sensor 4, a proximity sensor 5, a piezoelectric element 7, a first microphone 8a, and a second microphone. 8b, a third microphone 8c, a speaker 11, an in camera 12, an out camera 13, a panel 20, and a housing 40.

  The display 2 is a liquid crystal display (LCD: Liquid Crystal Display), an organic EL display (OELD: Organic Electro-Luminescence Display), or an inorganic EL display (IELD: Inorganic Electro-Luminescence Display). The display 2 displays characters, images, symbols, graphics, and the like.

  The button 3 receives an operation input from the user. The number of buttons 3 is not limited to the example shown in FIGS.

  The illuminance sensor 4 detects the illuminance of ambient light of the mobile phone 1A. Illuminance indicates light intensity, brightness, or luminance. The illuminance sensor 4 is used for adjusting the luminance of the display 2, for example. The proximity sensor 5 detects the presence of a nearby object without contact. The proximity sensor 5 detects the presence of an object based on a change in a magnetic field or a change in a feedback time of an ultrasonic reflected wave. For example, the proximity sensor 5 detects that the display 2 is brought close to the face. The illuminance sensor 4 and the proximity sensor 5 may be configured as one sensor. The illuminance sensor 4 may be used as a proximity sensor.

  When an electric signal (voltage corresponding to a sound signal) is applied, the piezoelectric element 7 expands or contracts or bends according to the electromechanical coupling coefficient of the constituent material. That is, the piezoelectric element 7 is deformed when an electric signal is applied. The piezoelectric element 7 is attached to the panel 20 and used as a vibration source for vibrating the panel 20. The piezoelectric element 7 is formed using, for example, ceramic or quartz. The piezoelectric element 7 may be a unimorph, bimorph, or multilayer piezoelectric element. The multilayer piezoelectric element includes a multilayer bimorph element in which bimorphs are stacked (for example, 16 layers or 24 layers are stacked). The multilayer piezoelectric element is composed of, for example, a multilayer structure including a plurality of dielectric layers made of PZT (lead zirconate titanate) and electrode layers disposed between the plurality of dielectric layers. A unimorph expands and contracts when an electrical signal (voltage) is applied. A bimorph bends when an electrical signal (voltage) is applied.

  The first microphone 8a is a microphone unit having a plurality of unidirectional microphone capsules arranged at a predetermined angle in the horizontal direction, and mainly inputs horizontal sound coming from the shooting direction of the out-camera 13. To do. For example, when shooting is started by starting the out-camera 13 while maintaining the posture of the mobile phone 1A as shown in FIG. 1, the first microphone 8a inputs sound coming from the right side or the left side in the shooting direction. .

  The second microphone 8b is a microphone unit having a plurality of unidirectional microphone capsules arranged at a predetermined angle in the vertical direction, and mainly inputs vertical sound coming from the shooting direction of the out-camera 13. To do. For example, when shooting is started by starting the out-camera 13 while maintaining the posture of the mobile phone 1A in the state shown in FIG. 1, the second microphone 8b inputs sound coming from above or below the shooting direction. To do.

  The third microphone 8c is an input unit that inputs a voice uttered from the user, for example, during a call. For example, the first microphone 8a, the second microphone 8b, and the third microphone 8c convert the input sound into an electrical signal and send it to the controller 10 described later.

  The speaker 11 is a sound output unit that outputs sound by an air conduction method. The speaker 11 is a dynamic speaker, for example, and can transmit a sound obtained by converting an electric signal to a person whose ear is not in contact with the mobile phone 1A. The speaker 11 is used, for example, for outputting sound during reproduction of moving image content or outputting music.

  The in camera 12 is a camera that photographs an object facing the display 2. The out camera 13 is a camera that photographs an object or the like facing away from the display 2. The in-camera 12 and the out-camera 13 convert the captured image into an electrical signal.

  The panel 20 vibrates as the piezoelectric element 7 deforms (expands or bends), and transmits the vibration as sound vibration to an ear cartilage (auricular cartilage) or the like that the user contacts with the panel 20. The panel 20 is an example of a sound generator. The panel 20 also has a function of protecting the display 2 and the piezoelectric element 7 from external force. The panel 20 is formed by synthetic resins, such as glass or an acryl, for example. The shape of the panel 20 is, for example, a plate shape. The panel 20 may be a flat plate. The panel 20 may be a curved panel whose surface is smoothly curved.

  The display 2 and the piezoelectric element 7 are attached to the back surface of the panel 20 by a joining member 30. The piezoelectric element 7 is spaced from the inner surface of the housing 40 by a predetermined distance while being arranged on the back surface of the panel 20. The piezoelectric element 7 may be separated from the inner surface of the housing 60 even in a stretched or bent state. That is, the distance between the piezoelectric element 7 and the inner surface of the housing 40 is preferably larger than the maximum deformation amount of the piezoelectric element 7. The piezoelectric element 7 may be attached to the panel 20 via a reinforcing member (for example, sheet metal or glass fiber reinforced resin). The joining member 30 is, for example, a double-sided tape or an adhesive having thermosetting property or ultraviolet curable property. The joining member 30 may be an optical elastic resin that is a colorless and transparent acrylic ultraviolet curable adhesive.

  The display 2 is disposed approximately in the center of the panel 20 in the short direction. The piezoelectric element 7 is disposed in the vicinity of a predetermined distance from the longitudinal end of the panel 20 so that the longitudinal direction of the piezoelectric element 7 is parallel to the lateral direction of the panel 20. The display 2 and the piezoelectric element 7 are arranged in parallel on the inner surface of the panel 20.

  A touch screen (touch sensor) 21 is disposed on almost the entire outer surface of the panel 20. The touch screen 21 detects contact with the panel 20. The touch screen 21 is used to detect a user's contact operation with a finger, pen, stylus pen, or the like. The gestures detected using the touch screen 21 include, but are not limited to, touch, long touch, release, swipe, tap, double tap, long tap, drag, flick, pinch in, and pinch out. The detection method of the touch screen 21 may be any method such as a capacitance method, a resistive film method, a surface acoustic wave method (or an ultrasonic method), an infrared method, an electromagnetic induction method, and a load detection method.

  The touch screen 21 is also used to detect auricular cartilage or the like that contacts the panel 20 in order to hear sound.

  The housing 40 is formed using resin or metal. The housing 40 supports the button 3, the illuminance sensor 4, the proximity sensor 5, the first microphone 8a, the second microphone 8b, the third microphone 8c, the speaker 11, the camera 12, the panel 20, and the like.

  The sound output by the mobile phone 1A according to the embodiment will be described in more detail with reference to FIGS. FIG. 4 is a diagram illustrating an example of the shape of the panel 20. FIG. 5 is a diagram illustrating an example of the vibration of the panel 20.

  An electric signal corresponding to the output sound is applied to the piezoelectric element 7. For example, ± 15 V higher than ± 5 V, which is an applied voltage of a so-called panel speaker that transmits sound by air conduction sound through the ear canal, may be applied to the piezoelectric element 7. Thereby, even when the user presses a part of his / her body against the panel 20 with a force of 3N or more (5N to 10N force), for example, the panel 20 generates sufficient vibration and is used. Vibration sound transmitted through a part of the person's body can be generated. The voltage applied to the piezoelectric element 7 can be appropriately adjusted according to the fixing strength of the panel 20 with respect to the housing 40 or the performance of the piezoelectric element 7.

  When an electric signal is applied, the piezoelectric element 7 expands or contracts in the longitudinal direction. The panel 20 to which the piezoelectric element 7 is attached is deformed in accordance with expansion / contraction or bending of the piezoelectric element 7. Thereby, the panel 20 vibrates and generates air conduction sound. Further, when the user brings a part of the body (for example, auricular cartilage) into contact with the panel 20, the panel 20 generates vibration sound that is transmitted to the user through the part of the body. That is, as the piezoelectric element 7 is deformed, the panel 20 vibrates at a frequency that is perceived as vibration sound with respect to an object that contacts the panel 20.

  For example, when an electric signal corresponding to the voice of the other party of the call or sound data such as a ring tone or music is applied to the piezoelectric element 7, the panel 20 generates air conduction sound and vibration sound corresponding to the electric signal. Let The sound signal output via the piezoelectric element 7 and the panel 20 may be based on sound data stored in the storage 9 described later. The sound signal output via the piezoelectric element 7 and the panel 20 may be based on sound data stored in an external server or the like and acquired via the network by the communication unit 6 described later.

  In the present embodiment, the panel 20 may be approximately the same size as the user's ear. The panel 20 may be larger than the user's ear, as shown in FIG. In this case, the user can bring the entire outer periphery of the ear into contact with the panel 20 when listening to the sound. Listening to the sound in this way makes it difficult for ambient sounds (noise) to enter the ear canal. In the present embodiment, at least the panel 20 has a length in the longitudinal direction (or short direction) corresponding to the distance between the lower leg of the anti-annulus (lower anti-annular leg) and the anti-tragus, and the pair of the panel 20 from the tragus. A region wider than a region having a length in the short direction (or the longitudinal direction) corresponding to the distance to the ear ring vibrates. The panel 20 has a length in the longitudinal direction (or short direction) corresponding to the distance from the region near the upper leg of the ear ring (upper pair leg) in the ear ring to the earlobe, and the vicinity of the ear ring in the ear ring from the tragus. A region having a length in the short direction (or the longitudinal direction) corresponding to the distance to the part may vibrate. The region having the above length and width may be a rectangular region, or an elliptical shape in which the length in the longitudinal direction is the major axis and the length in the lateral direction is the minor axis. Also good. The average size of the human ear can be known, for example, by referring to the Japanese human body size database (1992-1994) created by the Human Life Engineering Research Center (HQL).

  As shown in FIG. 5, the panel 20 vibrates not only in the attachment region 20a to which the piezoelectric element 7 is attached, but also in a region away from the attachment region 20a. The panel 20 has a plurality of locations that vibrate in a direction intersecting the main surface of the panel 20 in the vibrating region, and in each of the plurality of locations, the value of the amplitude of vibration changes from positive to negative with time, Or vice versa. At each moment, the panel 20 vibrates at a seemingly random or regular distribution in a substantially entire portion of the panel 20 with a portion having a relatively large vibration amplitude and a portion having a relatively small vibration amplitude. That is, vibrations of a plurality of waves are detected over the entire panel 20. If the voltage applied to the piezoelectric element 7 is ± 15 V as described above, even if the user presses the panel 20 against his / her body with a force of 5N to 10N, for example, The vibrations that occur are difficult to attenuate. For this reason, even if a user makes an ear contact the area | region away from the attachment area | region 20a on the panel 20, he can hear a vibration sound.

  In the present embodiment, the display 2 is attached to the panel 20. For this reason, the lower part of the panel 20 (the side on which the display 2 is attached) is increased in rigidity, and the vibration is smaller than the upper part of the panel 20 (the side on which the piezoelectric element 7 is attached). For this reason, the sound leakage of the air conduction sound due to the vibration of the panel 20 in the lower part of the panel 20 is reduced.

  The cellular phone 1 </ b> A can transmit air conduction sound and vibration sound via a part of the user's body (for example, auricular cartilage) to the user by the vibration of the panel 20. Therefore, when the mobile phone 1A outputs a sound having a volume equivalent to that of the dynamic receiver, the sound transmitted to the surroundings of the mobile phone 1A due to air vibration may be reduced compared to an electronic device having only a dynamic speaker. it can. Such a feature is suitable, for example, when listening to a recorded message in a place where there is another person nearby, such as in a train.

  Furthermore, the mobile phone 1 </ b> A transmits a vibration sound to the user by the vibration of the panel 20. Therefore, even if the user wears the earphone or the headphone, the user can hear the vibration sound due to the vibration of the panel 20 through the earphone or the headphone and a part of the body by bringing the mobile phone 1A into contact therewith. it can.

  Furthermore, the mobile phone 1 </ b> A transmits sound by the vibration of the panel 20. Therefore, when the mobile phone 1A does not include a separate dynamic receiver (for example, the speaker 11), it is not necessary to form an opening (sound emission port) in the housing 40 for transmitting the sound generated by the panel 20 to the outside. For this reason, when realizing a waterproof structure, the structure can be simplified. When it is necessary to form an opening such as a sound outlet of a dynamic speaker in the housing 40, the cellular phone 1A closes the opening with a member that allows gas to pass but not liquid to realize a waterproof structure. A structure may be adopted. An example of the member that allows gas to pass but not liquid is Gore-Tex (registered trademark).

  The functional configuration of the mobile phone 1A will be described with reference to FIG. FIG. 6 is a block diagram of the mobile phone 1A. As shown in FIG. 6, the mobile phone 1A includes a display 2, a button 3, an illuminance sensor 4, a proximity sensor 5, a communication unit 6, a piezoelectric element 7, a first microphone 8a, and a second microphone 8b. A third microphone 8 c, a storage 9, a controller 10, a speaker 11, an in-camera 12, an out-camera 13, a posture detection unit 15, a vibrator 18, and a touch screen 21.

  The communication unit 6 communicates wirelessly. The communication method supported by the communication unit 6 is a wireless communication standard. Examples of wireless communication standards include cellular phone communication standards such as 2G, 3G, and 4G. As cellular phone communication standards, for example, LTE (Long Term Evolution), W-CDMA (Wideband Code Multiple Access), CDMA2000, PDC (Personal Digital Cellular, GSM (registered trademark) mmloS). (Personal Handy-phone System). As wireless communication standards, for example, there are WiMAX (Worldwide Interoperability for Microwave Access), IEEE802.11, Bluetooth (registered trademark), IrDA (Infrared Data Association), NFC (NearCo), etc. The communication unit 6 may support one or more of the communication standards described above.

  The storage 9 stores programs and data. The storage 9 is also used as a work area for temporarily storing the processing result of the controller 10. The storage 9 may include any non-transitory storage medium such as a semiconductor storage medium and a magnetic storage medium. The storage 9 may include a plurality of types of storage media. The storage 9 may include a combination of a portable storage medium such as a memory card, an optical disk, or a magneto-optical disk and a storage medium reader. The storage 9 may include a storage device used as a temporary storage area such as a RAM (Random Access Memory).

  The programs stored in the storage 9 include an application executed in the foreground or the background and a control program that supports the operation of the application. For example, the application displays a screen on the display 2 and causes the controller 10 to execute processing according to the gesture detected by the touch screen 21. The control program is, for example, an OS. The application and the control program may be installed in the storage 9 via wireless communication by the communication unit 6 or a non-transitory storage medium.

  The storage 9 stores, for example, a control program 9A, a call application 9B, a music playback application 9C, a video playback application 9D, video content 9E, and setting data 9Z. The call application 9B provides a call function for a call by wireless communication. The music playback application 9C provides a music playback function for playing back sound from music data. The video playback application 9D provides a video playback function for playing back video and sound from the video data of the video content 9E. The setting data 9Z includes information related to various settings and processes related to the operation of the mobile phone 1A. The setting data 9Z is information on various settings and processes related to the operation of the mobile phone 1A, for example, on the moving image content 9E that can be regarded as a sound transmission point for each of a plurality of sounds included in the moving image content 9E. The coordinate value on the screen corresponding to the position is included. This coordinate value corresponds to the coordinate value on the screen when the moving image content 9E is reproduced and displayed on the display 2.

  The control program 9A provides functions related to various controls for operating the mobile phone 1A. The control program 9A realizes a call by controlling the communication unit 6, the piezoelectric element 7, the third microphone 8c, and the like, for example. The function provided by the control program 9A may be used in combination with a function provided by another program such as the call application 9B.

  The control program 9A includes, for example, the strength of sound pressure of sound input to the first microphone 8a and the second microphone 8b for each of a plurality of sounds input from the first microphone 8a and the second microphone 8b during moving image shooting. A function for narrowing down the position on the moving image that can be regarded as the transmission point of the sound based on the above and storing the coordinate value on the screen corresponding to the narrowed position in the setting data 9Z of the storage 9 is included. This coordinate value corresponds to the coordinate value on the screen when the video during moving image shooting is reproduced and displayed on the display 2.

  Furthermore, the control program 9 </ b> A includes a function for detecting the position of the ear that contacts the panel 20 using the touch screen 21, for example.

  Further, in the control program 9A, when the moving image content 9E is reproduced and displayed on the display 2, the vicinity of the position on the screen corresponding to the position of the ear that touches the panel 20 among the plurality of sounds included in the moving image content 9E. Includes a function for emphasizing and outputting a sound having a virtual sound source by vibrating the panel 20.

  The controller 10 is an arithmetic processing device. The arithmetic processing unit includes, for example, a CPU (Central Processing Unit), an SoC (System-on-a-chip), an MCU (Micro Control Unit), and an FPGA (Field-Programmable Gate Array), but is not limited thereto. The controller 10 implements various functions by comprehensively controlling the operation of the mobile phone 1A.

  Specifically, the controller 10 executes instructions included in the program stored in the storage 9 while referring to the data stored in the storage 9 as necessary. And the controller 10 controls a function part according to data and a command, and implement | achieves various functions by it. The functional unit includes, for example, the display 2, the communication unit 6, the piezoelectric element 7, the microphone 8, the speaker 11, and the vibrator 18, but is not limited thereto. The controller 10 may change the control according to the detection result of the detection unit. The detection unit includes, for example, the button 3, the illuminance sensor 4, the proximity sensor 5, the camera 12, the posture detection unit 15, and the touch screen 21, but is not limited thereto.

  For example, by executing the control program 9A, the controller 10 starts capturing a plurality of sounds input by the first microphone 8a and the second microphone 8b during moving image shooting, and for each of the plurality of sounds, Based on the strength of the sound pressure of the sound input to the first microphone 8a and the second microphone 8b, the position on the moving image that can be regarded as the transmission point of the sound is narrowed down, and the coordinate value on the screen corresponding to the narrowed position is stored. 9 is stored in the setting data 9Z.

  Further, the controller 10 executes a process of detecting the position of the ear in contact with the panel 20 using the touch screen 21 by executing the control program 9A, for example.

  Further, the controller 10, for example, executes a control program 9 </ b> A, and when a user's ear that contacts the panel 20 is detected when the moving image content 9 </ b> E is being reproduced, Among the above sounds, the sound associated with the vicinity of the position on the screen corresponding to the position of the ear in contact with the panel 20 is emphasized and output by vibrating the panel 20. For example, the controller 10 refers to the setting data 9Z, and uses the coordinate value on the screen corresponding to the sound transmission point of the moving image content 9E, in the vicinity of the position on the screen corresponding to the position of the ear touching the panel 20 The sound associated with is identified, and this sound is emphasized and output. The method of emphasizing includes increasing the volume, for example.

  The posture detection unit 15 detects the posture of the mobile phone 1A. The posture detection unit 15 includes at least one of an acceleration sensor, a direction sensor, and a gyroscope in order to detect the posture. Vibrator 18 vibrates a part or the whole of mobile phone 1A. The vibrator 18 includes, for example, a piezoelectric element or an eccentric motor in order to generate vibration. The vibration generated by the vibrator 18 is used not only to transmit sound but also to notify the user of various events such as incoming calls.

  In FIG. 6, some or all of the programs and data stored in the storage 9 may be downloaded from other devices by wireless communication using the communication unit 6. 6, some or all of the programs and data stored in the storage 9 may be stored in a non-transitory storage medium that can be read by the reading device included in the storage 9. Non-transitory storage media include, for example, optical disks such as CD (registered trademark), DVD (registered trademark), and Blu-ray (registered trademark), magneto-optical disks, magnetic storage media, memory cards, and solid-state storage media Including, but not limited to.

  The configuration of the mobile phone 1A shown in FIG. 6 is an example, and may be changed as appropriate without departing from the scope of the present invention. For example, the mobile phone 1A may include buttons such as a numeric keypad layout or a QWERTY layout as buttons for operations.

  7 and 8, for example, at the time of moving image shooting, for each of a plurality of sounds input by the first microphone 8 a and the second microphone 8 b, the sound of the sound input to the first microphone 8 a and the second microphone 8 b About the outline of the processing of the mobile phone 1A that narrows down the position on the moving image that can be regarded as the transmission point of the sound based on the strength of the pressure, and stores the coordinate value on the screen corresponding to the narrowed position in the setting data 9Z of the storage 9 explain.

  FIG. 7 is a diagram illustrating an example of a state in which a moving image being shot by the out camera 13 is displayed on the display 2. When the moving image shooting process is activated, the mobile phone 1A displays the moving image C1 acquired by the out-camera 13 on the display 2, and starts capturing the sound input by the first microphone 8a and the second microphone 8b. To do. When the mobile phone 1A takes a video, for example, as shown in FIG. 7, for example, CH1 (for example, a chirping of a bird “piyopiyo”), CH2 (for example, a river murmur “sarasara”), CH3 (for example, of trees) 3 types of sounds are captured.

  Subsequently, the mobile phone 1A narrows down the positions on the moving image that can be regarded as the transmission point of the sound based on the strength of the sound pressure for each of the captured sounds. For example, the mobile phone 1A first detects a microphone capsule having the maximum sound pressure of the CH1 sound from the unidirectional microphone capsules of the first microphone 8a and the second microphone 8b, and inputs the sound input direction. Is identified. For example, among the microphone capsules of the first microphone 8a and the second microphone 8b, when the sound pressure of the microphone capsule of the first microphone 8a that inputs sound coming from the left side in the shooting direction is maximum, the sound of CH1 Is specified as the left side of the shooting direction. Alternatively, among the microphone capsules of the first microphone 8a and the second microphone 8b, the microphone capsule of the first microphone 8a that inputs sound coming from the left side of the shooting direction and the sound that comes from the lower side of the shooting direction are input. When the sound pressure of the microphone capsule of the second microphone 8b is maximized, the input direction of the sound of CH1 is specified as the lower left side of the shooting direction. Subsequently, the mobile phone 1A estimates that the origin of the CH1 sound is present in the lower left half area P1 of the moving image C1 displayed on the display 2 in accordance with the input direction of the specified CH1 sound. Then, the cellular phone 1A acquires a coordinate value on the screen corresponding to the center position of the region P1, and stores it in the setting data 9Z in association with the sound of CH1. The coordinate value is, for example, a coordinate value in the XY coordinate system with the origin at the lower left corner of the screen displayed on the display 2.

  Similarly, the mobile phone 1 </ b> A estimates that the origin of the sound of CH <b> 2 exists in the lower left half area P <b> 2 of the moving image C <b> 1 displayed on the display 2. Then, the cellular phone 1A acquires a coordinate value on the screen corresponding to the center position of the region P2, and stores it in the setting data 9Z in association with the sound of CH2.

  Similarly, the cellular phone 1A estimates that the CH3 sound transmission point exists in the upper half area P3 of the moving image C1 displayed on the display 2. Then, the cellular phone 1A acquires a coordinate value on the screen corresponding to the center position of the region P3, and stores it in the setting data 9Z in association with the sound of CH3.

FIG. 8 is a diagram illustrating an example of coordinate values stored in the setting data 9Z. As shown in FIG. 8, the setting data 9 </ b> Z stores sound detection time, sound breakdown, and sound source position in association with each other. The sound detection time is the time when the sound pressure of the sound of CH1 is above a certain value, for example, the time when the sound corresponding to CH1 is input. The breakdown of the sound is, for example, the type of sound input by the first microphone 8a and the second microphone 8b when shooting a moving image. The coordinate value is a screen corresponding to the center position of the area on the moving image C1 that is estimated to include the transmission point of the sound input by the first microphone 8a and the second microphone 8b, as in the outline of the processing described above. This is the upper coordinate value. As shown in FIG. 8, the setting data 9Z includes, for example, a sound detection time “10:11:20 to 10:13:20”, a sound breakdown “CH1 (for example, a small bird cry)”, Coordinate values “X ₁ , Y ₁ ” corresponding to the position of the sound source are stored in association with each other. Similarly, the setting data 9Z corresponds to the sound detection time “10:10:20 to 10:14:20”, the sound breakdown “CH2 (for example, river murmur)”, and the position of the sound source. Coordinate values “X ₂ , Y ₂ ” are stored in association with each other. Similarly, the setting data 9Z corresponds to the sound detection time “10:12:20 to 10:14:00”, the sound breakdown “CH3 (for example, the noise of trees)”, and the position of the sound source. The coordinate values “X ₃ , Y ₃ ” are stored in association with each other.

  The mobile phone 1A stores the moving image C1 captured by the out camera 13 in the storage 9 as the moving image content 9E.

  The outline of the processing of the mobile phone 1A will be described with reference to FIG. The cellular phone 1A applies an audio signal based on the audio data of the moving image content 9E to the piezoelectric element 7. Then, the panel 20 is deformed to generate air conduction sound and vibration sound transmitted through a part of the user's body (for example, cartilage of the outer ear) that contacts the deformed panel 20. When the mobile phone 1A detects the contact of the user's ear with the panel 20 while the moving image content 9E is being reproduced, the mobile phone 1A is positioned at the position of the ear that contacts the panel 20 among the sounds of the plurality of channels included in the moving image content 9E. The sound associated with the corresponding position on the screen is emphasized from other channels.

  FIG. 9 is associated with the vicinity of the position on the screen corresponding to the position of the ear in contact with the panel 20 when the contact of the user's ear with the panel 20 is detected while the moving image content 9E is being reproduced. It is a figure which shows the outline | summary of the process of 1 A of mobile telephones which emphasize and output the sound which is being read.

  As illustrated in FIG. 9, the mobile phone 1A starts to reproduce the moving image content 9E, for example, in response to a user operation (step S11). Subsequently, when the mobile phone 1A detects the position of the ear E1 in contact with the panel 20 using the touch screen 21, the mobile phone 1A generates a sound having a virtual sound source in the vicinity of the position on the screen corresponding to the position of the ear E1. The panel 20 is vibrated and emphasized for output (step S12). For example, as shown in FIG. 9, the mobile phone 1A emphasizes and outputs the sound of CH1 (eg, “piyopiyo”) having a virtual sound source in the vicinity of the position on the screen corresponding to the position of the ear E1. To do.

  Specifically, the cellular phone 1 </ b> A detects the center (center of gravity) of the region where the touch of the ear is detected by the touch screen 21 as the position of the ear E <b> 1. The position of the ear E1 may be any of the vertices of the smallest rectangle including the region where the contact of the ear E1 is detected. Alternatively, the position of the ear E1 may be a position corresponding to a predetermined part of the ear E1. In this case, the position of the ear E1 is calculated from the relative positional relationship with the region where the contact of the ear E1 is detected based on the information regarding the general position of the part in the ear E1.

  Alternatively, the cellular phone 1 </ b> A may detect the position of the ear E <b> 1 by pattern matching between an image obtained based on the detection result of the touch screen 21 at the time of contact with the ear E <b> 1 and a sample prepared in advance. The image obtained based on the detection result of the touch screen 21 divides the detection area of the touch screen 21 into a lattice shape, and converts the detection state of contact of an object in each of the divided areas into a corresponding pixel state. Can be obtained. When the value detected by the touch screen 21 in each region varies depending on, for example, the distance between the touch screen 21 and the object, the pressure with which the object presses the touch screen 21, and the like, an image obtained based on the detection result of the touch screen 21 May be a multi-tone image. The specimen is an image that should be obtained in the same manner as the image obtained based on the detection result of the touch screen 21 in the area where the ear is in contact with the ear. The sample may be an image that should be obtained when the user of the mobile phone 1A touches the ear, or may be an image that should be obtained when the user touches a general ear. A plurality of specimens may be prepared such as a right ear image and a left ear image.

  The method of detecting the position of the ear E1 in contact with the panel 20 using the touch screen 21 is not limited to the above method, and other methods may be adopted.

  After detecting the position of the ear E1 in contact with the panel 20, the mobile phone 1A specifies the position on the screen of the moving image content 9E being reproduced corresponding to the position E1 of the ear. For example, the mobile phone 1A specifies the position on the screen of the moving image content 9E being reproduced as the position that intersects the display 2 when the ear position E1 is projected in the vertical direction. Subsequently, the cellular phone 1A refers to the setting data 9Z, and selects the one closest to the specified position on the screen from the positions indicated by the coordinate values associated with the sound included in the setting data. Then, the cellular phone 1A determines the selected sound as a sound associated with the vicinity of the position on the screen corresponding to the position of the ear E1, and emphasizes and outputs the sound by vibrating the panel 20 To do. That is, when the mobile phone 1A detects the contact of the user's ear, the user can feel as if the sound is being transmitted right now using the position of the moving image content 9E corresponding to the contact position of the ear as a sound source. , Emphasize the sound and output.

  A processing procedure of the mobile phone 1A according to the first embodiment will be described with reference to FIGS. The processing procedures shown in FIGS. 10 and 11 are realized by the controller 10 executing the control program 9A stored in the storage 9.

  FIG. 10 is a diagram illustrating an example of a processing procedure for storing coordinate values on a screen corresponding to positions on a moving image that can be regarded as transmission points of a plurality of input sounds during moving image shooting. The processing procedure shown in FIG. 10 is repeatedly executed until the moving image shooting is completed.

  As shown in FIG. 10, when shooting of a moving image is started, the controller 10 starts capturing a plurality of sounds input by the first microphone 8a and the second microphone 8b, and in step S101, for each captured sound. Further, the position on the moving image that can be regarded as the sound transmission point is narrowed down based on the strength of the sound pressure. For example, the controller 10 detects the microphone capsule with the maximum sound pressure of the input sound from the unidirectional microphone capsules of the first microphone 8a and the second microphone 8b, and specifies the sound input direction. To do. Subsequently, the controller 10 estimates the area of the moving image displayed on the display 2 where the sound transmission point exists in accordance with the input direction of the specified sound, thereby determining the position on the moving image that can be regarded as the sound transmission point. Narrow down. Then, in the controller 10, the mobile phone 1A acquires a coordinate value on the screen corresponding to the center position of this area, and stores it in the setting data 9Z in association with the sound.

  Subsequently, in step S102, the controller 10 stores, in the setting data 9Z, the coordinate value on the screen corresponding to the position narrowed down in step S101 for each captured sound in association with the sound. For example, in step S101, the controller 10 determines the center position of the moving image area displayed on the display 2 that is estimated to have a sound transmission point, and obtains a coordinate value on the screen corresponding to the center position. And stored in the setting data 9Z in association with the sound.

  FIG. 11 is associated with the vicinity of the position on the screen corresponding to the position of the ear in contact with the panel 20 when the contact of the user's ear with the panel 20 is detected while the moving image content 9E is being reproduced. It is a figure which shows an example of the process sequence which emphasizes and outputs the sound which has been read. The processing procedure shown in FIG. 11 is repeatedly executed until the reproduction of the moving image content 9E is completed.

  As shown in FIG. 11, when the controller 10 detects the contact of the user's ear with the panel 20 during the reproduction of the moving image content 9E as step S201, the controller 10 determines the position of the ear in contact with the panel 20. The detection is performed using the touch screen 21. Subsequently, in step S202, the controller 10 acquires a position on the screen of the moving image content 9E being reproduced corresponding to the contact position of the ear.

  Subsequently, in step S203, the controller 10 emphasizes and outputs the sound associated with the vicinity of the position on the screen identified in step S202 among the plurality of sounds included in the moving image content 9E. For example, the controller 10 refers to the setting data 9Z, and selects the position closest to the position on the screen identified in step S202 from the positions indicated by the coordinate values associated with the sound included in the setting data. Then, the cellular phone 1A determines the selected sound as a sound associated with the vicinity of the position on the screen corresponding to the position of the ear E1, and emphasizes and outputs the sound by vibrating the panel 20 To do. That is, when the mobile phone 1A detects the contact of the user's ear, the user can feel as if the sound is being transmitted right now using the position of the moving image content 9E corresponding to the contact position of the ear as a sound source. , Emphasize the sound and output.

  In the first embodiment described above, when the contact of the user's ear with the panel 20 is detected while the moving image content 9E is being reproduced, the position is close to the position on the screen corresponding to the position of the ear that contacts the panel 20. Emphasizes the associated sound and outputs it. For this reason, according to the first embodiment, it is possible to listen to a specific sound from a plurality of sounds output in the reproduced moving image content.

  In the first embodiment described above, the position on the moving image that can be regarded as the transmission point of the sound is narrowed down based on the strength of the sound pressure of the input sound at the time of moving image shooting, and the coordinate value on the screen corresponding to the narrowed position is set as the sound value. Store in association with. Therefore, according to the first embodiment, it is possible to easily generate a content that allows a specific sound to be heard from among a plurality of sounds that are output in the reproduced moving image content.

(Embodiment 2)
During reproduction of the moving image content 9E, a sound having a sound source in the vicinity of the contact position of the ear may be emphasized and output according to the amount of pressure when the user's ear contacts the panel 20. .

  For example, the controller 10 of the mobile phone 1A executes the control program 9A to acquire the magnitude of the pressure detected by the touch screen 21 when the user's ear contacts the panel 20, and the magnitude of the pressure. Accordingly, a process of emphasizing and outputting a sound having a sound source near the contact position of the ear is executed.

  FIG. 12 shows an outline of processing of the mobile phone 1A that emphasizes and outputs a sound whose sound source is in the vicinity of the contact position of the ear according to the magnitude of the pressure when the user's ear contacts the panel 20. FIG. As shown in FIG. 12, after the reproduction of the moving image content 9E is started, the mobile phone 1A detects the position of the ear E1 in contact with the panel 20, and specifies the position on the screen corresponding to the position of the ear E1. Thereafter, the magnitude of the pressure detected by the touch screen 21 and making the ear E1 contact the panel 20 is acquired.

  And if the magnitude | size of a press is below a predetermined value, 1 A of mobile telephones will be the sound of CH1 matched with the vicinity of the position on the screen corresponding to the position of the ear E1 among the sounds contained in the moving image content 9E. Is emphasized over the CH2 and CH3 sounds (step S21). On the other hand, if the magnitude of the press is equal to or smaller than the predetermined value, the mobile phone 1A, among the sounds included in the moving image content 9E, the sound of CH1 associated with the vicinity of the position on the screen corresponding to the position of the ear E1. Only the output is emphasized (step S22).

  FIG. 13 is a diagram illustrating an example of a processing procedure for emphasizing and outputting a sound having a sound source in the vicinity of the contact position of the ear according to the magnitude of the pressure when the user's ear contacts the panel 20. . The processing procedure illustrated in FIG. 13 is realized by the controller 10 executing the control program 9A stored in the storage 9. The processing procedure shown in FIG. 13 is repeatedly executed until the reproduction of the moving image content 9E is completed. In the processing procedure shown in FIG. 13, step S301 corresponds to step S201 shown in FIG. 11, and step S302 corresponds to step S202 shown in FIG.

  As shown in FIG. 13, the controller 10 detects the position of the ear in contact with the panel 20 using the touch screen 21 as step S301. Subsequently, in step S302, the controller 10 acquires a position on the screen of the moving image content 9E being reproduced corresponding to the contact position of the ear.

  Subsequently, the controller 10 acquires, from the touch screen 21, the magnitude of the pressure with which the ear contacts the panel 20 as step S <b> 303. Subsequently, in Step S304, the controller 10 determines whether or not the magnitude of the pressure is a predetermined value or less.

  As a result of the determination, if the magnitude of the press is equal to or smaller than the predetermined value (step S305, Yes), the controller 10 determines the position on the screen identified in step S302 among the sounds included in the moving image content 9E being reproduced. The sound associated with the vicinity of the sound is emphasized over other sounds and output.

  On the other hand, if the result of determination is that the magnitude of the pressure is not less than or equal to the predetermined value (No at step S305), the controller 10 determines that the sound included in the moving image content 9E being reproduced is on the screen identified at step S302. Only the sound associated with the vicinity of the position is emphasized and output.

  In the example of FIG. 13, the example has been described in which the mobile phone 1 </ b> A performs processing according to the magnitude of the pressure detected by the touch screen 21, but the present invention is not limited to this. The mobile phone 1 </ b> A may detect the contact area of the ear with the touch screen 21 and perform the above processing according to the size of the area of the contact area. In general, it is considered that the larger the pressure with which the user presses the ear against the panel 20, the larger the area of the ear that contacts the panel 20. In this case, when the area of the contact area is larger than a predetermined threshold, the mobile phone 1A emphasizes only the sound associated with the vicinity of the specified position on the screen among the sounds included in the moving image content 9E being reproduced. And output. When the area of the contact area is smaller than a predetermined threshold, the mobile phone 1A, among the sounds included in the moving image content 9E being reproduced, the sound associated with the vicinity of the specified position on the screen is compared with other sounds. Is also emphasized and output.

(Embodiment 3)
When the user's ear and the panel 20 are in contact with each other during the reproduction of the moving image content 9E, based on the contact position of the ear and the positional relationship between the ear and the panel 20, a plurality of sounds included in the moving image content 9E are included. You may make it select the sound output from.

  FIG. 14 is a diagram illustrating an example of a positional relationship between the ear E1 and the panel 20 when the user's ear E1 and the panel 20 are in contact with each other. FIG. 14 shows a state in which the user's head H1 in a state where the mobile phone 1A is in contact with the ear is viewed from information. As shown in FIG. 14, for example, the ear E1 is in contact with the panel 20 at a certain angle.

  FIG. 15 is a diagram showing a region on the moving image content 9E where sound is output when the ear E1 and the panel 20 are in the positional relationship shown in FIG. The mobile phone 1 </ b> A detects that the ear E <b> 1 and the panel 20 are in the positional relationship shown in FIG. 14 based on the detection result of the posture detection unit 15. Then, as shown in FIG. 15, the mobile phone 1 </ b> A outputs, for example, the sound of the region P <b> 4 on the side that is not in contact with the user's ear on the panel 20 in the moving image content 9 </ b> E being reproduced. For example, the cellular phone 1 </ b> A identifies the region P <b> 4 on the side that is not in contact with the user's ear on the panel 20 using the capacitance detected by the touch screen 21. The mobile phone 1 </ b> A may specify the region P <b> 4 based on the detection value of the illuminance sensor 4 or the proximity sensor 5. The cellular phone 1A may specify the region P4 based on an image (for example, an image of a user's face) taken by the in-camera 12.

(Embodiment 4)
In the above-described embodiment, an example in which the touch screen 21 is disposed on almost the entire surface of the panel 20 has been described. However, the touch screen 21 may be disposed so as not to overlap the panel 20. FIG. 16 is a front view of the mobile phone according to the fifth embodiment. 17 is a cross-sectional view taken along line cc of the mobile phone shown in FIG. A mobile phone 1 </ b> B disposed so that the touch screen 21 does not overlap the panel 20 will be described with reference to FIGS. 16 and 17.

  As shown in FIGS. 16 and 17, in the mobile phone 1 </ b> B, the display 2 is arranged side by side with the panel 20 so as to be in the same plane as the panel 20, not inside the panel 20. The touch screen 21 is disposed so as to cover almost the entire front surface of the display 2. That is, the touch screen 21 and the display 2 constitute a so-called touch panel (touch screen display).

  The piezoelectric element 7 is attached to the center of the back surface of the panel 20 by a joining member 30. When an electrical signal is applied to the piezoelectric element 7, the panel 20 vibrates in accordance with deformation (expansion / contraction or bending) of the piezoelectric element 7, and air conduction sound and a part of the human body that contacts the panel 20 (for example, an ear) And vibration sound transmitted through the cartilage. By disposing the piezoelectric element 7 in the center of the panel 20, the vibration of the piezoelectric element 7 is evenly transmitted to the entire panel 20, and the quality of the sound transmitted to the user is improved.

  Although the touch screen 21 is not disposed on the front surface of the panel 20, the panel 20 is disposed in the vicinity of the display 2 on which the touch screen 21 is disposed.

  Even when the panel 20 is disposed so as not to overlap the touch screen 21 as in the cellular phone 1B, the speaker is a dynamic speaker above the piezoelectric element 7 on the same side of the housing 40 where the piezoelectric element 7 is disposed. By providing the speaker 11, sound can be output in two modes as described above. As a result, it is possible for the user to output sound by a method suitable for the user.

(Embodiment 5)
In the above-described embodiment, an example in which at least a part of the touch screen 21 is disposed so as to overlap the display 2 has been described, but the touch screen 21 may be disposed so as not to overlap the display 2. FIG. 18 is a front view of a mobile phone according to the sixth embodiment. 19 is a cross-sectional view of the mobile phone shown in FIG. 18 taken along the line dd. A mobile phone 1 </ b> C that is arranged so that the touch screen 21 does not overlap the display 2 will be described with reference to FIGS. 18 and 19. A cellular phone 1C shown in FIGS. 18 and 19 is an example of a so-called folding cellular phone.

  As shown in FIGS. 18 and 19, in the mobile phone 1 </ b> C, the display 2 is arranged side by side with the panel 20 so as to be in the same plane as the panel 20 rather than inside the panel 20.

  The piezoelectric element 7 is attached to the center of the back surface of the panel 20 by a joining member 30. A reinforcing member 31 is disposed between the panel 20 and the piezoelectric element 7. That is, in the cellular phone 1 </ b> C, the piezoelectric element 7 and the reinforcing member 31 are bonded by the bonding member 30, and the reinforcing member 31 and the panel 20 are bonded by the bonding member 30. The piezoelectric element 7 may not be provided at the center of the panel 20.

  The reinforcing member 31 is an elastic member such as rubber or silicon. The reinforcing member 31 may be a metal plate made of aluminum or the like having a certain degree of elasticity, for example. The reinforcing member 31 may be a stainless plate such as SUS304. The thickness of a metal plate such as a stainless steel plate is suitably 0.2 mm to 0.8 mm, for example, depending on the voltage value applied to the piezoelectric element 7 or the like. The reinforcing member 31 may be a resin plate, for example. As resin which forms the resin-made board here, a polyamide-type resin is mentioned, for example. Examples of the polyamide-based resin include Reny (registered trademark), which is made of a crystalline thermoplastic resin obtained from metaxylylenediamine and adipic acid and is rich in strength and elasticity. Such a polyamide-based resin may be a reinforced resin reinforced with glass fiber, metal fiber, carbon fiber, or the like using itself as a base polymer. The strength and elasticity of the reinforced resin are appropriately adjusted according to the amount of glass fiber, metal fiber, carbon fiber, or the like added to the polyamide-based resin. The reinforced resin is formed, for example, by impregnating a resin into a base material formed by braiding glass fiber, metal fiber, carbon fiber or the like and curing it. The reinforced resin may be formed by mixing a finely cut fiber piece into a liquid resin and then curing it. The reinforced resin may be a laminate of a base material in which fibers are knitted and a resin layer.

  By disposing the reinforcing member 31 between the piezoelectric element 7 and the panel 20, the following effects can be obtained. When an external force is applied to the panel 20, the possibility that the external force is transmitted to the piezoelectric element 7 and the piezoelectric element 7 is damaged can be reduced. For example, when an external force is applied to the panel 20 by the mobile phone 1 </ b> C falling on the ground, the external force is first transmitted to the reinforcing member 31. Since the reinforcing member 31 has predetermined elasticity, it is elastically deformed by an external force transmitted from the panel 20. Therefore, at least a part of the external force applied to the panel 20 is absorbed by the reinforcing member 31, and the external force transmitted to the piezoelectric element 7 is reduced. As a result, damage to the piezoelectric element 7 can be reduced. When the reinforcing member 31 is disposed between the piezoelectric element 7 and the casing 40, the casing 40 is deformed, for example, when the mobile phone 1C falls on the ground, and the deformed casing 40 collides with the piezoelectric element 7 and the piezoelectric element. The possibility that 7 is damaged can be reduced.

  Vibration due to expansion / contraction or bending of the piezoelectric element 7 is first transmitted to the reinforcing member 31 and further transmitted to the panel 20. That is, the piezoelectric element 7 first vibrates the reinforcing member 31 having a larger elastic coefficient than the piezoelectric element 7 and further vibrates the panel 20. Therefore, the cellular phone 1 </ b> C does not include the reinforcing member 31, and it is possible to make the deformation of the piezoelectric element 7 less likely to be excessive as compared with a structure in which the piezoelectric element 7 is bonded to the panel 20 by the bonding member 70. Thereby, the deformation amount (degree of deformation) of the panel 20 can be adjusted. This structure is particularly effective in the case of the panel 20 that hardly obstructs the deformation of the piezoelectric element 7.

  Furthermore, by disposing the reinforcing member 31 between the piezoelectric element 7 and the panel 20, as shown in FIG. 20, the resonance frequency of the panel 20 is lowered and the acoustic characteristics in the low frequency band are improved. FIG. 20 is a diagram illustrating an example of a change in frequency characteristics due to the reinforcing member 31. FIG. 20 shows frequency characteristics when a sheet metal such as SUS304 is used as the reinforcing member 31, and frequency characteristics when reinforced resin such as Reny is used as the reinforcing member 31. . The horizontal axis represents frequency and the vertical axis represents sound pressure. The resonance point when reinforced resin is used is about 2 kHz, and the resonance point when sheet metal is used is about 1 kHz. The dip when using reinforced resin is about 4 kHz, and the dip when using sheet metal is about 3 kHz. That is, when the reinforced resin is used, the resonance point of the panel 20 is located in a higher frequency region and the frequency characteristic dip is located in a higher frequency region, compared to the case where a sheet metal is used. Since the frequency band used for the voice call of the mobile phone is 300 Hz to 3.4 kHz, when reinforced resin is used as the reinforcing member 31, the dip can be prevented from being included in the use frequency band of the mobile phone 1C. . Even when a sheet metal is used as the reinforcing member 31, the dip is not included in the use frequency band of the cellular phone 1C by appropriately adjusting the type or composition of the metal constituting the sheet metal or the thickness of the sheet metal. be able to. When the sheet metal and the reinforced resin are compared, the reinforced resin can reduce the influence on the antenna performance as compared with the sheet metal. Since reinforced resin is less likely to be plastically deformed than sheet metal, there is an advantage that acoustic characteristics are less likely to change. The reinforced resin suppresses the temperature rise at the time of sound generation compared to the sheet metal. Instead of the reinforcing member 31, a plate-like weight may be attached to the piezoelectric element 7 by the joining member 30.

  When an electrical signal is applied to the piezoelectric element 7, the panel 20 vibrates in accordance with deformation (expansion / contraction or bending) of the piezoelectric element 7, and air conduction sound and a part of the human body that contacts the panel 20 (for example, an ear) And vibration sound transmitted through the cartilage. The touch screen 21 is disposed so as to cover almost the entire front surface of the panel 20.

  Similarly to the above, in the cellular phone 1C, the first microphone 8a is disposed at the substantially central portion of the back surface of the housing 40 in which the piezoelectric element 7 is disposed. A second microphone 8b is disposed on the back end on the opposite side of the housing 40 where the piezoelectric element 7 is disposed. A second microphone 8b is disposed on the back end on the opposite side of the housing 40 where the piezoelectric element 7 is disposed. A first microphone 8c is disposed at the opposite end of the housing 40 where the piezoelectric element 7 is disposed.

(Other embodiments)
For example, when shooting a movie, by touching the ear to the panel 20, the microphone capsule for inputting sound is determined from the unidirectional microphone capsules of the first microphone 8a and the second microphone 8b. Also good. For example, when the user's ear touches the position of the area P1 shown in FIG. 7 during moving image shooting, the microphone capsule of the first microphone 8a that inputs sound coming from the left side of the shooting direction, and the lower direction of the shooting direction. The input of the sound of the microphone capsule of the second microphone 8b that inputs the sound coming from the side is made valid, and the input of the sound of the other microphone capsule is stopped.

  In the above embodiment, a moving image is captured by the out-camera 13, and the coordinate values on the screen corresponding to the positions on the moving image that can be regarded as the transmission points of the plurality of sounds input by the first microphone 8a and the second microphone 8b are obtained. The example of storing in the setting data 9Z of the storage 9 has been described. In other words, in the above-described embodiment, an example has been described in which content in which a specific sound can be listened to is created in advance from a plurality of sounds output in the reproduced moving image content. However, the present invention is not limited to this, and when there is moving image content in which the correspondence between the position on the screen when the moving image content is reproduced and the position of the sound source of the sound included in the moving image content is known in advance The same processing as in the above embodiment can also be applied when reproducing this moving image content. That is, during reproduction of this moving image content, the sound associated with the vicinity of the position on the screen corresponding to the position of the ear where the user's ear contacts the panel 20 is emphasized by vibrating the panel 20. Can be output. Further, by using the audio information originally present in the stereo content and using a technology that artificially converts the sound source of two channels into multiple channels, the output position of the sound of each channel can be estimated by calculation. The output position and the contact position of the ear on the touch panel can be associated with each other, and the sound corresponding to the contact position of the ear can be emphasized and output. Note that the number of sounds included in the moving image content is not limited to one but may be one.

  In the above embodiment, the panel 20 is deformed by the piezoelectric element 7 to generate air conduction sound and vibration sound. A part of the housing 40 may be deformed by the piezoelectric element 7 to generate air conduction sound and vibration sound. A part of the housing 40 may be a corner portion of the housing, for example.

  In the above-described embodiment, an example in which the display 2 is attached to the back surface of the panel 20 using the bonding member 30 as the mobile phone 1A has been described. However, the mobile phone 1A has a space between the panel 20 and the display 2. May be configured. By providing a space between the panel 20 and the display 2, the panel 20 easily vibrates, and the range on which the vibration sound can be easily heard on the panel 20 is widened.

  In the above embodiment, the example in which the piezoelectric element 7 is attached to the panel 20 has been described, but the piezoelectric element 7 may be attached to another place. For example, the piezoelectric element 7 may be attached to a battery lid. The battery lid is a member that is attached to the housing 40 and covers the battery. Since the battery lid is often attached to a surface different from the display 2 in a portable electronic device such as a cellular phone, according to such a configuration, the user can place a part of the body (for example, an ear) on a surface different from the display 2. You can hear the sound by touching.

  In said embodiment, although the reinforcement member 31 is a plate-shaped member, the shape of the reinforcement member 31 is not restricted to this. For example, the reinforcing member 31 may have a shape that is larger than the piezoelectric element 7 and has an end that curves toward the piezoelectric element 7 and covers the side of the piezoelectric element 7. The reinforcing member 31 may have, for example, a shape having a plate-like portion and an extending portion that extends from the plate-like portion and covers the side portion of the piezoelectric element 7. In this case, it is preferable that the extended portion and the side portion of the piezoelectric element 7 are separated by a predetermined distance. Thereby, it becomes difficult for the extending part to inhibit the deformation of the piezoelectric element.

  The bonding member that bonds the panel 20 and the piezoelectric element 7 and the bonding member that bonds the panel 20 and the casing 40 have been described as the bonding member 30 having the same reference numeral. However, different joining members may be used as appropriate depending on the members to be joined.

  In the above embodiment, a mobile phone has been described as an example of a device according to the appended claims. However, the device according to the appended claims is not limited to a mobile phone. The device according to the appended claims may be a mobile electronic device other than a mobile phone. Examples of the portable electronic device include, but are not limited to, a tablet, a portable personal computer, a digital camera, a media player, an electronic book reader, a navigator, and a game machine.

  The characterizing embodiments have been described in order to fully and clearly disclose the technology according to the appended claims. However, the appended claims should not be limited to the above-described embodiments, but all modifications and alternatives that can be created by those skilled in the art within the scope of the basic matters shown in this specification. Should be embodied by a simple configuration.

DESCRIPTION OF SYMBOLS 1A Mobile phone 2 Display 3 Button 4 Illuminance sensor 5 Proximity sensor 6 Communication unit 7 Piezoelectric element 8a 1st microphone 8b 2nd microphone 8c 3rd microphone 9 Storage 9A Control program 9B Call application 9C Music playback application 9D Video playback application 9E Video content 9Z Setting data 10 Controller 11 Speaker 12 Camera 15 Posture detection unit 18 Vibrator 20 Panel 21 Touch screen 30 Joining member 40 Housing

Claims (16)

A piezoelectric element;
A sound generator that vibrates by the piezoelectric element and generates air conduction sound and vibration sound transmitted by vibrating a part of the human body;
A display on which a screen of image content including sound is reproduced and displayed;
A detection unit that detects a contact position of a part of the human body with respect to the sound generation unit,
When a contact of a part of the human body with the sound generation unit is detected while the image content is being reproduced on the display, the sound generation unit is a sound included in the image content, and the contact position An electronic device that generates a sound associated with a position on the screen corresponding to The image content includes a plurality of sounds,
When contact of a part of the human body with the sound generation unit is detected while the image content is being reproduced on the display, the sound generation unit is configured to output the sound among the plurality of sounds included in the image content. The electronic device according to claim 1, wherein a controller that emphasizes and generates a sound associated with a position near the position on the screen corresponding to a position on the sound generation unit. The detection unit detects the amount of pressure on a part of the human body against the sound generation unit,
The electronic device according to claim 2, wherein the sound generation unit emphasizes a sound having a virtual sound source in the vicinity of a position on the screen in accordance with a pressing amount of a part of the human body. The detection unit detects a positional relationship between the part of the human body and the sound generation unit when the part of the human body contacts the sound generation unit,
The electronic device according to claim 2, wherein the sound generation unit generates a sound selected based on the contact position and the positional relationship from among a plurality of sounds included in the image content. The image content includes video content,
It further has a sound input part for inputting sound during movie shooting,
For each of a plurality of sounds input by the sound input unit at the time of moving image shooting, the position on the moving image that can be regarded as a transmission point of the sound is narrowed down based on the strength of the sound pressure of the sound, and the position corresponding to the narrowed position The electronic device according to claim 1, wherein coordinate values on the screen are stored in association with sounds.   The electronic device according to claim 5, wherein the controller changes the directivity of the sound input unit based on a position of the sound generation unit during standby for shooting a moving image.   The electronic device according to claim 1, wherein the sound generator is a panel. The panel vibrates in a region wider than a region having a length corresponding to the distance from the lower leg of the human ear to the antitragus and a width corresponding to the distance from the tragus to the antique. The electronic device according to claim 7. The electronic device according to claim 7 or 8, wherein the panel constitutes a part or all of a display panel, an operation panel, a cover panel, and a lid panel for allowing a rechargeable battery to be removed. When the panel is a display panel,
The piezoelectric element is disposed outside a display area for the display function.
The electronic device according to claim 9. The electronic device according to any one of claims 7 to 10, wherein the panel is deformed to transmit air conduction sound and human body vibration sound at any location of the panel. The panel has a plurality of locations that vibrate in a direction intersecting with the main surface of the panel in the vibration region, and the amplitude value of the vibration is increased from plus to minus with time or vice versa at each location. The electronic apparatus according to claim 7, wherein the electronic apparatus varies. A sound generator;
A display on which a screen of image content including sound is reproduced and displayed;
A detection unit that detects a contact position of a part of the human body with respect to the panel,
The image content includes an object to be displayed;
An electronic device that reproduces a sound associated with an object when a part of a human body comes into contact with the object. A piezoelectric element;
The electronic device according to claim 13, wherein the sound generation unit is vibrated by the piezoelectric element to generate air conduction sound and vibration sound transmitted by vibrating a part of a human body. A piezoelectric element, a sound generation unit that generates air conduction sound and vibration sound that is transmitted by vibrating a part of a human body, and a display that reproduces and displays a screen of image content including the sound. A control method executed by an electronic device including a detection unit that detects a contact position of a part of the human body with respect to the sound generation unit,
When the contact of a part of the human body with the sound generation unit is detected while the image content is being reproduced on the display, the sound included in the image content and corresponding to the contact position on the screen A control method including a step of causing the sound generation unit to generate a sound associated with the position of the sound. A piezoelectric element, a sound generation unit that generates air conduction sound and vibration sound that is transmitted by vibrating a part of a human body, and a display that reproduces and displays a screen of image content including the sound. In an electronic device comprising a detection unit that detects a contact position of a part of the human body with respect to the sound generation unit,
When the contact of a part of the human body with the sound generation unit is detected while the image content is being reproduced on the display, the sound included in the image content and corresponding to the contact position on the screen A control program for executing the step of causing the sound generating unit to generate a sound associated with the position of the sound.

Images