JP6196790B2 - Information device and control method of information device - Google Patents

Description

The present invention relates to an information device having a touch operation device that an operator touches to operate, and a control method for such an information device .

  Unlike a mechanical operation member such as a button, a touch operation device such as a touch panel performs operation input according to the detection result of the touch sensor. Here, the conventional touch operation device is configured by a hard flat surface such as glass. For this reason, even if the operator touches the finger with any position on the surface of the touch operation device, the feeling given to the finger is the same. Therefore, in the case of the conventional touch operation device, it is difficult for the operator to intuitively recognize whether the touch operation device has been correctly operated. As a proposal for enabling the operator to recognize the touch position without visual observation, Patent Document 1 proposes a portable terminal device that transmits the content of the image at the touch position in the touch operation device to the operator by braille display. doing.

JP 2012-156607 A

  When the content of the image at the touch position is transmitted to the operator by Braille display, the operator cannot recognize the content of the image at the touch position unless the operator has knowledge of Braille.

The present invention has been made in view of the above circumstances, and provides an information device having a touch operation device that can be operated without visual observation even if the operator has no special knowledge, and a control method for such an information device. For the purpose.

In order to achieve the above object, an information device according to one embodiment of the present invention includes a plurality of detection points associated with image positions, and a touch sensor that detects an operator's touch operation on the detection points. , Ta and the object specifying unit that specifies the name of a target on the shadow, the image determining unit to search the object has been name specified by the object specifying unit from the characteristics of the image, at the time of shooting The sensor can be touched by the operator's finger and is provided at a location different from the touch sensor. In the image searched for by the image determination unit, the sensory stimulus providing unit that gives sensory stimulus to the operator And a control unit that applies the sensory stimulus by the sensory stimulus providing unit according to a positional relationship between the position of the object and the touch position detected by the touch sensor.
In order to achieve the above object, an information device control method according to an aspect of the present invention includes a plurality of detection points associated with image positions, and detects an operator's touch operation on the detection points. a touch sensor, a control method for an information device including a sensory stimulation providing unit providing a sensory stimulus to the operator, and specifying the name of the object to be Taking target shadow, said specified An operator at the time of shooting in accordance with the positional relationship between the position of the target in the searched image and the touch position detected by the touch sensor. Providing a sensory stimulus to the operator by the sensory stimulus providing unit provided at a location where the finger can be touched and different from the touch sensor .

According to the present invention, it is possible to provide an information device having a touch operation device that can be operated without visual observation even if the operator has no special knowledge, and a control method for such an information device .

1 is an external view of a digital camera as an example of an information device equipped with a touch operation device according to an embodiment of the present invention. It is a block diagram which shows the detailed structure of a digital camera. 3 is a top perspective view of a piezoelectric element 1042. FIG. 4 is a cross-sectional view of a piezoelectric element 1042. FIG. 3 is a top perspective view of a piezoelectric element 1082. FIG. 3 is a cross-sectional view of a piezoelectric element 1082. FIG. It is a figure which shows the structural example of an application part. It is a figure which shows operation | movement of a piezoelectric element. It is a flowchart which shows the main operation | movement of a digital camera. It is the figure which showed the example of the menu screen displayed in an object only designation | designated mode. It is a figure which shows the modification of a position only designation | designated mode. It is a flowchart which shows the process of vibration control. It is a figure which shows the example of the vibration control of a piezoelectric element in case a touch position and the position of a target object are the same positions. It is a figure which shows the operation example of the vibration control in case a touch position and the position of a target object are the same positions. It is a figure which shows the example of the vibration control of a piezoelectric element in case a touch position and the position of a target object are different positions. It is a 1st figure which shows the operation example of the vibration control in case a touch position and the position of a target object are the same positions. It is a 2nd figure which shows the operation example of the vibration control in case a touch position and the position of a target object are the same positions. It is a flowchart which shows the process of reproduction | regeneration mode.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is an external view of a digital camera (hereinafter simply referred to as a camera) as an example of information equipment equipped with a touch operation device according to an embodiment of the present invention. Here, FIG. 1A is an external view seen from the front side (subject side) of the camera, and FIG. 1B is an external view seen from the back side of the camera. Here, a digital camera is shown as an example of information equipment, but the technology of the present embodiment can be applied even to a mobile phone, a mobile terminal, or the like. In addition, since it is a confirmation method that makes effective use of the five senses without relying only on vision, it has a wide range of applications and can be used for industrial and medical observation devices. Needless to say, since it does not impose visual restrictions, it can be applied to in-vehicle devices and surveillance cameras.

  In the case of a camera, the operator often watches the subject so as not to miss a photo opportunity. In this case, the operator's consciousness is directed toward the subject, and the operator executes the camera operation without checking with the eyes. In this embodiment, it is possible to perform correct camera operation without an operator confirming with eyes. For example, if the operator's attention is directed to a small screen, there may be a risk that a situation change occurring outside the screen may be overlooked. Even if the situation was not so serious, the other party who had conversation during the shooting of the interview sometimes changed their facial expressions during the shooting.

  As shown in FIG. 1A, a photographing lens 102 is provided on the front surface of the main body of the camera 1. The photographing lens 102 has a cylindrical lens frame and an optical system housed in the lens frame. A touch operation device is formed around the lens frame. An operator can perform manual focus and manual zoom using this touch operation device instead of the focus ring and zoom ring. Also, a release button 1141 is provided on the upper surface of the main body of the camera 1. The release button 1141 is an operation member for the operator to instruct the camera 1 to execute shooting. Here, the release button 1141 has a piezoelectric element and is configured to give a tactile stimulus to the operator's finger as necessary.

  Further, as shown in FIG. 1B, a display unit 106 is provided on the back of the main body of the camera 1. A touch operation device is formed on the display unit 106. The operator can perform various operations using the touch operation device.

  As shown in FIG. 1B, a finder display unit 110 and an eye sensor 112 are provided on the upper part of the back of the main body of the camera 1. The camera 1 can perform the same display as the display unit 106 on the finder display unit 110. Whether to display on the display unit 106 or to display on the finder display unit 110 is switched according to the detection result of the eye sensor 112. Details will be described later.

  Further, a microphone (microphone) 1161 is provided at the lower back of the main body of the camera 1.

  FIG. 2 is a block diagram showing a detailed configuration of the camera 1. As shown in FIG. 2, the camera 1 includes a photographing lens 102, a lens driving unit 103, a touch operation device 104, a display unit 106, a touch operation device 108, a viewfinder display unit 110, an eye sensor 112, and an operation. Unit 114, audio input unit 116, imaging unit 118, recording unit 120, clock unit 122, and control unit 124.

  As described above, the photographing lens 102 is configured by housing an optical system in a lens frame. The optical system of the photographic lens 102 includes a single lens or a plurality of lenses and a diaphragm, and condenses a light image of a subject (not shown) on the imaging surface of the imaging element 1181 of the imaging unit 118. The optical system is driven in the optical axis direction by the lens driving unit 103. The lens driving unit 103 includes a motor and its driving circuit, and drives the optical system according to a control signal from the control unit 124. Thereby, focus drive and zoom drive of the optical system are performed.

  The touch operation device 104 includes a touch sensor 1041, a piezoelectric element 1042, and an application unit 1043.

  The touch sensor 1041 has a plurality of detection points arranged along the circumferential direction of the lens frame of the photographing lens 102. Each detection point outputs a detection signal to the control unit 124 according to the touch of the operator's finger. Here, as the touch sensor 1041, various types of detection type touch sensors such as a pressure-sensitive touch sensor and a capacitive touch sensor can be used. The pressure-sensitive touch sensor is a touch sensor that detects a finger touch by detecting a change in pressure generated at the detection point when the finger touches the detection point. On the other hand, the capacitive touch sensor is a touch sensor that detects a finger touch by detecting a change in capacitance that occurs between the finger and the detection point when the finger approaches the detection point. Such a touch sensor 1041 allows the operator to perform a manual focus operation and a zoom operation. Here, when the zoom ring or the focus ring is provided in the lens frame, the touch sensor 1041 may be omitted.

  The piezoelectric element 1042 is disposed so as to correspond to the touch sensor 1041 and is configured to bend and displace when a voltage is applied. Here, the piezoelectric element 1042 in the present embodiment is configured to be able to apply a voltage to each part corresponding to the detection point of the touch sensor 1041, and thereby bends to each part corresponding to the detection point of the touch sensor 1041. To do. The configuration and operation of the piezoelectric element 1042 will be described in detail later.

  The applying unit 1043 is configured to apply a voltage independently for each portion of the piezoelectric element 1042. Here, the application unit 1043 applies positive and negative voltages with respect to the ground potential of the piezoelectric element 1042. The configuration and operation of the application unit 1043 will be described in detail later.

  The display unit 106 is, for example, a liquid crystal display or an organic EL display, and displays an image based on image data input from the control unit 124 serving as a display control unit.

  The touch operation device 108 includes a touch sensor 1081, a piezoelectric element 1082, and an application unit 1083.

  The touch sensor 1081 has a plurality of detection points arranged two-dimensionally on the display unit 106. Each detection point outputs a detection signal to the control unit 124 according to the touch of the operator's finger. Here, the touch sensor 1081 can use touch sensors of various detection methods as in the touch sensor 1041. With such a touch sensor 1081, the operator can perform various setting operations such as shutter speed setting, aperture setting, and exposure correction setting. The touch sensor 1081 also functions as an example of an object specifying unit. The operator can specify an object using the touch sensor 1081. An object is a subject that an operator is interested in, for example, which can be a subject of shooting or reproduction. Alternatively, the object may be an object that the operator should be interested in. By such an application, it is possible to find an object that the operator is not aware of but wants to photograph. Here, the candidate list of objects may be stored in a database. Such a database may be downloaded from outside the camera 1. Furthermore, the candidate for the object may be estimated by analyzing the behavior of the operator.

  The piezoelectric element 1082 that functions as an example of the sensory stimulus providing unit is disposed so as to correspond to the touch sensor 1081 and is configured to bend and displace when a voltage is applied. Here, the piezoelectric element 1082 in the present embodiment is configured to be able to apply a voltage to each portion corresponding to the detection point of the touch sensor 1081, thereby bending the portion corresponding to the detection point of the touch sensor 1081. To do. The configuration and operation of the piezoelectric element 1082 will be described in detail later.

  The application unit 1083 is configured to apply a voltage independently for each portion of the piezoelectric element 1082. Here, the application unit 1083 applies positive and negative voltages with respect to the ground potential of the piezoelectric element 1082. The configuration and operation of the application unit 1083 will be described in detail later.

  The finder display unit 110 is a display unit provided in the finder at the upper back of the main body of the camera 1. The finder display unit 110 is, for example, a liquid crystal display or an organic EL display, and displays an image based on image data input from the control unit 124 as a display control unit.

  The eye sensor 112 is provided in the vicinity of the finder, and detects the finder looking into the finder. The eye sensor 112 includes, for example, an infrared light emitting unit and an infrared light receiving unit. When the operator is looking into the viewfinder, the infrared light emitted from the infrared light emitting unit is reflected by the operator's face and received by the infrared light receiving unit. Thereby, the output signal in an infrared light-receiving part changes. On the other hand, when the operator does not look into the viewfinder, the infrared light emitted from the infrared light emitting unit is not reflected, so the output of the infrared light receiving unit does not change. Therefore, it is possible to identify whether the operator is looking into the finder from the output signal of the infrared light receiving unit.

  The operation unit 114 is a mechanical operation member provided in the main body of the camera 1 other than the touch operation device 104 and the touch operation device 108. The operation unit 114 includes, for example, a release button 1141. The release button 1141 has a piezoelectric element as described above, and applies tactile stimulation to the operator's finger as necessary.

  The voice input unit 116 that functions as an example of the object specifying unit inputs voice from the operator to the control unit 124. The audio input unit 116 includes a microphone (microphone) 1161 and an analog / digital (A / D) conversion unit 1162. The microphone 1161 converts the voice uttered by the operator into an electrical signal (voice signal). The A / D converter 1162 converts the audio signal obtained by the microphone 1161 into a digital audio signal (audio data).

  The imaging unit 118 includes an imaging element 1181 and an A / D conversion unit 1182. The image sensor 1181 is disposed on the optical axis of the optical system of the photographing lens 102 and at a position where a subject light beam is imaged by the optical system of the photographing lens 102. The imaging element 1181 has an imaging surface on which photodiodes constituting pixels are two-dimensionally arranged. Each photodiode generates a charge corresponding to the amount of light received. The electric charge generated in the photodiode is accumulated in a capacitor connected to each photodiode. The electric charge accumulated in this capacitor is read out as an image signal. The A / D converter 1182 converts the image signal read by the image sensor 1181 into a digital image signal (image data).

  The recording unit 120 is, for example, a flash memory built in the main body of the camera 1 or detachable from the main body of the camera 1, and is generated by the control unit 124 based on image data obtained through the imaging unit 118. Record the recorded image file.

  The clock unit 122 measures various times such as shooting date and time. For example, the shooting date and time is recorded in an image file.

  The control unit 124 controls various operations of the camera 1. For example, the control unit 124 has a function as an imaging control unit, and performs driving control of the lens driving unit 103 and driving control of the imaging element 1181. The control unit 124 also has a function as a display control unit, and controls the operation of the display unit 106 or the finder display unit 110 to display an image on the display unit 106 or the finder display unit 110. The control unit 124 includes an image processing unit, and performs image processing on the image data obtained by the imaging unit 118. In addition, the control unit 124 includes an image determination unit, and searches for an object specified by the operator by determining the characteristics (contour, color, etc.) of the image data. Further, the control unit 124 includes a voice processing unit, and recognizes the content of the voice uttered by the operator from the voice data obtained by the voice input unit 116. The control unit 124 has a function as a touch position identification unit, and identifies the position (touch position) of the detection point touched by the operator's finger in the touch sensor 1041 and the touch sensor 1081. The control unit 124 has a function as a piezoelectric control unit, and controls the application unit 1043 so as to drive the piezoelectric element 1042 corresponding to the touch position.

  Next, details of the piezoelectric element in the present embodiment will be described. Of course, the present invention is not limited to the piezoelectric element described below. First, the piezoelectric element 1042 will be described. FIG. 3 is a top perspective view of the piezoelectric element 1042. 4 is a cross-sectional view of the piezoelectric element 1042, in which (a) is a cross-sectional view taken along line 3A-3A in FIG. 3, (b) is a cross-sectional view taken along line 3B-3B in FIG. FIG. 4 is a cross-sectional view taken along line 3C-3C in FIG.

An example of the piezoelectric element 1042 is configured by sandwiching a polylactic acid layer between two electrodes. Hereinafter, the configuration of the piezoelectric element 1042 will be described in more detail.
For example, a number of signal electrodes 204 corresponding to the number of detection points of the touch sensor 1041 are formed on the lower insulating layer 202 which is a resin layer of polyimide or the like of about several tens of μm. Each signal electrode 204 is made of a transparent electrode having flexibility of about 10 μm, for example. Each signal electrode 204 is connected to a corresponding application unit. For example, one detection point has five signal electrodes 204 as shown in FIG. The same voltage is applied to these five signal electrodes 204.

  In addition, a lower ground (GND) electrode 206 is formed on the lower insulating layer 202. The lower GND electrode 206 is connected to the ground potential set in the camera 1.

  A polylactic acid layer 208 is stacked on the signal electrode 204 and the lower GND electrode 206. The polylactic acid layer 208 is a layer having piezoelectricity and flexibility, and is formed by laminating an L-polylactic acid layer 2081 and a D-polylactic acid layer 2082. The L-polylactic acid layer 2081 has at least one (three in the drawing) L-polylactic acid thin film sheet. On the other hand, the D-form polylactic acid layer 2082 has at least one (three in the figure) D-form polylactic acid thin film sheet.

  On the polylactic acid layer 208, an upper ground (GND) electrode 210 is formed. That is, the polylactic acid layer 208 is formed so as to be sandwiched between the signal electrode 204 and the upper GND electrode 210. A through hole 212 is formed from the upper GND electrode 210 to the lower GND electrode 206, and the upper GND electrode 210 and the lower GND electrode 206 are electrically connected by the through hole 212. As a result, the potential of the upper GND electrode 210 is fixed to the ground potential.

  Further, an upper insulating layer 214 is laminated on the upper GND electrode 210. The upper insulating layer 214 protects the electrodes and the like inside the piezoelectric element.

  Further, as shown in the cross-sectional view, the lower insulating layer 202 is formed on the base material layer 218 via the soft material layer 216. The soft material layer 216 is a layer such as silicone rubber, and the base material layer 218 is a hard material layer such as glass.

  Next, the piezoelectric element 1082 will be described. Since the laminated structure of the piezoelectric element 1082 is similar to that of the piezoelectric element 1042, the description will focus on different parts. FIG. 5 is a top perspective view of the piezoelectric element 1082. FIG. 6 is a sectional view of the piezoelectric element 1042 taken along line 5-5.

The piezoelectric element 1082 is obtained by arranging the piezoelectric elements 1042 shown in FIG. 3 as one piezoelectric element array and arranging a plurality of such piezoelectric element arrays. Hereinafter, the configuration of the piezoelectric element 1082 will be described in more detail.
A signal electrode 204 is formed on the lower insulating layer 202. One piezoelectric element array has a plurality (five in the figure) of signal electrodes 204 arranged in the vertical direction. A lead electrode 220 is formed on each signal electrode 204 constituting one piezoelectric element array. Each signal electrode 204 constituting one piezoelectric element array is connected to the application unit 1043 or 1083 via the extraction electrode 220.

  The lower insulating layer 202 has one lower GND electrode 206 for one piezoelectric element array.

  A polylactic acid layer 208 is stacked on the signal electrode 204 and the lower GND electrode 206. The polylactic acid layer 208 is configured by laminating an L polylactic acid layer 2081 and a D polylactic acid layer 2082.

  On the polylactic acid layer 208, one upper GND electrode 210 is formed for one piezoelectric element array. A through hole 212 is formed from the upper GND electrode 210 to the lower GND electrode 206, and the upper GND electrode 210 and the lower GND electrode 206 are electrically connected by the through hole 212.

  Further, an upper insulating layer 214 is laminated on the upper GND electrode 210. The upper insulating layer 214 protects the electrodes and the like inside the piezoelectric element.

  FIG. 7 is a diagram illustrating a configuration example of the application units 1043 and 1083. The application unit 1043 and the application unit 1083 have the same basic configuration. Therefore, here, the application unit 1043 and the application unit 1083 will be described without distinction. As shown in FIG. 7, the application unit includes constant voltage sources 3021 and 3022, switches 3041a and 3042a, switches 3041b and 3042b, a switch switching unit 306a, a switch switching unit 306b, and a decoder 308. ing. Here, the application section shown in FIG. 7 shows a configuration corresponding to the two signal electrodes 204. Actually, the application unit includes two switches and a switch switching unit corresponding to each of the signal electrodes 204.

  The constant voltage sources 3021 and 3022 are power supplies that generate a voltage to be applied to the signal electrode 204. Here, the constant voltage source 3021 is a power source that generates a voltage (for example, +5 V) having a positive potential with respect to the ground potential. The constant voltage source 3022 is a power source that generates a voltage (eg, −5 V) having a negative potential with respect to the ground potential.

  The switch 3041a is openably and closably connected between the constant voltage source 3021 and one signal electrode 204a of the signal electrodes 204, and the constant voltage source 3021 and the signal electrode 204a are connected in accordance with a control signal from the switch switching unit 306a. Is in a conductive state or an open state. The switch 3042a is openably and closably connected between the constant voltage source 3022 and the signal electrode 204a, and the constant voltage source 3022 and the signal electrode 204a are in a conductive state or an open state according to a control signal from the switch switching unit 306a. And

  The switch 3041b is openably and closably connected between the constant voltage source 3021 and another signal electrode 204b of the signal electrodes 204, and is connected to the constant voltage source 3021 and the signal electrode in accordance with a control signal from the switch switching unit 306b. A connection state or an open state is established between the terminal 204b and the terminal 204b. The switch 3042b is openably and closably connected between the constant voltage source 3022 and the signal electrode 204b, and is connected or disconnected between the constant voltage source 3022 and the signal electrode 204b in accordance with a control signal from the switch switching unit 306b. And

  The switch switching unit 306a is provided corresponding to the signal electrode 204a, and inputs a control signal to the switch 3041a and the switch 3042a in accordance with an input from the decoder 308. Here, the switch switching unit 306a inputs a control signal so that the switch 3042a is turned off when the switch 3041a is turned on, and the switch 3041a is turned off when the switch 3042a is turned on.

  The switch switching unit 306b is provided corresponding to the signal electrode 204b, and inputs a control signal to the switch 3041b and the switch 3042b in accordance with an input from the decoder 308. Here, the switch switching unit 306b inputs a control signal so that the switch 3042b is turned off when the switch 3041b is turned on, and the switch 3041b is turned off when the switch 3042b is turned on.

  The decoder 308 decodes the piezoelectric element driving instruction from the control unit 124, and inputs a control signal to the switch switching units 306a and 306b so that a voltage is applied to the signal electrode corresponding to the decoded piezoelectric element driving instruction.

  Next, the operation of the piezoelectric element will be described. The operation of the piezoelectric element is not different between the piezoelectric element 1042 and the piezoelectric element 1082. Therefore, here, the operation of the piezoelectric element 1042 and the operation of the piezoelectric element 1082 will be described without any particular distinction.

  When a positive voltage (for example, +5 V) is applied to the signal electrode 204 corresponding to a detection point of the touch sensor, the L polylactic acid layer 2081 extends due to an electric field generated between the signal electrode 204 and the upper GND electrode 210. As a result, the D polylactic acid layer 2082 shrinks. The displacement amount of the polylactic acid layer 208 is larger in the short side direction than in the long side direction. The polylactic acid layer 208 is laminated in the order of an L-polylactic acid layer 2081 and a D-polylactic acid layer 2082. Therefore, as a whole, the polylactic acid layer 208 is bent so as to be recessed as shown in FIG. The upper insulating layer 214, the upper GND electrode 210, the signal electrode 204, and the lower insulating layer 202 are also bent by the force generated by the bending of the polylactic acid layer 208, so that the piezoelectric element is deformed so as to be recessed as a whole. The force generated by the bending of the polylactic acid layer 208 is absorbed by the soft material layer 216 and is not transmitted to the base material layer 218.

  When a negative voltage (for example, −5 V) is applied to the signal electrode 204 corresponding to a detection point of the touch sensor, an L-type polylactic acid layer is generated by an electric field generated between the signal electrode 204 and the upper GND electrode 210. 2081 shrinks and the D polylactic acid layer 2082 extends. As described above, the amount of displacement of the polylactic acid layer 208 is greater in the short side direction than in the long side direction. The polylactic acid layer 208 is laminated in the order of an L-polylactic acid layer 2081 and a D-polylactic acid layer 2082. Therefore, the polylactic acid layer 208 is bent so as to swell as shown in FIG. 8B as a whole. The upper insulating layer 214, the upper GND electrode 210, the signal electrode 204, and the lower insulating layer 202 are also bent by the force generated by the bending of the polylactic acid layer 208, so that the piezoelectric element is deformed so as to swell as a whole. The force generated by the bending of the polylactic acid layer 208 is absorbed by the soft material layer 216 and is not transmitted to the base material layer 218.

  In this manner, the piezoelectric element in the present embodiment can directly dent or bulge the portion corresponding to each detection point of the touch sensor. Thereby, it is possible to give a tactile stimulus to the operator from an arbitrary position of the piezoelectric element.

  Here, the polylactic acid layer 208 is formed by laminating an L-polylactic acid layer 2081 and a D-polylactic acid layer 2082. This is to provide a large piezoelectric property (a large piezoelectric constant). Depending on the magnitude of the tactile stimulus given to the operator, only the L polylactic acid layer 2081 or the D polylactic acid layer 2082 may be laminated to form the polylactic acid layer 208.

  Further, the stacking order of the L polylactic acid layer 2081 and the D polylactic acid layer 2082 may be reversed. When the stacking order is reverse, the piezoelectric element bends so as to swell when a positive voltage is applied by the application unit, and bends so that the piezoelectric element dents when a negative voltage is applied.

  Next, the operation of the camera 1 having the touch operation device according to the present embodiment will be described. FIG. 9 is a flowchart showing the main operation of the camera 1. The operation in FIG. 9 is controlled mainly by the control unit 124. Further, the operation of FIG. 9 is started, for example, when the camera 1 is turned on.

  When the operation of FIG. 9 is started, the control unit 124 determines whether or not the operation mode of the camera 1 is the shooting mode (step S101). The camera 1 has a shooting mode and a playback mode as operation modes. The operation mode is set, for example, according to the operation of the operation unit 114 or the touch sensor 1081 by the operator.

  If it is determined in step S101 that the operation mode is the shooting mode, the control unit 124 causes the display unit 106 to display a through image (step S102). As a through image display, the control unit 124 activates the imaging unit 118 and acquires image data for through image display. After acquiring the image data, the control unit 124 performs general image processing for displaying a through image such as white balance correction and gamma correction on the acquired image data. After the through-image display image processing, the control unit 124 inputs the processed image data to the display unit 106 and causes the display unit 106 to display the image.

  After the through image is displayed on the display unit 106, the control unit 124 determines from the detection result of the eye sensor 112 whether the operator has looked into the finder display unit 110 (step S103). If it is determined in step S103 that the viewfinder display unit 110 is not looked into, the control unit 124 ends the through image display on the viewfinder display unit 110 (step S104). If it is determined in step S103 that the viewfinder display unit 110 is viewed, the control unit 124 causes the viewfinder display unit 110 to display a through image (step S105). The through image display on the finder display unit 110 is the same as the through image display on the display unit 106 except that the input destination of the image data is the finder display unit 110. General view image display image processing such as white balance correction and gamma correction is performed on the finder display unit. After the through-image display image processing, the control unit 124 inputs the processed image data to the display unit 106 and causes the display unit 106 to display the image. After the through image display on the finder display unit 110, the control unit 124 ends the through image display on the display unit 106 (step S106).

  After the through image display, the control unit 124 determines from the detection result of the touch sensor 1081 whether or not the operator has touched the display screen of the display unit 106 (step S107).

  If it is determined in step S104 that the operator has touched the display screen of the display unit 106, the control unit 124 determines whether or not an object for vibration control, which will be described in detail later, has already been specified. Is determined (step S108).

  If it is determined in step S108 that the target has been designated, the control unit 124 performs vibration control of the piezoelectric element (step S109). The vibration control will be described in detail later.

  If it is determined in step S108 that the object has not been specified, the control unit 124 determines whether or not the operation mode of the touch operation device 104 is the object specifying mode (step S110). The camera 1 includes a touch release mode and an object designation mode as operation modes of the touch operation device 104 during live view display. The touch release mode is a mode in which shooting is performed in response to an operator's touch operation on the touch operation device 104. On the other hand, the object designation mode is a mode for an operator to designate an object using the touch operation device 104. These operation modes are set by, for example, a touch operation on a menu screen (not shown).

  In step S110, when it is determined that the operation mode of the touch operation device 104 is not the object designation mode, that is, the touch release mode, the control unit 124 performs shooting and image recording operations (step S111). As a photographing operation, the control unit 124 performs AF control to focus the optical system of the photographing lens 102 and performs AE control to determine an exposure condition during photographing. Here, in step S111, AF control and AE control are performed so that the subject at the touched position is focused and exposed. Subsequently, the control unit 124 causes the imaging unit 118 to perform an imaging operation in accordance with the exposure condition determined by the AE control. After the imaging operation, the control unit 124 performs general recording image processing such as white balance correction, gamma correction, and compression processing on the image data obtained as a result of the imaging operation of the imaging unit 118. After the image processing for recording, the control unit 124 generates an image file based on the compressed image data, and causes the recording unit 120 to record the generated image file. After such a photographing operation, the control unit 124 shifts the process to step S119.

  When it is determined in step S110 that the object designation mode is set, the control unit 124 determines whether or not the designation mode is the simultaneous designation mode (step S112). The object designation mode has a simultaneous designation mode, an object only designation mode, and a position only designation mode as designation modes. These designation modes are set by a touch operation on a menu screen (not shown), for example. When it is determined in step S112 that the mode is the simultaneous designation mode, the control unit 124 acquires information on the position touched by the operator and information on the subject (target object) at the position, and acquires the acquired position information and The object information is stored in a storage unit (not shown) (step S113). Here, the object information includes information such as the name and shape of the subject. The name of the subject is specified by image recognition, for example. The shape of the subject is detected by contour extraction or the like. Here, when the object is a moving object, it is desirable to track the object and update the position information.

  If it is determined in step S112 that the mode is not the simultaneous designation mode, the control unit 124 determines whether or not the designation mode is the object only designation mode (step S114). If it is determined in step S114 that only the object is in the designation mode, the control unit 124 causes the display unit 106 to display a menu screen for allowing the operator to designate the object (step S115). FIG. 10 is a diagram illustrating an example of a menu screen. The operator touches an item of a desired object from the list of objects displayed on the menu screen. Upon receiving the touch operation, the control unit 124 searches for the object of the item touched by the operator from the image data obtained during the through image display (step S116). The search for an object can be performed using a well-known face detection technique, feature amount detection technique, or the like. In addition, for example, if the object is empty, the low cone trust portion at the top of the screen is identified as empty. Further, the feature of the important image may be read from a specific image database, and the object may be searched for by comparison with the important image. In the case of a dedicated device, there is a high possibility that important images can be set automatically. For example, for an in-vehicle device, an image of an oncoming vehicle (that is, an automobile) or a pedestrian (that is, a person) can be used as an important image. In addition, an image of a crack or disconnection can be used as an important image for an industrial device, and an image of a lesion can be used as an important image for a medical device. These important images may be associated with text or sound. After searching for the object, the control unit 124 stores the position information of the searched object on the touch operation device 104 and the object information in a storage unit (not shown) (step S117).

  If it is determined in step S114 that the object only mode is not specified, that is, the position only mode is selected, the control unit 124 causes the operator to store information on the touch position in a storage unit (not shown) (step S118). The position only designation mode here is a mode that enables designation of a point. Not only the position but also the range may be designated. In this case, for example, when the operator detects that a certain range is traced with the finger as shown by the illustrated arrow A as shown in FIG. 11, the control unit 124 shows the positional information of the range. Not stored in the storage unit.

  Subsequently, the control unit 124 determines whether or not the operator's voice has been input according to the signal from the voice input unit 116 (step S119). When determining in step S119 that there is no voice input, the control unit 124 shifts the process to step S123.

  When it is determined in step S119 that there is a voice input, the control unit 124 takes in the voice data and recognizes the contents of the fetched voice data (step S120). When the content of the recognized audio data matches the name of an object registered in advance (color or shape may be used), the control unit 124 obtains the recognized object as image data obtained during live view display. Search from (step S121). After searching for the object, the control unit 124 stores the position information of the searched object on the touch operation device 104 and the object information in a storage unit (not shown) (step S122).

  Subsequently, the control unit 124 determines whether or not a shooting operation has been performed by the operator (step S123). The photographing operation is, for example, a pressing operation of the release button 1141 of the operation unit 114. If it is determined in step S123 that no photographing operation has been performed, the control unit 124 returns the process to step S101.

  If it is determined in step S123 that a shooting operation has been performed, the control unit 124 performs shooting and image recording operations (step S124). As a photographing operation, the control unit 124 performs AF control to focus the optical system of the photographing lens 102 and performs AE control to determine an exposure condition during photographing. In step S124, for example, AF control is performed so that the center subject or the closest subject is in focus. Subsequently, the control unit 124 causes the imaging unit 118 to perform an imaging operation for recording in accordance with the exposure condition determined by the AE control. After the recording imaging operation, the control unit 124 performs general recording image processing such as white balance correction, gamma correction, and compression processing on the image data obtained as a result of the imaging operation of the imaging unit 118. Do. After the image processing for recording, the control unit 124 generates an image file based on the compressed image data, and causes the recording unit 120 to record the generated image file.

  After the photographing and image recording operations, the control unit 124 determines whether or not at least one of the object information and the position information is currently stored (step S125). If it is determined in step S125 that neither the object information nor the position information is stored, that is, the object is not specified, the control unit 124 returns the process to step S101.

  When it is determined in step S125 that at least one of the object information and the position information is stored, the control unit 124 records the stored object information or position information in step S124. (Step S126). Thereafter, the control unit 124 returns the process to step S101.

  If it is determined in step S101 that the operation mode is not the shooting mode, the control unit 124 determines whether or not the operation mode is the reproduction mode (step S127). When determining in step S127 that the operation mode is not the reproduction mode, the control unit 124 returns the process to step S101. If it is determined in step S127 that the operation mode is the playback mode, the control unit 124 performs a playback mode process (step S128). Details of the processing in the reproduction mode will be described later in detail.

  FIG. 12 is a flowchart illustrating vibration control processing. The vibration control process is a process for giving a tactile stimulus to the finger of the operator according to the positional relationship between the position touched by the operator and the position of the object.

  In FIG. 12, the control unit 124 determines whether or not both the object information and the position information are currently stored (step S201). When it is determined in step S201 that both the object information and the position information are stored, that is, when designation is made in a mode other than the position designation mode, the control unit 124 stores the position of the stored object. It is determined whether or not the touch position is the same position (step S202). Here, if the positional difference between the two is within an allowable range, the same position is assumed.

  When it is determined in step 202 that the position of the stored object and the touch position are the same position, the control unit 124 corresponds to the touch position detection point in the piezoelectric element 1082 as shown in FIG. The part which was made is vibrated (step S203). Thereafter, the control unit 124 ends the process of FIG. As described above, in the present embodiment, when the operator designates the object “empty” by voice, for example, as shown in FIG. 14 and touches the empty portion, vibration stimulation is given to the finger. As a result, the operator can know that the finger is touching the position of the object specified in advance, that is, the position of the object in the screen without looking at the display on the display unit 106. For this reason, the user can instantly determine that the object that he / she has specified is there, and can shoot without worrying about checking the screen one by one. It is possible to confirm that there is something other than visual.

  Here, FIG. 13 shows an example in which the piezoelectric element 1082 is vibrated. On the other hand, the piezoelectric element 1042 formed around the lens frame may be vibrated, or the release button 1141 may be vibrated. That is, it may be vibrated as long as it can be touched by an operator's finger during photographing. Furthermore, the position of the target object may be notified to the operator using other methods such as sensory stimuli other than vision other than tactile stimuli caused by vibration, for example, auditory stimuli using voice. When notifying by voice, the control unit 124 controls the speaker 123 to emit voice. When applied to wearable devices, vibration to the parts that come into contact with the human body is effective.For in-vehicle devices, the handle may vibrate, and for industrial and medical devices, the operating members should vibrate. It is effective.

  When it is determined in step 202 that the position of the stored object and the touch position are not the same position, the control unit 124 sets the touch position as shown in FIGS. 15A and 15B. The piezoelectric elements 1042 are sequentially dented from the corresponding detection point toward the detection point of the position of the object (step S204). The dent amount may be a fixed amount, or may be changed according to the amount of deviation between the position of the object and the touch position. Thereafter, the control unit 124 ends the process of FIG. For example, as shown in FIG. 16, when the designated object is a person (face) and the operator's touch position is shifted to the right from the face position, the operator's finger position is shifted to the left. The piezoelectric elements 1082 are sequentially dented in the direction. As shown in FIG. 17, when the designated object is a person (face) and the touch position of the operator is shifted upward from the face position, the operator moves downward from the finger position. The piezoelectric elements 1082 are sequentially dented in the direction. In this way, the finger can be guided to the position of the object by sequentially denting the portion from the touch position to the position of the object. Not only the vibration of the touched part, but also the vibration of another part or part corresponds to an image spread in two dimensions, and the deviation can be effectively transmitted if the vibration change direction is such that the X direction and the Y direction can be understood. It is also effective to replace this with the horizontal direction and the vertical direction to vibrate. By such a device, it is possible to recognize that there is a shift without visual observation. Of course, the vibration may be changed depending on the direction and the degree of displacement. The vibration pattern may be changed depending on the deviation in the Y direction and the deviation in the X direction. In addition, audio assistance may be provided, or a light emission pattern that does not require viewing the screen may be used.

  Here, FIG. 15A and FIG. 15B are examples in which the piezoelectric element 1082 is vibrated. On the other hand, the piezoelectric element 1042 may be vibrated. In this case, the direction of deviation and the vibration part may be associated with each other. For example, if the operator's touch position is shifted to the right from the face position, the right part of the lens frame is vibrated, and if the operator's touch position is shifted from the face position, the upper part of the lens frame Vibrate.

  In step S201, when it is determined that both the object information and the position information are not stored, that is, when designation is made in the position only designation mode, the control unit 124 stores the stored position and the object information. It is determined whether or not the position is the same position (step S205). Here, if the positional difference between the two is within an allowable range, the same position is assumed. The object is assumed to be predetermined. In the determination in step S205, the determined object is searched from the image data and its position is compared with the storage position.

If it is determined in step S205 that the stored position and the position of the target object are the same position, the control unit 124 corresponds to the touch position detection point on the piezoelectric element 1082, as in FIG. Is vibrated (step S206). Thereafter, the control unit 124 ends the process of FIG. In this way, the operator can know that the object is at a position designated in advance without looking at the display on the display unit 106. Further, when there is no vibration, it indicates that the object is not at the designated position. At this time, the operator is expected to move the camera 1 so that the object comes to a specific position while viewing the object without looking at the screen of the display unit 106. As a result, for example, even when the angle of view set in the camera 1 is narrow and the object cannot be confirmed on the display screen, it is possible to capture the object correctly within the display screen. is there.
Moreover, it is not necessary to limit to the object for photography. The technique of this embodiment may be applied to a dedicated observation device. In the case of such a dedicated observation device, an object may be automatically set, and it may be determined whether the set object is in the screen. Such an object corresponds to an oncoming vehicle or a pedestrian that is a vehicle-mounted device, a crack or a disconnection that is an industrial device, a lesion image that is a medical device, or the like. The image features of these objects may be associated with text or audio and can be specified. Further, instead of determining whether the object can be photographed, it may be merely determining whether the object has entered the image range.

  Here, in step S206, the piezoelectric element 1042 formed around the lens frame may be vibrated, or the release button 1141 may be vibrated. That is, it may be vibrated as long as it can be touched by an operator's finger during photographing. Furthermore, the operator may be notified of the position of the object using other methods such as sound other than the vibration of the piezoelectric element 1082.

  If it is determined in step 205 that the position of the stored object and the position of the object are not the same position, the control unit 124 moves from the touch position in the direction of deviation between the storage position and the position of the object. Then, the piezoelectric elements 1042 are sequentially dented (step S207). The dent amount may be a fixed amount, or may be changed according to the amount of deviation between the position of the object and the touch position. Thereafter, the control unit 124 ends the process of FIG.

  Here, in step S206, the piezoelectric element 1042 may be vibrated. In this case, the direction of deviation and the vibration part may be associated with each other. For example, if the operator's touch position is shifted to the right from the face position, the right part of the lens frame is vibrated, and if the operator's touch position is shifted from the face position, the upper part of the lens frame Vibrate.

  FIG. 18 is a flowchart showing processing in the playback mode. In the present embodiment, vibration control is performed not only in the shooting mode but also in the playback mode.

  In the playback mode, the control unit 124 causes the display unit 106 to display a list of image files recorded in the recording unit 120 (step S301). Subsequently, the control unit 124 determines whether an image file has been selected by the operator (step S302). If it is determined in step S302 that no image file has been selected, the control unit 124 shifts the process to step S319.

  If it is determined in step S302 that an image file has been selected, the control unit 124 reproduces the image file selected by the operator (step S303). In the reproduction process, the control unit 124 reads the image file selected by the operator from the recording unit 120, and decompresses the read image file. Then, the control unit 124 inputs the decompressed image data to the display unit 106 and displays the image.

  The processes in steps S304 to S318 after the reproduction of the image file are the object designation and vibration control processes described in steps S107 to S122. Therefore, the description is omitted. However, from FIG. 18, processing related to shooting is eliminated. Specifically, the shooting and image recording processing in step S111 and the designation mode only in the position in step S118 are eliminated.

  After the vibration control process and target object specification, the control unit 124 determines whether or not the operator has instructed the end of the reproduction mode (step S319). For example, when the operation mode is switched, it is determined that the end of the reproduction mode is instructed. If it is determined in step S319 that the end of the playback mode has not been instructed, the control unit 124 determines whether an instruction to change the image has been given by the operator (step S320). For example, when an instruction to end reproduction of an image being reproduced is given, it is determined that an instruction to change the image has been given. If it is determined in step S320 that an instruction to change the image has been given, the control unit 124 returns the process to step 301. If it is determined that an instruction to change the image has not been given, the control unit 124 returns the process to step 303.

  As described above, according to the present embodiment, the piezoelectric element is bent or vibrated according to the positional relationship between the touch position of the operator and the position of the object specified in advance, so that the finger of the operator Tactile stimulation is given. Thus, the operator can recognize the position of the object in the image without looking at the display screen of the display unit 106 or without special knowledge.

  Moreover, according to this embodiment, it is possible to bend a piezoelectric element for every part by forming a signal electrode for every part of the piezoelectric element formed in the touch sensor. Thereby, an arbitrary position on the touch sensor can be bent independently of the others. Thereby, it is possible to give a tactile stimulus directly to an operator's finger.

  In the present embodiment, a piezoelectric element is configured by sandwiching a polylactic acid layer between two electrodes. Since the polylactic acid layer is a transparent material, even if a piezoelectric element is formed on the display unit 106, the possibility that the display on the display unit 106 is obstructed can be reduced. Furthermore, it is possible to increase the bending displacement by laminating the D-form polylactic acid layer and the L-form polylactic acid layer.

  Here, the piezoelectric element can also be used as a touch sensor by connecting an electrode to any one of the polylactic acid layers. That is, the touch position can be detected by detecting a voltage signal corresponding to the bending of the polylactic acid layer when the operator's finger is touched. In such a configuration, the piezoelectric element and the touch sensor can be combined into one.

  Further, for example, if an object is specified by voice, display of an image by the display unit 106 or the finder display unit 110 is not necessary. That is, even if the display unit 106 or the finder display unit 110 is not provided, if the position of the detection point of the touch sensor is associated with each position of the image, vibration control similar to that illustrated in FIG. 12 is performed. Is possible. Here, the embodiment of polylactic acid that vibrates, senses position, and has excellent permeability, flexibility, and moldability has been described in detail, but it goes without saying that the same effect can be achieved by combining a plurality of other materials. Yes. Polylactic acid is easy to change the vibration position, but it can also be handled by moving the entire touch panel.

  Although the present invention has been described based on the above embodiments, the present invention can be applied to cameras other than consumer cameras, video cameras, and mobile devices, and can be applied to industrial, medical, and monitoring dedicated imaging devices and observation devices. Is also possible. Further, the present invention is not limited to the above-described embodiment, and various modifications and applications are possible within the scope of the gist of the present invention. Further, in the description of each operation flowchart described above, the operation is described using “first”, “next”, and the like for convenience, but this does not mean that it is essential to perform the operations in this order. Absent.

  In addition, each processing method by the image processing apparatus in the above-described embodiment, that is, the processing shown in each flowchart can be stored as a program that can be executed by the control unit 124. In addition, memory cards (ROM cards, RAM cards, etc.), magnetic disks (floppy disks, hard disks, etc.), optical disks (CD-ROM, DVD, etc.), storage media of external storage devices such as semiconductor memories, etc. are distributed. be able to. And the control part 124 can perform the process mentioned above by reading the program memorize | stored in the storage medium of this external storage device, and operation | movement being controlled by this read program.

  Further, the above-described embodiments include various stages of the invention, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some configuration requirements are deleted from all the configuration requirements shown in the embodiment, the above-described problem can be solved, and this configuration requirement is deleted when the above-described effects can be obtained. The configuration can also be extracted as an invention.

  DESCRIPTION OF SYMBOLS 1 ... Camera, 102 ... Shooting lens, 103 ... Lens drive part, 104 ... Touch operation apparatus, 106 ... Display part, 108 ... Touch operation apparatus, 110 ... Finder display part, 112 ... Eye sensor, 1141 ... Release button, 114 ... Operation 116: Audio input unit, 118 ... Imaging unit, 120 ... Recording unit, 122 ... Clock unit, 123 ... Speaker, 124 ... Control unit, 202 ... Lower insulating layer, 204 ... Signal electrode, 206 ... Lower ground ( GND) electrode, 208 ... polylactic acid layer, 210 ... upper ground (GND) electrode, 212 ... through hole, 214 ... upper insulating layer, 216 ... soft material layer, 218 ... base material layer, 1041,1081 ... touch sensor, 1042 , 1082 ... Piezoelectric element, 1043, 1083 ... Application section

Claims (10)

A touch sensor that has a plurality of detection points associated with the position of the image and detects an operator's touch operation on the detection points;
And the object specifying section that specifies the name of a target on shooting,
An image determination unit for searching the object having the name specified by the object specifying unit from features of the image;
A sensory stimulus providing unit for providing a sensory stimulus to the operator, which is provided at a location different from the touch sensor, where the operator's finger can be touched at the time of shooting ;
A control unit that applies the sensory stimulus by the sensory stimulus providing unit according to a positional relationship between the position of the object in the image searched by the image determination unit and the touch position detected by the touch sensor;
An information device comprising:   The information device according to claim 1, wherein the sensory stimulus is a tactile stimulus.   The information device according to claim 2, wherein when the position of the object is different from the touch position, the control unit applies the tactile stimulus by changing vibration according to the difference.   The information device according to claim 2, wherein the sensory stimulus providing unit applies the tactile stimulus by bending the touch position.   The said control part gives the said tactile stimulus by bending-vibrating the part of the said sensory stimulus provision part corresponding to the said touch position when the position of the said target object and the said touch position are equal. Information equipment described in   The sensory stimulus providing unit changes a bending position of the sensory stimulus providing unit according to a shift direction between the position of the object and the touch position when the position of the object and the touch position are not equal. 6. The information device according to claim 4, wherein the tactile stimulus is applied. A position designating unit for designating a position in the image;
The said control part gives the said sensory stimulus by the said sensory stimulus provision part according to the positional relationship of the position designated by the said position designation | designated part and the position of the said target object in the said image. The information device according to any one of 6.   The information device according to claim 1, wherein the object specifying unit specifies the object by voice input.   The information device according to claim 1, wherein the object designating unit designates the object by a touch operation. A touch sensor that has a plurality of detection points associated with the position of the image and detects an operator's touch operation on the detection points, and a sensory stimulus providing unit that provides sensory stimuli to the operator A control method for information equipment,
And to specify the name of a target on shooting,
Searching for the object of the specified name from image features;
Depending on the positional relationship between the position of the target object in the searched image and the touch position detected by the touch sensor, it can be touched by an operator's finger at the time of shooting and is different from the touch sensor. and providing a sensory stimulus to the operator by the sensory stimulation providing unit is provided,
A method for controlling an information device, comprising:

Images