JP2011193275A - Display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem: although understanding the feeling of a photographing person in photographing enables a viewer who views a photographed image to more greatly enjoy the image, a conventional device cannot give such enjoy to the viewer. <P>SOLUTION: This display device includes an input section for inputting biological information of a photographing person when photographing an image, a display section for displaying the image, and a processing section for estimating the photographing person's feeling in photographing from the biological information of the photographing person, and processing an image to be displayed on the display section on the basis of the estimated feeling. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a display device.

2. Description of the Related Art An imaging device that detects a photographer's biological information, estimates a photographer's emotion, and supports a photographing operation based on the estimated emotion is known. For example, the camera shake correction gain is adjusted in accordance with the height of emotion to improve the tracking characteristic of the correction lens.
[Prior art documents]
[Patent Literature]
[Patent Document 1] Japanese Unexamined Patent Application Publication No. 2009-210992

  Emotions such as feelings of sensation, uplifting, anxiety, and disappointment that a photographer has when shooting while facing the subject are all different. However, it is generally difficult to express such feelings at the time of shooting in the subject image itself. In some cases, the photographer may try not to express his / her emotion in the subject image. On the other hand, a viewer who appreciates a photographed image can enjoy the photographed image more deeply if he / she understands the photographer's emotion at the time of photographing.

  Similarly, what kind of emotions are held when an image is photographed by the photographer is also helpful for the viewer to enjoy the photographed image. It is also interesting for the viewers to see what emotions the viewers have when they make the subject see the images.

  However, such a way of enjoying an image cannot be realized by a conventional apparatus.

  In order to solve the above-described problem, a display device according to a first aspect of the present invention includes an input unit that inputs a photographer's biological information when an image is captured, a display unit that displays an image, and a biological image of the photographer. A processing unit that estimates an emotion at the time of shooting from the information and processes an image to be displayed on the display unit based on the estimated emotion.

  Moreover, in order to solve the said subject, the display apparatus in the 2nd aspect of this invention is the input which inputs an observer's biometric information according to operation of a display part which displays an image to an observer, and a display part And a processing unit that estimates an emotion during image observation from the biological information of the observer and processes an image displayed on the display unit based on the estimated emotion.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

It is principal part sectional drawing of the camera system which concerns on 1st Embodiment. It is an upper general view of a camera system. It is a figure which shows the 1st state which hold | gripped the imaging lens with the left hand. It is a figure which shows the 2nd state which hold | gripped the imaging lens with the left hand. It is a figure which shows the camera main body side biological sensor part provided in the camera main body. It is a figure which shows the structure of a heart rate detection apparatus and a pulse wave detection apparatus. It is a block diagram of a camera system. It is an operation | movement flowchart at the time of the imaging | photography mode which concerns on 1st Embodiment. It is an operation | movement flowchart at the time of the reproduction | regeneration mode concerning 1st Embodiment. It is a figure which shows the example of a character. It is a figure which shows the example of a display state. It is an external view of the liquid crystal display which concerns on 2nd Embodiment. It is a block diagram of a liquid crystal display.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

(First embodiment)
FIG. 1 is a cross-sectional view of a main part of a camera system 1 according to the first embodiment. In the present embodiment, a camera system 1 will be described as an example of a display device. The camera system 1 is a lens-interchangeable single-lens reflex camera that functions as an imaging device by combining a camera body 2 and a replaceable photographing lens 3. The camera system 1 includes a display device.

  The photographic lens 3 includes a lens group 4 including a focus lens, a zoom lens, and an anti-vibration lens, an aperture 5, an angular velocity sensor 6 that detects shake of the camera system 1, a driving device (not shown) that drives the lens group 4, and the like. The angular velocity sensor 6 detects an angular velocity around at least two axes orthogonal to the optical axis. The drive device has a plurality of motors composed of, for example, a vibration wave motor and a VCM, drives the focus lens in the optical axis direction, and drives the anti-vibration lens in a direction different from the optical axis direction.

  The photographic lens 3 has a lens CPU 7 that controls the entire photographic lens 3 and cooperates with the camera body 2, and controls the photographer's heart rate, blood flow, blood pressure, sweating, body temperature, and photographic lens 3. It has a lens-side biosensor unit 8 that detects a gripping pressure and the like.

  The camera body 2 retreats so that the light beam from the photographic lens 3 is reflected and guided to the finder optical system 26 and the light beam from the photographic lens 3 is incident on an image sensor 27 composed of a CCD or a CMOS. A main mirror 28 that swings at the retracted position is provided. A partial area of the main mirror 28 is a semi-transmissive area, and the camera body 2 includes a sub-mirror 30 that reflects the light beam transmitted through the semi-transmissive area to the focus detection sensor 29. The sub mirror 30 swings in conjunction with the main mirror 28, and when the main mirror 28 takes the retracted position, the sub mirror 30 also retracts from the light flux. The focus detection sensor 29 detects the focus state of the incident light beam by the phase difference method.

  The light beam reflected by the main mirror 28 at the reflection position is guided to the finder optical system 26 via the focusing screen 31 and the pentaprism 32. The finder optical system 26 is composed of a plurality of lenses, and the photographer can check the object field with the finder optical system 26.

  A part of the light beam passing through the pentaprism 32 is guided to the photometric sensor 40. The photometric sensor 40 measures the luminance distribution of the object scene by measuring the light beam incident on the photographing lens 3 for each of a plurality of regions. In addition, a GPS (Global Positioning System) module 41 is provided above the pentaprism 32 and receives signals from GPS satellites to acquire position information where the camera system 1 exists. . Further, the camera body 2 includes a microphone 42 that captures the sound of the object scene at a position that does not interfere with the photographic lens 3 in the vicinity of the mount portion of the photographic lens 3, and a speaker 43 near the finder optical system 26.

  When the main mirror 28 is at the retracted position, the light beam from the photographing lens 3 enters the image sensor 27 via the low-pass filter 33. An imaging board 34 is provided in the vicinity of the imaging element 27, and a rear monitor 37 is provided behind the imaging board 34 so as to face the outside. The rear monitor 37 constitutes a display device.

  A camera body side biosensor unit that detects the photographer's heart rate, blood flow, blood pressure, sweating volume, body temperature, pressure for gripping the camera body 2, and the like at the position where the photographer's right finger touches the camera body 2 16. The specific configuration and arrangement of the camera body side biosensor unit 16 will be described later.

  FIG. 2 is an upper schematic view of the camera system 1 according to the present embodiment. Specifically, it is a diagram showing a state where the operator holds the camera body 2 with the right hand and holds the photographing lens 3 with the left hand. As described above, the photographing lens 3 includes the lens-side biological sensor unit 8 that detects the photographer's heart rate, blood flow, blood pressure, sweating volume, body temperature, pressure for grasping the photographing lens 3, and the like. The sensor unit 8 is disposed at a position where the finger or palm of the left hand of the photographer touches.

In the figure, a state in which a heart rate detection device 9 and a pulse wave detection device 12 are provided as part of the lens-side biosensor unit 8 is shown. The heart rate detection device 9 has a plurality of electrode parts separated from each other, each composed of a reference electrode 9a and a detection electrode 9b, and detects the heart rate of the photographer. Pulse-wave detecting device 12 has a plurality of light emitting portions _{12a (12a} 1 _{~12a 4)} and a light receiving portion _12b corresponding to these (12b 1 ~12b ₄₎ are alternately arranged, the photographer's pulse wave detection To do. The pulse wave detection device 12 is used to measure the blood flow and blood pressure of the photographer as will be described later.

  As described above, the camera main body 2 has the camera main body side biosensor unit 16 at a position where the right finger of the photographer touches. When the photographer holds the camera body 2, the thumb of the right hand is located on the back of the camera body 2, and the index finger is located in the vicinity of the release SW 24, so that it is separated from the other three fingers located on the grip portion. . For this reason, the camera body-side biosensor unit 16 has a camera rear surface position corresponding to the thumb of the right hand, a position near the release SW 24 corresponding to the index finger, and a camera front position near the grip section corresponding to the other three fingers. And spaced apart from each other. In addition, the camera main body side biosensor unit 16 corresponding to the index finger may be provided on the surface of the release SW 24.

  In the camera body 2, at least one of the camera front surface position where the three fingers other than the thumb and index finger of the right hand hold the camera body 2 and the camera rear surface position corresponding to the thumb of the right hand determines the camera body 2. It is a holding part to hold. In addition, a number of operations SW are provided on the back of the camera body 2, and these operations SW are operated with the right thumb. A shooting mode SW 25 for setting a shooting mode is provided on the upper surface of the camera body 2.

  FIG. 3 is a diagram illustrating a first state in which the photographing lens 3 is held with the left hand. The first state is a state in which the back of the left hand is positioned on the lower side and the photographing lens 3 is gripped. FIG. 4 is a diagram illustrating a second state in which the photographing lens 3 is held with the left hand. The second state is a state where the back of the left hand is positioned on the left side and the photographing lens 3 is gripped.

  When the photographer performs a zoom operation or a manual focus operation while holding the photographing lens 3, the thumb of the left hand is separated from the other fingers. Also, the manner of holding the photographic lens varies depending on different photographers and different photographing situations (for example, horizontal position photographing and vertical position photographing). Therefore, a plurality of lens-side biosensor units 8 (8A to 8D) are provided on the circumference of the photographing lens 3.

  Specifically, the lens-side biosensor unit 8 is at least one of a zoom operation position and a manual focus operation position, a position corresponding to the thumb of the left hand, and a position corresponding to a finger other than the thumb. They are spaced apart. More specifically, the lens-side biosensor unit 8 is provided at a position where the zoom operation rubber and the focus operation rubber are provided so as to contact the left hand or to face the left hand. Yes.

  In addition to the above-described heart rate detection device 9 and pulse wave detection device 12, the lens-side biological sensor unit 8A includes a sweating sensor 13 that detects the amount of sweating of the photographer, a temperature sensor 14 that detects the body temperature of the photographer, and imaging. A pressure sensor 15 is provided for detecting the pressure with which the person holds the photographing lens 3.

  Similarly to the lens-side biosensor unit 8A, the lens-side biosensor units 8B to 8D each include a heart rate detection device 9, a pulse wave detection device 12, a sweat sensor 13, a temperature sensor 14, and a pressure sensor 15. In this way, by providing a plurality of lens-side biosensors 8 (8A to 8D) on the circumference of the photographing lens 3, it is possible to detect biometric information from the palm of the left hand.

  In the present embodiment, a plurality of lens-side biosensors 8 (8A to 8D) are provided according to the zoom operation position, the manual focus operation position, and the like. The plurality of lens-side biosensor units 8 may be provided at positions other than the above-described positions as long as the biometric information can be detected even when the gripping method is changed. Further, since the force with which the thumb of the left hand grips the taking lens 3 is not so large, the pressure sensor 15 corresponding to the thumb of the left hand may be omitted in the lens side biosensor units 8B and 8C. Similarly, when high detection accuracy is not required for the lens-side biological sensor unit 8, the number of parts of the photographing lens 3 can be suppressed by appropriately omitting the sensor at the position corresponding to the thumb of the left hand. Further, the lens CPU 7 may be controlled to emit light only when a finger is applied to the light emitting unit 12a of the pulse wave detecting device 12.

  FIG. 5 is a view showing the camera body-side biosensor unit 16 provided in the vicinity of the release SW 24 of the camera body 2. As shown in the drawing, the camera body side biosensor unit 16 includes a heart rate detection device 17 having the same configuration as the heart rate detection device 9 and a pulse wave detection device 20 having the same configuration as the pulse wave detection device 12. Have. The camera body-side biosensor unit 16 includes a sweat sensor 21 that detects the amount of sweat of the photographer, a temperature sensor 22 that detects the body temperature of the photographer, and a pressure sensor that detects the pressure at which the photographer holds the camera body 2. 23. As described above, the camera body-side biosensor unit 16 is not limited to the position corresponding to the right index finger shown in the figure, but also to the camera back position corresponding to the thumb and the camera front position corresponding to the other three fingers. Although provided, each has the same configuration.

  FIG. 6 is a diagram illustrating a configuration of the heart rate detection device 17 and the pulse wave detection device 20 included in the camera body-side biosensor unit 16. As shown in FIG. 6 (a), the heart rate detection device 17 has a plurality of electrode parts separated from each other, each composed of a reference electrode 17a and a detection electrode 17b, and detects the heart rate of the photographer. Further, as shown in FIG. 6B, the pulse wave detection device 20 is configured by alternately arranging a plurality of light emitting units 20a (20a1 to 20a4) and light receiving units 20b (20b1 to 20b4) corresponding thereto. Detect the photographer's pulse.

  FIG. 7 is a block diagram of the camera system 1 according to the present embodiment. The image pickup substrate 34 includes a drive circuit 10 that drives the image pickup device 27, an A / D conversion circuit 11 that converts the output of the image pickup device 27 into a digital signal, an image processing control circuit 18 including an ASIC, and signals from the image pickup device 27 A contrast AF circuit 19 for extracting the high-frequency component of the.

  The image processing control circuit 18 performs image processing such as white balance adjustment, sharpness adjustment, gamma correction, and gradation adjustment on the image signal converted into the digital signal, and also performs image compression such as JPEG to generate an image file. Generate. The image file has an area for storing information about the image as meta information. For example, various information can be additionally added as EXIF information in a JPEG file. Although specifically described later, in the present embodiment, the biological information of the photographer at the time of photographing is described as EXIF information. The generated image file is stored in the image recording medium 35. The image recording medium 35 may be a recording medium such as a flash memory that is detachable from the camera body 2, or may be a recording medium such as an SSD (Solid State Drive) built in the camera body 2. .

  The image signal subjected to the image processing is displayed on the rear monitor 37 under the control of the rear monitor control circuit 36. If the image signal captured immediately after the image capture is displayed for a predetermined time, a REC review display that allows the photographer to visually recognize an image corresponding to the image file recorded on the image recording medium 35 can be realized. Further, live view display can be realized by sequentially displaying an object scene image, which is continuously photoelectrically converted by the image sensor 27, on the rear monitor 37 without being recorded on the image recording medium 35. Further, an object scene image that is continuously photoelectrically converted by the image sensor 27 is converted into, for example, MPEG, H.264, or the like. If moving image compression processing such as H.264 is performed by the image processing control circuit 18 and recorded in the image recording medium 35, moving image shooting can be realized. At this time, the sound of the object scene collected by the microphone 42 is also compressed and recorded in synchronization with the moving image data. The frame rate of the generated moving image is selected and set from a plurality of frame rates such as 30 fps, for example.

  Further, when the reproduction mode is selected by the photographer, the image file after photographing can be displayed on the rear monitor 37 under the control of the rear monitor control circuit 36. At this time, the image file recorded on the image recording medium 35 is read by the camera body CPU 46 and developed by the image processing control circuit 19. When the photographer gives an instruction to display another image file by operating the operation member, the camera body CPU 46 develops the indicated image file and displays it on the rear monitor 37. A specific display flow will be described later.

  The contrast AF circuit 19 extracts a high-frequency component of the image signal from the image sensor 27 to generate an AF evaluation value signal, and detects a focus lens position where this becomes the maximum. Specifically, a predetermined high-frequency component is extracted from the image signal input from the image processing control circuit 18 using a bandpass filter, and detection processing such as peak hold and integration is performed to generate an AF evaluation value signal. . The generated AF evaluation value signal is output to the camera body CPU 46.

  The lens CPU 7 drives the anti-vibration lens in the photographing lens 3 in a direction different from the optical axis direction so as to cancel the camera shake detected by the angular velocity sensor 6 to realize optical camera shake correction. Camera shake correction is not limited to such optical camera shake correction, and a drive mechanism is provided to the image sensor 27, and image sensor drive-type camera shake correction that cancels camera shake by driving in a direction different from the optical axis direction is adopted. You can also. Furthermore, an electronic camera shake that calculates a motion vector between a plurality of images output from the image processing control circuit 18 and cancels camera shake by controlling an image reading position so as to cancel the calculated motion vector between images. Corrections can also be employed. Optical camera shake correction and image sensor-driven camera shake correction are particularly suitable for still image shooting and are also applied to moving image shooting. Electronic camera shake correction is suitable for moving image shooting. These methods can be selectively and additionally employed.

  As described above, the photometric sensor 40 measures the luminance distribution of the object scene by measuring the luminous flux incident on the photographing lens 3 for each of a plurality of areas, and outputs the measurement result to the camera body CPU 46. The camera body CPU 46 calculates an exposure value according to the selected photometry mode. As a metering mode, a split metering mode that balances bright and dark parts, a center-weighted metering mode that properly exposes the center of the screen, a spot metering mode that properly exposes a narrow area of the selected focus point, and the like can be selected. .

  The calendar unit 38 includes a crystal oscillator, a timekeeping integrated circuit, and the like, and holds calendar information such as year / month / day / hour / minute / second. The camera body CPU 46 can appropriately detect information related to time from the calendar unit 38. The GPS module 41 receives a signal from a GPS satellite and acquires latitude, longitude, and altitude information where the camera body 2 exists. The camera body CPU 46 can appropriately detect information regarding the position where the camera body 2 is present from the GPS module 41.

  The flash ROM 39 is an EEPROM (registered trademark) and is a storage device that stores various adjustment values and setting values in addition to a program for operating the camera system 1. Specifically, AF adjustment data, AE adjustment data, manufacturing date / time data, setting SW setting history, and the like are stored. The flash ROM 39 also stores the normal biological information value of the photographer. In the present embodiment, the flash ROM 39 stores a heart rate, a blood flow rate, a blood pressure, a body temperature, a pressure for gripping the camera body 2, and a pressure for gripping the photographing lens 3 as biometric information values.

  The RAM 44 is a high-speed RAM such as a DRAM in which a program stored in the flash ROM 39 is expanded and the camera body CPU 46 can access at high speed. In particular, various adjustment values and setting values that are frequently referred to are also copied from the flash ROM 39 to facilitate access from the camera body CPU 46. The character ROM 45 stores a plurality of characters to be displayed in a superimposed manner when a captured image is displayed on the rear monitor 37. Details will be described later.

  The release SW 24 is a two-stage switch. When the photographer presses the release SW 24 halfway, the camera body CPU 46 performs photographing preparation operations such as autofocus and photometry. Further, when the photographer fully presses the release SW 24, the camera body CPU 46 starts detecting the photographer's biological information using the lens-side biosensor unit 8 and the camera body-side biosensor unit 16, and also captures still images and moving images. Start shooting operation.

  The camera body CPU 46 controls the entire camera system 1 in cooperation with the lens CPU 7. In particular, in the present embodiment, the camera body CPU 46 acquires the photographer's biological information based on the outputs of the lens-side biosensor unit 8 and the camera-body side biosensor unit 16 in the photographing mode, and uses it as meta information of the photographed image. Record in an image file. In the playback mode, the biometric information is read from the meta information, the emotion of the photographer at the time of shooting is determined, the read shot image is processed based on the determined emotion, and then displayed on the rear monitor 37. Alternatively, biometric information when the captured image is displayed is acquired to determine the emotion, and the captured image to be displayed is processed based on the determined emotion, and then displayed on the rear monitor 37 again. Here, first, acquisition of the biometric information of the photographer by the lens side biosensor unit 8 and the camera body side biosensor unit 16 will be described.

  First, heart rate measurement will be described. As described above, the reference electrode 9a and the detection electrode 9b of the heart rate detection device 9 are provided at a position where the photographer holds the photographing lens 3 with the left hand, and the photographer grasps the camera body with the right hand. Are provided with a reference electrode 17a and a detection electrode 17b of the heart rate detection device 17. The detection potential from the detection electrodes 9b and 16b is output to the camera body CPU 46 after the potential difference is amplified by a differential amplifier (not shown). The camera body CPU 46 calculates the photographer's heart rate based on the potential difference between the detection electrodes 9b and 16b.

  For example, when the photographer does not hold the photographing lens 3, since the left hand of the photographer does not touch the reference electrode 9a and the detection electrode 9b, the space between the reference electrode 9a and the detection electrode 9b is open. . When the gap between the reference electrode 9a and the detection electrode 9b is open, the lens CPU 7 determines that the photographer is not holding the photographing lens 3. Similarly, the camera body CPU 46 determines that the photographer is not holding the camera body 2 when the reference electrode 17a and the detection electrode 17b of the heart rate detection device are open.

  Next, blood pressure measurement will be described. The pulse wave detection devices 12 and 20 measure the blood pressure of the photographer. Since pulse wave detection device 12 and pulse wave detection device 20 have the same configuration, pulse wave detection device 12 will be described as an example. The pulse wave detection device 12 emits, for example, infrared rays from the light emitting unit 12a, the infrared rays are reflected by the finger arteries, and the reflected infrared rays are received by the light receiving unit 12b which is an infrared sensor, whereby the pulse of the finger part of the hand is received. Detect waves. That is, the blood flow rate of the peripheral blood vessel is detected. The camera body CPU 46 calculates the blood pressure of the photographer based on the pulse wave from the pulse wave detection device 12. When the lens CPU 7 determines from the outputs of the reference electrode 9a and the detection electrode 9b of the heart rate detection device 9 that a finger of the photographer such as a little finger is not touching the photographing lens 3, the lens CPU 7 responds to the finger. By prohibiting the light emission of the arranged light emitting unit 12a, useless light emission is prevented and stray light is not emitted to the object scene. Similarly, the camera body CPU 46 determines that the pulse wave detection device 20 does not touch the camera body 2 based on the outputs of the reference electrode 17 a and the detection electrode 17 b of the heart rate detection device 17, for example. You may prohibit the light emission of the light emission part 20a.

  Next, perspiration measurement will be described. Sweating can be detected by measuring hand impedance. The perspiration sensors 13 and 21 have a plurality of electrodes and detect perspiration. The reference electrode 9a and the reference electrode 17a may be used as a part of the plurality of electrodes. The sweat sensor 13 is provided in each of the lens-side biosensors 8A to 8D. However, mental sweat such as impression, excitement, and tension has less sweat amount and shorter sweat time. You may provide only in the lens side biosensor part 8B and C located in the palm side of a middle hand with much quantity.

  Next, temperature measurement will be described. The temperature sensors 14 and 22 use a thermistor method in which the resistance value changes due to heat. Sweating includes the above-described mental sweating and thermal sweating for body temperature regulation, and mental sweating and thermal sweating interfere with each other. For this reason, the camera body CPU 46 can determine whether the photographer's sweating is mental sweating or thermal sweating based on the outputs of the sweating sensors 13 and 21 and the outputs of the temperature sensors 14 and 22. For example, if the temperature detected by the temperature sensor 22 is high and the sweat signal from the sweat sensor 21 is always detected, the camera body CPU 46 can determine that the sweat is thermal. Further, the camera main body CPU 46 can determine that the photographer is in a state of emotion, excitement, tension, etc., when the sweat signal from the sweat sensor 21 is output irregularly and is determined to be mental sweating. When the temperature sensors 14 and 22 are omitted, the main body CPU 44 determines that the sweat signal from the sweat sensors 13 and 21 is based on the positional information of the GPS module 41, the time information from the calendar unit 38, and the like. Or thermal sweating may be determined. Further, the lens CPU 7 can determine whether the sweat of the left hand is mental sweating or thermal sweating based on the outputs of the sweating sensor 13 and the temperature sensor 14.

  Next, pressure measurement will be described. The pressure sensor 15 is a capacitance type sensor, and measures the amount of deformation caused by the pressing force when the photographer holds the photographing lens 3. In the present embodiment, the pressure sensor 15 is provided below the operation rubber. The pressure sensor 23 is a similar capacitance type sensor, and measures the amount of deformation caused by the pressing force when the photographer holds the camera body 2. A strain gauge, an electrostrictive element, or the like may be used as the pressure sensors 15 and 23.

  As described above, the camera body CPU 46 cooperates with the lens CPU 7 to acquire the photographer's biological information based on the outputs of the lens side biological sensor unit 8 and the camera body side biological sensor unit 16. Hereinafter, the control flow of the camera system 1 will be specifically described.

  FIG. 8 is an operation flowchart in the shooting mode according to the first embodiment. The flow starts from a state in which the camera system 1 is turned on and waits for a shooting instruction from the photographer. In step S101, the camera body CPU 46 waits until SW1, which is a half-press of the release SW 24, is turned on. When SW1 is turned on by the photographer, the camera body CPU 46 executes a shooting preparation operation. Specifically, the exposure value is determined by the output of the photometric sensor 40, and autofocus is performed by a phase difference method using the focus detection sensor 29. The auto focus may be a contrast method using a contrast AF circuit 19.

  In step S103, the camera body CPU 46 waits until SW2, which is the full press of the release SW 24, is turned on. If the SW 2 is not turned on within a predetermined time, the camera body CPU 46 returns to step S101. If SW2 is turned on, the camera main body CPU 46 proceeds to step S104, and acquires the photographer's biometric information from at least one of the camera main body side biosensor unit 16 and the lens side biosensor unit 8. Further, the camera body CPU 46 executes the main photographing operation in step S105. In the actual photographing operation, the lens CPU 7 operates the diaphragm 5 according to the exposure value determined in step S102, and the camera body CPU 46 causes the focal plane shutter to travel to guide the subject light flux to the image sensor 27. Further, the camera body CPU 46 performs charge reading by multiplying the output of the image sensor 27 by a predetermined gain according to the determined imaging sensitivity. In addition, the order of step S104 and step S105 is not restricted to this, A reverse order may be sufficient and both may be processed in parallel.

  In step S106, the image processing control circuit 18 performs image processing and compression processing on the image signal thus generated to generate an image file. At this time, the camera body CPU 46 records the biometric information acquired in step S104 in the meta information of the image file. Further, in step S107, the camera body CPU 46 estimates the photographer's emotion from the acquired biometric information, and performs image file rating based on the estimated emotion. Rating is a grading according to the importance of a captured image. For example, 5 levels of 1 to 5 are assigned. In general, a photographer assigns a grade while visually recognizing a photographed image. However, in the present embodiment, the importance of an image is determined from the estimated emotion and the grade is automatically assigned. Examples of emotion classification will be described later. For example, if the estimated emotion is a positive emotion such as “love” or “joy”, 5 or 4 having high importance is assigned. Also, if the estimated emotion is “normal”, the average value of 3 is given, and 2 and 1, which are less important for negative emotions such as “indifference”, “anger”, “pain”, etc. Give.

  As described above, the lens-side biosensor unit 8 and the camera body-side biosensor unit 16 are configured as a collection of various sensors, and each sensor outputs different types of biometric information. By judging these outputs alone or in combination, the photographer's emotion can be estimated. For example, when a high heart rate and mental sweating are detected, it can be estimated that the photographer feels “impression”. The correspondence between the sensor output and the emotion is obtained in a verification manner, and a table indicating the correspondence can be stored in the flash ROM 39. Therefore, in emotion determination, it is determined whether the acquired biometric information matches a specific emotion pattern described in the table.

  When the camera body CPU 46 performs rating in step S107 and records the grade in the meta information of the image file, the camera body CPU 46 proceeds to step S108 and records the generated image file in the image recording medium 35. Thus, a series of operation flow in the shooting mode is completed. Note that the camera main body CPU 46 displays the confirmation of deletion again for the photographer when the photographer instructs to delete the image photographed by the operation SW and the rating is 4 or 5 is included. , Deletion of images with a rating of 4 or 5 may be prohibited. Thus, it is possible to prevent the photographer from deleting an image with a high rating by mistake.

  Next, the playback mode will be described. FIG. 9 is an operation flowchart in the reproduction mode according to the first embodiment. The playback mode starts when the photographer selects the playback mode by operating the operation member. Note that the playback mode can be operated even by another observer, not the photographer of the image.

  When the playback mode is selected by the photographer, the camera body CPU 46 determines whether the current playback mode is set to the shooting mode or the display mode in step S201. The shooting mode is a mode in which a photographer's emotion at the time of shooting is estimated and a captured image is processed and displayed based on the emotion. On the other hand, the display mode is a mode in which an emotion of the observer operating the camera system 1 in the playback mode is estimated when the captured image is displayed, and the captured image is processed and redisplayed based on the emotion. .

  First, the shooting mode will be described. If it is determined in step S201 that the shooting mode is set, the camera body CPU 46 reads out the first target image file from the image recording medium 35 using the image processing control circuit 18 in step S301, and the work memory of the RAM 44. Expand on top. Subsequently, in step S302, the biometric information of the photographer added at the time of photographing is read out from the meta information of the target image file. In step S303, the camera body CPU 46 estimates the photographer's emotion from the read biological information. As described above, the photographer's emotion is estimated from the biological information by referring to the table indicating the correspondence between the biological information stored in the flash ROM 39 and the emotion pattern, and the specific emotion in which the read biological information is described in the table. Determine if it matches the pattern.

  If the emotion is estimated in step S303, in step S304, the camera main body CPU 46 uses the image processing circuit 18 to process the captured image developed on the work memory of the RAM 44 based on the estimated emotion. Although a specific processing example will be described later, for example, a character corresponding to the estimated emotion is read from the character ROM 45 and is combined with a captured image.

  The captured image processed based on the estimated emotion is converted into a display signal by the rear monitor control circuit 36 and displayed on the rear monitor 37 in step S305. If there is an instruction to display the next image within a certain time, the process returns to step S301 again and the same display processing is repeated for the next image. The next image display instruction may be an operation of the operation member by the photographer or an automatic reproduction instruction such as a slide show mode. If there is no instruction to display the next image within a certain time, the playback mode is terminated. In this way, if the viewer who surrounds and appreciates the rear monitor 37 recognizes the emotion of the photographer at the time of photographing, the photographed image can be enjoyed more.

  Next, the display mode will be described. If it is determined in step S201 that the display mode is set, the camera body CPU 46 reads out the first target image file from the image recording medium 35 using the image processing control circuit 18 in step S401, and stores the work memory in the RAM 44. Expand on top. Subsequently, in step S <b> 402, the camera body CPU 46 displays the photographed image developed on the work memory of the RAM 44 on the rear monitor 37 using the rear monitor control circuit 36.

  When the target image is displayed and viewed by the observer, the camera main body CPU 46 performs the operation of the observer who operates the camera system 1 based on the outputs of the lens side biosensor unit 8 and the camera main body side biosensor unit 16 in step S403. Obtain biometric information. In step S404, the observer's emotion is estimated from the acquired biological information.

  After the emotion is estimated in step S404, in step S405, the camera body CPU 46 uses the image processing circuit 18 to convert the photographed image developed on the work memory of the RAM 44 based on the estimated emotion as in step S304. Process. The captured image processed based on the emotion estimated in this way is displayed on the rear monitor 37 in step S406 by being replaced with the pre-processed captured image already displayed by the rear monitor control circuit 36. If there is an instruction to display the next image within a predetermined time, the process returns to step S401 again and the same display processing is repeated for the next image. The next image display instruction may be an operation of the operation member by the photographer or an automatic reproduction instruction such as a slide show mode. If there is no instruction to display the next image within a certain time, the playback mode is terminated. In this way, if a viewer who surrounds and appreciates the rear monitor 37 recognizes the emotion of the operator at the time of display, the photographed image can be enjoyed more.

  Note that the reproduction of the captured image in the reproduction mode can select normal reproduction in which the recorded image file is expanded and reproduced in addition to the above-described image processing. The display method in the playback mode is selected and set by menu selection or the like.

  There are various methods for processing and expressing a photographed image in order to reflect the estimated emotion in the photographed image. For example, the camera body CPU 46 can change the size of the captured image to be displayed. When the feeling of “shame” is estimated from the acquired biometric information, the captured image is processed to be small and displayed. On the contrary, when the emotion of “joy” is estimated, it is gradually enlarged and displayed so as to zoom. At this time, the camera body CPU 46 shortens the display time when the feeling of “shame” is estimated to be shorter than the standard reproduction time, or lengthens the display time when the feeling of “joy” is estimated longer than the standard reproduction time. Or you may.

  Furthermore, the camera main body CPU 46 can perform processing for changing at least one of the luminance, the hue, and the hue of the captured image to be displayed. When the emotion of “love” is estimated from the acquired biometric information, a process for bringing the captured image to a warm color system is performed. On the other hand, when the feeling of “despair” is estimated, the entire captured image is processed and displayed dark.

  Or at the time of automatic reproduction in the slide show mode or the like, it is possible to change the display time of each continuously displayed captured image. When the feeling of “impression” is estimated from the acquired biometric information, the display time per sheet is shortened and displayed.

  As another example, the character corresponding to the emotion may be read from the character ROM 45 and combined with the captured image. FIG. 10 is a diagram illustrating an example of a character corresponding to an emotion. As shown in the figure, characters corresponding to various emotions such as "normal", "joy", "affection", "shame", "impression", "pain", "surprise", "confidence", "indifference", "sadness", "anger", "despair" It is stored in the character ROM 45. For character synthesis, an image signal may be generated by being directly embedded in an image, or an image signal may be generated by superimposing a layered image on a captured image by the rear monitor control circuit 36. Further, the display signal may be generated so that the character is displayed in a free area different from the area where the captured image of the rear monitor 37 is displayed.

  Although FIG. 10 shows characters corresponding to each emotion, a plurality of characters may be prepared for one emotion depending on the degree. From the biometric information, for example, it can be determined whether the person feels more intense or slightly impatient in the feeling of “impression”. Therefore, a plurality of characters are prepared so that the character showing the same “impression” can understand the degree of “impression” from the design. Then, a character corresponding to the selected emotion and its degree is selected, processed, and displayed. The degree of emotion can also be expressed by the character size, color, number, display time, and animation movement. For example, the camera body CPU 46 may display the rating grade and the number of characters the same, and when the rating grade is small, the character color of the character is made darker (gray or black), When the rating grade is large, a lighter color (red or sky blue) may be used. The camera body CPU 46 may process and change the read character itself based on the determined emotion (or biological information) and the degree thereof. For example, the camera body CPU 46 may blink the character in accordance with the photographer's heart rate, and may enlarge or reduce the photographed image according to the heart rate. Also good.

  FIG. 11 is a diagram illustrating an example of a display state in which characters are combined and displayed. The figure shows a state in which a joy character 47 representing “joy” moves around superimposed on a captured image. On the rear monitor 37, in addition to the display of the photographed image in which the characters are combined, a file name display 48 and a grade icon display 49 indicating the above-described five-step rating results are performed. The camera body CPU 46 may synthesize a character in addition to the photographed image and omit the display of the grade icon display 49, or the reverse display.

  In addition, it is not necessary to synthesize a character for all images, a character may be synthesized only to an image that is estimated to have a large emotional undulation, and the number of characters to be synthesized may be set on the image (by default) 5 or 10% of all captured images). Furthermore, when the playback image increase is a moving image, character composition may be performed in a scene where the undulation of the photographer's emotion during moving image shooting is large.

  Note that the reflection of the estimated emotion in the captured image is not limited to the one processing described above. Of course, processing may be performed in combination, or emotion may be reflected in the captured image using other expression methods. In addition, although the digital single-lens reflex camera was illustrated as 1st Embodiment, it cannot be overemphasized that it is applicable also to a compact camera, a mirrorless single-lens camera, a video camera, a mobile phone, etc.

(Second Embodiment)
Next, a display according to the second embodiment will be described. FIG. 12 is an external view of a liquid crystal display 51 according to the second embodiment. The display as a display device is not limited to the liquid crystal display 51 but may be a plasma display, an organic EL display, a projector, or the like. The basic concept in this embodiment is the same as that in the first embodiment. That is, also in this embodiment, an emotion is estimated based on biological information, and an image displayed based on the estimated emotion is processed and then displayed on the liquid crystal display 51. The liquid crystal display 51 includes a personal computer, a television, a digital photo frame, and the like.

  As shown in FIG. 12, the liquid crystal display 51 includes a display main body 52 and a remote controller 53 operated by an observer as main components. The remote controller 53 is provided with a biosensor unit 54 having the same configuration as the lens-side biosensor unit 8 and the camera body-side biosensor unit 16.

  Since the remote controller 53 is grasped by the observer, the biological information of the observer is acquired from the grasped palm. The acquired biological information is transmitted to the display main body 52 together with the operation command of the observer, for example, by infrared communication.

  The display main body 52 includes a slot 56 that receives an image recording medium 55 in which a photographed image file is stored. Note that this image recording medium 55 may be used as the image recording medium 35 of the first embodiment, and the biological information and rating grades of the photographer recorded as EXIF information may be transferred to the display main body 52. . Capture of the captured image file from the outside is not limited to the removable image recording medium 55, but can be realized by a communication means using a LAN or the like.

  The read captured image is displayed in the image display area 58 of the display screen 57. At the same time, a grade icon corresponding to the image file name and grade is displayed in the information display area 59. In the figure, the affection character 60 representing “affection” is shown moving around superimposed on the captured image. Note that the image file to be captured is not limited to the captured image file generated by the camera system 1 and may be any image file as long as the image file has a corresponding format.

  FIG. 13 is a block diagram of the liquid crystal display 51. The remote controller 53 includes a remote controller CPU 61 that controls the remote controller 53, a biosensor unit 54 that acquires biometric information of a viewer who operates the remote controller 53, an operation reception unit 62 that receives an instruction from the viewer by input of an operation member, and a display body 52 includes a remote control transmission unit 63 that transmits the biological information acquired to 52 and the operation command signal of the observer. The display main body 52 also includes a display CPU 64 that controls the display main body 52, a remote control receiving unit 65 that receives biological information and an operation command signal from the remote control transmission unit 63, and an IF ( Interface) 66 is provided.

  Furthermore, the liquid crystal display 51 includes an emotion determination unit 68 that determines the corresponding emotion from the biological information added to the captured image file or the observer's biological information transmitted from the remote control transmission unit. In addition to a program for operating the liquid crystal display 51, a flash ROM 69 that is a storage device that stores various adjustment values and setting values, and a RAM 70 in which a program stored in the flash ROM 69 is expanded. The RAM 70 also serves as a working memory. The character ROM 71 stores a plurality of the above characters. The image read out and processed by the image processing control circuit 67 is displayed in the image display area 58 of the display screen 57 by the display drive circuit 72.

  The display control of the liquid crystal display 51 is the same as the operation flow in the reproduction mode according to the first embodiment described with reference to FIG. That is, if the captured image file has the biometric information of the photographer at the time of photographing, the photographing mode can be selected, and the observer can perform photographing based on the photographer's emotion displayed on the display screen 57. You can enjoy images. If the display mode is selected, it is possible to capture the biological information of the observer who operates the remote controller 53 and enjoy a photographed image processed based on the emotion. If the captured image file does not have the photographer's biometric information at the time of shooting, the display mode is automatically selected.

  Further, when the captured image file includes the photographer's biological information, the display CPU 64 performs character display according to the photographer's biological information and character display according to the biological information at the time of observation by the observer. You may perform the control to perform. In this case, it is preferable to make the character according to the photographer's biological information different from the character according to the biological information at the time of observation by the observer. For example, even in the same image, you can see the gap between the photographer's emotion and the observer's emotion, and if the photographer himself becomes an observer, know the emotion at the time of shooting and the emotion at the time of observation Can enjoy new enjoyment of image observation.

  In addition, the display CPU 64 may shorten the display time when the feeling of “shame” is estimated to be shorter than the standard reproduction time, or make the display time when the feeling of “joy” is estimated longer than the standard reproduction time. May be. Similarly, the display CPU 64 processes and displays a small captured image when the feeling of “shame” is estimated or displays a large processed image of the captured image when the feeling of “joy” is estimated. Also good. The display CPU 64 may change the display order of captured images to a rating order, and prohibits display of images with low ratings or images such as “shame”, “pain”, “confused”, “sadness”, “anger”, and “despair”. You may control to do. In this case, the display main body 52 is provided with a photographing unit (camera) as disclosed in Japanese Patent Application Laid-Open No. 2008-165209, and the display CPU 64 cooperates with the photographing unit (camera) to display an image according to who is an observer. Prohibit display. More specifically, the display CPU 64 displays all images when a person whose face is registered is observing, and when the person who is not registered as a face observes the display CPU 64. It is sufficient to display some images like For face registration, a closeness is set (for example, large, medium, small) using a photographing unit (camera) provided in the display main body 52, and the display CPU 64 displays an image and a display to be displayed according to the closeness. You may determine the image to prohibit.

  The biometric information is not limited to being recorded in the meta information of the image file, and a biometric information storage file describing the link information to the image file and the biometric information may be generated.

  Moreover, in the said embodiment, the lens side biosensor part 8, the camera main body side biosensor part 16, and the biosensor part 54 were each arrange | positioned at the apparatus main body. However, the biological sensor may be configured independently so as to be directly attached to the body of the photographer or the observer. For example, a wristwatch-type biosensor as disclosed in JP-A-2005-270543 (US Pat. No. 7,538,890) may be used. In this case, the apparatus side includes a wired or wireless biometric information acquisition unit. Note that when a plurality of biosensors are provided, the respective outputs may be different. In such a case, it is possible to determine in advance which biosensor is prioritized for output, and it is possible to adopt measures such as calculating an average value of outputs.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. Moreover, it is also possible to combine 1st Embodiment and 2nd Embodiment suitably. Furthermore, it will be apparent to those skilled in the art that various modifications or improvements can be added to the above embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

DESCRIPTION OF SYMBOLS 1 Camera system, 2 Camera body, 3 Shooting lens, 4 Lens group, 5 Aperture, 6 Angular velocity sensor, 7 Lens CPU, 8 Lens side biological sensor part, 9 Heart rate detection apparatus, 10 Drive circuit, 11 A / D conversion circuit , 12 Pulse wave detection device, 13 Sweating sensor, 14 Temperature sensor, 15 Pressure sensor, 16 Camera body side biosensor unit, 17 Heart rate detection device, 18 Image processing control circuit, 19 Contrast AF circuit, 20 Pulse wave detection device, 21 Sweating sensor, 22 Temperature sensor, 23 Pressure sensor, 24 Release SW, 25 Shooting mode SW, 26 Viewfinder optical system, 27 Image sensor, 28 Main mirror, 29 Focus detection sensor, 30 Sub mirror, 31 Focus plate, 32 Penta prism, 33 Low-pass filter, 34 Imaging substrate, 35 Image recording medium, 36 Rear monitor control circuit, 37 Rear monitor, 38 Calendar section, 39 Flash ROM, 40 Photometric sensor, 41 GPS module, 42 Microphone, 43 Speaker, 44 RAM, 45 Character ROM, 46 Camera body CPU, 47 Joy character, 48 files Name display, 49 grade icon display, 51 liquid crystal display, 52 display main body, 53 remote control, 54 biosensor unit, 55 image recording medium, 56 slot, 57 display screen, 58 image display area, 59 information display area, 60 loving character, 61 remote control CPU, 62 operation reception unit, 63 remote control transmission unit, 64 display CPU, 65 remote control reception unit, 66 IF (Interface), 67 image processing control circuit, 68 determination unit, 69 flash R M, 70 RAM, 71 character ROM, 72 a display driving circuit

Claims (13)

An input unit for inputting the photographer's biometric information when the image is taken;
A display unit for displaying the image;
A display device comprising: a processing unit that estimates an emotion during shooting from the biological information of the photographer and processes the image displayed on the display unit based on the estimated emotion. A display for displaying an image to an observer;
An input unit for inputting the biological information of the observer in response to an operation of the display unit;
A display device comprising: a processing unit that estimates an emotion during image observation from the biological information of the observer and processes the image displayed on the display unit based on the estimated emotion.   The display device according to claim 1, wherein the processing unit processes the image into a form in which the emotion can be understood.   The display device according to claim 1, wherein the processing unit changes a size of the image based on the biological information input from the input unit.   The display device according to claim 1, wherein the processing unit changes a display time of the images that are continuously displayed based on the biological information input from the input unit. A character storage unit that stores a plurality of characters according to the type of emotion,
6. The processing unit according to claim 1, wherein the processing unit reads at least one of the plurality of characters from the character storage unit and combines it with the image based on biological information input from the input unit. Display device. A character storage unit that stores a plurality of characters according to the degree of emotion,
The said process part reads at least 1 of these characters from the said character memory | storage part based on the biometric information input from the said input part, and combines with the said image. Display device.   The display device according to claim 6, wherein the processing unit changes at least one of a character size, a color, a number, and a display time read from the character storage unit.   The display device according to claim 1, wherein the processing unit changes at least one of luminance, saturation, and hue of the image based on biological information input from the input unit.   The display device according to claim 1, further comprising a rating unit that uses an input result of the biological information for rating the image.   The display device according to claim 1, further comprising a biological information storage unit that stores an input result of the biological information in association with the image.   The display device according to claim 11, wherein the processing unit reads the input result from the biological information storage unit and processes the image when different from when the biological information is input. The input unit inputs biometric information of a photographer when the image is captured,
The said process part estimates the emotion at the time of an image photography from the said biometric information of the said photographer, and processes the said image displayed on the said display part based on the estimated said emotion. The display device described in 1.

Images