JP2021158559A - Electronic device, control method of the same, and program - Google Patents

Abstract

To provide a live view display suitable for binocular photography of images with a wide viewing angle.SOLUTION: An electronic device for displaying a live view of a captured image includes display control means for controlling to display a live view by deforming at least one live image, based on a cylindrical view method, of a first live image captured through a first optical system and a second live image captured through a second optical system having the same orientation as the first optical system and having a parallax relative to the first live image.SELECTED DRAWING: Figure 4

Description

The present invention relates to electronic devices, control methods thereof, programs and recording media.

There is known a technique for displaying a three-dimensional VR (Virtual Reality) image by acquiring an image with a wide viewing angle with parallax using two optical systems, mapping it on a virtual sphere, and displaying it. .. A twin-lens VR camera for capturing an image with parallax has two optical systems facing the same direction, and can acquire two images with parallax by taking one shot. Some binocular VR cameras can capture a wide range of 180 degrees (hemisphere, 90 degrees in all directions from the center of the image) in each optical system. In addition, as a method of displaying a VR image, there is a "single-eye VR display" in which a VR image is mapped to a virtual sphere to display one image, or a VR image for the left eye and a right eye is displayed in the left and right areas. There is known a "two-lens VR display" that displays images side by side.

Further, in Patent Document 1, stereoscopic imaging is performed in which the same subject is photographed from the left and right viewpoints by using two imaging units arranged side by side with parallax, and an image for the left eye and an image for the right eye are acquired respectively. The device is disclosed.

Japanese Patent No. 4875225

Conventionally, many twin-lens VR cameras cannot display the live view at the time of shooting, and there is room for improvement as to what kind of live view display is preferable at the time of shooting. For example, it is conceivable to display both left and right live images when shooting, but it may be difficult for the user to know which one to look at, making it difficult to shoot like a normal camera. .. Further, since the image captured as the image for VR display is an image with a wide viewing angle and is a distorted image as if it was taken with a fisheye lens, the situation of the subject can be confirmed by displaying the live view as it is. Becomes difficult.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to display a live view suitable for capturing an image with a wide viewing angle with two eyes.

The electronic device of the present invention is an electronic device that displays a captured image in a live view, and has the same orientation as that of the first live image captured via the first optical system and the first optical system. At least one of the second live images taken through the second optical system and having a parallax with respect to the first live image is deformed based on the cylindrical projection and displayed in the live view. It is characterized by having a display control means for controlling the image.

According to the present invention, it is possible to display a live view suitable for capturing an image with a wide viewing angle with two eyes.

It is a figure which shows an example of the appearance composition of a camera. It is a figure which shows an example of the internal structure of a camera. It is a schematic diagram which shows an example of the structure of a lens unit. It is a flowchart which shows an example of the processing of a camera. It is a figure which shows an example of a live view image. It is a figure which shows an example of the display mode setting screen. It is a figure which shows an example of a live view image. It is a figure which shows an example of the image to be photographed.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. In the present embodiment, a case where the electronic device is a digital camera (imaging device) will be described as an example.

FIG. 1 is a diagram showing an example of an external configuration of a digital camera 100 (hereinafter referred to as a camera). FIG. 1A is a perspective view of the camera 100 as viewed from the front, and FIG. 1B is a perspective view of the camera 100 as viewed from the back.
The camera 100 has a shutter button 101, a power switch 102, a mode changeover switch 103, a main electronic dial 104, a sub electronic dial 105, a moving image button 106, and an external display unit 107 on the upper surface of the camera 100. The shutter button 101 is an operation unit for preparing for shooting or instructing shooting. The power switch 102 is an operation unit that switches the power of the camera 100 on and off. The mode changeover switch 103 is an operation unit for switching various modes. The main electronic dial 104 is a rotary operation unit for changing set values such as a shutter speed and an aperture. The sub-electronic dial 105 is a rotary operation unit for moving a selection frame (cursor), advancing an image, and the like. The moving image button 106 is an operation unit for instructing the start and stop of moving image shooting (recording). The out-of-finder display unit 107 displays various set values such as the shutter speed and the aperture.

Further, on the back surface of the camera 100, a display unit 108, a touch panel 109, a direction key 110, a SET button 111, an AE lock button 112, an enlargement button 113, a play button 114, a menu button 115, an eyepiece unit 116, and an eyepiece detection unit 118 are provided. , Has a touch bar 119. The display unit 108 displays an image and various kinds of information. The touch panel 109 is an operation unit that detects a touch operation on the display surface (touch operation surface) of the display unit 108. The direction key 110 is an operation unit composed of keys (four-way keys) that can be pressed up, down, left, and right, respectively. The operation can be performed according to the position where the direction key 110 is pressed. The SET button 111 is an operation unit that is mainly pressed when determining a selection item. The AE lock button 112 is an operation unit that is pressed when the exposure state is fixed in the shooting standby state. The enlargement button 113 is an operation unit for switching on / off of the enlargement mode in the live view display (LV display) of the shooting mode. When the enlargement mode is on, the live view image (LV image) is enlarged or reduced by operating the main electronic dial 104. Further, the enlargement button 113 is used when the reproduced image is enlarged or the enlargement ratio is increased in the reproduction mode. The play button 114 is an operation unit for switching between the shooting mode and the play mode. In the shooting mode, pressing the playback button 114 shifts to the playback mode, and the latest image among the images recorded on the recording medium 227, which will be described later, can be displayed on the display unit 108.

The menu button 115 is an operation unit that is pressed when displaying a menu screen on which various settings can be made on the display unit 108. The user can intuitively make various settings by using the menu screen displayed on the display unit 108 and the direction keys 110 and the SET button 111. The eyepiece portion 116 is a portion for eyepieces with respect to the eyepiece finder (look-in type finder) 117. The user can visually recognize the image displayed on the EVF 217 (Electronic View Finder), which will be described later, via the eyepiece 116. The eyepiece detection unit 118 is a sensor that detects whether or not the user is in contact with the eyepiece 116.

The touch bar 119 is a line-shaped touch operation unit (line touch sensor) capable of receiving a touch operation. The touch bar 119 can be touch-operated with the thumb of the right hand while the grip portion 120 is held with the right hand (the state of being held with the little finger, ring finger, and middle finger of the right hand) so that the shutter button 101 can be pressed with the index finger of the right hand. It is placed in a possible position. That is, the touch bar 119 can be operated in a state (shooting posture) in which the eyepiece portion 116 is in contact with the eyepiece, the eyepiece finder 117 is viewed, and the shutter button 101 can be pressed at any time. The touch bar 119 can accept tap operations on the touch bar 119 (operations of touching and releasing without moving within a predetermined period), slide operations to the left and right (operations of touching and then moving the touch position while touching), and the like. be. The touch bar 119 is an operation unit different from the touch panel 109, and does not have a display function. The touch bar 119 of the present embodiment is a multifunction bar, and functions as, for example, an M-Fn bar.

Further, the camera 100 has a grip portion 120, a thumb rest portion 121, a terminal cover 122, a lid 123, a communication terminal 124, and the like. The grip portion 120 is a holding portion formed in a shape that can be easily gripped by the right hand when the user holds the camera 100. The shutter button 101 and the main electronic dial 104 are arranged at positions that can be operated by the index finger of the right hand while holding the camera 100 by holding the grip portion 120 with the little finger, ring finger, and middle finger of the right hand. Further, in the same state, the sub electronic dial 105 and the touch bar 119 are arranged at positions that can be operated by the thumb of the right hand. The thumb rest portion 121 (thumb standby position) is a grip portion provided on the back side of the camera 100 at a position where the thumb of the right hand holding the grip portion 120 can be easily placed without operating any operation portion. The thumb rest portion 121 is composed of a rubber member or the like for increasing the holding force (grip feeling). The terminal cover 122 protects a connector such as a connection cable that connects the camera 100 to an external device. The lid 123 protects the recording medium 227 and the slot by closing the slot for storing the recording medium 227 described later. The communication terminal 124 is a terminal for communicating with the lens unit 200 side, which will be described later, to which the camera 100 can be attached and detached.

FIG. 2 is a diagram showing an example of the internal configuration of the camera 100. The same configurations as those in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted as appropriate. A lens unit 200 is attached to the camera 100.
First, the lens unit 200 will be described.
The lens unit 200 is a kind of interchangeable lens that can be attached to and detached from the camera 100. The lens unit 200 is a single-lens lens and is an example of a normal lens.
The lens unit 200 includes an aperture 201, a lens 202, an aperture drive circuit 203, an AF (autofocus) drive circuit 204, a lens system control circuit 205, a communication terminal 206, and the like.
The aperture 201 is configured so that the opening diameter can be adjusted. The lens 202 is composed of a plurality of lenses. The diaphragm drive circuit 203 adjusts the amount of light by controlling the aperture diameter of the diaphragm 201. The AF drive circuit 204 drives the lens 202 to focus. The lens system control circuit 205 controls the aperture drive circuit 203, the AF drive circuit 204, and the like based on the instructions of the system control unit 50 described later. The lens system control circuit 205 controls the aperture 201 via the aperture drive circuit 203, and focuses by shifting the position of the lens 202 via the AF drive circuit 204. The lens system control circuit 205 can communicate with the camera 100. Specifically, communication is performed via the communication terminal 206 of the lens unit 200 and the communication terminal 124 of the camera 100. The communication terminal 206 is a terminal for the lens unit 200 to communicate with the camera 100 side.

Next, the camera 100 will be described.
The camera 100 includes a shutter 210, an imaging unit 211, an A / D converter 212, a memory control unit 213, an image processing unit 214, a memory 215, a D / A converter 216, an EVF 217, a display unit 108, and a system control unit 50. ..
The shutter 210 is a focal plane shutter that can freely control the exposure time of the imaging unit 211 based on the instruction of the system control unit 50. The image pickup unit 211 is an image pickup element (image sensor) composed of a CCD, a CMOS element, or the like that converts an optical image into an electric signal. The imaging unit 211 may have an imaging surface phase difference sensor that outputs defocus amount information to the system control unit 50. The A / D converter 212 converts the analog signal output from the imaging unit 211 into a digital signal. The image processing unit 214 performs predetermined processing (pixel interpolation, resizing processing such as reduction, color conversion processing, etc.) on the data from the A / D converter 212 or the data from the memory control unit 213. Further, the image processing unit 214 performs a predetermined calculation process using the captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the obtained calculation result. By this processing, TTL (through-the-lens) AF processing, AE (automatic exposure) processing, EF (flash pre-flash) processing, and the like are performed. Further, the image processing unit 214 performs a predetermined calculation process using the captured image data, and performs a TTL method AWB (auto white balance) process based on the obtained calculation result.

The image data from the A / D converter 212 is written to the memory 215 via the image processing unit 214 and the memory control unit 213. Alternatively, the image data from the A / D converter 212 is written to the memory 215 via the memory control unit 213 without going through the image processing unit 214. The memory 215 stores the image data obtained by the imaging unit 211 and converted into digital data by the A / D converter 212, and the image data to be displayed on the display unit 108 or the EVF 217. The memory 215 has a storage capacity sufficient to store a predetermined number of still images, moving images for a predetermined time, and audio. Further, the memory 215 also serves as a memory (video memory) for displaying an image.

The D / A converter 216 converts the image display data stored in the memory 215 into an analog signal and supplies it to the display unit 108 and the EVF 217. Therefore, the display image data written in the memory 215 is displayed on the display unit 108 or the EVF 217 via the D / A converter 216. The display unit 108 and the EVF 217 display according to the analog signal from the D / A converter 216. The display unit 108 and the EVF 217 are, for example, a display such as an LCD or an organic EL. The digital signal that has been A / D converted by the A / D converter 212 and stored in the memory 215 is converted into an analog signal by the D / A converter 216, and is sequentially transferred to the display unit 108 or EVF217 for display. The view is displayed.

The system control unit 50 is a control unit including at least one processor and / or at least one circuit. That is, the system control unit 50 may be a processor, a circuit, or a combination of a processor and a circuit. The system control unit 50 controls the entire camera 100. The system control unit 50 realizes each process of the flowchart described later by executing the program recorded in the non-volatile memory 219. The system control unit 50 also controls the display by controlling the memory 215, the D / A converter 216, the display unit 108, the EVF 217, and the like.

Further, the camera 100 has a system memory 218, a non-volatile memory 219, a system timer 220, a communication unit 221 and an attitude detection unit 222, and an eyepiece detection unit 118.
For the system memory 218, for example, RAM is used. In the system memory 218, constants and variables for the operation of the system control unit 50, a program read from the non-volatile memory 219, and the like are expanded. The non-volatile memory 219 is a memory that can be electrically erased and recorded, and for example, EEPROM is used. The non-volatile memory 219 records constants, programs, and the like for the operation of the system control unit 50. The program here is a program for executing the flowchart described later. The system timer 220 is a time measuring unit that measures the time used for various controls and the time of the built-in clock. The communication unit 221 transmits and receives video signals and audio signals to and from an external device connected by a wireless or wired cable. The communication unit 221 can also be connected to a wireless LAN (Local Area Network) and the Internet. The communication unit 221 can also communicate with an external device using Bluetooth (registered trademark) or Bluetooth Low Energy. The communication unit 221 can transmit an image (including a live image) taken by the imaging unit 211 and an image recorded on the recording medium 227, and can receive image data and various other information from an external device. The posture detection unit 222 detects the posture of the camera 100 with respect to the direction of gravity. Based on the posture detected by the posture detection unit 222, whether the image taken by the image pickup unit 211 is an image taken by holding the camera 100 horizontally or an image taken by holding the camera 100 vertically. It can be discriminated. The system control unit 50 can add orientation information according to the posture detected by the posture detection unit 222 to an image file of an image taken by the image pickup unit 211, or rotate and record the image. be. For the posture detection unit 222, for example, an acceleration sensor, a gyro sensor, or the like can be used. It is also possible to detect the movement (pan, tilt, lift, whether or not it is stationary, etc.) of the camera 100 by using the posture detection unit 222.

The eyepiece detection unit 118 can detect the approach of some object to the eyepiece unit 116 of the eyepiece finder 117 having the EVF 217 built-in. For the eyepiece detection unit 118, for example, an infrared proximity sensor can be used. When an object approaches, the infrared rays projected from the light projecting unit of the eyepiece detection unit 118 are reflected by the object and received by the light receiving unit of the infrared proximity sensor. The distance from the eyepiece 116 to the object can be determined by the amount of infrared rays received. In this way, the eyepiece detection unit 118 performs eyepiece detection that detects the close distance of the object to the eyepiece 116. The eyepiece detection unit 118 is an eyepiece detection sensor that detects the approach (eyepiece) and separation (eyepiece) of the eye (object) with respect to the eyepiece 116 of the eyepiece finder 117. When an object approaching the eyepiece portion 116 within a predetermined distance is detected from the non-eyepiece state (non-approaching state), it is detected that the eyepiece has been eyepieced. On the other hand, when the object that has detected the approach is separated by a predetermined distance or more from the eyepiece state (approaching state), it is detected that the eye has been removed. The threshold value for detecting the eyepiece and the threshold value for detecting the eyepiece may be different, for example, by providing hysteresis. Further, after the eyepiece is detected, the eyepiece is assumed to be in the eyepiece state until the eyepiece is detected. After the eyepiece is detected, it is assumed that the eyepiece is not in the eyepiece state until the eyepiece is detected. The system control unit 50 switches the display (display state) / non-display (non-display state) of the display unit 108 and the EVF 217 according to the state detected by the eyepiece detection unit 118. Specifically, when at least in the shooting standby state and the display destination switching setting is automatic switching, the display destination is set to the display unit 108 and the display is turned on and the EVF 217 is hidden during non-eye contact. Further, during eye contact, the display destination is set to EVF217, the display is turned on, and the display unit 108 is hidden. The eyepiece detection unit 118 is not limited to the case of an infrared proximity sensor, and another sensor may be used as long as it can detect a state that can be regarded as an eyepiece.

Further, the camera 100 includes an outside viewfinder display unit 107, an outside viewfinder display drive circuit 223, a power supply control unit 224, a power supply unit 225, a recording medium I / F 226, an operation unit 228, and the like.
The out-of-viewfinder display unit 107 displays various set values of the camera 100 such as the shutter speed and the aperture via the out-of-viewfinder display drive circuit 223. The power supply control unit 224 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching a block to be energized, and the like, and detects whether or not a battery is installed, the type of battery, the remaining battery level, and the like. Further, the power supply control unit 224 controls the DC-DC converter based on the detection result and the instruction of the system control unit 50, and supplies a necessary voltage to each unit including the recording medium 227 for a necessary period. The power supply unit 225 is a primary battery such as an alkaline battery and a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery and a Li battery, an AC adapter and the like. The recording medium I / F 226 is an interface with a recording medium 227 such as a memory card or a hard disk. The recording medium 227 is a memory card or the like for recording a captured image, and is composed of a semiconductor memory, a magnetic disk, or the like. The recording medium 227 may be removable or may be built-in.

The operation unit 228 is an input unit that receives an operation (user operation) from the user, and is used to input various instructions to the system control unit 50. The operation unit 228 includes a shutter button 101, a power switch 102, a mode changeover switch 103, a touch panel 109, another operation unit 229, and the like. Other operation units 229 include a main electronic dial 104, a sub electronic dial 105, a moving image button 106, a direction key 110, a SET button 111, an AE lock button 112, an enlargement button 113, a play button 114, a menu button 115, a touch bar 119, and the like. Is included.
The shutter button 101 has a first shutter switch 230 and a second shutter switch 231. The first shutter switch 230 is turned on by a so-called half-press (shooting preparation instruction) during the operation of the shutter button 101, and the first shutter switch signal SW1 is generated. The system control unit 50 starts shooting preparation processing such as AF processing, AE processing, AWB processing, and EF processing by the first shutter switch signal SW1. The second shutter switch 231 is turned on when the operation of the shutter button 101 is completed, that is, when the shutter button 101 is fully pressed (shooting instruction), and the second shutter switch signal SW2 is generated. The system control unit 50 starts a series of shooting processes from reading the signal from the imaging unit 211 to generating an image file including the captured image and writing it to the recording medium 227 by the second shutter switch signal SW2.

The mode changeover switch 103 switches the operation mode of the system control unit 50 to any one of a still image shooting mode, a moving image shooting mode, a playback mode, and the like. The modes included in the still image shooting mode include an auto shooting mode, an auto scene discrimination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), and a program AE mode (P mode). In addition, there are various scene modes, custom modes, etc. that are shooting settings for each shooting scene. The user can directly switch to any of the above-mentioned shooting modes by the mode changeover switch 103. Alternatively, the user can use the mode changeover switch 103 to temporarily switch to the shooting mode list screen, and then selectively switch to any of the displayed multiple modes using the operation unit 228. Similarly, the moving image shooting mode may include a plurality of modes.

The touch panel 109 is a touch sensor that detects various touch operations on the display surface of the display unit 108 (the operation surface of the touch panel 109). The touch panel 109 and the display unit 108 can be integrally configured. For example, the touch panel 109 is attached to the upper layer of the display surface of the display unit 108 so that the light transmittance does not interfere with the display of the display unit 108. Then, by associating the input coordinates on the touch panel 109 with the display coordinates on the display surface of the display unit 108, the GUI as if the user can directly operate the screen displayed on the display unit 108. (Graphical user interface) can be configured. For the touch panel 109, any one of various methods such as a resistive film method, a capacitance method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, an image recognition method, and an optical sensor method can be used. Depending on the method, there are a method of detecting that there is a touch due to contact with the touch panel 109 and a method of detecting that there is a touch due to the approach of a finger or a pen to the touch panel 109. You may.

The system control unit 50 can detect the following operations or states on the touch panel 109.
-A finger or pen that has not touched the touch panel 109 newly touches the touch panel 109, that is, the start of touch (hereinafter referred to as touch-down).
-A state in which the touch panel 109 is touched with a finger or a pen (hereinafter referred to as touch-on).
-The touch panel 109 is moved while being touched by a finger or a pen (hereinafter referred to as a touch move (Touch-Move)).
-The finger or pen touching the touch panel 109 is separated (released) from the touch panel 109, that is, the end of the touch (hereinafter referred to as touch-up).
-A state in which nothing is touched on the touch panel 109 (hereinafter referred to as touch-off).

When touchdown is detected, touch-on is also detected at the same time. After touchdown, touch-on usually continues to be detected unless touch-up is detected. When a touch move is detected, a touch-on is detected at the same time. Even if touch-on is detected, touch move is not detected unless the touch position is moved. After it is detected that all the fingers and pens that have been touched are touched up, the touch is turned off.

These operations / states and the position coordinates of the finger or pen touching the touch panel 109 are notified to the system control unit 50 via the internal bus. The system control unit 50 determines what kind of operation (touch operation) has been performed on the touch panel 109 based on the notified information. Regarding the touch move, the moving direction of the finger or pen moving on the touch panel 109 can also be determined for each vertical component and horizontal component on the touch panel 109 based on the change in the position coordinates. When it is detected that the touch move is performed for a predetermined distance or more, it is determined that the slide operation has been performed. The operation of quickly moving a finger on the touch panel 109 by a certain distance and then releasing it is called flicking. In other words, flicking is an operation of quickly tracing the touch panel 109 as if flicking it with a finger. When it is detected that the touch move is performed at a predetermined speed or more over a predetermined distance and the touch-up is detected as it is, it is determined that the flick has been performed (it can be determined that there was a flick following the slide operation). Further, a touch operation in which a plurality of points (for example, two points) are touched together (multi-touch) to bring the touch positions closer to each other is called pinch-in, and a touch operation to move the touch positions away from each other is called pinch-out. Pinch out and pinch in are collectively called pinch operation (or simply pinch).

FIG. 3 is a schematic view showing an example of the configuration of the lens unit 300. FIG. 3 shows a state in which the lens unit 300 is attached to the camera 100. Of the cameras 100 shown in FIG. 3, the same configurations as those described in FIG. 2 are designated by the same reference numerals, and the description thereof will be omitted as appropriate.

The lens unit 300 is a kind of interchangeable lens that can be attached to and detached from the camera 100. The lens unit 300 is a twin-lens lens capable of photographing with parallax between a left image and a right image. The lens unit 300 has two optical systems, each having a wide viewing angle of about 180 degrees, and can photograph a range of the front hemisphere. Specifically, the two optical systems of the lens unit 300 have a viewing angle (angle) of 180 degrees in the left-right direction (horizontal angle, azimuth angle, yaw angle) and 180 degrees in the vertical direction (vertical angle, elevation / depression angle, pitch angle), respectively. You can shoot a subject (minutes).

The lens unit 300 includes a right-eye optical system 301R having a plurality of lenses and a reflection mirror, a left-eye optical system 301L having a plurality of lenses and a reflection mirror, and a lens system control circuit 303. The right eye optical system 301R corresponds to an example of the first optical system, and the left eye optical system 301L corresponds to an example of the second optical system. In the right eye optical system 301R and the left eye optical system 301L, the lenses 302R and 302L located on the subject side are oriented in the same direction, and their optical axes are substantially parallel. The lens unit 300 of the present embodiment is a VR180 lens for capturing an image for a so-called VR180, which is a VR image format capable of binocular stereoscopic viewing. The VR180 lens has a fisheye lens capable of capturing a range of approximately 180 degrees for each of the right eye optical system 301R and the left eye optical system 301L. As for the lens for VR180, it is sufficient that the right eye optical system 301R and the left eye optical system 301L can acquire images capable of twin-lens VR display as VR180, respectively, and a wide viewing angle of about 160 degrees narrower than the range of 180 degrees. It may be a lens capable of capturing the range of. The VR180 lens has a right image (first image) formed via the right eye optical system 301R and a left image (second image) formed via the left eye optical system 301L having parallax from the right image. Can be formed on one or two image sensors of the mounted camera.
Further, the lens unit 300 is attached to the camera 100 via the lens mount portion 304 and the camera mount portion 305 of the camera 100. When the lens unit 300 is attached to the camera 100, the system control unit 50 of the camera 100 and the lens system control circuit 303 of the lens unit 300 pass through the communication terminal 124 of the camera 100 and the communication terminal 306 of the lens unit 300. And are electrically connected.

In the present embodiment, the right image formed via the right eye optical system 301R and the left image formed via the left eye optical system 301L having parallax from the right image are arranged side by side in the imaging unit of the camera 100. The image is formed on 211. That is, two optical images formed by the right eye optical system 301R and the left eye optical system 301L are formed on one image sensor. The imaging unit 211 converts the imaged subject image (optical signal) into an analog electrical signal. In this way, by using the lens unit 300, it is possible to simultaneously (set) acquire two images having parallax from two locations (optical systems) of the right eye optical system 301R and the left eye optical system 301L. .. Further, by dividing the acquired image into an image for the left eye and an image for the right eye and displaying them in VR, the user can view a three-dimensional VR image in a range of approximately 180 degrees, so-called VR180. ..

Here, the VR image is an image that can be displayed in VR, which will be described later. VR images include omnidirectional images (omnidirectional images) taken with an omnidirectional camera (omnidirectional camera) and panoramic images having a wider image range (effective image range) than the display range that can be displayed at once on the display unit. Etc. are included. Further, the VR image is not limited to a still image, but also includes a moving image and a live image (an image acquired from a camera in almost real time). The VR image has a maximum image range (effective image range) of 360 degrees in the horizontal direction and 360 degrees in the vertical direction. Further, the VR image has a wider angle of view than the angle of view that can be taken by a normal camera, or a display range that can be displayed on the display unit at once even if the angle is less than 360 degrees in the horizontal direction and less than 360 degrees in the vertical direction. Images with a wider video range are also included. The image taken by the camera 100 using the lens unit 300 described above is a kind of VR image. The VR image can be displayed in VR by setting the display mode of the display device (display device capable of displaying the VR image) to "VR view", for example. A VR image with an angle of view of 360 degrees can be displayed in VR, and the user can change the posture of the display device in the left-right direction (horizontal rotation direction) to view a seamless omnidirectional image in the left-right direction. can.

Here, the VR display (VR view) is a display method (display mode) in which the display range can be changed, which displays an image of the viewing range according to the posture of the display device among the VR images. The VR display includes a "single-eye VR display (single-eye VR view)" that displays one image by performing a deformation (deformation in which distortion correction is applied) that maps a VR image to a virtual sphere. Further, in the VR display, the VR image for the left eye and the VR image for the right eye are each modified to be mapped to a virtual sphere and displayed side by side in the left and right areas. There is. Stereoscopic viewing is possible by performing "two-eye VR display" using a VR image for the left eye and a VR image for the right eye, which have parallax with each other. In any VR display, for example, when a user wears a display device such as an HMD (head-mounted display), an image in a visual field range corresponding to the orientation of the user's face is displayed. For example, in a VR image, at a certain point in time, an image having a viewing range centered on 0 degrees in the left-right direction (a specific direction, for example, north) and 90 degrees in the up-down direction (90 degrees from the zenith, that is, horizontal) is displayed. Suppose you are. When the posture of the display device is turned upside down from this state (for example, the display surface is changed from south to north), 180 degrees in the left-right direction (opposite direction, for example, south) and up-down direction in the same VR image. The display range is changed to an image with a viewing range centered on 90 degrees. That is, when the user turns his face from north to south (that is, faces backward) while wearing the HMD, the image displayed on the HMD is also changed from the north image to the south image. The VR image taken by using the lens unit 300 of the present embodiment is an image of VR180 taken in a range of about 180 degrees forward, and there is no image in a range of about 180 degrees backward. When such an image of VR180 is displayed in VR and the posture of the display device is changed to the side where the image does not exist, a blank area is displayed.

By displaying the VR image in VR in this way, the user visually feels as if he / she is in the VR image (in the VR space). The VR image display method is not limited to the method of changing the posture of the display device. For example, the display range may be moved (scrolled) according to a user operation via a touch panel, a direction button, or the like. In addition, in VR display (when the display mode is "VR view"), in addition to changing the display range by changing the posture, the display range is changed according to touch move on the touch panel, drag operation with the mouse, etc., pressing of the direction button, etc. You may change it. A smartphone attached to VR goggles (head mount adapter) is a type of HMD.

The image taken through the lens unit 300 in the camera 100 configured as described above is divided into two because it is taken through the right eye optical system and the left eye optical system, and each has a wide field of view. Distortion occurs because it is a corner image. Therefore, even if the live image taken through the lens unit 300 is displayed in the live view on the display unit 108, it may be difficult for the user to confirm the condition of the subject. In the present embodiment, the processing of the camera 100 that displays a live view suitable for capturing an image with a wide viewing angle with two eyes will be described with reference to the flowchart of FIG.

FIG. 4 is a flowchart showing an example of processing of the camera 100. The flowchart of FIG. 4 is realized by the system control unit 50 expanding the program recorded in the non-volatile memory 219 into the system memory 218 and executing the program. Further, the flowchart of FIG. 4 is started when the shooting mode is set to the still image shooting mode.

In S401, the system control unit 50 acquires information on the type of the mounted lens unit, and determines whether or not the lens unit is a twin-lens lens, here, whether or not it is a VR180 lens. Specifically, the system control unit 50 can acquire information on the type of the lens unit from the lens unit by communicating with the mounted lens unit via the communication terminals 124, 206, and 306. If the mounted lens unit is a VR180 lens, the process proceeds to S408. On the other hand, for example, when the lens unit is not attached or the lens unit is attached is a normal lens unit such as a single-lens lens and is not a VR180 lens, the process proceeds to S402.

In S402, the system control unit 50 performs a live view display in which the live image captured by the imaging unit 211 is displayed on the display unit 108 via the lens unit (for example, a normal lens unit such as a single lens). To display the live view image. Here, the live view image means an image in which the live image is displayed in the display unit 108 in the live view.
FIG. 5 is a diagram showing an example of the live view image 500 displayed on the display unit 108.
The live view image 500 shown in FIG. 5 is displayed over the entire display screen of the display unit 108. By visually recognizing the live view image 500 displayed on the display unit 108, the user can confirm whether or not the desired shooting can be performed.

In S403, the system control unit 50 determines whether or not the shutter button 101 is half-pressed during the live view display and the first shutter switch 230 is turned on. Specifically, the system control unit 50 determines whether or not the first shutter switch signal SW1 has been received. When the first shutter switch 230 is turned on, that is, when the first shutter switch signal SW1 is received, the process proceeds to S404. On the other hand, if the first shutter switch 230 is not on, that is, if the first shutter switch signal SW1 is not received, the process proceeds to S405.
In S404, the system control unit 50 performs a shooting preparation process. Specifically, the system control unit 50 starts AF processing, AE processing, AWB processing, EF processing, and the like.

In S405, the system control unit 50 determines whether or not the shutter button 101 is fully pressed and the second shutter switch 231 is turned on during the live view display. Specifically, the system control unit 50 determines whether or not the second shutter switch signal SW2 has been received. When the second shutter switch 231 is turned on, that is, when the second shutter switch signal SW2 is received, the process proceeds to S406. On the other hand, when the second shutter switch 231 is not on, that is, when the second shutter switch signal SW2 is not received, the process returns to S401.
In S406, the system control unit 50 performs a shooting process. Specifically, the system control unit 50 performs a series of imaging processes from reading a signal from the imaging unit 211 to generating an image file including the captured image and writing it to the recording medium 227. The file name of the image file recorded here is, for example, a file name conforming to the DCF standard such as "IMG_3456.jpg". Further, like the image of the image file for VR180 recorded in S418 described later, extraction (acquisition) of the left eye image range 801L and the right eye image range 801R and deformation processing for VR180 (deformation using the cylindrical projection method). ) Etc. are not performed.

In S407, the system control unit 50 displays a quick review of the image captured by the imaging unit 211 in response to the imaging instruction. The quick review display is a process of displaying an image for the user to confirm the captured image immediately after the image is captured in response to a shooting instruction. However, S407 can be omitted when it is set not to display the quick review by the menu or the like. After that, the system control unit 50 returns to S401 and again determines whether or not the lens unit is a VR180 lens.

In S408, the system control unit 50 determines whether or not the display mode at the time of live view display is the first display mode. Specifically, the system control unit 50 determines whether or not the information of the first display mode is stored in the control variable stored in the non-volatile memory 219. If the information of the first display mode is stored, the process proceeds to S409, and if the information of the first display mode is not stored, the process proceeds to S410.
Here, in the present embodiment, when the VR180 lens is attached, the user can set in advance the display mode for displaying the live image taken through the VR180 lens in the live view.

FIG. 6 is a diagram showing an example of the display mode setting screen 600.
The display mode setting screen 600 is displayed on the display unit 108 by selecting a predetermined display mode setting item from the menu screen displayed by pressing the menu button 115, for example.
On the display mode setting screen 600, the first display mode selection button 601 to the fourth display mode selection button 604 are displayed so as to be selectable. The user presses the SET button 111 after moving the cursor 605 with the arrow keys 110 to select the selection button corresponding to the desired display mode among the first display mode selection buttons 601 to the fourth display mode selection buttons 604. Select by. Alternatively, the user directly touches and selects the selection button corresponding to the desired display mode among the first display mode selection buttons 601 to the fourth display mode selection buttons 604. The system control unit 50 sets the display mode by substituting the display mode information corresponding to the selection button selected by the user operation into the control variable and storing it in, for example, the non-volatile memory 219. Even if the display mode selection screen 600 is not displayed, each time the display mode switching button included in the other operation unit 229 is pressed during display in any of the first to fourth display modes. The display mode may be sequentially switched and set (toggle). Each display mode corresponding to the first display mode selection button 601 to the fourth display mode selection button 604 will be described later with reference to FIG. 7.

In S409, the system control unit 50 performs a live view display in which the live image captured by the imaging unit 211 via the VR180 lens is displayed on the display unit 108 in the first display mode. The first display mode is a mode in which the live image captured by the imaging unit 211 is displayed on the display unit 108 in a live view without being deformed based on the cylindrical projection described later. In the first display mode of the present embodiment, the live image captured by the imaging unit 211 is displayed on the display unit 108 as it is.

FIG. 7A is a diagram showing an example of the live view image 700 displayed in the live view in the first display mode. The live view image 700 includes a live view image 701R corresponding to a live image taken via the right eye optical system 301R and a live view image 701L corresponding to a live image taken via the left eye optical system 301L. Is included. Further, paying attention to the live view images 701R and 701L, they are circular, and the closer to the outer edge of the circular shape, the greater the distortion.
The first display mode can be realized relatively easily because the image processing can be reduced because the live view is displayed without deforming the live image. Further, it can be seen that two wide-field images are taken by the two optical systems, and it is easy to recognize that the VR180 lens is attached and the shooting is specified. On the other hand, the first display mode is not suitable for the user to check the focus, the facial expression of the subject, and the like in detail because the two images are displayed separately on one screen and are greatly distorted. Therefore, the user selects the first display mode, for example, when it is not necessary to check the focus, the facial expression of the subject, or the like in detail.

Next, in S410, the system control unit 50 determines whether or not the display mode at the time of live view display is the second display mode. Specifically, the system control unit 50 determines whether or not the information of the second display mode is stored in the control variable stored in the non-volatile memory 219. If the information of the second display mode is stored, the process proceeds to S411, and if the information of the second display mode is not stored, the process proceeds to S412.

In S411, the system control unit 50 performs a live view display in which the live image captured by the imaging unit 211 via the VR180 lens is displayed on the display unit 108 in the second display mode. In the second display mode, the live image taken through the right eye optical system and the live image taken through the left eye optical system are each deformed based on the cylindrical projection, and both are displayed on the display unit 108. This is the mode to display the live view. Specifically, the system control unit 50 causes the image processing unit 214 to perform image processing on the live image. The image processing unit 214 cuts out a region (first predetermined region) corresponding to the live image captured via the right eye optical system from the live image captured by the imaging unit 211, and the left eye optical system. A region (second predetermined region) corresponding to the live image captured via is cut out. The image processing unit 214 transforms each of the cut out live images by converting them based on the cylindrical projection.

FIG. 7B is a diagram showing an example of the live view image 702 displayed in the live view in the second display mode. The live view image 702 includes a live view image 703R obtained by deforming a live image taken through the right eye optical system based on a cylindrical projection and a live image taken through the left eye optical system in a cylindrical projection. A live view image 703L which is modified based on the above is included. Further, paying attention to each of the live view image 703R and the live view image 703L, the distortion of the outer edge is suppressed.

The cylindrical projection is a projection in which an image on the surface of a sphere is projected onto a cylindrical surface in contact with the sphere. In this embodiment, as the cylindrical projection, for example, an equirectangular projection in which parallels and meridians intersect at right angles and at equal intervals is used. Among the lenses for VR180, the live image taken through the right eye optical system and the live image taken through the left eye optical system can be regarded as substantially hemispherical live images, respectively. By deforming a substantially hemispherical live image based on a cylindrical projection, each is converted into a rectangular live image in which distortion of the outer edge is suppressed.

As described above, in the second display mode, since the live image is deformed based on the cylindrical projection and displayed in the live view, the distortion of the outer edge is suppressed, so that the user can check the image quality. On the other hand, the second display mode is not suitable for checking the focus, the facial expression of the subject, and the like in detail because the two images are displayed separately on one screen. Therefore, for example, the user does not need to check the focus, the facial expression of the subject, and the like in detail, but selects the second display mode when he / she wants to roughly grasp the situation of the subject.

Next, in S412, the system control unit 50 determines whether or not the display mode at the time of live view display is the third display mode. Specifically, the system control unit 50 determines whether or not the information of the third display mode is stored in the control variable stored in the non-volatile memory 219. If the information of the third display mode is stored, the process proceeds to S413. On the other hand, when the information of the third display mode is not stored, the information of the fourth display mode is stored and the process proceeds to S414.

In S413, the system control unit 50 performs a live view display in which the live image captured by the imaging unit 211 via the VR180 lens is displayed on the display unit 108 in the third display mode. In the third display mode, one of the live image taken through the right eye optical system and the live image taken through the left eye optical system is deformed based on the cylindrical projection and deformed. This is a mode in which the generated image is moved to the center side of the display unit 108 and displayed in a live view. The system control unit 50 causes the image processing unit 214 to perform image processing on the live image. The image processing unit 214 cuts out a region corresponding to the live image taken by, for example, the right eye optical system from the live image taken by the imaging unit 211, and converts the cut out live image based on the cylindrical projection. It transforms by doing.

FIG. 7C is a diagram showing an example of the live view image 704 displayed in the live view in the third display mode. The live view image 704 is, for example, an image obtained by deforming a live image taken through a right eye optical system based on a cylindrical projection, and is displayed near the center of the display screen. Further, the live view image 704 is displayed in a larger size than the live view image 703R shown in FIG. 7 (b). The live view image 704 is not limited to the case where the live image taken through the right eye optical system is deformed, and may be an image obtained by deforming the live image taken through the left eye optical system. ..
Further, whether the live view image 704 is a deformed image of the live image taken through the right eye optical system or a deformed image of the live image taken through the left eye optical system. , The user may be able to select. In this case, for example, on the display mode setting screen 600 shown in FIG. 6, after selecting the third display mode selection button 603, the live image captured via the right eye optical system or the left eye optical system is further deformed. It can be realized by letting you select.

As described above, in the third display mode, since the live image is deformed based on the cylindrical projection and displayed in the live view, the distortion of the outer edge is suppressed and only one image is displayed on one screen. .. Therefore, the user can check the focus, the facial expression of the subject, and the like in detail, and can shoot with the same feeling as when shooting with a normal lens. Therefore, the user selects the third display mode, for example, when he / she wants to check the situation of the subject in detail.

Next, in S414, the system control unit 50 does not display the live image captured by the imaging unit 211 via the VR180 lens on the display unit 108 as the fourth display mode. That is, the fourth display mode is a mode in which the live view image is not displayed.

FIG. 7D is a diagram showing a display example in the fourth display mode. Here, the live view image is not displayed, and the information 705 indicating that the live image is being taken through the VR180 lens is displayed. The reason why the live view image is not displayed in this way is that in an image with a wide viewing angle, the facial expression or the like may not be confirmed because the main subject captured as a whole is small. Further, since the image having a wide viewing angle has a very wide shooting range, the user can shoot at least the main subject even if the shutter button 101 is pressed at an approximate position.
As described above, the fourth display mode can prompt the user to directly visually recognize and shoot the subject with the naked eye by not displaying the live view image. However, when the user looks into the eyepiece finder 117 with the eyepiece 116, if nothing is displayed on the EVF 217, it cannot be visually recognized in the first place. Therefore, even if the system control unit 50 stores the information of the fourth display mode, if the eyepiece detection unit 118 detects the approach of the object, the EVF 217 is other than the fourth display mode. Live view display may be performed in any of the first to third display modes.

Next, in S415, the system control unit 50 determines whether or not the shutter button 101 is half-pressed during the live view display and the first shutter switch 230 is turned on. This process is the same as that of S403. When the first shutter switch 230 is turned on, the process proceeds to S416. On the other hand, if the first shutter switch 230 is not on, the process proceeds to S417.
In S416, the system control unit 50 performs a shooting preparation process. This process is the same as S404.

In S417, the system control unit 50 determines whether or not the shutter button 101 is fully pressed and the second shutter switch 231 is turned on during the live view display. This process is the same as S405. When the second shutter switch 231 is turned on, the process proceeds to S418. On the other hand, if the second shutter switch 231 is not on, the process returns to S401.
In S418, the system control unit 50 performs a shooting process. Specifically, the system control unit 50 generates an image file including an image taken through the right eye optical system and an image taken through the left eye optical system from the signal read from the image pickup unit 211. A series of shooting processes are performed until the image is written on the recording medium 227. Further, the system control unit 50 converts the image data into image data based on the file format of the VR 180 and records the data. Here, it is assumed that a moving image is recorded in the moving image mode and a still image is recorded in the still image mode.

The image captured by the imaging unit 211 in the photographing process of S418 is an image like the image 801 shown in FIG. 8A. The image 801 includes a left eye image range 801L imaged on the imaging unit 211 via the left eye optical system and a right eye image range 801R imaged on the imaging unit 211 via the right eye optical system. The black-painted part is also a part taken as an image, but the dark part is taken without both the left eye optical system and the right eye optical system. There is parallax between the subject images in the left-eye image range 801L and the right-eye image range 801R.

The still images recorded on the recording medium 227 in the photographing process of S418 are the image 802L and the image 802R shown in FIG. 8B. The system control unit 50 uses the image processing unit 214 to cut out (trim) the left eye image range 801L from the captured images 801 and record the image 802L drawn in a cylindrical projection in a substantially square shape. do. Further, the system control unit 50 uses the image processing unit 214 to cut out (trim) the right eye image range 801R from the captured images 801 and draw the image 802R in a substantially square projection in a cylindrical projection. To record. That is, the left eye image range 801L and the right eye image range 801R are extracted (acquired) from the captured image 801. The image 802L and the image 802R are recorded in the same still image file in a multi-picture format. The file name of the VR image of the still image is, for example, "123456.vr.jpg" so that it can be distinguished from the normal image file not for VR180 described in S406, and three characters before the extension ".jpg". Character string ".vr" is described. When the image saved as a still image file is displayed in a two-lens VR display with a device capable of VR display, a stereoscopic still image mapped to a hemisphere is displayed.

Further, the moving image recorded on the recording medium 227 in the photographing process of S418 is the image 803 shown in FIG. 8C. The system control unit 50 uses the image processing unit 214 to shift the left-eye image range 801L and the right-eye image range 801R of the captured images 801 to the left and right in the image of one moving image (moving image). Generate and record side-by-side) side-by-side video files. The file name of the VR image of the moving image is, for example, "123456.vr.mp4" so that it can be distinguished from the normal image file not for VR180 explained in S406, and three characters before the extension ".mp4". The character string ".vr" is described. When the image saved as a moving image file is displayed in a two-lens VR display with a device capable of VR display, a stereoscopic moving image mapped to a hemisphere is displayed.

In this way, the image recorded by the photographing process is based on whether a normal lens (not a VR180 lens) is attached as an interchangeable lens or a VR180 lens is attached (that is, the determination result of S401). (Depending on) changes. That is, whether the processing of S406 is performed and the recording is performed or the processing of S418 is performed and the recording is performed depends on whether or not the VR180 lens is attached.
The image recorded in the camera 100 is not limited to the one described in FIGS. 8 (b) and 8 (c), and is in the state of the image 801 described in FIG. 8 (a) (that is, trimming or deformation is performed). The photographed image that has not been subjected to the image may be recorded as an image file on the recording medium 227.

In S419, the system control unit 50 displays a quick review of the image captured by the imaging unit 211 in response to the imaging instruction. Specifically, the system control unit 50 cylinders the image taken through the right eye optical system and the image taken through the left eye optical system, regardless of the display mode when the live view is displayed. Both are displayed on the display unit 108 by being deformed based on the projection. By the quick review display, for example, it is displayed in the same display mode as the second display mode as shown in FIG. 7 (b). By being displayed in this way, the user recognizes that the captured image includes both the image captured through the right eye optical system and the image captured through the left eye optical system. At the same time, it is possible to check the appearance of the subject. Even if the quick review display is not set by the menu etc., if the display mode when displaying the live view is the fourth display mode, the user may want to check the captured image. , You may forcibly display the quick review. After that, the system control unit 50 returns to S401 and again determines whether or not the lens unit is a VR180 lens.

As described above, according to the present embodiment, at least one of the live image taken through the right eye optical system and the live image taken through the left eye optical system is deformed based on the cylindrical projection. Let me display the live view. By deforming the image based on the cylindrical projection and displaying it in the live view in this way, even when the image is photographed with two eyes, the user can perform a suitable image because the image to be photographed can be confirmed. ..
Further, according to the present embodiment, one of the live image taken through the right eye optical system and the live image taken through the left eye optical system is deformed based on the cylindrical projection method. Live view is displayed, and the other live image is not displayed in live view. Therefore, the user can check the focus, the facial expression of the subject, and the like in detail based on the live view display in which one of the live images is deformed based on the cylindrical projection.

Further, according to the present embodiment, when the display mode is set to the fourth display mode, a twin-lens lens capable of taking a picture using the parallax of the left image and the right image is attached as the second interchangeable lens. If so, the live image is not displayed in the live view on the display unit 108. On the other hand, when at least a monocular lens is attached as the first interchangeable lens, the live image is displayed in the live view on the display unit 108. As described above, when the twin-lens lens is attached, the live image is not displayed in the live view on the display unit 108, so that the user can be urged to directly visually recognize the subject with the naked eye and take a picture. Therefore, even when the image is taken with two eyes, the user can take a suitable picture.

(Other embodiments)
The present invention is also realized by executing the following processing. That is, in a process in which a program that realizes the functions of the above-described embodiment is supplied to a system or device via a network or various recording media, and a computer (CPU, MPU, etc.) of the system or device reads and executes the program code. be. In this case, the program and the recording medium in which the program is stored constitute the present invention.

Although the present invention has been described above with various embodiments, the present invention is not limited to these embodiments and can be modified within the scope of the present invention.
In the above-described embodiment, in S419 of FIG. 4, when displaying a quick review, an image taken through the right eye optical system and an image taken through the left eye optical system are each based on a cylindrical projection. The case where both are transformed and displayed on the display unit 108 has been described. However, this is not the case. For example, the system control unit 50 is not limited to the case of displaying the quick review, but is the same as the case of displaying the quick review even when displaying the image in response to the instruction to display the captured image, that is, when displaying in the playback mode. It may be displayed in. Further, for example, when displaying a quick review, the system control unit 50 uses a cylindrical projection of one of an image taken through the right eye optical system and an image taken through the left eye optical system. It may be deformed based on the above, and only the deformed image may be displayed on the display unit 108.

Further, in the above-described embodiment, the case where the first display mode to the fourth display mode can be set as the display mode when displaying the live view has been described, but the present invention is not limited to this case. For example, the display mode may be switched according to the user selecting at least two modes from the second display mode to the fourth display mode.

Further, in the above-described embodiment, the camera 100 in which two optical images formed by the right-eye optical system 301R and the left-eye optical system 301L are formed on one image sensor has been described, but the present invention is not limited to this case. .. For example, two image pickup elements may be provided, and the optical images formed by the right eye optical system 301R and the left eye optical system 301L may be applied to a twin-lens camera formed on the two image pickup elements, respectively.

Further, in the above-described embodiment, the case where the live image taken through the twin-lens lens is deformed based on the cylindrical projection and displayed in the live view has been described, but the present invention is not limited to this case. For example, if the situation of the subject can be confirmed, the live image taken through the binocular lens may be deformed based on another method and displayed in the live view, or the live view may be displayed without deforming the live image. It may be displayed.

Further, in the above-described embodiment, a case where it is determined whether or not the lens unit is a VR180 lens by acquiring information on the type of the lens unit from the mounted lens unit has been described, but only in this case. I can't. For example, the system control unit 50 displays a plurality of lens unit types on the display unit 108 so as to be selectable, and the system control unit 50 selects and sets the type of the lens unit worn by the user. You may get the kind of information. Further, the system control unit 50 may acquire information on the type of the lens unit by analyzing the image captured by the image pickup unit 211. For example, when the system control unit 50 analyzes that the image captured by the image pickup unit 211 includes two circular images, it can determine that the VR180 lens is attached.

It should be noted that the above-mentioned various controls described as those performed by the system control unit 50 may be performed by one hardware, or a plurality of hardware (for example, a plurality of processors and circuits) share the processing to be a device. The whole control may be performed.

Further, in the above-described embodiment, the case where the present invention is applied to an imaging device (camera) as an electronic device has been described, but the present invention is not limited to this case, and any device capable of live view display can be applied. That is, the present invention can be applied to smartphones, tablet PCs, portable personal computers, PDAs, portable image viewers, game machines, and the like.

Further, in the above-described embodiment, the case where the present invention is applied to a camera (imaging device main body) has been described, but the present invention is not limited to this case, and communication with an imaging device (including a network camera) is performed via wired or wireless communication. It can also be applied to a control device that remotely controls an imaging device. The device for remotely controlling the image pickup device is, for example, a device such as a smartphone, a tablet PC, or a desktop PC. The image pickup device can be controlled remotely by notifying the image pickup device of commands for performing various operations and settings from the control device side based on the operations performed on the control device side and the processes performed on the control device side. be. Further, the live image taken by the imaging device may be received via wired or wireless communication and displayed on the control device side.

50: System control unit 100: Camera 108: Display unit 200: Lens unit 211: Imaging unit 214: Image processing unit 300: Lens unit

Claims (21)

An electronic device that displays captured images in live view.
A first live image taken through the first optical system and a second optical system in the same orientation as the first optical system are taken and parallax with respect to the first live image. An electronic device comprising a display control means for controlling at least one live image of a second live image to be deformed based on a cylindrical projection and displayed in a live view. The display control means
Of the first live image and the second live image, one of the live images is deformed based on the cylindrical projection to be displayed in the live view, and the other live image is controlled not to be displayed in the live view. The electronic device according to claim 1. The display control means
When one of the first live image and the second live image is deformed and displayed based on the cylindrical projection, the deformed image of the one live image is displayed on the display screen. The electronic device according to claim 2, wherein the live view is controlled to be displayed closer to the center of the device. The display control means
When one of the first live image and the second live image is deformed and displayed based on the cylindrical projection, the deformed image of the one live image is enlarged. The electronic device according to claim 2 or 3, wherein the live view is controlled to be displayed. The display control means
When displaying an image immediately after being shot in response to a shooting instruction during the live view display, or when displaying an image in response to an instruction to display a shot image, the first optical system is used. The electronic device according to any one of claims 1 to 4, wherein both the image taken by the user and the image taken through the second optical system are controlled to be displayed. The display control means
The feature is that at least two modes, a mode in which one of the first live image and the second live image is displayed, a mode in which both are displayed, and a mode in which neither is displayed are switched according to the user's selection. The electronic device according to any one of claims 1 to 5. The display control means
Of the first live image and the second live image, one of the live images selected by the user is deformed based on the cylindrical projection and displayed in the live view, and the other live image is not displayed in the live view. The electronic device according to any one of claims 1 to 6, wherein the electronic device is controlled to the above. The display control means
When the first interchangeable lens is attached, the live image is not displayed in live view, and when the second interchangeable lens is attached, the live image is deformed based on the cylindrical projection and displayed in live view. The electronic device according to any one of claims 1 to 7, wherein the electronic device is controlled so as to be used. The display control means
When the second interchangeable lens is attached, an image obtained by cutting out a predetermined region from the image taken through the second interchangeable lens is deformed based on the cylindrical projection and displayed in a live view. The electronic device according to claim 8, wherein the electronic device is controlled so as to be used. The second interchangeable lens is
The electronic device according to claim 8 or 9, further comprising the first optical system and the second optical system. The electronic device according to any one of claims 1 to 7, wherein the electronic device is a twin-lens camera including two image pickup elements. In response to a shooting instruction during the live view display, an image file including an image taken through the first optical system and an image taken through the second optical system is generated, and the image is generated. The electronic device according to any one of claims 1 to 11, further comprising a control means for controlling to record a file. Acquisition means for acquiring a first image taken through the first optical system and a second image taken through the second optical system and having a parallax with respect to the first image. The electronic device according to any one of claims 1 to 12, further comprising. An acquisition method for acquiring an image taken through an interchangeable lens,
When the first interchangeable lens is attached, the live image acquired by the acquisition means is displayed in a live view on the display means.
An electronic device comprising: a display control means for controlling a live image acquired by the acquisition means so as not to display a live view on the display means when a second interchangeable lens is attached. The electronic device according to any one of claims 8 to 10 and 14, wherein the second interchangeable lens is a twin-lens lens capable of photographing with parallax between a left image and a right image. The electronic device according to any one of claims 8 to 10, 14 and 15, wherein the first interchangeable lens includes at least a monocular lens. The display control means
8. to 10, wherein the type of the attached interchangeable lens is determined based on at least one of communication with the interchangeable lens, setting by the user, and analysis of the captured image. The electronic device according to any one of 14 to 16. It is a control method for electronic devices that display captured images in live view.
A first live image taken through the first optical system and a second live image taken through the second optical system in the same orientation as the first optical system, with parallax relative to the first live image. A method for controlling an electronic device, which comprises a display control step for controlling at least one of the second live images to be deformed and displayed in a live view based on a cylindrical projection. With the acquisition step to acquire the image taken through the interchangeable lens,
When the first interchangeable lens is attached, the live image acquired in the acquisition step is displayed in live view on the display means.
An electronic device comprising: a display control step for controlling the live image acquired by the acquisition step so as not to display the live view on the display means when the second interchangeable lens is attached. Control method. A program for causing a computer to function as each means of the electronic device according to any one of claims 1 to 17. A computer-readable recording medium containing a program for causing the computer to function as each means of the electronic device according to any one of claims 1 to 17.

Images