JP2023047536A - Electronic device - Google Patents

Abstract

To provide a technique capable of suppressing an unwanted object, such as a user's (photographer's) finger, from being captured in an image by a novel method.SOLUTION: An electronic device has a camera that captures images using a plurality of lenses. The electronic device has posture detection means and control means. The posture detection means detects a posture of the electronic device. The control means controls to automatically change a lens used for capturing an image to be displayed on a display unit from a first lens to a second lens that is different from the first lens in accordance with detection by the posture detection means of a change in posture of the electronic device from a first orientation to a second orientation in a state in which an image captured using the first lens is displayed on the display unit.SELECTED DRAWING: Figure 3

Description

The present invention relates to electronic equipment capable of imaging.

When photographing, the finger may be captured in the photographed image. In particular, in smartphones and the like, the lens is often close to the edge of the main body, and there is a high degree of freedom in the orientation of the main body during shooting, so there is a high possibility that a finger will be captured in the image.

Japanese Patent Application Laid-Open No. 2002-200003 proposes a technique of detecting an unnecessary object from an image wider than the range of the main image and determining whether or not the unnecessary object appears in the main image. With the technique disclosed in Patent Document 1, the user can change the position of the camera to change the range of the main image so that unnecessary objects are not captured in the main image. Japanese Patent Application Laid-Open No. 2002-200003 proposes a technique of photographing a medical treatment by using imaging guideline information (information on the imaging direction and zoom magnification) corresponding to the medical treatment so as not to include unnecessary objects.

JP 2019-96926 A JP 2006-277486 A

However, in the conventional technology, since the photographing range, photographing direction, and zoom magnification are changed, it is impossible to photograph the range intended by the user.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a technique capable of suppressing unwanted objects such as a user's (photographer's) finger from appearing in a new image.

An electronic device of the present invention is an electronic device having a camera that captures images using a plurality of lenses, and includes orientation detection means for detecting the orientation of the electronic device and an image captured using a first lens. In a state where the display is displayed on the display unit, an image to be displayed on the display unit is captured in response to detection by the orientation detection means of a change in orientation of the electronic device from the first orientation to the second orientation. and control means for controlling to automatically change the lens to be used from the first lens to a second lens different from the first lens.

According to the present invention, it is possible to suppress unwanted objects such as the user's (photographer's) finger from appearing in a new image.

1 is an external view of a smartphone; FIG. 1 is a block diagram showing the configuration of a smartphone; FIG. 4 is a flowchart showing camera processing according to the first embodiment; 4 is a flowchart showing first lens change processing; 9 is a flowchart showing second lens change processing; FIG. 4 is a diagram showing a state in which a user holds a smartphone sideways; It is a figure which shows the imaged image. 6 is a diagram for explaining a specific example of the processing of S503 in FIG. 5; FIG. It is a rear view of a smart phone (modification). 9 is a flowchart showing camera processing according to the second embodiment; 6 is a flowchart showing image quality adjustment processing; FIG. 12 is a diagram for explaining a specific example of the processing of S1103 of FIG. 11;

Preferred embodiments (first embodiment and second embodiment) of the present invention will be described below with reference to the drawings.

1(a) and 1(b) are external views of a smartphone 100 as an example of a device (electronic device) to which the present invention can be applied. 1A is a front view of the smartphone 100, and FIG. 1B is a rear view of the smartphone 100. FIG.

A display 105 is a display unit provided on the front surface of the smartphone 100 that displays images and various types of information. The touch panel 106a is a touch operation member integrated with the display 105, and the smartphone 100 can detect a touch operation on the display surface (operation surface) of the display 105 using the touch panel 106a. Smartphone 100 can perform live view display (LV display) on display 105 of an image captured by out-camera 114 or in-camera 115 . LV display is display of an image representing a subject substantially in real time, and an image displayed in LV is called an LV image. The out-camera 114 includes a telephoto lens 114a, a standard lens 114b, a super-wide-angle lens 114c, a wide-angle lens 114d, and the like, which will be described later.

The power button 106 b is an operation button for switching on/off of the display 105 and switching on/off of the smartphone 100 . For example, when the user presses the power button 106b for a period of time shorter than a predetermined time (for example, three seconds) (short presses the power button 106b), the display 105 is turned on/off. When the user continues to press the power button 106b for a predetermined time (long presses the power button 106b), the power of the smartphone 100 is switched between ON and OFF.

The volume plus button 106c and the volume minus button 106d are volume buttons for adjusting the volume of the audio signal output from the audio output terminal 112a and the volume of the audio output from the speaker 112b. When the volume plus button 106c is pressed, the volume increases, and when the volume minus button 106d is pressed, the volume decreases. In the shooting standby state when using the camera, the volume plus button 106c and the volume minus button 106d function as shutter buttons for instructing shooting.

In addition, the user can arbitrarily set so that a specific function is executed when the power button 106b and the volume down button 106d are pressed at the same time, or when the volume down button 106d is quickly pressed several times. can.

Home button 106 e is an operation button for displaying a home screen on display 105 . When the user activates and uses various applications on the smartphone 100, the user presses the home button 106e to temporarily close the various activated applications and display the home screen on the display 105. can be made In this embodiment, the home button 106e is a button that can be physically pressed.

The audio output terminal 112a is an earphone jack, and is a terminal for outputting an audio signal to an earphone, an external speaker, or the like. The speaker 112b is a built-in speaker that outputs sound. For example, when an earphone or an external speaker is connected to the audio output terminal 112a, an audio signal is output from the audio output terminal 112a. When an earphone or an external speaker is not connected to the audio output terminal 112a, audio is output from the speaker 112b.

FIG. 2 is a block diagram showing a configuration example of the smartphone 100. As shown in FIG. CPU 101 , memory 102 , nonvolatile memory 103 , out-camera image processing unit 104 , display 105 , operation unit 106 , storage medium I/F 107 , external I/F 109 , and communication I/F 110 are connected to internal bus 150 . ing. Also connected to the internal bus 150 are an audio output unit 112 , a posture detection unit 113 , an out-camera 114 , an in-camera 115 and an in-camera image processing unit 116 . Each unit connected to the internal bus 150 can exchange data with each other via the internal bus 150 .

The CPU 101 is a control unit that controls the smartphone 100 as a whole, and includes at least one processor or circuit. The memory 102 is, for example, a RAM (a volatile memory using a semiconductor element, etc.). The CPU 101 controls each part of the smartphone 100 according to a program stored in the nonvolatile memory 103, for example, using the memory 102 as a work memory. The nonvolatile memory 103 stores image data, audio data, other data, various programs for the CPU 101 to operate, and the like. The nonvolatile memory 103 is composed of, for example, flash memory or ROM.

The out-camera 114 is a camera arranged on the back surface of the smartphone 100 (the surface opposite to the surface on which the display 105 is provided). The out-camera 114 includes an imaging unit 114s, a telephoto lens 114a, a standard lens 114b, a super-wide-angle lens 114c, and a wide-angle lens 114d.

The imaging unit 114s is an imaging element such as a CCD image sensor or a CMOS image sensor, and performs imaging using the telephoto lens 114a, the standard lens 114b, the ultra-wide-angle lens 114c, and the wide-angle lens 114d individually. The imaging unit 114s simultaneously (in parallel or can be done by time division). Of course, the imaging unit 114s can simultaneously perform only two or three types of imaging among the four types of imaging described above, or can perform only one type of imaging.

The focal length of the telephoto lens 114a is longer than the focal length of the standard lens 114b, and the imaging using the telephoto lens 114a is more telescopic than the imaging using the standard lens 114b. The focal length of the super wide-angle lens 114c is shorter than the focal length of the standard lens 114b, and the imaging using the ultra-wide-angle lens 114c is wider than the imaging using the standard lens 114b. The focal length of the wide-angle lens 114d is approximately intermediate between the focal length of the standard lens 114b and the focal length of the ultra-wide-angle lens 114c. That is, the telephoto lens 114a, the standard lens 114b, the wide-angle lens 114d, and the ultra-wide-angle lens 114c have shorter focal lengths, and accordingly wider angles of view (imaging ranges). In this embodiment, the zoom magnification (focal length) of the telephoto lens 114a is assumed to be a predetermined fixed value, but may be changeable by the user.

In this embodiment, one imaging unit 114s is used for all the lenses of the out-camera 114 (telephoto lens 114a, standard lens 114b, super wide-angle lens 114c, and wide-angle lens 114d). may be used. The number of lenses corresponding to one imaging unit is not particularly limited, and one imaging unit may be used for one lens, one imaging unit may be used for two lenses, One imaging unit may be used for three lenses. Four imaging units may be used: an imaging unit corresponding to the telephoto lens 114a, an imaging unit corresponding to the standard lens 114b, an imaging unit corresponding to the super wide-angle lens 114c, and an imaging unit corresponding to the wide-angle lens 114d.

The in-camera 115 is a camera arranged on the front surface of the smartphone 100 (the surface on which the display 105 is provided).

Both the image captured by the out-camera 114 and the image captured by the in-camera 115 can be LV-displayed on the display 105 . The user can operate the touch panel 106 a to select an image captured by which camera to be displayed on the display 105 as an LV. For example, the user (photographer) selects the out-camera 114 when capturing a scene in front of the user, and selects the in-camera 115 when taking a selfie.

When displaying an image captured by the out-camera 114 on the LV display on the display 105, the user operates the touch panel 106a to select which lens is used to display the image captured on the display 105 on the LV. can do. If the user selects the telephoto lens 114a, the user can LV-display on the display 105 an image that is more enlarged than when the standard lens 114b is selected. If the standard lens 114b is selected, an image with a wider angle than when the telephoto lens 114a is selected, and an image magnified more than when the super-wide-angle lens 114c or the wide-angle lens 114d is selected, can be displayed on the display 105 as an LV. If the super-wide-angle lens 114c is selected, a wider-angle image can be LV-displayed on the display 105 than when the telephoto lens 114a, the standard lens 114b, or the wide-angle lens 114d is selected.

Under the control of the CPU 101 , the out-camera image processing unit 104 performs various image processing (including object detection processing) using an image captured by the out-camera 114 . The out-camera image processing unit 104 includes a telephoto image processing unit 104a, a standard image processing unit 104b, a super-wide-angle image processing unit 104c, and a wide-angle image processing unit 104d. The telephoto image processing unit 104a performs various image processing using the image captured by the telephoto lens 114a. The standard image processing unit 104b performs various image processing using the image captured by the standard lens 114b. The ultra-wide-angle image processing unit 104c performs various image processing using the image captured by the ultra-wide-angle lens 114c. The wide-angle image processing unit 104d performs various image processing using the image captured by the wide-angle lens 114d. These four image processing units can operate simultaneously (in parallel or in time division).

Although four image processing units corresponding to four lenses are used in this embodiment, the number of lenses corresponding to one image processing unit is not particularly limited. One image processing unit may be used for two lenses, or one image processing unit may be used for three lenses. One image processor may be used for all lenses. That is, the out-camera image processing unit 104 does not have to include a plurality of image processing units. Even in that case, the out-camera image processing unit 104 may be able to simultaneously (parallelly or time-divisionally) execute a plurality of processes using a plurality of images.

Under the control of the CPU 101 , the in-camera image processing unit 116 performs various image processing using the image captured by the in-camera 115 .

Each image processing unit performs various image processing using images stored in the non-volatile memory 103 and storage medium 108, images obtained via the external I/F 109, images obtained via the communication I/F 110, and the like. can also be done. Image processing performed by each image processing unit includes A/D conversion processing, D/A conversion processing, image data encoding processing, compression processing, decoding processing, enlargement/reduction processing (resize), noise reduction processing, color conversion, processing, etc. Each image processing unit may be composed of a dedicated circuit block for performing specific image processing. Further, depending on the type of image processing, the CPU 101 can perform image processing according to a program without using each image processing unit.

The display 105 displays images, GUI screens that form a GUI (Graphical User Interface), and the like, under the control of the CPU 101 . The CPU 101 controls each unit of the smartphone 100 to generate a display control signal according to a program, generate an image signal to be displayed on the display 105 , and output the image signal to the display 105 . A display 105 displays an image based on the output image signal. Note that the smartphone 100 itself includes an interface for outputting an image signal to be displayed on the display 105, and the display 105 may be configured by an external monitor (such as a television).

The operation unit 106 is an input device for accepting user operations, including a character information input device such as a keyboard, a pointing device such as a mouse and a touch panel, buttons, dials, joysticks, touch sensors, and touch pads. The operation unit 106 includes operation members such as the touch panel 106a, the power button 106b, the volume plus button 106c, the volume minus button 106d, and the home button 106e.

A storage medium I/F 107 allows a storage medium 108 such as a memory card, CD, or DVD to be detachable. I do. Storage medium 108 may be internal storage embedded within smartphone 100 . The external I/F 109 is an interface for connecting with an external device via a wired cable or wirelessly and inputting/outputting image signals and audio signals. The communication I/F 110 is an interface for communicating with an external device, the Internet 111, etc., and transmitting/receiving various data such as files and commands.

The audio output unit 112 outputs audio of video data and music data, operation sounds, ringtones, various notification sounds, and the like. The audio output unit 112 includes the audio output terminal 112a and the speaker 112b described above. The audio output unit 112 may output audio through wireless communication or the like.

The orientation detection unit 113 detects the orientation (inclination) of the smartphone 100 with respect to the direction of gravity and the orientation (inclination) of the smartphone 100 with respect to each of the yaw, roll, and pitch axes. Based on the posture detected by the posture detection unit 113, whether the smartphone 100 is held horizontally, held vertically, directed upward, downward, oblique, or the like is determined. It is identifiable. Any one of a plurality of sensors such as an acceleration sensor, a gyro sensor, a geomagnetic sensor, a direction sensor, and an altitude sensor can be used as the orientation detection unit 113 . A combination of two or more of a plurality of sensors can also be used as the orientation detection unit 113 .

As described above, the operation unit 106 includes the touch panel 106a. The touch panel 106a is an input device that is superimposed on the display 105 and configured in a plane, and that outputs coordinate information according to the touched position. The CPU 101 can detect the following operations or states on the touch panel 106a.
- A finger or pen that has not touched the touch panel 106a newly touches the touch panel 106a, that is, the start of touch (hereinafter referred to as Touch-Down)
A state in which a finger or pen touches the touch panel 106a (hereinafter referred to as Touch-On)
・The finger or pen is moving while touching the touch panel 106a (hereinafter referred to as Touch-Move)
- The finger or pen touching the touch panel 106a is separated (released) from the touch panel 106a, that is, the end of the touch (hereinafter referred to as Touch-Up)
A state in which nothing is touched on the touch panel 106a (hereinafter referred to as Touch-Off)

When touchdown is detected, touchon is also detected at the same time. After touchdown, touchon continues to be detected unless touchup is detected. Touch-on is detected at the same time when touch-move is detected. Even if touch-on is detected, touch-move is not detected if the touch position does not move. Touch-off is detected when it is detected that all the fingers and pens that were in touch have touched up.

The CPU 101 is notified through the internal bus 150 of these operations/states and the coordinates of the position on the touch panel 106a touched by the finger or pen. Based on the notified information, the CPU 101 determines what kind of operation (touch operation) has been performed on the touch panel 106a. As for the touch move, the movement direction of the finger or pen moving on the touch panel 106a can also be determined for each vertical component/horizontal component on the touch panel 106a based on the change in the position coordinates. If it is detected that the touch-move has been performed for a predetermined distance or more, it is determined that the slide operation has been performed. An operation of touching the touch panel 106a with a finger and quickly moving it by a certain distance and then releasing it is called a flick. A flick is, in other words, an operation of quickly tracing the touch panel 106a as if flicking it with a finger. It can be determined that a flick has been performed when a touch-move of a predetermined distance or more at a predetermined speed or more is detected, and a touch-up is detected as it is (it can be determined that a flick has occurred following a slide operation). Further, a touch operation of touching (multi-touching) a plurality of points (for example, two points) to bring the touch positions closer to each other is called pinch-in, and a touch operation of moving the touch positions away from each other is called pinch-out. Pinch-out and pinch-in are collectively called pinch operation (or simply pinch). The touch panel 106a may use any type of touch panel of various types such as a resistive film type, an electrostatic capacitance type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, an optical sensor type, and the like. good. There is a method of detecting that there is a touch when there is contact with the touch panel, and a method of detecting that there is a touch when there is an approach of a finger or pen to the touch panel.

(First embodiment)
A first embodiment of the present invention will be described. In the first embodiment, an example of changing the lens used for capturing the image to be displayed on the LV so as to suppress unnecessary objects from appearing in the image captured by the out-camera 114 and displayed on the LV on the display 105 will be described. do. Although the unnecessary object is not particularly limited, it is assumed to be the user's (photographer's) finger in the first embodiment. An image captured by the out-camera 114 and displayed on the LV on the display 105 can also be regarded as an image recorded on the storage medium 108 .

FIG. 3 is a flowchart illustrating an example of camera processing of smartphone 100 . This processing is realized by the CPU 101 expanding the program stored in the nonvolatile memory 103 into the memory 102 and executing the program. For example, when the user gives an instruction to start a camera application, the camera processing in FIG. 3 starts. It is assumed that orientation detection unit 113 detects the orientation of smartphone 100 during camera processing in FIG. 3 . In addition, the imaging unit 114s performs imaging using the telephoto lens 114a, imaging using the standard lens 114b, imaging using the super-wide-angle lens 114c, and imaging using the wide-angle lens 114d simultaneously (in parallel or in time division). ). Also, the out-camera image processing unit 104 (telephoto image processing unit 104a, standard image processing unit 104b, ultra-wide-angle image processing unit 104c, and wide-angle image processing unit 104d) performs object detection processing. Object detection processing is processing for detecting an object that satisfies a predetermined condition from a captured image. The predetermined condition is a condition that a finger (unnecessary object) reflected in the captured image satisfies, for example, a black object (black area) located at the edge of the captured image and larger than a predetermined size. is.

In S301, the CPU 101 sets the default lens to the LV display lens (lens used to capture an image to be displayed on the LV). Although the default lens is not particularly limited, it is, for example, the standard lens 114b.

In S302, the CPU 101 determines whether a finger (an object satisfying a predetermined condition) has been detected from the image captured using the LV display lens, based on the result of object detection processing by the out-camera image processing unit 104. judge. That is, it is determined whether or not the finger is reflected in the image captured using the LV display lens. If it is determined that a finger has been detected (the finger is captured), the process proceeds to S303; otherwise, the process proceeds to S304.

In S303, the CPU 101 performs first lens change processing for changing (switching) the LV display lens. Details of the first lens changing process will be described later with reference to FIG.

In S<b>304 , the CPU 101 determines whether or not the posture of the smartphone 100 has changed based on the processing result of the posture detection unit 113 . If it is determined that the orientation of the smartphone 100 has changed, the process proceeds to S305; otherwise, the process proceeds to S308. In the first embodiment, it is assumed that a change in orientation of smartphone 100 from vertical to horizontal or a change in orientation of smartphone 100 from horizontal to vertical is detected as a change in orientation of smartphone 100 .

In S<b>305 , the CPU 101 determines whether or not the orientation (orientation) of the smartphone 100 is sideways based on the processing result of the orientation detection unit 113 . That is, it is determined whether or not the posture change detected in S304 is a change in orientation of the smartphone 100 from vertical to horizontal. If it is determined that the smartphone 100 is in landscape orientation, the process proceeds to S306; otherwise, the process proceeds to S307.

In S306, the CPU 101 performs second lens change processing for changing (switching) the LV display lens. Details of the second lens changing process will be described later with reference to FIG.

In S307, the CPU 101 sets the standard lens 114b as the LV display lens. The out-camera 114 is close to the upper end of the smartphone 100 , and the user often holds the lower half of the smartphone 100 when shooting while holding the smartphone 100 vertically. Therefore, when the smartphone 100 is oriented vertically, the risk of the finger being captured in the image captured by the standard lens 114b is low. Note that when the smartphone 100 is oriented vertically, a lens different from the standard lens 114b (for example, a default lens) may be set as the LV display lens.

In S<b>308 , the CPU 101 causes the display 105 to perform LV display of the image captured using the LV display lens.

In S309, the CPU 101 determines whether or not the user has instructed to change the imaging magnification (zoom magnification) using the touch panel 106a. If it is determined that an instruction to change the imaging magnification has been given, the process proceeds to S310; otherwise, the process proceeds to S312.

In S310, the CPU 101 sets the lens with the photographing magnification instructed in S309 as the LV display lens. For example, in a state where the standard lens 114b is set as the LV display lens, if there is an instruction to change the photographing magnification in the enlargement direction, the LV display lens is changed from the standard lens 114b to the telephoto lens 114a. In the state that the standard lens 114b is set as the LV display lens, if there is an instruction to change the shooting magnification in the reduction direction, the LV display lens is changed from the standard lens 114b to the super-wide-angle lens 114c or the wide-angle lens 114d. do. Changing the imaging magnification in the enlarging direction corresponds to narrowing the imaging range (angle of view), and changing the imaging magnification in the contracting direction corresponds to widening the imaging range (angle of view).

Through the processing of S309 and S310, the user can arbitrarily change the imaging magnification even after the LV display lens (imaging magnification) is automatically changed in S303, S306, or S307. Note that the change of the imaging magnification is not limited to the change of the LV display lens as in the processing of S310, and may be digital zoom or optical zoom that does not involve the change of the LV display lens.

In S<b>311 , the CPU 101 causes the display 105 to perform LV display of an image captured using the LV display lens. Here, the LV display lens is the lens newly set in S310.

In S312, the CPU 101 determines whether or not a shooting instruction has been issued. If it is determined that a photographing instruction has been issued, the process proceeds to S313; otherwise, the process proceeds to S314. For example, the user performs a shooting instruction by performing a touch operation on a shooting icon displayed on the display 105 . A dedicated physical button for issuing a photographing instruction may be arranged, and it may be determined that a photographing instruction has been issued by pressing the physical button. As described above, the photographing instruction may be given by pressing the volume plus button 106c or the volume minus button 106d. It may be determined that the shooting instruction has been issued by simultaneously pressing the power button 106b and the volume down button 106d or by quickly pressing the volume down button 106d several times.

In S313, the CPU 101 performs AF (autofocus) processing based on the display position of the AF frame, AE (auto exposure) processing, AWB (auto white balance processing), and other shooting preparation processing. Then, the CPU 101 performs a series of photographing processes until an image captured using the LV display lens is recorded in the storage medium 108 as an image file.

In S314, the CPU 101 determines whether or not to end the camera application. If the camera application is to be terminated, the camera processing in FIG. 3 is terminated; otherwise, the process proceeds to S302. For example, when the user gives an instruction to close the camera application or an instruction to switch the active application to another application, the CPU 101 determines to end the camera application.

FIG. 4 is a flowchart showing an example of the first lens change processing (S303 in FIG. 3) performed when a finger is captured in an image captured using the LV display lens. This processing is realized by the CPU 101 expanding the program stored in the nonvolatile memory 103 into the memory 102 and executing the program.

In S401, the CPU 101 determines whether or not the imaging magnification has been changed by an instruction from the user (that is, whether or not the LV display lens is the lens set in S310 of FIG. 3). If it is determined that the shooting magnification has been changed according to an instruction from the user, the first lens change processing in FIG. 4 is terminated, and if not, the process proceeds to S402.

In S<b>402 , the CPU 101 performs setting so as to exclude lenses whose number of pixels in the captured image is equal to or less than a predetermined number from the objects of the pickup processing performed in S<b>404 . This makes it possible to exclude, for example, a lens such as a macro lens whose number of pixels in an image to be captured is significantly smaller than that of a standard lens from the targets of the pick-up process. Note that the number of pixels of an image to be captured depends on the configuration of the imaging unit, the range of use of the imaging unit, and the like. Therefore, even if the lenses have different imaging units, or if the lenses have the same (common) imaging units, the number of pixels in the captured image may differ between the lenses. In addition, the target of the pick-up process can also be regarded as a candidate for the LV display lens after the change.

In S<b>403 , the CPU 101 sets to exclude the special-purpose lens from the camera pick-up processing targets performed in S<b>404 . As a result, for example, a lens used for special shooting, such as a monochrome lens, can be excluded from the targets of the pick-up process. Note that lenses satisfying other specific conditions may be excluded from targets of the pickup process.

In S404, the CPU 101 picks up a lens in which the finger is not captured in the captured image from the plurality of lenses of the out-camera 114 (pickup processing). If there are multiple lenses in which the finger is not captured in the captured image, all of the multiple lenses are picked up. Of course, lenses excluded in S402 or S403 are not picked up. The lens in which the finger is not captured in the captured image can also be regarded as the lens used to capture the image in which the finger was not detected.

In S405, the CPU 101 determines whether or not one or more lenses have been picked up in S404, that is, whether or not there is a lens in which the finger is not captured in the captured image. If it is determined that one or more lenses have been picked up, the process proceeds to S407; otherwise, the process proceeds to S406. The process proceeds to S406 when the finger is reflected in the image captured by all the lenses.

In S406, the CPU 101 makes a predetermined notification. In the first embodiment, the CPU 101 displays a warning on the display 105 to inform the user that a finger is covering the camera. For example, the display 105 is notified by text such as "Please check the camera." Since it was determined as No in S405, it can be understood that it is difficult to shoot with a lens other than the one covered by the finger in this step. Therefore, by performing the notification as described above, the possibility that the user will recognize that the camera (lens) is covered with a finger is increased, and it is possible to reduce the possibility of taking a photograph in which the finger is captured. be able to. Note that the predetermined notification may be realized by a display on the display 105, or may be realized by an audio output, lighting of a lamp, or the like.

In S407, the CPU 101 determines whether or not a plurality of lenses have been picked up in S404, that is, whether or not there are a plurality of lenses in which the finger is not captured in the captured image. If it is determined that a plurality of lenses have been picked up, the process proceeds to S408; otherwise, the process proceeds to S409. The process proceeds to S409 when one lens is picked up.

In S408, the CPU 101 sets the lens with the largest number of pixels of the image to be captured among the plurality of lenses picked up in S404 as the LV display lens.

In S409, the CPU 101 sets the lens picked up in S404 as the LV display lens.

FIG. 5 is a flowchart showing an example of the second lens changing process (S306 in FIG. 3) that is performed when the orientation (orientation) of the smartphone 100 changes from vertical to horizontal. This processing is realized by the CPU 101 expanding the program stored in the nonvolatile memory 103 into the memory 102 and executing the program.

In S<b>501 , the CPU 101 performs setting so as to exclude lenses whose number of pixels in an image to be captured is equal to or less than a predetermined number from the targets of the pickup processing performed in S<b>503 . In S<b>502 , the CPU 101 sets so as to exclude the special-purpose lens from the targets of camera pickup processing performed in S<b>503 . Note that lenses satisfying other specific conditions may be excluded from targets of the pickup process.

In S503, the CPU 101 picks up the lens closest to the center of the smartphone 100 in the horizontal direction (the direction perpendicular to the direction of gravity; the direction parallel to the sideways orientation of the smartphone 100) from the plurality of lenses of the out-camera 114 (pickup processing ). Of course, lenses excluded in S501 and S502 are not picked up. The details of this process will be described later with reference to FIG.

In S504, the CPU 101 determines whether or not a plurality of lenses have been picked up in S503. If it is determined that a plurality of lenses have been picked up, the process proceeds to S505; otherwise, the process proceeds to S506. The process proceeds to S506 when one lens is picked up.

In S505, the CPU 101 sets the lens with the largest number of pixels of the image to be captured among the plurality of lenses picked up in S503 as the LV display lens.

In S506, the CPU 101 sets the lens picked up in S503 as the LV display lens.

FIGS. 6A and 6B are diagrams showing an example of a state in which the user holds the smartphone 100 sideways. FIG. 6A is a front view of a user holding smartphone 100, and FIG. 6B is a top view of such a user. The angle of view 600 in FIG. 6B is the angle of view of the standard lens 114b (the range captured when the standard lens 114b is used), and the angle of view 601 is the angle of view of the super-wide-angle lens 114c (the range of the super-wide-angle lens 114c). range). In the example of FIG. 6B, the user's finger overlaps the angle of view 600 of the standard lens 114b.

FIGS. 7(a) and 7(b) are diagrams showing examples of images captured in the states of FIGS. 6(a) and 6(b). FIG. 7(a) shows an image captured using the standard lens 114b, and FIG. 7(b) shows an image captured using the super-wide-angle lens 114c. The user's finger 700 is reflected in the image of FIG. 7(a). On the other hand, the user's finger is not reflected in the image of FIG. 7(b). In the horizontal direction, the super-wide-angle lens 114c is closer to the center of the smartphone 100 than the standard lens 114b. The closer the lens is to the edge of the smartphone 100, the higher the risk of the finger being captured in the captured image. Therefore, using the lens closer to the center of the smartphone 100 can reduce the risk of being captured.

A specific example of the processing of S503 in FIG. 5 will be described with reference to FIG. FIG. 8 is a rear view showing smartphone 100 held sideways, and is a diagram showing an example of the distance from each lens of out-camera 114 to horizontal center position 800 of smartphone 100 . A horizontal center position 800 is the center position of the smartphone 100 in the horizontal direction (the direction parallel to the sideways orientation of the smartphone 100). A distance 801 from the ultra-wide-angle lens 114c and the wide-angle lens 114d to the horizontal center position 800 is shorter than a distance 802 from the telephoto lens 114a to the horizontal center position 800. FIG. A distance 802 from the telephoto lens 114 a to the horizontal center position 800 is shorter than a distance 803 from the standard lens 114 b to the horizontal center position 800 . That is, the super-wide-angle lens 114c and the wide-angle lens 114d are closest to the horizontal center position 800 among the telephoto lens 114a, the standard lens 114b, the super-wide-angle lens 114c, and the wide-angle lens 114d. Therefore, in the processing of S503 in FIG. 5, the super-wide-angle lens 114c and the wide-angle lens 114d are picked up.

Although an example of picking up the lens closest to the center of the smartphone 100 in the horizontal direction has been described, the present invention is not limited to this. Even if the lens is not closest to the center of the smartphone 100 in the horizontal direction, it is sufficient to pick up the lens closest to the center, and it is sufficient to pick up the lens present within a predetermined range from the center of the smartphone 100 in the horizontal direction.

9A and 9B are rear views of smartphones 910 and 920 (modifications) different from the smartphone 100. FIG. In the smartphone 910 of FIG. 9(a), the out-camera 911 includes eleven lenses 911a to 911k. The lenses 911a to 911k are closer to the horizontal end when the smartphone 910 is held sideways, and the lenses 911b to 911k are arranged in a circle around the lens 911a. For smartphone 910, lenses 911h-911k may be picked up. The lenses 911h to 911k are closer to the center of the smartphone 910 than the lenses 911a to 911g. In the smartphone 920 of FIG. 9B, the out-camera 921 includes four lenses 921a-921d. The lenses 921a to 921d are close to the ends in the horizontal direction when the smartphone 920 is held sideways, and are arranged in the horizontal direction. In the case of the smart phone 920, lenses 921c and 921d may be picked up. The lenses 921c and 921d are closer to the center of the smartphone 920 than the lenses 921a and 921b.

As described above, according to the first embodiment, the lens used for capturing an image to be displayed on the display unit changes from the first orientation to the second orientation according to the orientation change of the electronic device. lens is automatically changed to a second lens different from the first lens. Furthermore, in response to detection of an unnecessary object from an image captured using the first lens, the lens used for capturing the image to be displayed on the display unit is changed from the first lens to the first lens. is automatically changed to a different third lens. Such a novel method can suppress unwanted objects such as the user's (photographer's) finger from appearing in the image. In addition, in the case where the first lens and the second lens are the same type of lens provided on the same side (same surface) of the electronic device, the shooting range, shooting direction, and zoom magnification are not changed. The user can confirm the intended range and shoot.

It should be noted that only one of the change of the lens according to the posture change and the change of the lens according to the detection of the unnecessary object may be performed. For example, the processes of S302 and S303 of FIG. 3 may be omitted and the processes of S304 to S307 may be performed, or the processes of S304 to S307 may be omitted and the processes of S302 and S303 may be performed.

Also, an example in which the vertical orientation of the smartphone is the first orientation and the horizontal orientation is the second orientation has been described, but the present invention is not limited to this. Depending on the arrangement of the lens, the size of the electronic device, etc., the horizontal orientation of the electronic device may be the first orientation, and the vertical orientation may be the second orientation. For example, in the case of a tablet terminal that is larger than a smartphone, the horizontal orientation of the tablet terminal may be the first orientation, and the vertical orientation may be the second orientation.

(Second embodiment)
A second embodiment of the present invention will be described. In the second embodiment, an example will be described in which not only the lens used for capturing the image to be displayed on the LV is automatically changed, but also the angle of view of the image to be displayed on the LV is adjusted. The angle of view adjustment in the second embodiment is adjustment of the display range of the image, but the angle of view adjustment may adjust not only the display range but also the shooting range (recording range). Since processing other than angle-of-view adjustment is the same as in the first embodiment, only the angle-of-view adjustment will be described in detail below.

FIG. 10 is a flowchart illustrating an example of camera processing of smartphone 100 . This processing is realized by the CPU 101 expanding the program stored in the nonvolatile memory 103 into the memory 102 and executing the program. The processes of S1001 to S1007 and S1009 to S1014 are the same as the processes of S301 to S307 and S309 to S314 in FIG. 3, respectively. In the camera processing of FIG. 10, the processing of S1008 is performed instead of the processing of S308 of FIG. In S1008, the CPU 101 performs angle-of-view adjustment processing. In the field angle adjustment process, when the LV display lens is automatically changed in S1003, S1006, or S1007, the field angle (display range) of the image to be displayed on the LV is adjusted. Details of the angle-of-view adjustment processing will be described later with reference to FIG. 11 .

FIG. 11 is a flowchart showing an example of the angle-of-view adjustment process (S1008 in FIG. 10). This processing is realized by the CPU 101 expanding the program stored in the nonvolatile memory 103 into the memory 102 and executing the program.

In S1101, the CPU 101 determines whether or not the LV display lens was automatically changed in S1003, S1006, or S1007. If it is determined that the LV display lens has been automatically changed, the process advances to S1102; otherwise, the process advances to S1104.

In S1102, the CPU 101 determines whether the LV display lens after change has a wider angle than the LV display lens before change. If the wide-angle lens is determined, the process proceeds to S1103; otherwise, the process proceeds to S1104.

In S1103, the CPU 101 compares the angle of view (imaging range; object range) of the image captured using the changed LV display lens with the angle of view of the image captured using the changed LV display lens. Adjust to approximately the same. The angle of view may be adjusted by digital zoom or by optical zoom. The details of this process will be described later with reference to FIG.

In S<b>1104 , the CPU 101 causes the display 105 to perform LV display of the image captured using the LV display lens. If the processing of S1103 has been performed, the image of the angle of view after adjustment is displayed on the display 105 as an LV.

A specific example of the processing (angle of view adjustment) in S1103 of FIG. 11 will be described with reference to FIGS.

FIG. 12A shows the user holding the smartphone 100 vertically. Basically, the user uses the smartphone 100 while holding it vertically. FIG. 12(b) shows an image 1210 captured using the standard lens 114b in the situation of FIG. 12(a). In the situation of FIG. 12A, it is assumed that the standard lens 114b is set as the LV display lens.

FIG. 12(c) shows a state in which the user holds the smartphone 100 sideways. For example, a user holding the smartphone 100 vertically as shown in FIG. 12A turns the smartphone 100 horizontally for shooting. Accordingly, the LV display lens is changed from the standard lens 114b to the super-wide-angle lens 114c. FIG. 12(d) shows an image 1220 captured using the ultra-wide-angle lens 114c in the situation of FIG. 12(c).

Since the super-wide-angle lens 114c has a wider angle than the standard lens 114b, the angle of view (imaging range; object range) of the image 1210 in FIG. 12(d) is wider than the angle of view of the image 1220 in FIG. 12(b). Therefore, when the orientation (orientation) of the smartphone 100 is changed from portrait orientation to landscape orientation, if the image to be LV-displayed on the display 105 is changed from the image 1210 to the image 1220, the angle of view to be LV-displayed changes, and the user can change the LV-display angle. There is a possibility that the display may give a sense of incongruity. Therefore, in S1103 of FIG. 11, an image area 1221 corresponding to an imaging range that is substantially the same as the imaging range when using the standard lens 114b is cut out from the image 1220, and in S1104, the image area 1221 is displayed on the display 105 as an LV. . By doing this, when the LV display lens is changed, the imaging range displayed on the display 105 does not change significantly, and the user can continue to use the smartphone 100 without feeling uncomfortable with the LV display.

As described above, according to the second embodiment, not only the lens used for capturing the image to be displayed on the LV is automatically changed, but also the angle of view of the image to be displayed on the LV is adjusted, thereby making the user feel uncomfortable. You can do it without waking up.

Note that the various controls described above that are performed by the CPU 101 may be performed by one piece of hardware, or a plurality of pieces of hardware (for example, a plurality of processors or circuits) may share the processing to control the entire apparatus. may be performed.

Moreover, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention can be applied to the present invention. included. Furthermore, each embodiment described above merely shows one embodiment of the present invention, and it is also possible to combine each embodiment as appropriate.

Further, in the above-described embodiments, the case where the present invention is applied to a smartphone has been described as an example, but this is not limited to this example, and an electronic device capable of capturing images using a plurality of lenses individually. is applicable. For example, the present invention can be applied to personal computers, PDAs, mobile phone terminals, portable image viewers, printers, digital photo frames, music players, game machines, electronic book readers, video players, and the like. The present invention can also be applied to display devices (including projection devices), tablet terminals, digital cameras, AI speakers, home appliances, in-vehicle devices, medical devices, and the like.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

100: Smartphone 101: CPU
104: Out-camera image processing unit 104a: Telephoto image processing unit 104b: Standard image processing unit 104c: Super-wide-angle image processing unit 104d: Wide-angle image processing unit 105: Display 113: Posture detection unit 114: Out-camera 114a: Telephoto lens 114b: Standard lens 114c: Super wide-angle lens 114d: Wide-angle lens 114s: Imaging unit 910: Smartphone 911: Out camera 911a to 911k: Lens 920: Smartphone 921: Out camera 921a to 921d: Lens

Claims (16)

An electronic device having a camera that captures images using a plurality of lenses,
attitude detection means for detecting the attitude of the electronic device;
In response to the posture detection means detecting a change in posture of the electronic device from a first direction to a second direction while an image captured using the first lens is displayed on the display unit. and control means for automatically changing a lens used for capturing an image to be displayed on the display unit from the first lens to a second lens different from the first lens. An electronic device characterized by: 2. The electronic device according to claim 1, wherein the second lens is closer to the center of the electronic device than the first lens in a direction parallel to the second orientation. The second lens is a lens having the largest number of pixels of an image to be captured among a plurality of lenses existing within a predetermined range from the center of the electronic device in a direction parallel to the second direction. 3. The electronic device according to claim 2. 4. The electronic device according to claim 1, wherein the first orientation is vertical and the second orientation is horizontal. An electronic device having a camera that captures images using a plurality of lenses,
Object detection means for performing an object detection process for detecting an object that satisfies a specific condition from a captured image;
In response to the object detecting means detecting the object from the image captured using the first lens while the image captured using the first lens is being displayed on the display unit, and control means for automatically changing a lens used for capturing an image to be displayed on the display unit from the first lens to a third lens different from the first lens. and electronic devices. further comprising object detection means for performing object detection processing for detecting an object that satisfies a specific condition from the captured image;
The control means further controls the object detection means to detect the object from the image captured using the first lens while the image captured using the first lens is displayed on the display unit. is detected, the lens used for the image displayed on the display unit is automatically changed from the first lens to a third lens different from the first lens. The electronic device according to any one of claims 1 to 4. The object detection means performs the object detection process on each of a plurality of images captured using the plurality of lenses,
7. The electronic device according to claim 5, wherein the third lens is a lens used to capture an image in which the object is not detected. 3. The third lens is the lens with the largest number of pixels of the image to be captured when there are a plurality of lenses used to capture the image in which the object is not detected. 8. The electronic device according to 7. 9. The control unit according to claim 7, wherein the control unit further performs control so that a predetermined notification is made in response to detection of the object from all of the plurality of images by the object detection unit. electronics. The electronic device according to any one of claims 5 to 9, wherein the object is a user's finger. The control means, when performing control to automatically change the lens used for capturing an image to be displayed on the display unit from the first lens, does not perform control to change the lens to a lens that satisfies a specific condition. The electronic device according to any one of claims 1 to 10, characterized by: When the control means controls to automatically change a lens used for capturing an image to be displayed on the display unit from the first lens to a lens having a wider angle than the first lens, the lens after the change wherein the range displayed on the display unit of the image captured using is controlled to be substantially the same as the range of the image captured using the lens before the change. 12. The electronic device according to any one of 11. A control method for an electronic device having a camera that captures images using a plurality of lenses,
an orientation detection step of detecting an orientation of the electronic device;
In response to detecting a change in posture of the electronic device from a first direction to a second direction in the posture detection step while an image captured using the first lens is displayed on the display unit. and a control step of automatically changing a lens used for capturing an image to be displayed on the display unit from the first lens to a second lens different from the first lens. A control method characterized by: A control method for an electronic device having a camera that captures images using a plurality of lenses,
an object detection step of performing an object detection process for detecting an object that satisfies a specific condition from the captured image;
In response to detecting the object from the image captured using the first lens in the object detection step while the image captured using the first lens is displayed on the display unit, and a control step of automatically changing a lens used for capturing an image to be displayed on the display unit from the first lens to a third lens different from the first lens. control method. A program for causing a computer to function as each means of the electronic device according to any one of claims 1 to 12. A computer-readable storage medium storing a program for causing a computer to function as each means of the electronic device according to any one of claims 1 to 12.

Images