JP7030458B2 - Imaging control device, its control method, and program, and storage medium - Google Patents

Description

The present invention relates to an image pickup control device, a control method thereof, a program, and a storage medium, and more particularly to control during so-called self-shooting image pickup.

In recent years, so-called self-portrait imaging (hereinafter, simply referred to as self-portrait) has been performed in an image pickup control device provided in a smartphone and an image pickup control device such as a digital camera. For this reason, there is an image pickup control device provided with a so-called vari-angle display unit so that the display unit can be easily seen when taking a selfie. Further, there is an image pickup control device provided with an image pickup unit (in-camera) on the display unit side.

In order for the user to obtain a suitable photograph when taking a self-portrait, the user often selects one that has the best facial expression or background from a plurality of photographs obtained by continuous imaging. For this reason, the user may retake the picture many times. On the other hand, in a so-called photo sticker machine, it is possible to make corrections according to the user's preference, such as enlarging the eyes by image processing and further thinning the contour of the face.

Further, there is an imaging control device that generates a so-called interesting image by synthesizing preset eye / nose / mouth parts with an image obtained by self-shooting (Patent Document 1).

Japanese Unexamined Patent Publication No. 2009-140047

By the way, when taking a selfie, a peculiar technique such as taking an image in which the composition and the pose are determined is required. In addition, there are times when you take images many times and select the one that captures the best, but in order to obtain an image in which all the facial parts are captured in the best condition, a huge number of images must be taken. There is also.

Furthermore, when face parts are imaged separately and these face parts are combined to generate a self-portrait image, the imaging conditions for each face part are different, so the altitude such as color tone, exposure, focal length, and alignment is high. It is necessary to perform various image processing.

Therefore, an object of the present invention is to provide an image pickup control device capable of generating a user-desired self-portrait image by a simple operation at the time of self-portrait, a control method thereof, a program, and a storage medium. ..

In order to achieve the above object, the image pickup control device according to the present invention is the first of the image pickup control means for controlling the imager to perform continuous image pickup and the plurality of images obtained by the continuous image pickup. A selection means for selecting a partial area in an image according to an operation from a user, and a corresponding portion of a second image of the plurality of images different from the first image in which the partial area selected by the selection means is selected. It has an image processing means that performs a process of replacing by an area and a display control means that controls to display an image that has been replaced by the image processing means, and a state in which the amount of change in the live view image is a predetermined amount or less. Is continued for a predetermined time, and the image pickup control means starts the continuous image pickup when the image shake is equal to or more than a predetermined value .

According to the present invention, it is possible to generate a self-portrait image desired by the user by a simple operation at the time of self-portrait.

It is a figure which shows the appearance of an example of the image pickup control apparatus by 1st Embodiment of this invention. It is a block diagram which shows an example of the structure of the image pickup control apparatus shown in FIG. It is a figure which shows an example of the menu screen displayed on the liquid crystal display in the image pickup control apparatus shown in FIG. It is a figure for demonstrating the self-shooting performed by the image pickup control apparatus shown in FIG. It is a figure which shows an example of the image at the time of self-shooting continuous shooting or the automatic self-shooting in the image pickup control device shown in FIG. 1. It is a figure for demonstrating the face part replacement function screen displayed in the image pickup control apparatus shown in FIG. It is a flowchart for demonstrating the process at the time of the self-portrait image pickup performed by the image pickup control apparatus shown in FIG. It is a flowchart for demonstrating the process by the face part replacement mode shown in FIG. 7. It is a flowchart for demonstrating the process at the time of the automatic self-portrait image pickup performed by the image pickup control apparatus by the 2nd Embodiment of this invention. It is a figure for demonstrating the self-portrait image pickup of a plurality of people performed in the image pickup control apparatus by the 3rd Embodiment of this invention.

Hereinafter, an example of the image pickup control device according to the embodiment of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a diagram showing the appearance of an example of the image pickup control device according to the first embodiment of the present invention. 1 (a) is a front view, and FIG. 1 (b) is a rear view. Further, FIG. 1 (c) is a perspective view seen from the front side.

The illustrated image pickup control device is, for example, a smartphone (mobile phone terminal) with a camera. The image pickup control device has a vertically long main body 101, and the main body 101 includes a front face 101A, a back face 101B, and a side face 101C. The front face 101A is the front surface of the main body 101, and the back face 101B is the back surface of the main body 101. Further, the side face 101C is a side surface of the main body 101.

A touch screen display 102, a button 103D, an in-camera 104, an illuminance sensor 105, and a speaker 106 are arranged on the front face 101A. An out-camera 108 and a light 109 are arranged on the back face 101B. Further, the buttons 103A to 103C and the connector 107 are arranged on the side face 101C.

The button 103A functions as a sleep / wake button of the image pickup control device 1 by pressing the button 103A briefly. Further, the power can be turned on / off by pressing and holding the button 103A. The button 103B is a shutter button and is used in the case of image pickup when the image pickup control device is laterally supported. The shutter button is also displayed on the touch screen display 102, which will be described later, and this shutter button is used, for example, for image pickup when an image pickup control device including self-shooting is vertically supported.

The button 103C is used to switch the output of sounds such as the shutter sound and the warning sound output from the speaker 106 on (ringing) / off (silence). The button 103D is a home button and is used when displaying the home screen on the touch screen display 102. The button 103D can be used in place of the button 103A to wake the image pickup control device. A predetermined function can be assigned to the button 103D by user setting. In the following description, the buttons 103A to 103D may be collectively referred to as the button 103 without specifying which button it is.

The touch screen display 102 includes a liquid crystal display 102A and a touch screen 1022B. The touch screen 102B detects contact with the touch screen 102B by a finger, a stylus pen, or the like. Further, the touch screen 102B can detect contact by a plurality of fingers. As the detection method in the touch screen 2B, for example, a capacitance type or a resistance coating method is used.

In the image pickup control device, as will be described later, the type of gesture operation is determined based on the contact, the contact position, and the contact time detected by the touch screen 1022B. The gesture operation is an operation performed on the touch screen 2 by a finger or a stylus pen.

Gesture operations determined by the image pickup control device include general operations such as touch, release, tap, drag, swipe, flick, pinch-in, and pinch-out performed on a smartphone or the like.

Touch is a gesture operation in which a finger touches the touch screen 102B. Release is a gesture operation in which the finger is released from the touch screen 102B. Tap is a gesture operation that quickly touches and releases the touch screen 102B.

The drag is a gesture operation in which the object displayed on the liquid crystal display 102A is long-pressed and the finger is dragged in the touched state. Swipe is a gesture operation in which a finger is moved so as to be in contact with the touch screen 102B. Flick is a gesture operation that releases by flicking with a finger following a touch on the touch screen 102B. Pinch-in is a gesture operation of pinching (narrowing) with a plurality of fingers on the touch screen 102B. Pinch-out is a gesture operation that spreads with a plurality of fingers on the touch screen 102B.

In the image pickup control device, an operation corresponding to the above gesture operation performed on the touch screen 1022B is performed.

FIG. 2 is a block diagram showing an example of the configuration of the image pickup control device shown in FIG. 1. In FIG. 2, the same reference numbers are assigned to the same components as those shown in FIG. 1.

As described above, the image pickup control device includes the touch screen display 102, and characters, images, icons, and the like are displayed on the liquid crystal display 2A. The touch screen 102B detects the gesture operation. The in-camera 104 includes a lens and an image pickup device such as a CCD or CMOS that converts an optical image into an electric signal. The in-camera 104 is a small camera module having AF (autofocus), aperture, shutter speed adjustment function, and the like. Then, the in-camera 104 captures an object facing the front face 1101A.

The illuminance sensor 105 detects the illuminance in the image pickup control device. Then, the controller 14 sets the imaging conditions of the in-camera 4 based on the detected illuminance. Further, the controller 14 adjusts the brightness of the liquid crystal display 102A according to the illuminance detected by the illuminance sensor 5.

When the sound output is turned on by the switch 103C, the speaker 106 outputs a shutter sound and a warning sound at the time of imaging. Connector 107 The connector is used to connect the image pickup control device to the external device. For example, an AC adapter for charging the battery provided in the power supply module 11 described later is connected to the connector 107. Further, the connector 107 is used for input / output of image data, audio data, and the like to the non-volatile memory (hereinafter referred to as storage) 15 described later.

The connector 7 may be a terminal specially designed such as a Dock connector, or may be a general-purpose terminal such as a USB (Universal Serial Bus).

The out-camera 108 is a small camera module similar to the in-camera 104. The out-camera 108 takes an image of an object facing the back face 101B. The light 109 is a light emitting module and functions as a flash when imaging with the out-camera 108.

The communication module 10 communicates according to a predetermined wireless communication standard. For example, as a wireless communication standard, there are so-called Wi-Fi, Bluetooth (registered trademark), NFC, etc. of the IEEE802.11 standard, and the communication module 10 may support at least one. The communication module 10 is used for input / output of image data obtained by imaging and for downloading a function addition program module for an imaging control device.

The power supply module 11 includes a rechargeable battery and supplies power to the entire image pickup control device. As the battery provided in the power supply module 11, for example, a lithium ion battery or a nickel hydrogen battery is used. Then, the battery is charged by the AC adapter or USB via the connector 7.

The acceleration sensor 12 detects the direction and magnitude of the acceleration acting on the image pickup control device. The accelerometer 12 can detect in three axes in the XYZ directions. The gyro sensor 13 detects the angle and the angular velocity of the image pickup control device. The detection result by the acceleration sensor 12 and the gyro sensor 13 is used in the auto self-shooting function described later.

The controller 14 is a processor such as a one-chip system SoC (System-on-a-chip) designed for a general-purpose CPU or a smartphone. The controller 14 controls the entire image pickup control device. The controller 14 expands the control program stored in the storage 15 into the system memory 16 and executes it, and performs a process described later. The system memory 16 is also used as a work area of the controller 14.

The control program 15A, various data, and the like are stored in the storage 15. The storage 15 is a semiconductor memory, for example, an electrically erasable non-volatile memory (for example, a flash ROM). Further, the system memory 16 is, for example, a RAM.

As shown in the figure, the storage 15 stores a control program 15A, a photo editing module 15B, setting data 15C, image data 15D, an additional function module 15E, and the like. The control program 15A and the like stored in the storage 15 may be input via wireless communication by the communication module 10 or via the connector 7.

The control program 15A provides a function for controlling the entire image pickup control device. That is, by executing the control program 15A, the controller 14 controls each block shown in FIG. For example, by executing the control program 15A, the controller 14 controls the touch screen 102, the button 103, the in-camera 104, the illuminance sensor 105, the in-camera 108, the light 109, and the like to provide the camera function.

Further, by executing the control program 15A, the controller 14 executes a process based on the gesture operation detected by the touch screen 102B, and displays information according to the execution result on the display 102A. The control program 15A may be used in combination with a function provided by another program stored in the storage 15.

By executing the photo editing module 15B, the controller 14 edits and manages the image data obtained by imaging. Further, by executing the photo editing module 15B, the controller 14 changes the face parts described later.

The setting data 15C includes data related to settings related to various operations in the image pickup control device and settings of the additional function module. The image data 15D includes image pickup data by the in-camera 4 and the in-camera 5. The additional function module 15E is a module for adding a function to the image pickup control device. For example, the additional function module 15E includes a browser function for displaying a WEB page and the like.

FIG. 3 is a diagram showing an example of a menu screen displayed on a liquid crystal display in the image pickup control device shown in FIG.

The menu screen displayed on the liquid crystal display 102A has a menu area 301 and a status display area 302. The menu area 301 is an area for the user to select which function of the image pickup control device is to be executed. A plurality of icons 306 are displayed in the menu area 301, and each of the icons 306 is associated with a function of the image pickup control device. Then, when the controller 14 detects the tap operation on the icon 306, the controller 14 executes the function associated with the icon. In the illustrated example, as the icon 306, a calendar, a clock, a browser, an image pickup mode, a setting, a moving image image pickup, a photograph, and a camera are displayed in the menu area 301.

The status display area 302 is located on the upper side of the menu screen. In the illustrated example, the status display area 302 includes a level mark 303 indicating the strength of WiFi, a clock display 304 indicating the current time, and a battery indicating the remaining battery level. The mark 305 is displayed. The status display area 302 is also used when displaying various notification messages.

FIG. 4 is a diagram for explaining self-shooting performed by the image pickup control device shown in FIG. 4 (a) is a diagram showing a camera function screen, and FIG. 4 (b) is a diagram showing a dialog displayed during continuous self-shooting. Further, FIG. 4C is a diagram showing a self-shooting mark and a continuous shooting mark displayed at the time of auto self-shooting.

When the camera icon is tapped on the menu screen shown in FIG. 3, the controller 14 activates the camera function. Then, the controller 14 displays the camera function screen on the liquid crystal display 102A (see FIG. 4A). The camera function screen has a status display area 302 similar to the menu screen, and on the camera function screen, various statuses and notifications at the time of imaging are displayed in the status display area 302.

In the illustrated example, the flash mark 401 and the switching mark 402 are displayed in the status display area 302. The user can switch the flash on and off by tapping the flash mark 401. When the flash is turned on, the light 109 functions as a flash during imaging by the out-camera 108.

Further, the user alternately switches the camera used for imaging between the in-camera 104 and the out-camera 108 by tapping the switching mark 402.

The camera function screen has a live view display area 403, and the controller 14 displays the live view obtained by the camera currently used for imaging in the live view display area 403. Further, the camera function screen has an operation area 404.

In the illustrated example, the photo button 405, the shutter button 406, and the setting button 407 are displayed in the operation area 404. The photo button 405 has the same function as the photo icon 306 displayed on the menu screen. When the photo icon 405 is touch-operated, the controller 14 activates the photo editing module 15B.

When the shutter button 406 is tapped, the controller 14 captures a still image obtained by the in-camera 4 or the out-camera 108 and writes it in the storage 15 as image data 15D. The user can make various settings for the camera function by using the setting button 407.

When "selfie continuous shooting" described later is set, the controller 14 displays the dialog 408 in the live view display area 403 as shown in FIG. 4 (b). The dialog 408 is a dialog for asking the user whether or not to activate the face part change function, which is one of the functions of the photo editing module 15B, when the "selfie continuous shooting" is completed.

When the "auto self-shooting function" described later is set, the controller 14 displays the dialog 409 shown in FIG. 4 (c) in the live view display area 403 before the image pickup by the "auto self-shooting function". Further, the controller 14 displays the self-shooting mark 410 and the continuous shooting mark 411 in the status display area 302 to notify that the "auto self-shooting function" is activated. The dialog 409 shown in FIG. 4C may be displayed during continuous shooting.

The dialog 409 is a guide that urges the user to make a so-called "decisive face" at the time of continuous shooting and the number of seconds remaining until the continuous shooting imaging is automatically started in the "auto self-shooting function". The "decisive face" is a facial expression that is arranged to improve the image. Further, in the dialog 409, in order to obtain a self-portrait photograph with variation (diversity) (to obtain a variety of parts), it is urged to take various facial expressions during continuous shooting. In the illustrated example, the wording "various" encourages this.

It should be noted that other sentences may be used as long as the guide has such a meaning. Here, "selfie continuous shooting" means that continuous shooting of self-shooting by the in-camera 4 is performed by a user operation (holding down the image pickup button). Further, "auto self-shooting" means that the controller 14 determines whether or not the self-shooting continuous shooting by the in-camera 4 is "self-shooting" to take a self-shooting.

FIG. 5 is a diagram showing an example of an image taken by self-shooting continuous shooting or auto self-shooting in the image pickup control device shown in FIG. 1.

In the illustrated example, it is assumed that five self-portrait images are continuously shot in the self-portrait, and the captured images are arranged in order from left to right.

FIG. 6 is a diagram for explaining a face part replacement function screen displayed in the image pickup control device shown in FIG. 1. FIG. 6A is a diagram showing the display of a self-portrait image, and FIG. 6B is a diagram showing a state in which one image (base image) is selected. Further, FIG. 6C is a diagram showing a state in which replacement of the face part is instructed, and FIG. 6D is a diagram showing an image obtained by replacement of the face part.

The face part replacement function is one of the editing functions in the photo editing module 15B, and is a function for replacing a face part (partial area) in a continuous self-portrait image to obtain a desired self-portrait image.

When the face part replacement function is activated, the controller 14 controls the display so that the face part replacement function screen shown in FIG. 6A is displayed on the liquid crystal display 102A. Here, the self-portrait image shown in FIG. 5 is displayed as a thumbnail image on the face part replacement function screen.

Referring to FIG. 6A, here, the self-portrait image shown in FIG. 5 is displayed as a thumbnail image in the thumbnail display area 602. Then, when the self-portrait image is flicked in the left-right direction, the self-portrait image displayed in the front row is changed. In the illustrated example, the display area 608 shows a state in which the third self-portrait image 601 is generated in the front row. Further, a part of the other self-portrait image is displayed, which indicates that the self-portrait image exists before and after the self-portrait image 601.

When the "Undo" button 603 is operated, the controller 14 cancels the immediately preceding operation and restores the screen. When the "Exit" button 607 is operated, the controller 14 terminates the editing function of the photo editing module 15B.

When the "back" button 605 is operated, the controller 14 returns to the screen before the editing function is activated. Further, the controller 14 displays that the folder name of the image data 15D captured by the in-camera 104 or the out-camera 108 and stored in the storage 15 is album 606.

With reference to FIG. 6B, when the user selects the third self-portrait image 601 as the base image, the controller 14 displays the self-portrait image 601 in the display area 608. The "base image" is an image of a face in which facial parts such as the right eye, left eye, nose, and mouth are replaced. Here, when the self-portrait image 601 shown in FIG. 6 (a) is double-tapped, the controller 14 selects the self-portrait image as the "base image" and displays itself in the display area 608 as shown in FIG. 6 (b). The captured image 601 is enlarged and displayed.

When the self-portrait image 601 is enlarged and displayed, the controller 14 displays the imaging time 610 (“December 3, 4:56”) in which the selected self-portrait image 601 is captured.

With reference to FIG. 6 (c), the user double-taps the area of the face part (right eye / left eye / nose / mouth) that he / she wants to change in the “base image” with his / her finger (right eye in the example shown in FIG. 6 (c)). .. As a result, the controller 14 displays only the face part (right eye) selected as the change target in the self-portrait image obtained by continuous shooting in the thumbnail display area 602.

When the user selects and taps one of the face parts (right eye) displayed in the thumbnail display area 602, the controller 14 replaces the face part of the base image with the selected face part. Then, the user replaces the face parts (eyes / nose / mouth) until the desired self-portrait image is obtained. When the confirmation button 604 is touch-operated, the controller 14 confirms the face part replacement.

In addition, in FIG. 6C, the controller 14 displays the display (face part replacement 611) indicating the replacement of the face part.

FIG. 6D shows a self-portrait image obtained as a result of replacement by the face part replacement function. When the face part replacement is confirmed, the controller 14 displays the edited mark 609 indicating edited as a status. Then, the controller 14 displays the image obtained as a result of replacing the face part of the self-portrait image 601 with the selected face part.

FIG. 7 is a flowchart for explaining a process at the time of self-shooting image pickup performed by the image pickup control device shown in FIG. The process according to the illustrated flowchart is performed by the controller 14 expanding the control program stored in the storage 15 into the system memory 16 and executing the process.

In S702, when the user taps the camera icon 306 on the menu screen 301, the controller 14 activates the camera function and starts the image pickup mode.

In S703, the controller 14 displays the image (video) obtained by the out-camera 108 or the in-camera 104 on the display 102A (see FIG. 4A).

In S704, the controller 14 determines whether or not the camera being imaged is the in-camera 104.

When the liquid crystal display 102A is a vari-angle display unit, it is determined whether or not the controller 14 liquid crystal display 102A is oriented in substantially the same direction as the imaging direction. That is, the controller 14 determines whether the display surface of the liquid crystal display 102A is in a facing position visible from the subject or at another position (for example, a position where the display surface faces the back side of the camera opposite to the subject). In other words, the controller 14 determines whether or not the subject side and the display direction of the liquid crystal display 102A displaying the live view image are on the same side. Then, based on the determination result, the controller 14 determines whether or not the in-camera 104 is present. The vari-angle display unit is a display unit in which the position of the display unit can be changed relative to the camera.

In the case of the in-camera 104, the process proceeds to S705, and if not, the process proceeds to S706.

In S706, the controller 14 uses the out-camera 108 to perform imaging. Then, the controller 14 ends the self-shooting process.

In S705, the controller 14 determines whether or not the shutter button 406 has been touch-operated on the camera function screen shown in FIG. 4A. If the shutter button 406 is touch-operated, the process proceeds to S710, otherwise the process proceeds to S708.

In S708, the controller 14 determines whether or not the imaging mode has ended. Here, the controller 14 determines that the image pickup mode is terminated when the button 1033D is touch-operated. When the image pickup mode ends, the controller 14 ends the self-shooting process. If not, the process returns to S705.

In S710, the controller 14 determines whether or not the imaging is continuous shooting. Here, when the shutter button 406 is operated by a long touch operation (an operation that continues without releasing the touch for a predetermined time or longer) on the camera function screen, the controller 14 determines that continuous shooting is performed. On the other hand, if it is a short tap (an operation of releasing the touch in less than a predetermined time), the controller 14 determines that the shooting is not continuous shooting but single shooting.

It should be noted that the controller 14 may determine whether the self-shooting continuous shooting is continuous shooting if the self-shooting continuous shooting is set in advance and the self-shooting continuous shooting is set. Immediately after changing the self-shooting continuous shooting setting in advance, or when the camera function is activated in the case of the self-shooting continuous shooting setting, the controller 14 displays the guide shown in FIG. 4 (b). Further, the number of images taken per hour when the shutter button 406 is operated with a long touch can be changed by using the setting button 407 shown in FIG. 4 (a).

If it is continuous shooting, proceed to S713, and if not, proceed to S711.

In S711, the controller 14 records and controls one captured image obtained by imaging in the storage 19. Then, the controller 14 ends the self-shooting process.

In S713, the controller 14 stores the captured image obtained by continuously shooting the imager by the in-camera 104 in the storage 19. In S714, the controller 14 ends continuous shooting. The end of continuous shooting is performed when instructed by the operator. Furthermore, continuous shooting may be terminated when it is detected that the imaging conditions have changed significantly, such as when the orientation of the imaging control device has changed.

In S715, the controller 14 displays a message dialog (introduction message of the face part replacement mode) 408 prompting the activation of the face part replacement function shown in FIG. 4B. In S716, the controller 14 determines whether or not the operator has selected the face part replacement function. If the face part replacement function is selected, the process proceeds to S717. If not, the controller 14 ends the self-shooting process.

In S717, the controller 14 executes the face part replacement mode by the face part replacement function. Then, the controller 14 ends the self-shooting process.

FIG. 8 is a flowchart for explaining the process in the face part replacement mode shown in FIG. 7. The process according to the illustrated flowchart is performed by the controller 14 expanding the control program stored in the storage 15 into the system memory 16 and executing the process.

In S802, the controller 14 starts the face part replacement mode by using the face part replacement function. As described above, the face part replacement function is a part of the editing function of the photo editing module 15B.

In S803, the controller 14 displays the continuous self-portrait image (video) stored in the storage 15 on the liquid crystal display 102A. Here, the controller 14 displays the continuous self-portrait image stored in the storage 15 in S713 as shown in FIG. 6A.

In S804, the controller 14 determines whether or not the base image has been selected. Here, the controller 14 determines whether or not the user has selected the base image on the screen shown in FIG. 6A. If the base image is selected, the process proceeds to S805, and if not, the process returns to S804.

In S805, the controller 14 reads the base image from the storage 19 and temporarily stores it in the system memory 16. In S806, the controller 14 determines whether or not the face part selection operation for replacing the face part has been performed. Here, the controller 14 determines whether or not a selection operation of double-tapping the face part (eyes / nose / mouth) to be changed has been performed. If the face part selection operation is performed, the process proceeds to S809, and if not, the process proceeds to S807.

In S807, the controller 14 determines whether or not the face part replacement mode has ended. Here, the controller 14 determines whether or not the end button 607 has been touch-operated. When the face part replacement mode ends, the controller 14 ends the self-shooting process. If not, the process returns to S806.

In S809, the controller 14 extracts selected facial parts from the continuous self-portrait image. Here, the controller 14 extracts the face parts selected in S806 from the continuous self-portrait image. In S810, the controller 14 determines whether or not the face parts have been extracted from all of the continuous self-portrait images. In addition, all of the continuous self-portrait images refer to the images continuously shot and captured by one imaging instruction performed in S705 and S710 shown in FIG. 7, and the images captured by the previous imaging instructions are. Not included. If the face parts are extracted from all of the continuous self-portrait images, the process proceeds to S811, otherwise the process returns to S809.

In S811, the controller 14 displays a list of face parts extracted from all of the continuous self-portrait images in the thumbnail display area 602. Here, the controller 14 displays a partial region (face part) for each facial organ detected from the image by the facial organ detection. At this time, as shown in FIG. 6C, the controller 14 enlarges and displays the right eye portion, which is the selected face part, as can be seen by referring to the thumbnail display area 602.

In S812, the controller 14 determines whether or not one of the face parts listed in S811 is selected by the tap operation. If one is selected from the list of face parts, the process proceeds to S813, and if not, the process returns to S811.

In S813, the controller 14 replaces the face parts of the temporarily stored base image with the selected face parts. That is, the controller 14 performs image processing in which the selected face parts are superimposed on the corresponding face parts of the base image and synthesized.

In S814, the controller 14 previews the image in which the face parts are replaced in S813 on the liquid crystal display 2A. In S815, the controller 14 determines whether or not the confirmation button 604 has been touch-operated. If the confirmation button 704 is touch-operated, the process proceeds to S816, and if not, the process returns to S811.

In S816, the controller 14 clears the list of extracted face parts and displays the image obtained as a result of the face parts replacement on the liquid crystal display 102A as shown in FIG. 6D.

In S817, the controller 14 determines whether or not the end button 607 has been touch-operated. When the end button 607 is touch-operated, the controller 14 exits the photo editing module 15B and proceeds to S818, otherwise returns to S806. This allows other facial parts to be replaced in response to further user intervention.

In S818, the controller 14 writes the replaced image temporarily recorded in the system memory 16 to the storage 15. Then, the controller 14 ends the face part replacement mode. The controller 14 may transfer the replaced image to, for example, a server and store it in the server. Further, the controller 14 may print the replaced image using a printing device (not shown). That is, when the image pickup control device is also applied to the photo sticker machine, the image pickup control device has a printing device, and the replaced image is printed by the printing device.

As described above, in the first embodiment of the present invention, a desired self-portrait image is created while watching a preview by a simple operation of selecting and combining desired face parts from a group of continuously captured self-portrait images. be able to.

Since these self-portrait images are images continuously captured without changing their positions in a very short period of time, the imaging conditions are constant. Therefore, it is possible to generate a natural self-portrait image that is the same as an image obtained by a general imaging method without requiring special image processing such as color tone, exposure, and focal length adjustment and alignment.

[Second Embodiment]
Subsequently, an example of the image pickup control device according to the second embodiment of the present invention will be described. The configuration of the image pickup control device according to the second embodiment is the same as that of the image pickup control device shown in FIGS. 1 and 2.

FIG. 9 is a flowchart for explaining a process at the time of automatic self-shooting imaging performed by the imaging control device according to the second embodiment of the present invention. The process according to the illustrated flowchart is performed by the controller 14 expanding the control program stored in the storage 15 into the system memory 16 and executing the process.

In the above-mentioned first embodiment, an example of self-shooting by continuous shooting by operating the shutter button 506 has been described. On the other hand, in the second embodiment, self-portrait imaging is automatically performed according to the determination by the controller 14. In FIG. 9, the processing of S902 to S904 is the same as the processing of S702 to S704 shown in FIG. 7, and the processing of S906 and S908 is the same as the processing of S706 and S708 shown in FIG. Therefore, the description thereof is omitted here.

In S905, the controller 14 determines whether or not the face is detected in the image obtained by the image taken by the in-camera 104. If a face is detected, the process proceeds to S910, and if not, the process proceeds to S908. At the time of face detection, the controller 14 uses a known face detection algorithm.

When the face is detected in S905, the controller 14 presumes that the photographer (user) is trying to take a self-portrait image. In S910, the controller 14 determines whether or not the state in which the amount of change in the image (live view video) obtained by imaging is equal to or less than a predetermined amount continues for a predetermined time. Here, it is presumed that the controller 14 is trying to determine the imaging pose if the state in which the amount of change in the image being captured by the in-camera 104 is equal to or less than a predetermined amount continues for a certain period of time. If the state in which the amount of change is equal to or less than the predetermined amount continues for a predetermined time, the process proceeds to S911, and if not, the process proceeds to S912.

In S911, the controller 14 determines whether or not the shake of the image being imaged is equal to or greater than a predetermined value. If the shaking is equal to or greater than a predetermined value, the process proceeds to S914, and if not, the process proceeds to S912.

In S911, the controller 14 determines whether or not the runout is equal to or greater than a predetermined value based on the difference in the change in the image within the predetermined time in the image being captured by the in-camera 4. The runout may be determined based on the sensing result of the acceleration sensor 12. Further, the determination may be made by combining the change in the image and the sensing result of the acceleration sensor 12 and the gyro sensor 13.

The controller 14 determines whether or not the image pickup control device is on hand depending on whether or not there is runout. That is, when there is a shake, the controller 14 presumes that the user is holding the image pickup control device and taking an image. On the other hand, if there is no runout, it is presumed that the controller 14 is not handheld and the image pickup control device is fixed to a tripod or the like. In other words, if the image is taken by the in-camera and the face is detected in the image, the controller 14 presumes that the image is a self-portrait image. Subsequently, if the amount of change in the image during imaging continues to be less than or equal to a predetermined amount for a certain period of time, the controller 14 estimates that the user is in the process of determining the imaging pose, and the runout is equal to or greater than the predetermined value. If there is, it is presumed that it is a handheld image. Then, the controller 14 executes the automatic continuous shooting imaging after S914.

In S912, the controller 14 cancels the automatic self-shooting and performs normal in-camera imaging. Here, the controller 14 determines that the image pickup control device is fixed and the image pickup is performed, and performs normal in-camera image pickup instead of the automatic self-shooting. Then, the controller 14 ends the auto self-shooting process.

In S914, the controller 14 displays a message for starting continuous shooting on the liquid crystal display 102A. Here, the controller 14 notifies the number of seconds remaining until the start of continuous shooting imaging, and displays a prompt to make a so-called fixed face during continuous shooting imaging.

In S915, the controller 14 executes automatic continuous shooting imaging by the in-camera 4. That is, when the face is detected by in-camera imaging and the user decides the imaging pose and determines that the image is handheld, the controller 14 notifies the operator of the message of the start of automatic continuous shooting and continuous shooting. To execute.

Hereinafter, since the processing of S916 to S920 is the same as the processing of S713 to S717 shown in FIG. 7, the description thereof will be omitted.

As described above, in the second embodiment of the present invention, when there is a high possibility that the user is trying to take a "selfie", continuous shooting is automatically performed while notifying the user of a message. This eliminates the need for the user to instruct the start of continuous shooting, so that there is no risk of the composition once being broken and the user can concentrate on determining the pose and facial expression. As a result, it is possible to more easily obtain a self-portrait image desired by the user.

[Third Embodiment]
Next, an example of the image pickup control device according to the third embodiment of the present invention will be described. The configuration of the image pickup control device according to the third embodiment is the same as that of the image pickup control device shown in FIGS. 1 and 2.

By the way, when a plurality of people try to take a self-portrait image, the plurality of people may not fit in the optical angle of view. In this case, for example, a so-called selfie stick is used to increase the distance between the image pickup control device and a plurality of subjects.

On the other hand, in the third embodiment, a group of continuously captured images are joined together to generate an image that generates an image having a wider angle than the optical angle of view of the camera, and a state in which a plurality of people are imaged. do. For example, if there is an angle of view capable of stitching the self-portrait images captured by continuous images, the controller 14 displays the self-portrait images captured on the liquid crystal display 102A as an extended display image described later. Then, the user trims a desired area from the extended display image to obtain a wide-angle self-portrait image including a plurality of subjects.

FIG. 10 is a diagram for explaining self-shooting imaging of a plurality of people performed in the imaging control device according to the third embodiment of the present invention.

In the example shown in FIG. 10, eight wide-angle self-portrait images including a plurality of subjects obtained as a result of trimming are displayed in the display area 60. The controller 14 displays the area frame 1001 at the position of the face of each subject. When one of the area frames 1001 is selected by the user, the controller 14 enlarges and displays the face image displayed in the selected area frame 100 over the entire display area 608. Then, as described in the first embodiment, the face parts of the face image are replaced.

When the replacement of the face parts is completed, the controller 14 replaces the original face image of the subject with the replaced image, and displays a wide-angle self-portrait image for the plurality of subjects again. When replacing the face parts of another subject, if the area frame 1001 displayed at the position of the face of the other subject is selected, the process is performed in the same manner.

As described above, in the third embodiment of the present invention, even in the case of self-portrait imaging in which a plurality of subjects do not fit in the optical angle of view of the camera, the user desires for these plurality of subjects. You can create a captured image.

In the above-described embodiment, the image pickup control device including the out-camera 108 and the in-camera 104 has been described as an example. However, even if the in-camera is not provided, the display is a so-called vari-angle type. , The present invention can be carried out. In this case, the self-shooting image is taken by adjusting the vari-angle angle on the same surface as the image pickup lens.

The above-mentioned various controls described as being performed by the controller 14 may be performed by one hardware, or the entire device may be controlled by sharing the processing by a plurality of hardware.

Further, although the present invention has been described in detail based on the preferred embodiment thereof, the present invention is not limited to these specific embodiments, and various embodiments within the range not deviating from the gist of the present invention are also included in the present invention. included. Furthermore, each of the above-described embodiments is merely an embodiment of the present invention, and each embodiment can be appropriately combined.

Further, in the above-described embodiment, the case where the present invention is applied to a mobile terminal such as a smartphone has been described as an example, but the present invention is not limited to this example, and is a device having an image pickup unit and capable of continuous shooting. If there is, it is applicable. That is, the present invention can be applied to personal computers, PDAs, mobile phone terminals, portable image viewers, tablet terminals, smartphones, projection devices, in-vehicle devices, and the like.

The present invention can also be applied to a control device that communicates with an image pickup control device (including a network camera) via wired or wireless communication and remotely controls the image pickup control device. As a device for remotely controlling the image pickup control device, for example, there is a device such as a smartphone, a tablet PC, or a desktop PC. The image pickup control device is remotely controlled by notifying the image pickup control device of commands for performing various operations and settings based on the operations performed on the control device side and the processes performed on the control device side. It is possible. Further, the live view image captured by the image pickup control device may be received via wired or wireless communication and displayed on the control device side.

[Other embodiments]
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

14 controller 15 storage (non-volatile memory)
102 Touch screen display 103A-103D button 104 In-camera 105 Illuminance sensor 106 Speaker 107 Connector 108 Out-camera 109 Light

Claims (10)

An imaging control means that controls continuous imaging with the imager as the subject,
A selection means for selecting a partial region in the first image from the plurality of images obtained by the continuous imaging according to an operation from the user, and a selection means.
An image processing means that performs a process of replacing a partial region selected by the selection means with a corresponding partial region of a second image different from the first image among the plurality of images.
It has a display control means for controlling to display an image that has been replaced by the image processing means.
Imaging control characterized in that the imaging control means starts the continuous imaging when the state in which the amount of change in the live view image is equal to or less than a predetermined amount continues for a predetermined time and the shaking of the image is equal to or greater than a predetermined value. Device. The imaging control device according to claim 1, wherein the selection means selects a partial region of each facial organ detected from an image by facial organ detection. The image pickup control device according to claim 1 or 2, wherein the second image is an image selected by a user from the plurality of images excluding the first image. Further, it has an extraction means for extracting a partial region corresponding to the partial region selected by the selection means in each of the plurality of images.
The image pickup control device according to claim 3, wherein the display control means controls to display a partial region extracted by the extraction means when accepting the selection of the second image. The image pickup control device according to any one of claims 1 to 4, further comprising a recording control means for controlling to record an image that has been subjected to a replacement process by the image processing means. One of claims 1 to 5, wherein the display control means controls to display a guide for encouraging the imager to make various facial expressions before or during the continuous imaging. The imaging control device according to the section. The invention according to any one of claims 1 to 6 , further comprising an image generation means for joining the images obtained by the continuous imaging to generate an image having a wider angle than the optical angle of view. Image control device. It is a control method of an image pickup control device that performs image pickup.
An imaging control step that controls continuous imaging with the imager as the subject,
A selection step of selecting a partial region in the first image from the plurality of images obtained by the continuous imaging according to an operation from the user, and a selection step.
An image processing step of performing a process of replacing a partial area selected in the selection step with a corresponding partial area of a second image different from the first image among the plurality of images.
It has a display control step that controls to display an image that has been replaced by the image processing step.
A control characterized in that continuous imaging is started in the imaging control step when the state in which the amount of change in the live view image is equal to or less than a predetermined amount continues for a predetermined time and the shaking of the image is equal to or greater than a predetermined value. Method. A program for making a computer function as each means of the image pickup control device according to any one of claims 1 to 7 . A computer-readable storage medium containing a program for causing the computer to function as each means of the image pickup control device according to any one of claims 1 to 7 .

Images