JP2016100800A - Imaging device, imaging method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device, an imaging method and a program that can very easily achieve an excellent pickup image under self-photographing.SOLUTION: In an imaging device having imaging means containing an imaging lens for imaging a subject, and display means for successively displaying pickup images obtained by the imaging means, in parallel to an imaging operation of the imaging means, a visual line direction of a photographer is detected from a pickup image of the photographer which is obtained by the imaging means, and it is detected whether the visual line direction is a camera direction along which the imaging lens is oriented (S4). pickup images (frame images) for which the visual line direction is determined to be the camera direction are successively stored in a frame buffer (S10). After the number of images stored in the frame buffer exceeds a specified number (S8: YES), a moving picture comprising a series of stored frame images or a specific frame image is set as a final imaging result. In parallel to a self-photographing work, pickup images during a time period for which the visual line is directed to the imaging lens can be automatically obtained.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a photographing apparatus, a photographing method, and a program.

  In recent years, SNS (Social Networking Service) has become widespread along with improvements in the use environment of the Internet, and it is common practice to publish or share still images and moving images taken by the user on the Internet. In addition, mobile phone terminals with cameras such as smartphones and personal computers with built-in cameras are used on a daily basis, and users such as smartphones take pictures of their faces for the purpose of disclosure on the Internet. Opportunities to perform so-called “selfie” are increasing.

  On the other hand, as a technique useful for the above-described self-portrait, for example, the following technique is described in Patent Document 1 below. That is, in a camera in which a camera unit including a photographing lens is rotatably provided in a main body having a display unit used as a finder at the time of shooting, when the photographer rotates the camera unit by 180 ° and takes a self-portrait, A technique for preventing a sense of incongruity by inverting an image to be displayed upside down and using a mirror image is described.

JP 2008-53925 A

  However, in Japanese Patent Application Laid-Open No. 2004-228688, the following problems arise when taking a self-portrait using a camera whose camera unit can rotate with respect to a display unit used as a viewfinder.

  For example, the same applies to smartphones or the like, but in a configuration in which the camera unit and the display unit are arranged apart from each other, the photographer appears on the display unit because the distance between the photographer and the camera unit (shooting distance) is short. If you shoot while looking at your own face, you can only shoot a face with a line of sight.

  For this reason, in order to obtain a good captured image with no line of sight deviation, the photographer moves the line of sight to the camera once, and then returns the line of sight to the display unit to obtain the photographic result (moving image or The troublesome task of confirming (still images) has to be repeated, and there is a problem that it takes time and effort to take a selfie.

  The present invention has been made in view of such conventional problems, and an object of the present invention is to provide a photographing apparatus, a photographing method, and a program capable of extremely easily obtaining a good photographed image at the time of self-taking.

  In order to solve the above-described problem, in the present invention, the gaze direction of the photographer photographed by the photographing means is parallel to the photographing means including the photographing lens for photographing the subject and the photographing operation by the photographing means. A determination unit that determines whether or not the camera is facing the photographic lens; and an output unit that outputs a photographic image captured by the photographic unit while the line-of-sight direction is determined to be the photographic lens direction by the determination unit. It is provided with.

  According to the present invention, a good captured image can be obtained very easily during self-portrait.

It is a front view of the smart phone illustrated as an imaging device of the present invention. It is a block diagram which shows the electric constitution of a smart phone. It is a flowchart which shows the operation | movement content in self-shooting mode. It is a flowchart following FIG. It is a flowchart which shows a preservation | save process. It is a conceptual diagram which shows the frame image which comprises the moving image for a confirmation. (A) is an example of a through image when the line of sight is directed toward the display screen, (b) is an example of a through image when the line of sight is directed toward the camera unit, and (c) is an example of a moving image for confirmation. FIG.

  Hereinafter, an embodiment of the present invention will be described. FIG. 1 is a front view of a smartphone 1 having a function as a photographing apparatus of the present invention. The smartphone 1 is provided with a color liquid crystal display panel 3 for displaying characters and images on the surface of the device body 2, and a camera unit 4 and a speaker 5 in which a photographing lens is arranged on the upper side, and A plurality of operation buttons 6 are provided on the lower side.

  The camera unit 4 is mainly used for so-called self-portrait. On the back side of the device body 2 (not shown), another camera unit used for shooting other shooting targets other than the photographer is also provided. ing.

  FIG. 2 is a block diagram showing an outline of the electrical configuration of the smartphone 1. As shown in FIG. 2, the smartphone 1 includes a control unit 11, a power supply unit 12, a communication unit 13, an imaging unit 14, an image storage unit 15, a program storage unit 16, a frame buffer 17, a display unit 18, and an operation unit 19. It has each part.

  The control unit 11 includes a CPU (Central Processing Unit), its peripheral circuits, and a work internal memory such as a RAM (Random Access memory), and controls each unit of the smartphone 1. The control unit 11 includes an internal clock that counts the current date and time.

  The power supply unit 12 includes a rechargeable battery, a DC / DC converter, and the like, and supplies power necessary for each operation to each unit of the smartphone 1.

  The communication unit 13 includes a wireless communication circuit for performing wireless telephone communication and Internet communication based on TCP / IP (Transmission Control Protocol / Internet Protocol), a voice input circuit including a microphone (not shown) for performing a call, A receiving circuit for decoding and reproducing the audio signal, a reproducing circuit including the speaker 5 and the like.

  The imaging unit 14 includes the above-described camera unit 4 on the front surface side of the device main body 2, two imaging elements individually provided in the camera unit (not shown) on the back surface side of the device main body 2, an image processing engine, and the like. Consists of

  The imaging device is, for example, a CCD (Charge Coupled Device) or CMOS (Complementary Meta 10xide Semiconductor) type imaging device. The image processing engine performs AFE (Analog Front End) that performs amplification and conversion to digital signals of the image signals output from each image sensor, and performs various processing on the digital signals to obtain luminance (Y) components and color differences. The image processing circuit is configured to convert image data (YUV data) composed of (UV) components, and a frame memory that stores image data for at least one screen.

  While the camera mode is set in the smartphone 1, the imaging unit 14 sequentially captures a subject at a predetermined frame rate (for example, 30 fps) with one of the two imaging elements, and sequentially supplies image data of each frame to the control unit 11. To do.

  The image data supplied to the control unit 11 is sequentially stored in the frame buffer 17 and supplied to the display unit 18 including the color liquid crystal display panel 3 and displayed on the screen as a live view image as a moving image.

  The frame buffer 17 is composed of, for example, an SDRAM (Synchronous DRAM) capable of high-speed data transfer, and the frame buffer 17 has a memory capacity capable of storing image data for a plurality of frames.

  The display unit 18 includes the color liquid crystal display panel 3 and a display drive circuit that drives the color liquid crystal display panel 3 in accordance with image data supplied from the control unit 11.

  Further, the image data stored in the frame buffer 17 is compressed as still image data by the control unit 11 using a JPEG (Joint Photographic Expert Group) method or the like as necessary, and the image data is converted into a still image file to which various attribute information is added. It is stored in the storage unit 15. Further, the image data is sequentially compressed as moving image data in the control unit 11 as required by the MPEG (Motion Picture Experts Group) method or the like, and stored in the image storage unit 15 as a moving image file to which various attribute information is added. .

  Still image data or moving image data recorded as a still image file or a moving image file in the image storage unit 15 is read and expanded by the control unit 11 as necessary, and then supplied to the display unit 18 to be a color liquid crystal display panel. 3 is reproduced (displayed) as a still image or a moving image.

  The image storage unit 15 includes, for example, a flash memory built in the smartphone 1, a memory card that is detachable from the smartphone 1, and a card interface that inputs and outputs data to the memory card.

  The operation unit 19 includes a plurality of operation buttons 6 described above, a capacitive touch sensor (not shown) that is integrally provided on the surface of the color liquid crystal display panel 3 and forms a touch panel, and a drive circuit thereof. Is done. The operation state of the operation button 6 and the touch sensor in the operation unit 19 is monitored at any time by the control unit 11.

  The program storage unit 16 is composed of a nonvolatile memory such as a flash memory in which stored data can be rewritten as needed. The program storage unit 16 stores a control program for causing the control unit 11 to control each unit of the smartphone 1.

  In particular, the program storage unit 16 specifies the eyes of a person in the image for each frame image (hereinafter referred to as a frame image) captured by the imaging unit 14 to the control unit 11, and A program for performing processing to be described later, including visual line detection processing for detecting the visual line direction, is stored.

  In the smartphone 1 having the above configuration, a self-portrait mode is prepared as an operation mode that is a lower level of the shooting mode. The self-portrait mode is a photographing mode used when the photographer saves a photographed image of himself / herself mainly for the purpose of sharing with others. Also, there are two types of captured images to be saved: still images and moving images. When setting the self-portrait mode, the photographer sets in advance whether to save the still images or moving images. can do.

  Next, the operation | movement which concerns on this invention of the smart phone 1 is demonstrated, referring a figure. 3 to 5 are flowcharts showing processing contents of the control unit 11 when the above-described selfie mode is set by the user in the smartphone 1.

  As shown in FIG. 3, when the control unit 11 starts the operation together with the setting of the self-portrait mode, the control unit 11 immediately starts the photographing process (step S1). That is, the control unit 11 starts an imaging operation at a predetermined frame rate by the imaging unit 14 and a process of sequentially displaying the captured image, that is, a captured image (hereinafter referred to as a frame image) as a through image on the display unit 18 (Step S1). S1).

  Thereafter, the control unit 11 executes the processing described below in parallel with the above photographing operation. First, the control unit 11 performs initialization for deleting the image data stored in the frame buffer 17 and resets the counter for the number of stored images (n), that is, the number of stored frame images in the frame buffer 17 (step S2). .

  Then, every time a new frame image is acquired (step S3: YES), the control unit 11 performs a gaze direction determination process on the newly acquired frame image (step S4).

  Such processing detects the direction of the line of sight of the photographer who is taking a self-portrait, and determines whether or not it is the direction of the photographing lens facing the camera unit 4, that is, the photographing lens (hereinafter referred to as the camera direction). It is processing. In such processing, the method for determining the line-of-sight direction is arbitrary, and various known techniques can be used as appropriate.

  As an example, first, face detection is performed to detect a face area corresponding to a photographer's face present in the image, parts recognition is performed on the face area and the photographer's eyeball is specified, and then the 3D model of the eyeball is detected. Is estimated. That is, the eyeball center, eyeball radius, and pupil center are estimated based on information such as the positions of the corners of the eyes and the eyes and the contours of the black eyes. Here, since the line connecting the center of the pupil and the center of the eyeball is the line-of-sight direction, if both coincide with each other on the image, it can be estimated that the line-of-sight direction of the photographer is the camera direction.

  Therefore, the distance between the center of the eyeball and the center of the pupil on the image is calculated, and normalized by the distance face size is obtained as a deviation coefficient. If the deviation coefficient is equal to or greater than a threshold value, the line-of-sight direction is the camera direction. If the deviation coefficient is not greater than or equal to the threshold value, it is determined that the line-of-sight direction is the camera direction. That is, if the amount of deviation between the line-of-sight direction and the camera direction is greater than or equal to the reference, it is determined that the line-of-sight direction is not the camera direction. If the amount of deviation between the line-of-sight direction and the camera direction is not greater than the reference, the line-of-sight direction is the camera direction. Judge that there is.

  Note that the gaze direction determination process determines whether or not the detected gaze direction is the camera direction after detecting the gaze direction of the photographer who is taking a selfie. It may be determined whether or not the line-of-sight direction is facing the camera direction.

  Subsequently, when it is determined by the line-of-sight determination process in step S4 that the line-of-sight direction is not the camera direction (step S5: NO), if the recording mark is not being displayed (step S6: NO), the above-described image is immediately displayed. It is confirmed whether or not the stored number (n) exceeds the specified number (Nthresh) (step S7).

  Then, for example, when the number of stored images does not exceed the specified number as in the beginning of the process in the self-portrait mode (step S7: NO), the control unit 11 returns to the process of step S2.

  FIG. 7A is a diagram illustrating an example of the through image G1 when the photographer is turning his / her line of sight toward the display screen of the display unit 18 (color liquid crystal display panel 3). Then, the control unit 11 repeats the same processing as described above while the photographer turns his / her line of sight toward the display screen.

  Thereafter, when determining that the line-of-sight direction is the camera direction (step S5: YES), the control unit 11 executes the following processing. That is, the control unit 11 displays the recording mark M (see FIG. 7B) on the through image G2 (step S9), stores the frame image at that time in the frame buffer 17 (step S10), and stores the image. The number (n) is updated (incremented) (step S11).

  Then, the control unit 11 returns to the process of step S3, and determines a new frame while determining that the line-of-sight direction is the camera direction, that is, while the photographer is turning the line of sight toward the camera unit 4 (photographing lens). Every time an image is acquired, the processes in steps S9 to S11 are repeated. As a result, the captured images for a plurality of continuous frames, which are in a state where the photographer is looking at the camera unit 4, are accumulated in the frame buffer 17.

  FIG. 7B is a diagram illustrating an example of the through image G2 when the photographer turns his / her line of sight toward the camera unit 4.

  If the control unit 11 determines that the line-of-sight direction is not the camera direction while accumulating each frame image in the frame buffer 17 as described above (step S5: NO), that is, the photographer views the line of sight. 4 is moved to a display screen of the display unit 18 (hereinafter simply referred to as a display screen) or the like, the above-described processing of steps S6 to S8 is executed.

  First, since the control unit 11 is displaying the recording mark M on the through image at that time (step S6: YES), after the display of the recording mark M is finished (step S7), the number of stored images is the specified number. It is confirmed whether or not it exceeds (step S7).

  If the number of stored images does not exceed the specified number (step S7: NO), the control unit 11 returns to the process of step S2, initializes the frame buffer 17 once, and resets the number of stored images (n). After that, the above-described processing is repeated.

  On the other hand, when the number of stored images exceeds the specified number (step S7: YES), the control unit 11 continues to execute the process shown in FIG.

  First, the control unit 11 initializes the display frame number (t), and sets a number obtained by adding 1 to the predetermined number of excluded frames (Ntrim) as an initial value (step S12).

  Subsequently, the control unit 11 reads out the data of the t-th stored frame image from all the frame images stored in the frame buffer 17 (step S13) and displays it on the display screen (step S14).

  Thereafter, the control unit 11 executes the following processing until a predetermined save instruction operation is performed by the photographer (step S15: NO).

  That is, every time a new frame image is acquired (step S16: YES), the control unit 11 performs a line-of-sight direction determination process (step S17). Note that the specific processing content at this time is the same as the processing in step S4, and the direction of the photographer's line of sight is detected using various known techniques for a new frame image, and this is the camera direction. It is determined whether or not.

  When determining that the photographer's line-of-sight direction is not the camera direction (step S18: NO), the control unit 11 first increments the display frame number (t) (step S19).

  Subsequently, the control unit 11 determines that the new display frame number (t) is obtained by subtracting the number of excluded frames from the number of stored images (the number of all frame images stored in the frame buffer 17) (n-Ntrim). ) Is exceeded (step S20).

  If the new display frame number (t) does not exceed the above number (n−Ntrim) (step S20: NO), the control unit 11 returns to the process of step S13 and repeats the above-described process. That is, the control unit 11 reads out the data of the frame image stored at the t-th (with a new display frame number) from all the frame images stored in the frame buffer 17 and displays the data on the display screen (Step S1). S14).

  Thereafter, while the control unit 11 determines that the sight line direction of the photographer is not the camera direction (step S18: NO), until the new display frame number (t) exceeds the above number (n-Ntrim). (Step S20: NO), the above process is repeated.

  On the other hand, if the new display frame number (t) exceeds the number (n-Ntrim) while repeating the above process (step S20: YES), the control unit 11 returns to the process of step S12 at that time. Once the display frame number (t) is initialized, the above-described processing is repeated.

  That is, while it is determined that the photographer's line-of-sight direction is not the camera direction (step S18: NO), the confirmation moving image is repeatedly displayed on the display screen.

  FIG. 6 is a conceptual diagram showing frame images constituting the moving image Gm. The moving image Gm has a display frame number (t) of an initial value among all frame images stored in the frame buffer 17. It is a moving image composed of frame images from (Ntrim + 1) to a number (n−Ntrim) obtained by subtracting the number of excluded frames from the number of stored images (n). In other words, the moving image Gm is a frame period (hereinafter referred to as a confirmation frame period) obtained by removing a predetermined frame period at the beginning and the end from a frame period in which a series of frame images stored in the frame buffer 17 is captured. ).

  Thereby, the photographer can repeatedly confirm the moving image Gm simply by directing his / her line of sight toward the display screen.

  FIG. 7C is a diagram showing an example of the moving image Gm displayed on the display screen, and the content is displayed as a through image G2 when the photographer turns the line of sight toward the camera unit 4 immediately before. It is the same as the moving image in the confirmation frame period in the moving image that has been performed. Further, the recording mark M described above is not displayed while the moving image Gm is displayed.

  On the other hand, when the control unit 11 determines that the sight line direction of the photographer is the camera direction while displaying the confirmation moving image Gm on the display unit 18 (step S18: YES), the above-described FIG. The process from step S2 to step S11 is repeated. That is, the control unit 11 once initializes the frame buffer 17, and then again determines a frame image while determining that the photographer's line-of-sight direction is the camera direction, that is, the photographer directs the line of sight toward the camera unit 4. The captured image in the state is newly stored in the frame buffer 17.

  Thereafter, the control unit 11 repeats the above-described steps S12 to S20 of FIG. 4 when the number of stored images in the frame buffer 17 again exceeds the specified number (step S8: YES). This causes the photographer to repeatedly confirm a new moving image Gm.

  Then, while the processing of step S12 to step S20 is repeated, that is, while the moving image Gm for confirmation is being displayed, if a save instruction operation is performed by the photographer at any time (step S15: YES), FIG. The storage process shown in FIG. 5 is executed (step S21).

  If the type of the captured image set in advance as a storage target is a moving image during the storing process (step S101: YES), the control unit 11 displays the moving image Gm being displayed, that is, the confirmation frame period shown in FIG. Is stored in the image storage unit 15 (step S102).

  In addition, if the type of the captured image set in advance as the storage target is a still image (step S103: YES), the control unit 11 uses the frame image being displayed at the time when the storage instruction operation is performed as the still image. Save in the storage unit 15 (step S103).

  And the control part 11 complete | finishes all the processes by self-portrait mode with the completion | finish of the said preservation | save process.

  As described above, at the time of shooting in the self-portrait mode, a series of frame images taken sequentially is automatically stored in the frame buffer 17 while it is determined that the direction of the photographer's line of sight is the camera direction. Thereafter, a moving image composed of the series of frame images stored in the frame buffer 17 or a specific frame image, that is, a still image in the series of frame images, is stored in the image storage unit 15 as a photographing result.

  For this reason, the photographer does not repeat the complicated operation of moving the line of sight to the camera unit 4 and performing shooting, then shifting to the playback mode, returning the line of sight to the display unit 18 and checking the shooting result. In both cases, it is possible to reliably obtain a captured image (moving image or still image) with the line of sight directed toward the photographic lens. Therefore, the photographer can obtain a good photographed image very easily when taking a selfie.

  Further, when it is determined that the photographer's line-of-sight direction is no longer the camera direction, the series of frame images stored in the frame buffer 17 are displayed on the display unit 18 as a confirmation moving image Gm, and the moving image or a specific frame is displayed. The image is stored in the image storage unit 15 as a photographing result. Therefore, the photographer can easily confirm the photographing result to be stored in advance by simply returning the line of sight from the camera unit 4 to the display unit 18.

  In addition, since the confirmation moving image Gm is repeatedly displayed until a save instruction operation is performed by the photographer, the photographer can surely confirm the shooting result to be saved.

  Further, since the confirmation moving image Gm is displayed on the display unit 18 instead of the through image G2, for example, the photographer can compare with the case where the confirmation moving image Gm is displayed together with the through image on a part of the display screen. In, it is possible to more easily confirm the photographing result to be stored.

  Further, while a series of frame images are stored in the frame buffer 17, that is, while it is determined that the sight line direction of the photographer is the camera direction, the recording mark M is displayed on the through image G2. did. Therefore, the photographer can know in real time whether or not the photographed image to be stored can be photographed while the through image G2 is displayed, and at the same time, the moving image displayed during the photographing operation can be confirmed. It is possible to easily distinguish between the moving image Gm for use and the through image G2.

  In addition, since the shooting result to be saved is a shot image (moving image or still image) of a frame period excluding the predetermined frame period at the beginning and end of the series of frame images, the image is finally saved. From the photographed image, a photographed image immediately after the photographer's line-of-sight direction is determined to be the camera direction during photographing or immediately before it is determined that the photographer's line-of-sight direction is no longer the camera direction is excluded from the storage target.

  Therefore, even if the determination accuracy of the line-of-sight direction is low, the captured image to be saved is a good captured image in a state where the photographer's line of sight is directed to the photographing lens. Therefore, the photographer can obtain a good photographed image very easily and reliably when taking a selfie.

  In addition, when the photographed image to be saved is a still image, the frame image of the frame that was displayed when the save instruction operation was performed by the photographer while the confirmation moving image Gm was displayed was saved. . Therefore, it is possible for the photographer to obtain a good still image very easily and reliably when taking a selfie.

  Further, when it is determined that the sight line direction of the photographer is the camera direction while the confirmation moving image Gm is displayed on the display unit 18, in response thereto, the sight line direction of the photographer is again the camera direction. The frame image in between is determined and stored in the frame buffer 17. Therefore, if the photographer Gm for confirmation or the frame image constituting the photographer is not intended as a storage target, the photographer simply puts his / her line of sight while the confirmation movie Gm is displayed. It is possible to immediately shift to a new shooting operation simply by shifting to 4.

  Here, in the present embodiment, the photographer can select whether the captured image to be stored is a moving image or a still image as necessary. However, the captured image is stored in advance as a moving image and a still image. It is good also as a structure which is only any one of an image.

  Further, when the type of captured image to be saved can be selected as in the present embodiment, for example, as a method for performing a save instruction operation, a method for saving a moving image and a method for saving a still image, These two types of methods may be prepared, and the type of captured image may be selected simultaneously with the save instruction operation.

  Similarly to the present embodiment, when the photographer selects the type of captured image to be stored prior to the shooting operation, when a still image is selected as the type of captured image, only the still image can be stored. In such a case, the following processing may be performed.

  For example, the number of frame images stored in the frame buffer 17 is reduced, and when it is determined that the photographer's line-of-sight direction is not the camera direction, the series of frame images are slid and displayed at predetermined time intervals. May be.

  Further, for example, the frame buffer 17 stores only a captured image after several frames (after a predetermined time) after it is determined that the direction of the sight line of the photographer is the camera direction, and the captured image is displayed for a certain period of time. May be. Further, for example, in a configuration that can ensure a high accuracy as the determination direction of the line-of-sight direction, the frame buffer 17 stores only the captured image determined to be the camera direction of the photographer, and the captured image is constant. You may make it display a time.

  Similarly to the present embodiment, when the type of captured image to be stored is selected prior to the shooting operation, when a moving image is selected as the type of the captured image, or when only a moving image can be stored Alternatively, the following processing may be performed.

  For example, instead of the display process of the confirmation moving image Gm in the present embodiment, a display process for displaying all of the series of frame images stored in the frame buffer 17 is performed, and a moving image to be stored is confirmed in the present embodiment. A moving image having the same frame period as the moving image Gm for use may be used.

  On the other hand, in the present embodiment, the photographer can confirm the moving image in advance before storing the moving image. However, for example, when a predetermined number of frame images are stored in the frame buffer 17, the entire frame period is immediately stored. Alternatively, a moving image composed of the frame image or the frame image of a predetermined frame period may be automatically stored. This also applies to the case of storing a still image. For example, when a photographed image that is determined to be in the camera direction can be photographed while a through image is being displayed, the photographed image of the negative is immediately preserved. It is good also as a structure.

  In the present embodiment, the recording mark M is displayed on the through image G2 while the series of frame images are accumulated in the frame buffer 17, so that the moving image displayed during the shooting operation is for confirmation. The moving image Gm and the through image G2 can be easily distinguished from each other, but may be as follows.

  For example, while displaying the confirmation moving image Gm, a frame for clearly indicating it may be displayed on the outer peripheral portion of the display screen. Even in this case, it is possible to easily distinguish whether the moving image displayed during the photographing operation is the confirmation moving image Gm or the through image G2.

  Furthermore, in the above description, the case where the photographed image obtained by the self-portrait operation is mainly stored has been described. However, the present invention is not limited to confirming the captured image obtained by the self-portrait operation in advance, for example. The present invention can also be applied to the case of outputting to the outside in real time as in the case of transmitting in real time from 14.

  Moreover, in this embodiment, although the smart phone 1 was illustrated as an imaging device of this invention, it is not restricted to this, The camera part containing an imaging lens is rotatably provided in the main body which has a display part used as a finder at the time of imaging | photography. It can also be applied to other cameras. Even in such a case, when the photographer takes a self-portrait by rotating the camera unit by 180 °, a good captured image can be obtained very easily.

  Furthermore, the present invention can also be applied to a notebook type personal computer with a built-in camera, a personal computer with a camera, and the like. Even in this case, the photographer operating the personal computer can obtain a good captured image very easily when taking a selfie.

As mentioned above, although embodiment of this invention and its modification were demonstrated, as long as these are in the range with which the effect of this invention is obtained, it can change suitably, and embodiment after change is also in a claim. It is included in the scope of the invention described and equivalent invention. The invention described in the scope of the claims of the present application will be appended below.
[Claim 1]
Photographing means including a photographing lens for photographing a subject;
In parallel with the photographing operation by the photographing means, a judging means for determining whether or not the sight line direction of the photographer photographed by the photographing means faces the photographing lens;
An imaging apparatus comprising: output means for outputting a photographed image photographed by the photographing means while the line-of-sight direction is judged to be a photographing lens direction by the judging means.
[Claim 2]
Comprising display means for sequentially displaying captured images photographed by the photographing means;
Either the structure in which the photographing lens is arranged in a positional relationship that is separated from the display screen of the display means and the photographer's field of view, or the structure that can be arranged in a positional relationship that is separated in the photographer's field of view as necessary The photographing apparatus according to claim 1, comprising either one of the structures.
[Claim 3]
Image storage means for storing a photographed image;
The imaging apparatus according to claim 2, wherein the output unit outputs the captured image, which is determined by the determination unit to be the photographic lens direction, to the image storage unit.
[Claim 4]
The imaging apparatus according to claim 3, wherein the output unit outputs the captured image, which is determined by the determination unit as the sight line direction to be a photographing lens direction, as a still image to the image storage unit.
[Claim 5]
The output means outputs the captured images for a plurality of consecutive frames determined by the determination means as the direction of the photographic lens to the image storage means as a moving image. The imaging device described.
[Claim 6]
Temporary storage means for temporarily storing the photographed image photographed by the photographing means while the line-of-sight direction is judged to be the photographing lens direction by the judging means;
In response to determining that the line-of-sight direction is not the shooting lens direction by the determination unit, the display control unit displays the captured image stored immediately before in the temporary storage unit on the display unit,
The imaging apparatus according to claim 3, wherein the output unit outputs the captured image displayed on the display unit by the display control unit to the image storage unit.
[Claim 7]
The temporary storage means temporarily stores the captured images for a plurality of consecutive frames photographed by the photographing means while the line-of-sight direction is determined to be a photographing lens direction by the determination means,
The photographing apparatus according to claim 6, wherein the display control means causes the display means to display a moving image composed of the photographed images for the plurality of frames stored immediately before in the temporary storage means.
[Claim 8]
8. The photographing apparatus according to claim 7, wherein the display control means causes the display means to display a moving image composed of a plurality of frames of the photographed images in a predetermined frame period temporarily stored immediately before in the temporary storage means.
[Claim 9]
The predetermined frame period is a frame period excluding a predetermined frame period at the beginning and the end among frame periods in which the line-of-sight direction is determined to be a photographing lens direction by the determination unit. The imaging device according to claim 8.
[Claim 10]
10. The display control unit repeatedly displays the moving image on the display unit while the determination unit determines that the line-of-sight direction is not the photographing lens direction. Shooting device.
[Claim 11]
11. The photographing apparatus according to claim 7, wherein the display control unit causes the display unit to display the moving image instead of the photographed images sequentially photographed by the photographing unit. .
[Claim 12]
In the display control unit, the captured image displayed on the display unit is either the captured image sequentially captured by the capturing unit or the captured image stored immediately before in the temporary storage unit. The imaging apparatus according to claim 11, wherein an identification mark that clearly indicates is displayed on the display unit together with any one of the captured images.
[Claim 13]
The display control means is configured to obtain, as the identification mark, the photographed images for a plurality of consecutive frames photographed by the photographing means while the line-of-sight direction is determined to be a photographing lens direction by the determining means. 13. The photographing apparatus according to claim 12, wherein an identification mark indicating that it is being stored in the temporary storage means is displayed on the display means together with the photographed images sequentially photographed by the photographing means.
[Claim 14]
While the display control means is displaying on the display means a moving image composed of the captured images for the plurality of frames stored immediately before in the temporary storage means, the presence or absence of an instruction to save the captured image by the photographer is displayed. With confirmation means to confirm,
The output means is responsive to the confirmation by the confirmation means being confirmed by the photographer that the photographed image is stored, and the display means outputs the moving image being displayed to the image storage means. Item 14. The photographing device according to any one of Items 7 to 13.
[Claim 15]
While the display control means is displaying on the display means a moving image composed of the captured images for the plurality of frames stored immediately before in the temporary storage means, the presence or absence of an instruction to save the captured image by the photographer is displayed. With confirmation means to confirm,
The output means responds to confirmation by the confirmation means that the photographer saves the captured image, and the display means captures the captured image of the frame in the table in the moving image being displayed by the display means in the image storage means. The imaging apparatus according to claim 7, wherein the imaging apparatus outputs the image.
[Claim 16]
The display control means displays the moving image composed of the captured images for the plurality of frames stored immediately before in the temporary storage means on the display means while the line-of-sight direction is determined by the determination means. 16. The photographing apparatus according to claim 14 or 15, wherein the display of the moving image on the display unit is terminated in response to the determination that the direction is the direction.
[Claim 17]
In parallel with the photographing operation by the photographing means, it comprises a detecting means for detecting the line of sight of the photographer photographed by the photographing means,
The imaging apparatus according to claim 1, wherein the determination unit determines whether or not a line-of-sight direction detected by the detection unit is facing the imaging lens.
[Claim 18]
A photographing method in a photographing apparatus including photographing means including a photographing lens for photographing a subject,
In parallel with the photographing operation by the photographing means, a determination step of determining whether or not the sight line direction of the photographer photographed by the photographing means is facing the photographing lens;
An output step of outputting a photographed image photographed by the photographing means while the line-of-sight direction is determined to be a photographing lens direction by the determination step.
[Claim 19]
A computer having a photographing device including photographing means including a photographing lens for photographing a subject;
In parallel with the photographing operation by the photographing means, a judging means for determining whether or not the sight line direction of the photographer photographed by the photographing means faces the photographing lens;
A program that functions as an output unit that outputs a photographed image photographed by the photographing unit while the line-of-sight direction is determined to be a photographing lens direction by the determining unit.

DESCRIPTION OF SYMBOLS 1 Smart phone 2 Device main body 3 Color liquid crystal display panel 4 Camera part 11 Control part 13 Communication part 14 Imaging part 15 Image memory | storage part 16 Program memory | storage part 17 Frame buffer 18 Display part 19 Operation part G1, G2 Through image Gm Moving image M Recording mark

Claims (19)

Photographing means including a photographing lens for photographing a subject;
In parallel with the photographing operation by the photographing means, a judging means for determining whether or not the sight line direction of the photographer photographed by the photographing means faces the photographing lens;
An imaging apparatus comprising: output means for outputting a photographed image photographed by the photographing means while the line-of-sight direction is judged to be a photographing lens direction by the judging means. Comprising display means for sequentially displaying captured images photographed by the photographing means;
Either the structure in which the photographing lens is arranged in a positional relationship that is separated from the display screen of the display means and the photographer's field of view, or the structure that can be arranged in a positional relationship that is separated in the photographer's field of view as necessary The photographing apparatus according to claim 1, comprising either one of the structures. Image storage means for storing a photographed image;
The imaging apparatus according to claim 2, wherein the output unit outputs the captured image, which is determined by the determination unit to be the photographic lens direction, to the image storage unit.   The imaging apparatus according to claim 3, wherein the output unit outputs the captured image, which is determined by the determination unit as the sight line direction to be a photographing lens direction, as a still image to the image storage unit.   The output means outputs the captured images for a plurality of consecutive frames determined by the determination means as the direction of the photographic lens to the image storage means as a moving image. The imaging device described. Temporary storage means for temporarily storing the photographed image photographed by the photographing means while the line-of-sight direction is judged to be the photographing lens direction by the judging means;
In response to determining that the line-of-sight direction is not the shooting lens direction by the determination unit, the display control unit displays the captured image stored immediately before in the temporary storage unit on the display unit,
The imaging apparatus according to claim 3, wherein the output unit outputs the captured image displayed on the display unit by the display control unit to the image storage unit. The temporary storage means temporarily stores the captured images for a plurality of consecutive frames photographed by the photographing means while the line-of-sight direction is determined to be a photographing lens direction by the determination means,
The photographing apparatus according to claim 6, wherein the display control means causes the display means to display a moving image composed of the photographed images for the plurality of frames stored immediately before in the temporary storage means.   8. The photographing apparatus according to claim 7, wherein the display control means causes the display means to display a moving image composed of a plurality of frames of the photographed images in a predetermined frame period temporarily stored immediately before in the temporary storage means.   The predetermined frame period is a frame period excluding a predetermined frame period at the beginning and the end among frame periods in which the line-of-sight direction is determined to be a photographing lens direction by the determination unit. The imaging device according to claim 8.   10. The display control unit repeatedly displays the moving image on the display unit while the determination unit determines that the line-of-sight direction is not the photographing lens direction. Shooting device.   11. The photographing apparatus according to claim 7, wherein the display control unit causes the display unit to display the moving image instead of the photographed images sequentially photographed by the photographing unit. .   In the display control unit, the captured image displayed on the display unit is either the captured image sequentially captured by the capturing unit or the captured image stored immediately before in the temporary storage unit. The imaging apparatus according to claim 11, wherein an identification mark that clearly indicates is displayed on the display unit together with any one of the captured images.   The display control means is configured to obtain, as the identification mark, the photographed images for a plurality of consecutive frames photographed by the photographing means while the line-of-sight direction is determined to be a photographing lens direction by the determining means. 13. The photographing apparatus according to claim 12, wherein an identification mark indicating that it is being stored in the temporary storage means is displayed on the display means together with the photographed images sequentially photographed by the photographing means. While the display control means is displaying on the display means a moving image composed of the captured images for the plurality of frames stored immediately before in the temporary storage means, the presence or absence of an instruction to save the captured image by the photographer is displayed. With confirmation means to confirm,
The output means is responsive to the confirmation by the confirmation means being confirmed by the photographer that the photographed image is stored, and the display means outputs the moving image being displayed to the image storage means. Item 14. The photographing device according to any one of Items 7 to 13. While the display control means is displaying on the display means a moving image composed of the captured images for the plurality of frames stored immediately before in the temporary storage means, the presence or absence of an instruction to save the captured image by the photographer is displayed. With confirmation means to confirm,
The output means responds to confirmation by the confirmation means that the photographer saves the captured image, and the display means captures the captured image of the frame in the table in the moving image being displayed by the display means in the image storage means. The imaging apparatus according to claim 7, wherein the imaging apparatus outputs the image.   The display control means displays the moving image composed of the captured images for the plurality of frames stored immediately before in the temporary storage means on the display means while the line-of-sight direction is determined by the determination means. 16. The photographing apparatus according to claim 14 or 15, wherein the display of the moving image on the display unit is terminated in response to the determination that the direction is the direction. In parallel with the photographing operation by the photographing means, it comprises a detecting means for detecting the line of sight of the photographer photographed by the photographing means,
The imaging apparatus according to claim 1, wherein the determination unit determines whether or not a line-of-sight direction detected by the detection unit is facing the imaging lens. A photographing method in a photographing apparatus including photographing means including a photographing lens for photographing a subject,
In parallel with the photographing operation by the photographing means, a determination step of determining whether or not the sight line direction of the photographer photographed by the photographing means is facing the photographing lens;
An output step of outputting a photographed image photographed by the photographing means while the line-of-sight direction is determined to be a photographing lens direction by the determination step. A computer having a photographing device including photographing means including a photographing lens for photographing a subject;
In parallel with the photographing operation by the photographing means, a judging means for determining whether or not the sight line direction of the photographer photographed by the photographing means faces the photographing lens;
A program that functions as an output unit that outputs a photographed image photographed by the photographing unit while the line-of-sight direction is determined to be a photographing lens direction by the determining unit.

Images