JP2022086220A - Imaging apparatus, imaging method, and imaging control program - Google Patents

Abstract

To provide an imaging apparatus, an imaging method, and an imaging control program suitable to accurately control the imaging conditions of a subject.SOLUTION: The imaging apparatus comprises: an imaging unit which generates image data by imaging a subject; a display unit which displays an image indicated by the image data; a detection unit which detects an eye movement of a user; an identifying unit which identifies an area having been watched for a prescribed period or more by the user, out of areas of the image displayed by the display unit; and a control unit which controls the imaging conditions of the subject included in the image of the area identified by the identifying unit, in accordance with a prescribed eye movement of the user detected by the detection unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image pickup device, an image pickup method, and an image pickup control program, and relates to an image pickup device, an image pickup method, and an image pickup control program suitable for accurately controlling the image pickup conditions of a desired subject.

Patent Document 1 describes a first camera that captures an area to be monitored, a display device that displays an image captured by the first camera, and a second camera that captures a user who views the image displayed on the display device. Camera, a detection means for detecting the movement of the user's eyes based on the image taken by the second camera, and a control means for controlling the mode of shooting by the first camera according to the movement of the user's eyes. A video surveillance system equipped with the above is disclosed.

Japanese Patent No. 6276954

However, when the structure of Patent Document 1 is applied to a camera with a monitor for displaying a subject image (including a camera mounted on a mobile terminal such as a smartphone), the user can use the subject image on the monitor due to the influence of camera shake. It is not possible to specify whether you are looking at the image, and there is a possibility that an unintended subject will be selected as the control target for the shooting mode. In other words, there is a problem that the related technique cannot accurately control the imaging conditions of a desired subject.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide an image pickup device, an image pickup method, and an image pickup control program suitable for accurately controlling the image pickup conditions of a desired subject. ..

The image pickup device according to one aspect of the present embodiment includes an image pickup unit that captures an image of a subject and generates image data, a display unit that displays an image indicated by the image data, and a detection unit that detects the movement of the user's eyes. Of the area of the image displayed on the display unit, a specific portion that specifies a region that is stared at by the user for a predetermined period or longer, and an image of a subject included in the image of the region specified by the specific portion. A control unit that controls conditions according to a predetermined eye movement of the user detected by the detection unit is provided.

In the imaging method according to one aspect of the present embodiment, a subject is imaged to generate image data, the image indicated by the image data is displayed on the display unit, the movement of the user's eyes is detected, and the image data is displayed on the display unit. Of the areas of the image to be imaged, a region that is stared at by the user for a predetermined period or longer is specified, and the imaging condition of the subject included in the image of the specified region is set to the predetermined movement of the eyes of the user. Control accordingly.

The image pickup control program according to one aspect of the present embodiment is a process of capturing an image of a subject to generate image data, a process of displaying an image indicated by the image data on a display unit, and a process of detecting the movement of the user's eyes. The process of specifying the area of the image displayed on the display unit, which is the area stared at by the user for a predetermined period or longer, and the imaging conditions of the subject included in the image of the specified area. The computer is made to execute the process of controlling according to the predetermined movement of the eyes of the user.

According to the present embodiment, it is possible to provide an image pickup device, an image pickup method, and an image pickup control program suitable for accurately controlling the image pickup conditions of a desired subject.

It is a block diagram which shows the structural example of the image pickup apparatus which concerns on Embodiment 1. FIG. It is a flowchart which shows an example of the operation of the image pickup apparatus shown in FIG. It is a figure which shows an example of the image displayed on the display part mounted on the image pickup apparatus shown in FIG. 1. It is a figure which shows an example of the image displayed on the display part mounted on the image pickup apparatus shown in FIG. 1. It is a flowchart which shows the other example of the operation of the image pickup apparatus shown in FIG. It is a block diagram which shows the hardware configuration example of the image pickup control function of the image pickup apparatus which concerns on Embodiment 1. FIG.

Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. In each drawing, the same elements are designated by the same reference numerals, and duplicate explanations will be omitted as necessary for the sake of clarity of explanation.

<Embodiment 1>
FIG. 1 is a block diagram showing a configuration example of the image pickup apparatus 1 according to the first embodiment. The image pickup device 1 is a camera with a monitor (including a camera mounted on a mobile terminal such as a smartphone or a tablet) for displaying an image of the subject to be captured, and is stared at by a user (imaging person) for a predetermined period of time. The imaging conditions of the subject included in the image in the monitor area are controlled according to the predetermined movement of the eyes. As a result, the image pickup apparatus 1 can prevent the subject unintended by the user from being selected as the control target of the image pickup condition due to the influence of camera shake. Further, the image pickup apparatus 1 can control the image pickup condition of a desired subject without looking away from the display unit by controlling the image pickup condition by the movement of the eyes. That is, the image pickup apparatus 1 according to the first embodiment can accurately control the image pickup conditions of a desired subject without being affected by camera shake. Hereinafter, a specific description will be given.

As shown in FIG. 1, the image pickup apparatus 1 includes an image pickup unit 11, a display unit 12, a detection unit 13, a specific unit 14, and a control unit 15.

The image pickup unit 11 is a unit that captures an image of a subject and generates image data. Specifically, the image pickup unit 11 includes at least an optical system 111, an image pickup element 112, and an image processing unit 113.

The optical system 111 includes a focus lens, a zoom lens, an aperture, a shutter, and the like.

The image pickup device 112 is, for example, a CMOS image sensor, a CCD image sensor, or the like, and obtains an image of a subject formed via an optical system 111 to generate image pickup data. Here, the image sensor 112 generates image data captured by a user or an automatic shutter operation.

The image processing unit 113 performs predetermined processing on the image pickup data generated by the image pickup element 112 to generate image data. The image data generated by the image processing unit 113 is stored in a storage unit (not shown).

The display unit (monitor) 12 displays the image indicated by the image data generated by the image processing unit 113 or the image data stored in a storage unit (not shown).

The detection unit 13 detects the movement of the eyes of the user who is looking at the display unit 12. For example, the detection unit 13 is a camera mounted on the image pickup apparatus 1 so that the user who is looking at the display unit 12 can take a picture, and detects the movement of the user's eyes from the image taken by the camera.

The specific unit 14 specifies an area of the image displayed on the display unit 12 (monitor area) that is stared at by the user for a predetermined period (for example, 3 seconds) or longer as a control target area of the imaging condition. As a result, it is possible to prevent a region not intended by the user from being specified as a controlled target region of the imaging condition due to the influence of camera shake. The specific unit 14 estimates the region stared by the user from, for example, the movement of the eyes (line of sight) detected by the detection unit 13.

The control unit 15 controls the image pickup apparatus 1 in general. Further, the control unit 15 detects the imaging conditions of the subject included in the image in the area specified by the specific unit 14 in the image area displayed on the display unit 12 by the detection unit 13. Control according to the movement of. As a result, the user can control the imaging conditions of the desired subject without looking away from the monitor.

As described above, in the image pickup apparatus 1 according to the present embodiment, by setting the target stared at for a predetermined period or longer as the control target of the image pickup condition, the subject unintended by the user due to the influence of camera shake is selected as the control target of the image pickup condition. Can be prevented. Further, the image pickup apparatus 1 according to the present embodiment can control the image pickup condition of a desired subject without losing the line of sight from the display unit by controlling the image pickup condition by the movement of the eyes. That is, the image pickup apparatus 1 according to the first embodiment can accurately control the image pickup conditions of a desired subject without being affected by camera shake.

(flowchart)
Subsequently, the operation of the image pickup apparatus 1 will be described with reference to FIGS. 2 to 4.
FIG. 2 is a flowchart showing an example of the operation of the image pickup apparatus 1. 3 and 4 are diagrams showing an example of an image displayed on the display unit 12 mounted on the image pickup apparatus 1. Note that FIG. 3 shows an image of the display unit 12 before controlling the imaging conditions, and FIG. 4 shows an image of the display unit 12 after controlling the imaging conditions.

First, the user (imaging person) holds the image pickup apparatus 1 toward the subject (step S101). At this time, the image of the subject is displayed on the display unit 12 mounted on the image pickup apparatus 1. In the example of FIG. 3, an image including the subject A is displayed on the display unit 12.

After that, the user stares at the image of the desired subject among the images displayed on the display unit 12 (step S102). In the example of FIG. 3, the user stares at the image of the subject A. The specific unit 14 estimates the region stared by the user from, for example, the movement of the eyes (line of sight) detected by the detection unit 13.

When the user continues to stare at the desired subject for a predetermined period or longer, the specifying unit 14 specifies the region including the desired subject as the controlled target region of the imaging condition (YES → S103 in step S102). In other words, the specific unit 14 locks the gaze target. The locked area may be highlighted, for example, surrounded by a white frame.

In the example of FIG. 3, since the user keeps staring at the subject A for a predetermined period or longer, the specifying unit 14 specifies the region B including the subject A as the controlled target region of the imaging condition. In other words, the specific unit 14 locks the area B including the subject A.

If the user's gaze period is less than a predetermined period, the gaze target is not locked (NO in step S102).

After that, the imaging conditions of the locked region are controlled according to the movement of the user's eyes detected by the detection unit 13 (step S104). It should be noted that how to control the imaging conditions by the movement of the eyes is predetermined, but it can be changed by the user, and the number can be freely increased. User settings are summarized in a correspondence table and stored in a storage device (not shown).

For example, when the user moves his / her eyes up and down, the control unit 15 controls the focus of the locked region by using the focus lens provided in the optical system 111 (YES → S105 in step S104). Further, when the user moves his / her eyes left and right, the control unit 15 controls the zoom of the locked area by using the zoom lens provided in the optical system 111 (YES → S106 in step S104). Further, when the user rotates his / her eyes, the control unit 15 performs aperture control (brightness control) in the locked area by using the aperture provided in the optical system 111 (YES → S107 in step S104). ). Thereby, the user can control the imaging condition of the locked area without taking the line of sight from the monitor.

In the example of FIG. 4, the user moves his / her eyes left and right. As a result, the zoom control of the area B including the subject A is performed (that is, the area B including the subject A is enlarged).

If the user does not move his / her eyes, the imaging condition of the locked area is not controlled (NO in step S104).

After that, when the image is taken by the shutter operation, the lock of the gaze target is released (YES → S109 in step S108), but when the shutter is not operated, the lock is not released and the lock is released again. , The imaging condition of the locked region is controlled according to the movement of the user's eyes (NO in step S108). In this example, the lock is released when the shutter is operated, but the present invention is not limited to this. The setting may be changed so that the lock is released by another operation (for example, an operation of removing the line of sight from the monitor) instead of the shutter operation. This makes it possible to unlock the lock without taking a picture.

After that, when the power is turned off, the operation of the image pickup apparatus 1 ends (YES in step S110). If the power is not turned off, the process returns to the process of step S102 (NO in step S110).

As described above, in the image pickup apparatus 1 according to the present embodiment, by setting the target stared at for a predetermined period or longer as the control target of the image pickup condition, the subject unintended by the user due to the influence of camera shake is selected as the control target of the image pickup condition. Can be prevented. Further, the image pickup apparatus 1 according to the present embodiment can control the image pickup condition of a desired subject without losing the line of sight from the display unit by controlling the image pickup condition by the movement of the eyes. That is, the image pickup apparatus 1 according to the first embodiment can accurately control the image pickup conditions of a desired subject without being affected by camera shake.

In the present embodiment, the case where the lock is released each time the imaging condition is controlled has been described as an example, but the present invention is not limited to this. The image pickup apparatus 1 may be configured so that the lock is not released until the user's line of sight deviates from the display unit 12. Hereinafter, a brief description will be given with reference to FIG.

FIG. 5 is a flowchart showing another example of the operation of the image pickup apparatus 1.
In FIG. 5, after the imaging conditions are controlled, it is determined whether or not the user's line of sight deviates from the display unit 12 (monitor) (step S208). This determination is made, for example, based on the movement of the user's eyes detected by the detection unit 13.

If the user's line of sight deviates from the display unit 12, the lock is released (YES → S109 in step S208), but if the user's line of sight does not deviate from the display unit 12, the lock is released. Instead, the imaging condition of the locked region is controlled again according to the movement of the user's eyes (NO in step S208). Therefore, the user can continuously image the desired subject while changing the imaging conditions without looking away from the desired subject. The continuous imaging may be manual or automatic by the user.

Since the other operations shown in the flowchart of FIG. 5 are the same as the operations shown in the flowchart of FIG. 3, the description thereof will be omitted.

(Hardware configuration of the image pickup control function of the image pickup device 1)
The image pickup control process realized by the image pickup apparatus 1 can be realized by a general-purpose computer system. Hereinafter, a brief description will be given with reference to FIG.

FIG. 6 is a block diagram showing an example of the hardware configuration of the image pickup control function. The computer 100 includes, for example, a CPU (Central Processing Unit) 101, which is a control device, a RAM (Random Access Memory) 102, and a ROM (Read Only Memory) 103. The computer 100 further includes an IF (Inter Face) 104, which is an interface with the outside, and an HDD (Hard Disk Drive) 105, which is an example of a non-volatile storage device. Further, the computer 100 may be provided with an input device such as a keyboard and a mouse and a display device such as a display as other configurations (not shown).

The HDD 105 stores an OS (Operating System) (not shown) and an image pickup control program 106. The image pickup control program 106 is a computer program in which the image pickup control process according to the present embodiment is implemented.

The CPU 101 controls various processes in the computer 100, access to the RAM 102, ROM 103, IF 104, HDD 105, and the like. The computer 100 reads and executes the OS and the image pickup control program 106 stored in the HDD 105 by the CPU 101. As a result, the computer 100 realizes the image pickup control function of the image pickup apparatus 1 according to the present embodiment.

Although the invention made by the present inventor has been specifically described above based on the embodiments, the present invention is not limited to the embodiments already described, and various changes can be made without departing from the gist thereof. It goes without saying that it is possible.

In the above-described embodiment, the present invention has been described as a hardware configuration, but the present invention is not limited thereto. The present invention can also realize arbitrary processing by causing a CPU (Central Processing Unit) to execute a computer program. In addition, the above-mentioned programs can be stored and supplied to a computer using various types of non-transitory computer readable media. Non-temporary computer-readable media include various types of tangible storage media. Examples of non-temporary computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), optomagnetic recording media (eg, optomagnetic disks), CD-ROMs (Read Only Memory) CD-Rs, CDs. -R / W, including semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)). The program may also be supplied to the computer by various types of temporary computer readable media. Examples of temporary computer readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

Although the invention of the present application has been described above with reference to the embodiments, the invention of the present application is not limited to the above. Various changes that can be understood by those skilled in the art can be made within the scope of the invention in the configuration and details of the invention of the present application.

1 Imaging device 11 Imaging unit 12 Display unit 13 Detection unit 14 Specific unit 15 Control unit 100 Computer 101 CPU
102 RAM
103 ROM
104 IF
105 HDD
106 Image control program 111 Optical system 112 Image sensor 113 Image processing unit

Claims (5)

An image pickup unit that captures an image of a subject and generates image data,
A display unit that displays the image indicated by the image data, and
A detector that detects the movement of the user's eyes and
A specific unit that specifies an area of the image displayed on the display unit that is stared at by the user for a predetermined period or longer.
A control unit that controls the imaging conditions of the subject included in the image of the region specified by the specific unit according to the predetermined eye movement of the user detected by the detection unit.
An image pickup device equipped with. The specific unit is configured to infer a region stared by the user from the movement of the user's eyes detected by the detection unit.
The imaging device according to claim 1. The imaging unit includes a focus lens, a zoom lens, and an aperture.
The control unit controls at least one of a focus lens, a zoom lens, and an aperture according to a predetermined eye movement of the user.
The imaging device according to claim 1 or 2. Image the subject and generate image data,
The image indicated by the image data is displayed on the display unit, and the image is displayed on the display unit.
Detects user eye movements and
Among the areas of the image displayed on the display unit, the area stared at by the user for a predetermined period or longer is specified.
The imaging condition of the subject included in the image of the specified area is controlled according to the predetermined movement of the eyes of the user.
Imaging method. The process of capturing an image of a subject and generating image data,
The process of displaying the image indicated by the image data on the display unit,
The process of detecting the movement of the user's eyes and
A process of specifying an area of the image displayed on the display unit, which is the area stared at by the user for a predetermined period or longer.
A process of controlling the imaging conditions of the subject included in the image of the specified area according to the predetermined movement of the eyes of the user.
An imaging control program that causes a computer to execute.

Images