JP7387091B1 - Photography method, photography equipment and program - Google Patents

Abstract

[Problem] To enable a photographer to appropriately photograph an object to be photographed without looking at the screen of a photographing device. A photographing method according to an embodiment is a photographing method executed by a photographing device equipped with a camera, and includes an acquisition process of acquiring an image from the camera, and a detection process of detecting a target object from the image. , when the target object detected by the detection process is included in a first area of the image, a first pattern is notified, and the target object detected by the detection process is included in a second area of the image. notification processing of notifying in a second pattern different from the first pattern when the object is included in the first area or the second area; and photographing by the camera at predetermined time intervals while the object is included in the first area or the second area. and photographing processing. [Selection diagram] Figure 8

Description

The present invention relates to a photographing method, a photographing device, and a program.

In recent years, services that provide parents with photos of their children in the nursery schools and kindergartens have become popular. With such services, children may be photographed using cameras installed within the park, or staff members may photograph children using cameras such as smartphones. For example, cited document 1 discloses a technique for photographing children in a kindergarten using a surveillance camera.

JP 2022-71553 Publication

When staff take pictures of children with a camera such as a smartphone, they usually do so while looking at the smartphone screen. However, if staff members stare at smartphone screens, there is a risk that they will not be paying enough attention to children. On the other hand, if staff members take pictures without looking at the smartphone screen, it becomes difficult to take pictures that clearly show the children.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to enable a photographer to appropriately photograph a subject without looking at the screen of a photographing device.

A photographing method according to an embodiment is a photographing method executed by a photographing device equipped with a camera, and includes an acquisition process of acquiring an image from the camera, a detection process of detecting a target object from the image, and the detection process. A first pattern is used to notify when the target object detected by the process is included in a first area of the image, and when the target object detected by the detection process is included in a second area of the image. a notification process for notifying using a second pattern different from the first pattern; and a photographing process for photographing with the camera at predetermined time intervals while the object is included in the first area or the second area. ,including.

According to one embodiment, the photographer can appropriately photograph the object without looking at the screen of the photographing device.

3 is a diagram illustrating how to use the photographing device 1. FIG. FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing device. 1 is a diagram showing an example of a functional configuration of a photographing device 1. FIG. 2 is a flowchart illustrating an example of processing executed by the photographing device 1. FIG. FIG. 2 is a schematic diagram illustrating processing executed by the photographing device 1. FIG. 1 is a diagram showing an example of a functional configuration of a photographing device 1. FIG. 2 is a flowchart illustrating an example of processing executed by the photographing device 1. FIG. FIG. 2 is a schematic diagram illustrating processing executed by the photographing device 1. FIG.

Hereinafter, each embodiment of the present invention will be described with reference to the accompanying drawings. Note that in the descriptions of the specifications and drawings related to each embodiment, the same reference numerals are given to the constituent elements having substantially the same functional configuration to omit redundant explanation.

[First embodiment]
<Overview of photographing device 1>
First, an overview of the photographing device 1 according to the first embodiment will be explained. The photographing device 1 is an information processing device equipped with a camera, and is used by a user (photographer) to photograph a subject. The photographing device 1 is, for example, a smartphone, a tablet terminal, or a digital camera, but is not limited thereto. The photographing device 1 may be any information processing device equipped with a camera. The subject to be photographed may be a person or an animal, but is not limited thereto.

FIG. 1 is a diagram illustrating how to use the photographing device 1. As shown in FIG. In this embodiment, it is assumed that the user photographs the subject with the camera of the photographing device 1 while looking toward the subject. That is, when photographing a subject, the user does not look at the screen of the photographing device 1 that displays the image being photographed. Therefore, when photographing an object, the user points the camera toward the object, but does not know whether the object is actually being photographed with the camera.

Therefore, in this embodiment, the photographing device 1 determines whether the photographing target is included in the image based on the image acquired from the camera, and automatically photographs the photographing target if the photographing target is included. Thereby, the user can photograph the subject by simply pointing the camera roughly toward the subject without looking at the screen of the photographing device 1.

Further, the photographing device 1 notifies the user of the position of the object to be photographed in the image acquired from the camera using sound or vibration. Thereby, the user can grasp whether an image is being captured in which the subject is in a desired position (for example, in the center of the image) without looking at the screen of the photographing device 1.

In this way, according to the photographing device 1, the user can photograph an image in which the photographic subject is in a desired position without looking at the screen of the photographing device 1. As a result, the user can appropriately photograph the object while looking toward the object.

This photographing device 1 can be used, for example, in a daycare center or kindergarten where a staff member (photographer) takes a picture of a child while looking at the child, or at a nursing facility where a staff member (photographer) takes a photograph while looking at the cared person. This can be suitably used when taking a picture of a caregiver, or when a guardian (photographer) takes a picture of a child while looking at the child at the park.

<Hardware configuration of information processing device>
Next, the hardware configuration of the information processing device 100 will be explained. FIG. 2 is a diagram illustrating an example of the hardware configuration of the information processing device 100. As shown in FIG. 2, the information processing device 100 includes a processor 101, a memory 102, a storage 103, a communication I/F 104, an input/output I/F 105, and a drive, which are interconnected via a bus B. A device 106 is provided.

The processor 101 controls each configuration of the information processing device 100 and operates the functions of the information processing device 100 by loading various programs including an OS (Operating System) stored in the storage 103 into the memory 102 and executing them. Realize. The processor 101 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), or a combination thereof.

The memory 102 is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), or a combination thereof. The ROM is, for example, a PROM (Programmable ROM), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), or a combination thereof. The RAM is, for example, DRAM (Dynamic RAM), SRAM (Static RAM), or a combination thereof.

The storage 103 stores various programs and data including an OS. The storage 103 is, for example, a flash memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an SCM (Storage Class Memories), or a combination thereof.

The communication I/F 104 is an interface for connecting the information processing device 100 to an external device via the network N and controlling communication. The communication I/F 104 is, for example, an adapter compliant with Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark), Ethernet (registered trademark), or optical communication, but is not limited thereto. .

The input/output I/F 105 is an interface for connecting the input device 107 and the output device 108 to the information processing device 100. The input device 107 is, for example, a mouse, keyboard, touch panel, microphone, scanner, camera, various sensors, operation buttons, or a combination thereof. Output device 108 is, for example, a display, a projector, a printer, a speaker, a vibrator, or a combination thereof.

The drive device 106 reads and writes data on the recording medium 109. The drive device 106 is, for example, a magnetic disk drive, an optical disk drive, a magneto-optical disk drive, or an SD card reader. The recording media 109 is, for example, a CD (Compact Disc), a DVD (Digital Versatile Disc), an FD (Floppy Disk), an MO (Magneto-Optical disk), a BD (Blu-ray (registered trademark) Disc), or a USB (registered trademark). ) memory, SD card, or a combination thereof.

Note that in this embodiment, the program may be written into the memory 102 or the storage 103 during the manufacturing stage of the information processing device 100, may be provided to the information processing device 100 via the network N, or may be provided to the information processing device 100 via the recording medium. The information may be provided to the information processing apparatus 100 via a non-transitory computer-readable recording medium such as 109.

<Functional configuration of photographing device 1>
Next, the functional configuration of the photographing device 1 will be explained. FIG. 3 is a diagram showing an example of the functional configuration of the photographing device 1. As shown in FIG. As shown in FIG. 3, the photographing device 1 includes a communication section 11, a storage section 12, and a control section 13.

The communication unit 11 is realized by the communication I/F 104. The communication unit 11 transmits and receives information to and from an external device via the network. The network is, for example, a wired LAN (Local Area Network), a wireless LAN, the Internet, a public network, a mobile data communication network, or a combination thereof.

The storage unit 12 is realized by a memory 102 and a storage 103. The storage unit 12 stores image information 121 and photographed image information 122.

Image information 121 is information regarding the image im acquired by the camera. The image information 121 includes information such as image im, shooting date and time, and camera settings. The image information 121 is temporarily stored in the storage unit 12.

The photographed image information 122 is information regarding the image im photographed by the camera. Photographing here means storing the image im acquired by the camera as a photographed image. The photographed image information 122 includes information such as image im, photographing date and time, camera settings, and detection results by the detection unit 132.

The control unit 13 is realized by the processor 101 reading a program from the memory 102, executing it, and cooperating with other hardware configurations. The control unit 13 controls the entire operation of the photographing device 1. The control section 13 includes an acquisition section 131, a detection section 132, a notification section 133, and a photographing section 134.

The acquisition unit 131 acquires the image im from the camera and stores image information 121 including the image im in the storage unit 12. The image im may be a still image or a moving image.

The detection unit 132 detects the target object t from the image im stored in the storage unit 12. The target object t is a part or all of the object to be photographed, and is set depending on the object to be photographed. The target object t is, for example, a person, an animal, or a face thereof, but is not limited thereto. The detection unit 132 detects the target object t from the image im, for example, using a machine learning model learned to detect the target object t from the image. The machine learning model is, for example, a CNN (Convolutional Neural Network), but is not limited to this. The detection unit 132 can use any existing method capable of detecting a detection target.

The notification unit 133 controls the output device 108 (vibrator and speaker) and notifies the user according to the detection result of the target object t by the detection unit 132. The notification method will be described later.

The photographing unit 134 controls the camera and executes photographing. Specifically, the photographing unit 134 activates the camera and stores a part of the image im stored in the storage unit 12 in the storage unit 12 as a photographed image.

Note that the functional configuration of the photographing device 1 is not limited to the above example. For example, the photographing device 1 may include a part of the above functional configuration, and the user terminal 2 may include the rest. Further, the photographing device 1 may have functional configurations other than those described above. Further, as described above, each functional configuration of the imaging device 1 may be realized by software, or by hardware such as an IC chip, SoC (System on Chip), LSI (Large Scale Integration), or microcomputer. You can.

<Processing executed by photographing device 1>
Next, the processing executed by the photographing device 1 will be explained. FIG. 4 is a flowchart illustrating an example of a process executed by the photographing device 1. FIG. 5 is a schematic diagram illustrating processing executed by the photographing device 1. FIG. 5 shows a series of images im acquired over time from a camera, in which the detection target is a person.

When the user operates the photographing device 1 to cause the photographing device 1 to start processing, the processing from step S101 to step S113 continues in a predetermined manner until the user operates the photographing device 1 and causes the photographing device 1 to complete the process. Runs at time intervals.

(Step S101)
First, when the photographing device 1 starts processing, the photographing unit 134 starts the camera, and the acquiring unit 131 starts acquiring the image im from the camera (step S101). For example, the acquisition unit 131 acquires one image im from a camera every second, and temporarily stores it in the storage unit 12 as image information 121 in association with the shooting date and time. The time interval at which the acquisition unit 131 acquires the image im from the camera can be set arbitrarily.

(Step S102)
When the new image im is stored in the storage unit 12, the detection unit 132 performs a process of detecting the object t on the stored image im, and detects the object t from the image im (step S102). . When the image im includes a plurality of objects t, the plurality of objects t are detected.

(Step S103)
The detection unit 132 checks whether the target object t has been detected from the image im (step S103). If the target object t is not detected (step S103: NO), the image im is deleted and the process proceeds to step S111. If the target object t is detected (step S103: YES), the process proceeds to step S104.

In the example of FIG. 5, if an image im1 without a person is acquired, the process proceeds to step S111, and if an image im2 including a person is acquired, the process proceeds to step S104.

(Step S104)
The notification unit 133 determines whether the target object t detected from the image im is included in the first region r1 of the image im (step S104). Here, the first region r1 and the second region r2 will be explained.

The first area r1 is an area on the image im that is set to be photographed while the object t is located there, and is set as a range of positions preferable as the position of the object t in the image im. This is an area where The first region r1 is, for example, a region including the center of the image im (center region), but is not limited thereto. The image im in which the object t is included in the first region r1 is an image im obtained by appropriately photographing the object t.

The first region r1 may be a rectangular region whose vertical and horizontal lengths are both shorter than the outline of the image im (see FIG. 5), a rectangular region that cuts or crosses the center of the image im, or a circular or elliptical region. It can also be in the area of form. The shape and size of the first region r1 can be set arbitrarily.

The second area r2 is an area on the image im that is set to be photographed while the object t is located there, and is an area other than the first area r1. The second area r2 may be the entire area other than the first area r1 in the image im, or may be a part of the image im. The image im in which the object t is included in the second region r2 is an image im obtained by photographing the object t.

The second region r2 may be, for example, a rectangular region surrounding the first region r1, which is a rectangle whose length and width are both shorter than the outline of the image im, or a rectangular region that extends vertically or crosses the center of the image im. It may be a two-divided area sandwiching one area r1 left and right or top and bottom, or it may be a circular or elliptical area surrounding a circular or elliptical first area r1. The shape and size of the second region r2 can be set arbitrarily.

Note that, in this embodiment, the second region r2 is the entire area other than the first region r1. Therefore, when the target object t detected from the image im is not included in the first region r1, the target object t is included in the second region r2.

The notification unit 133 may determine that the object t is included in the first region r1 of the image im when the entire object t is included in the first region r1, or the center of gravity of the object t is in the first region r1. If the object t is included in the region r1, it may be determined that the object t is included in the first region r1 of the image im. When a plurality of objects t are detected from the image im, if even one object t is included in the first region r1, it is determined that the object t is included in the first region r1 of the image im. You can.

If the target object t is included in the first region r1 (step S104: YES), the process proceeds to step S105. If the target object t is not included in the first region r1 (included in the second region r2) (step S104: NO), the process proceeds to step S106.

In the example of FIG. 5, if an image im3 in which a person is located inside the first region r1 is acquired, the process proceeds to step S105, and if an image im2 in which a person is located outside the first region r1 is acquired, the process The process proceeds to step S106. In the example of FIG. 5, it is assumed that it is determined whether the person is included in the first region r1 based on the position of the center of gravity of the bounding box b1 corresponding to the detected person.

(Step S105)
When the target object t is included in the first region r1, the notification unit 133 controls the output device 108 to notify the user in the first pattern (step S105). For example, the notification unit 133 notifies the user that the object t is included in the first region r1 by vibrating the vibrator in the first pattern. Furthermore, the notification unit 133 may notify the user that the target object t is included in the first region r1 by outputting the first pattern of sound from the speaker. In this way, by notifying the user with vibration or sound, the user can know that the object t is included in the first region r1 without looking at the screen of the photographing device 1. In the example of FIG. 5, when the image im3 in which a person is included in the first region r1 is acquired, the user is notified of this in the first pattern.

(Step S106)
On the other hand, if the object t is not included in the first region r1 (the object t is included in the second region r2), the notification unit 133 controls the output device 108 to notify the user in a second pattern. (Step S106). The second pattern is an arbitrary pattern different from the first pattern. For example, the notification unit 133 notifies the user that the object t is included in the second region r2 by vibrating the vibrator in the second pattern. Furthermore, the notification unit 133 may notify the user that the target object t is included in the second region r2 by outputting the second pattern of sound from the speaker. In this way, by notifying the user with vibration or sound, the user can confirm that the object t is not included in the first region r1 (or that it is included in the second region r2) without looking at the screen of the photographing device 1. You can know that there are In the example of FIG. 5, when an image im2 in which a person is not included in the first region r1 is acquired, the user is notified of this using the second pattern.

(Step S107)
Thereafter, the photographing unit 134 checks whether currently photographing is in progress, that is, whether photographing has already started (step S107). If photographing is not yet in progress (step S107: NO), the process advances to step S109. If shooting is already in progress (step S107: YES), the process advances to step S108.

(Step S108)
If photographing is already in progress, the photographing unit 134 photographs the object t (step S108). Specifically, the photographing unit 134 stores the image im in the storage unit 12 as a photographed image. The photographed image is stored as photographed image information 122 in association with the date and time of photographing. After that, the process returns to step S101.

(Step S109)
If photographing is not yet in progress, the photographing unit 134 photographs the object t and starts photographing (step S109). The photographing unit 134 stores the image im in the storage unit 12 as a photographed image. The photographed image is stored as photographed image information 122 in association with the date and time of photographing. Thereafter, in step S107, it is determined that shooting is in progress. In the example of FIG. 5, when the image im3 is acquired, it is determined that the image is being photographed, and the image im3 is saved as the photographed image. The photographed image is stored as photographed image information 122 in association with the date and time of photographing.

(Step S110)
Thereafter, the notification unit 133 controls the output device 108 to notify the user that imaging has started. The notification unit 133 notifies the user of the start of shooting, for example, by outputting a sound from a speaker indicating that shooting has started. After that, the process returns to step S101. This allows the user to know that photography has started without looking at the photography device 1. Note that the notification of the start of shooting may be omitted.

In the present embodiment, while the object t is included in the first region r1 or the second region r2, the camera performs photographing at predetermined time intervals (for example, at 1-second intervals). The start of imaging is notified at the beginning of the period during which imaging is performed (at the start of imaging). In the example of FIG. 5, when image im2 is acquired, photographing is started, image im2 is saved as a photographed image, and the user is notified of the start of photographing.

Here, the process returns to step S103. In step S103, the detection unit 132 checks whether the target object t has been detected from the image im. If the target object t is not detected (step S103: NO), the image im is deleted and the process proceeds to step S111.

(Step S111)
If the target object t is not detected, the photographing unit 134 checks whether photographing is currently in progress, that is, whether photographing has already started (step S111). If the image im is not yet being photographed (step S111: NO), the image im is deleted and the process returns to step S101. If photographing is already in progress (step S111: YES), the process advances to step S112.

(Step S112)
If photographing is already in progress, the photographing unit 134 ends the photographing (step S112). Specifically, the photographing unit 134 deletes the image im.

(Step S113)
Thereafter, the notification unit 133 controls the output device 108 to notify the user that the shooting has ended. The notification unit 133 notifies the user of the end of shooting, for example, by outputting a sound from a speaker indicating that shooting has ended. After that, the process returns to step S101. Thereby, the user can know that the shooting has ended without looking at the shooting device 1. Note that the notification of completion of photographing may be omitted.

In the present embodiment, while the object t is included in the first region r1 or the second region r2, the camera performs photographing at predetermined time intervals (for example, at 1-second intervals). The end of photography is notified at the end of the period during which photography is performed (at the end of photography). In the example of FIG. 5, when image im6 is acquired, imaging ends, image im6 is deleted, and the user is notified of the end of imaging.

Here, the processing when the image im in FIG. 5 is acquired from a camera will be summarized. It is assumed that imaging has not yet started at the time when image im1 is acquired.

When image im1 is acquired (step S101), since no person is detected (step S103: NO) and the image is not being photographed (step S111: NO), image im1 is deleted.

When the image im2 is acquired (step S101), a person is detected (step S103: YES), the person is not included in the first region r1 (step S104: NO), and the image is not being photographed yet (step S107). : NO), the photographing device 1 vibrates in the second pattern (step S106), photographing is started (step S109), image im2 is saved as a photographed image, and a sound indicating the start of photographing is output (step S110). ). Thereby, the user can know that the photographing of the image im in which a person appears in the second region r2 has started without looking at the screen of the photographing device 1.

When images im3 and im4 are acquired (step S101), a person is detected (step S103: YES), the person is included in the first region r1 (step S104: YES), and the image is already being photographed ( Step S107: YES), the photographing device 1 vibrates in the first pattern (step S105), and images im3 and im4 are saved as photographed images (step S108). Thereby, the user can know that the image im in which a person appears in the first region r1 has been captured without looking at the screen of the photographing device 1.

When the image im5 is acquired (step S101), a person is detected (step S103: YES), the person is included in the second region r2 (step S104: NO), and the image is already being photographed (step S107). :YES), the photographing device 1 vibrates in the second pattern (step S106), and the image im5 is saved as a photographed image (step S108). Thereby, the user can know that the image im in which a person appears in the second region r2 has been captured without looking at the screen of the imaging device 1.

When image im6 is acquired (step S101), since no person is detected (step S103: NO) and photography is already in progress (step S111: YES), photography ends (step S112), and image im6 is acquired (step S101). is deleted, and a sound indicating the end of photography is output (step S113). Thereby, the user can know that the photographing of the image im in which the person has been captured has been completed without looking at the screen of the photographing device 1.

<Summary>
As explained above, the photographing device 1 acquires the image im from the camera, detects the target object t from the image im, and when the detected target object t is included in the first region r1 of the image im, the first When the detected object t is included in the second region r2 of the image im, a second pattern different from the first pattern is used. While included in r2, images are taken by the camera at predetermined time intervals.

Depending on whether or not there are notifications of the first pattern and the second pattern, the user (photographer) can know whether or not the object t is being photographed. Therefore, the user can photograph the object t without looking at the screen of the photographing device 1 by simply adjusting the direction of the photographing device 1 (camera) so that the first pattern or the second pattern is notified. can.

Furthermore, depending on whether notification of the first pattern or the second pattern is performed, the user can know where the object t being photographed is shown in the image im. Therefore, the user can photograph the object t at an appropriate position without looking at the screen of the photographing device 1 by simply adjusting the orientation of the photographing device 1 (camera) so that the first pattern notification is performed. can.

[Second embodiment]
<Overview of photographing device 1>
Next, an overview of the photographing device 1 according to the second embodiment will be explained. The photographing device 1 according to the present embodiment detects a person and a face from the image im, and notifies the user according to the detection results. Hereinafter, differences from the first embodiment will be mainly explained.

<Functional configuration of photographing device 1>
FIG. 6 is a diagram showing an example of the functional configuration of the photographing device 1. As shown in FIG. As shown in FIG. 6, the photographing device 1 includes a person detection section 132A and a face detection section 132B instead of the detection section 132. The notification unit 133 notifies the user according to the detection results of the person detection unit 132A and the face detection unit 132B. Other functional configurations are the same as in the first embodiment.

The person detection unit 132A detects a person from the image im stored in the storage unit 12. The person detection unit 132A detects a person from the image im, for example, using a machine learning model learned to detect a person from an image. The machine learning model is, for example, CNN, but is not limited to this. The person detection unit 132A can use any existing method capable of detecting a person.

The face detection unit 132B detects a human face from the image im stored in the storage unit 12. The face detection unit 132B detects a face from the image im, for example, using a machine learning model trained to detect faces from images. The machine learning model is, for example, CNN, but is not limited to this. The face detection unit 132B can use any existing method capable of detecting a face.

<Processing executed by photographing device 1>
Next, the processing executed by the photographing device 1 will be explained. FIG. 7 is a flowchart illustrating an example of a process executed by the photographing device 1. FIG. 8 is a schematic diagram illustrating the processing executed by the photographing device 1. FIG. 8 shows a series of images im acquired from a camera over time, and the detection target is a person.

When the user operates the photographing device 1 to cause the photographing device 1 to start processing, the processes in steps S201 to S215 continue in a predetermined manner until the user operates the photographing device 1 and causes the photographing device 1 to finish the process. Runs at time intervals.

(Step S201)
First, when the photographing device 1 starts processing, the photographing unit 134 starts the camera, and the acquiring unit 131 starts acquiring the image im from the camera (step S201). For example, the acquisition unit 131 acquires one image im from a camera every second, and temporarily stores it in the storage unit 12 as image information 121 in association with the shooting date and time. The time interval at which the acquisition unit 131 acquires the image im from the camera can be set arbitrarily.

(Step S202)
When the new image im is saved in the storage unit 12, the person detection unit 132A performs a person detection process on the saved image im, and detects a person from the image im (step S202). When image im includes multiple people, multiple people are detected.

(Step S203)
The person detection unit 132A checks whether a person is detected from the image im (step S203). If no person is detected (step S203: NO), the image im is deleted and the process proceeds to step S213. If a person is detected (step S203: YES), the process proceeds to step S204.

In the example of FIG. 8, if the image im1 that does not include a person is acquired, the process proceeds to step S213, and if the image im2 that includes a person is acquired, the process proceeds to step S204.

(Step S204)
When the new image im is stored in the storage unit 12, the face detection unit 132B performs face detection processing on the stored image im and detects a face from the image im (step S204). When image im includes multiple faces, multiple faces are detected.

(Step S205)
The face detection unit 132B checks whether a face is detected from the image im (step S205). If no face is detected (step S205: NO), the process proceeds to step S208. If a face is detected (step S205: YES), the process advances to step S206.

In the example of FIG. 8, if the image im1 in which the face is small and is not detected is obtained, the process proceeds to step S208, and if the image im2 in which the face is detected is obtained, the process proceeds to step S206.

(Step S206)
The notification unit 133 determines whether the face detected from the image im is included in the first region r1 of the image im (step S206). Here, the first region r1 and the second region r2 will be explained.

The first area r1 is an area on the image im in which photography is set to be performed while a person or face is located there, and is set as a range of positions preferable as the position of the face in the image im. It is an area. The first region r1 is, for example, a region including the center of the image im (center region), but is not limited thereto. The image im in which the face is included in the first region r1 is an image im in which the face is appropriately photographed.

The first region r1 may be a rectangular region whose vertical and horizontal lengths are both shorter than the outline of the image im (see FIG. 8), a rectangular region that cuts or crosses the center of the image im, or a circular or elliptical region. It can also be in the area of form. The shape and size of the first region r1 can be set arbitrarily.

The second area r2 is an area on the image im that is set to be photographed while a person or face is located there, and is an area other than the first area r1. The second area r2 may be the entire area other than the first area r1 in the image im, or may be a part of the image im. The image im in which a person or a face is included in the second region r2 is an image im of a person or a face.

The second region r2 may be, for example, a rectangular region surrounding the first region r1, which is a rectangle whose length and width are both shorter than the outline of the image im, or a rectangular region that extends vertically or crosses the center of the image im. It may be a two-divided area sandwiching one area r1 left and right or top and bottom, or it may be a circular or elliptical area surrounding a circular or elliptical first area r1. The shape and size of the second region r2 can be set arbitrarily.

Note that, in this embodiment, the second region r2 is the entire area other than the first region r1. Therefore, if the face detected from the image im is not included in the first region r1, the face is included in the second region r2.

The notification unit 133 may determine that the face is included in the first area r1 of the image im if the entire face is included in the first area r1, or may determine that the face is included in the first area r1 if the center of gravity of the face is included in the first area r1. , it may be determined that the face is included in the first region r1 of the image im. When a plurality of faces are detected from the image im, if at least one face is included in the first region r1, it may be determined that the face is included in the first region r1 of the image im.

If the face is included in the first region r1 (step S206: YES), the process proceeds to step S207. If the face is not included in the first region r1 (included in the second region r2) (step S206: NO), the process proceeds to step S208.

In the example of FIG. 8, if the image im4 with the face inside the first area r1 is acquired, the process proceeds to step S207, and if the image im3 with the face outside the first area r1 is acquired, the process The process proceeds to step S208. In the example of FIG. 8, it is assumed that it is determined whether the face is included in the first region r1 based on the position of the center of gravity of the bounding box b2 corresponding to the detected face.

(Step S207)
When the face is included in the first region r1, the notification unit 133 controls the output device 210 to notify the user using the first pattern (step S207). For example, the notification unit 133 notifies the user that the face is included in the first region r1 by vibrating the vibrator in the first pattern. Furthermore, the notification unit 133 may notify the user that the face is included in the first region r1 by outputting the first pattern of sound from the speaker. In this way, by notifying the user with vibration or sound, the user can know that the face is included in the first region r1 without looking at the screen of the photographing device 1. In the example of FIG. 8, when an image im3 in which a person is included in the first region r1 is acquired, the user is notified of this in the first pattern.

(Step S208)
On the other hand, if the face is not detected or the face is not included in the first region r1 (the face is included in the second region r2), the notification unit 133 controls the output device 210 to display the second pattern. The user is notified (step S208). The second pattern is an arbitrary pattern different from the first pattern. For example, the notification unit 133 notifies the user that the face is not detected or that the face is included in the second region r2 by vibrating the vibrator in the second pattern. Furthermore, the notification unit 133 may notify the user that the face is not detected or that the face is included in the second region r2 by outputting the second pattern of sound from the speaker. In this way, by notifying the user with vibration or sound, the user can confirm that the face has not been detected or that the face is not included in the first area r1 (the second area r2). In the example of FIG. 8, when an image im2 in which no face is detected is acquired, and an image im3 in which a face is not included in the first region r1 is acquired, the user is notified of this using the second pattern. Ru.

(Step S209)
After that, the photographing unit 134 checks whether currently photographing is in progress, that is, whether photographing has already started (step S209). If photographing is not yet in progress (step S209: NO), the process advances to step S211. If shooting is already in progress (step S209: YES), the process advances to step S210.

(Step S210)
If photographing is already in progress, the photographing unit 134 photographs the object t (step S210). Specifically, the photographing unit 134 stores the image im in the storage unit 12 as a photographed image. The photographed image is stored as photographed image information 122 in association with the date and time of photographing. After that, the process returns to step S201.

(Step S211)
If photographing is not yet in progress, the photographing unit 134 photographs the object t and starts photographing (step S211). The photographing unit 134 stores the image im in the storage unit 12 as a photographed image. The photographed image is stored as photographed image information 122 in association with the date and time of photographing. Thereafter, in step S209, it is determined that shooting is in progress. In the example of FIG. 8, when the image im3 is acquired, it is determined that the image is being photographed, and the image im3 is saved as the photographed image. The photographed image is stored as photographed image information 122 in association with the date and time of photographing.

(Step S212)
Thereafter, the notification unit 133 controls the output device 210 to notify the user that photography has started. The notification unit 133 notifies the user of the start of shooting, for example, by outputting a sound from a speaker indicating that shooting has started. After that, the process returns to step S201. This allows the user to know that photography has started without looking at the photography device 1. Note that the notification of the start of shooting may be omitted.

In the present embodiment, while the object t is included in the first region r1 or the second region r2, the camera performs photographing at predetermined time intervals (for example, at 1-second intervals). The start of imaging is notified at the beginning of the period during which imaging is performed (at the start of imaging). In the example of FIG. 8, when image im2 is acquired, photographing is started, image im2 is saved as a photographed image, and the user is notified of the start of photographing.

Here, the process returns to step S203. In step S203, the person detection unit 132A checks whether a person is detected from the image im. If no person is detected (step S203: NO), the image im is deleted and the process proceeds to step S213.

(Step S213)
If no person is detected, the photographing unit 134 checks whether photographing is currently in progress, that is, whether photographing has already started (step S213). If the image im is not yet being photographed (step S213: NO), the image im is deleted and the process returns to step S201. If shooting is already in progress (step S213: YES), the process advances to step S214.

(Step S214)
If photographing is already in progress, the photographing unit 134 ends the photographing (step S214). Specifically, the photographing unit 134 deletes the image im.

(Step S215)
Thereafter, the notification unit 133 controls the output device 210 to notify the user that the shooting has ended. The notification unit 133 notifies the user of the end of shooting, for example, by outputting a sound from a speaker indicating that shooting has ended. After that, the process returns to step S201. Thereby, the user can know that the shooting has ended without looking at the shooting device 1. Note that the notification of completion of photographing may be omitted.

In the present embodiment, while the object t is included in the first region r1 or the second region r2, the camera performs photographing at predetermined time intervals (for example, at 1-second intervals). The end of photography is notified at the end of the period during which photography is performed (at the end of photography). In the example of FIG. 8, when image im6 is acquired, imaging ends, image im6 is deleted, and the user is notified of the end of imaging.

Here, the processing when the image im in FIG. 8 is acquired from a camera will be summarized. It is assumed that imaging has not yet started at the time when image im1 is acquired.

When image im1 is acquired (step S201), since no person is detected (step S203: NO) and the image is not being photographed (step S213: NO), image im1 is deleted.

When image im2 is acquired (step S201), a person is detected (step S203: YES), a face is not detected (step S205: NO), and the image is not being photographed yet (step S209: NO). The photographing device 1 vibrates in a second pattern (step S208), photographing is started (step S211), image im2 is saved as a photographed image, and a sound indicating the start of photographing is output (step S212). Thereby, the user can know, without looking at the screen of the photographing device 1, that the photographing of the image im, which includes a person but whose face is not included in the first region r1, has started.

When image im3 is acquired (step S201), a person is detected (step S203: YES), a face is detected (step S205: YES), and the face is not included in the first region r1 (step S206: NO). Moreover, since the photographing is already in progress (step S209: YES), the photographing device 1 vibrates in the second pattern (step S208), and the image im3 is saved as the photographed image (step S210). Thereby, the user can know, without looking at the screen of the photographing device 1, that the image im, which includes a person but whose face is not included in the first region r1, has been photographed.

When image im4 is acquired (step S201), a person is detected (step S203: YES), a face is detected (step S205: YES), the face is included in the first region r1 (step S206: YES), Moreover, since the photographing is already in progress (step S209: YES), the photographing device 1 vibrates in the first pattern (step S207), and the image im4 is saved as the photographed image (step S210). Thereby, the user can know that the image im in which a person's face is included in the first region r1 has been taken without looking at the screen of the photographing device 1.

When image im5 is acquired (step S201), a person is detected (step S203: YES), a face is detected (step S205: YES), and the face is not included in the first region r1 (step S206: NO). Moreover, since the photographing is already in progress (step S209: YES), the photographing device 1 vibrates in the second pattern (step S208), and the image im5 is saved as the photographed image (step S210). Thereby, the user can know, without looking at the screen of the photographing device 1, that the image im, which includes a person but whose face is not included in the first region r1, has been photographed.

When image im6 is acquired (step S201), since no person is detected (step S203: NO) and photography is already in progress (step S213: YES), photography ends (step S214), and image im6 is acquired (step S201). is deleted, and a sound indicating the end of photography is output (step S215). Thereby, the user can know that the photographing of the image im in which the person has been captured has been completed without looking at the screen of the photographing device 1.

<Summary>
As explained above, the photographing device 1 acquires the image im from the camera, detects a person from the image im, detects a face from the image im, and detects that the detected face is included in the first region r1 of the image im. The first pattern is different from the first pattern when the face is not included in the first area r1 of the image im and the detected person is included in the first area r1 or the second area r2. Notification is made using a second pattern, and while the person is included in the first region r1 or the second region r2, photography is performed with the camera at predetermined time intervals.

Depending on whether there are notifications of the first pattern and the second pattern, the user (photographer) can know whether a person is being photographed. Therefore, the user can photograph a person without looking at the screen of the photographing device 1 by simply adjusting the direction of the photographing device 1 (camera) so that the first pattern or the second pattern is notified.

Furthermore, depending on whether the notification of the first pattern or the second pattern is performed, the user can know where in the image im the face of the person being photographed is shown. Therefore, the user can photograph a face at an appropriate position without looking at the screen of the photographing device 1 by simply adjusting the orientation of the photographing device 1 (camera) so that the first pattern notification is performed.

Note that in this embodiment, when a face is detected from the image im, the photographing device 1 may perform face authentication processing on the detected face. When a person included in the image im is identified through the face authentication process, information indicating the identified person may be stored as the photographed image information 122 in association with the image im (the photographed image).

Further, if the person included in the image im is identified through the face recognition process and the person is a predetermined person set in advance, the notification unit 133 may notify the user of this in a third pattern. good. The third pattern is a vibration or sound pattern different from the first pattern and the second pattern. Thereby, the user can know that a predetermined person is being photographed without looking at the screen of the photographing device 1.

Further, the photographing unit 134 may perform photographing only when the person included in the image im is identified by the face authentication process and the person is a predetermined person set in advance. Thereby, photography can be executed only when a predetermined person is included in the image im.

<Additional notes>
This embodiment includes the following disclosure.

(Additional note 1)
A photographing method performed by a photographing device equipped with a camera, the method comprising:
an acquisition process of acquiring an image from the camera;
a detection process of detecting a target object from the image;
If the target object detected by the detection process is included in a first area of the image, a first pattern is notified; if the target object detected by the detection process is included in a second area of the image, a first pattern is provided. a notification process of notifying with a second pattern different from the first pattern when the
a photographing process of photographing with the camera at predetermined time intervals while the object is included in the first region or the second region;
Shooting methods including.

(Additional note 2)
A photographing method performed by a photographing device equipped with a camera, the method comprising:
an acquisition process of acquiring an image from the camera;
a person detection process for detecting a person from the image;
face detection processing for detecting a face from the image;
If the face detected by the face detection process is included in a first area of the image, a first pattern is notified, and if the face is not included in the first area of the image and the person detection process notification processing of notifying with a second pattern different from the first pattern when the detected person is included in the first area or the second area;
a photographing process of photographing with the camera at predetermined time intervals while the person is included in the first region or the second region;
Shooting methods including.

(Additional note 3)
The photographing method according to appendix 1, wherein the object is a person, an animal, or a face thereof.

(Additional note 4)
The photographing method according to appendix 1 or 2, wherein the first area is an area including the center of the image.

(Appendix 5)
The photographing method according to appendix 1 or 2, wherein the second area is an area other than the first area of the image.

(Appendix 6)
The photographing method according to supplementary note 1 or supplementary note 2, wherein the notification process is performed by vibration.

(Appendix 7)
The photographing method according to supplementary note 1 or supplementary note 2, wherein the notification process is to notify by sound at least one of the start and end of shooting.

(Appendix 8)
further comprising face recognition processing for performing face recognition from the image,
The photographing method according to supplementary note 1 or supplementary note 2, wherein the notification process is to notify using a third pattern when a predetermined person is identified by the face authentication process.

(Appendix 9)
A photographing device equipped with a camera,
an acquisition unit that acquires an image from the camera;
a detection unit that detects a target object from the image;
A first pattern is used to notify when the object detected by the detection section is included in a first region of the image, and a first pattern is provided when the object detected by the detection section is included in a second region of the image. a notification unit that notifies using a second pattern different from the first pattern when the
a photographing unit that photographs the object with the camera at predetermined time intervals while the object is included in the first region or the second region;
A photographing device equipped with.

(Appendix 10)
A photographing method performed by a photographing device equipped with a camera, the method comprising:
an acquisition unit that acquires an image from the camera;
a person detection unit that detects a person from the image;
a face detection unit that detects a face from the image;
If the face detected by the face detection unit is included in the first area of the image, a first pattern is notified; if the face is not included in the first area of the image, and the person detection unit a notification unit configured to notify using a second pattern different from the first pattern when the detected person is included in the first area or the second area;
a photographing unit that photographs with the camera at predetermined time intervals while the person is included in the first region or the second region;
A photographing device equipped with.

(Appendix 11)
A photographing method performed by a photographing device equipped with a camera, the method comprising:
an acquisition process of acquiring an image from the camera;
a detection process of detecting a target object from the image;
If the target object detected by the detection process is included in a first area of the image, a first pattern is notified; if the target object detected by the detection process is included in a second area of the image, a first pattern is provided. a notification process of notifying with a second pattern different from the first pattern when the
a photographing process of photographing with the camera at predetermined time intervals while the object is included in the first region or the second region;
A program that causes a computer to execute shooting methods including

(Appendix 12)
A photographing method performed by a photographing device equipped with a camera, the method comprising:
an acquisition process of acquiring an image from the camera;
a person detection process for detecting a person from the image;
face detection processing for detecting a face from the image;
If the face detected by the face detection process is included in a first area of the image, a first pattern is notified, and if the face is not included in the first area of the image and the person detection process notification processing of notifying with a second pattern different from the first pattern when the detected person is included in the first area or the second area;
a photographing process of photographing with the camera at predetermined time intervals while the person is included in the first region or the second region;
A program that causes a computer to execute shooting methods including

The embodiments disclosed herein are illustrative in all respects and should not be considered restrictive. The scope of the present invention is indicated by the claims rather than the above-mentioned meaning, and it is intended that all changes within the meaning and range equivalent to the claims are included. Further, the present invention is not limited to the embodiments described above, and various modifications can be made within the scope of the claims, and can be obtained by appropriately combining technical means disclosed in different embodiments. The embodiments are also included in the technical scope of the present invention.

1: Imaging device 11: Communication unit 12: Storage unit 13: Control unit 101: Processor 102: Memory 103: Storage 104: Communication I/F
105: Input/output I/F
106: Drive device 107: Input device 108: Output device 109: Recording medium 121: Image information 122: Photographed image information 131: Acquisition section 132: Detection section 132A: Person detection section 132B: Face detection section 133: Notification section 134: Photography Department

Claims (7)

A photographing method performed by a photographing device equipped with a camera, the method comprising:
an acquisition process of acquiring an image from the camera;
a person detection process for detecting a person from the image;
face detection processing for detecting a face from the image;
If the face detected by the face detection process is included in a first region of the image , at least one of a first pattern of vibration and sound is notified, and the face is not included in the first region of the image; and a notification process of notifying at least one of vibration and sound of a second pattern different from the first pattern when the person detected by the person detection process is included in the first area or the second area. ,
a photographing process of photographing with the camera at predetermined time intervals while the person is included in the first region or the second region;
Shooting methods including. The photographing method according to claim 1 , wherein the first area is an area including the center of the image. The photographing method according to claim 1 , wherein the second area is an area other than the first area of the image. 2. The photographing method according to claim 1 , wherein the notification process includes notifying by sound at least one of the start and end of photographing. further comprising face recognition processing for performing face recognition from the image,
2. The photographing method according to claim 1 , wherein the notification process includes notification using a third pattern when a predetermined person is identified by the face authentication process. A photographing method performed by a photographing device equipped with a camera, the method comprising:
an acquisition unit that acquires an image from the camera;
a person detection unit that detects a person from the image;
a face detection unit that detects a face from the image;
If the face detected by the face detection unit is included in a first area of the image , at least one of a first pattern of vibration and sound is notified, and the face is not included in the first area of the image; and a notification unit that notifies at least one of vibration and sound of a second pattern different from the first pattern when the person detected by the person detection unit is included in the first area or the second area. ,
a photographing unit that photographs with the camera at predetermined time intervals while the person is included in the first region or the second region;
A photographing device equipped with. A photographing method performed by a photographing device equipped with a camera, the method comprising:
an acquisition process of acquiring an image from the camera;
a person detection process for detecting a person from the image;
face detection processing for detecting a face from the image;
If the face detected by the face detection process is included in a first region of the image , at least one of a first pattern of vibration and sound is notified, and the face is not included in the first region of the image; and a notification process of notifying at least one of vibration and sound of a second pattern different from the first pattern when the person detected by the person detection process is included in the first area or the second area. ,
a photographing process of photographing with the camera at predetermined time intervals while the person is included in the first region or the second region;
A program that causes a computer to execute shooting methods including