JP2014143595A - Image recorder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a user to record a user-specific image, without requiring any special operation.SOLUTION: An image recorder includes an imaging section 103a worn on the head of a user, capturing a subject in the sight line direction of the user and outputting a captured image, a gaze determination section 201 for determining whether or not the user is gazing the subject continuously for a predetermined time or more, and an automatic photographing section 204 for starting photographing operation of the imaging section 103a based on the determination results from the gaze determination section 201. An image captured by the photographing operation of the automatic photographing section 204 is recorded, as an image file, in a flash memory.

Description

  The present invention relates to an image recording apparatus.

  A life log that automatically captures and records daily scenes encountered by a user with a wearable camera that can be worn on the body attracts attention. Patent Document 1 discloses a wearable camera that performs automatic photographing using a sound such as a whistle sound as a trigger.

JP 2004-356970 A

  An object of the present invention is to provide an image recording apparatus capable of automatically recording an image unique to a user without any special operation by the user. The invention of Patent Document 1 requires a special operation of generating a predetermined sound as a trigger in order to automatically record an image.

  An image recording apparatus according to the present invention is an image recording apparatus to be worn on a user's head, and includes an imaging unit that captures a subject image in a user's line-of-sight direction and outputs a captured image; A gaze determination unit that determines whether or not the subject is gazed, an automatic shooting unit that causes the imaging unit to start a shooting operation based on a determination result of the gaze determination unit, and an image that is captured by a shooting operation by the automatic shooting unit And image recording means for recording as an image file.

  According to the present invention, a user-specific image can be automatically recorded without any special operation by the user.

1A is a schematic front view of an image recording apparatus according to an embodiment of the present invention. FIG. 2B is a schematic rear view of the image recording apparatus according to the embodiment of the present invention. 1 is a block configuration diagram of an image recording apparatus according to an embodiment of the present invention. 1 is a functional block diagram of an image recording apparatus according to an embodiment of the present invention. 5 is a flowchart relating to processing executed when the image recording apparatus is in an image recording mode. 6 is a flowchart relating to processing executed when the image recording apparatus is in an image reproduction mode. It is a figure which shows an example of the display screen for selecting the image file which performs image reproduction.

  FIG. 1A is a schematic front view of an image recording apparatus according to an embodiment of the present invention. FIG. 1B is a schematic rear view of the image recording apparatus illustrated in FIG. The image recording apparatus 100 is an electronic device that is worn on the user's head, and is, for example, a wearable camera or a head-mounted display. FIGS. 1A and 1B show an example in which the present invention is applied to a see-through type head mounted display.

  The image recording apparatus 100 includes a frame structure 101 similar to glasses, and is supported by a user's head. An image display unit 102 is attached to a predetermined position of the frame structure 101. The image display unit 102 displays an image to be visually recognized by the user on the left eye of the user via the display optical system 104.

  As shown in FIG. 1A, an image pickup unit 103a is attached to the front side of the image display unit 102 and picks up a subject image in the direction of the user's line of sight. The display optical system 104 is a lens provided in front of the user's left eye, and has a half mirror (not shown) in a part thereof. As shown in FIG. 1B, an eyeball imaging unit 103 b is provided on the back side of the image display unit 102 and images the left eye of the user.

  FIG. 2 is a block configuration diagram of the image recording apparatus 100. The image recording apparatus 100 includes a CPU 20, a RAM 22, a program ROM 24, a video RAM 26, and a flash memory 28.

  The CPU 20 controls various processes executed by the image recording apparatus 100. The RAM 22 serves as a work area when various processes are executed by the image recording apparatus 100. The program ROM 24 stores computer programs for various processes executed by the CPU 20 of the image recording apparatus 100.

  The video RAM 26 stores an image (image data indicating an image) displayed on the image display unit 102. The video RAM 26 stores a captured image (captured image data) captured by the imaging unit 103. The flash memory 28 stores data for an image stored in the video RAM 26 when the image display unit 102 displays an image.

  Further, the image recording apparatus 100 includes a video signal processing unit 30, a timing unit 32, a camera control unit 34, an operation unit 36, a display unit 102, an imaging unit 103a, and an eyeball imaging unit 103b.

  An image display unit 102 is connected to the video signal processing unit 30. The video signal processing unit 30 controls the image stored in the video RAM 26 to be displayed on the image display unit 102 in accordance with a command from the CPU 20. The image display unit 102 is configured by, for example, a retinal scanning display, and two-dimensionally scans image light corresponding to the image signal processed by the video signal processing unit 30. The scanned image light is guided to the user's left eye via the half mirror of the display optical system 104 and forms an image on the retina.

  An imaging unit 103a and an eyeball imaging unit 103b are connected to the camera control unit 34. The imaging unit 103a captures a subject image in the user's line-of-sight direction. The eyeball imaging unit 103b is attached to the frame structure 102 or the like, and images the user's left eye. The camera control unit 34 controls imaging by the imaging unit 103a and the eyeball imaging unit 103b in accordance with a command from the CPU 20. The camera control unit 34 stores the captured image captured by the imaging unit 103 a and the eyeball imaging unit 103 b in the video RAM 26.

  The timer unit 32 is controlled by the CPU 20 and detects an elapsed time from a predetermined timing. The operation unit 36 includes a power switch, a mode change switch, a release switch, and the like.

  The image recording apparatus 100 has at least an image recording mode and an image reproduction mode as operation modes. In the image recording mode, the image recording apparatus 100 records an image file based on the captured image of the imaging unit 103 a in the flash memory 28 under the control of the CPU 20. On the other hand, in the image playback mode, the image recording device 100 transfers the image file recorded in the flash memory 28 under the control of the CPU 20 via the video signal processing unit 30 and the image display unit 102. Display on the retina.

  FIG. 3 is a control block diagram of the image recording apparatus 100. The CPU 20 of the image recording apparatus 100 functions as the gaze determination unit 201, the person determination unit 202, the facial expression detection unit 203, and the automatic photographing unit 204 shown in FIG. 3 by executing the program stored in the program ROM 24. .

  The gaze determination unit 201 determines whether or not the user is gazing at a predetermined subject for a predetermined time or more. The gaze determination unit 201 includes a gaze detection unit 211 and a pupil detection unit 212. The line-of-sight detection unit 211 detects the user's line-of-sight direction based on, for example, an image of the user's left eye captured by the eyeball imaging unit 103b. For example, the pupil detection unit 212 detects the degree of opening (size) of the pupil of the user's left eye based on the image of the user's left eye captured by the eyeball imaging unit 103b.

  The gaze determination unit 201 indicates that when the user's gaze direction detected by the gaze detection unit 211 is continuously facing the same direction for a predetermined time or more, the user is gazing at a predetermined subject for a predetermined time or more. judge. Further, the gaze determination unit 201 detects that the pupil of the user's left eye detected by the pupil detection unit 212 has been continuously opened for a predetermined time or more and has a predetermined size or more. It is determined that the subject is being watched. Both the determination based on the detection result of the line-of-sight detection unit 211 and the determination based on the detection result of the pupil detection unit 212 may be used simultaneously, or one of them may be selected by the user. The predetermined time is, for example, 30 seconds, and is measured by the timer unit 32.

  The person determination unit 202 determines whether or not a person image exists in the captured image of the imaging unit 103a stored in the video RAM 26 using a known face detection technique. The facial expression detection unit 203 detects the facial expression of the person image based on the face image when the person determination unit 202 determines that a human image exists in the captured image of the imaging unit 103a stored in the video RAM 26. For example, the facial expression detection unit 203 detects a smile, an angry face, or the like based on features distributed over the entire face, such as the user's eyes, mouth, facial wrinkle shape, and mouth opening / closing state. For example, if a feature is detected from a face image, such as the corner of the eye is lowered, the corner of the mouth is raised, the mouth is open, or the fringe line is clear, the facial image is judged to be a smile. To do. Further, for example, when a feature such as the corner of the eye is raised, the corner of the mouth is lowered, or the wrinkles between the eyebrows are clear is detected from the face image, the expression of the face image is determined to be an angry face.

  The automatic photographing unit 204 controls the camera control unit 34 to automatically start the photographing operation by the imaging unit 103a. The captured image of the imaging unit 103a stored in the video RAM 26 by the camera control unit 34 is stored in the flash memory 28 after being subjected to predetermined image processing and predetermined image compression processing under the control of the CPU 20. .

  FIG. 4 is a flowchart relating to processing executed when the image recording apparatus 100 is in the image recording mode. In step S <b> 400, the CPU 20 determines whether or not the user is gazing at a predetermined subject continuously for a predetermined time or more by the processing of the gaze determination unit 201. CPU20 repeats the process of step S400, when step S400 is negatively determined, and when step S400 is positively determined, the process proceeds to step S401.

  In step S401, the CPU 20 determines whether a person image exists in the captured image of the imaging unit 103a by the process of the person determination unit 202. If the captured image does not include a person image, for example, only the landscape, the CPU 20 makes a negative determination in step S401 and advances the process to step S403. If the captured image includes a person image, the CPU 20 makes a positive determination in step S401 and advances the process to step S402.

  In step S <b> 402, the CPU 20 detects the facial expression of the person included in the captured image by the processing of the facial expression detection unit 203. If the facial expression of the person is a smile, the CPU 20 advances the process to step S403. If the facial expression of the person is an angry face, the CPU 20 returns the process to step S400. If it is determined that the person is angry, the CPU 20 determines that the person has refused to shoot by the user and does not record an image. Thereby, the privacy of the person who refuses to shoot can be protected.

  In step S <b> 403, the CPU 20 starts the photographing operation by the imaging unit 103 a by the process of the automatic photographing unit 204. The recorded image may be a still image or a moving image. If a moving image is being shot, an image indicating that shooting is being performed may be output from the image display unit 102. For example, the CPU 20 causes the image display unit 102 to output an image including characters such as “REC”. After the image file based on the image captured by the imaging unit 103a is stored in the flash memory 28 by the automatic imaging process, the CPU 20 returns the process to step S400.

  FIG. 5 is a flowchart relating to processing executed when the image recording apparatus 100 is in the image reproduction mode. In step S500, the CPU 20 acquires one or more image files stored in the flash memory 28. In step S501, the CPU 20 generates a thumbnail image for each of the image files acquired in step S500. In step S502, a display screen for displaying each thumbnail image generated in step S501 is generated.

  FIG. 6 is a diagram illustrating an example of the layout of the display screen generated in step S502. The display screen layout 600 shown in FIG. 6 includes display areas 601 a, 601 b, 601 c, and 601 d for displaying the thumbnail images generated in step S 501, and a pointer 602. The pointer 602 is displayed based on the line-of-sight direction detected by the line-of-sight detection unit 211 of FIG.

  Returning to FIG. 5 again, in step S503, the CPU 20 controls the video signal processing unit 30 so that the display screen generated in step S502 is displayed by the image display unit 102.

  In step S <b> 504, the CPU 20 determines whether or not the user's line-of-sight direction is continuously facing the same direction for a predetermined time or longer by the processing of the line-of-sight detection unit 211. That is, it is determined whether or not the position of the pointer 604 is at the same position continuously for a predetermined time (for example, 5 seconds). CPU20 advances a process to step S505, when step S504 is affirmed determination, and advances a process to step S501 when step S504 is negatively determined.

  In step S505, the CPU 20 determines whether or not the position of the pointer 602 is inside the thumbnail image display areas 601a to 601d. If step S505 is positively determined, the CPU 20 advances the process to step S506, and if step S505 is negatively determined, the CPU 20 advances the process to step S507.

  In step S506, the CPU 20 selects an image file corresponding to the display area where the pointer 602 is located, and starts reproducing the image file. For example, when a still image file is selected, the still image is displayed. When a moving image file is selected, the moving image is reproduced and displayed. After the reproduction is completed, the CPU 20 returns the process to step S502.

  In step S507, the CPU 20 determines that the user is not interested in the displayed thumbnail image, and changes the image file for displaying the thumbnail image to another image file. After changing the image file, the CPU 20 returns the process to step S500.

According to the embodiment described above, the following operational effects can be obtained.
The image recording apparatus 100 is a head-mounted display that is worn on the user's head, and includes an imaging unit 103a that captures a subject image in the user's line-of-sight direction and outputs a captured image, and the user continuously captures a subject for a predetermined time or more. A gaze determination unit 201 that determines whether or not the user is gazing, an automatic shooting unit 204 that starts a shooting operation of the imaging unit 103a based on a determination result of the gaze determination unit 201, and a shooting operation by the automatic shooting unit 204. And a CPU 20 that records the recorded image in the flash memory 28. By providing such a configuration, it is possible to automatically record a user-specific image without performing a special operation by the user simply by observing the subject according to his / her interest or the like.

The embodiment described above can be implemented with the following modifications.
[Modification 1]
In the above embodiment, the image display unit 102 guides the image to the left eye of the user and forms the image on the retina of the left eye. However, the image display unit 102 guides the image to the right eye of the user and displays the image on the retina of the right eye. You may decide to form. Further, the eyeball imaging unit 103b captures an image of the user's left eye, but may capture an image of the right eye. The left-eye and right-eye image display units 102 and the eyeball imaging unit 103b may be provided at the same time.

[Modification 2]
Although the above embodiment has been described with an example of an optical see-through head mounted display, the present invention can also be applied to a video see-through head mounted display or a wearable camera attached to the head.

[Modification 3]
Although the image display unit 102 is a retinal scanning display, other transmissive display methods may be used, and a non-transmissive liquid crystal display, an organic EL display, or the like may be used. You may further provide the sound collection part which collects an audio | voice simultaneously with image recording, and the speaker which reproduces | regenerates an audio | voice.

[Modification 4]
The process of step S402 in FIG. 4 can be omitted. When the process of step S402 is omitted, the CPU 20 may return the process to step S400 when an affirmative determination is made in step S401.

  The embodiments and modifications described above are merely examples, and the present invention is not limited to these contents as long as the features of the invention are not impaired. Further, the embodiments and modifications described above may be combined and executed as long as the features of the invention are not impaired.

20 CPU
24 Program ROM
28 Flash Memory 32 Timekeeping Unit 100 Image Recording Device 102 Image Display Unit 103a Imaging Unit 103b Eyeball Imaging Unit

Claims (6)

An image recording device worn on a user's head,
Imaging means for imaging a subject image in the user's line of sight and outputting a captured image;
Gaze determination means for determining whether or not the user is gazing at a subject continuously for a predetermined time or more;
Automatic photographing means for causing the imaging means to start photographing based on a determination result of the gaze determining means;
Image recording means for recording an image photographed by the photographing operation by the automatic photographing means as an image file;
An image recording apparatus comprising: The image recording apparatus according to claim 1,
The gaze determining means includes
Gaze detection means for detecting the gaze direction of the user,
When the line-of-sight detection means detects that the line-of-sight direction is substantially the same for a predetermined time or longer, it is determined that the user is gazing at the subject for a predetermined time or longer. Image recording device. The image recording apparatus according to claim 2,
Further comprising person determination means for determining whether or not a person image is included in the captured image,
The automatic photographing unit performs a photographing operation on the imaging unit when the person determining unit determines that the line-of-sight direction is substantially the same for a predetermined time or more and is not included in the captured image. The image recording apparatus is characterized in that when the person determination unit determines that a person image is included in the captured image, the imaging unit does not start a shooting operation. The image recording apparatus according to claim 3.
A facial expression detection means for detecting the facial expression of the person image when the person determination means determines that a person image is included in the captured image;
The automatic photographing unit causes the imaging unit to start a photographing operation when the facial expression detection unit detects a smile when the human determination unit determines that a person image is included in the captured image. An image recording apparatus, wherein when the facial expression detection means detects an angry face, the imaging means does not start a photographing operation. The image recording apparatus according to any one of claims 2 to 4,
Display means provided at a position facing at least one eye of the user and displaying an image;
Thumbnail display means for displaying a thumbnail image of the image file recorded in the image recording means on the display means;
An image further comprising reproduction means for causing the display means to reproduce and display the image file corresponding to the thumbnail image when the line-of-sight direction is continuously directed to the same thumbnail image for a predetermined time or more. Recording device. In the image recording device according to any one of claims 1 to 5,
The gaze determining means includes
Comprising pupil detection means for detecting the size of the user's pupil;
An image recording apparatus according to claim 1, wherein when the size of the pupil is a predetermined size or more continuously for a predetermined time or more, it is determined that the user is gazing at the subject for a predetermined time or more.

Images