JP2009130903A - Image recording apparatus, image recording method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image recording apparatus capable of extracting a face image existent in a photographed moving picture, generating a zoomed-up image of the face, and recording the zoomed-up image of the face into an optical recording medium as a multi-angle image together with a normal image. <P>SOLUTION: The image recording apparatus comprises: a signal processing circuit 9 in which it is detected whether a face image exists in an image relating to image data or not and when the face image is detected, metadata relating to detection timing, coordinates and size of the face image are output together with the image data; and a recording and reproducing circuit 11 in which the image data and metadata are stored on a hard disk 12 under control of a microprocessor 1, the image data and the metadata are read from the hard disk 12 in dubbing, the face image is extracted on the basis of the metadata to produce face image data, and the image data and the face image data are recorded on an optical recording medium via a DVD drive 13. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a recording apparatus or the like that records image data on an optical recording medium, and more particularly, to a multi-angle compatible image recording apparatus, an image recording method, and a program.

  2. Description of the Related Art Conventionally, in an imaging apparatus such as a digital camera, a face such as a person is detected from a captured image signal, and autofocus control or image quality correction is automatically performed on the face. Here, as a method of detecting a face from a captured image signal, there is a method of defining an average face image as a template, for example, from a sampled face image, and matching the image signal using the template.

  For example, in Patent Document 1, the number of face images included in an image, the face position, the face size, and the face orientation of each face image are detected, and the images are sequentially switched and displayed. In this case, an image processing apparatus that controls display of an image to which an image switching effect is applied in accordance with the detected face image is disclosed.

JP 2007-79894 A

  However, as disclosed in Patent Document 1, a method of extracting a person's face from an image and using it for searching for an image is well known in an imaging apparatus such as a digital camera. A technique for extracting a face image, cutting out only the up image of the face and reproducing it is not disclosed in Patent Document 1, and has not been proposed yet.

  On the other hand, although there are editing machines mainly for business use that can record content shot simultaneously by multiple cameras as a multi-angle image on a single optical recording medium, ordinary users can easily change to multi-angle. There is no technology for producing a corresponding optical recording medium.

  In view of this, the present invention extracts a face image present in a captured video, generates a face-up image, and records the face-up image together with a normal image as a multi-angle image on an optical recording medium. The challenge is to make it possible.

  An image recording apparatus according to a first aspect of the present invention detects whether or not a face image exists in an image related to input image data. When the face image is detected, the face image in the image data is detected. Signal processing means for outputting metadata relating to the detection timing, coordinates, and size together with image data, the image data and the metadata are stored in a first storage means, and the first storage is used during dubbing. Means for reading out the image data and the metadata, extracting a face image based on the metadata to generate face image data, and recording / reproducing means for recording the image data and the face image data on an optical recording medium; Prepare.

  Therefore, the recording / reproducing means records the image data and the face image data on the optical recording medium in a multi-angle state in a state where the face up image reproduction based on the face image data can be performed.

  In the first aspect, the signal processing unit may include a face detection unit, and the face detection unit may detect the presence of a face image in the image related to the image data at a predetermined time interval. Furthermore, the detection of the face image by the face detection means detects whether or not there is a part indicating the feature of the person's face in the image, and if there is a face image, whether the number of face detections has increased or decreased. If there is an increase or decrease in face detection, the increase / decrease data may be recorded as metadata. Further, in the first aspect, the image processing apparatus includes an imaging unit that captures a subject image and outputs an imaging signal. The signal processing unit performs analog / digital conversion on the imaging signal, performs predetermined signal processing, and performs image data processing. May be generated.

  The image recording apparatus according to the second aspect of the present invention detects whether or not a face image exists in an image related to input image data, and when the face image is detected, the face in the image data is detected. Signal processing means for outputting metadata relating to image detection timing, coordinates, and size together with compressed image data, the compressed image data and the metadata are stored in a first storage means, and at the time of dubbing The compressed image data and the metadata are read from the first storage means, the compressed image data is decompressed, and a face image is extracted from the decompressed image data based on the metadata to obtain face image data And recording / reproducing means for compressing the image data and the face image data and recording the compressed data on an optical recording medium.

  Accordingly, the image data and the face image data are compressed and multi-angle recorded on the optical recording medium in a state where the face up image reproduction based on the face image data can be performed by the recording / reproducing means.

  The image recording method according to the third aspect of the present invention detects whether or not a face image exists in the image related to the input image data, and when the face image is detected, the face in the image data is detected. A step of outputting metadata relating to the timing, coordinates, and size of image detection together with the compressed image data; storing the compressed image data and the metadata in a first storage means; The compressed image data and the metadata are read from one storage means, the compressed image data is decompressed, and a facial image is extracted from the decompressed image data based on the metadata to generate facial image data And compressing the image data and the face image data and recording them on an optical recording medium.

  Therefore, the image data and the face image data are recorded on the optical recording medium in a multi-angle state in a state where the face up image reproduction based on the face image data can be performed.

  A program according to a fourth aspect of the present invention detects whether or not a face image exists in an image related to input image data, and when the computer detects the face image, Signal processing means for outputting the metadata relating to the detection timing, coordinates and size of the face image together with the compressed image data, the compressed image data and the metadata are stored in the first storage means, and at the time of dubbing Reads the compressed image data and the metadata from the first storage means, decompresses the compressed image data, extracts a face image from the decompressed image data based on the metadata, and extracts a face image It functions as a recording / reproducing unit that generates data, compresses the image data and face image data, and records the compressed data on an optical recording medium.

  Therefore, by software processing based on the program, the image data and the face image data are recorded on the optical recording medium in a multi-angle state in a state where the face up image reproduction based on the face image data can also be performed.

  According to the present invention, it is possible to extract a face image present in a captured moving image, generate a face up image, and record the face up image together with a normal image as a multi-angle image on an optical recording medium. An image recording apparatus, an image recording method, and a program can be provided.

  The best mode for carrying out the present invention (hereinafter simply referred to as an embodiment) will be described below in detail with reference to the drawings.

  FIG. 1 shows and describes the configuration of an image recording apparatus according to a first embodiment of the present invention.

  The image recording apparatus shown in FIG. 1 is applicable as a hard disk DVD (Digital Versatile Disc) video camera recorder that realizes playback from a scene corresponding to a face index thumbnail list and a selected thumbnail. Details will be described below.

  As shown in FIG. 1, the image recording apparatus 100 includes a microprocessor 1 as a control unit that controls the whole. Power from the battery 2 or the AC power supply 4 is supplied to the macro processor 1 through the power supply circuit 3 and further supplied to other circuits through the power supply control unit 5. The microprocessor 1 is connected with a key input circuit 6 as operation input means. A CCD (Charge Coupled Device) 8 is disposed as an image pickup device on the optical path of the light incident on the photographing optical system 7, and the output of the CCD 8 is connected to a signal processing circuit 9 as signal processing means. . The signal processing circuit 9 includes a face detection engine 10 as face detection means. The signal processing circuit 9 is connected to a hard disk 12 and a DVD drive 13 as first storage means via a recording / reproducing circuit 11. The recording / reproducing circuit 11 is connected to the microprocessor 1.

  The microprocessor 1 is connected to an LCD (Liquid Crystal Display) panel 16 via an on-screen display (OSD) circuit 14 and a superimposing circuit 15. In this example, the LCD panel 16 is composed of a touch panel.

  In this example, the DVD is exemplified as a normal / face angle multi-angle recording destination, but more specifically, for example, a Blu-ray disc, HD (High Definition) DVD or the like is used as a recording destination for various optical recording media. Of course, it is possible to adopt.

  The photographing optical system 7 includes a focus lens for condensing subject light on the CCD 8, a drive mechanism for driving the focus lens to perform focusing and zooming, and subject light to the image sensor for a predetermined time by opening and closing operations. Conceptually includes an incident shutter mechanism and the like.

  In such a configuration, the microprocessor 1 performs A / D (Analog / Digital) conversion on the external key input voltage of the key input circuit 6 and determines which key is pressed from the obtained voltage value, or An input signal from the LCD panel 16 that is a touch panel is A / D converted, and a button on a GUI (Graphical User Interface) displayed on the LCD panel 16 is determined from the value to control each circuit. Issue a control command for

  Further, the microprocessor 1 controls the CCD 8, the signal processing circuit 9, and the recording / reproducing circuit 11 to switch between HD / SD (Standard Definition) recording / reproducing of the video signal.

  Further, the microprocessor 1 controls the recording / reproducing circuit 11, accesses the management file and the recording file recorded on the hard disk 12, decompresses (decodes) and processes the recorded video signal, and passes through the superimposing circuit 15. It is displayed on the LCD panel 16.

  Here, the “management file” is a file for managing metadata related to the face image corresponding to the image data. The metadata includes the face detection timing, coordinates and size in the image data, and also the face Increase / decrease data is managed. On the other hand, the “recording file” is a file for recording image data (normal angle).

  At the time of the display, the OSD circuit 14 interprets the command sent from the microprocessor 1 and sends the display signal created on the built-in video RAM to the superimposing circuit 15 in scanning units. In response to this, the superimposing circuit 15 displays the recording video signal or the reproduced video signal and the display signal from the OSD circuit 14 and outputs them on the LCD panel 16.

  On the other hand, a video signal obtained by being imaged and photoelectrically converted by the CCD 8 which is an image sensor is subjected to predetermined signal processing by the signal processing circuit 9 and then as image data by the recording / reproducing circuit 11 by the recording / reproducing circuit 11. Is recorded as a recording file on a DVD serving as a storage means. The predetermined signal processing by the signal processing circuit 9 is, for example, after converting into a digital image signal, preprocessing such as defective pixel correction is performed on the image signal, and auto white balance adjustment, contrast adjustment, etc. Predetermined image quality correction is performed. Further, the signal processing circuit 9 performs an encoding process for converting into compressed image data.

  The signal processing circuit 9 incorporates a face detection engine 10, detects a face in the image data by the face detection engine 10, and stores temporal and spatial position information in the detected face image in a video management file. To do. This face detection method includes a method of defining an average face image as a template, for example, from a sampled face image, and matching the image signal with the template, but is not limited thereto. .

  The video signal recording format is high-definition using MPEG-4 AVC / H.264 (Moving Picture Experts Group-4 Advanced Video Coding / H.264) compression and AVCHD (Advanced Video Codec High Definition) (registered trademark) standards. Supports (HD) video recording and standard (SD) signal recording using MPEG-2 compression.

  Hereinafter, characteristic processing by the image recording apparatus according to the present embodiment will be described.

  FIG. 2 conceptually shows and describes the flow of multi-angle recording on a DVD.

  During recording, the LCD panel 16 displays the state of the captured moving image together with the display of “REC” such as the screen 200.

  When a face is detected in the image at the time of capture, the signal processing circuit 9 supplies the metadata along with the image data to the recording / reproducing circuit 11 side.

  The recording / reproducing circuit 11 records the metadata simultaneously with the recording of the normal image data on the hard disk 12 under the control of the microprocessor 1 (# 1).

  Next, when this image data is dubbed to a DVD by the DVD drive 13, the encoded (compressed) compressed image data is decoded (expanded) by software processing by the microprocessor 1, and the face is determined based on the metadata. The centered image is cut out by the reproduction zoom function (# 2), and the image data cut out only near the face is encoded (compressed) again and recorded on the DVD together with the normal image data (# 3).

  Thus, multi-angle recording is performed (# 4).

  Since the dubbing operation does not need to be performed in real time, the dubbing operation can be realized with the processing capability of a consumer DVD video camera recorder.

  On the other hand, if there is sufficient processing capacity and buffer memory capacity, as shown in FIG. 3, it is possible to simultaneously generate face angle image data and perform multi-angle recording when recording normal image data. . That is, in this case, normal angle normal image data is stored in the first area of the intermittent buffer included in the recording / reproducing circuit 11 (# 11), and an image is cut out from the normal image data with the face at the center. Image data related to the face angle obtained by cutting out only the vicinity of the face is sequentially stored in the second storage area of the intermittent buffer (# 12). These are multi-angle recorded on the DVD by the DVD drive 13 (# 13).

  The image data recorded in a multi-angle manner in this way can be reproduced on a screen as shown in FIG. That is, FIG. 4A shows a display example related to the normal angle, and FIG. 4B shows a display example related to the face angle. In any of the screens 201 and 202, buttons 201a and 202a for switching the angle are provided on the GUI of the touch panel so that the angle can be switched at a timing desired by the user. When display reproduction is performed at the face angle, as shown in FIG. 4B, an icon 202b indicating that the display is performed at the face angle is displayed.

  Hereinafter, a series of processes related to multi-angle recording by the image recording apparatus according to the embodiment of the present invention will be described in more detail with reference to the flowchart of FIG. Part or all of the following processing corresponds to the image recording method according to the embodiment.

  The following processing is performed in units of 1 GOP (Group of Pictures), for example.

  The subject light incident through the photographing optical system 7 is received by the light receiving surface of the CCD 8, is photoelectrically converted, and a video signal is output. This video signal is input to the signal processing circuit 9 and subjected to the signal processing as described above to generate a digital image signal (step S1).

  The face detection engine 10 of the signal processing circuit 9 detects a face in the process of this signal processing. When a face is detected (step S2 branches to Yes), metadata is extracted (step S3). The image signal is encoded and output as image data to the recording / reproducing circuit 11 (step S4). The recording / reproducing circuit 11 buffers the image data and metadata related to the normal angle in the intermittent buffer (step S5).

  Subsequently, it is determined whether or not a face is detected in the image related to the image data. If the face is detected (Yes in step S6), the microprocessor 1 processes the normal angle image data by software processing. And a face angle image signal obtained by cutting out only the vicinity of the face based on the face metadata (step S7), and the face angle image signal is encoded again (step S8). Then, the compressed image data relating to the face angle is buffered in the intermittent buffer of the recording / reproducing circuit 11 (step S9).

  In this way, the recording / reproducing circuit 11 transfers the compressed normal angle image data and the face angle image data buffered in the intermittent buffer to the DVD drive 13 and writes them to the DVD as an optical recording medium (step). S10), a series of processing ends. Multi-angle recording is realized by the above processing.

  According to the above processing, the face angle image data is dubbed and recorded on the DVD by multi-angle recording. Therefore, when reproducing the DVD, the reproduction of the human face in the image is appropriately selected and executed. It becomes possible to do.

  It should be noted that when a plurality of faces are detected in normal angle image data, it is also possible to record a plurality of face angle image data by multi-angle recording.

  Hereinafter, with reference to the flowchart of FIG. 6, a series of processes related to the multi-angle recording of the improved example improved from such a viewpoint will be described in detail.

  Since the processing of steps S11 to S15 is the same as that of steps S1 to S5 in FIG. In step S16, it is determined whether or not one or more faces have been detected. If detected, the normal angle image data is decoded by software processing by the microprocessor 1 and based on the face metadata. Then, an image signal of a face angle obtained by cutting out only the vicinity of the face is generated (step S17), and the image signal of the face angle is encoded again (step S18). Then, the image data relating to the face angle is buffered in the intermittent buffer of the recording / reproducing circuit 11 (step S19).

  Then, it is determined whether or not all face image data has been buffered (step S20), and the processes of steps S17 to S19 are repeated until all the buffering is completed. Thus, when all buffering is completed, the recording / reproducing circuit 11 transfers the compressed normal angle image data and face angle image data buffered in the intermittent buffer to the DVD drive 13 and writes them to the DVD. This is done (step S21), and thus the series of processes according to the improved example is completed.

  According to the above processing, when there are a plurality of human faces in the image, for example, when a commemorative photo is taken when going out to a family trip or the like, the image data of each person's face angle is converted into multi-image data. Since recording can be performed by angle recording, it is possible to appropriately perform image reproduction of a desired person's face angle during reproduction of the recorded DVD.

  As described above in detail, according to one embodiment of the present invention, a human face is automatically detected from a moving image, the result is recorded as metadata on a management file, and the person's face is recorded based on the metadata. It is possible to provide an image recording apparatus such as a video camera recorder that records a face-up on a DVD as a multi-angle image. At this time, information such as appearance time and coordinates on the moving image regarding the face may be written as metadata on the moving image management file.

  The creation of an up-image of a person's face may be created as needed by cutting out an image near the corresponding coordinates from an image near the corresponding time of the moving image based on the face detection metadata information in the moving image management file. This image recording apparatus includes a face detection engine, for example, so that a face can be detected at regular time intervals when a moving image is captured. The face image is detected by detecting whether there is a portion indicating the characteristics of the human face in the image, and if there is a face image, whether there is an increase or decrease in the number of face detections. Of course, face detection information may be recorded as metadata on the management file when there is an increase or decrease.

  Therefore, according to the image recording apparatus according to the embodiment of the present invention, a multi-angle recording DVD can be created by a single image recording apparatus. Then, it is possible to switch and reproduce the face-up moving image and the normal moving image included in the scene.

  Although one embodiment of the present invention has been described above, the present invention is not limited to this, and it is needless to say that various improvements and changes can be made without departing from the spirit of the present invention.

  For example, in addition to face detection, smile detection may be performed, and an image near the face related to the smile may be cut out and recorded as multi-angle as face angle image data. For example, when there are a plurality of faces, a method of giving higher priority to smiles in the faces can be considered.

  Also, the face is first identified and registered, and even if a plurality of faces are detected thereafter, only the image data of the face angle related to the registered face is preferentially recorded in multi-angle. Also good. Alternatively, if the face has already been taken up during normal shooting, it may be assumed that the face angle image data and the normal angle image data have substantially the same size. The processing load may be reduced by not recording the face angle image data. Even when the recording is not performed at the face angle, the processing load can be prevented from increasing by disabling the data indicating the blank.

  In addition to the above, control may be performed so that the child's face in the image is given priority.

1 is a configuration diagram of an image recording apparatus according to a first embodiment of the present invention. The figure which shows notionally the flow of the process of the multi-angle recording to DVD. The figure which shows notionally the flow of the other process of the multi-angle recording to DVD. The figure which shows the example of a screen display at the time of the reproduction | regeneration by an LCD panel. 6 is a flowchart for explaining in more detail a series of processes related to multi-angle recording by the image recording apparatus according to the embodiment of the present invention. 6 is a flowchart for explaining in more detail an improvement example of a series of processes related to multi-angle recording by the image recording apparatus according to the embodiment of the present invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Microprocessor, 2 ... Battery, 3 ... Power supply circuit, 4 ... AC power supply, 5 ... Power supply control part, 6 ... Key input circuit, 7 ... Imaging optical system, 8 ... CCD, 9 ... Signal processing circuit, 10 ... Face Detection engine, 11 ... recording circuit, 12 ... hard disk, 13 ... DVD drive, 14 ... on-screen display circuit, 15 ... superimposition circuit, 16 ... LCD touch panel, 100 ... image recording device

Claims (7)

When detecting whether a face image exists in the image related to the input image data, and detecting the face image, metadata relating to the detection timing, coordinates, and size of the face image in the image data Signal processing means for outputting together with image data;
The image data and the metadata are stored in a first storage means. During dubbing, the image data and the metadata are read from the first storage means, and a face image is extracted based on the metadata. An image recording apparatus comprising: face image data, and recording / reproducing means for recording the image data and the face image data on an optical recording medium. 2. The image recording according to claim 1, wherein the signal processing means includes face detection means, and the face detection means detects the presence of a face image in the image related to the image data at a predetermined time interval. apparatus. In the detection of the face image by the face detection means, it is detected whether or not there is a part indicating the feature of the person's face in the image, and if there is a face image, whether or not the number of face detection is increased or decreased. The image recording apparatus according to claim 2, wherein when the face detection is detected, the increase / decrease data is also recorded as metadata. An imaging unit that captures a subject image and outputs an imaging signal is provided.
The image recording apparatus according to claim 1, wherein the signal processing unit performs analog / digital conversion on the imaging signal and performs predetermined signal processing to generate image data. When detecting whether a face image exists in the image related to the input image data, and detecting the face image, metadata relating to the detection timing, coordinates, and size of the face image in the image data Signal processing means for outputting together with compressed image data;
The compressed image data and the metadata are stored in the first storage means, and when dubbing, the compressed image data and the metadata are read from the first storage means, and the compressed image data is decompressed. And a recording / reproducing unit that extracts face images from the expanded image data based on the metadata to generate face image data, compresses the image data and the face image data, and records the compressed image data on an optical recording medium. An image recording apparatus characterized by that. When detecting whether a face image exists in the image related to the input image data, and detecting the face image, metadata relating to the detection timing, coordinates, and size of the face image in the image data Outputting with compressed image data;
The compressed image data and the metadata are stored in the first storage means, and when dubbing, the compressed image data and the metadata are read from the first storage means, and the compressed image data is decompressed. A step of extracting a face image from the decompressed image data based on the metadata to generate face image data, compressing the image data and the face image data, and recording the compressed image data on an optical recording medium. An image recording method characterized by the above. Computer
When detecting whether a face image exists in the image related to the input image data, and detecting the face image, metadata relating to the detection timing, coordinates, and size of the face image in the image data Signal processing means for outputting together with compressed image data;
The compressed image data and the metadata are stored in the first storage means, and when dubbing, the compressed image data and the metadata are read from the first storage means, and the compressed image data is decompressed. The image data is extracted from the expanded image data based on the metadata to generate face image data, and the image data and the face image data are compressed and recorded on an optical recording medium. A program characterized by that.

Images