JP2008098755A - Imaging apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To securely recognize a target subject. <P>SOLUTION: An imaging unit 103 picks up the image of the subject and a face image detection unit 104 detects a face image from the picked-up image; and a superimposing unit 106 superimposes a face mark indicating a target to be imaged on the image while being made to correspond to a targeted face image as a face image to be picked up by an imaging person among the detected face images, and a display control unit 107 controls display of the image with the face mark superimposed on the image display unit 23, so that the targeted subject can securely be recognized. The present invention is applicable to a video camera. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and method, and more particularly, to an imaging apparatus and method that can surely grasp a subject.

  In recent years, video cameras have become widespread. In conventional video camera photography, a person is often imaged as an object to be imaged, and in particular, the majority of cases are where children are imaged. As a situation at the time of imaging in that case, for example, a scene with other children such as an athletic meet or a music concert occupies a considerable proportion of the whole. In such cases, it is difficult for parents taking video camera shots to find my child from among many children. In particular, when the clothes of children are the same, such as athletic meet, it is difficult for parents in the spectator seat to identify my child who is far away when the clothes are difficult to judge.

  As a technique effective in such a case, at least one of the horizontal width, vertical length, or area of the face portion is performed by performing skin color extraction or face template matching as an index indicating the characteristics of a person from a moving image. Based on the above, a gesture recognition method for recognizing a person's gesture has been proposed (see, for example, Patent Document 1).

JP 2003-216955 A

  However, the above-mentioned Patent Document 1 only discloses a technique for recognizing a person's gesture, and in a scene of shooting with a general video camera, the photographer can grasp the subject even if only the gesture is known. Was difficult.

  In other words, in the case of a general video camera, image recognition that is effective in many imaging scenes is to identify the person to be imaged and to grasp the movement of the target person, and it is effective image recognition only with gestures. I can't say that. In other words, it is effective to be able to determine at a glance the target person that the photographer wants to capture from among an unspecified number of people, and it is effective to keep focusing on a specific person during imaging.

  The present invention has been made in view of such a situation, and by providing an environment in which it is possible to identify an imaging subject that is meaningful to the photographer and to grasp the movement of the imaging subject, it is possible to reliably target the subject. It is to be able to grasp.

  An imaging apparatus according to an aspect of the present invention includes an imaging unit that captures an image of a subject in the imaging apparatus that displays the captured image of the subject, and a predetermined part of the subject from the captured image. A first detection means for detecting a part image, which is a blurred image, and an imaging target corresponding to a target part image which is a part image to be imaged by the photographer among the detected part images And superimposing means for superimposing a first icon indicating the image on the image, and display control means for controlling the display of the image on which the first icon is superimposed on the display.

  The part image may be an image of the face of the subject.

  According to an instruction from the photographer, a selection unit that selects the target part image from the detected part images is further provided, and the superimposing unit corresponds to the selected target part image, and One icon can be superimposed on the image.

  Storage control means for storing the selected target part image in a predetermined recording medium can be further provided.

  Reading means for reading an image including the target part image from a predetermined recording medium storing the target part image, reading the part image detected from the captured image, and reading from the recording medium Collating means for collating the target part image included in the image that is included, and the superimposing means corresponds to the part image that matches the target part image based on the result of the collation, and Can be superimposed on the image.

  The first icon may be an image having a circular, square, or elliptical shape.

  Second detection means for detecting movement of the imaging target is further provided, and the superimposing means superimposes a second icon indicating the state of operation of the imaging target on the image based on the detected movement. be able to.

  The second icon may be an image indicating a position where the imaging target exists.

  The second icon may be an image indicating a predetermined operation method of a zoom button operated to display the image to be imaged that is not displayed on the display on the display.

  An imaging method according to an aspect of the present invention is an imaging method of an imaging apparatus that displays an image of a captured subject on a display, captures the image of the subject, and includes a predetermined part of the subject from the captured image. A first icon that indicates that the image is a target image that is a target image that is a target image that is a target image that is a target image of a target image picked up by the photographer. Is superimposed on the image, and the display of the image on which the first icon is superimposed on the display is controlled.

  In one aspect of the present invention, an image of a subject is picked up, a part image that is an image including a predetermined part of the subject is detected from the picked-up image, and the photographer of the detected part images A first icon indicating that it is an imaging target is superimposed on the image so as to correspond to the target part image that is the part image of the imaging target, and the display on the display of the image on which the first icon is superimposed is controlled. The

  As described above, according to one aspect of the present invention, it is possible to reliably grasp the target subject. In particular, according to one aspect of the present invention, it is possible to reliably grasp the target subject by providing a function that can identify the imaging subject meaningful to the photographer and grasp the movement of the imaging subject. Can do.

  Embodiments of the present invention will be described below. Correspondences between constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

  An imaging apparatus according to an aspect of the present invention captures an image of a subject in an imaging apparatus (for example, video camera 1 in FIG. 2) that displays the captured image of the subject on a display (for example, the display unit 23 in FIG. 2). A part image (for example, a face image) that is an image including a predetermined part (for example, the face of the subject) of the subject from the captured image (for example, the imaging unit 103 in FIG. 2) A first detection means (for example, the face image detection unit 104 in FIG. 2) for detecting the image and an object that is a part image of an object to be imaged (for example, the person to be imaged) among the detected part images A superimposing unit (for example, the superimposing unit 106 in FIG. 2) that superimposes a first icon (for example, a face mark) indicating that it is an imaging target in correspondence with the part image, and the first icon Before superimposed Display control means for controlling the display to the display of the image (e.g., the display control unit 107 of FIG. 2) and a.

  The part image may be an image of a face of the subject (for example, a face image).

  In accordance with an instruction from the photographer, there is further provided selection means (for example, the face image selection unit 108 in FIG. 2) for selecting the target part image from the detected part images, and the superimposing means is selected. The first icon can be superimposed on the image in correspondence with the target part image (for example, the process of step S21 in FIG. 4 by the superimposing unit 106 in FIG. 2).

  Storage control means (for example, the recording medium control unit 110 in FIG. 2) for storing the selected target part image in a predetermined recording medium (for example, the recording medium 28 in FIG. 2) can be further provided.

  Reading means (for example, the recording medium control unit 110 in FIG. 2) for reading out an image including the target region image from a predetermined recording medium (for example, the recording medium 28 in FIG. 2) storing the target region image. Matching means for collating the part image detected from the captured image with the target part image included in the image read from the recording medium (for example, the face image matching unit 105 in FIG. 2) And the superimposing unit superimposes the first icon on the image in correspondence with the part image that matches the target part image based on the result of the collation (for example, the superimposing unit in FIG. 2). 106, the process of step S60 of FIG.

  The first icon may be an image having a circular, square, or elliptical shape.

  Second detection means (for example, the state detection unit 201 in FIG. 9) for detecting the movement of the imaging target is further provided, and the superimposing means determines the state of operation of the imaging target based on the detected movement. A second icon (for example, a status mark) to be displayed can be superimposed on the image (for example, the process of step S96 in FIG. 13 by the superimposing unit 106 in FIG. 9).

  The second icon may be an image (for example, a state mark 303a in FIGS. 10 and 11) indicating a position where the imaging target exists.

  The second icon is an image (for example, a status mark 303b in FIG. 19) indicating a predetermined operation method of the zoom button operated to display the image to be imaged that is not displayed on the display on the display. To 303e).

  An imaging method according to one aspect of the present invention is an imaging method of an imaging device (for example, video camera 1 in FIG. 2) that displays a captured image of a subject on a display (for example, the display unit 23 in FIG. 2). An image is captured (for example, the process in step S11 in FIG. 4), and a part image that is an image including a predetermined part of the subject is detected from the captured image (for example, in step S13 in FIG. 4). Processing), a first icon (for example, a face mark) indicating that it is an imaging target is associated with a target part image that is a part image to be picked up by a photographer among the detected part images. The image is superimposed on the image (for example, the process in step S21 in FIG. 4), and the display of the image on which the first icon is superimposed on the display is controlled (for example, the process in step S22 in FIG. 4). ) Includes the step.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an embodiment of a video camera 1 to which the present invention is applied.

  The video camera 1 is an example of an imaging device having a function of displaying a captured image in real time. In this embodiment, the video camera 1 will be described as an example. However, the present invention is not limited to this. For example, an imaging function such as a digital still camera, a mobile phone with an imaging function, or a personal computer with an imaging function is used. The present invention can be applied to a device having a function of displaying a displayed image.

  The video camera 1 includes an optical block 11, a lens iris driver 12, a CCD (Charge Coupled Devices) 13, a timing generation circuit 14, a CCD driver 15, a preprocessing circuit 16, a DSP (Digital Signal Processor) 17, an SDRAM (Synchronous Dynamic Random Access). Memory) 18, CPU (Central Processing Unit) 19, RAM (Random Access Memory) 20, Flash ROM (Read Only Memory) 21, display controller 22, display unit 23, input unit 24, drive controller 25, recording media interface 27, And a bus 29. In addition, for example, two recording media, a removable medium 26 and a recording medium 28, are appropriately attached to the video camera 1, respectively.

  The optical block 11 includes, for example, an optical lens, a focus mechanism, a shutter mechanism, a diaphragm mechanism (iris) mechanism, and the like, and collects (reflects) light reflected on the subject, that is, receives light of the subject image on the CCD 13. The image is formed on the part. The optical block 11 is driven by a lens iris driver 12.

  The lens iris driver 12 generates a drive signal for operating the optical block 11 in accordance with control from the CPU 19, supplies the generated drive signal to the optical block 11, and operates the optical block 11. For example, the lens iris driver 12 supplies the optical block 11 with a drive signal for adjusting the light amount by focusing or adjusting the diaphragm based on a user instruction when an image is captured.

  The CCD 13 is composed of a CCD sensor, and converts light into an analog electrical signal by photoelectrically converting light (image) imaged by the optical block 11 in accordance with a drive signal supplied from the CCD driver 15. . Based on the timing signal from the timing generation circuit 14 controlled by the CPU 19, the CCD 13 supplies image data, which is a converted analog signal, to the preprocessing circuit 116.

  The CCD 13 is not limited to a CCD sensor, and may be an image sensor that generates an image signal in units of pixels, such as a CMOS (Complementary Metal Oxide Semiconductor) sensor.

  The timing generation circuit 14 generates a timing signal that is a temporal reference for the operation of the CCD 13 based on the control by the CPU 19, and supplies the generated timing signal to the CCD 13 and the CCD driver 15.

  The CCD driver 15 generates a drive signal for driving the CCD 13 based on the timing signal supplied from the timing generation circuit 14 and supplies the generated drive signal to the CCD 13.

  The pre-processing circuit 16 performs CDS (Correlated Double Sampling) processing on the image data that is an analog signal supplied from the CCD 13 so as to maintain a good S / N (Signal / Noise) ratio and AGC. (Automatic Gain Control) processing is performed to control the gain. Then, the preprocessing circuit 16 performs A / D (Analog / Digital) conversion, converts an analog signal into a digital signal, and supplies image data that is a digital signal obtained by the conversion to the DSP 17. Note that image data supplied from the preprocessing circuit 16 to the DSP 17 is stored in the SDRAM 18.

  The CPU 19 controls the operation of each part of the video camera 1, such as the lens iris driver 12, the timing generation circuit 14, or the DSP 17. For example, when capturing an image of a subject, the CPU 19 controls the lens iris driver 12 to generate a drive signal for operating the optical block 11 and supplies the generated drive signal to the optical block 11. Then, the optical block 11 is operated.

  Further, the CPU 19 executes various processes in accordance with programs stored in the flash ROM 21 or the RAM 20, for example.

  The RAM 20 appropriately stores programs executed by the CPU 19 and various data. The flash ROM 21 records programs executed by the CPU 19 and various data. These CPU 19, RAM 20, and flash ROM 21 are connected to each other by a bus 29.

  The bus 29 is further connected to a DSP 17, a display controller 22, an input unit 24, a drive controller 25, and a recording media interface 27.

  The DSP 17 is composed of, for example, a dedicated DSP that is a dedicated IC (Integrated Circuit) or a general-purpose DSP.

  The DSP 17 applies image processing to image data that is a digital signal supplied from the preprocessing circuit 16. The DSP 17 includes, for example, an AF (Auto Focus) processing unit 41, an AE (Auto Exposure) processing unit 42, an AWB (Auto White Balance) processing unit 43, a compression / decompression unit 44, and the like, and is supplied from the preprocessing circuit 16. A predetermined camera signal process is applied to the image data, and the image data subjected to various adjustments is compressed by a predetermined compression method by the predetermined camera signal process.

  The DSP 17 includes an SDRAM controller 45. The SDRAM controller 45 controls the SDRAM 18 based on the program executed by the DSP 17 and various data, thereby temporarily storing the program and various data in the SDRAM 18.

  The SDRAM 18 appropriately stores programs executed by the DSP 17 and various data based on the control of the SDRAM controller 45. For example, the SDRAM 18 temporarily stores the image data supplied from the preprocessing circuit 16, so that the DSP 17 compresses predetermined camera signal processing and image data for the temporarily stored image data. Processing is applied to generate compressed image data.

  Specifically, for the image data supplied from the preprocessing circuit 16 stored in the SDRAM 18, the AF processing unit 41 performs a process (so-called autofocus) for focusing on the subject to perform an AE process. The unit 42 performs a process of determining exposure based on the brightness of the subject (so-called automatic exposure), and the AWB processing unit 43 corrects the entire image data based on a predetermined reference color that is white. (So-called white balance) is performed.

  The compression / decompression unit 44 compresses variously adjusted image data by a predetermined compression method by predetermined camera signal processing. For example, the compression / decompression unit 44 compresses image data using a JPEG (Joint Photographic Experts Group) method. Thereby, the image data can be recorded on the removable medium 26 after the data amount of the image data is reduced.

  Under the control of the CPU 19, the DSP 17 reads the compressed image data from the SDRAM 18 and supplies the image data to the drive controller 25 via the bus 29, thereby recording the image data on the removable medium 26 as a file.

  The drive controller 25 is configured by a predetermined interface corresponding to the removable medium 26, and causes the image data supplied from the DSP 17 to be recorded on the removable medium 26 or read out the image data recorded on the removable medium 26, and the DSP 17 Or it supplies to CPU19 grade | etc.,.

  The removable medium 26 is, for example, a tape (for example, a magnetic tape) or an optical disk (for example, a recordable DVD (Digital Versatile Disc)), and is attached to the video camera 1 as necessary. The removable medium 26 can also be configured as the same recording medium as a recording medium 28 described later. As the removable medium 26, for example, a magnetic disk such as a hard disk, a semiconductor memory such as a memory card, or a magneto-optical disk such as MD (Mini-Disc) (trademark) may be used.

  Further, the DSP 17 reads the target image data from the removable medium 26 via the bus 29 in response to an operation input from the user received through the input unit 24 such as a control key or the touch panel 52. Then, the compressed image data supplied from the removable medium 26 is subjected to predetermined decompression processing (decompression processing) according to the data compression, so that the image data is decompressed, and the decompressed image data is transferred to the bus. It is supplied to the display controller 22 via 29.

  That is, when the image data is read from the removable medium 26, the compression / decompression unit 44 converts the compressed image data supplied from the removable medium 26 via the bus 29 to a predetermined value according to the compression method. The image data is decompressed by performing the decompression process (decompression process). The expanded image data is supplied to the display controller 22 via the bus 29.

  The display controller 22 generates an image signal to be supplied to the display unit 23 based on the image data supplied from the DSP 17 and supplies the generated image signal to the display unit 23. The display controller 22 controls the display of the display unit 23 by supplying an image signal to the display unit 23.

  The display unit 23 displays an image corresponding to the image signal on the display screen based on the image signal supplied from the display controller 22. Note that the display form of the image follows the control program recorded in the flash ROM 21. That is, the display controller 22 determines a display form of an image to be displayed on the screen of the display unit 23 according to a control program executed by the CPU 19.

  The display unit 23 includes a color liquid crystal display unit 51 and a touch panel 52.

  The color liquid crystal display unit 51 includes, for example, an LCD (Liquid Crystal Display) panel, an organic EL (Electro Luminescence) panel, and the like, and displays an image corresponding to the image signal based on the image signal supplied from the display controller 22. To do. Thereby, for example, an image corresponding to the image data recorded on the removable medium 26 is displayed on the screen of the color liquid crystal display unit 51.

  The touch panel 52 superimposed on the color liquid crystal display unit 51 is configured to transmit the screen of the color liquid crystal display unit 51, and allows the user to select an operation screen key or the like displayed on the color liquid crystal display unit 51. The CPU 19 receives the signal indicating the position information indicating the operation position. The CPU 19 controls the operation of each unit of the video camera 1 based on the signal supplied from the touch panel 52.

  The recording medium interface 27 includes a predetermined interface corresponding to the recording medium 28, records data supplied from the DSP 17 or the CPU 19, or reads out data recorded on the recording medium 28, and the DSP 17 or the CPU 19 or the like. To supply.

  The recording medium 28 is a memory card such as a Memory Stick (trademark), for example, and is attached to the video camera 1 as necessary. Of course, the recording medium 28 can be configured as the same recording medium as the removable medium 26.

  Although not shown in the figure, in the video camera 1, predetermined processing is performed on the audio signal captured from the microphone, and the audio data obtained by performing the predetermined processing is, for example, , Recorded on the removable medium 26. In other words, the data captured and recorded by the video camera 1 is generally composed of audio data and the like in addition to the image data, but in this embodiment, in order to simplify the explanation, video is recorded. Only the image data to be displayed will be described.

  Further, in the example of FIG. 1 described above, the video camera 1 has been described by taking as an example a configuration in which the recording medium interface 27 is provided and the recording medium 28 is attached thereto, but the data to be recorded on the recording medium 28 is removable media. It is also possible to adopt a configuration in which the recording media interface 27 is not provided by recording in the recording medium 26. That is, the recording medium attached to the video camera 1 may be only the removable medium 26. In this case, the data recorded on the recording medium 28 is recorded on the removable medium 26.

  Furthermore, the hardware configuration of the video camera 1 is not limited to the example of FIG. 1, and it is sufficient that it has at least the functional configuration of FIG.

  FIG. 2 is a block diagram illustrating an example of a functional configuration of the video camera 1 of FIG.

  The video camera 1 includes a control unit 101, an operation unit 102, an imaging unit 103, a face image detection unit 104, a face image collation unit 105, a superimposition unit 106, a display control unit 107, a face image selection unit 108, a face image storage unit 109, The recording medium control unit 110, the face image detection unit 111, the display unit 23, and the recording medium 28 are included.

  In FIG. 2, the same parts as those in FIG. 1 are denoted by the same reference numerals, and the description of the same parts will be omitted because they are repeated.

  In the present embodiment, since the video camera 1 has the hardware configuration shown in FIG. 1 described above, the control unit 101 corresponds to, for example, the CPU 19 shown in FIG. Corresponds to the input unit 24 or the touch panel 52. The imaging unit 103 corresponds to, for example, the optical block 11 to the preprocessing circuit 16 in FIG. 1, and the face image detection unit 104, the face image collation unit 105, the superimposition unit 106, and the face image detection unit 111 are, for example, This corresponds to the DSP 17 in FIG.

  Furthermore, the face image selection unit 108 corresponds to, for example, the touch panel 52 of FIG. 1, the display control unit 107 corresponds to, for example, the display controller 22 of FIG. 1, and the face image storage unit 109 corresponds to, for example, FIG. The recording medium control unit 110 corresponds to, for example, the recording medium interface 27 of FIG.

  However, the face image detection unit 104, the face image collation unit 105, the superimposition unit 106, the display control unit 107, the recording media control unit 110, or the face image detection unit 111 executes, for example, software (computer program) in FIG. It may be realized by the CPU 19. That is, the face image detection unit 104, the face image collation unit 105, the superimposition unit 106, the display control unit 107, the recording media control unit 110, or the face image detection unit 111 can be configured as hardware or configured as software. It can also be configured as a combination of software and hardware.

  The control unit 101 controls the operation of each unit of the video camera 1 in FIG. 2 based on an operation signal supplied from the operation unit 102 according to the operation of the photographer.

  The imaging unit 103 images a subject. The imaging unit 103 supplies the subject image obtained by the imaging to the face image detection unit 104 and the superimposition unit 106.

  In the following description, in the present embodiment, the simple description “image” includes both the meaning of the image itself and the image data for displaying the image.

  The face image detection unit 104 performs face image detection processing for detecting a face image from the image supplied from the imaging unit 103. The face image detection unit 104 supplies the detection result obtained by performing the face image detection process to the face image matching unit 105.

  Although details will be described later, the face image detection process is a process for detecting a human face image from the captured subject image. That is, the face image detection unit 104 detects a face image of a person shown in the image from the image captured by the image capturing unit 103 by executing a face image detection process.

  The face image selection unit 108 selects the face image of the person to be imaged from the images of the subject displayed on the display unit 23 based on, for example, an operation on the touch panel 52 by the imager. The face image selection unit 108 supplies the selected face image of the person to be imaged (or information on the face image) to the face image storage unit 109.

  Note that the face image selection unit 108 is not limited to an operation performed on the touch panel 52 by the photographer, for example, an imager operated on a focused face image in the subject image displayed on the display unit 23. The face image of the person to be imaged may be selected in response to the key input by. In short, it is only necessary to select the face image of the person to be imaged, and the means and method thereof are arbitrary.

  The face image storage unit 109 stores the face image (or information regarding the face image) of the person to be imaged captured by the image capturing unit 103 supplied from the face image selection unit 108. The face image storage unit 109 supplies the stored face image of the person to be imaged to the face image matching unit 105 in response to a request from the face image matching unit 105.

  The face image storage unit 109 stores a face image database 109a, and the face image of the person to be imaged and information related to the face image are registered in the face image database 109a. The face image storage unit 109 temporarily stores the face image of the person to be imaged without registering it in the face image database 109a, and supplies the temporarily stored face image to the face image matching unit 105. May be.

  The face image matching unit 105 performs face image matching processing based on the detection result of the face image supplied from the face image detection unit 104 and the face image of the person to be imaged read from the face image storage unit 109. The face image matching unit 105 supplies the matching result of the face image obtained by performing the face image matching process to the superimposing unit 106.

  Although details will be described later, the face image collation processing is performed by collating the face image detected from the captured subject image with the face image of the person to be imaged, and capturing the image from the detected face image. This is processing for specifying the face image of the subject.

  That is, the face image matching unit 105 is captured by collating the detection result of the face image supplied from the face image detecting unit 104 with the face image of the person to be imaged registered in the face image database 109a. The face image of the person to be imaged is specified from the face images of the subject image. As a result, the face image registered in the face image database 109a is specified from the captured image of the subject.

  The superimposing unit 106 is based on the face image matching result supplied from the face image matching unit 105, and in the subject image supplied from the imaging unit 103, the face image of the person to be imaged identified by the face image matching unit 105. A face mark is superimposed on. The superimposing unit 106 supplies an image of the subject on which the face mark is superimposed to the display control unit 107. Note that the superimposition unit 106 supplies the image supplied from the imaging unit 103 to the display control unit 107 as it is without superimposing the face mark when the collation result indicates that the face mark is not superimposed. .

  Here, the face mark is a mark for making the imaging subject recognize the imaging subject. For example, the face mark is formed of an icon having a circular, square, or elliptical shape, and the icon having the shape is superimposed on the face image of the person to be imaged.

  The display control unit 107 causes the display unit 23 (the color liquid crystal display unit 51) to display the image of the subject with the face mark superimposed supplied from the superimposition unit 106. Note that the display control unit 107 causes the display unit 23 to display an image of a subject on which no face mark is superimposed when an image on which no face mark is superimposed is supplied from the superimposition unit 106.

  The recording media control unit 110 reads the face image (or information related to the face image) of the person to be imaged registered in the face image database 109 a of the face image storage unit 109 and records the face image on the recording medium 28. Thereby, the recording medium 28 records the face image of the person to be imaged, information on the face image of the person to be imaged, and the like.

  The recording media control unit 110 reads an image (or a face image) recorded on the recording medium 28 and includes the face image of the person to be imaged, and supplies the image to the face image detection unit 111.

  The face image detection unit 111 performs a face image detection process for detecting a face image from an image including the face image of the person to be imaged supplied from the recording media control unit 110. The face image detection unit 111 supplies the detection result obtained by performing the face image detection process to the face image storage unit 109 and registers it in the face image database 109a. Thereby, the face image of the person to be imaged included in the image recorded on the recording medium 28 is registered in the face image database 109a.

  That is, in the face image database 109a, the face image of the subject to be imaged selected by the photographer from the image of the subject or the face image of the subject to be imaged included in the image recorded on the recording medium 28 is registered. Therefore, the face image collation unit 105 identifies the face image of the person to be imaged by collating those face images with the face image detected from the captured subject image.

  When the image recorded on the recording medium 28 is a face image, the face image detection unit 111 does not perform the face image detection process by the face image detection unit 111 and uses the face image read from the recording medium 28 as it is. It is registered in the face image database 109a.

  Further, in the present embodiment, for convenience of explanation, the face image detection unit 104 and the face image detection unit 111 are described as separate blocks. Of course, these blocks may be the same block.

  FIG. 3 is a diagram for explaining an outline of a method for registering a person to be imaged.

  In the video camera 1, as shown in FIG. 3, when an image 151 is captured by a photographer, the image 151 is displayed on the display unit 23 in real time. That is, the imager captures the subject corresponding to the image 151 while viewing the image 151 displayed on the display unit 23 in real time.

  For example, as shown in FIG. 3, when shooting a sports day with the video camera 1, the parent operating the video camera 1 checks the image 151 displayed on the display unit 23 in order to take an image of my child. However, it is difficult to find my child among many children wearing similar exercise clothes. Therefore, the video camera 1 detects a face image from the captured image of the subject, and superimposes a face mark 151a on the imaged person such as his / her child from the detected face image. For example, it becomes possible to intuitively identify a person to be imaged such as his child with respect to an imager such as a parent who images his child.

  That is, in the example of FIG. 3, in the image 151 displayed on the display unit 23 having the touch panel 52, for example, the person to be imaged such as his / her child touches the image 151. The face mark 151a is superimposed on the subject to be imaged. Incidentally, in the image 151 displayed on the display unit 23, an image of a child who is the imaging target person surrounded by the face mark 151a is touched by the imaging person.

  If the image is being captured, the face mark 151a is superimposed on the image of the child who is the subject of the image capture, so that the parent who is the image capturer captures the face mark 151a to capture the image of the athletic meet. Even in a situation where there is a high possibility of losing sight of such a child, it is possible to reliably capture an image of the child who is the imaging subject without departing from the image frame.

  In the video camera 1, the face image 152 (or an image including the face image 152) of the subject to be imaged selected by the photographer can be recorded on the recording medium 28. That is, by recording the face image 152 on the recording medium 28, for example, when the same person to be imaged is captured again, the video camera 1 does not perform an operation of selecting the face image to be imaged. The face image 152 recorded in the recording medium 28 can be read out, the person to be imaged can be specified from the face image 152, and the face mark 151a can be superimposed on the face image of the person to be imaged. In addition, by attaching the recording medium 28 to another video camera, the face image that becomes the face image 152 is identified from the captured images in the other video camera as well, and the face image is identified. On the other hand, a face mark can be superimposed.

  Next, the specific processing flow will be described.

  FIG. 4 is a flowchart for explaining the face mark superimposing process executed by the video camera 1 of FIG.

  In step S <b> 11, the imaging unit 103 images a subject, and supplies an image of the subject obtained by the imaging to the face image detection unit 104 and the superimposition unit 106.

  In step S <b> 12, based on the operation signal from the operation unit 102, the control unit 101 determines whether or not superimposition of the face mark is instructed, for example, when the photographer presses a button that instructs superimposition of the face mark. Determine whether.

  If it is determined in step S <b> 12 that superimposition of a face mark is instructed, in step S <b> 13, the face image detection unit 104 performs face image detection processing for detecting a face image from the image supplied from the imaging unit 103. .

  Here, the details of the face image detection process executed by the face image detection unit 104 corresponding to the process of step S13 will be described with reference to the flowchart of FIG.

  In step S <b> 31, the face image detection unit 104 detects a face image from the image supplied from the imaging unit 103.

  Here, a face image detection method is used. Generally, a method is used in which a human eye is identified from the image and the identified eye is a face image. In the case of this method, the face image detection unit 104 performs gray scale processing on the image supplied from the imaging unit 103, and determines the black-and-white part of the eye based on the area ratio, the shape, and the like from the obtained image. To detect.

  In step S <b> 32, the face image detection unit 104 extracts feature points that are facial organs (main parts) other than the eyes from the image supplied from the imaging unit 103.

  Here, as a feature point extraction method, the face image detection unit 104 detects, for example, a nose, a mouth, an eyebrow, an ear, or a hair that is a facial organ other than the eyes detected in the process of step S31. Thus, feature points are extracted. Note that although the organ detection method is used, it is not necessary to detect images of all organs. For example, only a part of an image of an organ may be detected, such as detecting only images of eyes and mouth. The face image detection unit 104 may detect a face image by performing skin color determination using a color filter and detecting a human skin image.

  There are various face image detection methods, and any method may be used as long as the face image can be detected from a predetermined image. For example, since the human head has an elliptical shape, it may be used to detect a face image. Specifically, the face image detection unit 104 detects a face image by detecting an ellipse image from the image supplied from the imaging unit 103 and using the ellipse image as a head. To do. Furthermore, as a method for detecting a face image, in addition to detecting an image having an elliptical shape, for example, a method for detecting an image having a shape similar to the contour line on the lower side of the face may be used. Is possible. In short, it is only necessary to detect a face image from a predetermined image, and the means and method thereof are arbitrary.

  Returning to the flowchart of FIG. 4, in step S <b> 14, the display control unit 107 causes the display unit 23 to display the image of the subject supplied from the imaging unit 103 via the superimposing unit 106. At this time, since the face image of the person to be imaged is not selected by the photographer, the display unit 23 displays the image of the subject that is supplied by the superimposing unit 106 and has no face mark superimposed thereon. As a result, the photographer captures an image while viewing the image of the subject displayed on the display unit 23.

  In step S <b> 15, the face image selection unit 108 selects the face image of the person to be imaged from the images of the subject displayed on the display unit 23 based on the operation on the touch panel 52 by the photographer.

  If it is determined in step S12 that the face mark superimposition has not been instructed, the process of step S13 is skipped, and the process proceeds to step S14. In this case, in step S15, the image displayed on the display unit 23 is not instructed to superimpose a face mark, and the face image detection process (the process of step S13) is not performed. The face image of the person to be imaged cannot be selected from the images displayed on the display unit 23.

  In step S <b> 16, the face image selection unit 108 determines whether or not the face image of the person to be imaged has been selected by the photographer.

  If it is determined in step S16 that the face image of the person to be imaged has been selected by the image pickup person, the face image storage unit 109 displays the face image of the object to be picked up selected by the face image selection unit 108 in step S17. Register in the image database 109a. As a result, the face image of the person to be imaged selected by the face image selection unit 108 is registered in the face image database 109a.

  In step S <b> 18, the control unit 101 selects the button based on the operation signal from the operation unit 102 by, for example, pressing a button that instructs the imager to record the face image of the person to be imaged on the recording medium 28. It is determined whether or not an instruction to record the face image of the person to be imaged is instructed.

  If it is determined in step S18 that recording of the face image of the selected person to be imaged has been instructed, in step S19, the recording media control unit 110 causes the face image of the person to be imaged registered in the face image database 109a. Is recorded on the recording medium 28. Thereby, since the face image of the person to be imaged selected by the photographer is recorded on the recording medium 28, the photographer can attach the recording medium 28 to another video camera, for example. It is possible to superimpose a face mark on the face image of the person to be imaged.

  On the other hand, if it is determined in step S18 that recording of the face image of the selected person to be imaged is not instructed, the process of step S19 is skipped, and the process proceeds to step S20. That is, in this case, the face image of the person to be imaged is not recorded on the recording medium 28.

  In step S <b> 20, the face image matching unit 105 performs face image matching processing and supplies the face image matching result to the superimposing unit 106. Specifically, the face image matching unit 105 is picked up by matching the detection result of the face image from the face image detecting unit 104 with the face image of the person to be imaged registered in the face image database 109a. The face image of the subject to be imaged selected by the photographer is specified from the face images of the subject image. Thereby, the same face image as the face image of the person to be imaged selected by the photographer is specified from among the captured images of the subject.

  In step S <b> 21, the superimposing unit 106 determines the face of the person to be imaged identified by the face image matching unit 105 among the images displayed on the display unit 23 based on the face image matching result from the face image matching unit 105. A face mark is superimposed on the image. The superimposing unit 106 supplies the image on which the face mark is superimposed to the display control unit 107.

  In step S <b> 22, the display control unit 107 causes the display unit 23 to display an image of the subject on which the face mark supplied from the superimposing unit 106 is superimposed.

  That is, as shown in FIG. 3, in the image 151 displayed on the display unit 23, when the parent who is the photographer selects his / her child as the person to be imaged, the display control unit 107 selects the selected image. The image 151 in which the face mark 151 a is superimposed on the face image of the child is displayed on the display unit 23. Thus, for example, a parent who has captured the state of an athletic meet with the video camera 1 surely follows his or her child on which the face mark 151a is superimposed from among a large number of children. Imaging can be performed.

  Moreover, although it is the shape of the face mark 151a superimposed on the face image of a person to be imaged, various shapes can be used as shown in FIG.

  That is, when a child as shown in the example of FIG. 6 is an imaging subject, as shown on the left side of FIG. 6, for example, the imaging subject of the image 171 displayed on the display unit 23 (in the example of FIG. 6). In addition to the circular shape (face mark 171a in FIG. 6) superimposed around the face of “child”), a square shape (face mark 171b in FIG. 6) as shown in the center of FIG. 6 may have an elliptical shape (face mark 171c in FIG. 6) or the like. Further, although not shown, in addition to a circle, a rectangle, or an ellipse, for example, an alternate long and short dash line or a wavy line may be used, or a triangle, a diamond shape, a rhombus, or the like may be used. In addition, when there are a plurality of persons to be imaged, the coloring may be changed for each person to be imaged. That is, the face mark 151a only needs to allow the imager to grasp the image of the person to be imaged from among the images captured, and the shape, color, size, and the like can be arbitrarily changed.

  Returning to the flowchart of FIG. 4, in step S <b> 23, based on the operation signal from the operation unit 102, the control unit 101 gives an instruction to end imaging when, for example, a photographer presses a button that instructs to stop imaging. It is determined whether or not it has been done.

  If it is determined in step S23 that the end of imaging has not been instructed, the process returns to step S11, and the above-described processes of steps S11 to S23 are repeated. That is, until it is determined in step S23 that the end of imaging has been instructed, in steps S11 to S23, the process of superimposing the face mark on the face image of the person to be imaged is repeated. Imaging is performed while following the face mark superimposed on the face image.

  On the other hand, if it is determined in step S23 that the end of imaging has been instructed, the face mark superimposing process in FIG. 4 ends.

  As described above, the video camera 1 can display the face mark superimposed on the face image of the person to be imaged selected by the imager. Thereby, since the photographer can grasp the specific person in the image displayed on the display unit 23, the desired person can be imaged without missing the imaging opportunity.

  In the above-described example, in the process of step S18, the control unit 101 determines based on the operation signal from the operation unit 102 whether or not the recording of the face image of the selected person to be imaged has been instructed. As described above, the face image of the person to be imaged selected by the photographer is forcibly recorded on the recording medium 28 by proceeding to the process of step S19 without performing the determination process of step S18. Good.

  By the way, in the above-described example, the face mark is displayed so as to be superimposed on the face image of the person to be imaged selected by the photographer. However, in the present embodiment, for example, the image is captured by a digital still camera. If the same person as the face image of the person to be imaged read from the recording medium 28 exists in the imaged subject using the image recorded on the recording medium 28, It is also possible to superimpose a face mark on the face of the person.

  Therefore, referring to FIGS. 7 and 8, instead of selecting the face image to be imaged from the captured images as described above, from the images recorded on the recording medium 28. A process for specifying a face image of a person to be imaged and superimposing a face mark on the person to be imaged will be described.

  FIG. 7 is a diagram for explaining an outline of a method for registering a person to be imaged using an image including a face image recorded on the recording medium 28.

  As shown in FIG. 7, when a recording medium 28 that records an image 192 including a face image of the person to be imaged (image of a child in the example of FIG. 7) is attached to the video camera 1, the video camera 1 reads an image 192 recorded on the recording medium 28. In the video camera 1, when the same person as the face image included in the image 192 read from the recording medium 28 exists in the captured image 191, the face mark 191 a is superimposed on the face of the person. The image 191 on which the face mark 191 a is superimposed is displayed on the display unit 23.

  As a result, the person to be imaged is specified from the image read from the recording medium 28, so that the imager can search and register the face image of the person to be imaged from the image displayed on the display unit 23 at the time of imaging. Therefore, it is possible to quickly and surely image the person to be imaged.

  Next, the specific processing flow will be described.

  FIG. 8 is a flowchart for explaining the face mark superimposing process using the face image included in the image recorded on the recording medium 28, which is executed by the video camera 1 of FIG.

  In step S51, as in the process of step S11 of FIG. 4, the subject is imaged by the imaging unit 103, and an image of the subject obtained thereby is supplied to the face image detecting unit 104 and the superimposing unit 106.

  In step S <b> 52, based on the operation signal from the operation unit 102, the control unit 101 determines whether or not superimposition of the face mark is instructed by, for example, pressing a button for superimposing the face mark by the photographer. Determine whether.

  If it is determined in step S52 that superimposition of the face mark is instructed, in step S53, the control unit 101 mounts the recording medium 28 that records the image including the face image on the recording medium interface 27. It is determined whether or not it has been done.

  When it is determined in step S53 that the recording medium 28 that records the image including the face image is attached, the recording medium control unit 110 reads the image from the attached recording medium 28 in step S54. And the image is supplied to the face image detection unit 111.

  In step S <b> 55, the face image detection unit 111 performs face image detection processing for detecting a face image from the image supplied from the recording media control unit 110, and supplies the face image detection result to the face image storage unit 109. Thereby, the face image of the person to be imaged is detected from the images recorded on the recording medium 28.

  The details of the face image detection process of FIG. 5 corresponding to the process of step S55 executed by the face image detection unit 111 are the same as the process of step S13 of FIG. 4 by the face image detection unit 104 described above. Therefore, the description thereof is omitted.

  In step S <b> 56, the face image storage unit 109 displays the face image of the imaging target person included in the image read from the recording medium 28 and detected by the face image detection unit 111. Register in the image database 109a. Thus, the face image of the person to be imaged detected from the image recorded on the recording medium 28 is registered in the face image database 109a.

  In step S57, the face image detection unit 104 performs face image detection processing for detecting a face image from the image captured by the image capturing unit 103 in the same manner as in step S13 of FIG. The image is supplied to the face image matching unit 105. Thereby, a face image is detected from the image of the subject imaged by the imaging unit 103.

  The details of the face image detection process of FIG. 5 corresponding to the process of step S57 executed by the face image detection unit 104 are the same as the process of step S13 of FIG. 4 by the face image detection unit 104 described above. Therefore, the description thereof is omitted.

  In step S <b> 58, the face image matching unit 105 performs face image matching processing, and supplies the face image matching result to the superimposing unit 106. Specifically, the face image matching unit 105 is picked up by matching the detection result of the face image from the face image detecting unit 104 with the face image of the person to be imaged registered in the face image database 109a. The face image of the person to be imaged recorded on the recording medium 28 is specified from the face images of the subject image. Thereby, the face image of the same person as the face image of the image recorded on the recording medium 28 is specified from the captured image of the subject.

  In step S <b> 59, the face image matching unit 105 identifies the face image of the person to be imaged included in the image recorded on the recording medium 28 from the face images of the captured subject image by the face image matching process. It is determined whether or not it has been done.

  When it is determined in step S59 that a face image has been specified, in step S60, the superimposing unit 106 displays an image displayed on the display unit 23 based on the face image matching result supplied from the face image matching unit 105. Among them, the face mark is superimposed on the face image of the person to be imaged identified by the face image matching unit 105. The superimposing unit 106 supplies the image on which the face mark is superimposed to the display control unit 107.

  In step S <b> 61, the display control unit 107 causes the display unit 23 to display an image of the subject on which the face mark supplied from the superimposing unit 106 is superimposed.

  That is, as shown in FIG. 7, in the video camera 1, a face image of a child who is an imaging target included in an image 192 recorded on the recording medium 28 from an image 191 displayed on the display unit 23. When the matching face image is specified, the display control unit 107 causes the display unit 23 to display an image 191 in which the face mark 191a is superimposed on the specified child face image. Thereby, for example, the parent who is capturing the state of the athletic meet with the video camera 1 follows his / her child on which the face mark 191a is superimposed from the image 191 displayed on the display unit 23, thereby You can keep shooting your child who is always the subject of imaging.

  On the other hand, if it is determined in step S59 that the face image is not specified, the process of step S60 is skipped, and the process proceeds to step S61. In step S <b> 61, the display control unit 107 causes the display unit 23 to display an image of the subject on which the face mark supplied from the superimposing unit 106 is not superimposed. That is, at this time, the face image is not specified by the face image collation process, and the face image of the person to be imaged included in the image recorded on the recording medium 28 does not exist in the captured subject image. Therefore, an image of the subject on which no face mark is superimposed is displayed on the display unit 23.

  In step S62, as in step S23 of FIG. 4, it is determined whether or not the end of imaging has been instructed by the control unit 101. If it is determined that the end of imaging has not been instructed, the processing in steps S51 to S62 is performed. Repeated. That is, the process of superimposing the face mark on the face image of the person to be imaged is repeated in steps S51 to S62 until it is determined in step S62 that the end of imaging has been instructed. The desired person to be imaged is imaged while following the face mark superimposed on.

  On the other hand, if it is determined in step S62 that the end of imaging has been instructed, the face mark superimposing process in FIG. 8 ends.

  As described above, the video camera 1 can display the face mark superimposed on the face image of the person to be imaged included in the image read from the attached recording medium 28. As a result, the photographer can know the movement of the person who is the imaging subject, and thus can always take an image centering on the person who is the imaging subject.

  In the above-described example, when the superimposition of the face mark is instructed by the process of step S52, the process of reading the image including the face image of the person to be imaged from the recording medium 28 and superimposing the face mark is performed. However, the present invention is not limited to this. For example, the process of step S52 may be omitted, and the process of superimposing a face mark may be performed when the recording medium 28 is attached to the video camera 1. Of course, in the case of this example, when the recording medium 28 is not attached to the video camera 1, the process of superimposing the face mark is not performed.

  By the way, for the photographer, it is ideal that the subject to be imaged is continuously taken. However, the subject to be imaged such as a child is always moving, and the image is taken while holding the video camera 1 in his / her hand. Therefore, there is a possibility that the person to be imaged will be out of the image frame and lost sight. In this case, as described above, even if an attempt is made to superimpose a face mark on the face image of the person to be imaged, the person to be imaged itself does not exist in the image displayed on the display unit 23 (out of the image frame). Because of this, face marks cannot be superimposed. Therefore, in the above-described example, the face mark is superimposed on the face of the person who is the imaging target in the captured image, but an icon indicating the direction of the position where the imaging target exists. By superimposing (hereinafter, referred to as a status mark), even if the imaging subject is lost sight from the screen, it becomes possible to quickly confirm the location of the imaging subject using the status mark. Therefore, the process of superimposing the state mark on the captured image will be described next with reference to FIGS.

  FIG. 9 is a block diagram illustrating another example of the functional configuration of the video camera 1.

  In FIG. 9, the same parts as those in FIG. 2 are denoted by the same reference numerals, and the description of the same parts will be omitted because they are repeated. That is, in FIG. 9, the video camera 1 is provided with a state detection unit 201 in addition to the video camera 1 of FIG. 2.

  The state detection unit 201 performs state detection processing for detecting the state of the person to be imaged registered in the face image database 109a from the image supplied from the image pickup unit 103. The state detection unit 201 supplies the detection result obtained by performing the state detection process to the superposition unit 106.

  Although details will be described later, the state detection process is a process of detecting the state of the imaging target person by a process of estimating the position of the imaging target person who has deviated from the image frame.

  The superimposing unit 106 superimposes a state mark indicating the state of the imaging subject on the subject image supplied from the imaging unit 103 based on the detection result of the imaging subject supplied from the state detection unit 201. The superimposing unit 106 supplies an image of the subject on which the state mark is superimposed to the display control unit 107. Further, as described above, the superimposing unit 106 is supplied from the imaging unit 103 when the person to be imaged exists in the image frame based on the face image matching result supplied from the face image matching unit 105. A face mark is superimposed on the face image of the person to be imaged in the subject image.

  Based on the image supplied from the superimposing unit 106, the display control unit 107 causes the display unit 23 to display an image of the subject on which the face mark or the state mark is superimposed.

  Next, an outline of the status mark will be described with reference to FIGS.

  In the video camera 1, as illustrated in FIG. 10, when a subject is imaged by the photographer, an image 301 of the captured subject is displayed on the display unit 23. In this image 301, a face mark 302a indicating that the person is an imaging target and a state mark 303a indicating the state of the imaging target person on which the face mark 302a is superimposed are superimposed.

  That is, in the example of FIG. 10, the photographer images a soccer game with the video camera 1, but registers (selects) a player who is paying attention in the game as an imaging target person. A face mark 302a is superimposed on the selected player. The state mark 303a indicates the state of the selected person to be imaged, that is, the state of the player on which the face mark 302a is superimposed. In the example of FIG. It is present in the.

  Specifically, in the state mark 303a, arrows indicating four directions, up, down, left, and right, are attached in a circle indicating the presence / absence of the person to be imaged. When it has deviated, the direction of the position where the person to be imaged is present is indicated. That is, the status mark 303a indicates that there is a person to be imaged in the direction of the arrow indicated by a solid line and there is no person to be imaged in the direction of the arrow indicated by a dotted line. That is, in the example of FIG. 10, since there are players (capturing subjects) with the face mark 302 a superimposed in the image 301, the circle of the state mark 303 a is indicated by a solid line, and the upper, lower, left, and right in the circle These four arrows are marked with dotted lines.

  In the example of FIG. 11, there is no player (imaging target person) on which the face mark 302 a is superimposed in the image 301, and the player exists at the left position of the image frame. In the circle of the mark 303a, only the left arrow is indicated by a solid line, and the other three arrows are indicated by dotted lines. In the example of FIG. 11, since there is no subject to be imaged in the image 301, a circle indicating the presence or absence of the subject to be imaged is indicated by a dotted line in the status mark 303a.

  In the state mark 303a, the way of using the solid line and the dotted line as described above is the same in other figures described later.

  Hereinafter, the state where the person to be imaged is present in the image frame in the state mark is referred to as “existing state”. At this time, the state mark is indicated by a solid circle and a four-direction arrow by a dotted line. Is done. Furthermore, a state where the person to be imaged is present at the left position of the image frame is referred to as a “left movement state”, and similarly, a state where the subject is present at the right direction is referred to as a “right movement state”. The existing state is referred to as an “upward movement state”, and the state existing in a downward direction is referred to as a “downward movement state”. At this time, in the state mark, the four arrows in the circle indicating the presence / absence of the imaging target person are indicated by solid lines only in the direction of the position where the imaging target person exists, and the other three directions are dotted lines. Will be written.

  As described above, the video camera 1 is configured such that even when the imaging target person does not exist in the screen displayed on the display unit 23, the imaging target person is out of the image frame by the status mark. It is possible to indicate the direction of the position where the person to be imaged is out of the image frame. Thereby, even if the person to be imaged is off the screen of the display unit 23, the imager confirms the direction of the position where the person to be imaged exists by the status mark, and the video camera 1 in the direction of the arrow of the status mark. By moving, it becomes possible to capture the person to be imaged within the image frame.

  Next, the specific processing flow will be described.

  FIG. 12 is a flowchart for explaining the face / state mark superimposing process executed by the video camera 1 of FIG.

  In step S71, similar to the processing in step S51 of FIG. 8, the subject is imaged by the imaging unit 103, and the image obtained thereby is supplied to the face image detection unit 104, the superimposition unit 106, and the state detection unit 201. Is done.

  In step S <b> 72, based on the operation signal from the operation unit 102, the control unit 101, for example, presses a button that instructs the imager to superimpose the face mark and the state mark, thereby superimposing the face mark and the state mark. Whether or not is instructed is determined. If it is determined in step S72 that superimposition of the face mark and the status mark is instructed, the process proceeds to step S73.

  In steps S73 to S80, as in steps S53 to S60 of FIG. 8, the face image detection unit 104, the face image collation unit 105, the superimposition unit 106, the face image storage unit 109, the recording media control unit 110, and the face image detection unit Based on the image recorded on the recording medium 28, the person to be imaged is identified from among the captured images of the subject, and a face mark is superimposed on the person to be imaged.

  In step S81, the imaging unit 103, the superimposing unit 106, and the state detecting unit 201 detect the state of the person to be imaged from the image captured by the imaging unit 103, and superimpose a state mark on the image. To do.

  Here, the details of the state detection process executed by the imaging unit 103, the superimposing unit 106, and the state detection unit 201 corresponding to the process of step S81 will be described with reference to the flowchart of FIG.

  In step S91, the state detection unit 201 reads the face image of the person to be imaged registered in the face image database 109a to identify the person to be imaged.

  In step S92, the imaging unit 103 focuses on the person to be imaged.

  In step S <b> 93, the state detection unit 201 determines a position where the specified imaging subject exists.

  Specifically, as illustrated in FIG. 14, for example, in the image of the Nth frame, the imaging subject who was present at the position P (6, 5) is the N + 1th frame, which is the next frame. In the image, when moving to the position Q (10, 8), the moving speed of the imaging subject who has moved from the position P to Q can be obtained from the moving distance from the imaging subject person position P to Q. .

  In the example of FIG. 14, one frame of this image has (X, Y) = (0,0) when the horizontal direction in the figure is the X direction and the vertical direction is the Y direction in order to simplify the description. 0), (20, 0), (0, 15), and (20, 15) have four corners, that is, 15 × 20 blocks. Also, in the image of one frame, the position of the person to be imaged is, for example, a position such as the center of gravity of the face of the person to be imaged.

  That is, when the focal length when focusing on the point X is Xf, the frame rate is fr (fps), the horizontal (horizontal) direction coordinate is x, and the vertical (vertical or depth) direction coordinate is y, As shown in FIG. 15, the focal length when the focus is set at the position Q where the person to be imaged is Qf (= F (expressed as “F” in the example of FIG. 15)).

  Further, the moving distance of the position Q of the person to be imaged per frame (that is, the moving distance of the person to be imaged) is uniquely determined by the relationship with the above Qf. That is, although it is the position Q of the person to be imaged, in the example of FIG. 15, the horizontal movement distance is Qx (m) and the vertical movement distance is Qy (m), so fr = 30 (fps). Then, the moving speed of the person to be imaged is 30 × Qx (m / s) in the horizontal direction (horizontal speed) and 30 × Qy (m / s) in the vertical direction (vertical speed).

Furthermore, as shown in FIG. 16, when the imaging elapsed time is T _n and the frame number at a certain imaging point is n (that is, the frame before n can be expressed as n−1), imaging starts. One second after the time T _n is fr = 30 (fps), and can be expressed as T ₃₀ . On the premise of such a relationship, the position of the person to be imaged on the screen can be expressed as the following coordinate (1).

Q _n (qx _n , qy _n ) (1)

That is, the state detection unit 201 determines, for example, Q _n (qx _n , qy _n ) indicated by coordinates (1) as the position where the imaging subject exists.

Returning to the flowchart of FIG. 13, in step S <b> 94, the state detection unit 201 determines whether or not the person to be imaged exists within the screen displayed on the display unit 23. For example, the state detection unit 201 determines whether or not the determined coordinates of the position where the person to be imaged exists, Q _n (qx _n , qy _n ), are the coordinates in the image of FIG.

  In step S94, when it is determined that the person to be imaged does not exist in the screen, the state detection unit 201 estimates the position of the person to be imaged in the screen in step S95. And the information indicating the position where the imaging target person exists, obtained by performing the imaging target person position estimation process, is supplied to the superimposing unit 106.

That is, the state detection unit 201 uses the coordinates (1) to project the imaging subject person from the position relationship between Q _n and Q _n-1 when the imaging subject person is out of the image frame and protrudes from the screen. Estimated position.

Specifically, for example, in FIG. 14, when an imaging subject who has moved from Q _n-1 (15, 5) to Q _n (18, 5) protrudes from the right end of the screen, the speed of the imaging subject Is constant, Q _{n + 1} (qx _{n + 1} , qy _{n + 1} ) can be estimated as Q _{n + 1} (21, 5) from the above-described relationship. That is, it is estimated that the person to be imaged that protrudes from the right end of the screen is out of the image frame and exists at the position of Q _{n + 1} (21, 5).

Further, as an example other than the right end of the direction in which the person to be imaged protrudes, for example, in FIG. 14, the person to be imaged who has moved from Q _n-1 (10, 10) to Q _n (10, 13) in FIG. In the case of overhanging, _assuming that the speed of the subject to be imaged is constant, Q _{n + 1} (qx _{n + 1} , qy _{n + 1} ) is Q _{n + 1} (10, 16) from the above-described relationship. Can be estimated. That is, it is estimated that the person to be imaged that protrudes from the upper end of the screen is out of the image frame and exists at the position of Q _{n + 1} (10, 16).

  In this way, the coordinates after protruding from the image frame can be obtained by using the coordinates before protruding from the imaging target person.

  In the above-described example, the example in which the person to be imaged protrudes from the right end or the upper end of the screen has been described. Of course, the coordinates when the object to be imaged protrudes from the left end or the lower end of the screen can be obtained in the same manner. Further, any method other than the method described above may be used as long as the method can estimate the position of the person to be imaged out of the image frame.

  Returning to the flowchart of FIG. 13, in step S <b> 96, the superimposing unit 106 moves out of the image frame to the image supplied from the imaging unit 103 based on the result of the imaging target person position estimation supplied from the state detection unit 201. A state mark indicating the position where the imaging subject is present is superimposed.

  On the other hand, if it is determined in step S94 that the person to be imaged exists in the screen, the process of step S95 is skipped, and in step S96, the superimposing unit 106 adds the image supplied from the imaging unit 103 to the image. A state mark indicating that the person to be imaged is present in the image frame is superimposed.

  At this time, the superimposing unit 106 supplies the display control unit 107 with image data in which both the face mark (the process in step S80 in FIG. 12) and the status mark (the process in step S96 in FIG. 13) are superimposed.

  Returning to the flowchart of FIG. 12, in step S <b> 82, the display control unit 107 causes the display unit 23 to display the image of the subject on which the face mark or the state mark is superimposed supplied from the superimposition unit 106.

  That is, as shown in FIG. 10, when the person to be imaged is present in the image 301, the display unit 23 displays the face mark 302 a for identifying the person to be imaged and the person to be imaged. A state mark 303 a indicating that the image 301 exists in the image 301 is displayed superimposed on the image 301. On the other hand, as shown in FIG. 11, when the person to be imaged does not exist in the image 301, for example, since the person to be imaged exists in the left direction of the image 301, only the left direction is indicated by a solid line Is superimposed on the image 301 and displayed.

  As a result, even if the photographer loses sight of the imaging subject, the direction in which the imaging subject exists can be grasped by the status mark, so that the imaging subject can be immediately found and imaged. it can.

  In the above-described example, an example in which only one person is an imaging target and an image in which a face mark and a state mark are superimposed is displayed on the imaging target is described. The present invention is not limited, and a plurality of persons can be imaged, and face marks and status marks can be superimposed on the image and displayed.

  FIG. 17 is a schematic diagram illustrating a configuration example of a screen on which state marks are superimposed on a plurality of imaging subjects.

  In the example of FIG. 17, the photographer images a soccer game with the video camera 1, but has registered (selected) four players who are interested in the game. Face marks 302b to 302e are respectively superimposed on the players who are present (in FIG. 17, since the players on which the face marks 302e are superimposed exist outside the image frame, the face marks 302e are superimposed on the image 301. Not) Further, on the left side of the image 301 displayed on the display unit 23, state marks 303b to 303e indicating the state of the player on which the face mark is superimposed are displayed corresponding to the face marks 302b to 302e.

  Specifically, the state mark 303b indicates the state of the player (hereinafter also referred to as player b) on which the face mark 302b is superimposed. Since the player b exists in the image frame, the state mark indicates “present state”. Similarly, the state mark 303c indicates the state of the player (hereinafter also referred to as the player c) on which the face mark 302c is superimposed, and indicates the “present state” because the player c exists in the image frame.

  Similarly, the state mark 303d indicates the state of the player (hereinafter also referred to as the player d) on which the face mark 302d is superimposed. Since the player d exists in the image frame, the “present state” is indicated. Yes. Furthermore, the state mark 303e indicates the state of the player (hereinafter also referred to as the player e) on which the face mark 302e is superimposed. Since the player e is out of the image frame and exists in the left direction, State ".

  Thereafter, in the video camera 1, for example, when the photographer is moved leftward in order to image the player in the “left movement state”, the player e enters the image frame. When the player b and the player d are out of the image frame, the display unit 23 displays a screen as shown in FIG. 18, for example.

  In the example of FIG. 18, the status mark 303b is displayed on the display screen because, for example, the player b moves off to the right of the image frame by moving the video camera 1 to the left by the photographer. Change from “Existing State” to “Right Moving State”. Similarly, the state mark 303c is maintained in the “present state” because the player c continues to exist in the image frame even when the video camera 1 is moved leftward by the photographer, for example. To do.

  Similarly, the status mark 303d indicates that the player d has moved to the left of the image frame by moving the video camera 1 to the left by the photographer. Change from "status" to "left move status". Furthermore, the status mark 303e is displayed when the video camera 1 is moved leftward by the photographer, for example, so that the player e enters the image frame. Change to "Status". That is, the photographer moves the video camera 1 to the left according to the state mark 303e on the screen of FIG. 17 displayed on the display unit 23, thereby capturing the player e as shown in FIG. Became possible.

  As described above, in the video camera 1, even when a plurality of imaging subjects are selected, the face mark and the state mark are superimposed on the captured image as in the case where one imaging subject is selected. be able to. This enables the imager to grasp the direction in which there is an imaging subject who has fallen out of the image frame even if the imaging subject has been lost when multiple imaging subjects are selected. Therefore, it is possible to immediately find an imaging subject and continue imaging.

  The display method of the state mark is not limited to the above-described example. For example, in the examples of FIGS. 17 and 18, the color may be changed for each player (imaging target person), or the player may correspond to the state mark. Information that can identify the person to be imaged, such as a face image or name, may be displayed. In addition, although the display position of the status mark is described as being displayed on the left side of the image 301 in the above-described example, the display is not limited to this, and for example, it may be displayed so as to be superimposed on the image 301. It may be displayed at any position on the top, bottom, left and right of 301.

  Further, in the above-described example, the example in which the position (direction) of the imaging subject who has deviated from the screen displayed on the display unit 23 has been described as the status mark. However, in the present embodiment, the status mark is In addition to indicating the position of the person to be imaged who has deviated from the image frame, a guide for operating the zoom button (hereinafter referred to as an operation guide) for putting the person to be imaged in the image frame is shown. May be.

  FIG. 19 is a schematic diagram illustrating a configuration example of a screen on which a state mark including an operation guide is superimposed.

  In the example of FIG. 19, as in FIGS. 17 and 18, the photographer who is capturing images with the video camera 1 captures a soccer game in a state where four players are selected as the imaging subjects. ing.

In the example of FIG. 19, the state mark 303b ₁ indicates the state of the player b, and the player b has deviated to the right of the image frame, and thus indicates “right movement state”. An operation guide 303b ₂ is displayed below the status mark 303b ₁ . The operation guide 303b ₂ captures the current zoom position indicated by a dotted line corresponding to the zoom position of the zoom mark 304 (hereinafter referred to as the current zoom position) and the player b who is the imaging target (that is, the player). A zoom position (hereinafter referred to as a target zoom position) indicated by a solid line capable of placing b in the image frame) is displayed.

  That is, the player b is not displayed without being displayed in the image frame when the zoom position is the current zoom position. However, when the player b zooms down the zoom position to the target zoom position, Fits within the frame.

  Note that, as described above with reference to FIGS. 14 to 16, the position where the imaging target person deviates from the image frame can be obtained, and thus the target zoom position is calculated using the position, for example.

Similarly, the state mark 303c ₁ indicates the state of the player c, and indicates the “present state” because the player c exists in the image frame. An operation guide 303c ₂ is displayed below the status mark 303c ₁ . The operation guide 303c ₂ displays only the current zoom position because “exist”, that is, the player c is within the image frame and does not need to display the target zoom position.

Further, the state mark 303d ₁ indicates the state of the player d, and the player d has deviated to the left of the image frame, and therefore indicates “left moving state”. An operation guide 303d ₂ is displayed below the status mark 303d ₁ and the current zoom position and the target zoom position are displayed. That is, the player d is not displayed in the image frame if the zoom position is the current zoom position, but “Tele”, that is, the player d displays the image by zooming down the zoom position to the target zoom position. Fits within the frame.

Further, the state mark 303e ₁ indicates the state of the player e, and indicates the “present state” because the player e exists in the image frame. An operation guide 303e ₂ is displayed below the status mark 303e ₁ and the current zoom position and the target zoom position are displayed. That is, in this case, the player e is within the image frame, but the face of the player e displayed in the image 301 is too small, so “Wide”, that is, the zoom position is zoomed to the target zoom position. The face will be reflected by uploading.

  As described above, the video camera 1 can display an operation guide for the zoom button for displaying the person to be imaged in the image frame as the status mark. Thereby, even when the photographer loses sight of the person to be imaged, he can grasp the sense of distance to the person to be imaged and can perform operations such as appropriate zoom-out and rotation operations.

  It should be noted that an actual distance (for example, “2.5 m”, etc.) to the imaging subject who is out of the image frame may be displayed as the status mark. Further, only the operation guide may be displayed as the status mark. Further, a speaker (not shown) provided in the video camera 1 may output the content indicated by the status mark such as “Please move the video camera 1 to the left” by voice.

  Furthermore, in the video camera 1, a state mark in which both “circle” and “arrow” are dotted lines is displayed for an imaging subject who has not been reflected on the screen since the start of imaging. Is in a state that is not “Tele”, “Wide”, or “exist”.

  In this embodiment, as shown in FIGS. 10 and 11 and FIGS. 17 to 19, the state mark is attached with arrows indicating four directions of up, down, left, and right in a circle indicating the presence or absence of the person to be imaged. However, the present invention is not limited to this, and various icons can be used as shown in FIG.

  In the example of FIG. 20, the status marks 311 a to 311 g are provided with direction keys for indicating the direction of the position where the imaging target person exists in four directions, and the direction in which the imaging target person exists by using these direction keys. Point to. That is, if the status mark is an icon that can indicate the position of the person to be imaged, its shape, color, size, and the like can be arbitrarily changed. In the present embodiment, the position of the person to be imaged is shown in four directions, but the direction can be arbitrarily changed, for example, eight directions. In short, it is only necessary to show the position of the person to be imaged to the imager, and the display form is arbitrary.

  Returning to the flowchart of FIG. 12, in step S <b> 83, as in step S <b> 62 of FIG. 8, it is determined whether or not the control unit 101 has instructed the end of imaging, and it is determined that the end of imaging has not been instructed. Steps S71 to S83 are repeated. That is, the predetermined process of superimposing the face mark and the state mark is repeated in steps S71 to S83 until it is determined in step S83 that the end of imaging has been instructed. As a result, the photographer performs imaging while following the face mark superimposed on the imaging subject, and moves the video camera 1 according to the status mark even when the imaging subject has deviated from the image frame. Thus, the person to be imaged can be reliably imaged.

  As described above, the video camera 1 provides a function capable of identifying the imaging subject meaningful to the photographer and grasping the motion of the imaging subject, thereby reliably and promptly taking the photographer. This makes it possible to grasp the target subject.

  In addition, since the photographer can select a plurality of persons to be photographed at the time of photographing, it is possible to adjust the image frame appropriately. Furthermore, since the photographer can accurately grasp the movement of the selected person to be imaged, it is possible to focus according to the scene.

  Furthermore, since the video camera 1 performs caution display and voice guidance when the person to be imaged is out of the image frame, the imager can immediately follow the object person to be imaged.

  In the present embodiment, the face image of the imaging target person is acquired from the recording medium 28 that records the image including the face image of the imaging target person. The face image of the person to be imaged may be acquired from an image acquired from another device via a network such as the Internet, for example, by a communication unit (not shown) provided in.

  In this embodiment, “child” is described as an example of the person to be imaged. However, the present invention is not limited to this, and the video camera 1 includes, for example, vehicles other than children, animals such as automobiles, and the like. Can be used for a subject to be imaged, particularly a moving subject. Furthermore, the image that identifies the person to be imaged is not limited to the face image of the subject, but may be an image of a part that can identify the person to be imaged.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.

  The recording medium is distributed to provide a program to the user separately from the computer, and includes a magnetic disk (including a flexible disk) on which the program is recorded, an optical disk (CD-ROM (Compact Disc-Read Only Memory)) , DVD)), magneto-optical disk (including MD (trademark)), or the removable medium 26 of FIG. The flash ROM 21 of FIG. 1 in which a program is recorded is configured.

  The program for executing the series of processes described above is installed in a computer via a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via an interface such as a router or a modem as necessary. You may be made to do.

  In the present specification, the step of describing the program stored in the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

It is a block diagram which shows the structure of one Embodiment of the video camera to which this invention is applied. It is a block diagram which shows the example of a functional structure of the video camera of FIG. It is a figure explaining the outline | summary of the registration method of an imaging subject person. It is a flowchart explaining a face mark superimposition process. It is a flowchart explaining the detail of a face image detection process. It is a schematic diagram which shows the structural example of a face mark. It is a figure explaining the outline | summary of the registration method of the imaging subject using the image recorded on the recording medium. It is a flowchart explaining a face mark superimposition process. It is a block diagram which shows the other example of a functional structure of a video camera. It is a figure explaining the outline | summary of a state mark. It is a figure explaining the outline | summary of a state mark. It is a flowchart explaining the outline | summary of a face and a state mark. It is a flowchart explaining the detail of a state detection process. It is a figure explaining the calculation method of the moving distance of an imaging subject person. It is a figure explaining the calculation method of the moving distance of an imaging subject person. It is a figure explaining the calculation method of the moving distance of an imaging subject person. It is a schematic diagram which shows the structural example of the screen on which the status mark was superimposed. It is a schematic diagram which shows the structural example of the screen on which the status mark was superimposed. It is a schematic diagram which shows the structural example of the screen on which the status mark was superimposed. It is a schematic diagram which shows the structural example of a state mark.

Explanation of symbols

  1 Video Camera, 17 DSP, 19 CPU, 22 Display Controller, 23 Display Unit, 24 Input Unit, 27 Recording Media Interface, 28 Recording Media, 52 Touch Panel, 101 Control Unit, 102 Operation Unit, 103 Imaging Unit, 104 Face Image Detection Unit, 105 face image collation unit, 106 superimposition unit, 107 display control unit, 108 face image selection unit, 109 face image storage unit, 109a face image database, 110 recording media control unit, 111 face image detection unit, 201 state detection unit

Claims (10)

In an imaging device that displays an image of a captured subject on a display,
Imaging means for capturing an image of a subject;
First detection means for detecting a part image that is an image including a predetermined part of the subject from the imaged image;
A superimposing unit that superimposes a first icon indicating an imaging target on the image in correspondence with a target part image that is a part image to be imaged by the photographer among the detected part images;
An image pickup apparatus comprising: display control means for controlling display on the display of the image on which the first icon is superimposed. The imaging device according to claim 1, wherein the part image is an image of a face of the subject. According to an instruction from the photographer, the image processing apparatus further includes a selection unit that selects the target part image from the detected part images.
The imaging apparatus according to claim 1, wherein the superimposing unit superimposes the first icon on the image in correspondence with the selected target part image. The imaging apparatus according to claim 3, further comprising storage control means for storing the selected target part image in a predetermined recording medium. Reading means for reading an image including the target part image from a predetermined recording medium storing the target part image;
Collating means for collating the part image detected from the imaged image and the target part image included in the image read from the recording medium,
The imaging apparatus according to claim 1, wherein the superimposing unit superimposes the first icon on the image in correspondence with a part image that matches the target part image based on a result of collation. The imaging device according to claim 1, wherein the first icon is an image having a circular, square, or elliptical shape. A second detecting means for detecting a movement of the imaging target;
The imaging apparatus according to claim 1, wherein the superimposing unit superimposes a second icon indicating an operation state of the imaging target on the image based on the detected movement. The imaging apparatus according to claim 7, wherein the second icon is an image indicating a position where the imaging target exists. The imaging according to claim 7, wherein the second icon is an image indicating a predetermined operation method of a zoom button operated to display the image of the imaging target that is not displayed on the display on the display. apparatus. In an imaging method of an imaging device that displays an image of a captured subject on a display,
Take an image of the subject,
From the captured image, a part image that is an image including a predetermined part of the subject is detected,
Corresponding to the target part image that is the part image to be imaged by the imager among the detected part images, a first icon indicating that the part is an imaging target is superimposed on the image,
An imaging method including a step of controlling display of the image on which the first icon is superimposed on the display.

Images