JP2010098354A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the following problem: a conventional imaging apparatus having a subject recognizing function needs to detect a motion vector, etc., of a subject and further needs to individually detect the subject moving out of a screen and being hidden behind an obstacle, which require the conventional imaging apparatus to perform complicated processing. <P>SOLUTION: The imaging apparatus includes an imaging section 10 which images the subject, an image processing section 11 which performs image processing on an output of the imaging section 10, a feature detection section 12 which detects specified subjects in an output image of the image processing section 11, an operation section 16 which receives an operation instruction including an operation to select at least one specified subject out of the specified subjects detected by the feature detection section 12, a display section 13 which displays a mark showing the selected specified subject together with the output image of the image processing section 11, and a control section 14 which divides the display section 13 into a plurality of display areas, and controls the display section 13 to display a different warning according to a display area where the marked specified subject is present. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging device, and more particularly to an imaging device that performs subject recognition and displays a recognized subject.

  Conventionally, in an imaging device having a function of recognizing a subject such as a digital camera, the imaging device is controlled in various ways by AE, AF, etc. so that the recognized subject can be photographed optimally. However, since the subject is recognized from the image information on the screen, if the subject is removed from the screen, control over AE and AF becomes impossible.

  On the other hand, there is a technique in which the subject is indicated by an arrow in the radial direction from the center of the screen and the operator is informed of the position of the subject, thereby making it difficult to remove the subject from the screen (Patent Document 1).

  Hereinafter, a conventional imaging apparatus having a subject recognition function will be described with reference to FIG.

In FIG. 9A, the photographer is informed of the position of the subject with a radial arrow 102 from the center of the monitor with respect to the subject 101 displayed on the display unit 13. In FIG. 9B, an arrow 102 is arranged on a line connecting the center of the screen and the subject with respect to the subject 101 (not actually displayed) that is off (not shown) from the display unit 13, and the subject is outside the screen. It is shown that.
JP 2005-341449 A

  However, in the imaging apparatus described above, it is necessary to detect a motion vector or the like of the subject 101, and further, it is necessary to detect separately when the subject 101 moves outside the screen and when the subject 101 is hidden by an obstacle. There is a problem that the processing becomes complicated.

  An imaging apparatus having a subject recognition function according to the present invention solves the above-described problem, and includes an imaging unit that images a subject, an image processing unit that performs image processing on an output of the imaging unit, and an image processing unit. A feature detection unit that detects a specific subject in the output image, a display unit that displays an output image of the image processing unit together with a mark indicating the specific subject, and the display unit is divided into a plurality of display areas, and the mark A control unit that determines the presence or absence of a warning or a warning method according to the display area in which the specific subject that displays is present.

  With the above configuration, the imaging apparatus of the present invention does not require detection of a motion vector or the like related to the subject, and can issue an appropriate warning with a simple configuration.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of a video camera 100 having a subject recognition function according to Embodiment 1 of the present invention. In FIG. 1, a video camera 100 is signal-processed with an imaging unit 10, a signal processing unit 11 that processes a signal of the imaging unit 10, a specific target detection unit 12 that detects a specific target from a signal-processed image. An image recording unit 17 that records images, a display unit 13 that displays detected specific objects, a controller 14 that controls them, and an operation unit 16 that is operated by a user from the outside are provided.

  The imaging unit 10 includes an optical system and an imaging element.

  The optical system includes an objective lens, a zoom lens, a diaphragm, an OIS unit, a focus lens, and the like. The optical system collects light from the subject and forms a subject image.

  The imaging element captures a subject image formed by the optical system and generates image data. The image sensor is a CMOS image sensor, a CCD image sensor, or the like.

  The signal processing unit 11 includes an AD converter, an image processing unit, a system encoder, and a system decoder.

  The AD converter converts image data generated by the image sensor into a digital signal.

  The image processing unit performs various processes on the image data converted by the AD converter. The image processing unit generates image data to be input to the system encoder, or generates image data to be displayed on the display unit 13. The image processing unit performs image processing on the image data output from the system decoder, generates image data to be displayed on the display unit 13, and generates image data for re-input to the system encoder. Or

  The image processing unit includes an image preprocessing unit, an image compression unit, and an image expansion unit.

  The image preprocessing unit performs various image processing such as gamma correction and white balance correction on the image data converted by the AD converter.

  The image compression unit compresses image data by performing DCT (Discrete Cosine Transform), Huffman coding, and the like. The image compression unit is, for example, MPEG-2 or H.264. The image data is compressed by a compression format conforming to the H.264 standard. However, the present invention relates to MPEG-2 or H.264. The image data is not limited to H.264 format.

  For example, when the compressed image data output from the system decoder is reproduced on the display unit 13, the image expansion unit decodes the image data into an uncompressed state.

  The system decoder decodes the AV data recorded in the image recording unit 17 and generates compressed image data. The system decoder outputs the generated compressed image data to the image expansion unit.

  The controller 14 is a control unit that controls the entire video camera 100. The controller 14 can be realized by a semiconductor element or the like. The controller 14 may be configured only by hardware, or may be realized by combining hardware and software. The controller 14 can be realized by a microcomputer.

  The specific target detection unit 12 analyzes the image data generated by the signal processing unit 11 and detects a specific target in the image data. The specific target is, for example, a human face, a specific animal such as a person or a cat, a moving object, or the like. The specific target detection unit 12 detects the specific target and calculates a reliability indicating the certainty of the specific target. In the present embodiment, the specific target detection unit 12 is provided separately from the controller 14, but may be realized by the controller 14.

  The image recording unit 17 includes a recording medium control unit and a recording medium. The recording medium is a memory card, a hard disk, a DVD (Digital Versatile Disc), or the like.

  The display unit 13 is a liquid crystal monitor or an electronic viewfinder. The display unit 13 can display image data captured by the imaging unit 10 and image data in AV data read from the image recording unit 17. The display unit 13 can display various setting information of the video camera 100, shooting time, and the like.

  The operation unit 16 is a component that collectively refers to various operation means. The operation unit 16 receives a user instruction and transmits the instruction to the controller 14.

  The imaging unit 10 is an example of the imaging unit of the present invention. The signal processing unit 11 is an example of an image processing unit of the present invention. The specific target detection unit 12 is an example of a feature detection unit of the present invention. The operation unit 16 is an example of the operation unit of the present invention. The display unit 13 is an example of the display unit of the present invention. The controller 14 is an example of a control unit of the present invention.

  The operation of the video camera 100 having the subject recognition function configured as described above will be described.

  Various corrections and the like are performed on the image data captured by the imaging unit 10 by the signal processing unit 11. The corrected signal is recorded by the image recording unit 17. Also. The specific target detection unit 12 detects the specific target based on the signal from the signal processing unit 11, and displays the detection location using a frame display or the like. At the same time, the controller 14 displays a warning corresponding to the corresponding area and a warning about a decrease in reliability on the detection position of the specific target using the display unit 13.

  The warning display process will be described below with reference to the flowchart of FIG.

  In a state where a plurality of subjects are displayed on the display unit 13, the photographer operates to select a subject to be recognized from the plurality of subjects (step S201). For example, when the display unit 13 is a touch panel, the photographer selects a subject by touching a specific subject on the display unit 13. When the photographer operates the operation button on the operation unit 16, the selection display is sequentially moved to a plurality of subjects, and the determination button is operated at the timing when the selection display is performed on the specific subject to be selected. Thus, the specific target may be selected.

  In the present embodiment, a case where one target is selected as a specific subject from a plurality of subjects will be described, but the present invention is not limited to this, and a plurality of specific subjects may be selected.

  The controller 14 displays a mark on the screen of the display unit 13 for the selected specific subject (step S202). A display example of the mark is shown in FIG. When the face 30 is selected as the specific subject, a frame display of a color different from the normal frame display is performed around the face 30. The frame display around the selected face 30 may be blinked.

  The controller 14 detects in which area the selected specific subject exists on the screen of the display unit 13 (step S203).

  An example of screen division is shown in FIG. The screen is divided into five areas: a center area 31, an upper area 32, a right area 33, a lower area 34, and a left area 35. The size of each area shown in FIG. 3 is an example, and the upper area 32, the right area 33, the lower area 34, and the left area 35 may be larger and the central area 31 may be smaller, or vice versa. May be. In the present embodiment, each of the upper region 32, the right region 33, the lower region 34, and the left region 35 has a trapezoidal shape, but each may have a rectangular shape. In FIG. 3, the controller 14 detects that the specific subject 30 exists in the right region 33.

  The controller 14 detects the reliability of the subject selected using the specific target detection unit 12 (step S204).

  The controller 14 detects whether the specific subject 30 exists in any of the four areas, the upper area 32, the right area 33, the lower area 34, and the left area 35, which are determined as the predetermined area (step S205). When the specific subject 30 exists in the four areas located around the screen, the specific subject 30 is likely to move outside the screen in the future. Therefore, the controller 14 issues a warning to the photographer in advance.

  As shown in FIG. 3, when the specific subject 30 exists in the predetermined area (Yes in step S205), the controller 14 compares the reliability of the specific subject 30 with a threshold value to be held (step S206). When the reliability is equal to or lower than the threshold (Yes in Step S206), the controller 14 deletes the displayed mark (display frame) (Step S208) and determines the position on the display screen where the specific subject 30 is equal to or lower than the threshold. The data is stored on an internal memory (not shown) (step S209). When the reliability exceeds the threshold (No in Step S206), the controller 14 displays a warning on the display unit 13 (Step S207).

  When the reliability of the specific subject 30 is lowered, generally, the specific subject 30 may have moved away, hidden behind an obstacle, or moved out of the screen. By detecting the motion vector of the specific subject 30, it can be determined whether or not the subject is moving out of the screen, but the reliability has decreased due to being out of the screen, or has been hidden by an obstacle at the edge of the screen Is difficult to distinguish. Also, when the specific subject 30 is at the edge of the screen, if the obstacle moves and hides the specific subject 30, or if the specific subject 30 moves away at the end of the screen, the reliability of the specific subject is The motion vector is not detected and no warning can be issued. Therefore, in the present embodiment, it is difficult to recognize the specific subject 30 by issuing a warning in accordance with the region where the specific subject 30 is located and its reliability, without making these distinctions. Inform the photographer. Thus, the photographer can take various actions so that the specific subject 30 can be easily recognized. That is, an appropriate warning can be issued with a simple configuration without detecting a motion vector related to the subject.

  As shown in FIG. 3, an example of a warning display when the specific subject 30 exists in the right region 33 is shown in FIG. In FIG. 4, the controller 14 blinks the warning display bar 43 in the right area as a warning display. As a result, the photographer can recognize that the specific subject 30 is located in the right region, and thus can take an action such as directing the video camera 100 in that direction.

  The controller 14 measures the elapsed time when the reliability is equal to or lower than the threshold value (Yes in Step S206), and displays it when the elapsed time exceeds a predetermined time previously held (Yes in Step S210). Is not displayed (step S211), and the process is started again from the selection operation of the specific subject (step S201). This case corresponds to the case where the specific subject 30 has moved out of the screen and the photographer has not performed an action corresponding to the warning display indicating the direction for a predetermined period of time. Since it is considered that the user wants to select a specific subject, the process is started from the specific subject selection operation (step S201).

  The controller 14 continues the warning display until the predetermined time elapses (No in step S210) (step S207). In other words, even if the reliability of the specific subject 30 decreases and the user loses sight, the photographer can help the photographer find the specific subject 30 by continuing the warning display for a certain period of time.

  The controller 14 determines that the reliability of the specific subject 30 existing in the predetermined area (Yes in Step S205) is equal to or lower than the threshold (Yes in Step S206) and the specific subject before the predetermined time elapses (No in Step S210). When the reliability of 30 is recovered and exceeds the threshold value (No in step S206), the warning display is continued (step S207). However, in this case, when the reliability of the specific subject 30 decreases, the specific subject 30 is mistakenly recognized as another subject, and then the reliability of the erroneously recognized subject increases, so that the wrong subject is selected. A continuing situation can occur. In order to avoid this situation, the controller 14 stores in the internal memory (not shown) the disappearance position on the display screen when the reliability of the specific subject 30 falls below the threshold value (step S209). Whether or not the specific subject before and after the decrease in reliability matches is determined based on whether or not the appearance position when the reliability of the subject recovers and exceeds the threshold is in the vicinity of the disappearance position. When the appearance position of the specific subject is not in the vicinity of the disappearance position, the controller 14 determines that a different specific subject is selected, stops the mark display and the warning display, and starts the specific subject selection operation (step S201). Start processing.

  Note that whether or not the specific subject matches before and after the decrease in reliability may be determined by comparing characteristics such as the color and shape of the specific subject.

  FIG. 5 shows a difference in the display method of the warning display bar depending on the area. As a warning display, the controller 14 blinks the warning display 42 when the specific subject 30 is in the upper region 32, blinks the warning display 43 when the specific subject 30 is in the right region 33, and the specific subject 30 is in the lower region. When the specific object 30 is in the left area 35, the warning display 45 flashes.

  By performing such warning display, the ease of recognition differs depending on the size, position, etc. of the arrow in the case of the arrow display compared to the case of displaying the arrow indicating the subject. It is possible to display a warning that is easy for the photographer to view without causing problems such as obstructing the confirmation of the subject.

  As shown in FIG. 6, the controller 14 does not display a warning when the specific subject 30 exists in the central region 31 other than the predetermined region (No in step S205).

  Note that when the specific subject 30 exists in the central region and the reliability thereof is compared with a threshold value to be held and the reliability is equal to or lower than the threshold value, the controller 14 may perform a warning display illustrated in FIG. In FIG. 8, the controller 14 blinks all warning display bars 42, 43, 44, and 45. As a result, the photographer is losing sight of the fact that the specific subject 30 is located in the central area and the reliability of the specific subject 30 is lowered, that is, the specific subject 30 is located behind an obstacle such as a tree. Accordingly, the photographer can move and take an appropriate action such as moving to a position where the specific subject 30 can be seen.

  In the present embodiment, when the specific subject 30 is in the central region 31 and the reliability exceeds the threshold, no warning is given, but all the warning display bars 42, 43, 44, and 45 are turned on. It is also possible to notify the photographer.

  Thus, the video camera 100 can perform appropriate warning display with a simple configuration by changing the warning display method using the region where the specific subject 30 is located and the reliability.

  As a warning display method, as shown in FIG. 8, warning display units 91, 92, 93, and 94 made of LEDs or the like are provided outside the display unit 10, and the warning display units 91, 92, 93, and 94 are Depending on the position of the subject and the reliability, it may be lit or blinking.

  The image pickup apparatus according to the present invention uses the method of displaying the subject existing area and simultaneously displaying the reliability of the subject, so that the target subject is likely to be off the screen or the target subject is likely to be lost. It is possible to inform the photographer of this. Further, since the warning display is located at the periphery of the screen, the warning can be performed without disturbing the confirmation of other subjects, which is useful for digital cameras, video cameras, and the like.

1 is a block diagram illustrating a configuration of an imaging device according to an embodiment of the present invention. The flowchart which shows the process of the warning display in one embodiment of this invention The figure which shows the screen area division | segmentation in one embodiment of this invention The figure which shows the warning display in one embodiment of this invention The figure which shows the warning display according to the area | region in one embodiment of this invention The figure which shows the to-be-photographed object located in the division | segmentation screen area in one embodiment of this invention The figure which shows the warning display in one embodiment of this invention The figure which shows the warning display method in other embodiment of this invention. Figure showing a conventional warning display

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image pick-up part 11 Signal processing part 12 Specific object detection part 13 Display part 14 Controller 16 Operation part 17 Screen recording part 100 Video camera

Claims (8)

An imaging unit for imaging a subject;
An image processing unit for image processing the output of the imaging unit;
A feature detection unit for detecting a specific subject in the output image of the image processing unit;
A display unit for displaying an output image of the image processing unit together with a mark indicating the specific subject;
A control unit that divides the display unit into a plurality of display regions and determines whether or not there is a warning or determines a warning method according to the display region where the specific subject displaying the mark exists;
An imaging apparatus comprising: An operation unit that receives an operation instruction including an operation of selecting at least one specific subject from the specific subject detected by the feature detection unit;
The display unit displays an output image of the image processing unit together with a selection mark indicating the selected specific subject,
The control unit divides the display unit into a plurality of display areas, and determines whether or not there is a warning or determines a warning method according to the display area where the specific subject displaying the selection mark is present. The imaging apparatus according to 1. The feature detection unit calculates the reliability of the detected specific subject,
The control unit displays the warning for a predetermined time when the reliability of the specific subject displaying the mark falls below a threshold when the warning is displayed on the display unit, and after the predetermined time has elapsed. The imaging apparatus according to claim 1, wherein control is performed so that the warning is not displayed. The control unit determines whether or not the specific subject matches the specific subject selected by the operation unit when the reliability of the specific subject exceeds the threshold before the predetermined time elapses. The image pickup apparatus according to claim 3, wherein the image pickup device is controlled to continue displaying the warning when they match. The control unit determines whether the specific subject before and after the change in reliability matches.
Storing the disappearance position of the specific subject in the display unit when the reliability of the specific subject displaying the mark is equal to or lower than a threshold;
The imaging apparatus according to claim 4, wherein the position of the specific subject whose reliability exceeds the threshold is determined based on whether or not the position is near the disappearance position. The imaging apparatus according to claim 1, wherein the display area includes five areas of a center area, an upper area, a lower area, a left area, and a right area of the display unit. The control unit displays a warning near a region where the specific subject exists in the display unit when the specific subject displaying the mark exists in the upper region, the lower region, the left region, or the right region. The imaging device according to claim 6. The imaging device according to claim 1, wherein the feature detection unit detects a human face as the specific subject.

Images