WO2011052120A1

WO2011052120A1 - Imaging control device and imaging control method

Info

Publication number: WO2011052120A1
Application number: PCT/JP2010/005067
Authority: WO
Inventors: 敏信秦野
Original assignee: パナソニック株式会社
Priority date: 2009-11-02
Filing date: 2010-08-16
Publication date: 2011-05-05
Also published as: JP2011097502A

Abstract

Disclosed is an imaging control device provided with: a face detection unit which detects the face of an imaged subject in image data; a recognition region setting unit which monitors whether the position of the detected face is stable and determines an image recognition region within a prescribed area with reference to the stable detected face; a motion vector extraction unit which examines the set image recognition region and extracts changes of motion vectors on the time axis for any motion in the image recognition region; an image recognition unit which recognizes whether the motion vector change is a prescribed change; and an imaging control unit that generates a prescribed imaging control signal corresponding to the result of recognition by the image recognition unit.

Description

Imaging control apparatus and imaging control method

The present invention performs wirelessly controlled shooting control without using a remote control transmitter, and relates to a shooting control device such as a digital camera capable of automatic shooting based on image recognition. The present invention relates to a technique for improving operability and efficiency to surely perform shooting at an intended timing. The present invention also relates to a shooting control method.

Conventionally, cameras equipped with a remote control system using infrared light have been proposed. As a result, it is possible to shoot by pressing the shooting button of the remote control transmitter at a distant position instead of directly pressing the release button of the camera body. However, it is also a fact that it causes an increase in the cost and size of the camera. In view of this, there has been proposed a camera capable of a release operation by remote operation without using a remote controller. It is automatic shooting based on image recognition.

In Patent Document 1, shape recognition such as a V sign in a subject (see (a) in FIG. 10) or recognition of a movement locus such as an S-curve in the subject (see FIG. 10) for image data acquired by an image sensor. 10 (see (b)) or a pattern or color recognition such as a barcode included in the subject (see (c) in FIG. 10), the release control is disclosed when a corresponding item is detected. ing.

Also, in Patent Document 2, a shooting operator designates a target mark on the display screen of the apparatus main body, and then the shooting operator moves so as to be reflected in the shooting field of view and puts his face at a position corresponding to the target mark. A method is disclosed that triggers if placed. That is, the photographing operator moves the target mark displayed on the through image display screen to designate a desired position in the photographing field of view. Thereafter, image recognition of the face of the specific person is performed for the target area corresponding to the designated position, and if it is recognized, a predetermined operation relating to photographing is performed using that as a trigger.

Also, Patent Document 3 eliminates the inconvenience of shooting at an unintended timing when shooting is immediately triggered by the establishment of autofocus. For example, if the V-sign of the subject person's hand is used as a shooting trigger, shooting can be performed at the intended timing. However, if autofocus is targeted at the V-sign, it is impossible to focus on the face of the subject person. The technique disclosed in Patent Document 3 is to clear these two problems. There, two camera operators are assumed. A person who operates the shutter button of the camera, and a person who performs a shooting trigger operation reflected in the camera. First, the operator A recognizes the face of the operator B as the subject by an operation with the camera body, and then performs autofocus. The operator A who fully presses the shutter button approaches the operator B and himself becomes a subject. Then, the operator B issues a V sign. When the V-sign and the skin color of the hand are recognized and recognized, a shooting process is executed using the V-sign as a shooting trigger (paragraphs [0035] to [0051], especially [0047], [0048], [0050], [0051]. ] And FIGS. 2, 3, and 5).

JP 2002-122934 A JP 2008-283502 A JP 2009-147574 A

In the case of Patent Document 1, predetermined shape data, motion data, and pattern data that are used as a reference for pattern recognition such as V-sign, shape recognition such as S-curve, and pattern recognition such as barcode are registered in a predetermined storage area in advance. There is annoyance that must be kept. In order to detect, for example, an “S” -shaped trajectory of a palm whose arm is swayed or a trajectory in which a human can operate (a trajectory of a circle, triangle, square, alphabet, hiragana, mark, etc.) As long as it is continuous both spatially and spatially, it is necessary to perform pre-processing for performing continuous shooting before the release operation and detecting the trajectory. However, such continuous shooting and preprocessing take time, and this makes it difficult to capture images at the intended timing. In order to realize this with high detection accuracy, it is necessary to mount a large-scale CPU with a high function, but it is difficult to realize from the viewpoint of cost. In addition, the area for detecting motion in the acquired image data is the entire area of the screen, but monitoring the entire area of the screen places a heavy load on the CPU. In addition, because the entire screen area is monitored, if a moving object such as a bird or airplane enters the background, it may be mistakenly recognized as a sign and may be shot at an unintended timing. . The cause of these inconveniences is that the monitoring area (image recognition area) is the entire area of the screen and is not limited to the small area, but is widely diffused.

In the case of Patent Document 2, there are too many tasks forcing the photographing operator prior to photographing. It is necessary to perform operations such as registering a specific person's image, setting the self-timer mode, selecting a specific person within the registered person, and setting the target area by moving the target mark. It has become. Furthermore, regarding the designation of the target area, the imaging operator does not necessarily designate the optimum area, and there is a problem that the accuracy of motion detection of the recognition target tends to deteriorate. In addition, depending on the size of the designated target area, the amount of data to be subjected to image processing becomes too large, and it takes a long time to perform the release process, which makes it impossible to perform a release operation at an optimal timing.

In the case of Patent Document 3, as in Patent Document 1, the area in which the signature is detected in the acquired image data G is the entire area of the screen, but monitoring the entire area of the screen places a heavy load on the CPU. It will be. In addition, because the entire screen area is monitored, if a moving object such as a bird or airplane enters the background, it may be mistakenly recognized as a sign and may be shot at an unintended timing. . Further, since the movement of the hand that draws the character “1” is detected by the motion vector between the image data, the amount of data to be processed becomes enormous, and it is necessary to mount a large-scale and high-performance CPU. However, it is difficult to realize it from the cost aspect. If the detection accuracy is low, it takes time for actual use, and it takes a long time for the release process, and the release operation may not be performed at an optimal timing.

The present invention has been created in view of such circumstances, and in a shooting control apparatus that performs shooting control of remote operation wirelessly without using a remote control transmitter, shooting at the timing intended by the shooting operator is ensured. However, it is an object of the present invention to obtain higher operability and faster speed while the operation required from the photographing operator as a subject is very simple.

The present invention solves the above problems by taking the following measures.

In the present invention, the following is considered. First, the face area of the subject is detected, and then it is monitored whether the detected face area is stabilized in position, and if it is stabilized, the image recognition area (trigger area) is set within a predetermined range based on the stable detected face area. Determine. The image recognition area is arbitrary such as a certain range of the chest, a certain range of the side of the face, a certain range of the upper side of the head, a certain range of the side of the waist, and its size and number are also arbitrary. This image recognition area may be single or plural. In the case of a single case, the distance is determined in a certain direction and a certain distance from the detected face area. In the case of a plurality of image recognition areas, a desired one is selected from a plurality of image recognition areas having various directions and distances from the detected face area. Subsequently, the determined image recognition area is examined to monitor whether there is movement in the image recognition area and whether the movement is predetermined. About this movement, a hand is moved (rotation, right-and-left reciprocation (waving a hand), up-and-down reciprocation, slanting reciprocation, etc.), and it is arbitrary. When the detected movement of the subject is a predetermined one, a predetermined control signal related to photographing is generated according to the recognition result of the movement. The predetermined thing refers to rotation, left and right reciprocation, up and down reciprocation, slant reciprocation, and open by connecting. “Predetermined thing” and “authentication result” may be just one item or a plurality of items. When there is only one item, it is only necessary to move in the image recognition area. In this case, the predetermined control is control related to one operation such as releasing a shutter if a motion is detected in the image recognition area. In the case of a plurality of items, the first control is performed for the first motion detection, the second control is performed for the second motion detection, and the nth motion detection is performed for the nth motion detection. It is configured to perform control. Alternatively, when the movement of A is detected and then the movement of B is detected, the first control is performed. Conversely, when the movement of B is detected and then the movement of A is detected, the second control is performed. Configure.

Detecting that the detected face area is stable in position can be called “static” detection. Further, detecting a motion and whether the motion is a predetermined one can be referred to as “motion” detection. The transition from the detection of “static” to the detection of “motion” is one condition. The shooting operator who is also the subject first stops the movement of the body (the face position is stable) within the shooting field of view, and then moves a part of the body such as the hand within a specific range. As a result, the imaging control device generates a predetermined control signal. The shooting operator only needs to stop the movement of the body within the shooting field of view and move a part of the body such as the hand within a specific range. Just by that, you can shoot at the intended timing.

If this is observed from the surface of the image data, it means determination of stabilization of the position of the detected face region, setting of the image recognition region, and detection of a motion vector change in the image recognition region. Various techniques for detecting the face area have already been developed. It is also easy to determine the stabilization of the position of the detected face area. Various techniques for changing motion vectors in the image recognition area have already been developed. The technical point of the present invention is to set the image recognition area within a predetermined range based on the detected face area, but this is not technically difficult. As for the position of the image recognition area, it is only necessary to determine in advance how many pixels each position is displaced in the horizontal and vertical scanning directions from the detected face area where the position is stabilized. It suffices to determine the number of pixels in both the horizontal and vertical scanning directions in advance. Therefore, if a detected face area whose position is stabilized is determined, it is easy to determine and set an image recognition area based on the detected face area. Although it is not technically difficult to set the image recognition area within the predetermined range based on the detected face area, this is very effective in solving the problem, and is a technical point of the present invention. ing.

The present invention thus
[1] Detection of a face area and a step of waiting for stabilization of the position of the detected face area (this is a so-called “static” detection step)
[2] Step for determining and setting an image recognition area based on a detected face area whose position is stabilized [3] A step for detecting a predetermined motion in the image recognition area (this is a so-called “motion” detection step) )
A series of steps (“static” detection → image recognition area → “motion” detection).

The technical point of the present invention lies in organically linking these three technical items in order to solve the problems recognized in the prior art.

Hereinafter, the configuration of the present invention will be described with reference to FIG. In order to realize the combination of the series of steps described above, the imaging control apparatus of the present invention includes an imaging device 1, a face region detector 2, a recognition region setting unit 3, a motion vector extractor 4, an image recognition unit 5, and an imaging control. A container 6 is provided. The face area detector 2 takes in the image data G of the subject obtained by the imaging device 1, detects a face area K composed of position information and size information of the face of the subject in the image data G, and detects the face area K It is configured to have a function of passing information to the recognition area setting unit 3. The recognition area setting unit 3 monitors whether the face area K detected by the face area detector 2 is positionally stable, determines an image recognition area N within a predetermined range with reference to the stable detection face area K, and It is configured to have a function of passing information on the defined image recognition area N to the motion vector extractor 4. The motion vector extractor 4 examines the image recognition area N set by the recognition area setting unit 3, extracts a motion vector change V on the time axis for an arbitrary operation in the image recognition area N, and extracts the extracted motion vector change V. Is transferred to the image recognizer 5. The image recognizer 5 is configured to recognize whether or not the motion vector change V is a predetermined one and to pass the recognition result to the imaging controller 6. The imaging controller 6 is configured to generate a predetermined control signal related to imaging according to the recognition result by the image recognizer 5.

In short, the imaging control apparatus according to the present invention is
A face area detector 2 for detecting a face area K composed of position information and size information of the face of the subject in the image data G sent from the image pickup device 1;
A recognition area setting unit 3 that monitors whether the face area K detected by the face area detector 2 is stable in position and determines an image recognition area N within a predetermined range with reference to the stable detection face area K;
A motion vector extractor 4 that examines the image recognition area N set by the recognition area setting unit 3 and extracts a motion vector change V on the time axis for an arbitrary operation in the image recognition area N;
An image recognizer 5 for recognizing whether the motion vector change V by the motion vector extractor 4 is a predetermined one;
A shooting controller 6 that generates a predetermined control signal related to shooting according to a recognition result by the image recognizer 5;
It is set as the structure provided with.

In this configuration, the image pickup device 1 is not an essential component but an external component.

The above solver is described as an imaging control method as follows. That is, the shooting control method according to the present invention is:
Detecting a face region K composed of position information and size information of the face of the subject in the image data G sent by imaging;
Monitoring whether the detected face area K is positionally stable, and determining the image recognition area N within a predetermined range with reference to the stable detected face area K;
Examining the set image recognition area N and extracting a motion vector change V on the time axis for any operation in the image recognition area N;
Recognizing whether the motion vector change V is a predetermined one;
Generating a predetermined control signal related to shooting according to the recognition result of the motion vector change V;
Is included.

In the above configuration of the present invention, the photographing operator who is also a subject stops the movement of the body within the photographing field of view, determines the face position, and then moves a part of the body such as the hand within a specific range corresponding to the image recognition area N. The imaging control device can generate a predetermined control signal. That is, the movement of the subject is captured by the image pickup device 1 and passed to the face area detector 2. The face area detector 2 detects a face area K including position information and size information of the face of the subject in the captured image data G of the subject (see (b) in FIG. 1), and information on the detected face area K. To the recognition area setting unit 3. The recognition area setting unit 3 monitors whether the detected face area K is stabilized in position, and if it is stabilized, sets the image recognition area N within a predetermined range with reference to the stable detected face area K (see FIG. 1 (see (c)), the motion vector extractor 4 is notified. The motion vector extractor 4 examines the image recognition area N, extracts a motion vector change V on the time axis for an arbitrary operation in the image recognition area N, and passes the extracted motion vector change V to the image recognizer 5. The image recognizer 5 recognizes whether or not the motion vector change V is a predetermined one (see (d) in FIG. 1), and passes the recognition result to the imaging controller 6. The shooting controller 6 generates a predetermined control signal related to shooting according to the recognition result by the image recognizer 5.

In this series of processes, the camera operator only needs to stop the body movement in the object scene and move a part of the body such as the hand within a specific range corresponding to the image recognition area N. With that alone, you can shoot at the intended timing. The image recognition area N is determined within a predetermined range with reference to the detected face area K, and in what direction and how far away from the detected face area K is determined in advance. The accuracy of motion detection is high. The photographing field of view is wider than the detected face area K. It is a heavy burden in the prior art to search all areas of the imaging field of view in order to determine where the sign is generated within the wide imaging field of view, but in the present invention, the detection face area K is used as a reference. Thus, since a relatively small image recognition area N is determined within a predetermined range, the determination is quick and easy, and the accuracy is high. As a result, malfunctions due to detection errors are suppressed, and the burden on the device control unit and the calculation unit is greatly reduced. That is, the trigger timing is early. As a result, photographing at the timing intended by the photographing operator is ensured. This is a wireless remote controller that does not require a transceiver, and can also be applied to remote release control.

The characteristics of the present invention can be stated as follows. The means for realizing the arrangement is not integrated in the apparatus as in the prior art, but the apparatus and the shooting operator are linked via the instruction manual and are stored in the shooting operator's memory. Matters occupy a big weight. That is, it is a memory of “stopping the body movement within the photographing field of view, determining the face position, and then moving a part of the body such as the hand within a specific range”. However, the memory and operation of “stopping the body movement within the shooting field of view, determining the face position, and then moving a part of the body such as the hand within a specific range” is hardly a burden on the shooting operator. . The apparatus side also detects the face area K from the image data G of the subject, waits for the stability of the detected face area K, sets the image recognition area N based on the detected face area K, and moves the motion vector in the image recognition area N. Extracting and determining the change V is relatively insignificant as compared to the computation of a huge amount of processing that takes the complicated steps accepted in the prior art. The device can be realized without imposing much burden of increase in circuit scale and cost. The image recognition area N can be set quickly and easily, and the processing speed and efficiency in this respect can be improved.

According to the above configuration of the present invention, it is possible to achieve the above-mentioned total and reliable shooting effect at the timing intended by the shooting operator under simple operability while suppressing an increase in circuit scale. It is possible.

(2) In the imaging control device having the above-described configuration (1), prior to starting the sequence including the above-described series of steps, the imaging control device is inspected for shaking and confirmed to be free from shaking. It is conceivable that the above sequence is started for the first time. In this case, as shown in FIG. 2, a camera shake detector 7 for detecting the vibration of the apparatus main body is further used. The camera shake detector 7 determines that the imaging control device is fixed to, for example, a tripod by detecting a continuous vibration-free state of the imaging control device. That is, it is automatic detection of the remote photographing situation itself by hand shake detection.

In short, in the imaging control device having the above configuration (1), when the vibration of the imaging control device is further detected and it is determined that there is no continuous vibration, a camera shake detector that activates the recognition area setting unit 3 and the image recognition unit 5. 7 is provided.

With this configuration, it is possible to automatically detect that the camera is in the remote shooting state by using a detection result (negative result: no vibration) of the camera shake detection of the apparatus main body by the camera shake detector 7. If the camera shake detector 7 determines that there is vibration of the imaging control device, the recognition area setting unit 3 and the image recognizer 5 are not activated, and the above sequence does not function. If it is determined that there is no continuous vibration, the recognition area setting unit 3 and the image recognition unit 5 are activated, and the above-described series of sequences is made to function.

In a normal hand-held shooting process that is not remote control, when a shooting operator who has the main body wants to take a picture at his intended timing, a person other than the shooting operator who is the subject Due to the movement, it is possible to avoid the inconvenience that the control is performed at a timing contrary to the intention of the photographing operator.

Determining that the imaging control device is in a fixed state can be called “static” detection. As a condition prior to the above [1], [2], [3], [0] If the fixed state determination of the imaging control device that is detection of “static” is put,
[0] Fixed state determination of imaging control device that is detection of “still” [1] Waiting for detection of face area and stabilization of position of detected face area that is detection of “still” [2] Position is stabilized Determination of Image Recognition Area Based on Detected Face Area [3] This is a combination of a series of steps for detecting a predetermined motion in the image recognition area N, which is detection of “motion”. “Still” → “Still” → Image recognition area → “Motion”. Since one condition is added as compared with the case of (1) above, it is possible to improve the reliability of the control, and since the function of the camera shake detector 7 that is generally mounted is used, There is no need to overcomplicate the configuration.

According to the present invention, the image recognition area is determined based on the detected face area whose position is stabilized, the motion vector change in the image recognition area is recognized as a predetermined one, and the result is recognized. As a shooting operator, it is only necessary to stop the movement of the body within the field of view and move a part of the body such as the hand within a specific range equivalent to the image recognition area. Reliable shooting at the timing intended by the operator can be realized. In addition, it is technically relatively easy to determine the image recognition area within a predetermined range on the basis of the detected face area. The image recognition area is relatively small and the accuracy of motion vector change determination is high. Since malfunctions are suppressed, the burden on the device control unit and the calculation unit is greatly reduced, and it is possible to suppress an increase in circuit scale and increase processing speed and efficiency.

FIG. 1 is a block diagram showing a basic configuration of a photographing control apparatus according to the present invention. FIG. 2 is a block diagram (No. 1) showing the configuration of one aspect of the basic configuration of the imaging control apparatus according to the present invention. FIG. 3 is a block diagram (No. 2) showing the configuration of one aspect of the basic configuration of the imaging control apparatus according to the present invention. FIG. 4 is an explanatory diagram of an operation of one aspect of the basic configuration of the imaging control apparatus according to the present invention. FIG. 5 is a block diagram showing the configuration of the imaging control apparatus in the embodiment of the present invention. FIG. 6 is a flowchart showing the operation of the imaging control apparatus in the embodiment of the present invention. FIG. 7 is a subject composition for explaining the operation of the photographing control apparatus in the embodiment of the present invention. FIG. 8 is an explanatory diagram (part 1) of the operation of the imaging control apparatus in the embodiment of the present invention. FIG. 9 is an explanatory diagram (part 2) of the operation of the photographing control apparatus according to the embodiment of the present invention. FIG. 10 is an explanatory diagram of image processing for realizing the conventional automatic photographing mode.

The imaging control device of the present invention having the above-described configurations (1) and (2) can be further advantageously developed in the following embodiment.

(3) Regarding the camera shake detector 7 having the configuration of (2) above, there is a preferable aspect in which it is configured by a gyro sensor. If it is implemented using a gyro sensor, the accuracy of camera shake detection is high while being simple.

(4) The recognition area setting unit 3 configured as described above in (1) to (3) has the following preferable modes. That is, a chest area corresponding to a predetermined range of the chest of the subject person in the detected face region K is determined for the image recognition region N determined within a predetermined range with reference to the stable detected face region K by the face region detector 2. It is. The motion vector change V on the time axis is extracted for an arbitrary operation in this chest area. An imaging operator who is also a subject, when performing an operation for generating a predetermined control signal related to imaging, is the area where the operation should be performed, the area of his chest is the easiest to understand, memorize, and operate easily It is an area. Even on the apparatus side, the relative positional relationship (direction and distance) from the detected face region K is very simple, and the amount of calculation processing can be reduced.

(5) The face area detector 2 configured as described in the above (1) to (4) has the following preferable modes. The face area detector 2 detects the face area of a person in the image data G of a predetermined frame among a plurality of frames constituting the moving image during display of the through image from the image pickup device 1. At this time, it shall detect continuously for every flame | frame.

In general, a series of sequence processing (image signal processing, resizing processing, face detection processing, motion vector detection processing, compression processing, and parallel processing of display processing) for the image data G from the image pickup device 1 is performed while a through image is being displayed. There is a monitor mode to perform. In the sequence process in the monitor mode, the face detection process is also performed. The face area (face position and size information) of a person in a predetermined frame image among a plurality of frames realizing a moving image is continuously detected for each frame, and the information obtained by the detection is used as a motion vector in the next stage. This is passed to the extractor 4.

(6) In the configuration having the camera shake detector 7 of (2) above, the motion vector extractor 4 and the image recognizer 5 have the following preferable modes. If it is determined that the image device is fixed by the camera shake detector 7 during the display of the through image from the image pickup device 1, the motion vector extractor follows the face region detector 2 and the recognition region setter 3. 4 and the image recognizer 5 are activated. This organically connects a series of operations by these plural vessels.

(7) The motion vector extractor 4 configured as described in the above (1) to (6) has the following preferable modes. This will be described with reference to FIG. The moving picture compressor 8 performs moving picture compression on the resized image, but the motion vector extractor 4 is configured to function as an intermediate process of moving picture compression in the moving picture compressor 8, and a motion vector in units of basic blocks. And the temporal change between a plurality of frames constituting the moving image is extracted. Using such a motion vector extractor 4, a motion vector change V on the time axis is extracted for an arbitrary operation in the image recognition region N. Here, the resized image may be an arbitrarily resized moving image of the moving image standard size, or may be an image resized to the display size in the apparatus main body. Generally, the moving picture compressor 8 mounted on the photographing control apparatus has a function of extracting a motion vector as an intermediate process of the moving picture compression. This is to use the motion vector extractor 4.

(8) The image recognizer 5 for recognizing whether the motion vector change V having the configurations (1) to (7) is a predetermined one has the following preferable modes. When the image recognizer 5 recognizes that the motion vector change V during a certain period by the motion vector extractor 4 is a predetermined one during display of the through image from the image pickup device 1, a shooting trigger signal is generated. It is to do. This means that the processing of the main theme of the present invention is performed in a so-called monitor mode.

(9) The image recognizer 5 also has the following preferred modes. The condition for generating the shooting trigger signal is that the motion vector change V during a certain period is a predetermined one, and this is that the motion vector change V is a periodic change. That is, when the image recognizer 5 recognizes that the motion vector change V by the motion vector extractor 4 is a periodic change, it generates a shooting trigger signal. Since the periodic change is captured, the detection accuracy of the motion vector change V is increased.

(10) In addition, the image recognizer 5 may recognize a continuous rotational motion for a certain period in the image recognition area N while displaying the through image from the image pickup device 1.

(11) Further, the image recognizer 5 may recognize a continuous linear motion for a certain period in the image recognition area N while displaying the through image from the image pickup device 1.

(12) When the image recognizer 5 recognizes that the motion vector change V by the motion vector extractor 4 is a large change exceeding the range of the image recognition area N, it determines whether the change is repeated periodically. In addition, there is also an aspect in which a shooting trigger signal is generated when a repetition is recognized. As shown in FIGS. 4A and 4B, when there is a periodic motion vector change V within the image recognition area N, it is relatively easy to determine that it is periodic. On the other hand, when the motion vector change V protrudes from the image recognition area N as shown in (c) and (d) in FIG. 4, the movement is not periodic but transient in the image recognition area N. It will be judged. Therefore, when the motion vector change V protrudes, it is determined whether or not the image recognition region N is repeatedly entered and exited from the edge in the same manner. . When the image recognizer 5 recognizes that the same movement is repeated, the recognition result to that effect is passed to the shooting controller 6, and a predetermined control signal related to shooting is generated by the shooting controller 6. Become.

(13) The imaging controller 6 having the configuration (1) to (12) has the following preferable modes. The control of the release operation is executed when the image recognition of the predetermined recognition object by the image recognizer 5 is established, and the continuation of the image recognition operation is executed when the image recognition is not established.

(14) Regarding the imaging control method described in [Means for Solving the Problems] above,
A step of displaying that the position of the photographing control device is fixed;
Displaying a rectangular frame indicating the face area of the displayed image and its chest area;
When a photographing operator performs a predetermined operation in the chest area, performing a preset recognition confirmation completion operation;
A step for performing a release operation;
Those containing are preferred.

The outline of the present invention has been described above. Embodiments of the present invention will be described below with reference to the drawings.

(Example)
FIG. 5 is a block diagram showing a configuration of an image pickup apparatus on which the photographing control apparatus according to the embodiment of the present invention is mounted. The imaging apparatus 100 is a single-plate digital camera that converts an optical image of a captured subject into digital image data G and records it on a recording medium 35. First, an overall outline of the imaging apparatus 100 will be described, and then the description will be shifted to an explanation of the part corresponding to the invention.

[Explanation of overall overview]
The imaging unit 10 includes an optical lens 11, an optical low-pass filter (LPF) 12, a color filter 13, an imaging device 14, and an analog front end unit 15. The image sensor 14 is an image sensor represented by a CCD type or a CMOS type. A large number of photodiodes (photosensitive pixels) are two-dimensionally arranged on the light receiving surface of the image sensor 14 and photoelectrically convert subject information that has passed through the optical lens 11. The optical low-pass filter 12 has an action of removing a high frequency component equal to or higher than the sampling frequency depending on the pixel pitch of the image sensor 14 and the like (the high frequency component of the key signal is reduced in the final image after image reproduction (signal processing)). Occurrence of a phenomenon that turns back to a low frequency component). The color filter 13 has a predetermined color arrangement such that any one of R, G, and B is present at a position corresponding to one pixel of the image sensor 14, and the color of light incident on the photodiode that is a light receiving element Make a selection. The light that has passed through the optical lens 11 passes through the optical low-pass filter 12 and the color filter 13 and enters the image sensor 14. The subject image formed on the light receiving surface of the image sensor 14 is converted into a signal charge of an amount corresponding to the amount of incident light by each photodiode, and a voltage corresponding to the signal charge based on a pulse supplied from a driver circuit (not shown). It is sequentially read out as a signal (image signal). The image sensor 14 has an electronic shutter function that controls the charge accumulation time (shutter speed) of each photodiode according to the timing of the shutter gate pulse. The operation (exposure, reading, etc.) of the image sensor 14 is controlled by the CPU 21. The image signal output from the image sensor 14 is sent to the analog front end unit 15 and is converted into a digital signal by A / D conversion processing after processing such as analog gain and CDS (correlated double sampling). In addition, in the image pickup device 14 represented by a CMOS type, as a means for realizing high-speed reading, there is a form in which a noise processing unit and an A / D converter are mounted in the device and output as a digital signal directly from the image pickup device. . The A / D-converted image data G is recorded on the recording medium 35 through necessary signal processing according to the operation mode of the imaging apparatus 100 or without signal processing. The imaging apparatus 100 of this example includes a JPEG format, MPEG format, H.264 format. It is possible to record still images and moving images in a compressed data format such as H.264 format, and to record RAW image data (raw data) immediately after A / D conversion. Further, the shake of the image pickup apparatus 100 is detected using the shake detection unit 40, and the detected shake amount is fed back to the sensor unit unit including the optical lens 11 and the image pickup device 14, so that the optical shake correction and the shake correction by the sensor shift are performed. Is called. The camera shake detection unit 40 uses a gyro sensor.

When recording in the compressed data format, the A / D converted image data G is sent to the image signal processing unit 27 via the preprocessing unit 24. The pre-processing unit 24 and the image signal processing unit 27 are synchronized (processing for calculating the color of each point by interpolating a spatial shift of the color signal associated with the color filter array), white balance (WB) adjustment, and gamma correction. , An image processor that performs various processes such as luminance / color difference signal generation, contour enhancement, scaling (enlargement / reduction) processing using an electronic zoom function, and pixel number conversion (resizing) processing. Process the signal. The pre-processing unit 24 and the image signal processing unit 27 include an image memory 26 that can temporarily store an image being processed via the memory control unit 25, and performs image signal processing while using the image memory 26. The image data G that has undergone predetermined signal processing in the preprocessing unit 24 and the image signal processing unit 27 is recorded as standardized size image data or displayed on the liquid crystal monitor via the monitor interface 33. The processing unit 29 changes the image size to the standard size.

For the resized image data G, the face area detection unit 30 detects information such as the position / size / tilt of the person's face as necessary. Further, the resized image data G is sent to the compression / decompression unit 28 and is compressed according to various compression formats. A compression encoding algorithm corresponding to the compression format used at this time is used. MPEG format, H.264 When the image data G is compressed at a standardized size according to a moving image compression data format such as H.264 format, the resize data or the resized image is input from the image memory 26 via the memory control unit 25 in the compression / decompression unit 28 by parallel processing. After the frame data to be read and input are compressed, the compressed data is stored in the memory space by writing back to the image memory 26. At that time, motion detection vector detection processing in units of basic blocks is performed as intermediate processing of moving image compression, and the detected motion detection vector data is stored in the image memory 26 by the memory control unit 25 or inside the compression / decompression unit 28. Can be stored in registers. The compressed image data is recorded on the recording medium 35 via the recording medium interface 31. The recording medium 35 for storing image data is not limited to a semiconductor memory represented by a memory card, and various media such as a magnetic disk, an optical disk, and a magneto-optical disk can be used. Further, the recording medium (internal memory) built in the imaging apparatus 100 is not limited to a removable medium.

The CPU 21 is a control unit that performs overall control of the camera system according to a predetermined program, and controls the operation of each circuit in the imaging apparatus 100 based on an instruction signal from the operation panel 34. The ROM 22 stores programs executed by the CPU 21 and various data necessary for control, and the RAM 23 is used as a work area for the CPU 21.

The operation panel 34 is a device for a user to input various instructions to the imaging apparatus 100. For example, a mode selection switch for selecting an operation mode of the imaging apparatus 100, a menu item selection operation (cursor movement operation), and the like. A cross key for inputting instructions such as frame advance / rewind of playback images, an execution key for instructing selection (registration) and execution of an operation, and a selection item for erasing a desired target and canceling the instruction Various operation devices such as a cancel key, a power switch, a zoom switch, and a release switch are included.

The CPU 21 controls the image pickup unit 10 such as the image pickup device 14 according to various shooting conditions (exposure conditions, presence / absence of strobe light emission, shooting mode, etc.) in accordance with an instruction signal input from the operation panel 34, and automatic exposure (AE). Control, automatic focus adjustment (AF) control, auto white balance (AWB) control, lens drive control, image processing control, read / write control of the recording medium 35, and the like are performed.

For example, the CPU 21 performs automatic focus adjustment (AF) control when detecting half-pressing of the release switch, and starts exposure and reading control for capturing a recording image when detecting full-pressing of the release switch. Further, the CPU 21 sends a command to a strobe control circuit (not shown) as necessary to control the light emission of a flash light emitting tube (light emitting unit) such as a xenon tube.

The pre-processing unit 24 includes an automatic calculation unit that performs calculations necessary for automatic exposure control and automatic focus adjustment control, and performs focus evaluation value calculation and AE calculation based on an image signal captured in response to half-pressing of the release switch. And the result of the calculation is transmitted to the CPU 21. When the release switch is fully pressed, the CPU 21 controls a lens driving motor (not shown) based on the result of the focus evaluation value calculation, moves the optical lens 11 to the in-focus position, and moves the aperture and electronic shutter. To control exposure. Thus, the captured image data G is recorded on the recording medium 35 according to the recording mode. In addition, the digital camera has a display processing unit 32 and a monitor interface 33 in order to realize a display constituted by a liquid crystal monitor.

When carrying out the photographing control method related to the present invention, after the power is turned on, the operation starts in a basic mode called a monitor mode, and the preprocessing is performed on the frame data continuously output from the image sensor 14. , Image signal processing, resizing processing, face area detection processing, compression processing, motion vector detection processing, and display processing are performed in parallel as one sequence processing, and a through image is displayed on the liquid crystal monitor.

[Explanation of the equivalent part of the invention]
The image pickup unit 10 corresponds to the image pickup device 1 in [Means for Solving the Problems] described above, the face region detection unit 30 corresponds to the face region detector 2, and the CPU 21 recognizes the recognition region setting device 3 and the image recognition device 5. The compression / decompression unit 28 includes the function of the motion vector extractor 4, and the program of the CPU 21 corresponds to the shooting controller 6 that generates a predetermined control signal related to shooting according to the recognition result of a predetermined recognition target. The camera shake detection unit 40 corresponds to the camera shake detector 7.

FIG. 6 shows a flowchart of basic operations when the imaging control method related to the present invention is performed using the imaging apparatus 100. In step S <b> 1, the imaging apparatus 100 starts a shooting operation by a power-on operation assigned to the operation panel 34.

Next, in step S2, when the CPU 21 recognizes that the operation mode assigned to the operation panel 34 is the photographing mode, the pre-processing, image signal, and the like are performed on the frame data continuously output from the image sensor 14. Continuous parallel processing is performed with processing, resizing processing, and display processing as one sequence processing. This is called a monitor mode.

Next, in step S3, during the monitor mode operation, a determination process is performed to determine whether the main body of the imaging apparatus 100 is in a handheld state or is fixedly installed. In the determination process, the CPU 21 uses the camera shake detection unit 40 to detect that there is almost no camera vibration, and thus determines that the camera is not held but is fixed in an arbitrary place. The camera shake detection unit 40 is easy to implement using a gyro sensor and has high accuracy. When the camera vibration is detected and it is determined that the camera is hand-held, a normal hand-held shooting process is performed without performing the shooting control process related to the present invention. If it is determined that the shooting is fixed installation, the process proceeds to the next step S4.

In step S4, the CPU 21 performs face area detection processing using the face area detection unit 30. The face area detection process continuously detects the position and size information of a face in a predetermined frame image among a plurality of frames constituting the moving image while displaying a through image from the imaging unit 10 in the monitor mode. The detection information is read by the CPU 21.

Next, in step S5, a motion vector is extracted. That is, when the compression operation of the compression / decompression unit 28 is started and the intermediate processing function is activated, the image is resized to an arbitrary moving image standard size or the display size of the main body, and a motion vector in units of basic blocks is extracted and extracted from the resized image. The motion vector is transferred to the image memory 26. This operation is performed during live view image display in the monitor mode. The motion vector stored in the image memory 26 is read by the CPU 21 via the memory control unit 25.

Next, in step S6, the CPU 21 waits for the position of the detected face area to be stabilized while displaying the through image. If stable, the process proceeds to the next step S7. Next, in step S <b> 7, the CPU 21 determines an area of a predetermined range of the chest position below the human face in the frame image as the image recognition area N.

Next, in step S8, only the motion vector information in the breast area N is extracted, transferred to the image memory 26 via the memory control unit 25, and read out by the CPU 21 to limit the data transfer amount. The CPU 21 serving as the image recognizer 5 detects an assumed sign operation from the temporal change information of the motion vector in the chest area N.

Next, in step S9, the CPU 21 recognizes the motion vector change in the chest area N as an image recognition process. If there is no predetermined change, the process returns to step S6, and recognition processing is performed again to determine whether there is any change in the position and size of the face. Thereafter, steps S6 → S7 → S8 → S9 are repeated, and the CPU 21 simultaneously recognizes the change in the position and size information of the person's face in the field of view and the change in the motion vector in the chest area N as an image recognition process. Go. At that time, when the motion vector in the chest area N changes for a certain period under the condition that the position and size of the face hardly change, the process proceeds to step S10, and a shooting trigger signal for starting the setting operation is obtained. appear.

Next, a case will be described in which the motion trigger signal is generated by recognizing that the motion vector in the chest area N changes periodically for a certain period as a predetermined change during a certain period.

Referring to FIG. 7, a case where family photos are taken will be described. FIG. 7 is a conceptual diagram of setting of the chest area N and image recognition in an image in a family photographic composition. It is assumed that there are three persons in the screen, that is, a father, a mother, and a child, and the photographing operator is a father. After each face area K is detected, a chest area N is set with a rectangular frame at the chest position, and the detection of the face area K and the motion vector at the breast position are detected at the same time to show the state of image recognition. Here, the predetermined operation starts by performing an operation in which the father turns his hand at the chest position. “The action of turning the hand” is not registered as a promise in advance. Specifically, by continuously rotating the hand over a certain period at the chest position, it is recognized that the motion vector periodically changes on the time axis.

A method for recognizing that the motion vector periodically changes V on the time axis by rotating the hand continuously at the chest position over a certain period using (a) and (b) in FIG. Will be described in detail.

(A) in FIG. 8 is a conceptual diagram for explaining a periodic rotation operation in the breast area N of the subject person. FIG. 8B is a conceptual diagram showing a motion vector change V at an arbitrary position on the circumference of the rotation operation.

The motion vector is detected for each of the basic small blocks a, b, c, and d shown in a rectangular shape with a thick solid line in the chest area N surrounded by the dotted line (a) in FIG. When the photographing operator performs a continuous rotational motion at the chest position, the hand moves on the circumference. As a basic operation of motion vector detection, it is possible to detect that the subject person is turning his / her hand at the chest by detecting the temporal change of the motion vector on the circumference by the CPU 21.

Further, by recognizing that the motion vector periodically changes in the chest area N for a certain period of time by rotating the hand continuously, all motion vector changes V in the chest area N are detected. Instead, only a change in the motion vector at a position on an arbitrary circumference in the area is detected assuming circular motion. As the rotation detection, it is possible to recognize clockwise rotation and counterclockwise rotation.

A method for recognizing that a motion vector periodically changes on a time axis by reciprocating a hand up and down or left and right continuously at the chest position using (a) to (d) in FIG. Will be described in detail.

(A) in FIG. 9 is a conceptual diagram for explaining the periodic vertical movement in the chest area N. FIG. 9B is a conceptual diagram showing the motion vector change V at an arbitrary position in the periodic vertical movement. (C) in FIG. 9 is a conceptual diagram for explaining the periodic left-right movement in the chest area N. (D) in FIG. 9 is a conceptual diagram showing a motion vector change V at an arbitrary position in the periodic left-right motion.

The motion vector is detected in the chest area N surrounded by a dotted line, for example, with a basic small block a shown in a rectangular shape with a thick solid line. When the photographing operator continuously moves in the vertical direction at the chest position, the hand moves up and down in an arbitrary a area as shown in FIG. As a basic operation of motion vector detection, the CPU 21 detects a temporal change in the motion vector on an arbitrary area a to detect that the subject person moves his / her hand vertically on the chest. Can do.

When the photographing operator performs a continuous left / right movement at the chest position, the hand moves left and right in an arbitrary a area as shown in FIG. As a basic operation of motion vector detection, the CPU 21 detects a temporal change in the motion vector on an arbitrary a area to detect that the subject person moves his / her hand in the left / right direction at the chest. Can do.

As a predetermined operation based on such recognition, a photographic release operation of a digital camera fixedly installed on a tripod or the like can be automatically performed by performing an operation in which a father, who is also a subject, turns his hand at the chest position.

In particular, in this embodiment, since the target position is automatically specified using the face area detection within the photographing field of view, the periodic operation (cue) can be surely recognized at the specified chest position. it can. Therefore, the size of the target area described above can be minimized as a part of the chest area N, and thereby the processing burden associated with the image recognition (the processing burden on the CPU in this embodiment) can be reduced. it can.

Also, in actual motion vector detection, the vector change of an arbitrary image part may be a large change exceeding the detection range. In that case, it outputs detection flag data of a large change, recognizes that the position of the large change area periodically changes in the chest area, and the subject person periodically moves his / her hand vertically on the chest. It is possible to detect that it is operating quickly.

Further, in the automatic shooting mode in the present embodiment, the processing state on the camera side can be confirmed by notifying the shooting operator that the operation of the target area within the shooting field of view has been recognized by the start of LED blinking. Therefore, the shooting operator or the like can clearly know the operation timing related to the subsequent shooting, and this is also convenient.

As described above, in the present embodiment, the photographing operator who is the subject by periodically rotating the hand at the chest position in the photographing field of view, or periodically performing the vertical movement and the horizontal movement, for example. It is possible to reliably perform shooting at a timing intended by itself.

In other words, a wireless remote control system that does not require a transceiver provides a camera that can be released remotely, automatically detects the remote shooting situation itself by detecting camera shake, and at the time of automatic shooting based on image recognition By simply performing a predetermined periodic sign operation at the chest position with the face of the subject standing still, photographing at the intended timing can be performed reliably. In addition, the specific contents of the photographing operation at that time can be selected as appropriate.

The above embodiment is merely an example, and it goes without saying that various modifications can be made without departing from the spirit of the present invention.

In addition, it is also possible to implement | achieve the procedure of the control processing at the time of implementing this invention, and the function of the means related to it with a program on computers, such as a personal computer and a microcomputer. Further, part or all of the image processing is not limited to a dedicated hardware (signal processing circuit) mode, and a part of the processing may be realized by a program. A program therefor and various recording (storage) media on which the program is recorded are also included in the present invention. Further, it is natural that the processing method according to such a procedure is also included in the present invention.

The present invention is useful for an image processing apparatus such as a digital still camera that performs a release operation remotely through acquired image data.

DESCRIPTION OF SYMBOLS 1 Image pick-up device 2 Face area detector 3 Recognition area setting device 4 Motion vector extractor 5 Image recognition device 6 Shooting controller 7 Camera shake detector 8 Movie compressor 10 Image pick-up part 14 Image pickup device 21 CPU
24 Pre-processing unit 25 Memory control unit 26 Image memory 27 Image signal processing unit 28 Compression / decompression unit (including motion vector detection)
29 Resize processing section 30 Face area detection section 40 Camera shake detection section 100 Imaging device K Detection face area V Motion vector change N Image recognition area (chest area)

Claims

A face area detector for detecting a face area composed of position information and size information of a subject's face in image data sent from an imager;
A recognition area setting unit that monitors whether the face area detected by the face area detector is positionally stable, and determines an image recognition area within a predetermined range based on the stable detection face area;
A motion vector extractor that examines the image recognition region set by the recognition region setter and extracts a motion vector change on a time axis for an arbitrary operation in the image recognition region;
An image recognizer for recognizing whether the motion vector change by the motion vector extractor is a predetermined one;
A shooting controller that generates a predetermined control signal related to shooting according to a recognition result by the image recognizer;
An imaging control device comprising:
Furthermore, when detecting the vibration of the imaging control device and determining that there is no continuous vibration, the camera includes a camera shake detector that activates the recognition area setting device and the image recognizer.
The imaging control apparatus according to claim 1.
The hand shake detector is composed of a gyro sensor,
The imaging control apparatus according to claim 2.
The recognition area setting unit is a chest area corresponding to a predetermined range of a breast of a subject person in the detected face area with respect to the image recognition area defined within a predetermined range with reference to a stable detection face area by the face area detector. Configured to determine,
The imaging control apparatus according to claim 1.
The face area detector is configured to detect the face area of a person in a predetermined frame image among a plurality of frames constituting a moving image for each frame during display of a through image from the imaging device. ,
The imaging control apparatus according to claim 1.
When the motion vector extractor determines that the image device is fixed by the camera shake detector during display of the through image from the image pickup device, the face region detector, the recognition region Configured to activate the motion vector extractor and the image recognizer following a setter;
The imaging control apparatus according to claim 2.
The motion vector extractor is configured to function as an intermediate process of moving image compression, and in a moving image of a moving image standard size that is arbitrarily resized or an image that is resized to a display size in the apparatus main body, motion in units of basic blocks Configured to extract vector direction and temporal changes between multiple frames that make up the video,
The imaging control apparatus according to claim 1.
The image recognizer generates a shooting trigger signal when the motion vector change by the motion vector extractor during a certain period is recognized as a predetermined one during display of the through image from the image pickup device. Configured to,
The imaging control apparatus according to claim 1.
The image recognizer is configured to generate a shooting trigger signal when recognizing that the motion vector change by the motion vector extractor is a periodic change.
The imaging control apparatus according to claim 1.
The image recognizer is configured to recognize a continuous rotational motion for a certain period within the image recognition area during display of a through image from the image pickup device.
The imaging control apparatus according to claim 1.
The image recognizer is configured to recognize continuous linear motion for a certain period in the image recognition area during display of a through image from the image pickup device.
The imaging control apparatus according to claim 1.
When the image recognizer recognizes that the motion vector change by the motion vector extractor is a large change exceeding the range of the image recognition area, the image recognizer determines whether the change is repeated periodically, and repeats Configured to generate the shooting trigger signal when recognizing
The imaging control apparatus according to claim 1.
The imaging controller executes release operation control when image recognition of a predetermined recognition target by the image recognizer is established, and continues image recognition operation when image recognition is not established.
The imaging control apparatus according to claim 1.
Detecting a face area K composed of position information and size information of a subject's face in image data sent by imaging;
Monitoring whether the detected face area K is positionally stable, and determining the image recognition area N within a predetermined range with reference to the stable detected face area K;
Examining the set image recognition area N and extracting a motion vector change V on the time axis for any operation in the image recognition area N;
Recognizing whether the motion vector change V is a predetermined one;
Generating a predetermined control signal related to shooting according to the recognition result of the motion vector change V;
Including a shooting control method.
A step of displaying that the position of the photographing control device is fixed;
Displaying a rectangular frame indicating the face area of the displayed image and its chest area;
When a photographing operator performs a predetermined operation in the chest area, performing a preset recognition confirmation completion operation;
A step for performing a release operation;
The imaging | photography control method of Claim 14 containing these.