JP6330283B2 - Subject tracking device, imaging device, and subject tracking program - Google Patents

Description

  The present invention relates to a subject tracking device, an imaging device, and a subject tracking program.

  There has been provided an imaging device having a so-called subject tracking function for tracking a subject included in an image. In an imaging apparatus having the subject tracking function, for example, when performing live view display, a tracking frame is displayed for a subject to be tracked so that the photographer can easily recognize the frame (see Patent Document 1). When this subject tracking function is used, if the subject repeats minute movements, the tracking frame also follows the subject and repeats minute movements, so that the photographer cannot observe live view display in a concentrated manner. Therefore, it is considered that the tracking frame is not moved when the moving distance of the subject on the live view display is equal to or less than a threshold value.

JP 2010-81587 A

  However, for example, in a live view display in which a subject located at the peripheral edge of the display screen further moves toward the peripheral edge of the display screen, the subject is located at a position where the distance from the edge of the display screen is equal to or less than the threshold value described above. When detected, even if the subject moves, the tracking frame is fixed at a position set immediately before the movement. Similarly, when the subject stops after a minute movement, the tracking frame is fixed at a position based on the position of the subject immediately before the minute movement, not the current subject position. Therefore, there is a problem that the tracking state of the subject in the live view display cannot be properly recognized.

  An object of the present invention is to provide a subject tracking device, an imaging device, and a subject tracking program that allow a photographer to appropriately recognize the tracking state of a subject.

In order to solve the above-described problem, the subject tracking device according to the present invention includes a subject detection unit that detects a subject included in continuously acquired images, and the position of the subject in the first image. Tracking of the subject set for the second image when the position of the subject of the second image acquired after one image moves within a predetermined range and does not satisfy the predetermined condition controls on the position of the tracking frame set for the first image the position of the tracking frame indicating that the position of the tracking frame when satisfying the moving and the predetermined condition within the predetermined range And a controller that moves the subject to the position of the subject in the second image.

The imaging device of the present invention includes the above-described subject tracking device, an imaging device that can continuously acquire images by capturing subject light, a display unit that displays the images, and the display unit And a display control unit that superimposes the tracking frame on the image to be displayed.

The subject tracking program of the present invention is acquired after the first image with respect to a subject detection step for detecting a subject included in continuously acquired images and the position of the subject in the first image. The position of the tracking frame indicating that the subject is being tracked, which is set for the second image when the position of the subject of the second image moves within a predetermined range and does not satisfy the predetermined condition the controlling a position of the tracking frame set for the first image, of the object of the position of the tracking frame when satisfying the moving and the predetermined condition within the predetermined range and the second image The step of moving to a position can be executed by a computer.

  According to the present invention, the photographer can be made to properly recognize the tracking state of the subject.

It is a functional block diagram which shows an example of an imaging device. It is a flowchart which shows the flow of the process at the time of imaging | photography. It is a flowchart which shows the flow of a tracking process. It is a figure which shows the through image acquired, and the change of the position of the tracking frame set based on the to-be-photographed object detected from this through image.

  Embodiments of the present invention will be described below. FIG. 1 is a functional block diagram illustrating an example of an imaging apparatus according to the present embodiment. In addition to a shooting mode for acquiring still images and moving images, the imaging apparatus 10 includes a playback mode for playing back on the monitor 22 image data acquired by shooting and images based on image data recorded on the recording medium 28. ing. Switching between the photographing mode and the reproduction mode described above is executed by operating the operation member 24. In the present embodiment, an image obtained in a shooting standby state in which the release button is not operated and an image obtained after a half-press operation of the release button are referred to as a through image, and when the release button is fully pressed. The acquired image will be described as a captured image.

  The imaging apparatus 10 includes an imaging optical system 15, an imaging element 16, a lens driving circuit 17, an imaging element driving circuit 18, a signal processing circuit 19, a data processing circuit 20, a display control circuit 21, a monitor 22, a compression / expansion circuit 23, and an operation member. 24, a photometric circuit 25, a shake detection sensor 26, a CPU 27, and a recording medium 28. Here, the recording medium 28 includes a nonvolatile memory such as a memory card, an optical disk such as a hard disk, a DVD, and the like. The recording medium 28 may be built in the imaging device 10 or may be detachably attached to the imaging device 10.

  The imaging optical system 15 includes a plurality of lens groups, and forms a subject image on the imaging surface of the imaging element 16. The imaging optical system 15 includes a focus lens (not shown), and the lens drive circuit 17 drives the focus lens to move back and forth in the optical axis direction, thereby performing focus adjustment in the imaging optical system 15. The imaging optical system 15 includes a zoom lens (not shown), and the lens driving circuit 17 drives the zoom lens to move back and forth in the optical axis direction, thereby performing zoom adjustment of the imaging optical system 15. The lens driving circuit 17 generates a lens driving signal in accordance with a lens driving command output from the CPU 27, and moves each lens by driving a lens driving mechanism (not shown) with the generated lens driving signal.

  Further, the imaging optical system 15 moves a predetermined lens of the plurality of lenses in a direction perpendicular to the optical axis of the imaging optical system 15 based on the output of the shake detection sensor 26, thereby obtaining an imaging element. The position of the subject imaged on 16 is shifted. Thereby, shake correction is performed. The movement of the predetermined lens described above is executed via a drive mechanism (not shown). Here, any one of the plurality of lenses constituting the imaging optical system 15 is not configured to move in a direction perpendicular to the optical axis of the imaging optical system 15, but the imaging element 16 is arranged to the optical axis of the imaging optical system 15. It may be configured to move in a direction perpendicular to the direction.

  The image sensor 16 can capture a still image, a continuous image of a still image, and a moving image. The image sensor 16 is composed of, for example, a CCD image sensor or a CMOS image sensor.

  The image sensor drive circuit 18 generates a drive signal in response to a command output from the CPU 27 and supplies the generated drive signal to the image sensor 16. The image sensor 16 accumulates signal charges and reads the accumulated signal charges according to the supplied drive signal. The CPU 27 obtains information on the brightness of the object scene using the photometry data of the object scene obtained by the photometry circuit 25, and based on the obtained information on the brightness, the charge accumulation time of the image sensor 16 and the imaging optical system 15. The aperture, the amplification degree of the image signal output from the image sensor 16 and the like are determined.

  Upon receiving a command from the CPU 27, the signal processing circuit 19 performs amplification processing, direct current reproduction processing, A / D conversion processing, white balance processing, and gamma processing on the input signal, and uses the data after the signal processing as image data. The data is output to the data processing circuit 20.

  The data processing circuit 20 outputs the image data output from the signal processing circuit 19 to the compression / decompression circuit 23 in response to a command from the CPU 27 and also performs resolution conversion processing (pixels) necessary for displaying an image on the monitor 22. (Number conversion) processing is performed on the image data, and the image data after the resolution conversion processing is output to the display control circuit 21. Note that, when performing data zoom processing, the data processing circuit 20 performs resolution conversion processing on input image data and outputs the result to the compression / decompression circuit 23 and the display control circuit 21.

  In addition to this, the data processing circuit 20 performs a tracking process described later when a through image is captured or a moving image is captured in a shooting standby state in which shooting is not performed.

  In response to a command from the CPU 27, the display control circuit 21 performs predetermined signal processing on the image data input from the data processing circuit 20 and outputs the image data to the monitor 22. The display control circuit 21 performs processing for superimposing overlay image data such as a tracking frame indicating the position of the subject in addition to the shooting menu and the cursor on the image data. As a result, an image on which the overlay image is superimposed is displayed on the monitor 22.

  The compression / decompression circuit 23 performs a compression process on the image data output from the data processing circuit 20 in a predetermined format such as a JPEG format in response to a command from the CPU 27, and the compressed data is stored in the recording medium 28. Write. When image data is recorded on the recording medium 28 during shooting, an image corresponding to the image data to be recorded is displayed on the monitor 22. Note that in the case where an instruction to perform non-compression of image data is given by the operation of the operation member 24, the compression / decompression circuit 23 records the image data on the recording medium 28 without performing compression processing.

  When reproducing the image data recorded on the recording medium 28, the compression / decompression circuit 23 reads the image data recorded on the recording medium 28 in accordance with a command from the CPU 27, and decodes the read image data. Process. The decoded image data is output to the data processing circuit 20. The data processing circuit 20 performs resolution conversion processing on the decoded image data and outputs it to the display control circuit 21. As a result, an image based on the image data read from the recording medium 28 is displayed on the monitor 22.

  The operation member 24 includes a release button and outputs an operation signal corresponding to the operation content to the CPU 27. When a release operation signal based on the pressing operation of the release button is input from the operation member 24, the CPU 27 corresponds to a focus detection area set in advance in the imaging screen in the image signal read from the imaging element 16. A known contrast AF (autofocus) operation is performed using the signal.

  Specifically, the CPU 20 sets the lens so that the integrated value (so-called focus evaluation value) of the high-frequency component for the data corresponding to the focus detection area among the image data signal-processed by the signal processing circuit 19 is maximized. A drive command (focus adjustment signal) is output to the lens drive circuit 17. The position of the focus lens that maximizes the focus evaluation value is a focus position that eliminates blurring of the edge of the subject image captured by the image sensor 16 and maximizes the contrast of the image (increases sharpness).

  In addition to the contrast AF operation, a phase difference AF operation by a known pupil division method may be performed. Further, in place of the contrast AF operation, the phase difference AF can be performed by using the imaging element 16 in which focus detection pixels are embedded in the imaging surface.

  The operation member 24 includes a zoom operation member. When a zoom operation signal based on the zoom operation is input from the operation member 24, the CPU 27 outputs a command to the data processing circuit 20, and changes the conversion ratio of the resolution conversion process for the image data based on the operation signal. Thereby, the image displayed on the monitor 22 is enlarged or reduced, that is, electrical zoom adjustment is executed. The resolution conversion ratio corresponds to the electronic zoom magnification. When the data processing circuit 20 changes the conversion ratio in the direction of increasing the electronic zoom magnification, a part of the image to be reproduced is enlarged and displayed on the monitor 22. In this case, the enlargement ratio is increased, but the display range of the image to be reproduced is narrowed. On the other hand, when the data processing circuit 20 changes the conversion ratio in the direction of decreasing the electronic zoom magnification, the enlargement ratio of the reproduced image displayed on the monitor 22 is reduced, but the display range of the reproduced image is widened.

  The shake detection sensor 26 includes, for example, an angular velocity sensor, a gyro sensor, and the like. The shake detection sensor 26 is provided in the housing of the imaging device 10 and detects the shake of the imaging device 10 in, for example, two directions in a plane orthogonal to the optical axis of the imaging optical system 15.

  When shake occurs in the housing, the shake detection sensor 26 detects this and outputs the above-described two-way shake amount data to the CPU 27. The CPU 27 provides a correction optical system constituting the imaging optical system 15 via the lens driving circuit 17 so that the movement of the subject image formed on the imaging surface of the imaging element 16 due to the shake of the housing is canceled. Move. As a result, shake correction is performed.

  Next, the configuration of the data processing circuit 20 will be described. The data processing circuit 20 includes a subject detection unit 31, a subject tracking unit 32, a coring processing unit 33, and a filtering processing unit 34.

  The subject detection unit 31 detects the subject included in the image based on the image data in the initial frame where the subject tracking operation is started, using the image data that has been subjected to the resolution conversion process. As a method for detecting a subject, for example, feature information (for example, feature information of a face) indicating characteristics of a person as a main subject (main subject), or a feature indicating characteristics of a vehicle (car, train, aircraft, etc.) Based on the information or the like, the area of the subject included in the image is detected. Then, image data (hereinafter referred to as subject image) data of the detected main subject region is generated. The subject detection unit 31 also generates subject information indicating the position (coordinates) of the detected subject in the screen and the size of the main subject.

  Here, the generation of the subject image data of the subject detection unit 31 is not limited to automatic generation. For example, the photographer instructs the subject image displayed on the monitor 22 by operating the operation member 24. Thus, the subject detection unit 31 detects the area of the designated subject, generates subject image data, and generates subject information indicating the position (coordinates) of the detected subject in the screen and the size of the main subject. It is good also as a structure.

  The subject tracking unit 32 performs, for example, a template matching process using the subject image data generated by the subject detection unit 31 in the initial frame and the image data subjected to the resolution conversion process. Since this template matching process is well known, its details are omitted here. When the subject is detected in a frame following the initial frame by performing the template matching process, the subject tracking unit 32 generates subject information based on the detected subject. In the above description, the example in which the subject tracking unit 32 detects the subject by the template matching process in the frame following the initial frame is not limited to this, and the initial frame is not limited to this by using other methods such as a particle filter. A configuration may be adopted in which the subject of the frame following is detected.

  The coring processing unit 33 performs the coring process using the position information of the subject detected from each of the through images that move back and forth in time and the position information of the tracking frame displayed on the monitor 22. Here, the tracking frame displayed on the monitor 22 is a tracking frame set for a through image that is temporally previous among through images that are temporally forward and backward.

The coring process is executed according to the following procedure. First, the coring processing unit 33 determines a distance D ₀ between the position of the tracking frame with respect to the through image that is temporally ahead and the position of the subject that is detected from the through image that is temporally later. Is determined (first determination). The position of the tracking frame and the position of the subject are the center position of the tracking frame and the center position of the subject, respectively. As a result of the first determination, when the above-mentioned distance D is equal to or less than the predetermined threshold D ₀ , the coring processing unit 33 confirms whether the positions of the subjects detected from the through images that move back and forth in time coincide with each other. It is determined whether or not (second determination). As a result of the second determination, when the positions of the subjects detected from the through images that are temporally different from each other do not match, the coring processing unit 33 determines the position of the tracking frame with respect to the through image obtained before in time ( In other words, the position of the tracking frame displayed on the monitor 22) is set as the position of the tracking frame with respect to the through image obtained later in time. Therefore, the position of the tracking frame is fixed.

In the second determination described above, when it is determined that the positions of the subjects detected from the through images that move forward and backward in time match, the coring processing unit 33 is detected from each of the through images that move back and forth in time. A determination (third determination) is made as to whether or not the number of times S determined that the positions of the subject to be matched are equal to or greater than a preset threshold value S ₀ . In this third determination, when the number of times S determined that the positions of the subjects detected from the through images that move forward and backward coincide with each other is less than a preset threshold value S ₀ , the coring processing unit 33 The position of the tracking frame with respect to the through image obtained earlier in time is set as the position of the tracking frame with respect to the through image obtained later in time. Therefore, also in this case, the position of the tracking frame is fixed.
In the first determination described above, the distance D between the position of the tracking frame with respect to the through image obtained earlier in time and the position of the subject detected from the through image obtained later in time is a predetermined threshold D. If it exceeds ₀ , the coring processing unit 33 ends the coring processing and executes the processing in the filtering processing unit 34.

Further, in the third determination, when the number of times S determined that the positions of the subjects detected from the through images that move back and forth in time coincide with each other is equal to or greater than a preset threshold value S ₀ , the coring processing unit 33 Then, the coring process is terminated. In this case, the position of the subject detected from the through image obtained later in time is set as the position of the tracking frame in this frame. That is, the tracking frame position is released.

  The filtering processing unit 34 uses the position information of the tracking frame superimposed on the through image obtained earlier in time and the subject information based on the subject detected from the through image obtained later in time. When calculating the display position (coordinates) of the tracking frame, filtering in the time axis direction is performed. For example, the filtering process is a process for obtaining an average value of the position of the tracking frame superimposed on the through image obtained before in time and the position of the subject detected from the through image obtained later in time. It is. In the present embodiment, when the tracking frame position is released, filtering processing is executed.

  Hereinafter, the flow of processing of the imaging apparatus 10 when the photographing mode is set will be described based on the flowchart of FIG.

  Step S101 is processing for capturing a through image. The CPU 27 calculates the brightness of the object scene using the photometric data obtained from the photometric circuit 25. Based on the calculated information about the brightness of the object scene, the CPU 27 determines the charge accumulation time of the image sensor 16, the diaphragm in the image pickup optical system 15, the amplification degree of the image signal output from the image sensor 16, and the like. Then, the CPU 27 outputs a drive signal for driving the image sensor 16 to the image sensor drive circuit 18. Upon receiving this drive signal, the image sensor drive circuit 18 drives the image sensor 16. Therefore, an image signal is output from the image sensor 16 at a frame rate of 30 fps, for example.

  Step S102 is image processing. The signal processing circuit 19 performs amplification processing, DC reproduction processing, A / D conversion processing, white balance processing, and gamma processing on the input image signal, and the data processing circuit 20 uses the signal-processed data as through image data. Output to. The data processing circuit 20 performs resolution conversion processing (pixel number conversion) processing necessary for displaying an image on the monitor 22 on the input through image data, and controls display of the through image data after the resolution conversion processing. Output to the circuit 21.

  Step S103 is processing to display a through image. The display control circuit 21 performs predetermined signal processing on the through image data input from the data processing circuit 20 and outputs the processed signal to the monitor 22. As a result, a through image is displayed on the monitor 22.

  Step S104 is processing for determining whether or not the number n of imaging processing is n ≧ 2. The CPU 27 counts the number n of imaging processes. When the count value n is n ≧ 2, the CPU 27 sets the result of the determination process in step S104 to Yes. In this case, the process proceeds to step S105. On the other hand, when the count value n is not n ≧ 2, in other words, when the count value n is n = 1, the CPU 27 sets the result of the determination process in step S104 to No. In this case, the process proceeds to step S106.

  Step S105 is subject tracking processing. The process of step S105 will be described later.

  Step S106 is subject detection processing. The data processing circuit 20 detects a subject using through image data that has been subjected to resolution conversion processing. Then, the data processing circuit 20 generates subject image data based on the detected subject area. Further, the data processing circuit 20 generates subject information including the detected position and size of the subject.

  Step S107 is processing to display a tracking frame. The data processing circuit 20 outputs the generated subject information to the display control circuit 21. The display control circuit 21 generates a tracking frame based on the input subject information as overlay image data, and superimposes this overlay image data on the acquired through image data. Then, the display control circuit 21 displays the through image data on which the overlay image data is superimposed on the monitor 22. Accordingly, a through image on which the tracking frame is superimposed is displayed on the monitor 22.

  Step S108 is an AF process. The CPU 27 performs a contrast AF (autofocus) operation using the subject area detected by the data processing circuit 20 as a focus detection area. Although the detected subject area is used as the focus detection area, the tracking frame area may be used as the focus detection area.

  Step S109 is a process of determining whether or not there is a half-press operation of a release button that is one of the operation members 24. Although not shown, the release button is provided with a switch (first switch) for detecting the half-press operation and a switch (second switch) for detecting the full-press operation. When the output signal from the first switch of the release button is input to the CPU 27, the CPU 27 sets the result of the determination process in step S109 to Yes. In this case, the process proceeds to step S110. On the other hand, when the output signal from the first switch of the release button is not input to the CPU 27, the CPU 27 sets the result of the determination process in step S109 to No. In this case, the process returns to step S101. Then, the processing from step S101 to step S108 is repeatedly executed until the result of the determination processing in step S109 is Yes.

  Hereinafter, the processing from step S110 to step S115 is the same as the processing of step S101, step S102, step S103, step S105, step S107, and step S108. Hereinafter, description of these processes is omitted. These processes are executed with AE (Auto Exposure) fixed. When the processing from step S110 to step S115 is executed, the process proceeds to step S116.

  Step S116 is processing for determining whether or not the release button is fully pressed. When the output signal from the second switch described above is input to the CPU 27, the CPU 27 sets the result of the determination process in step S116 to Yes. In this case, the process proceeds to step S117. On the other hand, when the output signal from the second switch is not input to the CPU 27, the CPU 27 sets the result of the determination process in step S116 to No. In this case, the process returns to step S110. In this case, the processing from step S110 to step S115 is repeatedly executed until the result of the determination processing in step S116 is Yes.

  Step S117 is a captured image capturing process. The CPU 27 finely adjusts the position of the focus lens constituting the imaging optical system 15 so that the subject area included in the tracking frame is in focus. And an imaging process is performed using the determined imaging condition. By performing this imaging process, an image signal is input from the imaging element 16 to the signal processing circuit 19.

  Step S118 is image processing. The signal processing circuit 19 performs amplification processing, DC reproduction processing, A / D conversion processing, white balance processing, and gamma processing on the input image signal, and the data processing circuit 20 uses the data after the signal processing as photographed image data. Output to.

  Step S119 is processing for displaying a captured image. The display control circuit 21 performs predetermined signal processing on the captured image data input from the data processing circuit 20 and outputs the result to the monitor 22. As a result, the captured image is displayed on the monitor 22.

  Step S120 is processing for recording a captured image. The signal processing circuit 19 outputs the captured image data subjected to the image processing to the compression / decompression circuit 23. The compression / decompression circuit 23 performs compression processing on the input image data in a predetermined format, and records the compressed data on the recording medium 28. When recording image data without compression, the compression / decompression circuit 23 records the image data as it is on the recording medium 28 without performing the above-described compression processing.

  Next, the tracking process in step S105 and step S113 mentioned above is demonstrated using the flowchart of FIG. Here, the processing from Step S203 and Step S207 to Step S209 and Step S211 is the coring processing.

  Step S201 is a process for performing template matching. The data processing circuit 20 executes a template matching process using the subject image data obtained by performing the process of step S106 and the image data subjected to the resolution conversion process. When the subject is detected by performing the template matching process, the data processing circuit 20 generates subject information based on the detected subject.

In step S202, the difference ΔX, ΔY between the position of the tracking frame displayed in the previous frame among the frames preceding and following in time and the position of the subject detected in the subsequent frame among the frames preceding and following in time are calculated. It is processing. The data processing circuit 20 calculates position differences ΔX and ΔY using the position information of the tracking frame displayed on the monitor 22 and the subject information generated in step S201. For example, the position of the tracking frame displayed in the previous frame 1 among the frames that move forward and backward in time (x ₁ , y ₁ ), Assuming that the position of the detected subject is (X ₂ , Y ₂ ), the difference ΔX, ΔY between the position of the tracking frame displayed on the monitor 22 and the position of the subject detected by the process of step S201 is It is calculated by the following formulas (1) and (2).

ΔX = | x ₁ −X ₂ | (1)
ΔY = | y ₁ −Y ₂ | (2)
Step S203 is a tracking frame is determined (first determination) process distance D is whether exceeds the threshold value D ₀ to the object. The data processing circuit 20 obtains the distance D from the tracking frame to the subject using ΔX and ΔY obtained by the above-described equations (1) and (2). The distance D is calculated | required by (3) Formula, for example.

D = [(ΔX) ² + (ΔY) ² ] ^1/2 (3)
The data processing circuit 20 determines whether or not compared with the threshold value D ₀ a predetermined distance D which is obtained by the equation (3) described above, the distance D exceeds the threshold value D _0. If the distance D exceeds the threshold value _{D 0,} the data processing circuit 20, the result of the determination processing in step S203 and Yes, the process proceeds to step S204. That is, if the coring process has been continuously performed before this, the coring process ends. On the other hand, when the distance D becomes the threshold value _{D 0} or less, the data processing circuit 20, the result of the determination processing in step S203 and No, the process proceeds to step S207.

The threshold value D ₀ described above, and the resolution in the through image used is a value set according to the frame rate when acquiring the through image. In other words, when the frame rate is small, the elapsed time from the acquisition of the through image that is temporally previous to the acquisition of the through image that is temporally later from the acquisition of the through image that is temporally previous is acquired. . That is, since the moving distance of the subject detected from the frame images chronologically successive increases, setting a large threshold value D _0. On the other hand, when the frame rate is high, the elapsed time from when the through image that is temporally ahead is acquired until the through image that is temporally later is acquired becomes short. That is, since the moving distance of the subject detected from the frame images chronologically successive shorter, setting a small threshold value D _0.

Here, as the processing in step S203, it determined distance D is determined whether exceeds the threshold value D _0, is not limited thereto, threshold value related to the distance, such as [Delta] X + [Delta] Y It is also possible to set it as the process which determines whether it exceeds.

Step S204 is a filtering process. The data processing circuit 20 obtains the position of the tracking frame with respect to the through image from the position (x ₁ , y ₁ ) of the displayed tracking frame and the position (X ₂ , Y ₂ ) of the subject detected in step S201. Specifically, using the following equations (4) and (5), the position (x ₂ , y ₂ ) of the tracking frame in the subsequent frame among the temporally preceding and following frames is obtained. In other words, the tracking frame position is released by performing the process of step S204.

x ₂ = (x ₁ + X ₂ ) / 2 (4)
y ₂ = (y ₁ + Y ₂ ) / 2 (5)
Step S205 is processing for setting the position of the tracking frame. The data processing circuit 20 sets the value (position) obtained by performing the filtering process in step S204 as the position information of the tracking frame. The data processing circuit 20 then outputs the set tracking frame position information to the display control circuit 22.

  Step S206 is processing to set the value S of the frame counter to 0. The data processing circuit 20 resets the value S of a frame counter (not shown) (S = 0). Here, in the coring process described later, the frame counter determines the position of the subject of the image obtained later in time between the two images preceding and following in time, and the position of the subject of the image obtained earlier in time. Is counted.

In step S203, if the distance D is the threshold _{D 0} below, the process proceeds to step S207.

  Step S207 is processing for determining whether or not the position of the subject detected this time matches the position of the subject detected last time (second determination). Here, the position of the subject detected this time indicates the position of the subject detected from the through image obtained later in time among the images (frames) that change in time. Further, the position of the subject detected last time indicates the position of the subject detected from the through image obtained before in time among the images (frames) that move back and forth in time. When the position of the subject detected this time coincides with the position of the subject detected last time, the data processing circuit 20 sets the result of the determination process in step S207 to Yes. In this case, the process proceeds to step S208. On the other hand, when the position of the subject detected this time does not match the position of the subject detected last time, the data processing circuit 20 sets the result of the determination process in step S207 to No. In this case, the process proceeds to step S211. The processing in step S207 may be performed using only the X coordinate or only the Y coordinate, or may be performed using both the X coordinate and the Y coordinate.

  Step S208 is an increment process for adding 1 to the value S of the frame counter. The data processing circuit 20 has a frame counter (not shown). The data processing circuit 20 adds +1 to the value S of the frame counter.

Step S209 determines whether or not the value S of the frame counter is the threshold value S ₀ or more (third determination) is processed. The data processing circuit 20 determines, by referring to the value S of the frame counter, whether the value S of the frame counter is the threshold value S ₀ or more. If the value S of the frame counter is the threshold value _{S 0} or more, the data processing circuit 20, a Yes result in decision process in step S208. In this case, the process proceeds to step S210. That is, in this case, the coring process ends. On the other hand, if the value S of the frame counter is the threshold S ₀ or less, the data processing circuit 20, and No the result of the determination processing in step S209. In this case, the process proceeds to step S211.

  Step S210 is processing for setting the position of the subject detected this time as the position of the tracking frame. The data processing circuit 20 sets the position of the detected subject as the position of the tracking frame for displaying. That is, the position of the tracking frame is released by executing the process of step S210. When the process of step S210 is executed, the process proceeds to step S206.

  If the result of the determination process in step S207 is No, the process proceeds to step S211.

  Step S211 is processing for fixing the position of the tracking frame. The data processing circuit 20 sets the position information of the tracking frame displayed last time as the position information of the tracking frame displayed this time. That is, the position of the tracking frame is fixed by executing the process of step S211.

  Hereinafter, with respect to the through image and the position of the tracking frame superimposed on the through image when the subject in the position A from time t2 to time t3 moves in the X direction until time t2, with reference to FIG. explain.

  Since the subject is at position A at time t1, the tracking frame 40 is displayed in accordance with the position of the subject detected from the through image P1. Also, the subject is in position A at time t2. That is, the distance D between the position of the tracking frame 40 and the position of the subject detected from the through image P2 at time t1 is D = 0. Therefore, the position of the tracking frame 40 at time t2 is the same position as at time t1.

As described above, the subject moves from position A to position B (in the X direction) while time t2 to time t3 have elapsed. For example, when the X coordinate of the subject detected at time t2 is X _t2 = 90 and the X coordinate of the subject detected at time t3 is X _t3 = 150, the time t3 from the position of the tracking frame 40 at time t2 The distance D to the position of the subject at is D = 60. Here, if the threshold value D ₀ is set to 20, the distance D> threshold D ₀ is satisfied. In such a case, the coring process ends and the filtering process is executed. Here, the position _{x t2} of tracking frame 40 at time _t2, a _x t2 = 90, the position _{X 3} of the object at time t3 _is the X t3 = 0.99. Therefore, the position _{x t3} of the tracking frame 40 at time t3 _{becomes x t3 = (x t2 + X} t3) / 2 = (90 + 150) / 2 = 120.

After time t3, the subject does not move in the X and Y directions. For example, the subject is at position B at time t4. Since the position x t3 of the tracking frame 40 at time _t3 is x _t3 = 120 and the X coordinate of the subject detected at time t4 is X _t4 = 150, the position x _{t3 from} the position of the tracking frame 40 at time t3 is the time t4. The distance D to the subject position is D = 30. Again, the distance D> coring is not performed because satisfying the threshold D _0, the filtering process is executed. Here, the position _{x t3} of the tracking frame 40 at time t3 _is an x t3 = 120, the position _{X t4} of the object at time t4 _is the X t4 = 0.99. Therefore, the position _{x t4} of the tracking frame 40 at time t4 _is a _{x t4 = (x t3 + X} t4) / 2 = (120 + 150) / 2 = 135.

The subject at time t5 is at position B, as is time t4. The position x _t4 = 135 of the tracking frame 40 at time t4 and the position X _t5 of the subject at time _t5 are X _t5 = 150. Therefore, the distance D from the position of the tracking frame 40 at time t4 to the position of the subject at time t5 is D = 15. In this case, it does not satisfy the distance D> threshold D _0. Further, the position of the subject at time t5 has not changed from the position of the subject at time t4. Therefore, the value S of the frame counter is S = 1. Here, if the threshold value S ₀ in the third determination is S ₀ = 2, S ≧ S ₀ is not satisfied. In this case, the position of the tracking frame 40 at time t 5 is the position of the tracking frame 40 at time t 4. Is set at the same position.

The subject at time t6 is at position B, as is time t5. The position x _t5 = 135 of the tracking frame 40 at time t5 and the position X _t6 of the subject at time _t6 are X _t6 = 150. Therefore, the distance D from the position of the tracking frame 40 at time t5 to the position of the subject at time t6 is D = 15. Similar to time t5, even at time t6, it does not satisfy the distance D> threshold _{D 0.} Further, the position of the subject at time t6 has not changed from the position of the subject at time t5. Therefore, the value S of the frame counter is S = 2. As described above, since S ₀ = 2 in the third determination, S ≧ S ₀ is satisfied. Therefore, in this case, the position of the tracking frame 40 at time t6 is the position of the subject at time t6.

  As described above, in the movement of the subject detected from the through image, when the moving distance between the through images is large and exceeds a preset threshold value (first determination is Yes), the coring process is performed. If so, the coring process is terminated and a filtering process is performed. By this filtering process, the position of the tracking frame is released and the position of the tracking frame is newly set. In addition, the movement distance of the subject is small and is equal to or less than a preset threshold (first determination is No), and the positions of the subjects obtained from images that are temporally different do not match (second determination is No). The tracking frame position is fixed by the coring process.

  Further, when the first determination is No and the positions of the subjects obtained from the images that move forward and backward match (the second determination becomes Yes), the positions of the subjects that are obtained from the forward and backward images match. It is determined whether or not the number of times exceeds a threshold value. When the number of times that the positions of the subjects obtained from images that are temporally matched are equal to or greater than the threshold (the third determination is Yes), the coring process is terminated. Accordingly, the fixing of the position of the tracking frame is released, and the detected position of the subject is newly set as the position of the tracking frame. On the other hand, when the number of times the position of the subject obtained from images that are temporally matched is less than the threshold (the third determination is No), the position of the tracking frame is fixed by the coring process.

That is, when the subject is moving slightly, the distance to the subject when the tracking frame position is used as a reference is small. Therefore, the position of the tracking frame set for the image obtained when the subject is moving slightly is fixed. Furthermore, after the object is moved, when the subject stops, the position of the tracking frame is moves gradually closer to the position of the subject, does not satisfy the distance D> threshold D ₀ and determination It is fixed at the set position before being done. However, if the actual subject position is stopped, it is desirable to set a tracking frame at the position of the subject to be stopped. Therefore, in the present invention, the number of times that the position of the subject in the through image that moves back and forth in time coincides is counted, and if the count is equal to or greater than the threshold, it is determined that the subject has not moved temporarily. Then, the position of the tracking frame is set to the position of the detected subject. Therefore, it is possible to display the tracking frame appropriately while preventing a minute movement of the tracking frame.

Further, in the conventional object tracking process fixed, the subject is located in the threshold D ₀ the range from the end portion of the through-image, also, to a position before the subject when moving toward the end portion, the tracking frame is to be moved Is done. However, in the subject tracking according to the present invention, the subject is, the subject is in the range from the end following threshold D ₀ of the through-image, also, when the subject moves toward the end portion, the position of the object is one If the number of times played is a predetermined number, it is determined that the subject is at that position, and the tracking frame is moved to that position. Therefore, it is possible to display the tracking frame appropriately.

In the present embodiment, the tracking frame is displayed on the through image obtained in the shooting standby state. However, the present invention is not limited to this, and the tracking process is performed in the process of capturing the moving image. It can also be applied to the case where it is performed.
(Modification)
In the present embodiment, when the distance D from the tracking frame to the subject becomes the threshold value D ₀ or less, and when the number of times the position of the object does not change is continuous predetermined number, the position of the detected object as a position of the tracking frame It is set. However, the present invention is not limited to this. For example, when the number of times that the position of the subject does not change is equal to or greater than a predetermined number in a through image obtained in a predetermined period, the detected position of the subject is set as the position of the tracking frame. May be.

In the above-described embodiment, the configuration for determining whether or not to continue the coring process in step S207 based on whether or not the position of the subject detected this time coincides with the position of the subject detected last time will be described. However, instead of this, in step S207, it is determined whether or not the distance between the position of the subject detected this time and the position of the subject detected last time is less than a predetermined value P. If the determination is affirmed, the process proceeds to step S208. If the determination is negative, the process may proceed to step S211. Here, the value of the predetermined value P is smaller than the threshold value D _0. In this way, the tracking frame display is fixed by comparing a predetermined value of a value related to the distance between the position of the subject detected from the first image and the position of the subject detected from the second image. If it is configured to cancel when the number of times the magnitude relationship matches exceeds a preset threshold, the position of the subject detected this time and the position of the subject detected last time are accurate due to the influence of noise included in the image data. Even if the position is slightly shifted, the coring process can be performed. Note that the process of determining whether the distance between the position of the subject detected this time and the position of the subject detected last time is less than the predetermined value P is not performed in place of step S207, but in conjunction with the determination in step S207. Is also possible.

  In the present embodiment, the imaging apparatus is described as an example, but the configuration may be a data processing circuit illustrated in FIG. 1 or a subject tracking apparatus capable of executing the processing of the flowchart in FIG. . In addition, a subject tracking program that allows the computer to execute the configuration of the data processing circuit shown in FIG. 1 or the processing of the flowchart of FIG. 3 may be used. The subject tracking program is preferably recorded on a computer-readable recording medium such as an optical disk such as a memory card or a hard disk or a magnetic disk.

  DESCRIPTION OF SYMBOLS 10 ... Imaging device, 16 ... Imaging device, 20 ... Data processing circuit, 21 ... Display control circuit, 22 ... Monitor, 31 ... Subject detection part, 32 ... Subject tracking part, 33 ... Coring processing part, 34 ... Filtering processing part

Claims (7)

A subject detection unit for detecting a subject included in continuously acquired images;
When the position of the subject of the second image acquired after the first image moves within a predetermined range with respect to the position of the subject of the first image and does not satisfy the predetermined condition, the second The position of the tracking frame that is set for the image and indicates that the subject is being tracked is controlled to the position of the tracking frame that is set for the first image, and moves within the predetermined range and A control unit that moves the position of the tracking frame to the position of the subject in the second image when a predetermined condition is satisfied;
Ru includes a subject tracking apparatus. The subject tracking device according to claim 1,
When the number of times that the position of the subject detected from the first image matches the position of the subject detected from the second image exceeds a predetermined threshold, the control unit determines that satisfies the subject to move the position of the tracking frame to be set for the second image, the position of the object of the second image from the position of the tracking frame set for the first image Tracking device. The subject tracking device according to claim 2,
A first determination unit that determines whether the position of the subject detected from the first image matches the position of the subject detected from the second image;
A counting unit that counts the number of times the position of the subject in the first image matches the position of the subject in the second image;
A second determination unit that determines whether the number of times counted by the counting unit exceeds the threshold;
A subject tracking device comprising: The subject tracking device according to claim 1,
The control unit satisfies the predetermined condition when a distance between a position of the tracking frame with respect to the first image and a position of the subject detected from the second image exceeds a preset value. determined, the position of the tracking frame to be set for the second image, the object tracking device for moving the position of the object of the second image from the position of the tracking frame set for the first image. The subject tracking device according to claim 4,
The preset value is a subject tracking device that changes in accordance with a frame rate of continuously acquired images. A subject tracking device according to any one of claims 1 to 5,
An image sensor capable of continuously acquiring images by capturing subject light; and
A display unit for displaying the image;
A display control unit for displaying the image on the display unit and superimposing the tracking frame on the image to be displayed;
An imaging apparatus comprising: A subject detection step of detecting a subject included in continuously acquired images;
When the position of the subject of the second image acquired after the first image moves within a predetermined range with respect to the position of the subject of the first image and does not satisfy the predetermined condition, the second The position of the tracking frame that is set for the image and indicates that the subject is being tracked is controlled to the position of the tracking frame that is set for the first image, and moves within the predetermined range and Moving the position of the tracking frame to the position of the subject in the second image when a predetermined condition is satisfied;
Subject tracking program that can be executed by a computer.

Images