JP2019197310A - Video analysis device and program thereof - Google Patents

Abstract

To provide a video analysis device capable of displaying a moving amount of a subject in real time.SOLUTION: A vide analysis device 1 includes: down conversion means 20 for generating a reduced image by down converting a photographed video; optical flow calculation means 30 for calculating an optical flow of the reduced image; moving amount association means 40 for linearly complementing a moving amount of the optical flow in association with a resolution and a frame rate of the photographed video; and moving amount display means 50 for synthetically displaying a moving direction of the optical flow and the linearly complemented moving amount of the optical flow in the reduced image.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a video analysis apparatus that analyzes the amount of movement of a subject included in a video and a program thereof.

  Conventionally, in a surveillance system such as a security camera, an optical flow is used to determine a specific movement of a subject (Patent Documents 1 and 2). There are several calculation methods for this optical flow. As a typical method, a feature point having a large eigenvalue is extracted, and an iterative calculation is performed from a small image pyramid by an LK (Lucas-Kanade) method.

JP, 2006-207185, A JP 2012-22370 A

  When shooting a subject, the photographer mainly checks the camera image using the viewfinder and adjusts the angle of view according to the size and movement of the subject. Here, when there is a motion in the camera image, since the appearance of the image differs depending on the resolution and the frame rate, it is necessary to know the amount of movement of the subject suitable for the resolution and the frame rate. In other words, in video production, it is necessary for the photographer to know the movement of the subject suitable for the resolution and the frame rate, and a method using the optical flow as an index is examined. For example, by calculating the optical flow from temporally continuous images, the amount of movement of the subject may be digitized as a motion vector, and the motion vector corresponding to the amount of movement of the subject may be displayed on the viewfinder and visualized.

  However, when the optical flow is used as an index for video production, there are the following problems. Here, as the frame rate of the camera video increases, the movement of the subject decreases between temporally continuous images, so that an optical flow with high accuracy can be calculated. On the other hand, when the resolution and frame rate of the camera image are increased, the calculation of the optical flow is not in time, so that the display frame rate is reduced and the real-time property is lost.

  Therefore, an object of the present invention is to provide a video analysis apparatus capable of displaying a movement amount of a subject in real time and a program thereof.

  In view of the above-described problems, a video analysis apparatus according to the present invention is a video analysis apparatus that analyzes a movement direction and a movement amount of a subject, and includes a down-conversion unit, an optical flow calculation unit, and a movement amount association unit. And a moving amount display means.

According to this configuration, the video analysis device generates a reduced video by down-converting at least one of the resolution and the frame rate of the input video by the down-conversion unit.
The video analysis device calculates the optical flow of the reduced video by the optical flow calculation means.
As described above, the video analysis apparatus can suppress the amount of calculation of the optical flow by down-conversion.

The video analysis apparatus linearly complements the movement amount of the optical flow calculated by the optical flow calculation unit in association with the resolution and the frame rate of the input video by the movement amount association unit.
The video analysis apparatus uses the movement amount display means to calculate the movement direction of the optical flow calculated by the optical flow calculation means and the movement amount of the optical flow linearly complemented by the movement amount association means as the movement direction and movement amount of the subject. , And composite display on the reduced video.
As described above, the video analysis apparatus linearly complements the movement amount of the optical flow in association with the resolution and the frame rate of the input video.

  Note that the video analysis apparatus according to the present invention can also be realized by a video analysis program that causes a general computer to operate cooperatively as the above-described means.

According to the present invention, the following excellent effects can be obtained.
According to the present invention, the amount of optical flow calculation is suppressed by down-conversion, and the amount of movement of the optical flow is linearly complemented in association with the resolution and frame rate of the input video, so that the amount of movement of the subject is displayed in real time. Can do.

1 is an external view of a camera equipped with a video analysis device according to a first embodiment. It is a block diagram which shows the structure of the video analysis apparatus which concerns on 1st Embodiment. It is explanatory drawing explaining the reduction | restoration image | video in 1st Embodiment. It is explanatory drawing explaining an example of the movement amount matching in a horizontal direction in 1st Embodiment. It is explanatory drawing explaining an example of the movement amount matching to a perpendicular direction in 1st Embodiment. It is explanatory drawing explaining an example of the movement amount matching in the diagonal direction in 1st Embodiment. It is explanatory drawing explaining an example of the synthetic | combination image | video in 1st Embodiment. It is explanatory drawing explaining the example of a display of the optical flow in 1st Embodiment. It is explanatory drawing explaining an example of the synthetic | combination image | video in 1st Embodiment, (a) represents when the moving amount of a to-be-photographed object is small, (b) represents when the moving amount of a to-be-photographed object is large. It is explanatory drawing explaining an example of the warning display in 1st Embodiment. It is a flowchart which shows operation | movement of the video-analysis apparatus of FIG. It is a block diagram which shows the structure of the video analysis apparatus which concerns on 2nd Embodiment. In 3rd Embodiment, it is an external view of the video monitoring apparatus incorporating a video analysis apparatus. It is a block diagram which shows the structure of the video analysis apparatus which concerns on 3rd Embodiment. It is explanatory drawing explaining the reduction | restoration image | video in the modification 1. FIG.

(First embodiment)
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. In each embodiment, the same means is denoted by the same reference numeral, and description thereof is omitted.

[camera]
With reference to FIG.1 and FIG.2, the structure of the video-analysis apparatus 1 which concerns on 1st Embodiment is demonstrated.
In the first embodiment, it is assumed that the video analysis device 1 is attached to the camera C as shown in FIG.
First, prior to the description of the video analysis apparatus 1, the camera C will be briefly described. For example, the camera C is a general photographing camera capable of photographing an ultra-high definition video (for example, resolution 7680 × 4320 pixels, frame rate 120 Hz). The camera C includes a camera body _{C B,} and a viewfinder _{C V.} The camera body C _B is constituted by an optical system and an imaging element such as a lens or the like, and generates a captured image that includes the subject (input image). The viewfinder _CV displays a reduced image of the captured image so that the cameraman can check the captured image.

[Video analysis equipment]
With reference to FIG. 2, the configuration of the video analysis apparatus 1 will be described.
As shown in FIG. 2, the image analysis apparatus 1 is input captured image from the camera body C _B, down-converts the input captured image to generate a reduced image, the generated reduced image to the viewfinder C _V indicate. At this time, the video analysis apparatus 1 analyzes the moving direction and moving amount of the subject and synthesizes and displays the analysis result on the reduced video. The video analysis apparatus 1 includes a video input unit 10, a down-conversion unit 20, an optical flow calculation unit 30, a movement amount association unit 40, and a movement amount display unit 50.

The video input means 10 receives a captured video from the camera C (camera body C _B ), and acquires the resolution and frame rate (for example, resolution 7680 × 4320 pixels, frame rate 120 Hz) of the input captured video. . Then, the video input unit 10 outputs the input captured video to the down-conversion unit 20. Further, the video input unit 10 outputs pre-reduction resolution / frame rate information indicating the resolution and frame rate of the captured video to the movement amount association unit 40.

  The down-conversion unit 20 generates a reduced video by down-converting the captured video from the video input unit 10. In the present embodiment, the down-conversion means 20 down-converts the captured video to a realistic resolution and frame rate at which optical flow calculation and composite display can be performed in real time (for example, resolution 1920 × 1080 pixels, frame rate 30 Hz). . Further, since the entire captured video L is set as a reduced area, the entire captured video L is reduced and a reduced video S is generated as shown in FIG. The reduced area is an area where the reduced image S is generated in the captured video L.

  Here, the down-conversion means 20 can perform down-conversion by a general method. For example, the down-converting means 20 thins out pixels after applying a smoothing filter to the captured video for resolution, and thins out the frames constituting the captured video at equal intervals for the frame rate. In addition, the down-converting means 20 may down-convert using frame blending such as outputting an average of two frames as one frame.

  Thereafter, the down-conversion unit 20 outputs the down-converted reduced video to the optical flow calculation unit 30 and the movement amount display unit 50. Further, the down-conversion unit 20 outputs post-reduction resolution / frame rate information indicating the resolution and frame rate of the reduced video to the movement amount association unit 40.

  The optical flow calculation unit 30 calculates the optical flow of the reduced video from the down-conversion unit 20. That is, the optical flow calculation means 30 calculates the optical flow of feature points having eigenvalues from consecutive frames in the reduced video. For example, the optical flow calculation unit 30 can calculate the optical flow by a general method such as the LK method or the Horn-Shunk method. In the present embodiment, the optical flow calculation unit 30 calculates an optical flow by the LK method.

Here, the optical flow calculation unit 30 may extract the feature points from the entire reduced video and calculate the optical flow.
On the other hand, the optical flow calculation means 30 may extract the feature points only from the detection area set in advance in the reduced video and calculate the optical flow. This detection area is an area for which an optical flow is to be calculated in the reduced video, and a user such as a cameraman can set an arbitrary size and shape. In the present embodiment, it is assumed that the detection area is set in a rectangular shape at the center of a reduced video image where there is a high possibility that a subject exists. In this case, the optical flow calculation unit 30 can calculate the optical flow with higher accuracy because the spatial frequency component of the detection region is less likely to be lost than the entire reduced image.
Thereafter, the optical flow calculation unit 30 outputs the optical flow calculated for each feature point to the movement amount association unit 40.

  The movement amount association unit 40 linearly complements the movement amount of the optical flow calculated by the optical flow calculation unit 30 in association with the resolution and the frame rate of the captured video.

  First, the movement amount association unit 40 uses the pre-reduction resolution / frame rate information from the video input unit 10 and the post-reduction resolution / frame rate information from the down-conversion unit 20 to use the resolution reduction rate A and the frame rate. A reduction rate B is calculated. In this embodiment, the resolution reduction rate A is based on the ratio of the number of pixels of the reduced video and the shot video because the entire shot video is reduced (the number of pixels of the reduced video is the number of pixels of the shot video). division). For example, since the captured video is 7680 × 4320 pixels and the reduced video is 1920 × 1080 pixels, the resolution reduction ratio A = (1920 × 1080) / (7680 × 4320) = 1/16. The frame rate reduction rate B is based on the ratio of the frame rate between the reduced video and the shot video (the frame rate of the reduced video is divided by the frame rate of the shot video). For example, since the captured video is 120 Hz and the reduced video is 30 Hz, the frame rate reduction rate B = 30/120 = 1/4. In this way, the movement amount association means 40 calculates the resolution reduction rate A and the frame rate reduction rate B from the ratio between the resolution and the frame rate before and after down-conversion regardless of whether or not there is a filter or frame blending.

  Next, the movement amount association means 40 makes the movement amount of the optical flow obtained by the optical flow calculation means 30 linearly complement with both the resolution and the frame rate to correspond to the captured video. Specifically, the movement amount associating means 40 uses the following equation (1), where x is the movement amount of the optical flow obtained by the optical flow calculating means 30, and the optical flow associating with the captured video. The amount of movement is calculated. Hereinafter, the movement amount after linear interpolation is assumed to be y.

  Thereafter, the movement amount associating means 40 sends movement direction / movement amount information indicating the movement direction of the optical flow from the optical flow calculation means 30 and the movement amount y of the optical flow with linear interpolation to the movement amount display means 50. Output.

<Association of movement amounts in the horizontal and vertical directions>
With reference to FIGS. 4 to 6, the correspondence of the movement amounts in the horizontal direction, the vertical direction, and the oblique direction will be specifically described.
In the upper part of FIGS. 4 to 6, reduced video (resolution 1920 × 1080 pixels, frame rate 30 Hz) is illustrated. 4 to 6 show the first photographed video (resolution 3840 × 2160 pixels, frame rate 60 Hz). 4 to 6 show the second photographed video (resolution: 7680 × 4320 pixels, frame rate: 120 Hz).
4 to 6, the squares represent pixels, the hatched squares represent pixels that are feature points, and the horizontal axis represents the time axis.
4 to 6 show captured images with different resolutions and frame rates for the sake of simplicity, actually, it is not necessary to handle captured images with different resolutions and frame rates at the same time.

  First, consider the relationship between the reduced video and the first captured video. As shown in FIGS. 4 and 5, it is assumed that the feature point has moved one pixel horizontally or vertically between frames of the reduced video (x = 1). Since the resolution of the first captured video is 3840 × 2160 pixels, the resolution reduction rate A = 1/4, and one pixel movement in the reduced video corresponds to two pixels of the first captured video. In other words, when the frame rate reduction rate B in the equation (1) is ignored, the movement amount y = 1 / √A = 1 / √ (1/4) = 2 pixels.

  Further, since the frame rate of the first photographed video is 60 Hz, the frame rate reduction rate B = 1/2, and the movement amount per frame (1/30 frame → 2/30 frame) in the reduced video is the first photographed video. This corresponds to a movement amount of 2 frames (1/60 frame → 3/60 frame). At this time, the amount of movement per frame in the first captured video of 60 Hz can be obtained by linearly interpolating (× 1/2) the amount of movement of two frames of the first captured video. Specifically, the movement amount y per frame in the first photographed video is 1 ÷ √ (1/4) × 1/2 = 1 pixel from the equation (1).

  In this way, consider a case where one pixel is moved between 1/30 frame and 2/30 frame in the reduced video (see the upper part of FIGS. 4 and 5). In this case, in the first photographed image, one pixel moves between 1/60 frame and 2/60 frame, and one pixel moves between 2/60 frame and 3/60 frame. (Refer to the middle of FIGS. 4 and 5).

  Next, consider the relationship between the reduced video and the second captured video. Since the resolution of the second captured video is 7680 × 4320 pixels, the resolution reduction ratio A = 1/16, and one pixel movement per frame in the reduced video corresponds to 4 pixels of the second captured video. Further, since the frame rate of the second captured video is 120 Hz, the frame rate reduction rate B = 1/4, and the movement amount per frame (1/30 frame → 2/30 frame) in the reduced video is the second captured video. This corresponds to a movement amount of 4 frames (1/120 frame → 5/120 frame). At this time, the amount of movement per frame in the second photographed image at 120 Hz can be obtained by linearly interpolating the amount of movement for four frames of the second photographed image (× 1/4). Specifically, the movement amount y per frame in the second photographed image is 1 ÷ √ (1/16) × 1/4 = 1 pixel from the equation (1).

  In this way, consider a case where one pixel is moved between 1/30 frame and 2/30 frame in the reduced video (see the upper part of FIGS. 4 and 5). In this case, in the second shot image, one pixel is moved between 1/120 and 2/120 frames, and one pixel is moved between 2/120 and 3/120 frames. Similarly, since one pixel is moved, a total of four pixels are moved (see the lower part of FIGS. 4 and 5).

<Association of movement amount in diagonal direction>
As shown in FIG. 6, the amount of diagonal movement per frame is obtained as the square of the sum of the square of the horizontal movement amount component and the square of the vertical movement amount component. Therefore, the resolution reduction rate A and the frame rate reduction rate B are all the same in the horizontal direction, the vertical direction, and the oblique direction. Here, since the horizontal movement amount component is one pixel and the vertical movement amount component is one pixel in the reduced video, the horizontal movement direction and the vertical movement direction as described with reference to FIGS. 4 and 5. Is the same correspondence.

  As described above, even when the resolution and frame rate of the captured video are different, the movement amount per frame in the reduced video can be associated with the movement amount in the captured video regardless of the horizontal, vertical, and diagonal directions. Thereby, the movement amount of the optical flow F in 2/60 frames in FIGS. 4 to 6 and the movement amount of the optical flow F in 2/120 frames to 4/120 frames can be linearly complemented.

Returning to FIG. 2, the description of the configuration of the video analysis device 1 will be continued.
The movement amount display means 50 combines the movement direction / movement amount information from the movement amount association means 40 with the reduced image from the down-conversion means 20 and displays it on the viewfinder _CV as the movement direction and movement amount of the subject. To do. That is, the movement amount display means 50 combines the movement direction of the optical flow F calculated by the optical flow calculation means 30 and the movement amount y of the optical flow F linearly complemented by the movement amount association means 40 into a reduced image. Generate video.

In the present embodiment, the movement amount display means 50 synthesizes movement direction / movement amount information with a delay of one frame with respect to the reduced video. In the example of FIG. 4, the moving amount display means 50 combines the moving direction / moving amount information obtained from the reduced video from 1/30 frame to 2/30 frame with the reduced video of the 2/30 frame ( (Not shown in FIG. 3). Note that the movement amount display means 50 may combine the obtained movement direction / movement amount information with a reduced image by delaying it by one frame or more.
Thereafter, the movement amount display means 50 outputs the generated composite video to the viewfinder _CV .

<Example of composite video>
An example of the composite image generated by the movement amount display means 50 will be described with reference to FIG. 7. As shown in FIG. 7, in the composite image α, the moving direction and the movement amount of the subject T are represented by arrows.
In FIG. 7, a rectangular detection region β is set at the center of the synthesized video α. Therefore, the optical flows F ₁ and F _{2 of the} subjects T ₁ and T ₂ included in the detection region β among the three subjects T included in the synthesized video α are calculated. On the other hand, the object T ₃ which is not included in the detection area β is the optical flow is not calculated. Therefore, in the composite video α, only the optical flows F ₁ and F _{2 of} the subjects T ₁ and T ₂ are displayed, and the optical flow of the subject T ₃ is not displayed. In FIG. 7, the detection area β may not be displayed in the synthesized video α.

  The direction of the optical flow F represents the movement direction of the subject T, and the length of the optical flow F represents the movement amount of the subject T. That is, the longer the optical flow F, the larger the movement amount of the subject T, and the shorter the optical flow F, the smaller the movement amount of the subject T.

Here, the movement amount display means 50 may color-divide the optical flow F for each of several stages of movement amount ranges and change the length of the optical flow F. The value, color, and length of the movement amount range can be arbitrarily set by the user at each stage. In FIG. 7, the movement range and color of the optical flow F are 1 to less than 5 dark blue, 5 to less than 15 light blue, 15 to less than 30 green, 30 to less than 100 yellow, 100 to less than 500 red. As shown in FIG. Further, in FIG. 7, the length of the optical flow F is represented in five stages in correspondence with this movement amount range. For example, the optical flow F ₁ of the object T _1, since the amount of movement is less than 15 or more 30, are displayed in green at moderate length. Also, optical flow F ₂ of the object T ₂ are, the amount of movement since 5 or more and less than 15, less than the optical flow F _1, are displayed in light blue.

Further, the movement amount display means 50 may display the set movement amount range value and color on the left side of the composite video α as setting information. Further, the movement amount display means 50 may display the ratio of the optical flow F included in the movement amount range as the analysis information. In Figure 7, the optical flow F is the total of 10, contains five optical flows F ₁ to the movement amount range of less than 15 or more 30, five in the movement amount range of less than 5 or 15 optical flow F ₂ It is included. For this reason, the ratio of the optical flow F is 50% in the movement amount range of 5 or more and less than 15 and 15 or more and less than 30, and 0% in the other movement amount ranges.

Further, the movement amount display means 50 may display the number of feature points corresponding to the optical flow F on the lower left side of the synthesized video α as analysis information. In FIG. 7, the number of feature points is ten, the same as the number of optical flows F.
Furthermore, the movement amount display means 50 may calculate the movement amount average value of the optical flow F in each of the omni-direction, the vertical direction, and the horizontal direction, and display it at the lower center of the synthesized video α. In FIG. 7, the movement amount average value of the optical flow F is 20.2 in all directions, 14.5 in the vertical direction, and 15.7 in the horizontal direction.

Further, the movement amount display means 50 may display only the optical flow F only in the movement amount range when the user sets a movement amount range to be displayed. For example, the movement amount display means 50 displays only the optical flow F having a movement amount range of 15 or more and less than 30.
Further, the movement amount display means 50 may display the peripheral side of the arrow representing the optical flow F in a different color as shown in FIG. For example, when the inside of the arrow representing the optical flow F is green or light blue, the peripheral side of the arrow is black or white.

In FIG. 8, the synthetic video α is illustrated abstractly in order to explain the optical flow F, but actually, it is as shown in FIG. 9 (an example of photographing a natural landscape). In FIG. 9A, since the amount of movement of the subject is small compared to FIG. 9B, the arrow indicating the optical flow F is shorter.
Note that the movement amount display means 50 does not need to display the setting information and the analysis information as shown in FIG. 9 when it is difficult to see the setting information and the analysis information on the synthesized video α.

<Example of warning display>
An example of warning display by the movement amount display means 50 will be described with reference to FIG. 10. As shown in FIG. Therefore, it is preferable to avoid sudden camera work. Therefore, the movement amount display means 50 determines a sudden camera work based on the movement amount of the feature point (movement amount of the optical flow F), and displays a warning when there is a concern about motion sickness.

Specifically, the movement amount display means 50 obtains the number of feature points whose movement amount exceeds a preset first threshold value. Next, the movement amount display means 50 calculates a ratio between the number of feature points exceeding the first threshold and the number of all feature points extracted from the reduced video. Then, the movement amount display means 50 displays a warning by an arbitrary method when the calculated ratio exceeds a preset second threshold value. The first threshold and the second threshold can be arbitrarily set by the user. In the present embodiment, the movement amount display means 50 displays a warning message W “Warning!” On the lower right side of the composite video α in FIG. 10 as a warning display.
Note that the warning message W is arbitrary, and may be, for example, “Rate of Arrow5 Exceed ○%” (“◯” is a numerical value indicating an excess with respect to the second threshold).

  By this warning display, the video analysis apparatus 1 can prompt a user such as a cameraman to suppress the pan speed. Thereby, the user can intentionally avoid a movement that gives a warning in his camera work.

[Operation of video analyzer]
The operation of the video analysis device 1 will be described with reference to FIG.
As shown in FIG. 11, the image input unit 10, the captured image from the camera body C _B are input, obtains a resolution and frame rate of the input captured image (step S1).

  The down-conversion unit 20 generates a reduced video by down-converting both the resolution and the frame rate of the captured video input to the video input unit 10. For example, the down-conversion unit 20 down-converts the captured video by thinning out pixels and frames of the captured video (step S2).

  The optical flow calculation unit 30 calculates the optical flow F of the reduced video generated by the down-conversion unit 20. For example, the optical flow calculation means 30 calculates the optical flow F by the LK method (Step 3).

The movement amount association unit 40 linearly complements the movement amount of the optical flow F calculated by the optical flow calculation unit 30 in association with the resolution and frame rate of the captured video.
First, the movement amount association means 40 calculates the resolution reduction rate A and the frame rate reduction rate B from the ratio between the resolution and frame rate of the captured video and the resolution and frame rate of the reduced video.
Next, the movement amount associating means 40 uses the above-described equation (1) to linearly complement the movement amount of the optical flow F obtained in step S3 with both the resolution and the frame rate to obtain a captured video. Correspond (step S4).

The movement amount display means 50 combines the movement direction of the optical flow F calculated in step S3 and the movement amount of the optical flow F linearly complemented in step S4 into a reduced image as the movement direction and movement amount of the subject T, The image is displayed on the viewfinder _CV (step S5).

[Action / Effect]
As described above, the video analysis apparatus 1 suppresses the calculation amount of the optical flow F by down-conversion, and linearly complements the movement amount of the optical flow F in association with the resolution and the frame rate of the captured video. The amount of movement can be displayed.
Thereby, in the video analysis apparatus 1, when the cameraman visually recognizes the viewfinder _CV , the movement amount of the subject can be known in real time, and when the movement amount to be photographed is determined from the resolution and the frame rate of the produced video, The zoom and angle of view of the camera C can be adjusted according to the amount of movement. Furthermore, in the video analysis apparatus 1, since the movement amount of the feature point of the captured video is displayed on the viewfinder _CV , the cameraman can easily perform camera work in accordance with the resolution and frame rate of the produced video.

(Second Embodiment)
With reference to FIG. 12, the video analysis apparatus 1B according to the second embodiment will be described while referring to differences from the first embodiment.
The video analysis device 1B is different from the first embodiment in that the moving amount of the optical flow F is corrected according to the viewing distance. As shown in FIG. 12, the video analysis apparatus 1B includes a video input unit 10, a down-conversion unit 20, an optical flow calculation unit 30, a movement amount association unit 40, a movement amount display unit 50B, and a movement amount correction. Means 60.
In addition, since it is the same as that of 1st Embodiment except movement amount display means 50B and movement amount correction means 60, description is abbreviate | omitted.

  Here, since the movement amount of the optical flow F is defined in units of pixels, the movement amount of the optical flow F differs from the movement amount of the eye on the retina depending on the viewing distance during video viewing. For example, the resolution of the display screen is 7680 × 4320 pixels, 3840 × 2160 pixels or 1920 × 1080 pixels, and the viewing distance is 0.75 times (0.75H), 1.5 (1.5H) times the screen height or There are various viewing conditions for the captured image, such as 3.0 (3.0H) times. When this viewing condition is known, the amount of movement of the optical flow F itself or the movement amount range of the optical flow F is changed according to the viewing condition, thereby associating with the amount of movement of the eye on the retina. be able to.

  For example, the moving distance range is set to be 5 or more and less than 10 and 10 or more and less than 30 with the viewing distance being 0.75H as a reference. In this case, the movement amount of the optical flow F represents the movement amount at a position (viewing distance 0.75H) separated from the screen center by the screen height × 0.75. When the viewing distance is doubled (1.5H), the movement amount of the optical flow F is apparently half. That is, twice the movement amount at a viewing distance of 1.5H corresponds to the movement amount at a viewing distance of 0.75H.

  Therefore, the movement amount correction unit 60 refers to the movement direction / movement amount information from the movement amount association unit 40 and changes the movement amount range of the optical flow F so as to be proportional to the viewing distance. Specifically, when the viewing distance is doubled, the movement amount correction unit 60 corrects the movement amount range to double (10 or more and less than 20, or 20 or more and less than 60). By this correction, the movement amount of the optical flow F is substantially halved. Furthermore, when the viewing distance is quadrupled, the movement amount correction unit 60 corrects the movement amount range to four times (20 or more and less than 40, 40 or more and less than 120).

Thereafter, the movement amount correction unit 60 obtains the movement direction / movement amount information representing the movement direction y of the optical flow from the movement amount association unit 40 and the movement amount y of the optical flow corrected according to the viewing distance. It outputs to the display means 50B.
Note that the movement amount correction means 60 may correct the movement of the optical flow F itself so that it is not proportional to the movement amount range of the optical flow F but is inversely proportional to the viewing distance.

The movement amount display means 50B combines the movement direction / movement amount information from the movement amount correction means 60 with the reduced video from the down-conversion means 20, and displays it on the viewfinder _CV as the movement direction and movement amount of the subject. Is.
Note that the composite video displayed by the movement amount display means 50B is the same as that in the first embodiment, and thus further description thereof is omitted.

[Action / Effect]
As described above, the video analysis apparatus 1B corrects the movement amount of the optical flow F according to the viewing condition (viewing distance), so that the accurate movement amount of the subject T can be synthesized and displayed.

(Third embodiment)
With reference to FIG.13 and FIG.14, the video analysis apparatus 1 which concerns on 3rd Embodiment is demonstrated about a different point from 1st Embodiment.
As shown in FIG. 13, the point that the video analysis device 1 is built in the video monitoring device 100 is different from the first embodiment.
As shown in FIG. 14, the video monitoring device 100 monitors the amount of movement of the subject for the video stored in the video storage device 200, and includes a video analysis device 1 and a monitor 110.

The video analysis device 1 receives the stored video (input video) from the video storage device 200, down-converts the input stored video image to generate a reduced video, and displays the generated reduced video on the monitor 110. At this time, the video analysis apparatus 1 analyzes the moving direction and moving amount of the subject, and synthesizes and displays the analysis result on the reduced video.
The video analysis apparatus 1 is connected to the video storage apparatus 200 via a wired or wireless network, and the stored video is input from the video storage apparatus 200 via this network. Since the video analysis apparatus 1 is the same as that of 1st Embodiment, the further description is abbreviate | omitted.

[Action / Effect]
As described above, even when the video analysis device 1 is built in the video monitoring device 100, the movement amount of the subject can be displayed in real time as in the first embodiment, so that appropriate video monitoring can be performed. .

(Modification 1)
As mentioned above, although each embodiment of this invention was explained in full detail, this invention is not limited to above-mentioned each embodiment, The design change etc. of the range which does not deviate from the summary of this invention are included.
In the above-described embodiment, the reduced video is generated from the entire captured video. However, the present invention is not limited to this. For example, a reduced video may be generated by cutting out a partial area of the shot video. That is, in the above-described embodiment, the entire captured video is set as a reduced area, whereas in Modification 1, a part of the captured video is set as a reduced area.

With reference to FIG.2 and FIG.15, the video analysis apparatus 1D which concerns on the modification 1 is demonstrated about a different point from 1st Embodiment.
The video analysis apparatus 1D according to the first modification includes a video input unit 10, a down-conversion unit 20B, an optical flow calculation unit 30, a movement amount association unit 40B, and a movement amount display unit 50.

As shown in FIG. 15, the down-conversion means 20D generates a reduced video S by cutting out a rectangular reduced area M from the center of the captured video L, as shown in FIG. In the first modification, the captured video L is 7680 × 4320 pixels, the reduced area M is 3840 × 2160 pixels, and the reduced video S is 1920 × 1080 pixels. The position and size of the reduced area M are arbitrary.
Since the movement amount association means 40B has reduced a partial area of the captured video L, it calculates the resolution reduction rate A based on the ratio of the number of pixels of the reduced video S and the reduced area M (pixels of the reduced video S). The number of pixels in the reduced area M). For example, the resolution reduction ratio A = (1920 × 1080) / (3840 × 2160) = 1/4.
The other points, the video analysis device 1D, are the same as those in the first embodiment, and thus the description thereof is omitted.

(Other)
In each of the above-described embodiments, it has been described that linear interpolation is performed using Expression (1), but the method of linear interpolation is not limited to this.
In each of the embodiments described above, when the detection area is set in the reduced video, the feature points are extracted from the detection area of the reduced video. However, the present invention is not limited to this. For example, even when a detection area is set for a reduced image, feature points may be extracted from the entire reduced image.

The above-described video analysis apparatus records the optical flow linearly complemented by the movement amount association unit in an HDD (Hard Disk Drive) or USB (Universal Serial Bus) memory, and performs video analysis processing (for example, Video analysis means (not shown) for performing subject movement determination and tracking may be provided.
In each of the above-described embodiments, the resolution and the frame rate are exemplified for each of the captured video and the reduced video, but the resolution and the frame rate are not particularly limited.
In each of the above-described embodiments, it has been described that the resolution and the frame rate of the captured video are manually set. However, the resolution and the frame rate of the reduced video may be automatically detected.

  In each of the above-described embodiments, it has been described that both the resolution and the frame rate are down-converted. However, only one of the resolution and the frame rate may be down-converted. If the frame rate is not down-converted, the frame rate reduction rate B is 1. When the resolution is not down-converted, the resolution reduction rate A is 1.

  In the above-described embodiment, the video analysis apparatus has been described as independent hardware, but the present invention is not limited to this. For example, the present invention can be realized by a program that causes hardware resources such as a CPU, a memory, and a hard disk included in a computer to operate cooperatively as the video analysis apparatus described above. These programs may be distributed via a communication line, or may be distributed by writing in a recording medium such as a CD-ROM or a flash memory.

1, 1B Video analysis apparatus 10 Video input means 20 Down-conversion means 30 Optical flow calculation means 40 Movement amount association means 50, 50B Movement amount display means 60 Movement amount correction means 100 Video monitoring device 110 Monitor C Camera C _B Camera body C _V viewfinder

Claims (7)

A video analysis device that analyzes the direction and amount of movement of a subject,
Down-converting means for generating a reduced video by down-converting at least one of the resolution and the frame rate of the input video;
An optical flow calculating means for calculating an optical flow of the reduced video;
Movement amount association means for linearly complementing the movement amount of the optical flow calculated by the optical flow calculation means in association with the resolution and frame rate of the input video;
The movement direction of the optical flow calculated by the optical flow calculation means and the movement amount of the optical flow linearly complemented by the movement amount association means are combined and displayed on the reduced video as the movement direction and movement amount of the subject. Movement amount display means;
A video analysis apparatus comprising: The down-converting means generates the reduced video by down-converting a reduced area preset in all or part of the input video,
The movement amount associating means calculates a resolution reduction rate A based on a ratio of the number of pixels between the reduced video and the reduced area of the input video, and based on a frame rate ratio between the reduced video and the input video. The frame rate reduction rate B is calculated using the following equation (1):

The video analysis apparatus according to claim 1, wherein the movement amount x of the optical flow is linearly complemented. The optical flow calculation means calculates an optical flow of a detection area preset in the reduced video,
The video analysis apparatus according to claim 1, wherein the movement amount display unit synthesizes and displays a movement direction and a movement amount of the optical flow included in the detection area. A movement amount correcting unit that corrects the movement amount of the optical flow linearly complemented by the movement amount association unit according to a preset viewing distance;
4. The movement amount display means synthesizes and displays the movement direction of the optical flow and the movement amount of the optical flow corrected by the movement amount correction means. The video analysis device described in 1. The optical flow calculation means extracts feature points from the reduced video, calculates the optical flow for the extracted feature points,
The movement amount display means calculates a ratio between the number of feature points whose movement amount exceeds a preset first threshold and the number of all feature points extracted from the reduced video, and the calculated ratio is calculated in advance. The video analysis apparatus according to any one of claims 1 to 4, wherein a warning is issued when a set second threshold value is exceeded.   6. The video analysis unit according to claim 1, further comprising: a video analysis unit that performs video analysis processing based on a movement amount of the optical flow linearly complemented by the movement amount association unit. Video analysis equipment.   A program for causing a computer to function as the video analysis apparatus according to any one of claims 1 to 6.

Images