JP2010021761A - Video image automatic recording control device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video image automatic recording control device capable of determining the contents of a video image of continuous image frames, and transmitting control signals for controlling recording of the video image. <P>SOLUTION: The video image automatic recording control device includes a feature vector generating means 4 which, for determining the continuity of image contents of continuous image frames, extracts image feature from two image frames of a predetermined inter-frame distance, for generating a feature vector, a similarity determining means 7 which, about each of feature vectors having been generated, sequentially compares similarities each other of the image contents of the continuous image frames until dissimilarity appears from two image frames, for determining a time point that represents dissimilarity, and a video image recording control means 8 which transmits control signals to start recording of video image or to change a recording density so that the video image to be recorded has high resolution at the time point when the dissimilarity is identified. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention determines the video content of successive image frames and automatically starts video recording when detecting a situation in which the continuity of the image content changes greatly, or automatically sets the resolution of the video to be recorded. The present invention relates to an automatic video recording control device.

  As a conventional device that senses a situation in which the continuity of image content changes greatly from video analysis, a device that uses a difference image (for example, see Patent Document 1) and a device that uses a hue (for example, see Patent Document 1). An apparatus for detecting a moving object (see, for example, Patent Document 1) is disclosed.

  These devices basically use a threshold value that is appropriately tuned each time in the environment in which they are used, or compare them with values learned in advance in a neural network according to the object to be used. It is configured to sense the situation where continuity changes greatly.

Japanese Patent No. 2525810 Japanese Patent No. 2878759 Japanese Patent No. 3295482

  However, in these methods, it is necessary to grasp the state in the environment where the device is installed in advance, and in order to adapt to a new environment, it is necessary to set appropriately or to learn each time. , Often troublesome.

  It is often difficult to know the state of the environment in advance in the location in broadcasting, especially in a location such as a natural documentary.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide an automatic video recording control apparatus that judges the content of video of successive image frames and sends a control signal for controlling video recording.

  The video automatic recording control device of the present invention analyzes video and senses or determines a situation in which the continuity of the image content changes greatly in a device that records video in a long-term location and a video recording device of a surveillance camera. In accordance with the determination result, control is performed so as to start recording and increase the resolution of the video (higher recording density).

  That is, the video automatic recording control apparatus of the present invention is an automatic video recording control apparatus that automatically controls video recording based on the video content of continuous image frames that are input. In order to determine the characteristics, a feature vector generating unit that extracts an image feature from two image frames having a predetermined interframe distance and generates a feature vector, and the generated feature vector Similarity determination means for sequentially comparing the similarities of the image contents of the successive image frames until the dissimilarity is expressed with respect to the image frames, and when the dissimilarity is determined Video recording control means for starting a video recording or sending a control signal for changing a recording density so as to increase the resolution of the video to be recorded. .

  In the video automatic recording control apparatus of the present invention, the image feature includes a hue represented by a position of an edge area for each image frame, a motion vector between image frames, and a pixel value in an effective motion vector. .

  In the video automatic recording control apparatus of the present invention, the feature vector is a vector component that represents the edge area, the motion vector, and the image feature for each hue by the number of pixels for each direction of the two image frames. It is characterized by comprising a combination of

  In the automatic video recording control apparatus of the present invention, the feature vector includes a vector component based on the image feature of the hue only when the density of effective motion vectors other than the edge area is equal to or higher than a predetermined threshold. It is characterized by.

  The automatic video recording control apparatus according to the present invention further comprises transmission means for remotely transmitting a control signal of the video recording control means.

  According to the video automatic recording control apparatus of the present invention, when recording a video in a long-term location or recording a video of a surveillance camera, unattended automatic start of recording or higher resolution of video (higher recording density) ), And long-term video recording can be performed efficiently.

  A video automatic recording control apparatus according to an embodiment of the present invention will be described below.

  FIG. 1 is a block diagram of an automatic video recording control apparatus according to an embodiment of the present invention. The automatic video recording control apparatus 1 according to the present embodiment includes an image feature extraction unit 2, an image feature determination unit 3, a feature vector generation unit 4, an effective motion pixel count unit 5, a similarity determination switching unit 6, Similarity discriminating means 7 and video recording control means 8 for recording the discriminated video in the recording device 9 are provided. The similarity determination unit 7 includes a similarity identification unit 71 and an identification learning unit 72. The video automatic recording control device 1 of this embodiment is a device that automatically controls video recording based on the content of video (for example, camera video) of continuous image frames that are input.

  The image feature extraction means 2 cuts the input camera video into frame images for each frame, and extracts image features for determining the continuity of the image contents of each successive image frame. As will be described later, the image features to be extracted include the “edge area” position for each image frame, the “motion vector (speed and direction of motion)” between image frames, and the “hue” represented by the pixel value of the effective motion vector It consists of.

  For this “edge area” extraction, a known canny edge detection algorithm is preferably used, but other edge detection algorithms such as a known Sobel edge detection algorithm may be used.

For example, as shown in FIG. 2, a certain N (N is a natural number of 1 or more) frame image F _N is divided into predetermined pixel blocks (for example, 5 × 5 pixels or 7 × 7 pixels), and each pixel It is determined whether or not the block is an edge pixel, and the determination result can be identified by a flag or the like. Next, an 8-neighbor expansion process is performed around the detected edge pixel to determine an edge area. At this time, the application frequency of the 8-neighbor expansion process may be repeated as appropriate twice or three times to improve the accuracy of the edge area to be determined.

Frame image F _N in FIG. 2, a "tree" I _S that represents the part of the background of continuity between image frames, fly as a discontinuity between image frames in an image frame including a "bird" Im Suppose there is. The frame image _FN is divided into predetermined pixel blocks, and an edge area is determined by performing, for example, 8-neighbor expansion processing. For example, the area B2 having no background edge is not set as the edge area.

  However, in such an edge area, a pixel block such as a tree branch or leaf that should belong to the background (for example, B1 in the figure) or a pixel block such as a fine feather of a bird that is a moving object (for example, illustrated) B3) can be gathered and extracted densely in the edge area. In this case, a large number of pixel blocks may be identified as edge areas in the vicinity of such edge areas (B1, B3), such as tree branches or leaves that should belong to a background that moves in the wind. Can also be discriminated as frame images having a large change in continuity in subsequent processing. Therefore, in order to improve the accuracy of determining whether or not the frame image has a large change in continuity in the subsequent processing, the edge density of the pixel block once identified as the edge area is calculated, and the edge area to be processed is calculated. It is preferable to narrow down.

For example, the number of pixel blocks that have been identified as peripheral edge areas is accumulated around the pixel block that has been identified as the edge area, and if this accumulated value is equal to or greater than a predetermined threshold T _density , the pixel block at the center is The flag value identified as the edge area as the background area is changed. In this way, the edge area to be processed is narrowed down, and the edge area that should belong to the background is excluded from the processing target as much as possible, so that the detection accuracy of a large change in continuity, which is a subsequent process, is improved. .

  It is preferable to use the known Lucas-Kanade motion detection for the “motion” extraction, but other motion extraction algorithms such as a known block matching method can be used.

For example, as shown in FIG. 3, of the image of the successive image frames are input, between the image frame F _{N + 1} subsequent to the image frame F _N to be processed _(may not necessarily be the next frame), a motion Calculate the vector. Since the motion vector is calculated regardless of the edge area described above, it is possible to use a pixel block having a different size from the pixel block used to extract the edge area. A motion vector is extracted with the same size and arrangement of pixel blocks. In the example of FIG. 3, the pixel block 10 in the image frame F _N is the size and orientation of the motion vector 13 in moving to the pixel block 11 in the image frame F _{N + 1} (moved in the shown 11 shown 10a in the same frame position) θ will represent the "speed of movement", and "orientation of the motion" in the image frame F _N respectively processed.

The extraction of "Hue" is the speed of motion of the motion vectors obtained as described above is set as valid motion vector when a predetermined value T _velOSity above, the function of the effective motion pixel number counting means 5 counting the number of pixels of blocks of a valid motion vector in an image frame F _N to be processed, the average of this when the count value is the predetermined value T _Vnumber above, within the effective motion vector pixel blocks each hue (or pixel block (Pixel value) is calculated and obtained. For this hue, three-dimensional data in a color space such as RGB can be used.

Image feature judgment unit 3, the image features extracted by the image feature extraction unit 2, i.e. the "edge area", "motion (speed and direction)", and an image frame F _N combinations discrimination processing target in "Hue" Do. Specifically, as shown in Table 1, the combination of image features, the area in the image frame F _N to be processed is classified into whether an edge area, then whether the edge area is the threshold value T _density than Whether the area is other than the edge area or not is classified as a threshold T _vnumber or more.

With this division, if it is less than the threshold value T _density for the edge areas, in a reliable edge, two image frames will be described later when the contrast "edge orientation" and "motion (speed and direction) "Is used as a guideline representing the movement of the image content.

Further, if the edge area is equal to or greater than the threshold value T _density , the reliability of the edge is low, and it is not always necessary to use a guideline representing the movement of the image content in consideration of the processing load.

Further, except for the edge area, when the count value of the number of pixels of blocks of a valid motion vector in an image frame F _N to be processed is a predetermined value T _Vnumber above, since there is likely to be image content in motion, there motion frame In addition to “movement (speed and azimuth)”, the image characteristic of “hue” is also used as a guide for representing the movement of the image content.

Further, except for the edge area, when the count value of the number of pixels of blocks of a valid motion vector in an image frame F _N to be processed is less than the predetermined value T _Vnumber, not be said that there is likely to be image content in motion, Considering the processing load, the image feature of “movement (speed and direction)” is also used as a guide for representing the movement of the image content.

  In this embodiment, the image feature discriminating means 3 is provided and the image features extracted by the image feature extracting means 2 are classified. However, when the conditions classified in advance as shown in Table 1 are satisfied, It is also possible to extract image features and generate feature vectors. That is, the functions of the image feature extraction unit 2 and the image feature discrimination unit 3 can be combined with the feature vector generation unit 4 to be described later so that only the necessary image features can be extracted simultaneously when the feature vector generation unit 4 generates the feature vectors. Furthermore, the processing load can be reduced.

  The feature vector generation means 4 is used for determining the continuity of image features for each of “edge area”, “motion vector (speed and direction of motion)”, and “hue” for each orientation of two image frames. For example, the image feature discriminating means 3 generates a feature vector obtained by connecting image features classified according to conditions. That is, the feature vector is expressed by a vector obtained by connecting the image feature of the edge area and the image feature other than the edge area. Specifically, the following two types of feature vectors satisfying the conditions in Table 1 are represented by the number of pixels obtained by dividing 0 ° to 180 ° by a histogram using image features related to the edge area and motion.

[Feature vector when the number of moving pixels is _{equal to} or greater than a predetermined value T _vnumber ]
(Feature vector component of edge area)
x _i ^(ee) : i × 180 / N ^(ee) ≦ θ _T ≦ (i + 1) × 180 / N ^(ee)
Meet the number of pixels in the target frame of the "edge area" with orientation ^{_{θ T x i (ev):}} i × 180 / N (ev) ≦ θ T ≦ a ^{(i + 1) × 180 /} N (ev)
Satisfies theta with the orientation of _T number of pixels in the target frame of the "size of the motion" ^{_{x i (es): i ×}} SpMax / N (es) ≦ θ T ≦ (i + 1) × SpMax / N a ^(es)
Satisfies θ number of pixels in the target frame with the orientation of _T "speed of movement" (feature vector component other than the edge area)
^{_{x i (nev): i ×}} 180 / N (nev) ≦ θ T ≦ (i + 1) × 180 / N a ^(nev)
The number of pixels in the target frame of the “edge area” having the azimuth direction θT satisfying: x _i ^(nes) : i × SpMax / N ^(nes) ≦ θ _T ≦ (i + 1) × SpMax / N ^(nes)
The number of pixels x _i ^(col) in the target frame of “the magnitude of motion” having the orientation of θT that satisfies: For example, using RGB pixel values (other hue expression methods may be used)
col = 0 (R), col = 1 (G), col = 2 (B)
When
The feature vector x is expressed by the following equation using the above six vector components for each N (*).

  Here, each N (*) represents the number of divisions when creating a histogram (* is a symbol representing each component), and SpMax is an upper limit value of an assumed motion speed (can be arbitrarily set). I represents an arbitrary position divided when creating a histogram.

[Feature vector when the number of moving pixels is _{equal to} or greater than a predetermined value T _vnumber ]
In this case, since the hue information is not used, the feature vector x is expressed as follows.

Effective motion pixel number counting means 5 counts the number of pixels of blocks of a valid motion vector in an image frame F _N to be processed calculated by the image feature extraction unit 2 has a function of sending back the count value to the image feature extraction unit 2 .

  The similarity determination switching means 6 is a switch function that allows selection as desired whether or not to execute the function of the similarity identification means 71 via the identification learning means 72.

  The similarity identification unit 71 compares similarities for each feature vector obtained by extracting two image frames having a predetermined interframe distance (set within several frames in order to compare changes in motion), and determines dissimilarity. Determine. For example, when determining the similarity between the image contents of two image frames by paying attention to the shape of the histogram, the inner product of feature vectors for two images at a specific time interval is calculated, and the inner product value is equal to or greater than a threshold value. This is done by determining whether or not there is. Alternatively, a feature vector in two image frames at a specific time interval is obtained by using a determination range obtained by machine learning with respect to an identification function predetermined by the identification learning unit 72 selected according to the similarity determination switching unit 6. The similarity can also be determined by comparison.

  For example, the similarity identifying unit 71 sequentially compares the similarities of the image contents of successive image frames in the input camera video until each generated feature vector represents dissimilarity with respect to two image frames. And determine the time point representing dissimilarity. Specifically, the similarity identifying unit 71 determines 1 if the two image frames are similar (when the compared feature vector is within a predetermined range), and determines that they are not similar. When the comparison is performed (when the compared feature vector is outside the predetermined range), −1 is output every time the image frame comparison is performed. Here, when it is −1, dissimilarity is indicated, and it can be determined that the two image frames have changed greatly.

  The predetermined range used for the comparison of the feature vectors defines the range of similarity determination for each component, and if it is within the range of all similarity determinations, it is determined that there is similarity, and the range of similarity determination If there are a predetermined number (for example, 2) or more of vector components that are not included, it is determined that there is no similarity. The range of similarity determination and the predetermined number of settings used for the determination are preferably set for each application.

  The similarity identifying unit 71 can also apply an averaging filter such as a median filter to the similarity determination result for a certain period in order to stabilize the system. Further, the similarity identifying means 71 outputs −1 unconditionally when comparing feature vectors of different dimensions (when comparing feature vectors x of different formulas described above).

  The identification learning unit 72 applies a determination range obtained by machine learning for a predetermined identification function to the similarity identification unit 71. Specifically, a support vector machine that exhibits high performance for binary identification can be used as the identification learning means 72. In addition, any kind of discriminant function generation algorithm that can perform prediction determination for a new input value can be used. Further, a function of looping for machine learning between the similar identification unit 71 and the identification learning unit 72 may be further provided.

  The video recording control means 8 starts video recording at the time of determining the dissimilarity, or sends a control signal for changing the recording density so that the video to be recorded has a higher resolution. Specifically, when the video recording control unit 8 obtains a discrimination result indicating the dissimilarity of the similarity identifying unit 71, the video recording control unit 8 records the recording for a certain time, for example, when recording the camera video input to the recording device 9. The start is turned on, or the recording density is controlled so as to increase the resolution.

  According to the video automatic recording control device of the present embodiment, whether the video content of successive image frames is judged and video recording is automatically started when a situation in which the continuity of the image content changes greatly is detected. Or the resolution of the recorded video can be automatically increased.

  For example, in a long-term location for broadcasting, a discrimination result indicating dissimilarity can be obtained when recording and shooting animals that do not know when they appear in the field or events that are predicted to occur but do not know when they occur Then, the video recording control means 8 can control the recording device 9 so as to record the video for a certain period of time (for example, a period of 1 minute, for example).

  If it is desired to add a margin for a short period before the target video, a skip back recorder can be connected in front of the recording device and the output of the skip back recorder can be recorded in the recording device 9. When used for crime prevention, the video recording control means 8 sets the number of frames for a certain period, for example, when the number of frames of the normal video is set low and a discrimination result indicating dissimilarity is obtained. The compression rate of the recording device 9 is controlled so as to increase. Also in this case, it can be used in combination with a skip back recorder.

  Although the above embodiments have been described as representative examples, it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the invention. For example, although it has been described that the recording apparatus 9 is controlled, a transmission means for remotely transmitting the control signal of the video recording control means 30 can be further provided, and remote control can be performed through a communication line such as the Internet instead of the recording apparatus 9. . Accordingly, the invention should not be construed as limited by the embodiments described above, but only by the claims.

  According to the video automatic recording control apparatus of the present invention, it is possible to sense and record or transmit a video expected by an unmanned person, which is useful for an application in which long-term camera shooting is desired.

1 is a block diagram of an automatic video recording control apparatus according to an embodiment of the present invention. FIG. It is a figure showing the image frame in the image feature extraction of the video automatic recording control apparatus of one Example by this invention. It is a figure showing the image frame used for the motion detection in the image feature extraction of the image | video automatic recording control apparatus of one Example by this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image automatic recording control apparatus 2 Image feature extraction means 3 Image feature discrimination means 4 Feature vector generation means 5 Effective motion pixel number counting means 6 Similarity discrimination switching means 7 Similarity discrimination means 8 Video recording control means 9 Recording apparatus 71 Similarity identification Means 72 Discrimination learning means

Claims (5)

An automatic video recording control device for automatically controlling video recording based on video content of continuous image frames input,
In order to determine the continuity of the image content of successive image frames, feature vector generation means for extracting image features from two image frames at a predetermined interframe distance and generating a feature vector;
For each of the generated feature vectors, the similarity is determined by sequentially comparing the similarities of the image contents of the successive image frames until the dissimilarities are expressed with respect to the two image frames. Discrimination means;
Video recording control means for sending a control signal for starting recording of video or changing recording density so as to increase the resolution of video to be recorded at the time of determining dissimilarity;
An automatic video recording control apparatus comprising:   2. The video automatic recording control apparatus according to claim 1, wherein the image feature includes a hue represented by a position of an edge area for each image frame, a motion vector between the image frames, and a pixel value in an effective motion vector. .   The feature vector comprises a combination of vector components representing the edge area, the motion vector, and the image feature for each hue by the number of pixels for each orientation of the two image frames. The video automatic recording control apparatus according to 2.   4. The video according to claim 3, wherein the feature vector includes a vector component based on the image feature of the hue only when the density of effective motion vectors other than the edge area is equal to or higher than a predetermined threshold. Automatic recording control device.   The video automatic recording control apparatus according to any one of claims 1 to 4, further comprising a transmission unit that remotely transmits a control signal of the video recording control unit.

Images