JP2019215889A - Computer-implemented method, imaging system, and image processing system - Google Patents

Abstract

To provide a low-cost and computationally efficient video-based vehicle classification method.SOLUTION: A vehicle classification method comprises the steps of: capturing a vehicle detection objective region with a traffic monitoring camera 305 as an imaging capturing device oriented to include a visual field that spans the region, and generating a cluster of motion vectors representative of a vehicle detected within a target region; associating one or more attributes with the cluster of motion vectors; and classifying the detected vehicle on the basis of the one or more attributes associated with the cluster of motion vectors.SELECTED DRAWING: Figure 3

Description

  Automated vehicle detection, counting and classification are important tools widely used by traffic planners. These will help local governments determine the enforcement of automobile traffic laws as well as critical hourly traffic flow, maintenance schedules and optimal road traffic law enforcement periods. These tools also support accident detection, verification, and response.

  Conventional vehicle classification systems are based on laser scanners and underground sensors typically used at toll booths for automatic toll collection (ETC) (http://www.osilaserscan.com/Products/Vehicle-Detection-and-). -Classification.aspx). These systems gather detailed information about the vehicle, such as the number of axles, height, width, weight, length, profile and volume, and determine the amount to be charged to the vehicle based on the information obtained. I do. While these systems have high accuracy and precision, they are expensive to install and / or maintain and are not easily scalable to other applications such as monitoring urban roadways and bridges. . Imagine that states and municipalities have numerous ordinances that restrict trucks and buses of a certain size on certain roads. In the case of automatic enforcement by video imaging, only rough vehicle classification is required to distinguish trucks and buses from other types of vehicles.

  The present invention provides a relatively low cost, computationally efficient, video-based solution to vehicle classification that can meet traffic law enforcement requirements in some jurisdictions.

  According to one embodiment of the present invention, a computer-implemented method for classifying a vehicle captured by an imaging device oriented to include a field of view spanning a vehicle detection target region, comprising: a) detecting a vehicle within a target region; Generating a cluster of motion vectors representative of the vehicle that has occurred, b) associating one or more attributes with the cluster of motion vectors, and c) detecting based on one or more attributes associated with the cluster of motion vectors. Classifying the identified vehicles.

  According to another embodiment of the present invention, there is provided an imaging system for classifying vehicles captured by an imaging system, comprising: an imaging device oriented to include a field of view spanning a vehicle detection target area; and an imaging device. An image processor operably associated with the image processor, the image processor performing a method of classifying the vehicle captured by the imaging device, the method comprising: a) motion representing the vehicle detected in the target area. Generating clusters of vectors; b) associating one or more attributes with clusters of motion vectors; c) classifying detected vehicles based on one or more attributes associated with the clusters of motion vectors. And an imaging system.

  According to yet another embodiment of the present invention, a computer-implemented method for classifying a vehicle captured by an imaging device associated with a field of view including a vehicle detection target region, comprising: a) detecting a vehicle detected within the target region. Extracting a cluster of motion vectors; b) associating one or more attributes with the cluster of motion vectors; c) determining a vehicle detected based on the one or more attributes associated with the cluster of motion vectors. Classifying. A computer-implemented method comprising the steps of:

  According to yet another embodiment of the present invention, there is provided an image processing system for classifying a vehicle captured by an imaging device, comprising: a) a motion vector of the vehicle detected in a target area associated with the imaging device. B) associating one or more attributes with a cluster of motion vectors; c) classifying detected vehicles based on one or more attributes associated with the cluster of motion vectors. And an image processing system that includes an image processor that performs the method.

FIG. 1 is a diagram showing a video camera network system. FIG. 2 is a diagram schematically illustrating reference (I) and non-reference (P and B) frames in a video compression technique according to an exemplary embodiment of the present invention. FIG. 3 is a schematic diagram illustrating an imaging system including an off-line vehicle classification process using a motion vector according to an exemplary embodiment of the present invention. FIG. 4 is a schematic diagram illustrating an imaging system including an inline vehicle classification process using a motion vector according to an exemplary embodiment of the present invention. FIG. 5 is a diagram illustrating a block matching algorithm according to an exemplary embodiment of the present invention, and FIG. 5 is a diagram illustrating a reference frame including a reference block and a search window. FIG. 6 illustrates a block matching algorithm according to an exemplary embodiment of the present invention, and FIG. 6 illustrates a target frame including a target block, ie, a motion block. FIG. 7 is a diagram illustrating a result of a block-based motion estimation performed according to an exemplary embodiment of the present invention, and FIG. 7 is a diagram illustrating a reference frame. FIG. 8 illustrates the results of a block-based motion estimation performed according to an exemplary embodiment of the present invention, and FIG. 8 illustrates a target frame. FIG. 9 illustrates the results of block-based motion estimation performed according to an exemplary embodiment of the present invention, and FIG. 9 illustrates the resulting motion vector field. FIG. 11 illustrates the results of block-based motion estimation performed according to an exemplary embodiment of the present invention, and FIG. 10 illustrates an estimated frame. FIG. 11 illustrates another result of a block-based motion estimation algorithm performed by an exemplary embodiment of the present invention, and FIG. 11 illustrates a reference frame. FIG. 12 illustrates another result of a block-based motion estimation algorithm performed according to an exemplary embodiment of the present invention, and FIG. 12 illustrates a target frame. FIG. 13 illustrates another result of a block-based motion estimation algorithm performed according to an exemplary embodiment of the present invention, and FIG. 13 illustrates a resulting motion vector field. FIG. 14 is a diagram illustrating another result of a block-based motion estimation algorithm performed according to an exemplary embodiment of the present invention, and FIG. 14 is a binary image showing the resulting active motion blocks. FIG. 15 is a diagram showing a video frame including each of the captured trucks and passenger cars. FIG. 16 is a diagram showing a video frame including each of the captured trucks and passenger cars. FIG. 17 is a diagram illustrating the active motion blocks associated with each of FIGS. 15 and 16. FIG. 18 is a diagram illustrating the active motion blocks associated with each of FIGS. 15 and 16. FIG. 19 is a diagram illustrating a vehicle classification method by estimating the length of a detected vehicle in a region of interest using a cluster of active motion blocks, according to an exemplary embodiment. FIG. 20 is an example illustrating a track and / or bus that has been correctly classified using a compressed motion vector according to an exemplary embodiment of the present invention. FIG. 21 is an example showing tracks and / or buses correctly classified using compressed motion vectors according to an exemplary embodiment of the present invention. FIG. 22 is an example illustrating a track and / or bus that has been correctly classified using a compressed motion vector according to an exemplary embodiment of the present invention. FIG. 23 is an example showing tracks and / or buses that have been correctly categorized using compressed motion vectors according to an exemplary embodiment of the present invention.

  For the purpose of disclosing the present invention, the term "vehicle" is defined as being used to convey persons or objects such as cars, trucks, motorcycles, and the like.

  The present invention provides a method and system for automated video-based vehicle classification that can operate within a compressed video stream and includes the following steps (a) during an initialization step, generally during installation or setup of the system. Determining the location of a virtual target area within the camera's field of view spanning the region of interest to be performed, the target area defining a location in the captured image where vehicle detection and classification is performed; (B) capturing the video using a camera or reading into a compressed video previously captured by the camera; and (c) incoming live uncompression when the vector is of the type used for video compression. If motion vectors are determined from the video stream, or if processing already compressed video Extracting a motion vector from the data stream; and (d) detecting the presence of a vehicle traveling in the virtual target area by analyzing the temporal persistence of the cluster of motion vectors from step (c); (E) categorizing the detected vehicle into one of the truck / bus or other passenger vehicle categories by analyzing the cluster of motion vectors associated with the detected vehicle, and the relevant information about the vehicle class is metadata. And (f) the (arbitrary) frames (hereinafter referred to as “frames”) in which the vehicle is included in the field of view of the camera are referred to in order to facilitate the subsequent search. -) Can be encoded as a frame. Of note, incorporating the vehicle classification directly into the compressor allows the classification process to be embedded directly inside the camera.

  It should be noted that the presently taught methods and systems of the present invention are centered on determining the offenders of motor vehicle laws and / or regulations associated with vehicle classification. Actual law enforcement requires some further thought. For example, the automatic issuance of tickets requires vehicle identification, which is commonly used in many transportation imaging applications related to law enforcement for red light, speed, parking, etc. ). Another use of the detection method and system according to the invention involves alerting local police to stop identified vehicles. Another use is to detect certain violations and monitor traffic conditions to ensure that appropriate signage or law enforcement functions are performed. Another use is the acquisition of traffic statistics used by highway departments for applications such as routing, traffic signal optimization, understanding of road rehabilitation required as a function of traffic flow. An advantage of the method of the present invention is that it can be easily integrated as a further feature of existing traffic cameras.

  The following description is directed primarily to a method and / or system for classifying vehicles captured by an image sensor. As mentioned above, the classification of vehicles detected as trucks, buses, passenger cars, etc., allows further processing to provide a collection of traffic flow statistics as well as enforcement of laws and regulations relating to vehicle types.

  The exemplary embodiments described below use motion vectors and / or blocks associated with the video compression process to classify vehicles, but other motion vector types unrelated to video compression may also be used. It will be appreciated that they can be used and do not depart from the scope of the invention. Further, while the exemplary embodiment is directed to a video-based system for capturing a vehicle in a target area, such as a particular road, a variety of multiple image frame capture systems may be used, These do not depart from the scope of the present invention. Some examples of video or multiple image frame capture systems include RGB cameras, infrared camera systems, thermal camera systems, and satellite imaging camera systems.

  Video compression is essential in applications where high quality video transmission and / or archiving is required. Consider the surveillance system shown in FIG. 1 and consisting of a set of dome cameras 105, 110, 115, 120 and 125 that relay video data to a central processing and archiving facility. Although the communication network 140 used to carry the video stream between the camera and the central facility is at the pinnacle of patented technology, traffic management centers have recently begun the transition to Internet Protocol or IP compliant networks.

  In any case, the underlying communication network generally has bandwidth constraints that dictate the use of video compression techniques on the camera side before transmission. For conventional analog cameras, compression is performed as an external encoder attached to the camera, while digital cameras typically integrate the encoder within the camera itself. Typical transmission rates on IP (IP) networks must limit the frame rate of multi-megapixel video streams to less than 5 frames per second (fps). The latest video compression standards allow the use of camera capabilities at maximum frame rate to transmit high resolution video over the same network bandwidth (Claudio R. Lima et al., Sprint-Nextel, 1 Adrian Court, Burlingame, CA 94010, USA), “High Definition Video Broadcast Over Core IP Networks”, page 11). For example, when transmitting 1080-pixel HD (high-resolution) uncompressed video, a bandwidth of 1.5 gigabytes (Gbps) per second is required, but 250 megabits per second (Mbps) is required for the compressed video. Only. Therefore, even though the same network infrastructure is used, it is possible to transmit a compressed video whose frame speed is six times the frame speed of the version of the non-compressed video.

  Video compression involves two types of redundancy in a video stream, spatial redundancy between adjacent pixels in a frame, and temporal redundancy between adjacent frames in a frame. This is achieved by utilizing redundancy. This approach separates into two different types of prediction, namely intra-frame prediction and inter-frame prediction, as shown in FIG. 2 so that two different types of coded frames, namely the reference frame and Obtain a non-reference frame. The reference frame, i.e., I-frame 205, is encoded in a stand-alone manner (intra-frame) using a compression method similar to that used to compress digital images. The compression of non-reference frames, i.e., P-frame 215 and B-frame 210, generally involves a three-step process in which the target frame is estimated or predicted from pre-encoded frames. Involves the use of interframe or motion compensated prediction methods.

(I) Motion estimation in which a motion vector is estimated using a frame that has been encoded in advance. The target frame is segmented into pixel blocks called target blocks, and the estimated or predicted frame is assembled by suturing the block from a pre-coded frame that best matches the target block. The motion vector describes the relative displacement between the location of the original block in the reference frame and the location in the predicted frame. Although motion compensation for P-frames depends only on the immediately preceding frame, the immediately preceding and subsequent frames are typically used to predict the B-frame.

(Ii) a residual calculation in which an error between the predicted frame and the target frame is calculated, and

(Iii) Compression in which the error residual and the extracted motion vector are compressed and stored. Iain E. See Richardson, "The H.264 Advanced Video Compression Standard."

  For images captured by a fixed camera (having the configuration of most traffic cameras currently deployed), the main factor of the change between frames arranged next to each other is the movement of the object corresponding to the movement of the object. I have. In this setting, the output from the motion compensation stage is a flow of optical blocks describing the movement of the pixel blocks between adjacent frames. Thus, the encoded set of motion vectors is a good descriptor of the apparent motion of the object in the field of view associated with the fixed camera.

  As mentioned above, although the vehicle classification method disclosed in the present invention has been described as using a compression type motion vector, the method generally applies to other types of motion vectors. The detailed description of the compression type vectors is primarily for the benefit of the significant bandwidth associated with video compression.

  The present invention provides a method and system for automated video-based vehicle classification that can operate within a compressed video stream. The exemplary embodiment uses motion vectors associated with video compression, which are calculated as one of the compression steps prior to archiving or transmission, or are easily obtained from a compressed data stream. It is possible. Building the vehicle classification directly on the compression pipeline adds a small amount of conductive computation to real-time performance. This embedded embodiment of the classification process eliminates the need for further processing on the server, such as decompression (decompression), vehicle classification, and recompression. In another embodiment, the classification process is performed in a central processing unit having access to the compressed video data. According to this embodiment, vehicle classification is enabled by decompression of motion vectors only, rather than decompression of full video.

  One advantage of the classification process according to the present invention is bandwidth reduction, which is achieved when vehicle classification is performed at or near a video camera rather than a central server. This configuration reduces bandwidth by simply sending images of interest to a central location for use for archival and evidential purposes, for example, sending images of trucks, buses or other passenger cars can do. Alternatively, a video frame containing the subject vehicle may be encoded as a reference frame when a complete video feed is transmitted and stored, to facilitate subsequent video searches. Alternatively, transmission of a classification result with or without a corresponding video alone is sufficient for traffic analysis applications.

  A high-level overview of two exemplary embodiments of the present invention is shown in FIGS. First, the traffic monitoring camera 305 captures an image of a target area, generally, an entrance or exit of a highway or a vehicle.

  Referring to FIG. 4, vehicle classification is performed with some modification to the video compression procedure, as described in more detail below. According to one exemplary embodiment, when a vehicle passes through a road and is classified, information about the vehicle class is embedded in the compressed video stream in the form of metadata. If desired, frames where the vehicle is within the field of view of the camera may be encoded as reference frames to facilitate subsequent searches.

  The procedure involved in the implementation of the algorithm is as follows.

a) During the initialization step, determine the location of a virtual target area in the field of view of the camera spanning the area of interest, the target area defining a location in the captured image where vehicle detection and classification is performed. I do. This step is typically performed during system installation and setup.

b) Capture video using camera 305. Alternatively, a compressed video taken by a camera is obtained.

c) Determine the motion vector from the incoming live uncompressed video stream 405 when the vector is of the type used for video compression. Alternatively, when a compressed video has been read in advance, a motion vector is extracted from the compressed data stream (see FIG. 3).

d) Detect the presence of a vehicle traveling in the virtual target area by using the motion vector from step c).

e) If a vehicle traveling in the virtual target area is detected, classify the detected vehicle into one of the truck / bus or other passenger vehicle categories. The vehicle class is embedded in the compressed video stream in the form of metadata.

f) The (arbitrary) frame within the camera's field of view where the vehicle is optimal, relative to the specific application, is encoded as a reference frame, facilitating further searching and quick searching for evidence images. Can be

  Classification of detected vehicles may be performed in one or more of several possible ways, including:

(1) Based on geometric attributes (eg, area, length, height, width, bias, combinations thereof, etc.) of clusters of detected motion blocks associated with vehicles detected on the image plane. Do it. For example, the area, length, height, width, bias, and combinations thereof are associated with the detected motion block cluster.

(2) Estimate the physical length and / or width of the detected vehicle by using camera coordinate techniques that map pixel coordinates to real world length units.

  Since the procedure depends on the processing of the information sent by the motion vectors, or this method does not require the video to be decompressed, as shown in FIG. Being executable, the classification process is computationally more efficient than naive methods that perform complex operations such as background estimation, motion detection, tracking, and feature extraction on fully decompressed video streams It can be something. Further, the vehicle classification method of the present invention may also be useful in processing stored videos that have been pre-compressed for use in historical data analysis.

  The following detailed description provides further details of the method steps outlined above for classifying a vehicle according to an exemplary embodiment of the present invention.

  Implementation of the vehicle classification method of the present invention requires only a small amount of further processing for the video compression algorithm at the time of execution, and is used as an analog-to-digital converter in the case of an analog camera, and can be implemented in digital or IP. In the case of a camera, it is used as the camera itself

a) During the initialization step, determine the location of the virtual target area within the camera's field of view spanning the region of interest.

  Typical virtual target areas are not limited to single or multiple virtual polygons, but typically include one virtual polygon per monitored traffic lane. The location of the virtual target area is manually entered as it depends on the geometry of the particular camera settings. Virtual polygons are used for both occlusion and vehicle detection.

b) Capture video using a camera. Alternatively, a compressed image obtained in advance by the camera is read.

  Conventional traffic cameras or other video cameras can be used to capture live video. Implementations of the present invention require few changes on IP cameras that perform embedded video compression. Alternatively, the compressed video is available from a video database.

c) Determine the motion vector from the incoming live uncompressed video stream when the vector is of the type used for video compression. Alternatively, a motion vector is extracted from a compressed data stream obtained in advance.

  Standard implementations of video compression algorithms typically use a fixed rate to include about one I-frame reference or I-frame every 30-50 frames. Since I-frames do not have an associated motion vector, they are not used for monitoring and detection. Since I-frames represent only a few frames, omitting processing of I-frames does not significantly affect vehicle detection.

  Motion vectors are extracted as part of the motion estimation stage in the compression process. The motion vector that grasps the complete video frame is calculated at the time of compression. From the description of the subsequent steps, the processing of the motion vector arranged in the virtual target area's confinement makes the stop event robust monitoring. It will be clear that is enough for. The following briefly describes how motion vectors are extracted (see "The H.264 Advanced Video Compression Standard" by Iain E. Richardson).

  The motion vector between two adjacent frames in a video sequence is determined by the pixel-level light flow method B. K. P. Horn) and B.H. G. Schunkk, "Artificial Intelligence, 17, 185-203 (issued in 1981)", which requires a motion vector to be calculated for each pixel in each non-reference frame. Costly. However, compression algorithms such as H264 and MPEG4 generally use a block-based approach. Iain E. See Richard H., 264, "The H.264 Advanced Video Compression Standard." The motion vector in the block-based method describes a motion of matching blocks through frames arranged adjacent to each other, and when compared with a pixel-level method, these operations are computation resources. The demand for is not very high. 5 and 6 show a graphical description of the block matching algorithm.

The block matching algorithm divides the frame to be compressed, i.e., the target frame, into pixel blocks of a predetermined size; Indicate the size of the block. The search is performed in the reference frame for the block that is most similar to the current mxn target pixel block. Since the search and similarity metrics are computationally expensive processes, the search window is typically defined around the location of the target motion block, as shown in FIG. For example, the criteria of the similarity between blocks are a mean square error (MSE) and a mean absolute difference (MAD), and are calculated as follows.

In the formula, B (k, l, j) represents a pixel arranged in the k-th column and the l-th row of m × n blocks of pixels in the j-th frame, and (d ₁ , d ₂ ) is a vector that describes the displacement between the target block and the candidate block. In this case, the j-1st frame is the already encoded frame used as the reference frame, and the jth frame is the target frame. MSE and MAD together measure how dissimilar the two blocks are, and then the block similarity measure can be defined as the reciprocal or negative MSE or MAD. The motion vector for the target pixel block is the vector (d ₁ , d ₂ ) that maximizes the block similarity measure between the target block and the reference block. The search for the best matching block in the search window can be performed using a complete extended search, a binary search, a three-stage search, a spiral search algorithm, or the like. In this regard, Y. W. Huang et. al., VLSI (Ultra Large Scale Integrated Circuit) Signal Processing System, entitled "Survey on Block Matching Motion Estimation Algorithms and Architectures with New Results", Vol. 42 (2006). FIG. 9 shows a motion field of view obtained from an application of an 8 × 8 pixel block-based motion estimation algorithm by 16 × 16 pixel search for the reference frame shown in FIG. 7 and the target frame shown in FIG. FIG. 10 shows a predicted image obtained by suturing the best matching reference block. In this scene, the camera is stationary and the car is traveling from right to left. As a result, all apparent movements are in the area where the car is located in the image plane.

d) Detect the presence of a vehicle traveling in the virtual target area by using the motion vector from c).

  To avoid false positives due to spurious sources of motion such as camera shake, moving leaves, clouds, water waves, etc., only motion vectors having a magnitude greater than a predetermined threshold T are assumed. Motion blocks associated with such motion vectors are referred to as active motion blocks and indicate the presence of an object moving through the area captured by these blocks. Figures 11-14 illustrate how motion vector knowledge for a given target frame can be used with respect to the location of the target virtual area to trigger a vehicle detection event. FIGS. 11-14 show two consecutive video frames, their corresponding motion vectors and active motion blocks. A motion vector is calculated for each 4 × 4 pixel block. In both figures, a target virtual area as a sample and a virtual polygon drawn as a rectangular box including a monitored road lane are superimposed.

As the vehicle moves across the virtual polygon, a number of active motion vectors are placed inside the polygon. Two thresholds are set to avoid false positives due to active motion vectors generated by apparent motion of objects different from the vehicle. These thresholds, the threshold N ₁ of Teigizukeru the minimum connection cluster virtual polygon inside the active motion vectors before the vehicle detection is triggered, connects the cluster that has an active motion vectors of at least N ₁ is, trigger vehicle detection the threshold N ₂ for Teigizukeru the minimum number of consecutive frames in the virtual polygon before being consists.

The value of N ₁ is generally, the geometry of the camera settings, the size of the virtual polygon, as well as the resolution of the video sequence, depends on the size of the blocks used in the motion estimation algorithm. The value of N _2, the value of N _1, the geometry of the camera settings, depends on the average velocity of the size of the virtual polygon, is the frame rate, and monitoring road. Vehicle detection event is triggered at least N ₁ one concatenated motion vector of the cluster virtual polygon inside arranged the N on the _second consecutive frames.

  When a vehicle is detected, morphological actions such as closing, opening, fitting, etc. can be performed on the binary image describing the location of the active motion block.

  e) determining whether a running vehicle is detected in the virtual target area and classifying the detected vehicle into one of trucks and / or any other passenger car category that may include a bus or a single vehicle. I do. The vehicle class can be embedded in the compressed video stream in the form of metadata.

  When a vehicle is detected in the area of interest, the detected vehicle is classified into one of the bus and / or truck or other passenger car categories. As mentioned above, vehicle classification can be implemented in a number of ways. Some examples of these algorithms are given below.

(1) based on the geometric attributes (eg, area, length, height, width, bias, combinations thereof, etc.) of the cluster of detected motion blocks associated with the detected vehicle on the image plane; These geometric attributes can be determined directly from the shape of the cluster itself or from the shape of a primitive geometric feature (eg, an ellipse) that fits the contour of the cluster. And / or

(2) Estimating the physical length of the detected vehicle can be achieved using camera calibration techniques that map pixel coordinates to real world length units.

A relatively simple method for vehicle classification is based on the measured area of a cluster of active motion blocks. For a given camera setting geometry, the bus / truck area is typically much larger than the area of other passenger cars on the image plane, as shown in FIGS. In these figures, the blob 1705 detected for the track is much larger than the blob 1805 for the SUV. Based on the binary image showing the active motion block, the area S of the detected blob is:

Can be calculated as

Where I is the frame size binary mask output from the previous module and x and y are pixel coordinates. Thereafter, the calculated blob area was is compared to a predetermined thresholds T ₁ in order to distinguish the bus / truck from other cars. Predetermined thresholds T ₁ may be set particular camera configuration, geometry, and depending on parameters such as camera resolution, the algorithm during installation and / or setup of the front of the camera to initialize the.

Another method for performing vehicle classification is by estimating the physical length d of the detected vehicle. This can be achieved by first determining the start and end points of the cluster of active motion blocks. Draw a line through each of the start and end points perpendicular to the direction of the street on the image plane. Thereafter, the line connecting the start point and the end point is projected on a line perpendicular to the line passing through the start point and the end point. Thereafter, the length of the vehicle is estimated as the length of the projected line. This technique is shown in FIG. The physical length of the projected line can be estimated by a calibration process that maps pixel coordinates to real coordinates (from IEEE Trans. On Pattern Analysis and Machine Intelligence, by Z. Zhang, "Aflexible new technology." camera calibration "(published in 2000, Volume 22, Chapter 11, pages 1330 to 1334). Thereafter, the length d that is estimated is compared with a threshold value _{T 2} which is Teigizuke advance. estimated length threshold If it is larger, the vehicle is classified as a bus and / or truck.Notably, the above technique used to estimate the length of the detected vehicle is such that the image is directly behind or ahead of the vehicle. Or a camera oriented diagonally It should be understood that the estimated vehicle width of the vehicle can then be used to estimate the width of the captured vehicle from various classes of vehicles such as, for example, a single versus a passenger car. It can be used to provide vehicle classification based on different widths.

  If desired, vehicle class information can be embedded directly in the captured video. This results in additional computer processing savings because the compression standard allows parsing the compressed video stream without actually decoding the video and / or audio content. The ability to include various types of metadata is a key element in MPEG4 and H264.

f) For a particular application, the (arbitrary) frame in the field of view of the camera where the vehicle is optimal is encoded as a reference frame, facilitating a future or quick search for evidence images. be able to.

  The video sequence or the still frames in the sequence are viewed by a person for vehicle class verification and this image can be used as legal evidence. Fast search is possible by selecting a frame as an I-frame. For example, a quick decompression (decompression) to view an image of a driving vehicle passing through a virtual target area may require an I-frame when the vehicle is in the center of the target area or where its license plate is readable. It is enabled by encoding.

  The method provided herein was tested on a video sequence captured on a local road in Webster Village, New York. The video was taken with a commercially available Vivotek IP8352 surveillance camera. The footage monitored the street for about 15 minutes and had 25K frames. The captured video had a frame rate of 30 frames per second (fps) and a resolution of 1024 × 1280 pixels. To enable faster processing, the frames were sub-sampled four times vertically and horizontally. As a result, the video sequence after the spatial thinning had a frame rate of 30 fps and a resolution of 320 × 256 pixels.

  The target area was defined in the captured video, and vehicles passing through the virtual polygon corresponding to the target area were monitored. 11 to 14 show the field of view of the camera and virtual polygons on the image plane.

To evaluate the performance of the classification method provided herein, the compressed motion vectors were calculated as in a typical implementation of MPEG4 and H264. The motion estimation block size was set to 8 × 8 pixels and the search window size was set to 16 × 16. The block size selection determines a minimum inter-vehicle distance that the algorithm can solve among other performance parameters. That is, a block size of m × n pixels renders an algorithm that cannot distinguish the distance between vehicles less than m + 1 pixels as the vehicle moves through the virtual target area. The algorithm parameters were set as follows: N ₁ = 10, N ₂ = 1, and T ₁ = 100. As described in step e), vehicle classification was performed based on the detected cluster area of the active motion block.

The footage included a total of 31 passenger cars and 4 trucks and / or buses in transit via the monitored lane. Table 1 shows the performance with respect to the classification ability of the algorithm.

  The algorithm correctly classified all four trucks / buses and correctly classified the thirty passenger cars across the target area. Tracks / buses that are well classified are shown in FIGS. It should be noted that the target area may capture vehicles traveling in different directions, but the detection and monitoring of these vehicles was automatically avoided using the information contained in the orientation of the motion vectors. Please note.

  Some portions of the detailed description herein are directed to operations on data bits performed by conventional computer components, including a central processing unit (CPU), a memory storage device for the CPU, and an attached display device. (Operation) algorithms and symbolic representations are presented. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to ensure that their work is most effectively conveyed to others skilled in the art. An algorithm is generally recognized as a coherent sequence of steps that can produce a desired result. These steps are those requiring physical manipulations of physical quantities. Usually, but not necessarily, these quantities take the form of electrical or magnetic signals that can be stored, transferred, combined, compared, or otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals by bits, values, elements, symbols, characters, terms, or numbers.

  However, it will be understood that all of the above and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise noted, as will be apparent from the description herein, throughout this description, reference will be made to "processing" or "operation" or "calculation" or "judgment (decision)" or "display." Description in terms of terms refers to the actions and processes of a computer system or similar computing device, which manipulates data represented as physical (electronic) quantities in registers and memories of the computer system, and Or registers or storage or transmission of such information, or conversion to other data, also represented as physical quantities in the display device.

  The exemplary embodiments also relate to an apparatus that performs the operations described herein. The apparatus may be specially constructed for the required purpose, or may comprise a general purpose computer selectively activated or reconfigured by a computer program stored on the computer. Such computer programs include, but are not limited to, any type of floppy disk, optical disk, CD-ROM, magneto-optical disk, read only memory (ROM), random access memory (RAM), EPROM), EEPROM), magnetic or It can be stored on a computer readable storage medium, such as any type of disk, including an optical card, or any type of medium suitable for storing electronic instructions and each coupled to a computer system bus.

  The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general-purpose systems may be used with the programs in accordance with the teachings herein, or it may be convenient to construct a more specialized apparatus to perform the methods described herein. I know. The structure for a variety of these systems will be apparent from the description above. In addition, the exemplary embodiments are not described with reference to any particular programming language. It will be appreciated that various programming languages may be used to implement the teachings of the exemplary embodiments described herein.

  A machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (eg, a computer). For example, machine-readable media includes, but is not limited to, read-only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory devices, and electrical, Includes optical, acoustic, or other forms of propagated signals (eg, carrier waves, infrared signals, digital signals, etc.).

  The methods illustrated throughout the specification can be implemented in a computer program product that can be implemented on a computer. The computer program product may include a non-transitory computer readable recording medium on which a control program is recorded, such as a disk, a hard drive, or the like. Common forms of non-transitory computer readable storage media are, for example, floppy disks, flexible disks, hard disks, magnetic tapes, or any other magnetic storage media, CD-ROM, DVD, or any other optical media, Includes RAM, PROM, EPROM, FLASH-EPROM or other memory chips or cartridges, or any other tangible medium from which a computer can be read and used.

  Alternatively, the method includes the step of transmitting the control program to a temporary carrier such as a transmittable carrier embodied as a data signal using a transmission medium such as an acoustic or light wave as generated during wireless and infrared data communications. It can be implemented in a medium.

Claims (8)

A computer-implemented method for classifying a vehicle captured by an imaging device oriented to include a field of view spanning a vehicle detection area, the method comprising:
a) generating a cluster of connected motion vectors representing vehicles detected in the target area, said clusters having only motion vectors having a magnitude greater than a predetermined threshold T; Generating the cluster;
b) associating one or more attributes with the cluster of motion vectors;
c) classifying detected vehicles based on the one or more attributes associated with the connected cluster of motion vectors;
Including
The motion vector is a compression type motion vector,
Step a) generates a cluster of compression-type motion vectors representing vehicles detected in a virtual target area determined in the field of view of said imaging device;
Computer implementation method.   Step c) classifies the detected vehicle as one of a relatively large vehicle, a relatively small vehicle, a truck, a bus, a passenger car, and a single vehicle, and wherein the one or more attributes is the motion vector The computer-implemented method of claim 1, comprising a geometric attribute associated with a cluster, wherein the geometric attribute comprises one or more of area, length, height, width, and bias.   The computer-implemented method of claim 1, wherein the vehicle classification is embedded in compressed data representing one or more image frames captured by the imaging device. An imaging system for classifying vehicles captured by an imaging system, wherein the imaging system comprises:
An imaging device oriented to include a field of view spanning the vehicle detection target area;
An image processor operably associated with the imaging device and performing a method of classifying vehicles captured by the imaging device;
Including
The method comprises
a) generating a cluster of connected motion vectors representing vehicles detected in the vehicle detection target area, wherein the clusters have only motion vectors having a magnitude greater than a predetermined threshold T; Generating the cluster, characterized by:
b) associating one or more attributes with the cluster of motion vectors;
c) classifying detected vehicles based on the one or more attributes associated with the connected cluster of motion vectors;
Including
The motion vector is a compression type motion vector,
Step a) generates a cluster of compression-type motion vectors representing vehicles detected in a virtual target area determined in the field of view of said imaging device;
Imaging system. A computer-implemented method for classifying a vehicle captured by an imaging device associated with a field of view including a vehicle detection target area, the method comprising:
a) extracting clusters of connected motion vectors representing vehicles detected in the vehicle detection target area, wherein the clusters have only motion vectors having a magnitude greater than a predetermined threshold. Extracting the cluster, characterized by:
b) associating one or more attributes with the cluster of motion vectors;
c) classifying detected vehicles based on the one or more attributes associated with the connected cluster of motion vectors;
Including
The motion vector is a compression type motion vector,
Step a) extracting a cluster of compression-type motion vectors representing vehicles detected in a virtual target area determined in the field of view of said imaging device;
Computer implementation method.   Step c) classifies the detected vehicle as one of a relatively large vehicle, a relatively small vehicle, a truck, a bus, a passenger car, and a single vehicle, and wherein the one or more attributes is the motion vector The computer-implemented method of claim 5, including a geometric attribute associated with the cluster, wherein the geometric attribute includes one or more of area, length, height, width, and bias. An image processing system for classifying vehicles captured by an imaging device,
a) extracting clusters of connected motion vectors representing vehicles detected in the vehicle detection target area, wherein the clusters have only motion vectors having a magnitude greater than a predetermined threshold. Extracting the cluster;
b) associating one or more attributes with the cluster of motion vectors;
c) classifying detected vehicles based on the one or more attributes associated with the connected cluster of motion vectors;
Including
The motion vector is a compression type motion vector,
Step a) extracting a cluster of compression-type motion vectors representing vehicles detected in a virtual target area determined in the field of view of said imaging device;
An image processor configured to perform the method.
Image processing system.   The image processing system according to claim 7, wherein the imaging device is one of a visible light video camera, an infrared video camera, a thermal video camera, and a satellite video camera.