CN105138525B - Traffic video processing unit and method and retrieval device and method - Google Patents

Traffic video processing unit and method and retrieval device and method Download PDF

Info

Publication number
CN105138525B
CN105138525B CN201410241113.0A CN201410241113A CN105138525B CN 105138525 B CN105138525 B CN 105138525B CN 201410241113 A CN201410241113 A CN 201410241113A CN 105138525 B CN105138525 B CN 105138525B
Authority
CN
China
Prior art keywords
video
traffic
index
retrieval
frame cluster
Prior art date
Application number
CN201410241113.0A
Other languages
Chinese (zh)
Other versions
CN105138525A (en
Inventor
黄跃峰
成斌
Original Assignee
株式会社日立制作所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社日立制作所 filed Critical 株式会社日立制作所
Priority to CN201410241113.0A priority Critical patent/CN105138525B/en
Publication of CN105138525A publication Critical patent/CN105138525A/en
Application granted granted Critical
Publication of CN105138525B publication Critical patent/CN105138525B/en

Links

Abstract

The present invention provides a kind of traffic video processing unit and method and retrieval device and method, and wherein traffic video processing unit includes: that the unit of retrieval object is extracted from traffic video data;The unit of topological Index relationship is established to video captured by adjacent traffic cameras;The video that traffic cameras is shot is divided into the unit of multiple frame clusters;The unit of index relative is established to the frame cluster in the video with topological Index relationship captured by adjacent traffic cameras with semantic feature correlation;With topological Index relationship is established to all videos using topological Index relation unit and topological Index relationship based on all videos and frame cluster index relative establish index relative to the frame cluster of all videos, constitute the unit in video index library.Unnecessary workload when retrieval traffic video can be reduced when carrying out traffic video retrieval based on traffic video processing unit of the invention and method and retrieval device and method, and can be avoided the influence caused by retrieval accuracy of unrelated video.

Description

Traffic video processing unit and method and retrieval device and method

Technical field

The present invention relates to field of video retrieval more particularly to a kind of traffic video based on traffic flow space-time characterisation to handle, Retrieve device and method.

Background technique

With the rapid development of multimedia technology, the mankind carry out information storage and propagate from single verbal description data hair The digital video data being made of media such as audio, text, pictures is opened up, and digital video is huge, at any time It increases sharply.Therefore, in field of video retrieval, current problems faced has no longer been digital video scarcity, but in face of huge Digital video data, how quickly and accurately to retrieve interested content.For this problem, there has been proposed be based on Video frequency searching (Content Based Video retrieval, CBVR) technology of content.So-called content based video retrieval system The certain semantic feature (color, texture, shape, movement etc.) in video data is exactly extracted, is then examined in video database The process of the matched image of rope or video clip, this, which is crossed range request and is automatically performed, is not having the case where manually participating in automatically Find matched image or video clip.Content based video retrieval system is different from traditional retrieval method based on keyword, It is an interleaving techniques, needs to make using the knowledge in the fields such as image procossing, pattern-recognition, computer vision, image understanding Based on.

Traffic video monitoring system is the indispensable subsystem of traffic guidance system, is a weight of intelligent transportation system Component part is wanted, while being also the technology that modern city traffic and management generally use.Traffic flow refers to automobile on road The Vehicle flow to be formed is continuously driven, the traffic flow of broad sense further includes non power driven vehicle stream and pedestrian stream.Traffic flow is regarded in traffic Has the characteristics that space-time expending in frequency monitoring network, i.e., if traffic rules allow between the adjacent traffic video in spatial position It is directly current, then the traffic flow of adjacent video record has continuity in time.For example, passing through any in transportation network The vehicle of video camera coverage centainly appears in the shooting model of the adjacent video camera in spatial position after a time interval Within enclosing.With the continuous expansion of city road network, the scale of traffic video monitoring system is also being gradually increased, this results in traffic Video data increases rapidly as a kind of huge data.How from vast as the open sea traffic video data user is retrieved Interested traffic information is of great significance to the management of traffic with control.

Patent application CN103186634A disclose a kind of intelligent traffic monitoring video based on content search method and Device, the invention are not matched that for comprehensive targetedly retrieve with user of prior art MPEG-7 standard semantic description The problem of, it proposes that the traffic data only extracted in intelligent traffic monitoring video carries out semantic description, then carries out video frequency searching Technical solution.However, the invention is with only content based video retrieval system technology without considering that traffic flow is regarded in traffic Have the characteristics that space-time expending in frequency monitoring network, need to retrieve all videos in traffic video monitoring system, Cause to be distributed on time and space scale in retrieving due to traffic video extensively, data scale is excessive and causes to retrieve Efficiency is lower and retrieval precision is low, compromises user experience.

Summary of the invention

For well-known technique the above deficiency, the present invention provides a kind of traffic video processing, retrieval device and Traffic video processing, search method.

The traffic video processing unit of the first aspect of the present invention includes: retrieval object extracting unit, is imaged from traffic Retrieval object is extracted in the video data of machine shooting;Adjacent video topological Index relationship establishes unit, is based on traffic flow space-time Characteristic establishes topological Index relationship to video captured by adjacent traffic cameras;Frame cluster sorts out unit, utilizes the inspection The video that the traffic cameras is shot is divided into multiple frame clusters by the semantic feature of rope object;Frame cluster index relative establishes unit, It is to the frame cluster in the video with topological Index relationship captured by adjacent traffic cameras with semantic feature correlation Establish index relative;Unit is established in video index library, establishes unit to all using the adjacent video topological Index relationship All videos of traffic cameras shooting establish topological Index relationship, and the topological Index relationship based on all videos and described The frame cluster index relative that frame cluster index relative establishes unit foundation establishes index relative to the frame cluster of all videos, constitutes video rope Draw library.

The traffic video retrieval device of the second aspect of the present invention includes retrieval unit, is handled using above-mentioned traffic video The semantic feature in the video index library that device is established, the retrieval object based on user's input carries out traffic video retrieval, obtains The traffic video of the retrieval object.

The traffic video processing method of the third aspect of the present invention includes: retrieval object extracting step, is imaged from traffic Retrieval object is extracted in the video data of machine shooting;Adjacent video topological Index relationship establishment step, takes the photograph adjacent traffic Video captured by camera establishes topological Index relationship;Frame cluster classifying step, will using the semantic feature of the retrieval object The video of the traffic cameras shooting is divided into multiple frame clusters;Frame cluster index relative establishment step images adjacent traffic Frame cluster in video with topological Index relationship captured by machine with semantic feature correlation establishes index relative;Video rope Draw library establishment step, owns using the adjacent video topological Index relationship establishment step to all traffic cameras shootings Video establishes topological Index relationship, and the topological Index relationship based on all videos and the frame cluster index relative establishment step The frame cluster index relative of middle foundation establishes index relative to the frame cluster of all videos, constitutes video index library.

The traffic video search method of the fourth aspect of the present invention includes searching step, is handled using above-mentioned traffic video The semantic feature in the video index library established in method, the retrieval object based on user's input carries out traffic video retrieval, obtains The traffic video of the retrieval object.

Technical effect

The present invention establishes topological Index relationship according to space-time characterisation of the traffic flow in traffic video monitoring network, so that When carrying out traffic video retrieval using traffic video processing unit of the invention and method and retrieval device and method, retrieved It only needs to retrieve the video with topological Index relationship in journey, reduces unnecessary video frequency searching, improve recall precision.Root The index relative that structuring unit is established according to the correlation of adjacent video structuring unit semantic feature, so that in retrieving It only needs to retrieve the structural unit with incidence relation, avoids and the entire content of target video is retrieved, improve Recall precision.Simultaneously as the reduction in video frequency searching space and video frequency searching content, so that non-targeted video in retrieving Amount of interference reduce, to improve the precision of video frequency searching.

Detailed description of the invention

Fig. 1 is the overall schematic of traffic video processing unit and retrieval device of the invention.

Fig. 2 is the overall schematic of traffic video processing method and search method of the invention.

Fig. 3 is the structure chart of traffic data in video.

Fig. 4 is the flow chart that traffic data extracts.

Fig. 5 is the schematic diagram and flow chart that adjacent camera topological Index relationship is established.

Fig. 6 is the flow chart and schematic diagram of traffic video structuring.

Fig. 7 is the flow chart and schematic diagram that adjacent video structuring unit incidence relation is established.

Fig. 8 is the flow chart that video index library is established.

Fig. 9 is the schematic diagram that video index library is established.

Figure 10 is the flow chart of video frequency searching.

Figure 11 is the schematic diagram of video frequency searching.

Specific embodiment

The present invention relates to a kind of traffic video processing unit and method based on traffic flow space-time characterisation and retrieval device and Method.In the following, being described with reference to the accompanying drawings embodiments of the present invention.

Fig. 1 is the overall schematic for indicating traffic video processing unit and retrieval device of the invention.As shown in Figure 1, handing over Intervisibility frequency processing device mainly includes with lower unit: the retrieval of retrieval object is extracted from the video data that traffic cameras is shot Object extracting unit;Topological Index relationship is established to video captured by adjacent traffic cameras based on traffic flow space-time characterisation Adjacent video topological Index relationship establish unit;The traffic cameras is shot using the semantic feature of the retrieval object Video be divided into multiple frame clusters frame cluster sort out unit;To captured by adjacent traffic cameras with topological Index relationship The frame cluster index relative that frame cluster in video with semantic feature correlation establishes index relative establishes unit;Utilize the phase Adjacent video topological Index relationship establishes unit and establishes topological Index relationship, and base to all videos that all traffic cameras are shot The frame cluster index relative of unit foundation is established to all in the topological Index relationship of all videos and the frame cluster index relative Unit is established in the video index library that the frame cluster of video establishes index relative.In addition, retrieving device as traffic video of the invention Possessed retrieval unit, the video index library established using above-mentioned traffic video processing unit, the inspection based on user's input The semantic feature of rope object carries out traffic video retrieval, obtains the traffic video of the object to be retrieved.

Traffic video processing method and search method of the invention shown in Fig. 2 are corresponding with device shown in FIG. 1, mainly into Row following steps: the step of extracting traffic data;The step of establishing adjacent video topological Index relationship;The step of frame cluster is sorted out; The step of establishing frame cluster index relative;With the step of establishing video index library.In addition, as traffic video retrieval side of the invention The searching step that method is carried out, using the video index library established in above-mentioned traffic video processing method, based on user's input The semantic feature for retrieving object carries out traffic video retrieval, obtains the traffic video of the object to be retrieved.Wherein, of the invention In traffic video processing unit and method, retrieval object is mainly motor-driven and non power driven vehicle, pedestrian.The semantic feature master of vehicle It include: type of vehicle, vehicle brand, license plate number, vehicle color, vehicle texture, vehicle shape and state of motion of vehicle Deng the semantic feature of pedestrian mainly includes face characteristic.

Each step is specifically described below.

1, in video traffic data extraction

Fig. 3 is the structure chart of traffic data in monitor video.It can be seen from the figure that traffic data relates generally to two sides Face: traffic object and traffic events.Wherein, traffic object mainly includes traffic participant (motor vehicles, non power driven vehicle, row People etc.) and traffic infrastructure (road, signal lamp, traffic sign, traffic marking, isolation strip and podium etc.).Traffic events It mainly include motor vehicle event (motor vehicles straight trip, motor vehicles left-hand rotation, motor vehicles right-hand rotation and motor vehicle violation etc.), non-machine Motor-car event (non power driven vehicle straight trip, non power driven vehicle are turned left, non power driven vehicle is turned right and non-motor vehicle is violating the regulations etc.) and pedestrian Event (street crossing of pedestrian's plane, the street crossing of pedestrian's solid and pedestrians disobeying traffic rule etc.).Traffic video processing unit provided by the invention and side Method and the retrieval device and method object to be handled and be retrieved mainly object relevant to traffic flow, including it is motor vehicle, non- Motor vehicle and pedestrian.During extracting traffic data, intelligent recognition algorithm (vehicle identification, vehicle can be utilized according to demand Board identification, pedestrian's identification, recognition of face, non-motor vehicle identification etc.) and manual identified the traffic data in video is extracted, As shown in Figure 4.

2, the foundation of the video surveillance network index database based on traffic flow space-time characterisation

(1) foundation of the topological Index relationship of video captured by adjacent camera

Traffic flow has the characteristics that space-time expending in traffic video monitoring network, as shown in fig. 5, it is assumed that regarding in traffic Frequently it is respectively video camera 1, video camera 2, video camera 3 and video camera 4 there are 4 adjacent video cameras in monitoring network, and passes through The traffic flow for crossing 1 coverage of video camera flows to video camera 2 and video camera 3 respectively.This, which means that, shoots model by video camera 1 The traffic flow object enclosed is in the coverage for centainly appearing in video camera 2 or video camera 3 after certain time interval It is interior, i.e., it is bound to occur in video camera 2 or 3 in the traffic flow object that 1 video of video camera occurred.On the contrary, video camera 1 and taking the photograph Camera 4 is adjacent on geographical location, but due to being limited by traffic rules, road equipment, coverage and other factors, Traffic flow object by 1 coverage of video camera does not pass through the coverage of video camera 4, i.e., shoots and regard in video camera 1 The traffic flow object for the existing mistake that occurs frequently will not centainly occur in 4 video of video camera in a reasonable time interval, unless around Other roads of row spend a relatively long time.According to the space-time expending feature of traffic flow, in video surveillance network The video of arbitrary neighborhood video camera shooting establishes topological Index relationship, indicate between them whether can at a reasonable time, it is empty Between within it is interrelated.As shown in figure 5, video camera 1 and video camera 2, video camera 1 and video camera 3 are there are incidence relation, foundation is opened up Flutter index relative.Although video camera 1 and 4 geographical location of video camera are closed on, incidence relation is not present, does not establish topological Index. The foundation of adjacent camera topological Index can reduce unnecessary video frequency searching, improve recall precision.For example, user will retrieve When the traffic object occurred in video camera 1, need to retrieve if not establishing video index captured by video camera 2,3 and 4 Three videos, only need to retrieve two videos captured by video camera 2,3 if establishing video index.

(2) traffic video structuring and the foundation of structuring unit incidence relation

In above-mentioned (1) the step of, according to the continuity of traffic flow over time and space, establish adjacent camera it Between topological Index relationship, improve recall precision.Since original video data does not have good hierarchy and structural, such as Fruit user thinks lookup and the video interested content of positioning associated, just has to completely retrieve original video, and inquiry is caused to be imitated Rate is lower, and user experience is poor.For this problem, the present invention carries out structuring processing to original video and establishes adjacent video The incidence relation of structuring unit.The successive frame with identical semantic feature is returned firstly, carrying out structuring processing to video It is merged into adjacent frame cluster when frame cluster is too small for same frame cluster, this is because the too small quantity that will increase frame cluster of frame cluster, drop Low search efficiency.Detailed process is as shown in the flow chart of the traffic video structuring of Fig. 6.A video is taken out, the video First frame is as present frame cluster and extracts the semantic feature of the frame cluster, if next frame and present frame cluster have the identical semanteme Feature, then the next frame is merged into the video cluster, repeatedly above procedure until a certain frame do not have the identical semantic feature, this Sample just obtains a frame cluster (video-frequency band), extracts semantic feature to the frame for not having the identical semantic feature and as another frame The first frame of cluster, then repeatedly above procedure, the classification until completing last frame complete the structuring of video.From Fig. 6 Video structural after schematic diagram can be seen that one section of video is divided into several video frame clusters (video clip), each video Semantic feature having the same between each frame of frame cluster.Then, the incidence relation for establishing adjacent video structuring unit is established Index relative between adjacent video frames cluster.Since the traffic object in the traffic flow video with topological relation is in traffic video The space-time expending feature having in monitoring network, so the semantic feature between adjacent video frames cluster is over time and space With correlation.The index relative between frame cluster is established according to the correlation of frame cluster semantic feature.Detailed process such as Fig. 7's is adjacent Shown in the flow chart that video structural unit incidence relation is established.Two are taken out in traffic video monitoring network, and there is topology to close The video of system carries out video structural processing respectively, traverses second with the semantic feature of first frame cluster of first video The semantic feature of each frame cluster of video establishes index relative if there are semantic dependencies between frame cluster, then, with The semantic feature of second frame cluster of one video traverses all frame clusters of second video, if there is semantic phase between frame cluster Guan Xing then establishes index relative, repeatedly above procedure, the traversal of the last one frame cluster until completing first video, in this way With regard to completing the foundation of the institutional unit index relationship of two videos.The lower half portion of Fig. 7 is the association of adjacent video structuring unit Index relative figure between frame cluster afterwards, as can be seen from the figure three sections of videos have done structuring processing respectively, and establish structure Change the index relative of unit, arrow indicates that there are index relatives between frame cluster in figure.

In order to more clearly description traffic video structuring and structuring unit incidence relation establish process, below with One example is illustrated.Assuming that there are two video cameras with topological relation in traffic video monitoring network, with vehicle Video structural processing is carried out to the video of two video camera shootings respectively for traffic object and establishes the association of structuring unit Relationship.The semantic feature of vehicle specifically includes that type of vehicle, vehicle brand, license plate number, vehicle color, vehicle texture, vehicle Shape and state of motion of vehicle etc., this example are described video frame using all license plate numbers in a frame as semantic feature.Depending on Frequency structurizing process are as follows: take out a video, be using the first frame of the video as the semantic feature of present frame cluster and the frame cluster The license plate number code collection of first frame, if next frame and present frame cluster include identical license plate number code collection, which is merged into The video cluster, repeatedly above procedure until a certain frame do not have identical license plate number code collection, thus obtain a frame cluster (video Section).Meanwhile using above-mentioned that frame without identical license plate number code collection as the first frame of next frame cluster, included by it Semantic feature of all license plate number code collections as next frame cluster, the then above procedure repeatedly of the frame to the frame and after it, directly To the classification for completing last frame, that is, complete the structuring of video.Same processing is done to second video, completes structure Change.Then set up the incidence relation between video frame cluster.Process is as follows: with the semantic feature of first frame cluster of first video That is the license plate number code collection in first frame cluster traverses the license plate number code collection of all frame clusters of second video, is if there is intersection There are identical license plate numbers between two frame clusters, then establish index relative, then, with the language of second frame cluster of first video All frame clusters that adopted feature traverses second video then establish index relative, repeatedly above procedure if there is intersection, until The traversal of the last one frame cluster of first video is completed, building for two institutional unit index relationships of video is thus completed It is vertical.

(3) foundation of traffic video monitoring network index database

Space-time expending is had the characteristics that in traffic video monitoring network according to traffic flow in the step of above-mentioned (1), it is right In the step of adjacent video establishes topological Index relationship, above-mentioned (2) according to the semantic dependency between frame and frame to video into Structuring of having gone handles and establishes index relative to adjacent video according to the semantic dependency between frame cluster.The two steps It is handled mainly for adjacent video, this part will establish video index library to entire monitoring network based on upper two steps. Fig. 8 and Fig. 9 is respectively the flow chart and schematic diagram for establishing video index library.As shown in figure 8, the establishment process in video index library is such as Under, a camera video is arbitrarily chosen first from road network as current video, then arbitrarily chooses the phase of the video Adjacent video, judges whether current video and adjacent video have topological Index relationship, if having topological Index relationship, establishes Index relative between the two and the index relative for establishing video frame cluster between the two;Then judge current video and next phase Adjacent video of the index relative of adjacent video until having judged the last one current video, thus rope of the current video in road network Draw relationship foundation completion and the video is not re-used as adjacent video and is judged.Then, that optionally completes index relative works as forward sight The adjacent video of frequency is as current video, and above step is completed to establish road network until having traversed all videos in road network repeatedly Video index library.Fig. 9 is the schematic diagram for establishing video index library, and current video (i.e. current camera is established in a and b expression in figure The video of shooting) and the adjacent video video of shooting (i.e. adjacent camera) between topological Index relationship, arrow indicates video Between there are topological relation, cross indicates that topological relation is not present between video, builds if video is there are topological Index relationship The index relative of vertical video frame cluster, c and d indicate to establish topology of the adjacent video of current video in traffic video monitoring network Index relative and frame cluster index relative, c, d obtain the index between the topological Index relationship of whole network video and frame cluster repeatedly Relationship is as shown bye.

For the process that more clearly description traffic video monitoring network index database is established, said with an example It is bright.Example carries out semantic description to video frame characterized by license plate numbers all in a frame, and to all views in monitoring network Frequency carries out structuring processing.The establishment process in video index library is as follows, and the view of a video camera is arbitrarily chosen first from road network Frequency is used as current video, then arbitrarily chooses the video of the adjacent camera of the video camera, judges current video and adjacent Whether video has topological Index relationship, if having topological Index relationship, establishes index relative between the two and root The index relative for establishing video frame cluster between the two according to whether there is identical license plate number between two frame clusters;Then judgement is current Adjacent video of the index relative of video and next adjacent video until having judged the last one current video, thus current video Index relative in road network establishes completion and the video is not re-used as adjacent video and is judged.Then, index is optionally completed The adjacent video of the current video of relationship is as current video, above step repeatedly, until all videos in road network have been traversed, Road network video index library is established in completion.

3, traffic video is retrieved

Mainly introduce the video frequency searching carried out based on traffic video monitoring network index database in this part.Retrieving is specifically such as Shown in the video frequency searching flow chart of Figure 10, user inputs the semantic feature of the object to be retrieved according to demand, retrieves and semantic After one video of characteristic matching, positioning and the matched video frame cluster of semantic feature and the frame cluster navigated to is set as present frame Cluster;According to traffic video monitoring network index database, all frame clusters of retrieval and present frame cluster with index relative, if do not deposited , then terminate to obtain search result, if it is present the semantic feature that the frame cluster retrieved and user input is matched, If retrieve with the matched frame cluster of semantic feature, the frame cluster of successful match is set as present frame cluster, if retrieval less than The frame cluster matched then terminates, and above procedure is until retrieving less than matched frame cluster repeatedly.Figure 11 is the schematic diagram of video frequency searching, in figure Arrow between video indicates there is topological Index relationship between video, and cross indicates that there is no topological Indexes to close between video It is that the arrow between frame cluster indicates there is index relative between video frame cluster, the representation of video shot in ellipse retrieves the view of target Frequently, the frame cluster of filled black is indicated with the matched frame cluster of semantic feature, the video that grey representation of video shot was retrieved, but it is not mesh Mark video.What 1 expression in Figure 11 in a retrieved includes first video with the matched frame cluster of semantic feature, corresponding frame Cluster filled black, 2 in a indicate and 1 has topological Index relationship, and frame cluster therein and the frame cluster retrieved have index Relationship retrieves the frame cluster in 2 with index relative, does not retrieve matched target, and 2 are expressed as grey, indicates inspection The video of target is crossed but do not retrieved to rope.In b 3 as 2 situations in a i.e. there are topological Index relationships and frame cluster rope Draw relationship, but does not retrieve matched target.1 in 4 and a in c has the index relative of topological Index relationship and frame cluster, And retrieve target from the frame cluster with index relative, then the frame cluster filled black, it is set as present frame cluster, then repeatedly Above procedure, until retrieving less than matched frame cluster, final search result such as e, shown in f.It can be seen from figure 11 that user's root All and semantic relevant video is retrieved according to demand, need to only retrieve the part in video index library with topological Index relationship Video, without being retrieved to all videos in video network, and during retrieving a video, it is only necessary to tool There is the video frame cluster of semantic association relationship to be retrieved, is retrieved without the full content to video, thus significantly Improve the efficiency of video frequency searching.Simultaneously as the reduction in video frequency searching space and video frequency searching content, so that in retrieving The amount of interference of non-targeted video reduces, and the probability of false retrieval and erroneous detection reduces, to improve the precision of video frequency searching.

For the process that more clearly description traffic video is handled, it is illustrated with an example.Traffic video index The semantic feature of library frame cluster is the set of all license plate numbers occurred in the frame cluster, and the index relative between frame cluster is two frames There are there are identical license plate numbers between intersection i.e. two frame cluster between cluster.If user wants retrieval vehicle A, then user Need to input the license plate number of vehicle A.Herein with the semantic feature of the license plate number of vehicle A, but in order to more accurately retrieve It, also can be using two or more features such as the license plate number of vehicle A and the color of vehicle A, texture as semantic spy to vehicle A Sign.The video that first vehicle A occurs and positioning video frame cluster are retrieved, and the frame cluster is positioned as present frame cluster, so Enter traffic video network index database afterwards to be retrieved, retrieval and present frame cluster have all frame clusters of index relative i.e. and current There are the frame clusters of intersection for frame cluster license plate number set, if retrieval terminates to obtain less than the frame cluster of the license plate number containing vehicle A Search result, if retrieving the frame cluster of the license plate number containing vehicle A, which becomes present frame cluster, repeatedly above procedure, Until can not find the video frame cluster comprising vehicle A license plate number, the frame cluster all comprising vehicle A license plate number is finally obtained.These The frame cluster that spatially there is frame cluster adjacent characteristic i.e. vehicle A to occur next time is spatially centainly adjacent with present frame cluster, It one is scheduled on before or after present frame cluster in time on time with sequencing, that is, vehicle A frame cluster occurred next time, therefore this A little frame clusters can form a frame cluster string over time and space.

Embodiment 1

It is retrieved based on traffic video processing unit of the invention and method and the vehicle route for retrieving device and method

In some cases, public security department and traffic control department need to carry out the path that vehicle driving is crossed by traffic video Retrieval, the present embodiment establish vehicle route in combination with the space-time characterisation of traffic flow as semantic feature using license plate number and retrieve video Index database, and vehicle route is retrieved based on index database.Process is as follows:

1, it extracts vehicle in video and is used as retrieval object;

2, vehicle route retrieval video index library is established based on traffic flow space-time characterisation, which is divided into three steps, first The video topological Index relationship of adjacent camera is first established according to space-time characterisation of the traffic flow in traffic video monitoring network, It is secondary, the video of each video camera is carried out at structuring with the semantic feature that all license plate numbers in video frame are retrieval object Reason, and be that judgment criteria establishes structuring unit with the presence or absence of intersection according to the license plate number code collection of adjacent video structuring unit Index relative;Third step establishes the vehicle route retrieval video index library of traffic video monitoring network based on first two steps;

3, the license plate number of user's input is obtained;

4, video frequency searching is carried out based on vehicle route retrieval video index library, the part is according to establishing in video index library The topological Index relationship of video camera and the index relative of adjacent video structural unit carry out video frequency searching to retrieval information, if it exists The video of retrieval then returns to the video retrieved;

5, due to the position of video camera, the position of video camera is fixed adjacent to each other and in traffic video monitoring network, so can The road of the traveling of vehicle is determined with the position of the video occurred according to vehicle, so that it is determined that the path of vehicle.

Embodiment 2

It is retrieved based on traffic video processing unit of the invention and method and the special car for retrieving device and method

In this example special car refer to can the differentiation of visually significant and common vehicle vehicle, such as armoured van, There is significant area on taxi, ambulance, sprinkling truck etc. due to the particularity of these vehicle functions with common vehicle from the appearance Not, thus these vehicles semantic feature (such as type of vehicle, vehicle shape, vehicle color, vehicle texture) visually and The semantic feature of the vision of common vehicle makes a big difference.According to this feature, the present embodiment is with the appearance language of special car Adopted feature establishes special car video index library in combination with the space-time characterisation of traffic flow, and based on index database to special car into Row retrieval.Process is as follows:

1, it extracts a certain special car in video and is used as retrieval object;

2, special car video index library is established based on traffic flow space-time characterisation, which is divided into three steps, first root The video topological Index relationship of adjacent camera is established according to space-time characterisation of the traffic flow in traffic video monitoring network, secondly, With the appearance (such as type of vehicle, vehicle shape, vehicle color, vehicle texture) of the special car in the video frame of each video camera For retrieve object semantic feature to video carry out structuring processing, and according to adjacent video structuring unit whether all include Special car is the index relative that judgment criteria establishes structuring unit;Third step establishes traffic video prison based on first two steps Control the special car video index library of network;

3, the appearance semantic feature of the special car of user's input is obtained;

4, video frequency searching is carried out based on special car video index library, the part is according to the camera shooting established in video index library The topological Index relationship of machine and the index relative of adjacent video structural unit carry out video frequency searching to retrieval information, retrieve if it exists Video, then return to the video retrieved;

5, the video and video clip in video index library comprising special car can be then retrieved according to above procedure.

Embodiment 3

Pedestrian retrieval based on traffic video processing unit of the invention and method and retrieval device and method

The present embodiment establishes pedestrian retrieval video in combination with the space-time characterisation of traffic flow using face characteristic as semantic feature Index database, and pedestrian is retrieved based on index database.Process is as follows:

1, it extracts pedestrian in video and is used as retrieval object;

2, pedestrian retrieval video index library is established based on traffic flow space-time characterisation, which is divided into three steps, first root The video topological Index relationship of adjacent camera is established according to space-time characterisation of the traffic flow in traffic video monitoring network, secondly, Structuring processing is carried out to video with the semantic feature that all face characteristics in the video frame of each video camera are retrieval object, and And integrate the rope for establishing structuring unit as judgment criteria with the presence or absence of intersection according to the face characteristic of adjacent video structuring unit Draw relationship;Third step establishes the pedestrian retrieval video index library of traffic video monitoring network based on first two steps;

3, the face characteristic of user's input is obtained;

4, video frequency searching is carried out based on pedestrian retrieval video index library, the part is according to the camera shooting established in video index library The topological Index relationship of machine and the index relative of adjacent video structural unit carry out video frequency searching to retrieval information, retrieve if it exists Video, then return to the video retrieved;

5, the video and video clip in video index library comprising pedestrian can be then retrieved according to above procedure.

Three above embodiment establishes topological rope all in accordance with space-time characterisation of the traffic flow in traffic video monitoring network Draw relationship, so that retrieving only needs to retrieve the video with topological Index relationship, reduces unnecessary video frequency searching, mention High recall precision;It is closed according to the index that the correlation of adjacent video structuring unit semantic feature establishes structuring unit System, so that only needing to retrieve the structural unit with incidence relation in retrieving, avoids to the entire of target video Content is retrieved, and recall precision is improved.Simultaneously as the reduction in video frequency searching space and video frequency searching content, so that inspection The amount of interference of non-targeted video reduces during rope, to improve the precision of video frequency searching.

Claims (8)

1. a kind of traffic video processing unit characterized by comprising
Object extracting unit is retrieved, retrieval object is extracted from the video data that traffic cameras is shot;
Adjacent video topological Index relationship establishes unit, based on traffic flow space-time characterisation to captured by adjacent traffic cameras Video establish topological Index relationship;
Frame cluster sorts out unit, is divided into the video that the traffic cameras is shot using the semantic feature of the retrieval object more A frame cluster;
Frame cluster index relative establishes unit, in the video captured by adjacent traffic cameras with topological Index relationship Frame cluster with semantic feature correlation establishes index relative;With
Unit is established in video index library, is closed using the topological Index that the adjacent video topological Index relationship establishes unit foundation Whether system, the video for judging that the video of the traffic cameras shooting of any selection is shot with adjacent traffic cameras have Topological Index relationship is established topological Index relationship with topological Index relationship, and is closed based on frame cluster index System establishes the frame cluster index relative of unit foundation to the selected traffic cameras shooting with topological Index relationship The video that video and adjacent traffic cameras are shot establishes video frame cluster index relative, by opening up to described in the foundation of all videos Index relative and video frame cluster index relative are flutterred to constitute video index library.
2. traffic video processing unit as described in claim 1, it is characterised in that:
The frame cluster sorts out unit and the identical frame of semantic feature of the retrieval object is classified as the same frame cluster.
3. traffic video processing unit as claimed in claim 1 or 2, it is characterised in that:
The retrieval object includes motor vehicle, non-motor vehicle and pedestrian,
The semantic feature of the retrieval object is resemblance, the vehicle color feature, vehicle textural characteristics, license plate number of vehicle At least one of feature, type of vehicle feature, vehicle brand feature and face characteristic.
4. a kind of traffic video retrieves device characterized by comprising
Including retrieval unit, the video established using traffic video processing unit described in any one of any one of claims 1 to 33 The semantic feature of index database, the retrieval object based on user's input carries out traffic video retrieval, obtains the friendship of the retrieval object Intervisibility frequency.
5. a kind of traffic video processing method characterized by comprising
Object extracting step is retrieved, retrieval object is extracted from the video data that traffic cameras is shot;
Adjacent video topological Index relationship establishment step, based on traffic flow space-time characterisation to captured by adjacent traffic cameras Video establish topological Index relationship;
The video that the traffic cameras is shot is divided into more by frame cluster classifying step using the semantic feature of the retrieval object A frame cluster;
Frame cluster index relative establishment step, in the video captured by adjacent traffic cameras with topological Index relationship Frame cluster with semantic feature correlation establishes index relative;With
Video index library establishment step utilizes the topological Index established in the adjacent video topological Index relationship establishment step Whether relationship, the video for judging that the video of the traffic cameras shooting of any selection is shot with adjacent traffic cameras have There is topological Index relationship, topological Index relationship is established with topological Index relationship, and index based on the frame cluster The frame cluster index relative established in relationship establishment step claps the selected traffic cameras with topological Index relationship The video that the video and adjacent traffic cameras taken the photograph are shot establishes video frame cluster index relative, by establishing institute to all videos Topological Index relationship and video frame cluster index relative are stated to constitute video index library.
6. traffic video processing method as claimed in claim 5, it is characterised in that:
In the frame cluster classifying step, the identical frame of semantic feature of the retrieval object is classified as the same frame cluster.
7. such as traffic video processing method described in claim 5 or 6, it is characterised in that:
The retrieval object includes motor vehicle, non-motor vehicle and pedestrian,
The semantic feature of the retrieval object is resemblance, the vehicle color feature, vehicle textural characteristics, license plate number of vehicle At least one of feature, type of vehicle feature, vehicle brand feature and face characteristic.
8. a kind of traffic video search method, it is characterised in that:
Including searching step, the video established in traffic video processing method described in any one of claim 5 to 7 is utilized The semantic feature of index database, the retrieval object based on user's input carries out traffic video retrieval, obtains the friendship of the retrieval object Intervisibility frequency.
CN201410241113.0A 2014-05-30 2014-05-30 Traffic video processing unit and method and retrieval device and method CN105138525B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410241113.0A CN105138525B (en) 2014-05-30 2014-05-30 Traffic video processing unit and method and retrieval device and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410241113.0A CN105138525B (en) 2014-05-30 2014-05-30 Traffic video processing unit and method and retrieval device and method

Publications (2)

Publication Number Publication Date
CN105138525A CN105138525A (en) 2015-12-09
CN105138525B true CN105138525B (en) 2019-10-01

Family

ID=54723876

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410241113.0A CN105138525B (en) 2014-05-30 2014-05-30 Traffic video processing unit and method and retrieval device and method

Country Status (1)

Country Link
CN (1) CN105138525B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106777078B (en) * 2016-12-13 2020-12-22 广东中星电子有限公司 Video retrieval method and system based on information database
CN108205895A (en) * 2016-12-16 2018-06-26 杭州海康威视数字技术股份有限公司 A kind of method and device for building vehicle style training sample picture library
CN107920280A (en) * 2017-03-23 2018-04-17 广州思涵信息科技有限公司 The accurate matched method and system of video, teaching materials PPT and voice content
CN109325432A (en) * 2018-09-13 2019-02-12 联想(北京)有限公司 A kind of recognition methods of three dimensional object, equipment and computer-readable storage media

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101075376A (en) * 2006-05-19 2007-11-21 北京微视新纪元科技有限公司 Intelligent video traffic monitoring system based on multi-viewpoints and its method
CN200990147Y (en) * 2006-05-19 2007-12-12 北京微视新纪元科技有限公司 Intelligent video traffic monitoring system based on multi-view point
CN101742267A (en) * 2009-12-31 2010-06-16 武汉市公安科学技术研究所 Video monitoring system of full digitalized networking safe city
WO2010097599A1 (en) * 2009-02-27 2010-09-02 British Telecommunications Public Limited Company Video segmentation
CN102289520A (en) * 2011-09-15 2011-12-21 山西四和交通工程有限责任公司 Traffic video retrieval system and realization method thereof
CN102724482A (en) * 2012-06-18 2012-10-10 西安电子科技大学 Intelligent visual sensor network moving target relay tracking system based on GPS (global positioning system) and GIS (geographic information system)
CN202563526U (en) * 2012-03-22 2012-11-28 北京尚易德科技有限公司 Transportation vehicle detection and recognition system based on video
CN102880692A (en) * 2012-09-19 2013-01-16 上海交通大学 Retrieval-oriented monitoring video semantic description and inspection modeling method
CN102929903A (en) * 2012-07-04 2013-02-13 北京中盾安全技术开发公司 Rapid video retrieval method based on layered structuralized description of video information
CN103716587A (en) * 2013-12-12 2014-04-09 深圳先进技术研究院 Video tracking method based on GIS network analysis and buffer area analysis

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7761900B2 (en) * 2006-08-02 2010-07-20 Clarendon Foundation, Inc. Distribution of content and advertisement

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101075376A (en) * 2006-05-19 2007-11-21 北京微视新纪元科技有限公司 Intelligent video traffic monitoring system based on multi-viewpoints and its method
CN200990147Y (en) * 2006-05-19 2007-12-12 北京微视新纪元科技有限公司 Intelligent video traffic monitoring system based on multi-view point
WO2010097599A1 (en) * 2009-02-27 2010-09-02 British Telecommunications Public Limited Company Video segmentation
CN101742267A (en) * 2009-12-31 2010-06-16 武汉市公安科学技术研究所 Video monitoring system of full digitalized networking safe city
CN102289520A (en) * 2011-09-15 2011-12-21 山西四和交通工程有限责任公司 Traffic video retrieval system and realization method thereof
CN202563526U (en) * 2012-03-22 2012-11-28 北京尚易德科技有限公司 Transportation vehicle detection and recognition system based on video
CN102724482A (en) * 2012-06-18 2012-10-10 西安电子科技大学 Intelligent visual sensor network moving target relay tracking system based on GPS (global positioning system) and GIS (geographic information system)
CN102929903A (en) * 2012-07-04 2013-02-13 北京中盾安全技术开发公司 Rapid video retrieval method based on layered structuralized description of video information
CN102880692A (en) * 2012-09-19 2013-01-16 上海交通大学 Retrieval-oriented monitoring video semantic description and inspection modeling method
CN103716587A (en) * 2013-12-12 2014-04-09 深圳先进技术研究院 Video tracking method based on GIS network analysis and buffer area analysis

Also Published As

Publication number Publication date
CN105138525A (en) 2015-12-09

Similar Documents

Publication Publication Date Title
Chang et al. Argoverse: 3d tracking and forecasting with rich maps
US10152644B2 (en) Progressive vehicle searching method and device
Serna et al. Detection, segmentation and classification of 3D urban objects using mathematical morphology and supervised learning
Karagiorgou et al. On vehicle tracking data-based road network generation
Xiao et al. Vehicle detection and tracking in wide field-of-view aerial video
CN104303193B (en) Target classification based on cluster
Cordts et al. The cityscapes dataset
Zyner et al. A recurrent neural network solution for predicting driver intention at unsignalized intersections
US9251425B2 (en) Object retrieval in video data using complementary detectors
Rasouli et al. Are they going to cross? a benchmark dataset and baseline for pedestrian crosswalk behavior
CN102254429B (en) Video identification-based detection method of detection device of violation vehicles
Tian et al. Hierarchical and networked vehicle surveillance in ITS: a survey
KR101589711B1 (en) Methods and systems for processing of video data
Hua et al. Recurrently exploring class-wise attention in a hybrid convolutional and bidirectional LSTM network for multi-label aerial image classification
Fernyhough et al. Constructing qualitative event models automatically from video input
Tang et al. Pamtri: Pose-aware multi-task learning for vehicle re-identification using highly randomized synthetic data
Ding et al. Network-matched trajectory-based moving-object database: Models and applications
Jeung et al. Trajectory pattern mining
CN105335702B (en) A kind of bayonet model recognizing method based on statistical learning
Cobo et al. A bibliometric analysis of the intelligent transportation systems research based on science mapping
EP2840786A2 (en) Apparatus, system and method
US20200342242A1 (en) Method, apparatus, and system for generating synthetic image data for machine learning
CN107729818B (en) Multi-feature fusion vehicle re-identification method based on deep learning
CN104778245B (en) Similar track method for digging and device based on magnanimity license plate identification data
JP2018503160A (en) Near-online multi-target tracking using aggregate local flow descriptor (ALFD)

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant