CN110275983B - Retrieval method and device of traffic monitoring data - Google Patents

Retrieval method and device of traffic monitoring data Download PDF

Info

Publication number
CN110275983B
CN110275983B CN201910485817.5A CN201910485817A CN110275983B CN 110275983 B CN110275983 B CN 110275983B CN 201910485817 A CN201910485817 A CN 201910485817A CN 110275983 B CN110275983 B CN 110275983B
Authority
CN
China
Prior art keywords
data
monitoring data
traffic monitoring
index
license plate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910485817.5A
Other languages
Chinese (zh)
Other versions
CN110275983A (en
Inventor
刘祥
刘雪莉
孙论强
郝旭宁
雷迅
王彬
王文建
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hisense TransTech Co Ltd
Original Assignee
Hisense TransTech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hisense TransTech Co Ltd filed Critical Hisense TransTech Co Ltd
Priority to CN201910485817.5A priority Critical patent/CN110275983B/en
Publication of CN110275983A publication Critical patent/CN110275983A/en
Application granted granted Critical
Publication of CN110275983B publication Critical patent/CN110275983B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/71Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7847Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/7867Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/787Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10Services
    • G06Q50/26Government or public services

Abstract

The disclosure discloses a retrieval method and a retrieval device of traffic monitoring data, which are applied to a server of a retrieval system and comprise the following steps: receiving a query request initiated by a client, wherein the query request comprises query conditions configured for querying traffic monitoring data; inquiring and determining index data matched with the inquiry conditions in an index database according to the inquiry conditions to obtain monitoring data identifications associated with the determined index data; acquiring traffic monitoring data stored by taking a monitoring data identifier as a main key in a full database, wherein the traffic monitoring data is obtained by taking the monitoring data identifier as the main key in the full database and performing full storage on corresponding structured attribute data; and returning the acquired traffic monitoring data to the client. Therefore, the speed of data return is improved, and the retrieval can be carried out according to the free combination of the retrieval fields.

Description

Retrieval method and device of traffic monitoring data
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for retrieving traffic monitoring data.
Background
With the rapid construction of the skynet project, a large amount of traffic monitoring videos are acquired every day. During the storage of the traffic monitoring video, the traffic monitoring video is usually extracted in a structured manner, and effective data is extracted from the traffic monitoring video to obtain traffic monitoring data of the traffic monitoring video. Instead of storing traffic monitoring video, traffic monitoring data is stored.
However, as the number of collected traffic monitoring videos increases, the number of stored traffic monitoring data also increases, and for a user, how to quickly retrieve required data from a large amount of monitoring data becomes a problem to be solved urgently today.
Disclosure of Invention
In order to solve the problems in the related art, the present disclosure provides a method and an apparatus for retrieving traffic monitoring data.
In a first aspect, a method for retrieving traffic monitoring data is applied to a server of a retrieval system, the retrieval system is provided with an index database and a full database, and the method comprises the following steps:
receiving a query request initiated by a client, wherein the query request comprises query conditions configured for querying traffic monitoring data;
according to the query condition, searching and determining index data matched with the query condition in the index database, and obtaining a monitoring data identifier associated with the determined index data, wherein the index data is obtained by performing data extraction on each retrieval field according to the retrieval fields configured for the index database and according to the structural attribute data of the traffic monitoring video;
acquiring traffic monitoring data stored by taking the monitoring data identifier as a main key in the full database, wherein the traffic monitoring data is obtained by taking the monitoring data identifier as the main key in the full database and performing full storage on the corresponding structural attribute data;
returning the acquired traffic monitoring data to the client
In a second aspect, a traffic monitoring data retrieval device is applied to a server of a retrieval system, the retrieval system is provided with an index database and a full database, and the device comprises:
a query request receiving module configured to: receiving a query request initiated by a client, wherein the query request comprises query conditions configured for querying traffic monitoring data;
a monitoring data identification obtaining module configured to: according to the query condition, querying and determining index data matched with the query condition in the index database, and obtaining a monitoring data identifier associated with the determined index data, wherein the index data is obtained by performing data extraction on each retrieval field according to the retrieval field configured for the index database and structured attribute data obtained by performing structured processing on each traffic monitoring video acquired in real time, and the index database is used for associating and storing the index data corresponding to each structured attribute data and the monitoring data identifier for identifying the structured attribute data;
a traffic monitoring data acquisition module configured to: acquiring traffic monitoring data stored by taking the monitoring data identifier as a primary key in the full database, wherein the traffic monitoring data is obtained by taking the monitoring data identifier as a primary key in the full database and performing full storage on the corresponding structured attribute data;
a return module configured to: and returning the acquired traffic monitoring data to the client.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
because the matching is only carried out in the index database storing the index data and the monitoring data identification according to the query condition, but not in the full database storing the traffic monitoring data, and compared with the full database, the data amount stored in the index database is less, the speed of matching in the index data according to the query condition is greatly improved compared with the speed of matching in the full database according to the query condition, thereby ensuring that a user can quickly retrieve the required traffic monitoring data in a retrieval system. The data return rate of the traffic monitoring data is greatly improved. And the user can freely combine the query conditions by utilizing the retrieval fields in the index database according to the requirements, thereby realizing the multi-angle and multi-granularity free combined query.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
FIG. 1 is a schematic illustration of an implementation environment according to the present disclosure;
FIG. 2 is a block diagram illustrating a server in accordance with an exemplary embodiment;
FIG. 3 is a flow chart illustrating a method of retrieving traffic monitoring data in accordance with an exemplary embodiment;
FIG. 4 is a flow diagram in one embodiment of steps preceding step S130 of the corresponding embodiment of FIG. 3;
FIG. 5 is a flow diagram in one embodiment of steps after step S130 of the corresponding embodiment of FIG. 3;
FIG. 6 is a flow diagram of steps in one embodiment before step S143 of the corresponding embodiment of FIG. 5;
FIG. 7 is a flow diagram of steps in one embodiment before step S220 of FIG. 6 is implemented correspondingly;
FIG. 8 is a flow diagram of steps in one embodiment before step S310 of the corresponding implementation of FIG. 7;
FIG. 9 is a flow diagram illustrating retrieval of a monitoring data identification from an index database in accordance with one particular embodiment;
FIG. 10 is a flow diagram illustrating a method of retrieving traffic monitoring data according to another embodiment;
FIG. 11 is a flow chart diagram illustrating a method of retrieving traffic monitoring data according to another exemplary embodiment;
FIG. 12 is a flow diagram illustrating a data logging retrieval system in accordance with one particular embodiment;
FIG. 13 is a block diagram illustrating a traffic monitoring data retrieval device in accordance with an exemplary embodiment;
fig. 14 is a block diagram illustrating a traffic monitoring data retrieval device according to another exemplary embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.
FIG. 1 is a schematic illustration of an implementation environment according to the present disclosure. The implementation environment includes: at least one terminal 101 (only two are shown in the figure), a server 102 of the retrieval system and a structured extraction server 103, wherein the server 102 of the retrieval system is provided with an index database server 1021 and a full database server 1022.
The retrieval system is used for retrieving the traffic monitoring data. The traffic monitoring data is data reflecting content in the traffic monitoring video obtained by performing data extraction and structuring processing according to the acquired traffic monitoring video, such as acquisition time information of the traffic monitoring video, vehicle information of vehicles in the traffic monitoring video (for example, license plate numbers of the vehicles, brands of the vehicles, current speed of the vehicles, and the like), storage position information of effective pictures extracted from the traffic monitoring video, and the like.
The index database server 1021 is used as a carrier of the index database and is used for storing index data and monitoring data identification obtained through the traffic monitoring video; the full database server 1022 serves as a carrier of the full database and is used for storing traffic monitoring data obtained through the traffic monitoring video. In the server 102 of the retrieval system, the index database server 1021 and the full database server 1022 are in communication connection with each other, so that after the index database server 1021 determines index data matched with a query condition according to the query condition, and further obtains a monitoring data identifier associated with the index data, traffic monitoring data using the monitoring data identifier as a main key is correspondingly obtained in the full database server 1022 according to the monitoring data identifier.
The client program of the retrieval system is run in the terminal 101, and the run client program is the client of the retrieval system, so that interaction between a terminal user and the server 102 of the retrieval system is realized, for example, a query request is initiated to the server 102 of the retrieval system through the client of the terminal 101 to obtain traffic monitoring data required to be queried. The client of the retrieval system may be an application client, or a web page client, which is not specifically limited herein, and the terminal 101 may be a desktop computer, a notebook computer, a tablet computer, or the like, which is not specifically limited herein.
For the index data and the traffic monitoring data stored by the server 102 of the retrieval system, both are derived from the traffic monitoring video, in the technical scheme of the disclosure, in order to facilitate the retrieval of the traffic monitoring data, before the data is stored in the server 102 of the retrieval system, the acquired traffic monitoring video is subjected to structuring processing to obtain the structured data of the traffic monitoring video. The structured extraction server 103 is used for carrying out structured processing on the traffic monitoring video.
By establishing a communication connection between the structured extraction server 103 and the server 102 of the retrieval system, the server 102 of the retrieval system obtains the structured data of the traffic monitoring video from the structured extraction server 103, and further generates the index data of the traffic monitoring video, the traffic monitoring data and the monitoring data identifier according to the structured data, so as to construct a link between the index data and the traffic monitoring data through the monitoring data identifier.
FIG. 2 is a block diagram illustrating a server in accordance with an example embodiment. Server 200 may act as server 102 of the retrieval system in the embodiment of fig. 1.
It should be noted that the server 200 is only an example adapted to the present invention, and should not be considered as providing any limitation to the scope of the present invention. The server 200 is also not to be construed as necessarily dependent upon or having one or more components of the exemplary server 200 shown in fig. 2.
The hardware structure of the server 200 may be greatly different due to different configurations or performances, as shown in fig. 2, the server 200 includes: a power supply 210, an interface 230, at least one memory 250, and at least one Central Processing Unit (CPU) 270.
The power supply 210 is used to provide operating voltage for each hardware device on the server 200.
The interface 230 includes at least one wired or wireless network interface 231, at least one serial-to-parallel conversion interface 233, at least one input/output interface 235, and at least one USB interface 237, etc. for communicating with external devices.
The storage 250 is used as a carrier for resource storage, and may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon include an operating system 251, an application 253 or data 255, etc., and the storage manner may be a transient storage or a permanent storage. The operating system 251 is used for managing and controlling various hardware devices and application programs 253 on the server 200 to implement the computation and processing of the mass data 255 by the central processing unit 270, and may be Windows server, mac OS XTM, unixTM, linux, freeBSDTM, freeRTOS, and the like. The application 253 is a computer program that performs at least one specific task on top of the operating system 251, and may include at least one module (not shown in fig. 2), each of which may contain a series of computer-readable instructions for the server 200. The data 255 may be index data, monitoring data identification, etc. stored in an index database, or monitoring data, etc. stored in a full database.
Central processor 270 may include one or more processors and is configured to communicate with memory 250 via a bus for computing and processing mass data 255 in memory 250.
As described in detail above, a server 200 to which the present invention is applicable will implement the methods of the present disclosure by central processor 270 reading a series of computer readable instructions stored in memory 250.
Furthermore, the present invention can be implemented by hardware circuitry or by a combination of hardware circuitry and software instructions, and thus, implementation of the present invention is not limited to any specific hardware circuitry, software, or combination of both.
FIG. 3 is a flow chart illustrating a method of retrieving traffic monitoring data in accordance with an exemplary embodiment. The method for retrieving traffic monitoring data is used for the server 102 of the retrieval system of the implementation environment shown in fig. 1. As shown in fig. 3, the method for retrieving traffic monitoring data may be executed by the server 200, wherein the retrieval system is provided with an index database and a full database, and the method includes:
step S110, receiving a query request initiated by the client, where the query request includes query conditions configured for querying traffic monitoring data.
With the rapid construction of skynet engineering, a large amount of traffic monitoring videos can be acquired every day through cameras deployed on traffic roads. The traffic monitoring data is a traffic information set extracted from the traffic monitoring video and storage information of effective pictures extracted from the traffic monitoring video. With the collection of mass traffic monitoring videos, traffic monitoring data is increased, and how to retrieve traffic monitoring data required by users, such as police, from the mass traffic monitoring data is a problem to be solved urgently today.
In view of this, the retrieval system of the present disclosure is proposed, and further the retrieval method of the traffic monitoring data of the present disclosure is proposed based on the data stored in the index database and the full database of the retrieval system.
The client is the client of the retrieval system, and the user configures the contents such as query conditions and the like in the client to further initiate a query request to the server of the retrieval system. And after receiving the query request initiated by the client, the server retrieves the traffic monitoring data according to the query request. The client may be an application client or a web page client, and is not limited in this respect.
Step S130, according to the query condition, determining index data matched with the query condition in the index database, and obtaining a monitoring data identifier associated with the determined index data, wherein the index data is obtained by performing data extraction on each retrieval field according to the retrieval fields configured for the index database and the structured attribute data of the traffic monitoring video.
In the retrieval system of the present disclosure, an index database and a full database are deployed, wherein the index database is used for storing index data obtained according to structured attribute data of a traffic monitoring video and a monitoring data identifier generated for the structured attribute data in an associated manner.
The retrieval fields configured in the index database are convenient for a client user to input corresponding retrieval words under the retrieval fields configured correspondingly for retrieval, and the configured retrieval fields include, for example, a license plate character field configured for each license plate character in a license plate, a vehicle type field configured for a vehicle type, a vehicle color field configured for a vehicle color, a brand field configured for a vehicle brand, a yearly money field configured for a vehicle, a safety belt field representing whether a person in the vehicle fastens a safety belt, and the like. In a specific embodiment, more types of retrieval fields can be configured according to an actual query scenario, so that a client user can conveniently retrieve traffic monitoring data.
The structured attribute data of the traffic monitoring video refers to data obtained by carrying out structured processing on the acquired traffic monitoring video. The method comprises the following steps of carrying out structural processing on the traffic monitoring video, for example, extracting acquisition information of the traffic monitoring video, such as acquisition time and acquisition place; identifying vehicles in the traffic monitoring video, obtaining vehicle information of the vehicles, such as identifying license plate numbers of the vehicles in the traffic monitoring video, identifying vehicle types of the vehicles in the traffic monitoring video, identifying whether people in the vehicles wear safety belts or not, identifying colors of the vehicles, identifying annual check marks attached to the vehicles, identifying annual money of the vehicles, identifying brands of the vehicles, identifying lane numbers where the vehicles are located, identifying and determining vehicle speeds of the vehicles, determining longitude and latitude where the vehicles are located, identifying whether sun visors of the vehicles are opened or not, and identifying whether the vehicles run red lights or not in the video; and obtaining the storage position information of the effective pictures according to the effective pictures extracted and stored from the traffic monitoring video. In a specific embodiment, more multidimensional information can be extracted from the traffic monitoring video according to actual needs. In other words, in different application scenarios, the structured attribute data may be data with more or less dimensions than the above example, specifically, effective data required for corresponding extraction from the traffic monitoring video according to the actual application scenario.
If the structured attribute data includes information extracted from the traffic monitoring video, and each information is identified by a field, called a vehicle passing attribute field, the vehicle passing attribute field configured for the structured processing of the traffic monitoring video is more than the retrieval field configured in the index database, so that the structured attribute data of the traffic monitoring video contains more dimensions of information than the index data of the traffic monitoring video.
The index data of each traffic monitoring video is obtained by extracting the attribute value of each retrieval field from the structured attribute data of the traffic monitoring video according to the retrieval fields configured in the index database, namely data extraction, so that the extracted set of the attribute values of all the retrieval fields forms the index data of the traffic monitoring video.
The monitoring data identification is structured attribute data used for uniquely identifying the corresponding traffic monitoring video.
The index database of the retrieval system disclosed by the invention not only stores the index data of the traffic monitoring video, but also stores the monitoring data identifier generated for the structured attribute data corresponding to the traffic monitoring video, and the index data and the monitoring data identifier are stored in the index database in a correlation manner.
Therefore, when the query is performed according to the query condition in the query request initiated by the client, the index data matched with the query condition is determined first, and then the monitoring data identifier corresponding to the index data, namely associated with the index data, is obtained based on the determined index data.
Step S150, traffic monitoring data stored by taking the monitoring data identifier as a main key is obtained in the full database, and the traffic monitoring data is obtained by taking the monitoring data identifier as a main key and performing full storage on corresponding structured attribute data in the full database.
The full database is used for storing the structural attribute data in full, the structural attribute data stored in full are used as traffic monitoring data of the corresponding traffic monitoring video, and the traffic monitoring data in the full database are stored by taking the corresponding monitoring data identifier as a main key. In other words, after the monitoring data identifier is obtained, the traffic monitoring data with the monitoring data identifier as the main key may be correspondingly obtained according to the monitoring data identifier.
And step S170, returning the acquired traffic monitoring data to the client.
Therefore, in the process of retrieving the traffic monitoring data, matched index data is determined through query conditions, monitoring data identification related to the index data is further obtained, and finally corresponding traffic monitoring data is correspondingly obtained from the full-scale database according to the obtained monitoring data identification.
Because the matching is only carried out in the index database storing the index data and the monitoring data identification according to the query condition, but not in the full database storing the traffic monitoring data, and compared with the full database, the data amount stored in the index database is less, the speed of matching in the index data according to the query condition is greatly improved compared with the speed of matching in the full database according to the query condition, thereby ensuring that a user can quickly retrieve the required traffic monitoring data in a retrieval system. The data return rate of the traffic monitoring data is greatly improved.
In the prior art, a mode of searching traffic monitoring data by pre-establishing a multidimensional cubic model (CUBE) exists, that is, the multidimensional CUBE model is established by periodically performing batch preprocessing calculation on data and is stored in a database. And when the user queries, selecting the corresponding CUBE query according to different combination conditions and returning a result. However, the method is suitable for a scene with fixed query conditions, if a query dimension is added, the original CUBE model cannot be supported, and the method can provide the query only after the CUBE model is pre-established in batches, so that the query delay exists, and the user experience is poor.
In the prior art, there is also a method of storing traffic monitoring data in a database, for example, HBase, and establishing a plurality of secondary indexes in the database, that is, establishing a plurality of secondary indexes in the database according to specific scenarios. When a client side initiates a query request, different indexes are selected in the database according to query conditions, and then data in a list in the database are returned according to index values. In this way, free combination query of any dimensionality cannot be achieved, the number of the secondary indexes is exponentially increased along with the query dimensionality, the complexity of the system is high, and the method is not suitable for complex and changeable service scenes in the traffic field.
According to the technical scheme, the index data and the traffic monitoring data are separated in the database structure layer, namely the index data are stored in the index database, and the traffic monitoring data are stored in the full database. On one hand, the matching is only carried out in the index database with less stored data according to the query conditions, and the traffic monitoring data is only required to be correspondingly obtained from the full database according to the obtained monitoring data identifier, so that the return rate of the traffic monitoring data during retrieval is greatly improved. On the other hand, since the index database is configured with the search fields having multiple dimensions, the search fields configured at the time of search can be freely combined. Therefore, the user can use the search field free combination query condition in the index database to realize multi-angle and multi-granularity free combination query according to the requirement, and the real-time quick query can be carried out after the data is stored in the index database and the full database.
In an embodiment, the query condition includes a character string input for performing traffic monitoring data retrieval through the license plate number, the retrieval field includes a license plate character field configured for each license plate character in the license plate number, as shown in fig. 4, and before step S130, the method further includes:
step S121, performing character segmentation on the character string to obtain a plurality of license plate characters included in the target license plate number.
And step S122, assigning values to corresponding license plate character fields according to the positions of the license plate characters in the target license plate number.
And S123, performing logic AND operation on each assigned license plate character field to obtain a spliced character string.
And step S124, replacing the character strings in the query conditions by splicing the character strings, so as to query in the index database by the replaced query conditions.
In the index database, a license plate character field is configured for each license plate character in the license plate number. For example, the number plate number of china is 7, and a plate character field is correspondingly configured for each plate character: p1, p2, p3, p4, p5, p6, p7, for example the license plate number: when storing the license plate number, lub 12345 stores the license plate number as { p1= lu, p2= B, p3=1, p4=2, p5=3, p6=4, and p7=5 }.
When inputting a character string in a license plate number for the purpose of traffic monitoring data retrieval by the license plate number, for example, inputting: and LuB 1, performing character segmentation according to the step S131 to obtain license plate characters included in the license plate number to be searched: lu, B, 1; according to the input character string, the first, the second and the third license plate characters are respectively as the first, the second and the third license plate characters, so that according to the position of the license plate character in the target license plate number, the license plate character field of the corresponding position is correspondingly assigned and is logically ANDed to obtain a spliced character string: { p1= shanand p2= Band p3=1}. Therefore, when the query is carried out in the index database according to the query condition, the query is carried out according to the spliced character strings, the fuzzy query is converted into the accurate query, and the query performance and the return speed of result data are greatly improved. In a specific embodiment, tests show that the speed of the mode of converting the fuzzy inquiry of the license plate number into the accurate inquiry of the license plate number is about 10 times faster than that of the conventional mode of carrying out the fuzzy inquiry on the license plate number.
In an embodiment, the query request includes the page number and the number of displays per page requested to be displayed by the client, the monitoring data identifier indicates the generation time of the monitoring data identifier, as shown in fig. 5, and after step S130, the method further includes:
step S141, sorting the obtained monitoring data identifiers according to the time information indicated by the obtained monitoring data identifiers, and obtaining a serial number of each monitoring data identifier in the sorting. And
and step S142, determining a sequence number interval of the monitoring data identification required to be acquired for the query request according to the page number and the display quantity of each page.
And step S143, acquiring the monitoring data identifier of the sequence number in the determined sequence number interval.
In this embodiment, step S150 includes:
step S151, obtaining the traffic monitoring data with the obtained monitoring data identifier as the main key in the full database as the traffic monitoring data displayed in the page indicated by the page code.
After a client sends a query request to a server based on a query condition, the server queries according to the query condition to obtain traffic monitoring data meeting the query condition, and then the traffic monitoring data is limited by the number of displays on each page in the client or by bandwidth limitation in a network.
Thus, before returning the acquired traffic monitoring data to the client, paging is performed according to the display number per page in the query request. Specifically, since the monitoring data identifier is obtained from the index database, and the traffic monitoring data using the monitoring data identifier as the main key is correspondingly obtained from the full database, in this embodiment, the monitoring data identifier associated with the index data matching the query condition in the slave index database is paged.
In the process of carrying out initial query according to query conditions, the queried and obtained monitoring data identifications are sequenced through self-generated time information indicated by the monitoring data identifications, namely the time information indicates the generation time of the monitoring data identifications, and the data ordering is ensured.
In a specific embodiment, a general algorithm for generating a GUID (global Unique Identifier) is improved, after the improvement, 10 bytes in the GUID are reserved, and another 6 bytes are used for identifying the generation time of the GUID, so as to obtain the monitoring data identification.
Sequencing the obtained monitoring data identifications based on the indicated self-generation time to obtain the sequence number of each monitoring data identification in the sequencing, namely sequentially configuring the sequence number starting from 1 for each monitoring data identification correspondingly according to the sequencing: 1,2,3, \ 8230; \ 8230;.
Therefore, the server side of the retrieval system pages the obtained monitoring data identifiers according to the sequence numbers of the sorted monitoring data identifiers and the display number of each page in the query request, and the paging is performed, namely, the sequence number interval of the monitoring data identifiers requested to be obtained is determined for the query request according to the display number of each page in the query request and the paging number, so that the monitoring data identifiers with the sequence numbers in the sequence number interval are correspondingly obtained from the sorting. For example, if the paging number in the query request is the third page and the display number of each page is 50, the sequence number interval for acquiring the monitoring data identifier requested by the query request is [101, 150].
And correspondingly acquiring the traffic monitoring data with the monitoring data identifier in the sequence number interval as a main key from the full database, and returning the traffic monitoring data to the client as the traffic monitoring data displayed in the page indicated by the page number in the query request. When the client needs to request the traffic monitoring data with the monitoring data identifiers with the sequence numbers in other sequence number intervals as the main keys, the client user initiates an inquiry request again through the client, correspondingly obtains the monitoring data identifiers in the sequence number intervals indicated by the paging codes according to the paging numbers and the display quantity of each page in the initiated inquiry request again, and correspondingly obtains the traffic monitoring data displayed in the paging indicated by the paging codes and returns the traffic monitoring data to the client.
By the technical scheme of the embodiment, the paging of the retrieval result on the server side of the retrieval system is realized. Compared with the prior art, the method has the advantages that the retrieval results of the retrieval are simultaneously returned to the client, and the client performs paging according to needs, so that the data transmission is ensured not to occupy too much network bandwidth, the terminal where the client is located is not required to cache all the retrieval results, and the cache pressure of the client is reduced. Especially, under the condition of more query results, if all the search results are returned to the client at the same time, the bandwidth occupied by data transmission and the read-write I/O of the terminal disk where the client is located become the bottleneck of the search system, and the memory overflow of the terminal where the client is located may be caused more seriously.
In an embodiment, as shown in fig. 6, before step S143, the method further includes:
step S210, writing the monitoring data identifiers in the sequence into the buffer queue according to the preset buffer amount of the buffer queue.
In this embodiment, step S143 includes:
step S220, acquiring the monitoring data identifier of the sequence number in the determined sequence number interval in the buffer queue.
In this embodiment, the monitoring data identifications obtained from the queries in the index database are buffered by the configured buffer queue. The number of buffers is preset for the buffer queue, so that when the monitoring data identifier obtained by querying is written into the buffer queue configured by the query condition for the first time, the monitoring data identifiers corresponding to the number of buffers are written into the buffer queue in sequence according to the preset number of buffers. For example, the number of buffers is 500, and the first 500 monitor data identifiers in the sequence are written into the buffer queue.
Therefore, in order to acquire the traffic monitoring data with the monitoring data identifier as the main key in the full database, the monitoring data identifier in the determined sequence number interval is correspondingly acquired from the buffer queue.
In an embodiment, as shown in fig. 7, before step S220, the method further includes:
step S310, judging whether the product of the page number and the display number of each page is less than the cache number.
If yes, go to step S320: and adjusting the cache number according to the page number and the display number of each page, and writing the monitoring data identifications of the corresponding number in the sequence into a cache queue according to the adjusted cache number, wherein the adjusted cache number is not less than the product.
If not, go to step S220: and acquiring the monitoring data identifier of the sequence number in the determined sequence number interval in the buffer queue.
If the product of the paging code and the display number of each page in the query request is less than the cache number, indicating that the monitoring data identifier cached in the cache queue cannot meet the display requirement in the page indicated by the paging code in the query request; otherwise, if the number of the sequence numbers is not smaller than the preset value, it indicates that the monitoring data identifier cached in the cache queue can meet the display requirement in the page indicated by the page number in the query request, and the monitoring data identifier with the sequence number in the determined sequence number interval can be directly obtained from the cache queue.
Therefore, when the product of the paging code and the display number of each page is smaller than the cache number, the cache number needs to be adjusted so that the adjusted cache number is not smaller than the product of the paging code and the display number of each page, and the monitoring data identifiers which are ordered into the adjusted cache number are written into the cache queue from the ordering according to the adjusted cache number.
In an embodiment, when the monitoring data identifier in the sorting is written into the buffer queue for the first time, the query identifier generated according to the query condition is simultaneously stored in the buffer queue, as shown in fig. 8, before step S310, the method further includes:
step S410, determining whether there is a query identifier generated according to the query condition in the buffer queue.
If so, go to step S310.
If not, go to step S130.
If the query identifier generated according to the query condition exists in the cache queue, it indicates that the monitoring data identifier obtained according to the query condition has been stored in the cache queue, for example, after the traffic monitoring data in the first page is returned to the client after the first query according to the query condition, the client user initiates a query request for requesting the traffic monitoring data in the second page to the server based on the same query condition, so that the step S310 is directly executed without performing a second query in the index database according to the query condition again.
On the contrary, if the query identifier generated according to the query condition does not exist in the cache queue, it indicates that the monitoring data identifier obtained according to the query condition is not stored in the cache queue, and further indicates that the server does not perform the query in the index database according to the query condition, so that step S130 needs to be performed to perform the query in the index database according to the query condition.
In one embodiment, step S320 includes:
the number of buffers is adjusted to: (product of paging code and display number per page/buffer number + 1) × buffer number.
The adjusted cache number can be ensured not to be less than the product of the number of pages in the query request and the display number of each page through adjustment.
In a specific embodiment, when the monitoring data identifier obtained according to the query condition is written into the buffer queue for the first time, the total number of the obtained monitoring data identifiers is written into the buffer queue. Before the cache number is adjusted to (the product of the paging code and the display number of each page/the cache number + 1) × the cache number, judging: (product of paging code and display quantity per page/cache quantity + 1) whether the cache quantity does not exceed the total quantity, if so, adjusting the cache quantity to the total quantity; otherwise, if not, the buffer amount is adjusted to: (product of paging code and display number per page/buffer number + 1) × buffer number.
FIG. 9 illustrates a flow diagram for retrieving monitoring data identifiers from an index database in one embodiment, which is a managed database of the retrieval system. As shown in fig. 9, upon receiving a query request,
1. calling the drive-client;
2. generating a query identifier key according to a query condition in the query request;
3. judging whether the query identifier key exists in a cache queue redis; if the query identifier key exists in the cache queue redis, turning to 6; if the query identifier key does not exist in the cache queue redis, turning to 4;
4. entering an index database to be queried according to the query condition;
5. according to the preset cache quantity, the monitoring data identifications with the preset cache quantity in the sequence are written into a cache queue redis;
6. judging whether the product of the paging code in the query request and the display number of each page is larger than the cache number, if so, turning to 9; if not, turning to 7;
7. taking out corresponding paging data from the buffer queue, namely monitoring data identification with a sequence number positioned in a sequence number interval determined according to the paging number and the display quantity of each page;
8. acquiring traffic monitoring data with a monitoring data identifier as a main key from a full database;
9. and calling the Druid query to adjust the cache number to be: (product of paging code and display number per page/buffer number + 1) × buffer number;
10. extracting monitoring data identifications of the number of the buffers after the previous adjustment in the sequencing according to the number of the buffers after the adjustment and writing the monitoring data identifications into a buffer queue redis; and then 7.
In an embodiment, as shown in fig. 10, the method for retrieving traffic monitoring data further includes:
step S510, obtaining structured data obtained by performing structured processing on the traffic monitoring video collected in real time, where the structured data indicates collection information of the traffic monitoring video, vehicle information of vehicles in the traffic monitoring video, and storage location information of effective pictures extracted from the traffic monitoring video.
And step S520, verifying the structured data, and taking the verified structured data as the structured attribute data of the traffic monitoring video.
Step S530, according to the retrieval fields configured for the index database, data extraction is carried out on the retrieval fields from the structured attribute data, and the index data of the traffic monitoring video is obtained. And
and step S540, taking the structured attribute data as traffic monitoring data of the traffic monitoring video.
And step S550, storing the index data and the monitoring data identifier generated for the structured attribute data in an index database in an associated manner, and storing the traffic monitoring data in a full database by taking the monitoring data identifier as a main key.
As described above, the index database in the retrieval system is used for storing the index data of the traffic monitoring video and the monitoring data identifier in an associated manner, and the full database is used for storing the traffic monitoring data with the monitoring data identifier as a main key. The storage of the index data of the traffic monitoring video and the traffic monitoring data in the retrieval system is realized through steps S510-550.
In step S520, the integrity of the data in the structured data is checked, including checking the data value and checking the attribute field. The checking of the data values checks, for example, whether or not there is data in the structured data for which the required value cannot be empty. For example, if the values of license plate number, acquisition place and acquisition time cannot be empty, whether the license plate number, the acquisition place and the acquisition time in the structured data are empty or not needs to be checked, and if the license plate number, the acquisition place and the acquisition time are empty, the license plate number, the acquisition place and the acquisition time are discarded; if not, the subsequent steps are carried out. Checking the attribute field, for example, whether the license plate number accords with the license plate number rule, and whether the acquisition time is in a set time interval, thereby discarding the data which do not meet the requirements;
secondly, the consistency of the data is checked, invalid data in the structured data is eliminated, meanwhile, a default missing value is set, data support is provided for the accuracy of subsequent query, the problem that the attribute in the data is not matched with an attribute dictionary (namely a value range set by the attribute) is avoided, for example, five dictionary values (0, 1,2,3 and 4) of license plate colors exist, the value of the vehicle color in the structured data is-1, so that the fact that-1 does not exist in the dictionary value can be judged, the-1 is converted into the default value 0, and the problem that the display is abnormal due to the fact that the value of the vehicle color in the result of the subsequent query is-1 can be avoided.
And obtaining the structured attribute data of the traffic monitoring video through verification. Correspondingly, data extraction is carried out on the retrieval fields from the structured attribute data to obtain index data of the traffic monitoring video, and the structured attribute data is used as the traffic monitoring data of the traffic monitoring video. And stored in the manner of step S550.
In one embodiment, the index data includes index data of license plate numbers, the vehicle information includes license plate numbers of vehicles, the search field includes a license plate character field configured for characters of each character position in the license plate numbers, as shown in fig. 11, the method further includes:
and step S610, performing character segmentation on the license plate number in the structured attribute data according to the position sequence in the license plate number to obtain license plate characters at each character position in the license plate number.
And step S620, assigning values to license plate character fields corresponding to the character positions according to the character positions of the license plate characters in the license plate number to obtain license plate number index data.
The license plate number is divided into license plate characters through the steps S610-620, and the license plate characters are stored, so that the fuzzy condition splicing is conveniently carried out if the query conditions comprise the character strings input for license plate number cable, and the fuzzy search is converted into the accurate search.
FIG. 12 is a flowchart illustrating the storing of data in a retrieval system according to an embodiment in which the index database of the retrieval system is a Druid database and the full database is an HBase database, wherein a kafka distributed publish-subscribe message system is further configured in the retrieval system for processing action flow data in storing data in the retrieval system; and configuring a batch-processed streaming computing framework spark streaming for processing the pushed structured data in real time.
As shown in fig. 12, the retrieval system obtains the structured data of the traffic monitoring video from the structured extraction server in a message publishing manner, i.e., a message publishing (TOPIC 1), then kafka pushes the obtained structured data into spark streaming in a message subscribing manner, i.e., a message subscribing (TOPIC 1), and the spark streaming processes the structured data in real time, wherein the processing includes checking the structured data, splitting the license plate number in the structured data to obtain the structured attribute data, generating the monitoring data identifier for the structured attribute data, and extracting the data from the structured attribute data to obtain the index data of the traffic monitoring video for the retrieval field.
Thus, on one hand, the structured attribute data is stored in the HBase database as the whole traffic monitoring data with the monitoring data identification as the primary key. In the HBase database, traffic monitoring data is stored in a single column. Meanwhile, in order to improve the retrieval performance, the HBase stores data in a mode of dividing a table according to months and pre-establishing partitions.
On the other hand, the obtained index data and the obtained monitoring data identification are pushed into kafka in a message publishing mode of kafka, namely message publishing (TOPIC 2), so that the index data and the monitoring data identification are pushed into the real-time processing flow kafka-index-service in a message subscribing mode of kafka, namely message subscribing (TOPIC 2), and the real-time processing flow kafka-index-service stores the index data and the monitoring data identification in a drift association mode.
The storage position information of the effective pictures in the structured attribute data indicates the storage positions of the effective pictures in the picture server, such as URLs of the effective pictures.
In the HBase, a monitoring data identifier and traffic monitoring data are associated through a row key rowkey, namely the traffic monitoring data is stored by taking the monitoring data identifier as a main key. In a specific embodiment, a hash strategy is adopted during rowkey design to ensure that data are uniformly distributed in different storage areas, so that the time for acquiring traffic monitoring data from HBase through rowkey is in millisecond level, and the query performance is ensured.
In order to improve the index data query efficiency, the index data is subjected to fragmentation processing according to the time according to the monitoring data identification by combining the service scene of the traffic field and the characteristic that all queries are based on the time period. When the Druid accesses data in real time, firstly, segmenting according to time to generate Segment, adopting an LSM-tree model, firstly, adding the Segment into an increment index of a memory, wherein the increment index of the memory adopts SkiList; when a certain threshold value is reached, the threshold value can be that the memory increment reaches the maximum number, or the memory increment is accumulated to a certain time, then the memory increment index is converted into the inverted index by adopting an asynchronous thread and is durably stored in a disk, and meanwhile, a new memory increment index is generated to receive data. In cycles, when the set Segment Granularity is reached, all persistent indexes within those Segment Granularity are merged into one Segment and pushed to the Deep Storage.
The following is an embodiment of the apparatus of the present disclosure, which may be used to execute an embodiment of a method for retrieving traffic monitoring data executed by the server 102 of the above-mentioned retrieval system of the present disclosure. For details not disclosed in the embodiments of the disclosed device, please refer to the embodiments of the traffic monitoring data retrieval method disclosed in the present disclosure.
Fig. 13 is a block diagram illustrating a traffic monitoring data retrieval apparatus according to an exemplary embodiment, the traffic monitoring data retrieval apparatus is applied to a server side of a retrieval system, the retrieval system is provided with an index database and a full database, and the traffic monitoring data retrieval apparatus includes:
a query request receiving module 110 configured to: and receiving a query request initiated by the client, wherein the query request comprises query conditions configured for querying traffic monitoring data.
A monitoring data identity obtaining module 130 configured to: the method comprises the steps of inquiring and determining index data matched with inquiry conditions in an index database according to the inquiry conditions, and obtaining monitoring data identifications associated with the determined index data, wherein the index data are obtained by performing data extraction on each retrieval field according to the retrieval field configured for the index database and structured attribute data obtained by performing structured processing on each traffic monitoring video acquired in real time, and the index database is used for storing the index data corresponding to each structured attribute data and the monitoring data identifications used for identifying the structured attribute data in an associated manner.
A traffic monitoring data acquisition module 150 configured to: and acquiring traffic monitoring data stored by taking the monitoring data identifier as a main key in a full database, wherein the traffic monitoring data is obtained by taking the monitoring data identifier as a main key in the full database and performing full storage on corresponding structured attribute data.
A return module 170 configured to: and returning the acquired traffic monitoring data to the client.
The implementation process of the functions and actions of each module in the above device is detailed in the implementation process of the corresponding step in the above method for retrieving traffic monitoring data, and is not described in detail here.
It is understood that these modules may be implemented in hardware, software, or a combination of both. When implemented in hardware, these modules may be implemented as one or more hardware modules, such as one or more application specific integrated circuits. When implemented in software, the modules may be implemented as one or more computer programs executing on one or more processors, such as programs stored in memory 250 for execution by central processor 270 of FIG. 2.
In one embodiment, the query condition includes a character string input for searching the traffic monitoring data through the license plate number, the search field includes a license plate character field configured for each license plate character in the license plate number, and the search device for the traffic monitoring data further includes:
a character segmentation unit configured to: and performing character segmentation on the character string to obtain a plurality of license plate characters included in the target license plate number.
An assignment unit configured to: and assigning values to the corresponding license plate character fields according to the positions of the license plate characters in the target license plate number.
A logical operation unit configured to: and performing logic AND operation on each assigned license plate character field to obtain a spliced character string.
A replacement unit configured to: and replacing character strings in the query conditions by splicing the character strings so as to query in the index database through the replaced query conditions.
In one embodiment, the query request includes the paging code and the number of displays per page requested to be displayed by the client, the monitoring data identifier indicates the generation time of the monitoring data identifier, and the retrieving device of the traffic monitoring data further includes:
a ranking module configured to: and sequencing the obtained monitoring data identifications according to the time information indicated by the monitoring data identifications, and obtaining the serial number of each monitoring data identification in the sequencing. And
a sequence number interval determination module configured to: and determining a sequence number interval of the monitoring data identification required to be acquired for the query request according to the page number and the display quantity of each page.
A monitoring data identification acquisition module configured to: and acquiring the monitoring data identifier of the sequence number in the determined sequence number interval.
The traffic monitoring data acquisition module 150 includes:
a traffic monitoring data acquisition unit configured to: and acquiring the traffic monitoring data with the acquired monitoring data identifier as a main key in the full database, wherein the traffic monitoring data is used as the traffic monitoring data displayed in the page indicated by the page code.
In one embodiment, the retrieving device of traffic monitoring data further comprises:
a writing unit configured to: and writing the monitoring data identifiers in the sequence into the buffer queue according to the preset buffer number of the buffer queue.
The monitoring data identification acquisition module comprises:
a monitoring data identification acquisition unit configured to: and acquiring the monitoring data identifier of the sequence number in the determined sequence number interval in the buffer queue.
In one embodiment, the retrieving device of traffic monitoring data further comprises:
a first determination module configured to: and judging whether the product of the page number and the display number of each page is less than the cache number.
A buffer number adjustment module configured to: if the first judging module judges that the product of the page number and the display quantity of each page is smaller than the cache quantity, the cache quantity is adjusted according to the page number and the display quantity of each page, so that the monitoring data identifications of the corresponding quantity in the sequence are written into the cache queue according to the adjusted cache quantity, and the adjusted cache quantity is not smaller than the product.
A first hopping module configured to: and if the first judging module judges that the product of the page number and the display quantity of each page is not less than the cache quantity, jumping to a monitoring data identification acquisition unit.
In an embodiment, when the monitoring data identifier in the sorting is written into the buffer queue for the first time, the query identifier generated according to the query condition is stored in the buffer queue at the same time, and the apparatus further includes:
a second determination module configured to: and judging whether the cache queue has a query identifier generated according to the query condition.
A second skip module configured to: and if the second judging module judges that the query identifier generated according to the query condition exists in the cache queue, skipping to the first judging module.
The second skipping module is configured to skip to the monitoring data identifier obtaining module 130 if the second determining module determines that the query identifier generated according to the query condition does not exist in the cache queue.
In one embodiment, the buffer amount adjusting module includes:
a buffer number adjustment unit configured to: the number of buffers is adjusted to: (product of paging code and display number per page/buffer number + 1) × buffer number.
In one embodiment, the retrieving device of traffic monitoring data further comprises:
a structured data acquisition module configured to: the method comprises the steps of obtaining structured data obtained by carrying out structured processing on traffic monitoring videos collected in real time, wherein the structured data indicate collected information of the traffic monitoring videos, vehicle information of vehicles in the traffic monitoring videos and storage position information of effective pictures extracted from the traffic monitoring videos.
A verification module configured to: and checking the structured data, and taking the checked structured data as the structured attribute data of the traffic monitoring video.
An index data obtaining module configured to: and according to the retrieval fields configured for the index database, performing data extraction on the retrieval fields from the structured attribute data to obtain the index data of the traffic monitoring video. And
a traffic monitoring data acquisition module configured to: and taking the structured attribute data as traffic monitoring data of the traffic monitoring video.
A storage module configured to: and storing the index data and the monitoring data identifier generated for the structured attribute data in an index database in an associated manner, and storing the traffic monitoring data in a full database by taking the monitoring data identifier as a main key.
In one embodiment, the index data includes license plate number index data, the vehicle information includes license plate numbers of vehicles, the search field includes license plate character fields configured for characters of each character position in the license plate numbers, and the search device for traffic monitoring data further includes:
a second segmentation module configured to: and performing character segmentation on the license plate number in the structured attribute data according to the position sequence in the license plate number to obtain license plate characters at each character position in the license plate number.
A license plate number index data obtaining module configured to: and assigning a value to the license plate character field corresponding to the character position according to the character position of the license plate character in the license plate number to obtain license plate number index data.
The implementation process of the functions and actions of each module/unit in the above device is specifically described in the implementation process of the corresponding step in the above method for retrieving traffic monitoring data, and is not described herein again.
Optionally, the present invention further provides a device for retrieving traffic monitoring data, where the device for retrieving traffic monitoring data may be configured in the server 102 of the retrieval system in the implementation environment shown in fig. 1, and execute all or part of the steps of the method for retrieving traffic monitoring data shown in any method embodiment above. As shown in fig. 14, the retrieving device 1000 of traffic monitoring data includes but is not limited to: a processor 1001 and a memory 1002.
Wherein the memory 1002 has stored thereon computer readable instructions which, when executed by the processor 1001, implement the method of any one of the above method implementations.
Wherein the executable instructions, when executed by the processor 1001, implement the method in any of the above embodiments. Such as computer readable instructions, which when executed by the processor 1001, read stored in the memory via the communication line/bus 1003 connected to the memory.
The specific manner in which the processor in this embodiment performs the operations has been described in detail in the embodiment of the method for retrieving traffic monitoring data, and will not be elaborated upon here.
In an exemplary embodiment, a storage medium is also provided, which is a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of the above method embodiments. Such as, but not limited to, a memory of instructions executable by the central processor of the server 200 to perform the method of retrieving the traffic monitoring data.
The specific manner in which the processor in this embodiment performs the operations has been described in detail in the embodiment of the method for retrieving traffic monitoring data, and will not be elaborated upon here.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims (9)

1. A retrieval method of traffic monitoring data is applied to a server side of a retrieval system, and is characterized in that the retrieval system is provided with an index database and a full database, and the method comprises the following steps:
acquiring structured data obtained by carrying out structured processing on a traffic monitoring video acquired in real time, wherein the structured data indicates acquisition information of the traffic monitoring video, vehicle information of vehicles in the traffic monitoring video and storage position information of effective pictures extracted from the traffic monitoring video;
verifying the structured data, and taking the verified structured data as the structured attribute data of the traffic monitoring video;
according to the retrieval fields configured for the index database, performing data extraction on the retrieval fields from the structured attribute data to obtain index data of the traffic monitoring video; and
taking the structured attribute data as traffic monitoring data of the traffic monitoring video;
storing the index data and the monitoring data identifier generated for the structured attribute data in the index database in an associated manner, and storing the traffic monitoring data in the full database by taking the monitoring data identifier as a primary key;
receiving a query request initiated by a client, wherein the query request comprises query conditions configured for querying traffic monitoring data;
according to the query condition, searching and determining index data matched with the query condition in the index database, and obtaining a monitoring data identifier associated with the determined index data, wherein the index data is obtained by performing data extraction on each retrieval field according to the retrieval fields configured for the index database and according to the structural attribute data of the traffic monitoring video;
acquiring traffic monitoring data stored by taking the monitoring data identifier as a primary key in the full database, wherein the traffic monitoring data is obtained by taking the monitoring data identifier as a primary key in the full database and performing full storage on the corresponding structured attribute data;
and returning the acquired traffic monitoring data to the client.
2. The method of claim 1, wherein the query condition comprises a character string input for searching traffic monitoring data through license plate numbers, the search field comprises a license plate character field configured for each license plate character in license plate numbers, and before searching the index database for index data matching the query condition according to the query condition and obtaining the monitoring data identification associated with the determined index data, the method further comprises:
performing character segmentation on the character string to obtain a plurality of license plate characters included in the target license plate number;
assigning values to corresponding license plate character fields according to the positions of the license plate characters in the target license plate number;
performing logic and operation on each assigned license plate character field to obtain a spliced character string;
and replacing the character strings in the query conditions by the spliced character strings so as to query in the index database by the replaced query conditions.
3. The method according to claim 1, wherein the query request includes a paging code and a display number per page requested to be displayed by the client, the monitoring data identifier indicates a generation time of the monitoring data identifier, the index database is queried according to the query condition to determine index data matching the query condition, and after obtaining the monitoring data identifier associated with the determined index data, the method further includes:
sequencing the obtained monitoring data identifications according to the time information indicated by the self, and obtaining the serial number of each monitoring data identification in sequencing; and
determining a sequence number interval of the monitoring data identification required to be acquired for the query request according to the paging number and the display quantity of each page;
acquiring a monitoring data identifier of a sequence number in the determined sequence number interval;
the acquiring of the traffic monitoring data stored by taking the monitoring data identifier as a primary key in the full database comprises:
and acquiring the traffic monitoring data with the acquired monitoring data identifier as a main key from the full database, and taking the traffic monitoring data as the traffic monitoring data displayed in the page indicated by the page code.
4. The method of claim 3, wherein the obtaining the sequence number precedes the monitoring data identification in the determined sequence number interval, the method further comprising:
writing the monitoring data identifiers in the sequence into the cache queue according to the preset cache number of the cache queue;
the acquiring the monitoring data identifier of the sequence number in the determined sequence number interval includes:
and acquiring the monitoring data identifier of the sequence number in the determined sequence number interval in the buffer queue.
5. The method of claim 4, wherein before obtaining the monitoring data identifier of the sequence number in the determined sequence number interval in the buffer queue, the method further comprises:
judging whether the product of the page number and the display quantity of each page is smaller than the cache quantity or not;
if the number of the monitoring data marks is less than the product, the cache number is adjusted according to the page number and the display number of each page, so that the monitoring data marks with the corresponding number in the sequence are written into the cache queue according to the adjusted cache number, and the adjusted cache number is not less than the product;
and if not, executing the monitoring data identifier of the sequence number in the determined sequence number interval acquired in the cache queue.
6. The method according to claim 5, wherein when the monitoring data identifier in the sequence is written into the buffer queue for the first time, the query identifier generated according to the query condition is simultaneously stored into the buffer queue, and before the product of the page number and the display number per page is judged to be smaller than the buffer number, the method further comprises:
judging whether a query identifier generated according to the query condition exists in the cache queue or not;
if so, executing the step of judging whether the product of the page number and the display quantity of each page is smaller than the cache quantity;
if not, the step of inquiring and determining the index data matched with the inquiry condition in the index database according to the inquiry condition and obtaining the monitoring data identification associated with the determined index data is executed.
7. The method of claim 5, wherein the adjusting the amount of buffering according to the page number and the display amount per page comprises:
adjusting the buffer amount as: (product of the page number and the display number per page/the number of caches + 1) × the number of caches.
8. The method of claim 1, wherein the index data comprises license plate number index data, wherein the vehicle information comprises a license plate number of a vehicle, wherein the search field comprises a license plate character field configured for characters at each character position in the license plate number, and wherein the method further comprises:
carrying out character segmentation on the license plate number in the structured attribute data according to the position sequence in the license plate number to obtain license plate characters of each character position in the license plate number;
and assigning a license plate character field corresponding to the character position according to the character position of the license plate character in the license plate number to obtain the license plate number index data.
9. A retrieval device of traffic monitoring data is applied to a server side of a retrieval system, and is characterized in that the retrieval system is provided with an index database and a full database, and the device comprises:
a structured data acquisition module configured to: acquiring structured data obtained by carrying out structured processing on a traffic monitoring video acquired in real time, wherein the structured data indicates acquisition information of the traffic monitoring video, vehicle information of vehicles in the traffic monitoring video and storage position information of effective pictures extracted from the traffic monitoring video;
a verification module configured to: verifying the structured data, and taking the verified structured data as the structured attribute data of the traffic monitoring video;
an index data obtaining module configured to: according to the retrieval fields configured for the index database, performing data extraction on the retrieval fields from the structured attribute data to obtain index data of the traffic monitoring video; and
a traffic monitoring data acquisition module configured to: taking the structured attribute data as traffic monitoring data of the traffic monitoring video;
a storage module configured to: storing the index data and the monitoring data identifier generated for the structured attribute data in the index database in an associated manner, and storing the traffic monitoring data in the full database by taking the monitoring data identifier as a primary key;
a query request receiving module configured to: receiving a query request initiated by a client, wherein the query request comprises query conditions configured for querying traffic monitoring data;
a monitoring data identification obtaining module configured to: according to the query condition, searching and determining index data matched with the query condition in the index database, and obtaining a monitoring data identifier associated with the determined index data, wherein the index data is obtained by performing data extraction on each retrieval field according to the retrieval field configured for the index database and structured attribute data obtained by performing structured processing on each traffic monitoring video acquired in real time, and the index database is used for storing the index data corresponding to each structured attribute data and the monitoring data identifier used for identifying the structured attribute data in an associated manner;
a traffic monitoring data acquisition module configured to: acquiring traffic monitoring data stored by taking the monitoring data identifier as a primary key in the full database, wherein the traffic monitoring data is obtained by taking the monitoring data identifier as a primary key in the full database and performing full storage on the corresponding structured attribute data;
a return module configured to: and returning the acquired traffic monitoring data to the client.
CN201910485817.5A 2019-06-05 2019-06-05 Retrieval method and device of traffic monitoring data Active CN110275983B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910485817.5A CN110275983B (en) 2019-06-05 2019-06-05 Retrieval method and device of traffic monitoring data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910485817.5A CN110275983B (en) 2019-06-05 2019-06-05 Retrieval method and device of traffic monitoring data

Publications (2)

Publication Number Publication Date
CN110275983A CN110275983A (en) 2019-09-24
CN110275983B true CN110275983B (en) 2022-11-22

Family

ID=67961952

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910485817.5A Active CN110275983B (en) 2019-06-05 2019-06-05 Retrieval method and device of traffic monitoring data

Country Status (1)

Country Link
CN (1) CN110275983B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112579633A (en) * 2019-09-30 2021-03-30 北京国双科技有限公司 Data retrieval method, device, equipment and storage medium
CN110851431B (en) * 2019-10-30 2022-12-20 中盈优创资讯科技有限公司 Data processing method and device for data center station
CN111405242A (en) * 2020-02-26 2020-07-10 北京大学(天津滨海)新一代信息技术研究院 Ground camera and sky moving unmanned aerial vehicle linkage analysis method and system
CN111405243A (en) * 2020-02-28 2020-07-10 北京大学(天津滨海)新一代信息技术研究院 Target monitoring method and system based on multiple cameras
CN111797094A (en) * 2020-05-25 2020-10-20 北京旷视科技有限公司 Data processing method and device and electronic equipment
CN112003956B (en) * 2020-10-27 2021-01-15 武汉中科通达高新技术股份有限公司 Traffic management system
CN112395517B (en) * 2020-11-16 2023-09-29 贝壳技术有限公司 House source searching and displaying method and device and computer readable storage medium
CN112269859B (en) * 2020-11-16 2023-05-26 国网山东省电力公司聊城供电公司 Power supply and distribution data optimization rapid processing system, equipment and readable storage medium
CN112527210A (en) * 2020-12-22 2021-03-19 南京中兴力维软件有限公司 Storage method and device of full data and computer readable storage medium
CN112667830A (en) * 2020-12-25 2021-04-16 厦门博海中天信息科技有限公司 Lightweight view storage method, device, medium and equipment

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102831174A (en) * 2012-07-27 2012-12-19 福建富士通信息软件有限公司 Method and system for rapidly checking structured information
CN103092894A (en) * 2011-11-08 2013-05-08 阿里巴巴集团控股有限公司 Structured information retrieval method and system
CN104050247A (en) * 2014-06-04 2014-09-17 上海美琦浦悦通讯科技有限公司 Method for realizing quick retrieval of mass videos
CN104216957A (en) * 2014-08-20 2014-12-17 北京奇艺世纪科技有限公司 Query system and query method for video metadata
CN105893510A (en) * 2016-03-30 2016-08-24 北京格灵深瞳信息技术有限公司 Video structurization system and target search method thereof
CN108564254A (en) * 2018-03-15 2018-09-21 国网四川省电力公司绵阳供电公司 Controller switching equipment status visualization platform based on big data
CN108595664A (en) * 2018-04-28 2018-09-28 尚谷科技(天津)有限公司 A kind of agricultural data monitoring method under hadoop environment
CN109376176A (en) * 2018-08-22 2019-02-22 阿里巴巴集团控股有限公司 A kind of data configuration method and device, a kind of calculating equipment and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2660599C1 (en) * 2017-06-01 2018-07-06 Общество с ограниченной ответственностью "Синезис" Method of video data indexing for facet classification
CN109130935B (en) * 2018-08-30 2021-08-31 广东电网有限责任公司 Intelligent charging monitoring system for electric automobile

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103092894A (en) * 2011-11-08 2013-05-08 阿里巴巴集团控股有限公司 Structured information retrieval method and system
CN102831174A (en) * 2012-07-27 2012-12-19 福建富士通信息软件有限公司 Method and system for rapidly checking structured information
CN104050247A (en) * 2014-06-04 2014-09-17 上海美琦浦悦通讯科技有限公司 Method for realizing quick retrieval of mass videos
CN104216957A (en) * 2014-08-20 2014-12-17 北京奇艺世纪科技有限公司 Query system and query method for video metadata
CN105893510A (en) * 2016-03-30 2016-08-24 北京格灵深瞳信息技术有限公司 Video structurization system and target search method thereof
CN108564254A (en) * 2018-03-15 2018-09-21 国网四川省电力公司绵阳供电公司 Controller switching equipment status visualization platform based on big data
CN108595664A (en) * 2018-04-28 2018-09-28 尚谷科技(天津)有限公司 A kind of agricultural data monitoring method under hadoop environment
CN109376176A (en) * 2018-08-22 2019-02-22 阿里巴巴集团控股有限公司 A kind of data configuration method and device, a kind of calculating equipment and storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Building a database on S3;Matthias Brantner et al.;《Proceedings of the 2008 ACM SIGMOD international conference on Management of data》;20080609;251-264 *
城市轨道交通线网数据中心与评估决策平台;张铭;《智能系统学报》;20170317;第13卷(第3期);458-468 *
基于高速公路车流量的管控系统设计与实现;林向铭;《中国优秀硕士学位论文全文数据库 工程科技II辑》;20190515(第5期);C034-499 *

Also Published As

Publication number Publication date
CN110275983A (en) 2019-09-24

Similar Documents

Publication Publication Date Title
CN110275983B (en) Retrieval method and device of traffic monitoring data
US7593981B2 (en) Detection of search behavior based associations between web sites
CN110109953B (en) Data query method, device and equipment
CN108304444B (en) Information query method and device
US8762383B2 (en) Search engine and method for image searching
US8463785B2 (en) Method and system for generating search collection of query
US20150227517A1 (en) Trend response management
CN109766318B (en) File reading method and device
US8880504B2 (en) Tag management device, system and recording medium
CN113609374A (en) Data processing method, device and equipment based on content push and storage medium
CN111552884A (en) Method and apparatus for content recommendation
CN111465932A (en) Integrating responses from queries to heterogeneous data sources
CN108154024B (en) Data retrieval method and device and electronic equipment
CN109408616A (en) Content similarities short text querying method, equipment, system and storage medium
CN110442614B (en) Metadata searching method and device, electronic equipment and storage medium
CN107291875B (en) Metadata organization management method and system based on metadata graph
CN115729965A (en) Information stream processing method, device, stream server and storage medium
CN109271580B (en) Search method, device, client and search engine
CN112231481A (en) Website classification method and device, computer equipment and storage medium
CN112269837A (en) Data processing method and device
CN113377831B (en) Resource data query method and device, electronic equipment and storage medium
CN115795184B (en) RPA-based scene get-on point recommendation method and device
CN111522941A (en) Text clustering method and device, electronic equipment and computer storage medium
CN116305288B (en) Method, device, equipment and storage medium for isolating database resources
CN116719584B (en) Data processing method, apparatus, computer, storage medium, and program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant