CN111651631A - High-concurrency video data processing method, electronic equipment, storage medium and system - Google Patents
High-concurrency video data processing method, electronic equipment, storage medium and system Download PDFInfo
- Publication number
- CN111651631A CN111651631A CN202010350998.3A CN202010350998A CN111651631A CN 111651631 A CN111651631 A CN 111651631A CN 202010350998 A CN202010350998 A CN 202010350998A CN 111651631 A CN111651631 A CN 111651631A
- Authority
- CN
- China
- Prior art keywords
- video data
- server
- cache
- data
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 46
- 238000000034 method Methods 0.000 claims abstract description 24
- 230000007246 mechanism Effects 0.000 claims abstract description 16
- 230000005540 biological transmission Effects 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 9
- 238000000638 solvent extraction Methods 0.000 claims 1
- 238000012544 monitoring process Methods 0.000 description 10
- 238000007726 management method Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 206010000117 Abnormal behaviour Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/71—Indexing; Data structures therefor; Storage structures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a high-concurrency video data processing method, which relates to the technical field of video data processing and comprises the following steps: receiving video data; based on the locking mechanism, the video data is written into the cache server and is written into the database server through the queue server. The method adopts the atomicity operation of the locking mechanism, ensures the consistency of the written data in the database server and the cache server, improves the stability of storing a large amount of video data, and solves the problem of data divergence caused by falling the data in the cache server to the database server in the isolated time. The invention also discloses an electronic device, a computer storage medium and a high-concurrency video data processing system.
Description
Technical Field
The present invention relates to the field of video data processing technologies, and in particular, to a high-concurrency video data processing method, an electronic device, a storage medium, and a system.
Background
The smart city system integrates multi-party resources such as emergency, public security traffic police, fire fighting, city management and the like by relying on various digital city technologies such as computer network, mobile communication, computer telecommunication integration, space information, grid management, city component management and the like, and realizes the exchange and sharing of various service data. The smart city system can effectively improve the city management regulation and control and emergency handling capacity of the government, and provide good digital, intelligent and humanized working and living environments for people, wherein the city safety is the guarantee of the whole smart city, the effective safety prevention means is a core ring of the whole city safety guarantee, and the video monitoring system is an indispensable part of the system.
In the project application of a video monitoring system, video data with high acquisition frequency and extremely large data volume is often received and stored, and high concurrent viewing requests for a large amount of video data are also required to be responded in real time. The existing video data processing method is characterized in that cache is added, then cache data are fallen to a database at intervals, however, the security and consistency of the data are not considered enough by the processing method, especially when high-concurrency video data are processed, once some data are lost, for example, some data in the cache are lost, the data are not fallen to the database at the moment, so that data divergence between the database and the cache can be caused, the consistency can not be ensured, and a problem can be caused when the data are recovered.
Disclosure of Invention
In order to overcome the defects of the prior art, an object of the present invention is to provide a method for processing high-concurrency video data, which adopts a lock mechanism to control an atomic operation for writing video data into a cache server and a database server simultaneously, and solves the problems of data security and consistency.
One of the purposes of the invention is realized by adopting the following technical scheme:
receiving video data;
and writing the video data into a cache server based on a locking mechanism, and simultaneously writing the video data into a database server through a queue server.
Further, the queue server employs a multi-threaded mechanism, each thread for processing transactions of a set of producers and consumers, the producers receiving the video data, and the consumers writing the video data to the database server.
Further, after receiving video data, detecting the data volume of the video data and the number of threads of the queue server, and adding a thread when the ratio of the data volume to the number of threads is greater than or equal to a thread threshold value.
Further, after receiving video data, detecting the data volume of the video data, performing library division when the data volume is larger than a performance threshold value, and newly adding a cache server and a database server.
Furthermore, each piece of video data in the cache server is marked with expiration time, and the cache server stores the video data of which the current time is less than the expiration time.
Further, each cache server maintains a cache index table, each database server maintains a database index table, the key value in each index table is the transmission time stamp of each piece of video data, and the value in each index table is the position of each piece of video data in the cache server or the database server.
Further, receiving a viewing request;
searching the cache index table according to the target transmission time, and judging whether a key value corresponding to the target transmission time is found in the cache index table;
if yes, returning target video data from the cache server;
if not, searching the database index table, returning the target video data from the database server, and writing the target video data into the cache server.
It is a further object of the present invention to provide an electronic device comprising a processor, a storage medium and a computer program, the computer program being stored in the storage medium, the computer program being such that when executed by the processor, the method for processing high concurrency video data, the method being one of the objects of the present invention.
It is a further object of the invention to provide a computer readable storage medium storing one of the objects of the invention, having stored thereon a computer program which, when executed by a processor, implements a high concurrency video data processing method which is one of the objects of the invention.
The fourth objective of the present invention is to provide a high-concurrency video data processing system, which distributes video data received from a terminal through a load balancing server, and writes the video data into a cache server and a database server simultaneously by a processing and control server according to the data volume of the distributed video data, thereby solving the problems of data security and consistency.
The fourth purpose of the invention is realized by adopting the following technical scheme:
a high concurrency video data processing system, comprising: the system comprises a load balancing server, a plurality of processing and control servers, a cache server, a queue server and a database server;
the load balancing server is used for receiving video data; distributing the video data to a plurality of the processing and control servers;
the processing and control server is used for writing the distributed video data into the cache server based on a locking mechanism and simultaneously writing the distributed video data into the database server through the queue server; detecting the data volume of the distributed video data and the thread number of the queue server; when the ratio of the data volume to the thread number is greater than or equal to a thread threshold value, adding a thread; and when the data volume is larger than the performance threshold value, adding a cache server and a database server.
Compared with the prior art, the invention has the beneficial effects that:
the invention adopts the atomicity operation of the locking mechanism, writes the video data into the cache server, and simultaneously writes the video data into the database server through the queue server, thereby ensuring the consistency of the data written in the database server and the cache server, improving the stability of storing a large amount of video data, and solving the problem of data divergence caused by falling the data in the cache server to the database server due to the cut-off time.
Drawings
Fig. 1 is a flowchart of a high-concurrency video data processing method according to a first embodiment of the present invention;
fig. 2 is a flowchart of a high concurrency video data processing method according to a fourth embodiment of the present invention;
fig. 3 is a block diagram of an electronic device according to a fifth embodiment of the present invention.
Detailed Description
The present invention will now be described in more detail with reference to the accompanying drawings, in which the description of the invention is given by way of illustration and not of limitation. The various embodiments may be combined with each other to form other embodiments not shown in the following description.
Example one
The embodiment one provides a high-concurrency video data processing method, which aims to ensure the consistency of data written in a database server and a cache server, improve the storage stability of a large amount of video data and solve the problem of data divergence caused by falling the data in the cache server to the database server due to the separation time by adopting a lock mechanism to control the atomicity operation of writing the video data into the cache server and the database server at the same time.
Referring to fig. 1, a method for processing high-concurrency video data includes the following steps:
and S110, receiving video data.
In the practical application of the video monitoring system project, a plurality of acquisition terminals simultaneously acquire video data at a high frequency, so that a large amount of video data which are highly concurrent by the plurality of acquisition terminals can be generated. The acquisition terminal comprises but is not limited to a singlechip and a microcomputer for acquiring monitoring video data. A large amount of video data are distributed to a plurality of processing and control servers through the load balancing server, and the phenomenon that a single processing and control server processes overlarge video data to cause time delay or even downtime is avoided. Each processing and control server receives the video data distributed by the load balancing server and processes the video data.
And S120, writing the video data into a cache server based on a locking mechanism, and simultaneously writing the video data into a database server through a queue server.
If the video data are directly landed on the database server, abnormal connection of the database server can be caused, so that the video data are firstly written into the queue server, and a large amount of video data are landed on the database server by the queue server. The queue server has a buffering effect on a large amount of video data, avoids abnormal connection of the database server, and improves the storage stability of the large amount of video data.
In order to ensure the data in the database server and the cache server are consistent, code programs realized by different program design voices are optimized, a locking mechanism can be adopted to write video data into the database server and the cache server to realize atomic operation, the video data can be written in at the same time, and if the video data is failed to be written into the database server or the cache server, the other party cannot write in. The database server backs up all video data, and when the cache server has data loss, the database server can be used as a backup source to quickly restore the lost data, so that the storage stability of a large amount of video data is improved.
Preferably, when the video data fails to be written simultaneously, the abnormal information is sent to the log server, so that the video data which fails to be written can be traced and analyzed conveniently, and the stability is further improved.
Example two
And the second embodiment is an improvement on the first embodiment, and the threads are quantitatively increased according to the data volume of the video data and the thread number of the queue server, so that the high-efficiency floor backup of the video data is ensured, and the storage stability is improved.
The queue server adopts a multithreading mechanism, each thread is used for processing transactions of a group of producers and consumers, the producers receive the video data, and the consumers place the video data in the database server. When only one producer and one consumer exist in the thread, the consumer can read the data only after the producer puts the data, and the producer goes to produce after the consumer takes the data, so that the data synchronization is realized; when the producer receives the data, the consumer can not fall to the ground, or when the consumer falls to the ground, the producer can not receive the data, otherwise, the data access/fetching failure of the producer and the consumer is caused, the consistency of the data written into the queue server and the cache server is further ensured, and the storage stability is improved.
After receiving video data, detecting the data quantity q of the video data and the group number n (namely the thread number) of producers and consumers in a queue server, and when the ratio of the data quantity q to the group number n of the producers and the consumers is more than or equal to a thread threshold Wt, namely
At least, increase
Group producers and consumers, i.e. newly added
And the threads further ensure that a large amount of video data can be landed on a database server in time, so that the stability of data storage is improved. The thread threshold Wt may be preset according to actual service requirements in the video monitoring system project. A large amount of video data is landed in the database server through each thread in the queue server.
EXAMPLE III
The third embodiment is an improvement on the basis of the first embodiment and/or the second embodiment, and a cache server and a database server are added according to the data volume of the video data, so that the capacity expansion is realized for a large amount of video data, and the storage stability is further improved.
The performance threshold value can be preset according to actual service requirements in a video monitoring system project, after video data are received, when the data volume of the video data is larger than the performance threshold value, the storage performance of the cache server and the database server on the data is reduced, the cache server and the database server are newly added at the moment, the transverse expansion of the storage capacity is realized, then the video data are stored, and the video data are separately stored in different caches and databases, so that the storage stability of a large amount of video data can be further improved, and the video searching can be conveniently and quickly responded. In order to effectively control the cost and avoid the condition that the cache server is down due to overlarge stored data, preferably, each piece of video data in the cache server is marked with failure time, the cache server only stores the video data of which the current time is less than the failure time, the monitoring video data of the latest period of time is guaranteed to be cached, and when a video viewing request exists, the data can be taken from the cache server to be quickly responded.
Example four
The fourth embodiment is an improvement on the third embodiment, and the cache index table and the database index table are successively searched according to the target transmission time, so that the high-concurrency data query request can be responded in real time.
In order to further improve the storage stability, each cache server maintains a cache index table, each database server maintains a database index table, the key value in each index table is the transmission time stamp of each piece of video data, and the value in each index table is the position of each piece of video data in the cache server or the database server. Each cache server and each database server can also quickly respond to video viewing requests by maintaining respective index tables.
Referring to fig. 2, the method for processing high-concurrency video data of the present embodiment includes the following steps:
and S210, receiving a viewing request.
When a plurality of output terminals simultaneously make requests to view a large amount of video data, highly concurrent viewing requests are generated. The output terminal may be a client or web browser on a cell phone or tablet that presents the monitoring data. The high-concurrency viewing requests are distributed to the processing and control servers through the load balancing server, and therefore the situation that a single processing and control server processes a large number of viewing requests to cause delay and even downtime is avoided. Each processing and control server receives the viewing request distributed by the load balancing server and responds to the viewing request. S220, searching the cache index table according to the target transmission time, and judging whether a key value corresponding to the target transmission time is found in the cache index table.
After each piece of video data is stored through different cache servers or database servers, in order to ensure that each piece of video data has a unique corresponding key value, the key value of each piece of video data adopts a coding scheme of a transmission time stamp, a database table serial number and a bigdata-hash, so that the target video data can be accurately searched according to the transmission time of the target video data.
And S230, if so, returning the target video data from the caching server. If the target transmission time is within the latest period of time, the key value in the cache index table can be directly inquired to obtain the position of the target video data in the cache server, and then the target video data is returned to complete the response to the viewing request. Therefore, the response speed to the high concurrent viewing request can be increased, the concurrent access amount of the terminal is greatly increased, and the requirement of subsequent business operation is met.
S240, if not, searching the database index table, returning the target video data from the database server, and writing the target video data into the cache server.
If the key value corresponding to the target transmission time is not found in the cache index table, the target transmission time is not in the latest period of time or the target video data in the cache server is lost unexpectedly, the key value in the database index table is inquired at the moment, the target video data in the database server is further obtained, and the response to the viewing request is completed. Meanwhile, the target video data is written into the cache server again, the cache data can be directly recovered without stopping service, and the terminal can conveniently respond for the second time.
EXAMPLE five
Fig. 3 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present invention, as shown in fig. 3, the electronic device includes a processor 310, a memory 320, an input device 330, and an output device 340; the number of the processors 310 in the computer device may be one or more, and one processor 310 is taken as an example in fig. 3; the processor 310, the memory 320, the input device 330 and the output device 340 in the electronic apparatus may be connected by a bus or other means, and the connection by the bus is exemplified in fig. 3.
The memory 320 is a computer-readable storage medium for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the high-concurrency video data processing method in the embodiments of the present invention. The processor 310 executes various functional applications and data processing of the electronic device by running software programs, instructions and modules stored in the memory 320, that is, implements the high-concurrency video data processing methods of the first to fourth embodiments.
The memory 320 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 320 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 320 may further include memory located remotely from the processor 310, which may be connected to the electronic device through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 330 may be used to receive video data, viewing requests, and the like. The output device 340 may include a display device such as a display screen.
EXAMPLE six
An embodiment of the present invention further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, perform a method for high-concurrency video data processing, the method including:
receiving video data;
and writing the video data into a cache server based on a locking mechanism, and simultaneously writing the video data into a database server through a queue server.
Preferably, after receiving video data, detecting a data amount of the video data and a thread number of the queue server, and adding a thread when a ratio of the data amount to the thread number is greater than or equal to a thread threshold.
Preferably, after receiving the video data, detecting the data volume of the video data, performing library splitting when the data volume is greater than a performance threshold, and newly adding a cache server and a database server.
Preferably, each cache server maintains a cache index table, each database server maintains a database index table, the key value in each index table is the transmission timestamp of each piece of video data, and the value in each index table is the position of each piece of video data in the cache server or the database server.
Preferably, a request to view is received;
searching the cache index table according to the target transmission time, and judging whether a key value corresponding to the target transmission time is found in the cache index table;
if yes, returning target video data from the cache server;
if not, searching the database index table, returning the target video data from the database server, and writing the target video data into the cache server.
Of course, the storage medium containing the computer-executable instructions provided by the embodiments of the present invention is not limited to the method operations described above, and may also perform related operations in the high-concurrency video data processing method provided by any embodiments of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes instructions for enabling an electronic device (which may be a mobile phone, a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
EXAMPLE seven
The seventh embodiment is a high-concurrency video data processing system corresponding to the high-concurrency video data processing method in the foregoing embodiments, and the processing and control server writes the video data into the cache server and the database server at the same time according to the data volume of the distributed video data, so that the problems of data security and consistency are solved, and the stability of storing a large amount of video data is improved.
A high concurrency video data processing system, comprising: the system comprises a load balancing server, a plurality of processing and control servers, a cache server, a queue server and a database server.
The load balancing server is used for receiving the video data; distributing the video data to a plurality of processing and control servers;
the processing and control server is used for writing the distributed video data into the cache server based on a locking mechanism and simultaneously writing the video data into the database server through the queue server; detecting the data volume of the distributed video data and the number of threads of the queue server; when the ratio of the data volume to the number of the threads is greater than or equal to the thread threshold value, adding the threads; and when the data volume is larger than the performance threshold value, adding a cache server and a database server.
The load balancing server can be connected with a plurality of acquisition terminals or output terminals, video data are obtained through the acquisition terminals in the video monitoring system, the acquisition terminals can be single-chip microcomputers, mini-computers and the like, the output terminals can be mobile phones or tablet clients or web browsers for displaying the monitoring data, and when the output terminals provide data checking requests, target video data are returned to the output terminals for displaying. The load balancing server is connected with the processing and control servers, high-concurrency video data or viewing requests are distributed to the processing and control servers, and delay and even downtime caused by the fact that a single processing and control server processes overlarge data or requests are prevented. The processing and control server supports horizontal expansion and can respond to requests with different magnitudes.
The processing and control server is connected with at least one cache server and is connected with at least one database server through a queue server. The cache server and the database server can transversely expand according to the data volume of the video data, and the stability of storing a large amount of video data is improved.
Preferably, the high-concurrency video data processing system further comprises a log server for recording abnormal behavior logs of the whole system, and the log server is connected with other servers in the system.
Preferably, each cache server maintains a cache index table, each database server maintains a database index table, the key value in each index table is the transmission timestamp of each piece of video data, and the value in each index table is the position of each piece of video data in the cache server or the database server. The load balancing server is used for receiving the high-concurrency viewing request and distributing the viewing request to the plurality of processing and control servers;
the processing and control server is used for receiving the distributed viewing request;
searching a performance threshold cache index table according to the target transmission time, and judging whether a key value corresponding to the performance threshold target transmission time is found in the performance threshold cache index table;
if yes, returning the target video data from the performance threshold cache server;
if not, searching the index table of the performance threshold database, returning the target video data of the performance threshold from the performance threshold database server, and writing the target video data of the performance threshold into the performance threshold cache server.
The processing and control server searches the cache index table and the database index table in sequence according to the target transmission time, and can respond to the highly concurrent data query request in real time.
It should be noted that, in the embodiment of the method and the system for processing high concurrent video data, the servers included in the embodiment are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be implemented; in addition, the specific names of the function servers are only for convenience of distinguishing from each other, and are not used to limit the protection scope of the present invention.
Various other modifications and changes may be made by those skilled in the art based on the above-described technical solutions and concepts, and all such modifications and changes should fall within the scope of the claims of the present invention.
Claims (10)
1. A method for processing high-concurrency video data, comprising: the method comprises the following steps:
receiving video data;
and writing the video data into a cache server based on a locking mechanism, and simultaneously writing the video data into a database server through a queue server.
2. The method of claim 1, wherein: the queue server adopts a multithreading mechanism, each thread is used for processing transactions of a group of producers and consumers, the producers receive the video data, and the consumers write the video data into the database server.
3. The method of claim 2, wherein: after receiving video data, detecting the data volume of the video data and the thread number of the queue server, and adding a thread when the ratio of the data volume to the thread number is greater than or equal to a thread threshold value.
4. The method of claim 1, wherein: after receiving video data, detecting the data volume of the video data, performing database partitioning when the data volume is larger than a performance threshold value, and newly adding a cache server and a database server.
5. The method of claim 1, wherein: and marking the expiration time of each piece of video data in the cache server, and storing the video data of which the current time is less than the expiration time by the cache server.
6. A method for highly concurrent video data processing as claimed in any of claims 1 to 5, wherein: each cache server maintains a cache index table, each database server maintains a database index table, the key value in each index table is the transmission timestamp of each piece of video data, and the value in each index table is the position of each piece of video data in the cache server or the database server.
7. The method of claim 6, wherein:
receiving a viewing request;
searching the cache index table according to the target transmission time, and judging whether a key value corresponding to the target transmission time is found in the cache index table;
if yes, returning target video data from the cache server;
if not, searching the database index table, returning the target video data from the database server, and writing the target video data into the cache server.
8. An electronic device comprising a processor, a storage medium, and a computer program, the computer program being stored in the storage medium, wherein the computer program, when executed by the processor, performs the high concurrency video data processing method of any one of claims 1 to 7.
9. A computer storage medium having a computer program stored thereon, characterized in that: the computer program, when executed by a processor, implements the high concurrency video data processing method of any of claims 1 to 7.
10. A high concurrency video data processing system, characterized by: the method comprises the following steps: the system comprises a load balancing server, a plurality of processing and control servers, a cache server, a queue server and a database server;
the load balancing server is used for receiving video data; distributing the video data to a plurality of the processing and control servers;
the processing and control server is used for writing the distributed video data into the cache server based on a locking mechanism and simultaneously writing the distributed video data into the database server through the queue server; detecting the data volume of the distributed video data and the thread number of the queue server; when the ratio of the data volume to the thread number is greater than or equal to a thread threshold value, adding a thread; and when the data volume is larger than the performance threshold value, adding a cache server and a database server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010350998.3A CN111651631B (en) | 2020-04-28 | 2020-04-28 | High concurrency video data processing method, electronic equipment, storage medium and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010350998.3A CN111651631B (en) | 2020-04-28 | 2020-04-28 | High concurrency video data processing method, electronic equipment, storage medium and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111651631A true CN111651631A (en) | 2020-09-11 |
| CN111651631B CN111651631B (en) | 2023-11-28 |
Family
ID=72348174
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010350998.3A Active CN111651631B (en) | 2020-04-28 | 2020-04-28 | High concurrency video data processing method, electronic equipment, storage medium and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111651631B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113434344A (en) * | 2021-07-22 | 2021-09-24 | 咪咕数字传媒有限公司 | File storage method and device, computing equipment and computer storage medium |
| CN114461151A (en) * | 2022-04-11 | 2022-05-10 | 武汉中科通达高新技术股份有限公司 | Video storage method, computer equipment and storage medium of national standard camera |
| CN116192816A (en) * | 2022-06-01 | 2023-05-30 | 中兴通讯股份有限公司 | Data transmission method, transmission server, reception server, and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102023973A (en) * | 2009-09-14 | 2011-04-20 | 中国移动通信集团公司 | Method, device and system for synchronizing database with application cache server |
| CN102117309A (en) * | 2010-01-06 | 2011-07-06 | 卓望数码技术（深圳）有限公司 | Data caching system and data query method |
| CN102331986A (en) * | 2010-07-12 | 2012-01-25 | 阿里巴巴集团控股有限公司 | Database cache management method and database server |
| CN103076994A (en) * | 2012-12-31 | 2013-05-01 | 北京中科大洋科技发展股份有限公司 | Method for realizing off-line file writing in SAN (Storage Area Network) shared-file system |
| CN105160572A (en) * | 2015-09-30 | 2015-12-16 | 努比亚技术有限公司 | Device and method for controlling order to generate, and seckilling system |
| CN106257456A (en) * | 2016-07-08 | 2016-12-28 | 北京京东尚科信息技术有限公司 | The method of data base's stability, Apparatus and system is improved under high concurrent request |
| CN108595487A (en) * | 2018-03-14 | 2018-09-28 | 摇了购（武汉）电子商务有限公司 | The method and system of data are accessed under a kind of big data high concurrent |
| CN108664520A (en) * | 2017-04-01 | 2018-10-16 | 北京京东尚科信息技术有限公司 | Safeguard method, apparatus, electronic equipment and the readable storage medium storing program for executing of data consistency |
| CN109033292A (en) * | 2018-07-13 | 2018-12-18 | 南京邮电大学 | A kind of database distributed caching system and method |
| CN109271252A (en) * | 2018-08-29 | 2019-01-25 | 华为技术有限公司 | A kind of method and device adjusting Thread Count |
-
2020
- 2020-04-28 CN CN202010350998.3A patent/CN111651631B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102023973A (en) * | 2009-09-14 | 2011-04-20 | 中国移动通信集团公司 | Method, device and system for synchronizing database with application cache server |
| CN102117309A (en) * | 2010-01-06 | 2011-07-06 | 卓望数码技术（深圳）有限公司 | Data caching system and data query method |
| CN102331986A (en) * | 2010-07-12 | 2012-01-25 | 阿里巴巴集团控股有限公司 | Database cache management method and database server |
| CN103076994A (en) * | 2012-12-31 | 2013-05-01 | 北京中科大洋科技发展股份有限公司 | Method for realizing off-line file writing in SAN (Storage Area Network) shared-file system |
| CN105160572A (en) * | 2015-09-30 | 2015-12-16 | 努比亚技术有限公司 | Device and method for controlling order to generate, and seckilling system |
| CN106257456A (en) * | 2016-07-08 | 2016-12-28 | 北京京东尚科信息技术有限公司 | The method of data base's stability, Apparatus and system is improved under high concurrent request |
| CN108664520A (en) * | 2017-04-01 | 2018-10-16 | 北京京东尚科信息技术有限公司 | Safeguard method, apparatus, electronic equipment and the readable storage medium storing program for executing of data consistency |
| CN108595487A (en) * | 2018-03-14 | 2018-09-28 | 摇了购（武汉）电子商务有限公司 | The method and system of data are accessed under a kind of big data high concurrent |
| CN109033292A (en) * | 2018-07-13 | 2018-12-18 | 南京邮电大学 | A kind of database distributed caching system and method |
| CN109271252A (en) * | 2018-08-29 | 2019-01-25 | 华为技术有限公司 | A kind of method and device adjusting Thread Count |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113434344A (en) * | 2021-07-22 | 2021-09-24 | 咪咕数字传媒有限公司 | File storage method and device, computing equipment and computer storage medium |
| CN114461151A (en) * | 2022-04-11 | 2022-05-10 | 武汉中科通达高新技术股份有限公司 | Video storage method, computer equipment and storage medium of national standard camera |
| CN116192816A (en) * | 2022-06-01 | 2023-05-30 | 中兴通讯股份有限公司 | Data transmission method, transmission server, reception server, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111651631B (en) | 2023-11-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111651631B (en) | High concurrency video data processing method, electronic equipment, storage medium and system | |
| CN109407977B (en) | Big data distributed storage management method and system | |
| US9807168B2 (en) | Distributed shared log for modern storage servers | |
| CN112559637B (en) | Data processing method, device, equipment and medium based on distributed storage | |
| CN110837423A (en) | Method and device for automatically acquiring data of guided transport vehicle | |
| US9075722B2 (en) | Clustered and highly-available wide-area write-through file system cache | |
| CN107346270B (en) | Method and system for real-time computation based radix estimation | |
| CN110784498B (en) | Personalized data disaster tolerance method and device | |
| CN111404737B (en) | Disaster recovery processing method and related device | |
| CN107566341B (en) | Data persistence storage method and system based on federal distributed file storage system | |
| CN116226139B (en) | Distributed storage and processing method and system suitable for large-scale ocean data | |
| CN106815318B (en) | Clustering method and system for time sequence database | |
| CN104580536A (en) | Metadata cluster load balance realization method | |
| CN116701352A (en) | Database data migration method and system | |
| CN113010373B (en) | Data monitoring method and device, electronic equipment and storage medium | |
| CN114661818A (en) | Method, system, and medium for real-time synchronization of data between clusters in a graph database | |
| CN113515518A (en) | Data storage method and device, computer equipment and storage medium | |
| CN115543970B (en) | Data page processing method, data page processing device, electronic equipment and storage medium | |
| CN104407918B (en) | A kind of preservation of avionics system task data and restoration methods | |
| CN111382200A (en) | Information loading method and device, server and storage medium | |
| CN117093367B (en) | Service data processing method, device and storage medium | |
| US20230179399A1 (en) | Method and system for preventing transfer error of big data | |
| CN113282405B (en) | Load adjustment optimization method and terminal | |
| CN113434509A (en) | Updating method and device of incremental index, storage medium and electronic equipment | |
| Steinmaurer et al. | Combining stream processing engines and big data storages for data analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Ye Junyan Inventor after: Tang Zhuo Inventor after: Lu Bin Inventor after: OuYang Jiming Inventor before: Tang Zhuo Inventor before: Ye Junyan Inventor before: Lu Bin Inventor before: OuYang Jiming |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |