CN117041685A - Road disease display method, system, device, equipment and readable storage medium - Google Patents
Road disease display method, system, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN117041685A CN117041685A CN202310922791.2A CN202310922791A CN117041685A CN 117041685 A CN117041685 A CN 117041685A CN 202310922791 A CN202310922791 A CN 202310922791A CN 117041685 A CN117041685 A CN 117041685A
- Authority
- CN
- China
- Prior art keywords
- code stream
- detection result
- road disease
- disease detection
- road
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 201000010099 disease Diseases 0.000 title claims abstract description 151
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 151
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000001514 detection method Methods 0.000 claims abstract description 170
- 238000004590 computer program Methods 0.000 claims description 20
- 238000009877 rendering Methods 0.000 claims description 15
- 230000001934 delay Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 11
- 230000006854 communication Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 9
- 230000000875 corresponding effect Effects 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000011664 signaling Effects 0.000 description 3
- 230000007175 bidirectional communication Effects 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000001747 exhibiting effect Effects 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/637—Control signals issued by the client directed to the server or network components
- H04N21/6373—Control signals issued by the client directed to the server or network components for rate control, e.g. request to the server to modify its transmission rate
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
- H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
- H04N21/26216—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints involving the channel capacity, e.g. network bandwidth
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
- H04N21/6437—Real-time Transport Protocol [RTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64746—Control signals issued by the network directed to the server or the client
- H04N21/64761—Control signals issued by the network directed to the server or the client directed to the server
- H04N21/64769—Control signals issued by the network directed to the server or the client directed to the server for rate control
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Databases & Information Systems (AREA)
- Processing Or Creating Images (AREA)
Abstract
The application discloses a road disease display method, a road disease display device, computer equipment and a readable storage medium. The method comprises the following steps: the playing end pulls the auxiliary code stream from the image acquisition equipment, receives the road disease detection result sent by the detection end, pulls the main code stream from the image acquisition equipment, and obtains the road disease detection result according to the main code stream; the playing end renders the road disease detection result to a video picture generated by the auxiliary code stream according to the time stamp of the road disease detection result and the time stamp of the auxiliary code stream; and delaying the video frames in the auxiliary code stream by a preset time to generate video pictures. The detection end only sends the road disease detection result to the playing end. The playing end superimposes the road disease detection result on the auxiliary code stream. The playing end is free from the limitation of a higher code rate of a main code stream when playing the video, and the clamping and the stopping can not be caused. The auxiliary code stream processed by the playing end has low code rate, the memory of the superimposed road disease detection result is small, and the final result is smoothly played.
Description
Technical Field
The application relates to the technical field of data processing, in particular to a road disease display method, a system, a device, equipment and a readable storage medium.
Background
As vehicles increase, road surface damage speeds increase, and thus, road damage detection work is becoming more and more important. Road hazards generally include transverse cracks, longitudinal cracks, blocky cracks, crazes, pits, repairs, etc. of the road surface. Road disease detection systems generally include an acquisition system and a disease analysis system. The acquisition system is arranged on the detection vehicle and can acquire images and videos of roads. The disease analysis system may detect road diseases based on the images and the videos.
In the prior art, for the accuracy of road disease detection, a high-resolution main code stream is required to be used for visual analysis; and the detection end directly superimposes the detection result on the main code stream and sends the main code stream to the playing end for playing. But the resolution of the video finally displayed by the playing end is limited by the resolution of the main code stream, and the video transmission speed of the main code stream with high resolution is low, so that the requirements on bandwidth and speed are high.
Disclosure of Invention
In view of this, embodiments of the present application provide a road disease display method, system, device, apparatus, and readable storage medium, which can smoothly play video streams, is not limited by the resolution of the main code stream, and reduces the requirements of bandwidth and speed.
In a first aspect, the present application provides a road disease display method, including:
the playing end pulls the auxiliary code stream from the image acquisition equipment, receives the road disease detection result sent by the detection end, pulls the main code stream from the image acquisition equipment, and obtains the road disease detection result according to the main code stream;
the playing end renders the road disease detection result to a video picture generated by the auxiliary code stream according to the time stamp of the road disease detection result and the time stamp of the auxiliary code stream; and delaying the video frames in the auxiliary code stream by a preset time to generate video pictures.
In a second aspect, the present application provides a road disease display system comprising: the device comprises a playing end, a detecting end and image acquisition equipment;
the image acquisition equipment is used for shooting the road and obtaining a main code stream and an auxiliary code stream corresponding to the road;
the detection end is used for pulling the main code stream from the image acquisition equipment and obtaining a road disease detection result according to the main code stream;
the playing end is used for pulling the auxiliary code stream from the image acquisition equipment and receiving the road disease detection result sent by the detection end; rendering the road disease detection result to a video picture generated by the auxiliary code stream according to the time stamp of the road disease detection result and the time stamp of the auxiliary code stream; and delaying the video frames in the auxiliary code stream by a preset time to generate video pictures.
In a third aspect, the present application provides a road disease display device comprising:
the acquisition unit is used for shooting the road and obtaining a main code stream and an auxiliary code stream corresponding to the road;
the detection unit is used for pulling the main code stream from the image acquisition equipment and obtaining a road disease detection result according to the main code stream;
the playing unit is used for pulling the auxiliary code stream from the image acquisition equipment and receiving the road disease detection result sent by the detection end; rendering the road disease detection result to a video picture generated by the auxiliary code stream according to the time stamp of the road disease detection result and the time stamp of the auxiliary code stream; and delaying the video frames in the auxiliary code stream by a preset time to generate video pictures.
In a fourth aspect, the present application provides a computer device comprising a memory storing a computer program and a processor implementing the steps of the method described above when the processor executes the computer program.
In a fifth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method described above.
In a sixth aspect, the application provides a computer program product comprising a computer program which, when executed by a processor, carries out the steps of the method described above.
Therefore, the technical scheme provided by the embodiment of the application has the following beneficial effects:
according to the road disease display method provided by the embodiment of the application, the detection end does not need to superimpose the main code stream and the road disease detection result, but only sends the road disease detection result to the play end. The playing end does not need to process the main code stream, but superimposes the road disease detection result on the auxiliary code stream. Therefore, the playing end is not limited by the higher code rate of the main code stream when playing the video, and the receiving speed of the code stream is not influenced by the limitation of the bandwidth of the main code stream, so that the blocking is not caused. Because the auxiliary code stream processed by the playing end has low code rate, the memory of the superimposed road disease detection result is small, and therefore, the final result can be played smoothly.
Drawings
Fig. 1 is an application scenario diagram of a road disease display method provided by an embodiment of the present application;
fig. 2 is a flowchart of a method for displaying road diseases according to an embodiment of the present application;
fig. 3 is a signaling diagram of a road disease detection method according to an embodiment of the present application;
fig. 4 is a schematic diagram of a road disease display system according to an embodiment of the present application;
fig. 5 is a schematic view of a road disease display device according to an embodiment of the present application;
FIG. 6 is a diagram illustrating an internal architecture of a computer device according to an embodiment of the present application;
FIG. 7 is an internal block diagram of another computer device according to an embodiment of the present application;
fig. 8 is an internal structural diagram of a computer-readable storage medium according to an embodiment of the present application.
Detailed Description
In order to make the purposes, technical schemes and advantages of the application more clear, application scenes of the technical scheme provided by the embodiment of the application are introduced.
The road disease display method provided by the embodiment of the application is applied to a road disease display system, and the system can detect road diseases and display the road diseases. In order to enable the playing end to smoothly play the video, the playing end directly obtains the auxiliary code stream from the image acquisition equipment, and the transmission speed is high because the code rate of the auxiliary code stream is low, the requirement on the playing end is not high, the computational requirement of the playing end can be reduced, and the video memory resource can be saved. Because the detection end only transmits the detection result to the playing end, and the main code stream is not transmitted, the transmission of the detection result can reduce the requirements on bandwidth and speed. Therefore, the technical scheme can improve the playing smoothness of the playing end and reduce the hardware requirement of the detecting end.
In the traditional technology, the detection end directly superimposes the detection result on the main code stream, recodes and transmits the detection result to the playing end, and the playing end directly plays, so that the hardware requirement on the detection end is higher, and the requirement on bandwidth and speed is higher when the detection end transmits the main code stream superimposed detection result to the playing end.
The road disease display method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a communication network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices, portable wearable devices, and the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, etc. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.
In order to facilitate understanding of the technical solution of the embodiments of the present application, technical terms related to the embodiments of the present application will be described first.
Main code stream: the main code stream output by the video encoder contains all information of video images, and has higher code rate and better quality.
Auxiliary code stream: the auxiliary code stream output by the video encoder contains partial information of video images, and has lower code rate and poorer quality.
Video character Overlay (OSD) refers to the overlay of information such as text and images On video pictures, and is commonly used in the fields of monitoring, broadcasting, or television.
The JS Object Notation (JSON, javaScript Object notification) is a lightweight data exchange format. It stores and presents data in a text format that is completely independent of the programming language based on a subset of ECMAScript (European Computer Manufacturers Association, JS specification by the european computer institute). The compact and clear hierarchical structure makes JSON an ideal data exchange language. Easy to read and write, easy to analyze and generate by a machine, and effectively improves the network transmission efficiency.
The real time streaming protocol (RTSP, real Time Streaming Protocol) is an application layer protocol in the TCP/IP protocol architecture that defines how efficiently a one-to-many application program transmits multimedia data over an IP network.
JPEG (Joint Photographic Experts Group) is a product of the JPEG standard, which is formulated by the international organization for standardization (ISO), and is a compression standard for continuous tone still images. The JPEG format is the most commonly used image file format, with the suffix named. Jpg or. JPEG.
In order that the above-recited objects, features and advantages of the present application will become more readily apparent, a more particular description of embodiments of the application will be rendered by reference to the appended drawings and appended drawings.
Referring to fig. 2, the flowchart of a method for displaying road diseases according to an embodiment of the present application is shown.
The road disease display method provided by the embodiment of the application comprises the following steps:
s201: the playing end pulls the auxiliary code stream from the image acquisition equipment, receives the road disease detection result sent by the detection end, pulls the main code stream from the image acquisition equipment, and obtains the road disease detection result according to the main code stream;
for example, the image acquisition device may be implemented using a camera, and a general camera may support multiple code streams, i.e., may directly obtain a main code stream and an auxiliary code stream. For example, the camera main code stream is set to be a high-resolution high-code-rate jpeg code stream with a code rate of 40960kb/s and a resolution of 5M. The camera auxiliary stream is set to 4096kb/s,1080p high definition h264 video stream. The above data is merely illustrative, and it should be understood that the primary and secondary code streams may be other code rate streams, where the code rate of the primary code stream is higher than the code rate of the secondary code stream.
The detection end obtains a main code stream from the image acquisition equipment, and the play end obtains an auxiliary code stream from the image acquisition equipment. The detection end obtains a road disease detection result by using the main code stream, and the playing end plays the video by using the auxiliary code stream.
S202: the playing end renders the road disease detection result to a video picture generated by the auxiliary code stream according to the time stamp of the road disease detection result and the time stamp of the auxiliary code stream; and delaying the video frames in the auxiliary code stream by a preset time to generate video pictures.
Because the main code stream and the auxiliary code stream are obtained by the image acquisition equipment, the time stamp of the main code stream is consistent with the time stamp of the auxiliary code stream. And because the road disease detection result is obtained according to the main code stream, the time stamp of the road disease detection result is consistent with the time stamp of the main code stream, and then the time stamp of the road disease detection result and the time stamp of the auxiliary code stream can be used for rendering the road disease detection result on a video picture generated by the auxiliary code stream with the consistent time stamp, so that the road disease detection result is rendered at an accurate position when the video is played.
According to the road disease display method provided by the embodiment of the application, the detection end does not need to superimpose the main code stream and the road disease detection result, but only sends the road disease detection result to the play end. The playing end does not need to process the main code stream, but superimposes the road disease detection result on the auxiliary code stream. Therefore, the playing end is not limited by the higher code rate of the main code stream when playing the video, and the receiving speed of the code stream is not influenced by the limitation of the bandwidth of the main code stream, so that the blocking is not caused. Because the auxiliary code stream processed by the playing end has low code rate, the memory of the superimposed road disease detection result is small, and therefore, the final result can be played smoothly. The technical scheme provided by the embodiment can reduce the hardware requirement of the playing end.
A specific implementation is described in detail below in connection with signaling diagrams.
Referring to fig. 3, the signaling diagram of a road disease detection method according to an embodiment of the present application is shown.
S301: the detection end B pulls the main code stream from the image acquisition equipment A.
And the detection end B pulls the main code stream from the image acquisition equipment through a real-time streaming protocol RTSP protocol and receives the main code stream pictures with high resolution and high code rate.
S302: the playing end C pulls the auxiliary code stream from the image acquisition equipment A.
It should be understood that the sequence of S301 and S302 is not specifically limited in the embodiment of the present application, and may be performed simultaneously or sequentially.
The playing end pulls the auxiliary code stream from the image acquisition equipment, and specifically comprises the following steps:
the playing end pulls the auxiliary code stream from the image acquisition equipment through a real-time streaming protocol (RTSP);
the method for receiving the road disease detection result sent by the detection end specifically comprises the following steps:
the playing end is connected with the detecting end through an application layer protocol WebSocket (WS for short) protocol interface, and receives the road disease detection result in the JSON format sent by the detecting end. WS-protocols may enable bi-directional communication, for example, may enable bi-directional communication between a browser and a server, and may enable persistent connection between a client and a server.
S303: and the detection end B obtains a road disease detection result according to the main code stream.
The detection end B obtains a road disease detection result according to the main code stream, and comprises the following steps:
and inputting the main code stream chips into a road disease detection model, outputting a road disease detection result by the road disease detection module, and converting the road disease detection result into a JSON format.
S304: the playing end C presses the video frames of the auxiliary code stream into a buffer queue.
For example, the playing end C renders the video frames in the buffer queue to the frame after a preset time delay. The embodiment of the application is not particularly limited to the length of the preset time, and may be specifically set according to actual needs, for example, the preset time is set to be fixed 200 milliseconds.
The playing end C presses the video frames into the buffer queue, and renders the video frames after the preset time, so that preparation is made for smooth playing of the video. Since the detection end B needs a certain time to perform detection, i.e. execution and reasoning of the algorithm. In addition, the detection end B sends the road disease detection result to the play end C, and a certain time is required in the data transmission process, so that the play end C delays the preset time and then renders the road disease detection result, which is to enable the road disease detection result to be synchronously rendered on the video picture.
In addition, the playing end C sets a preset time delay, and since the time difference between two adjacent video frames of the received auxiliary code stream may not be fixed, direct playing may cause the video frames to look stuck, which is not smooth. In addition, the playing end C presses the auxiliary code stream into the buffer queue first, hardware pressure of the playing end can be relieved, if the auxiliary code stream is directly decoded, time is possibly short, video is blocked, playing is not smooth, the auxiliary code stream is pressed into the buffer queue first, and therefore decoding and playing can be carried out according to fixed time, and video playing is smooth.
It should be understood that the sequence of S303 and S304 is not specifically limited in the embodiment of the present application, and may be performed simultaneously, because different execution bodies are operating. In addition, the embodiment of the present application also does not specifically limit the sequence of S304 and S305.
S305: the detection end B sends a road disease detection result to the playing end C. The playing end C is connected to the detecting end B through a WS protocol interface and receives the detection result push of the JSON format of the detecting end B in real time.
S306: and the playing end C correlates the road disease detection result with the video frames in the buffer queue according to the time stamp, and hierarchically renders the correlated road disease detection result and the video frames to the video picture.
The playing end C generates a video picture according to the video frame delay preset time in the auxiliary code stream, and the method comprises the following steps: the playing end delays a preset time from the buffer queue to render the video frames to the video pictures.
The playing end renders the road disease detection result to a video picture generated by the auxiliary code stream according to the time stamp of the road disease detection result and the time stamp of the auxiliary code stream, and the method comprises the following steps: and searching video frames with consistent time stamps from the cache queue according to the time stamps of the road disease detection results, associating the road disease detection results with the video frames and the next frames of the video frames, and layering and rendering the video characters of the road disease detection results on video frames and the video frames next to the video frames by overlapping OSD.
Layering and rendering the road fault detection result video character superposition OSD onto a video picture generated by a video frame and a next frame of the video frame, wherein the method comprises the following steps: setting a video frame and a corresponding position transparent of a next frame of the video frame according to coordinates of the road disease detection result; and layering and rendering the video character superposition OSD of the road disease detection result to a transparent position of a video picture generated by the video frame and a next frame of the video frame.
The playing end can render the road disease detection result and the video frame in a layered mode, the playing end renders a video picture, the coordinate position corresponding to the detection result on the video picture is specially used for drawing a transparent window of the detection result, the detection result is directly overlapped on the transparent window, and therefore the playing end does not need to decode the detection result.
When the video frames in the buffer queue have no associated road disease detection result, the receiving and playing end directly takes the video frames from the buffer queue and renders the video frames to the video frames.
The road disease display method provided by the embodiment of the application can display the road disease detection result at the playing end in real time, and simultaneously improves the flexibility of selecting the video stream at the playing end, so that the video playing at the playing end is not limited by the high code rate of the video stream at the detecting end any more, and a proper video stream can be flexibly selected according to the configuration of the playing end.
The embodiment of the application is not particularly limited to the type of the playing end, and can be a tablet personal computer, a playing platform or the like.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a road disease display system. The implementation of the solution provided by the device is similar to that described in the above method, so the specific limitation in the embodiments of the road disease display system or road disease display systems provided below may be referred to the limitation of the image matching method hereinabove, and will not be repeated here.
Referring to fig. 4, a schematic diagram of a road disease display system according to an embodiment of the present application is shown.
The road disease display system 400 provided by the embodiment of the application comprises: the device comprises a playing end C, a detecting end B and image acquisition equipment A;
the image acquisition equipment A is used for shooting a road and obtaining a main code stream and an auxiliary code stream corresponding to the road;
the detection end B is used for pulling the main code stream from the image acquisition equipment and obtaining a road disease detection result according to the main code stream;
the playing end C is used for pulling the auxiliary code stream from the image acquisition equipment and receiving the road disease detection result sent by the detection end; rendering the road disease detection result to a video picture generated by the auxiliary code stream according to the time stamp of the road disease detection result and the time stamp of the auxiliary code stream; and delaying the video frames in the auxiliary code stream by a preset time to generate video pictures.
In addition, the embodiment of the application also provides a road disease display device, which is described in detail below with reference to the accompanying drawings.
As shown in fig. 5, an embodiment of the present application provides a road disease display device 500, including:
the acquisition unit 501 is used for shooting a road and obtaining a main code stream and an auxiliary code stream corresponding to the road;
the detection unit 502 is configured to pull the main code stream from the image acquisition unit, and obtain a road disease detection result according to the main code stream;
a playing unit 503, configured to pull the auxiliary code stream from the image acquisition device, and receive the road disease detection result sent by the detection unit; rendering the road disease detection result to a video picture generated by the auxiliary code stream according to the time stamp of the road disease detection result and the time stamp of the auxiliary code stream; and delaying the video frames in the auxiliary code stream by a preset time to generate video pictures.
In some embodiments, the playing unit renders the road disease detection result onto the video frame generated by the auxiliary code stream according to the timestamp of the road disease detection result and the timestamp of the auxiliary code stream, and the playing unit 503 is specifically configured to:
and searching video frames with consistent time stamps from the cache queue according to the time stamps of the road disease detection results, associating the road disease detection results with the video frames and the next frames of the video frames, and layering and rendering the video characters of the road disease detection results on video frames and the video frames next to the video frames by overlapping OSD.
In some embodiments, the playing unit 503 pulls the auxiliary code stream from the collecting unit, specifically includes:
the playing unit pulls the auxiliary code stream from the acquisition unit through a real-time streaming protocol (RTSP);
the play unit 503 is specifically configured to receive the road disease detection result sent by the detection end: and the detection unit is connected with the WS protocol interface, and receives the road disease detection result in the JSON format sent by the detection unit.
In some embodiments, the detection unit 502 obtains a road disease detection result according to the main code stream, specifically for: and inputting the main code stream into a road disease detection model, outputting a road disease detection result by the road disease detection module, and converting the road disease detection result into a JSON format.
In some embodiments, the playing unit 503 hierarchically renders the road hazard detection result video character overlay OSD onto the video frame generated by the video frame and the next frame of the video frame, specifically for: setting a video frame and a corresponding position transparent of a next frame of the video frame according to coordinates of the road disease detection result; and layering and rendering the video character superposition OSD of the road disease detection result to a transparent position of a video picture generated by the video frame and a next frame of the video frame.
In some embodiments, the playing unit 503 is further configured to: the playing end presses the video frames in the auxiliary code stream into a buffer queue; and delaying a preset time from the buffer queue to render the video frames to the video pictures.
The above-mentioned various modules in the road disease display device may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In some embodiments, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 6. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used to store the design drawings. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by the processor, implements the steps in the road disease exhibiting method described above.
In some embodiments, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input device. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement the steps in the image matching method described above. The display unit of the computer device is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer equipment can be a touch layer covered on a display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structures shown in fig. 6 or 7 are merely block diagrams of portions of structures associated with aspects of the application and are not intended to limit the computer device to which aspects of the application may be applied, and that a particular computer device may include more or fewer components than those shown, or may combine certain components, or may have a different arrangement of components.
In some embodiments, a computer device is provided, comprising a memory storing a computer program and a processor implementing the steps of the method embodiments described above when the computer program is executed.
In some embodiments, an internal structural diagram of a computer-readable storage medium is provided as shown in fig. 8, the computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method embodiments described above.
In some embodiments, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region.
Those skilled in the art will appreciate that implementing all or part of the above described embodiment methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.
Claims (10)
1. A road disease display method, characterized by comprising:
the playing end pulls the auxiliary code stream from the image acquisition equipment and receives the road disease detection result sent by the detection end, and the detection end pulls the main code stream from the image acquisition equipment and obtains the road disease detection result according to the main code stream;
the playing end renders the road disease detection result to a video picture generated by the auxiliary code stream according to the time stamp of the road disease detection result and the time stamp of the auxiliary code stream; and generating the video picture by delaying the video frame in the auxiliary code stream by a preset time.
2. The method as recited in claim 1, further comprising: the playing end presses the video frames in the auxiliary code stream into a buffer queue;
generating the video picture by delaying a video frame in the auxiliary code stream by a preset time, wherein the video picture comprises the following steps:
and the playing end delays a preset time from the buffer queue to render the video frame to the video picture.
3. The method according to claim 2, wherein the playing end renders the road disease detection result onto a video frame generated by the auxiliary code stream according to the timestamp of the road disease detection result and the timestamp of the auxiliary code stream, including:
and searching a video frame with consistent time stamp from the buffer queue according to the time stamp of the road disease detection result, associating the road disease detection result with the video frame and the next frame of the video frame, and layering and rendering the road disease detection result video character superposition OSD onto a video picture generated by the video frame and the next frame of the video frame.
4. The method of claim 1, wherein the playing end pulls the auxiliary code stream from the image capturing device, specifically comprising:
the playing end pulls the auxiliary code stream from the image acquisition equipment through a real-time streaming protocol (RTSP);
the receiving the road disease detection result sent by the detection end specifically comprises the following steps:
the playing end is connected with the detecting end through a WS protocol interface and receives the road disease detection result in the JSON format sent by the detecting end.
5. The method according to any one of claims 1 to 4, wherein the obtaining the road disease detection result from the main code stream includes:
and inputting the main code stream into a road disease detection model, outputting the road disease detection result by the road disease detection module, and converting the road disease detection result into a JSON format.
6. The method of claim 3, wherein hierarchically rendering the road hazard detection result video character overlay OSD onto a video picture generated by the video frame and a next frame of the video frame, comprises:
setting the corresponding positions of the video frame and the next frame of the video frame to be transparent according to the coordinates of the road disease detection result;
and layering and rendering the video character superposition OSD of the road disease detection result to a transparent position of a video picture generated by the video frame and a next frame of the video frame.
7. A road disease display system, comprising: the device comprises a playing end, a detecting end and image acquisition equipment;
the image acquisition equipment is used for shooting a road and obtaining a main code stream and an auxiliary code stream corresponding to the road;
the detection end is used for pulling the main code stream from the image acquisition equipment and obtaining the road disease detection result according to the main code stream;
the playing end is used for pulling the auxiliary code stream from the image acquisition equipment and receiving the road disease detection result sent by the detection end; rendering the road disease detection result to a video picture generated by the auxiliary code stream according to the timestamp of the road disease detection result and the timestamp of the auxiliary code stream; and generating the video picture by delaying the video frame in the auxiliary code stream by a preset time.
8. A road disease display device, characterized by comprising:
the acquisition unit is used for shooting a road and obtaining a main code stream and an auxiliary code stream corresponding to the road;
the detection unit is used for pulling the main code stream from the image acquisition equipment and obtaining the road disease detection result according to the main code stream;
the playing unit is used for pulling the auxiliary code stream from the image acquisition equipment and receiving a road disease detection result sent by the detection end; rendering the road disease detection result to a video picture generated by the auxiliary code stream according to the timestamp of the road disease detection result and the timestamp of the auxiliary code stream; and generating the video picture by delaying the video frame in the auxiliary code stream by a preset time.
9. A computer device, characterized in that it comprises a processor and a memory, in which computer program instructions are stored, which processor, when executing the computer program instructions, realizes the steps in the method according to any of claims 1-6.
10. A computer readable storage medium, characterized in that it has stored therein computer program instructions which, when executed by a processor, implement the steps in the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310922791.2A CN117041685A (en) | 2023-07-25 | 2023-07-25 | Road disease display method, system, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310922791.2A CN117041685A (en) | 2023-07-25 | 2023-07-25 | Road disease display method, system, device, equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117041685A true CN117041685A (en) | 2023-11-10 |
Family
ID=88621797
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310922791.2A Pending CN117041685A (en) | 2023-07-25 | 2023-07-25 | Road disease display method, system, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117041685A (en) |
-
2023
- 2023-07-25 CN CN202310922791.2A patent/CN117041685A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8639086B2 (en) | Rendering of video based on overlaying of bitmapped images | |
| WO2020082870A1 (en) | Real-time video display method and apparatus, and terminal device and storage medium | |
| CN110868625A (en) | Video playing method and device, electronic equipment and storage medium | |
| CN109040601B (en) | Multi-scale unstructured billion pixel VR panoramic photography system | |
| CN103248797A (en) | Video resolution enhancing method and module based on FPGA (field programmable gate array) | |
| JP7177034B2 (en) | Method, apparatus and stream for formatting immersive video for legacy and immersive rendering devices | |
| CN110166795B (en) | Video screenshot method and device | |
| WO2021233123A1 (en) | Video processing method and apparatus, and computer device and storage medium | |
| US20180343459A1 (en) | Image compression method and image decompression method | |
| WO2022111717A1 (en) | Image processing method and apparatus, and electronic device | |
| US7692562B1 (en) | System and method for representing digital media | |
| CN114374853A (en) | Content display method and device, computer equipment and storage medium | |
| CN114051120A (en) | Video alarm method, device, storage medium and electronic equipment | |
| CN117041685A (en) | Road disease display method, system, device, equipment and readable storage medium | |
| US20080007648A1 (en) | Real time scaling and rotating decoded image data | |
| CN116700943A (en) | Video playing system and method and electronic equipment | |
| CN113645486A (en) | Video data processing method and device, computer equipment and storage medium | |
| TW201216148A (en) | Remote management system and the method thereof | |
| TWI482470B (en) | Digital signage playback system, real-time monitoring system, and real-time monitoring method thereof | |
| CN109429067B (en) | Dynamic picture compression method and device, computer equipment and storage medium | |
| RU2339998C1 (en) | Contour method of compressing graphical files | |
| WO2020224158A1 (en) | Editing method, computer-readable storage medium, and terminal | |
| US20210344918A1 (en) | Optimization of multi-sink wi-fi display with intelligent multi-session encoding | |
| US11588870B2 (en) | W3C media extensions for processing DASH and CMAF inband events along with media using process@append and process@play mode | |
| US11799943B2 (en) | Method and apparatus for supporting preroll and midroll during media streaming and playback |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |