WO2023071861A1

WO2023071861A1 - Data visualization display method and apparatus, computer device, and storage medium

Info

Publication number: WO2023071861A1
Application number: PCT/CN2022/125844
Authority: WO
Inventors: 符峥; 龙良曲
Original assignee: 影石创新科技股份有限公司
Priority date: 2021-10-29
Filing date: 2022-10-18
Publication date: 2023-05-04
Also published as: CN114003160A; CN114003160B

Abstract

The present application relates to a data visualization display method and apparatus, a computer device, and a storage medium. The method comprises: in response to a video display operation of a user, displaying video data to be processed; in response to a video frame selection operation of the user, determining a target video frame in the video data to be processed, and displaying the target video frame; in response to a data analysis selection operation of the user, determining a target data analysis identifier corresponding to the target video frame; obtaining, according to the video frame identifier of the target video frame and the target data analysis identifier, a target data analysis result corresponding to the target video frame; and drawing and rendering the target data analysis result to the target video frame. By means of the method, redundancy and information blur in data visualization can be reduced.

Description

Data visualization display method, device, computer equipment and storage medium

technical field

The present application relates to the technical field of data processing, in particular to a data visualization display method, device, computer equipment and storage medium.

Background technique

With the popularization of video shooting and playback equipment, video data is gradually becoming an important carrier of information transmission. As video data contains both spatial attributes (position structure of objects in the image) and temporal attributes (image frame sequence), it is necessary to consider the relationship and change of variables in the spatial and temporal dimensions of data during analysis.

In the traditional technology, for the analysis results such as charts corresponding to the video data and the spatial structure data such as the detection frame, the corresponding layers are obtained by rendering first, and then the layers are displayed on the video data for visual display.

However, the traditional method, because the layer is directly displayed to the video data, will cause redundancy in data visualization and blurred information.

Contents of the invention

Based on this, it is necessary to address the above technical problems and provide a data visualization display method, device, computer equipment and storage medium that can reduce the redundancy and information ambiguity of data visualization.

A data visualization display method, the method comprising:

Display video data to be processed in response to user video display operations;

In response to the user's video frame selection operation, determine the target video frame in the video data to be processed, and display the target video frame;

In response to the user data analysis selection operation, determine the target data analysis identifier corresponding to the target video frame;

Acquiring a target data analysis result corresponding to the target video frame according to the video frame identifier and the target data analysis identifier of the target video frame;

Draw the target data analysis results and render to the target video frame.

In one embodiment, the data visualization display method further includes:

Obtain video data to be processed;

Perform frame extraction processing on the video data to be processed to obtain a set of video frames corresponding to the video data to be processed;

Performing information analysis on the video frames in the video frame set respectively to obtain data analysis results corresponding to the video frames;

Obtain the video frame identification of the video frame, classify the same data analysis result, generate the corresponding data analysis identification, and establish the association relationship between the video frame identification and the data analysis result according to the data analysis identification.

In one embodiment, information analysis is performed on the video frames in the video frame set, and the data analysis results corresponding to the video frames include:

Carry out target detection on the video frames in the video frame set respectively, and obtain the target detection results corresponding to the video frames, and the target detection results include detection frame information.

In one embodiment, drawing the analysis result of the target data and rendering it to the target video frame includes:

Determine the data type of the target data analysis results;

Call the graphics type according to the data type, draw and render the target data analysis results to the target video frame.

In one embodiment, calling the graphics type according to the data type, drawing and rendering the target data analysis result to the target video frame includes:

When the data type is numerical data, draw the same item data analysis results corresponding to all video frames in the video frame set as a numerical curve;

Mark the target data analysis results on the numerical curve;

Render the marked numerical curve to the target video frame.

When the data type is structured data, the drawing coordinates are determined according to the target data analysis results;

According to the drawing coordinates, draw and render the target data analysis results to the target video frame.

In one embodiment, the data visualization display method further includes:

Create a data analysis selection interactive control, which is used to respond to user data analysis selection operations.

A data visualization display device, the device comprising:

The video display module is used to respond to the user's video display operation and display the video data to be processed;

The video frame display module is used to respond to the user's video frame selection operation, determine the target video frame in the video data to be processed, and display the target video frame;

The data analysis response module is used to respond to the user data analysis selection operation and determine the target data analysis identification corresponding to the target video frame;

A data acquisition module, configured to obtain a target data analysis result corresponding to the target video frame according to the video frame identifier and the target data analysis identifier of the target video frame;

The processing module is used to draw the target data analysis result and render it to the target video frame.

A computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

Draw the target data analysis results and render to the target video frame.

A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

Draw the target data analysis results and render to the target video frame.

The above data visualization display method, device, computer equipment and storage medium display the data to be processed by responding to the video display operation of the user, respond to the video frame selection operation of the user, determine the target video frame in the video data to be processed, display the target video frame, and respond The user data analysis selection operation determines the target data analysis identifier corresponding to the target video frame, obtains the target data analysis result corresponding to the target video frame according to the video frame identifier and the target data analysis identifier of the target video frame, and performs an analysis on the target data analysis result. Draw and render to the target video frame. During the whole process, by responding to user operations, real-time display control of the data analysis results can be realized, instead of directly displaying the corresponding layer of the data analysis results to the video data, which can reduce the redundancy and complexity of data visualization. Information is vague.

Description of drawings

Fig. 1 is a schematic flow chart of a data visualization display method in an embodiment;

Figure 2 is a schematic diagram of responding to user operations in an embodiment;

Fig. 3 is a schematic flow chart of drawing target data analysis results and rendering them to target video frames in one embodiment;

Fig. 4 is a schematic flow chart of an embodiment rendering the marked numerical curve to the target video frame;

Fig. 5 is a schematic flow chart of a data visualization display method in another embodiment;

Fig. 6 is a structural block diagram of a data visualization display device in an embodiment;

Figure 7 is an internal block diagram of a computer device in one embodiment.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

In one embodiment, as shown in FIG. 1 , a data visualization display method is provided. In this embodiment, the method is applied to a terminal for example. It can be understood that this method can also be applied to a server, and can also be applied to It is based on a system including a terminal and a server, and is realized through the interaction between the terminal and the server. Among them, the terminal can be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, portable wearable devices, sports cameras, panoramic cameras or pan-tilt cameras, etc., and the server can be composed of an independent server or multiple servers server cluster to achieve. In this embodiment, the method includes the following steps:

Step 102, display the video data to be processed in response to the user's video display operation.

Wherein, the user video display operation refers to an operation in which the user wants to display a video on the terminal. The video data to be processed refers to the video data that the user wants to display on the terminal.

Specifically, when data visualization is required, the user first needs to select the video data corresponding to the data visualization. The user can interact with the terminal to perform video display operations, that is, select the video data to be displayed, and the terminal will respond to the user's video display. Operation, according to the user's video display operation, acquire and display the video data to be processed. For example, a video list is displayed on the display interface of the terminal, and the user can make a selection according to the video list, and select the video data to be displayed on the terminal.

Step 104: In response to the user's video frame selection operation, determine the target video frame in the video data to be processed, and display the target video frame.

Wherein, the user's video frame selection operation refers to an operation in which the user wants to display any video frame in the video data to be processed on the terminal. The target video frame refers to the video frame that the user wants to display on the terminal.

Specifically, after the terminal displays the video data to be processed, the user will further interact with the terminal to perform a video frame selection operation, that is, select the video frame data to be displayed, and the terminal will respond to the user's video frame selection operation, and determine according to the user's video frame selection operation The target video frame in the video data to be processed displays the target video frame. For example, when displaying video data to be processed, the terminal will synchronously display a video progress bar corresponding to the video data to be processed, and the user can select video frames by dragging the video progress bar or clicking anywhere on the video progress bar.

Step 106, in response to the user data analysis selection operation, determine the target data analysis identifier corresponding to the target video frame.

Wherein, the user data analysis selection operation refers to an operation in which the user wants to display any data analysis result on the terminal. The target data analysis identifier refers to the data analysis identifier corresponding to the data analysis result that the user wants to display on the terminal. The data analysis identifier is used to distinguish the data analysis results. Different data analysis results correspond to different data analysis identifiers. For example, the target data analysis identifier may specifically refer to a character string corresponding to the data analysis result that the user wants to display on the terminal.

Specifically, after the terminal displays the target video frame, the user will further interact with the terminal to perform a data analysis selection operation, that is, select the target data analysis identification corresponding to the data analysis result to be displayed, and the terminal will respond to the user data analysis selection operation, according to the user The data analysis selection operation determines the target data analysis identifier corresponding to the target video frame. For example, when displaying the target video frame, the terminal will simultaneously display the data analysis selection interactive control, and the user can complete the data analysis selection operation by selecting the data analysis selection interactive control.

For example, as shown in Figure 2, the terminal will display the video data to be processed by responding to the video display operation of the user. While displaying the video data to be processed, the terminal will simultaneously display the video progress bar corresponding to the video data to be processed. The video frame selection operation can be performed by dragging the video progress bar or clicking anywhere on the video progress bar. The terminal will respond to the user's video frame selection operation and display the target video frame. While displaying the target video frame, the terminal will simultaneously display the data analysis Select the interactive control, and the user can select the data analysis selection operation by selecting the data analysis selection interactive control to determine the data analysis results that need to be displayed on the terminal.

Furthermore, when the user needs to view other video frames in the video data to be processed and the corresponding data analysis results, he can reselect the target video frame by dragging the video progress bar, clicking anywhere on the video progress bar, etc., and change the data The selection of the analysis selection interactive control for data analysis selection operation. It should be noted that after re-selecting the target video frame, the user may not select the interactive control for data analysis, and the data analysis result displayed at this time is the same as the data analysis result displayed on the target video frame of the previous frame. Corresponding, belonging to the same data analysis result.

Step 108, according to the video frame identifier and the target data analysis identifier of the target video frame, obtain the target data analysis result corresponding to the target video frame.

Wherein, the video frame identifier is used to distinguish video frames. For example, the video frame identifier may specifically refer to a sequence number of a video frame. For another example, the video frame identifier may specifically refer to a time stamp corresponding to the video frame. The target data analysis result refers to the data analysis result that the user wants to display on the terminal. The data analysis result refers to the analysis of the spatial information and time information in the video frame by using an appropriate statistical analysis method. Analysis information that is useful for readers or users and can form conclusions, usually expressed in the form of numerical values or characters. For example, the data analysis result may specifically refer to global image information such as brightness values of video frames. For another example, the data analysis result may specifically refer to a target detection result of an object in a video frame detected by using a target detection method, and the target detection result includes detection frame information. For another example, the data analysis result may specifically refer to the action category information of the person in the video frame.

Specifically, the terminal will obtain the video frame identifier of the video frame in advance, and classify the same data analysis result, generate the corresponding data analysis identifier, establish the association relationship between the video frame identifier and the data analysis result according to the data analysis identifier, and determine the target After the video frame identification and target data analysis identification, the terminal will query the established association relationship according to the video frame identification and target data analysis identification of the target video frame, and obtain the target data analysis result corresponding to the target video frame.

Step 110, drawing and rendering the target data analysis result to the target video frame.

Specifically, when performing information analysis on the video frames in the video data to be processed in advance, the terminal will specify the data type of the data analysis result, and save the data type and the data analysis result together. Therefore, after obtaining the target data analysis result, the terminal can synchronously obtain the data type of the target data analysis result, and then call the corresponding graphics type according to the data type to draw and render the target data analysis result to the target video frame. Wherein, the data type includes numeric data, text data, structured data, and the like.

The above data visualization display method, by responding to the user's video display operation, displays the data to be processed, responds to the user's video frame selection operation, determines the target video frame in the video data to be processed, displays the target video frame, and responds to the user's data analysis and selection operation. The target data analysis identifier corresponding to the target video frame, according to the video frame identifier and the target data analysis identifier of the target video frame, obtains the target data analysis result corresponding to the target video frame, draws the target data analysis result and renders it to the target video frame, In the whole process, by responding to user operations, real-time display control of the data analysis results can be realized, instead of directly displaying the corresponding layer of the data analysis results to the video data, which can reduce the redundancy and information ambiguity of data visualization.

In one embodiment, the data visualization display method further includes:

Obtain video data to be processed;

Among them, the video data to be processed refers to the video data captured by the shooting equipment that can obtain digital video. The shooting equipment includes but is not limited to SLR cameras, mirrorless cameras, mobile phones with camera and video functions, sports cameras, panoramic cameras, PTZ cameras, drones, etc.

Specifically, when the video data needs to be analyzed, the terminal will obtain the video data to be processed, perform frame extraction processing on the video data to be processed, obtain a set of video frames corresponding to the video data to be processed, and apply image processing based on digital image processing. Analysis and video analysis methods, respectively analyze the information of the video frames in the video frame set to obtain the data analysis results corresponding to the video frames, and obtain the video frame identification of the video frames, and classify the same data analysis results to generate the corresponding The data analysis identifier establishes an association relationship between the video frame identifier and the data analysis result according to the data analysis identifier. Wherein, the frame extraction method of the video data to be processed may be a uniform frame extraction method, or a frame extraction method may be set as required, which is not specifically limited in this embodiment.

Among them, image analysis and video analysis methods based on digital image processing are used to analyze the information of the video frames in the video frame set, mainly to obtain global or local information of the video frames. For example, performing information analysis may specifically refer to determining global image information such as brightness values of video frames. As another example, performing information analysis may specifically refer to obtaining detection frame information of an object in a video frame using a target detection method. For another example, performing information analysis may specifically refer to determining a category of a person's action in a video frame, and the like. In this embodiment, the data analysis results are expressed in the form of numerical values or characters. For example, the brightness value can be directly expressed by numerical value, the detection frame information can be expressed by numerical coordinates, and the character action category can be expressed by character.

Among them, establishing the association relationship between the video frame identifier and the data analysis result is to generate the corresponding data analysis identifier after classifying the same data analysis result, and add the time attribute synchronized with the video frame to the data analysis result. The analysis flag stores the data analysis results added with the time attribute synchronized with the video frame, so as to facilitate the dynamic synchronous display of the video frame and the data analysis results. The video frame identifier here may specifically refer to a sequence number of a video frame or a corresponding time stamp. It should be noted that when analyzing the video data to be processed, the number of video frames to be analyzed and the type of information to be analyzed can be set according to the needs. If the information analysis is performed on all video frames in the video frame set , then directly establish the association relationship between all video frame identifiers and data analysis results. If the information analysis is not performed on all video frames in the video frame set, that is, only part of the video frames are analyzed, then after completing the information analysis on the video frames in the entire video data to be processed, when establishing the association relationship, it is necessary to The data analysis results are supplemented or interpolated so that the number of data analysis results is consistent with the number of video frame identifiers, that is, for video frames that have not been analyzed for information in the video frame set, it is also necessary to establish the video frame identifier and data corresponding to the video frame Relationships between analysis results.

It should be noted that supplementing or interpolating the data analysis results may be performed when establishing an association relationship or when displaying the data analysis results. Wherein, performing when displaying the data analysis result means that when the data analysis result is displayed on the video frame, if the corresponding data analysis result is not obtained in the association relationship according to the video frame identification corresponding to the video frame, then according to the association relationship The existing data analysis results are supplemented or interpolated to obtain the data analysis results of the video frame.

Among them, the association relationship refers to the use of structured sequence data to represent the corresponding relationship between video frame identifiers and data analysis results. Structured sequence data includes but is not limited to tables, csv (Comma-Separated Values, comma-separated values), json (JavaScript Object Notation, JS Object Notation) data, etc. The specific format is arranged in order of video frames or time, and contains the data corresponding to the data analysis results of each frame or time. Taking csv data as an example, the relationship can be expressed as: frame_id, value (frame number, value): 0, 1.000; 1, 1.000; 2, 1.234.

In this embodiment, the acquisition of the data analysis results and the association between the video frame identifier and the data analysis results can be realized, so that the alignment and accurate display can be realized when the data analysis results need to be displayed on the video frames.

Target detection is performed on the video frames in the video frame set respectively, and target detection results corresponding to the video frames are obtained, and the target detection results include detection frame information.

Wherein, the detection frame information refers to the coordinate information of the object detected in the video frame. Take a detection frame information in csv format as an example, frame_id is the frame number, bbox_id is the frame detection frame number, x1, y1 are the image coordinates of the upper left corner of the detection frame, x2, y2 are the image coordinates of the lower right corner of the detection frame. The detection frame information can be specifically expressed as: frame_id, bbox_id, x1, y1, x2, y2: 0, 0, 20, 20, 40, 30; 0, 1, 15, 15, 25, 35; 1, 2, 30 , 30, 45, 50 such a form.

Specifically, the terminal obtains a pre-trained target detection model, uses the pre-trained target detection model to perform target detection on the video frames in the video frame set, and obtains target detection results corresponding to the video frames, and the target detection results include detection frame information. Among them, the pre-trained target detection model is obtained by training the sample image data carrying the target frame. When pre-training the target detection model, the terminal will first obtain the initial target detection model, and then input the sample image data into the initial target detection model. The initial target detection model outputs the predicted detection frame corresponding to the sample image data. By comparing the predicted detection frame and the target frame, the loss function corresponding to the initial target detection model is obtained, and the model parameters of the initial target detection model are adjusted. Using the model The initial target detection model after parameter adjustment performs target detection on the sample image data again to obtain the corresponding loss function until the loss function converges to obtain the pre-trained target detection model.

In this embodiment, target detection results corresponding to the video frames can be obtained by respectively performing target detection on the video frames in the video frame set, and the target detection results include detection frame information to implement information analysis on the video frames. For example, the information analysis of the video frame can specifically mean that the target object included in the video frame and the type and location of the target object can be determined according to the target detection result, so that the corresponding target detection can be displayed when the video frame is displayed. result.

Determine the data type of the target data analysis results;

Specifically, when performing information analysis on the video frames in the video data to be processed in advance, the terminal will specify the data type of the data analysis result, and save the data type and the data analysis result together. Therefore, after obtaining the target data analysis result, the terminal can synchronously obtain the data type of the target data analysis result, and then call the corresponding graphics type according to the data type to draw and render the target data analysis result to the target video frame. Wherein, the data type includes numeric data, text data, structured data, and the like. Further, when drawing and rendering, different graphic colors and fonts may be used to identify different target data analysis results.

Specifically, the terminal can use the programmable GUI (Graphical User Interface, Graphical User Interface) library to draw the analysis results of the target data and render them to the target video frame. , GUI code library for operation, it can draw target data analysis results into specific shapes, charts or text according to data types, and render them on the target video frame. For example, a schematic flow chart of drawing target data analysis results and rendering them to target video frames may be shown in FIG. 3 . The program in the terminal will read the analysis result data (that is, the target data analysis result), and call the GUI library (that is, the programmable GUI library) corresponding to the chart or graph type according to the data type specified in the analysis result file, so that the GUI library calls the system graphics The interface interface draws and renders analysis result charts, graphics and target video frames, and displays the rendering results on the system graphical interface of the terminal.

In this embodiment, by determining the data type of the target data analysis result and calling the graphic type according to the data type, the target data analysis result can be drawn and rendered to the target video frame.

Mark the target data analysis results on the numerical curve;

Render the marked numerical curve to the target video frame.

Specifically, when the data type is numerical data, the terminal will draw the same item data analysis results corresponding to all video frames in the video frame set as a numerical curve, mark the target data analysis results on the numerical curve, and draw the marked numerical curve Render to target video frame. For example, when the data analysis result is the brightness value of a video frame, its data type is numerical data. When it is necessary to display the corresponding brightness value on the target video frame, the terminal will first draw the corresponding brightness values of all video frames is a numerical curve, and then mark the brightness value corresponding to the target video frame on the numerical curve, and render the marked numerical curve to the target video frame.

For example, the flowchart of rendering the marked numerical curve to the target video frame can be shown in Figure 4. Here, for the convenience of display, the marked numerical curve and the target video frame are arranged in the form of layout instead of layer superposition It can be seen from Figure 4 that the numerical curve can be drawn by using the same item data analysis results corresponding to all video frames. For the left video frame and the right video frame, the corresponding target data analysis results are marked on the numerical curve , where the target data analysis result corresponding to the left video frame is 0.9, and the target data analysis result corresponding to the right video frame is 0.2.

In this embodiment, when the data type is numerical data, the same item data analysis results corresponding to all video frames in the video frame set are drawn as numerical curves, and the target data analysis results are marked on the numerical curves, and the marked numerical values The curve is rendered to the target video frame, which can realize the visual display of numerical data.

Specifically, when the data type is structured data, the terminal determines the drawing coordinates according to the target data analysis result, and draws and renders the target data analysis result to the target video frame according to the drawing coordinates. For example, when the data analysis result is the detection frame information of the video frame, its data type is structured data, and when the corresponding detection frame information needs to be displayed on the target video frame, the terminal will determine the drawing coordinates according to the detection frame information, That is, the coordinates of the detection frame, and then draw and render the detection frame to the target video frame according to the coordinates of the detection frame.

In this embodiment, when the data type is structured data, the drawing coordinates are determined according to the analysis results of the target data, and the analysis results of the target data are drawn and rendered to the target video frame according to the drawing coordinates, so that the visual display of the structured data can be realized.

In one embodiment, the data visualization display method further includes:

Specifically, the terminal will create data analysis and selection interactive controls through the programmable GUI library. The data analysis and selection interactive controls correspond to the number of data analysis results and are used to respond to user data analysis and selection operations. Users can use data analysis to select interactive controls in real time. Which data analysis results are displayed on the video frame. Furthermore, the terminal will also create a data type selection interactive control to respond to the user's data type selection operation, and control the data analysis results of which data types are displayed on the video frame in real time.

In this embodiment, by creating an interactive control for data analysis and selection, the interactive control for data analysis and selection can be used to respond to the user's data analysis and selection operation, thereby realizing real-time control of data visualization.

In one embodiment, as shown in Figure 5, a schematic flow chart is used to illustrate the data visualization display method of the present application, and the data visualization display method specifically includes the following steps:

Step 502, acquiring video data to be processed;

Step 504, performing frame extraction processing on the video data to be processed to obtain a set of video frames corresponding to the video data to be processed;

Step 506, respectively performing information analysis on the video frames in the video frame set to obtain data analysis results corresponding to the video frames;

Step 508, obtaining the video frame identifier of the video frame, and classifying the same data analysis result, generating a corresponding data analysis identifier, and establishing a relationship between the video frame identifier and the data analysis result according to the data analysis identifier;

Step 510, responding to the user's video display operation, displaying the video data to be processed;

Step 512, responding to the user's video frame selection operation, determining the target video frame in the video data to be processed, and displaying the target video frame;

Step 514, creating a data analysis selection interactive control, which is used to respond to the user's data analysis selection operation;

Step 516, in response to the user data analysis selection operation, determine the target data analysis identifier corresponding to the target video frame;

Step 518, according to the video frame identifier and the target data analysis identifier of the target video frame, obtain the target data analysis result corresponding to the target video frame;

Step 520, determine the data type of the target data analysis result, when the data type is numerical data, jump to step 522, and when the data type is structural data, jump to step 528;

Step 522, drawing the same item data analysis results corresponding to all video frames in the video frame set as a numerical curve;

Step 524, marking the target data analysis results on the numerical curve;

Step 526, rendering the marked numerical curve to the target video frame;

Step 528, determine the drawing coordinates according to the target data analysis results;

Step 530, draw and render the target data analysis result to the target video frame according to the drawing coordinates.

It should be understood that although the steps in the flow charts involved in the above embodiments are shown sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flow charts involved in the above embodiments may include multiple steps or stages, and these steps or stages are not necessarily executed at the same time, but may be executed at different times. These steps Or the execution sequence of the stages is not necessarily performed sequentially, but may be executed in turn or alternately with other steps or at least a part of steps or stages in other steps.

In one embodiment, as shown in FIG. 6 , a data visualization display device is provided, including: a video display module 602, a video frame display module 604, a data analysis response module 606, a data acquisition module 608 and a processing module 610, wherein :

Video display module 602, used to respond to user video display operation, display video data to be processed;

The video frame display module 604 is used to respond to the user's video frame selection operation, determine the target video frame in the video data to be processed, and display the target video frame;

The data analysis response module 606 is used to respond to the user data analysis selection operation and determine the target data analysis identification corresponding to the target video frame;

A data acquisition module 608, configured to acquire a target data analysis result corresponding to the target video frame according to the video frame identifier and the target data analysis identifier of the target video frame;

The processing module 610 is configured to draw the analysis result of the target data and render it to the target video frame.

The above data visualization display device displays the data to be processed by responding to the user's video display operation, responds to the user's video frame selection operation, determines the target video frame in the video data to be processed, displays the target video frame, responds to the user's data analysis and selection operation, determines and The target data analysis identifier corresponding to the target video frame, according to the video frame identifier and the target data analysis identifier of the target video frame, obtains the target data analysis result corresponding to the target video frame, draws the target data analysis result and renders it to the target video frame, In the whole process, by responding to user operations, real-time display control of the data analysis results can be realized, instead of directly displaying the corresponding layer of the data analysis results to the video data, which can reduce the redundancy and information ambiguity of data visualization.

In one embodiment, the data visualization display device further includes an analysis module, the analysis module is used to obtain the video data to be processed, perform frame extraction processing on the video data to be processed, obtain a set of video frames corresponding to the video data to be processed, and analyze the video frames respectively Perform information analysis on the video frames in the collection, obtain the data analysis results corresponding to the video frames, obtain the video frame identifiers of the video frames, and classify the same data analysis results, generate corresponding data analysis identifiers, and establish video frames based on the data analysis identifiers Identify the association relationship with the data analysis results.

In one embodiment, the analysis module is further configured to perform target detection on the video frames in the video frame set to obtain target detection results corresponding to the video frames, and the target detection results include detection frame information.

In one embodiment, the processing module is further configured to determine the data type of the target data analysis result, call the graphics type according to the data type, and draw and render the target data analysis result to the target video frame.

In one embodiment, the processing module is also used to draw the same item data analysis results corresponding to all video frames in the video frame set as a numerical curve when the data type is numerical data, and mark the target data analysis result on the numerical curve, Render the marked numerical curve to the target video frame.

In one embodiment, the processing module is further configured to determine the drawing coordinates according to the target data analysis results when the data type is structured data, and draw and render the target data analysis results to the target video frame according to the drawing coordinates.

In one embodiment, the data visualization display device further includes a creation module, the creation module is used to create a data analysis selection interaction control, and the data analysis selection interaction control is used to respond to the user's data analysis selection operation.

For specific embodiments of the data visualization display device, reference may be made to the above-mentioned embodiments of the data visualization display method, which will not be repeated here. Each module in the above-mentioned data visualization display device can be fully or partially realized by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.

In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure may be as shown in FIG. 7 . The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, an operator network, NFC (Near Field Communication) or other technologies. When the computer program is executed by the processor, a data visualization display method is realized. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball or a touch pad provided on the casing of the computer device , and can also be an external keyboard, touchpad, or mouse.

Those skilled in the art can understand that the structure shown in Figure 7 is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation to the computer equipment on which the solution of this application is applied. The specific computer equipment can be More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.

In one embodiment, a computer device is provided, including a memory and a processor, a computer program is stored in the memory, and the processor implements the following steps when executing the computer program:

Draw the target data analysis results and render to the target video frame.

In one embodiment, when the processor executes the computer program, the following steps are also implemented: acquiring video data to be processed, performing frame extraction processing on the video data to be processed, obtaining a video frame set corresponding to the video data to be processed, Perform information analysis on the video frame to obtain the data analysis result corresponding to the video frame, obtain the video frame identification of the video frame, and classify the same data analysis result, generate the corresponding data analysis identification, and establish the video frame identification and The relationship between data analysis results.

In one embodiment, when the processor executes the computer program, the following steps are further implemented: respectively performing object detection on the video frames in the video frame set to obtain object detection results corresponding to the video frames, the object detection results including detection frame information.

In one embodiment, when the processor executes the computer program, the following steps are further implemented: determining the data type of the target data analysis result, calling the graphics type according to the data type, drawing and rendering the target data analysis result to the target video frame.

In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the data type is numerical data, the same item data analysis results corresponding to all video frames in the video frame set are drawn as a numerical curve, and on the numerical curve Mark the target data analysis results, and render the marked numerical curve to the target video frame.

In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the data type is structured data, determine the drawing coordinates according to the target data analysis results, and draw and render the target data analysis results to the target video frame according to the drawing coordinates .

In one embodiment, when the processor executes the computer program, the following steps are further implemented: creating an interactive control for data analysis selection, and the interactive control for data analysis selection is used for responding to a user's data analysis selection operation.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

Draw the target data analysis results and render to the target video frame.

In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: acquiring the video data to be processed, performing frame extraction processing on the video data to be processed, obtaining a set of video frames corresponding to the video data to be processed, and separately processing the set of video frames Perform information analysis on the video frame, obtain the data analysis result corresponding to the video frame, obtain the video frame identification of the video frame, and classify the same data analysis result, generate the corresponding data analysis identification, and establish the video frame identification according to the data analysis identification Correlation with data analysis results.

In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: respectively performing target detection on the video frames in the video frame set to obtain target detection results corresponding to the video frames, and the target detection results include detection frame information.

In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: determining the data type of the target data analysis result, calling the graphics type according to the data type, drawing and rendering the target data analysis result to the target video frame.

In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the data type is numerical data, the same item data analysis results corresponding to all video frames in the video frame set are drawn as a numerical curve, and the numerical curve Mark the target data analysis results, and render the marked numerical curve to the target video frame.

In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the data type is structured data, determine the drawing coordinates according to the target data analysis results, draw and render the target data analysis results to the target video according to the drawing coordinates frame.

In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: creating an interactive control for data analysis selection, and the interactive control for data analysis selection is used for responding to a user's data analysis selection operation.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware, and the computer programs can be stored in a non-volatile computer-readable memory In the medium, when the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any references to memory, storage, database or other media used in the various embodiments provided in the present application may include at least one of non-volatile memory and volatile memory. Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

The technical features of the above embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered to be within the range described in this specification.

The above-mentioned embodiments only represent several implementation modes of the present application, and the description thereof is relatively specific and detailed, but it should not be construed as limiting the scope of the patent for the invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of protection of the patent application should be based on the appended claims.

Claims

A data visualization display method, characterized in that the method comprises:

Display video data to be processed in response to user video display operations;

In response to the user's video frame selection operation, determine the target video frame in the video data to be processed, and display the target video frame;

In response to the user data analysis selection operation, determine the target data analysis identifier corresponding to the target video frame;

Acquiring a target data analysis result corresponding to the target video frame according to the video frame identifier of the target video frame and the target data analysis identifier;

Drawing and rendering the analysis result of the target data to the target video frame.
The method according to claim 1, further comprising:

Obtain video data to be processed;

performing frame extraction processing on the video data to be processed to obtain a set of video frames corresponding to the video data to be processed;

Performing information analysis on the video frames in the video frame set respectively, to obtain data analysis results corresponding to the video frames;

Acquire the video frame ID of the video frame, classify the same data analysis result, generate a corresponding data analysis ID, and establish an association relationship between the video frame ID and the data analysis result according to the data analysis ID.
The method according to claim 2, wherein said performing information analysis on the video frames in the video frame set respectively, and obtaining the data analysis results corresponding to the video frames comprises:

Target detection is performed on the video frames in the video frame set respectively to obtain target detection results corresponding to the video frames, and the target detection results include detection frame information.
The method according to claim 1, wherein said drawing and rendering the target data analysis result to the target video frame comprises:

Determine the data type of the target data analysis result;

Calling a graphic type according to the data type, drawing and rendering the target data analysis result to the target video frame.
The method according to claim 4, wherein calling a graphics type according to the data type, drawing and rendering the target data analysis result to the target video frame comprises:

When the data type is numerical data, the same item data analysis results corresponding to all video frames in the video frame set are drawn as a numerical curve;

marking the target data analysis result on the numerical curve;

Rendering the marked numerical curve to the target video frame.
The method according to claim 4, wherein calling a graphics type according to the data type, drawing and rendering the target data analysis result to the target video frame comprises:

When the data type is structural data, determine the drawing coordinates according to the analysis result of the target data;

Drawing and rendering the target data analysis result to the target video frame according to the drawing coordinates.
The method according to claim 1, further comprising:

A data analysis selection interaction control is created, and the data analysis selection interaction control is used for responding to a user's data analysis selection operation.
A data visualization display device, characterized in that the device comprises:

The video display module is used to respond to the user's video display operation and display the video data to be processed;

The video frame display module is used to respond to the user's video frame selection operation, determine the target video frame in the video data to be processed, and display the target video frame;

The data analysis response module is used to respond to the user data analysis selection operation and determine the target data analysis identification corresponding to the target video frame;

A data acquisition module, configured to acquire a target data analysis result corresponding to the target video frame according to the video frame identifier of the target video frame and the target data analysis identifier;

A processing module, configured to draw the analysis result of the target data and render it to the target video frame.
A computer device, comprising a memory and a processor, the memory stores a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.
A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are realized.