CN116489336A

CN116489336A - Equipment monitoring method, device, equipment, medium and product based on virtual film production

Info

Publication number: CN116489336A
Application number: CN202211491783.9A
Authority: CN
Inventors: 李锐
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-11-25
Filing date: 2022-11-25
Publication date: 2023-07-25

Abstract

The application discloses a device monitoring method, device, equipment, medium and product based on virtual film making, and relates to the technical field of virtual film making. The method comprises the following steps: acquiring running state data and running video data respectively corresponding to a plurality of hardware devices; matching the operation state data of the hardware equipment with the monitoring index of the equipment type corresponding to the hardware equipment, and displaying a matching result between the operation state data of the hardware equipment and the monitoring index; displaying a video picture corresponding to the operation video data of the hardware equipment; and carrying out data statistical analysis on the operation performance data of the hardware equipment, and displaying the statistical analysis result of the operation performance data in a chart form. The hardware monitoring system has the advantages that the automation of data detection of all equipment of a shooting site of the virtual production in terms of hardware is realized, the monitoring efficiency of the hardware equipment in the shooting site of the virtual production is improved, and the workload of site workers is reduced.

Description

Equipment monitoring method, device, equipment, medium and product based on virtual film production

Technical Field

The embodiment of the application relates to the technical field of virtual production, in particular to a device monitoring method, a device, equipment, a medium and a product based on virtual production.

Background

Virtual production refers to a series of computer-aided production and visual film production methods, and at the shooting site of virtual production, a light emitting diode (Light Emitting Diode, LED) screen is used for displaying virtual content, and in front of the LED screen, there is an actual scene making prop, and a camera in virtual production can shoot the LED screen and the screen foreground scene at the same time, so as to obtain video of a picture fused with the LED screen and the screen foreground scene. The hardware involved in the shooting process of virtual production is many, for example: and rendering the collected fusion video in real time by using a rendering machine at the virtual film-making shooting site.

In the related art, after confirming that the renderer can work normally, a worker can operate the renderer to render the collected fusion video in real time, and after shooting is finished, the worker can check the video material obtained by rendering.

However, because the rendering machine is rendered in real time, the staff can not find the problem of the rendering machine in time in the shooting process, and the video material obtained by rendering can be checked only when in post-processing, if the problem occurs in the video material at this time, the video needs to be shot again, more manpower and material resources are needed, and the shooting efficiency of the virtual film production is lower.

Disclosure of Invention

The embodiment of the application provides a device monitoring method, device, equipment, medium and product based on virtual film making, which improves the monitoring efficiency of hardware equipment in a shooting site of virtual film making, and the technical scheme is as follows:

in one aspect, a device monitoring method based on virtual production is provided, the method comprising:

acquiring running state data and running video data respectively corresponding to a plurality of hardware devices, wherein the plurality of hardware devices are devices positioned on a virtual production shooting site, the plurality of hardware devices are used for building and collecting shooting scenes of the virtual production shooting site, the plurality of hardware devices correspond to at least two device types, the running state data comprise running performance data, and the running performance data are used for representing the hardware performance of the hardware devices in running;

matching the operation state data of the hardware equipment with the monitoring index of the equipment type corresponding to the hardware equipment, and displaying a matching result between the operation state data of the hardware equipment and the monitoring index, wherein the monitoring index is used for indicating the operation condition which needs to be met by the hardware equipment corresponding to the equipment type in the operation process, and the matching result is used for indicating the operation fault condition corresponding to the hardware equipment of the virtual film-making shooting site;

Displaying a video picture corresponding to the operation video data of the hardware equipment, wherein the video picture refers to a working scene picture of the hardware equipment at the virtual film-making shooting site;

and carrying out data statistical analysis on the operation performance data of the hardware equipment, and displaying the statistical analysis result of the operation performance data in a chart form.

In another aspect, there is provided a device for monitoring equipment based on virtual production, the device comprising:

the system comprises an acquisition module, a control module and a control module, wherein the acquisition module is used for acquiring running state data and running video data respectively corresponding to a plurality of hardware devices, the plurality of hardware devices are devices positioned on a virtual production shooting site, the plurality of hardware devices are used for constructing and acquiring shooting scenes of the virtual production shooting site, the plurality of hardware devices correspond to at least two device types, the running state data comprise running performance data, and the running performance data are used for representing the hardware performance of the hardware devices in running;

the first display module is used for matching the operation state data of the hardware equipment with the monitoring index of the equipment type corresponding to the hardware equipment, displaying a matching result between the operation state data of the hardware equipment and the monitoring index, wherein the monitoring index is used for indicating the operation condition which needs to be met by the hardware equipment corresponding to the equipment type in the operation process, and the matching result is used for indicating the operation fault condition corresponding to the hardware equipment of the virtual film-making shooting site;

The first display module is further configured to display a video frame corresponding to operation video data of the hardware device, where the video frame refers to a working scene frame of the hardware device at the virtual production shooting site;

the first display module is further configured to perform data statistics analysis on the operation performance data of the hardware device, and display a result of the statistics analysis on the operation performance data in a chart form.

In another aspect, a method for monitoring equipment based on virtual production is provided, the method comprising:

receiving device monitoring operation of a plurality of hardware devices, wherein the plurality of hardware devices are devices positioned on a virtual production shooting site, the plurality of hardware devices are used for building and collecting shooting scenes of the virtual production shooting site, and the plurality of hardware devices correspond to at least two device types;

responding to the equipment monitoring operation, displaying a matching result between the running state data of the hardware equipment and the monitoring index of the equipment type corresponding to the hardware equipment, wherein the monitoring index is used for indicating the running condition which needs to be met by the hardware equipment corresponding to the equipment type in the running process, the matching result is used for indicating the running fault condition corresponding to the hardware equipment of the virtual film-making shooting site, the running state data comprises running performance data, and the running performance data is used for representing the hardware performance of the hardware equipment in the running process;

Displaying a video picture corresponding to the operation video data of the hardware equipment, wherein the video picture refers to a working scene of the hardware equipment at the virtual film-making shooting site;

and in response to receiving an analysis operation of the operation performance data of the hardware equipment, displaying a statistical analysis result of the operation performance data in a graph form, wherein the analysis operation is used for carrying out data statistical analysis on the specified operation state data.

the device monitoring system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving device monitoring operation of a plurality of hardware devices, the plurality of hardware devices are devices positioned on a virtual production shooting site, the plurality of hardware devices are used for building and collecting shooting scenes of the virtual production shooting site, and the plurality of hardware devices correspond to at least two device types;

the second display module is used for responding to the equipment monitoring operation, displaying a matching result between the running state data of the hardware equipment and the monitoring index of the equipment type corresponding to the hardware equipment, wherein the monitoring index is used for indicating the running condition which needs to be met by the hardware equipment corresponding to the equipment type in the running process, the matching result is used for indicating the running fault condition corresponding to the hardware equipment of the virtual film-making shooting site, the running state data comprises running performance data, and the running performance data is used for representing the hardware performance of the hardware equipment in running;

The second display module is further configured to display a video frame corresponding to the operation video data of the hardware device, where the video frame refers to a working scene of the hardware device at the virtual production shooting site;

the second display module is further configured to graphically display a statistical analysis result of the operation performance data in response to receiving an analysis operation of the operation performance data of the hardware device, where the analysis operation is used to perform a data statistical analysis on the specified operation state data.

In another aspect, a computer device is provided, the computer device comprising a processor and a memory having stored therein at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, code set, or instruction set being loaded and executed by the processor to implement a virtual production-based device monitoring method as in any of the above embodiments.

In another aspect, a computer readable storage medium having stored therein at least one instruction, at least one program, code set, or instruction set loaded and executed by a processor to implement a virtual production-based device monitoring method as in any of the above embodiments is provided.

In another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the virtual production-based device monitoring method described in any one of the above embodiments.

The beneficial effects that technical scheme that this application embodiment provided include at least:

on the one hand, by matching the running state data of various hardware devices with the monitoring indexes of the device types corresponding to the hardware devices, displaying the matching result between the running state data and the monitoring indexes of the hardware devices in the monitoring terminal device and displaying the working scene pictures of the various hardware devices on the virtual production site, a worker can overall manage the various hardware devices, remotely monitor the various hardware devices, realize the automation of data detection of all devices on the hardware aspect of the shooting site of the virtual production, improve the monitoring efficiency of the hardware devices on the shooting site of the virtual production, and reduce the workload of the field worker; on the other hand, by carrying out data statistics analysis on the operation performance data of the hardware equipment, the statistics analysis result of the operation performance data is displayed in a chart form, the performance of the hardware equipment in the operation process is displayed more intuitively, the current operation condition of the hardware equipment is conveniently known in real time by staff, and the possible hardware operation problem is solved in time.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a device monitoring system provided in an exemplary embodiment of the present application;

FIG. 2 is a flow chart of a virtual production-based device monitoring method provided in one exemplary embodiment of the present application;

FIG. 3 is an interface diagram of a real-time display of operational status data provided by an exemplary embodiment of the present application;

FIG. 4 is a flowchart of a virtual production-based equipment monitoring method provided in another exemplary embodiment of the present application;

FIG. 5 is an interactive flow chart of a virtual production-based device monitoring method provided in one exemplary embodiment of the present application;

FIG. 6 is a flow chart of a virtual production-based equipment monitoring method provided in accordance with yet another exemplary embodiment of the present application;

FIG. 7 is a schematic diagram of an interface of a hardware status check module in device monitoring software according to an exemplary embodiment of the present application;

FIG. 8 is a schematic diagram of an interface of a hardware real-time status module in device monitoring software according to an exemplary embodiment of the present application;

FIG. 9 is a schematic diagram of an interface of a data alert module in device monitoring software according to an exemplary embodiment of the present application;

FIG. 10 is a schematic interface diagram of a history log module in device monitoring software according to one exemplary embodiment of the present application;

FIG. 11 is a schematic interface diagram of a video-in-place monitoring module in device monitoring software provided in an exemplary embodiment of the present application;

FIG. 12 is a schematic interface diagram of a performance data collection module in device monitoring software provided in an exemplary embodiment of the present application;

FIG. 13 is a block diagram of a virtual production-based equipment monitoring device according to an exemplary embodiment of the present application;

FIG. 14 is a block diagram of a virtual production-based equipment monitoring device provided in accordance with another exemplary embodiment of the present application;

fig. 15 is a block diagram of a monitoring end device according to an exemplary embodiment of the present application.

Detailed Description

For the purposes of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

The terms "first," "second," and the like in this application are used for distinguishing between similar elements or items having substantially the same function and function, and it should be understood that there is no logical or chronological dependency between the terms "first," "second," and no limitation on the amount or order of execution.

Virtual production refers to a series of computer-aided production and visual film production methods, and at the shooting site of virtual production, a light emitting diode (Light Emitting Diode, LED) screen is used for displaying virtual content, and in front of the LED screen, there is an actual scene making prop, and a camera in virtual production can shoot the LED screen and the screen foreground scene at the same time, so as to obtain video of a picture fused with the LED screen and the screen foreground scene. The hardware involved in the shooting process of virtual production is many, for example: and rendering the collected fusion video in real time by using a rendering machine at the virtual film-making shooting site. In the related art, after confirming that the renderer can work normally, a worker can operate the renderer to render the collected fusion video in real time, and after shooting is finished, the worker can check the video material obtained by rendering. However, because the rendering machine is rendered in real time, the staff can not find the problem of the rendering machine in time in the shooting process, and the video material obtained by rendering can be checked only when in post-processing, if the problem occurs in the video material at this time, the video needs to be shot again, more manpower and material resources are needed, and the shooting efficiency of the virtual film production is lower.

An embodiment of the present application provides a device monitoring method based on virtual production, and referring to fig. 1 schematically, a schematic diagram of a device monitoring system provided by an exemplary embodiment is shown. The device monitoring system includes a monitoring end device 110, a plurality of hardware devices, and a server 130. The computer device 100 and the server 130 are connected through a wired network or a wireless network, and the plurality of hardware devices and the server 130 are connected through a wired network or a wireless network.

Optionally, the plurality of hardware devices 120 includes: a hardware device such as a renderer 121, an LED processor 122, an operator 123, a shooting camera 124, and a monitoring camera 125 for constructing and collecting a shooting scene of a virtual production shooting site. Optionally, a first application program is installed in each piece of hardware, which may be a conventional application program, a cloud application program, an applet or an application module in a host application program, or a web platform, which is not limited in this embodiment. The first application program is provided with data collection and transmission functions for collecting the running state data of the hardware device and sending the running state data to the monitoring end device 110.

Optionally, a second application program is installed in the monitoring end device 110, and the second application program is provided with a data display and detection function, and is used for displaying the acquired running state data and running video data of the hardware device; and meanwhile, analyzing the running state data and displaying an analysis result.

Server 130 provides background services for monitoring end device 110 and multiple hardware devices 120. Optionally, the server 130 includes a data server 131 and a network (WEB) server 132.

Illustratively, as shown in fig. 1, the shooting site further includes a first switch 140, where each of the above hardware devices sends its corresponding operation status data to the first switch 140, and the first switch 140 sends the operation status data to the data server 131; the data server 131 receives the operation state data and forwards the operation state data to the web server 132, and the web server 132 sends the operation state data to the monitoring end device 110. After the monitoring end device 110 receives the operation state data, the operation state data may be displayed on a screen of the device.

Optionally, after receiving the operation state data, the monitoring end device 110 may match the operation state data of the hardware device based on the monitoring index, and display a matching result between the operation state data of the hardware device and the monitoring index on a screen of the device; optionally, the running state data of the hardware device further includes running performance data, where the running performance data is used to represent the hardware performance of the hardware device when running, and after the monitoring end device 110 receives the running performance data, the running performance data of the hardware device may be subjected to data statistical analysis, and a result of the statistical analysis on the running performance data is displayed in a graph form.

In some alternative embodiments, the first application program is further configured to collect video data of the hardware device and send the video data to the monitoring end device 110. Wherein the video data comprises: (1) A screen display of the operator 123 for recording a software operation process in the shooting process; (2) Capturing a material captured by the camera 124 (i.e., a video for producing a movie); (3) The operating scene of each hardware device photographed by the camera 125 is monitored.

Illustratively, as shown in fig. 1, the shooting site further includes a second switch 150 and a push device 160, and the server 130 further includes a video server 133. The manipulator 123, the capturing camera 124, and the monitoring camera 125 transmit their respective corresponding video data to the streaming device 160, the streaming device 160 pushes the video data to the second switch 150, and the second switch 150 transmits the video data to the video server 133; the video server 133 receives the video data and forwards the video data to the web server 132, and the web server 132 transmits the video data to the monitoring end device 110. After the monitoring end device 110 receives the video data, a video picture corresponding to the video data can be displayed on a screen of the device.

It should be noted that the server 130 can be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and can also be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs (Content Delivery Network, content delivery networks), and basic cloud computing services such as big data and artificial intelligence platforms.

Cloud Technology (Cloud Technology) refers to a hosting Technology that unifies serial resources such as hardware, software, network and the like in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The cloud technology is based on the general names of network technology, information technology, integration technology, management platform technology, application technology and the like applied by the cloud computing business model, can form a resource pool, and is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing. Optionally, the server 130 may also be implemented as a node in a blockchain system.

It should be noted that, before collecting relevant data (e.g., running state data and running video data) of a user and during the process of collecting relevant data of a user, a prompt interface, a popup window or output voice prompt information may be displayed, where the prompt interface, the popup window or the voice prompt information is used to prompt the user to collect relevant data currently, so that the present application only starts to execute the relevant step of obtaining relevant data of the user after obtaining the confirmation operation of the user on the prompt interface or the popup window, otherwise (i.e., when the confirmation operation of the user on the prompt interface or the popup window is not obtained), ends the relevant step of obtaining relevant data of the user, i.e., does not obtain relevant data of the user. In other words, all user data collected in the present application is collected with the consent and authorization of the user, and the collection, use and processing of relevant user data requires compliance with relevant laws and regulations and standards of the relevant country and region.

In connection with the above description, the method for monitoring equipment based on virtual production according to the embodiment of the present application is described, and fig. 2 shows a flowchart of the method for monitoring equipment based on virtual production according to an exemplary embodiment of the present application, where the method is applied to the monitoring end device 110 shown in fig. 1, and is described by taking as an example, the method includes:

Step 201, acquiring running state data and running video data respectively corresponding to a plurality of hardware devices.

The hardware devices are devices positioned on the virtual production shooting site, the hardware devices are used for building and collecting shooting scenes of the virtual production shooting site, the hardware devices correspond to at least two device types, and the running state data comprise running performance data which are used for representing the hardware performance of the hardware devices in running.

Optionally, the plurality of hardware devices includes at least two of a renderer, an LED processor, an operator, a camera for shooting, a camera for monitoring, and the like, each hardware device corresponding to one device type.

The rendering machine is a computer device provided with a rendering engine and is used for processing rendering tasks in shooting process, for example: rendering a picture required for shooting on an LED display screen of a virtual film-making site; the LED processor is used for converting the image into a signal which can be received by the LED display screen; various software control systems are installed in the operating machine and are used for controlling on-site equipment, for example: controlling the light of a shooting site through a light control system in the operating machine; the shooting camera is used for shooting films; the monitoring camera is used for shooting the working scene of each device on site and the manual operation scene of staff on the device.

Optionally, the operation state data corresponding to the hardware device is used to indicate the operation condition of each hardware component in the hardware device, and each hardware device has its corresponding operation state data. Illustratively, for a renderer and an operator, the run state data includes: central processing unit (Central Processing Unit, CPU) occupancy, CPU temperature, graphics processor (Graphics Processing Unit, GPU) occupancy, GPU temperature, frame rate, synchronization signals, hard disk conditions, memory occupancy, process focus, graphics card resolution, graphics card color depth, graphics card color exposure values, and the like. For an LED processor, the operational status data includes: input signal resolution, output signal resolution, input signal frame rate, output signal frame rate, color gamut setting, color space, exposure parameters, brightness setting, gamma setting, color temperature, display status, LED shutter refresh rate, LED synchronization status, LED phase offset, etc. For a camera, the operational status data includes: shooting status, machine hard disk condition, shooting resolution, shooting frame rate, shooting color gamut, camera power, time code, synchronization status, color temperature, aperture, etc.

Optionally, the method for the monitoring end device to obtain the running state data corresponding to each of the plurality of hardware devices includes at least one of the following methods:

1. and acquiring running state data corresponding to the plurality of hardware devices respectively in response to receiving the device monitoring operation.

The device monitoring operation is an operation performed in a monitoring end device, and illustratively, a target application program is installed in the monitoring end device and is used for detecting the running states of a plurality of hardware devices, a staff on a shooting site clicks a control for starting detection on the target application program, and the monitoring end device sends a data acquisition instruction to a server; the server transmits the instruction to each hardware device, each hardware device starts to collect real-time running state data and transmits the real-time running state data to the server, and the server transmits the obtained running state data to the monitoring end device.

2. And acquiring running state data corresponding to the plurality of hardware devices respectively based on a preset time interval.

Optionally, the plurality of hardware devices send respective corresponding real-time running state data to the server at regular time, and the server sends the acquired running state data to the monitoring end device. Optionally, the plurality of hardware devices send real-time operation state data (for example, once every 5 seconds) to the server according to fixed time intervals, and the server sends the acquired operation state data to the monitoring end device.

3. And responding to the change of the specified running state data corresponding to the hardware equipment, and acquiring the specified running state data corresponding to the hardware equipment.

Optionally, the plurality of hardware devices periodically send respective corresponding real-time operation state data to the server, the server analyzes the operation state data, and if the designated operation state data is changed, the designated operation state data is sent to the monitoring end device. Illustratively, the camera 1 sends the acquired shooting resolution, shooting frame rate, camera electric quantity and the like to the server at regular time, when the camera 1 starts to operate, the shooting resolution acquired by the monitoring terminal equipment is 1920 x 1080, the shooting frame rate is 30 frames per second, and the camera electric quantity is 100%; after the camera 1 operates for 5 minutes, the shooting resolution 1920 x 1080, the shooting frame rate 30 frames per second, the camera electric quantity 99% and the like acquired at the moment are sent to the server, the server analyzes according to the acquired data to obtain that other operation state data of the camera 1 except the camera electric quantity are unchanged, the camera electric quantity 99% is sent to the monitoring end equipment, and the monitoring end equipment updates the camera electric quantity data of the camera 1 to 99%.

It should be noted that, the above method for acquiring the running state data corresponding to each of the plurality of hardware devices is merely an illustrative example, and the embodiment of the present application is not limited thereto.

The operation video data are used for indicating the working scene of the hardware equipment in the virtual production shooting site.

Optionally, the method for the monitoring end device to obtain the running video data corresponding to each of the plurality of hardware devices includes at least one of the following methods:

1. and responding to the received device monitoring operation, and acquiring operation video data corresponding to the plurality of hardware devices respectively.

Optionally, after receiving the device monitoring operation on the monitoring end device, the monitoring end device sends an operation video data acquisition instruction to the server, and after receiving the instruction, the server sends an operation video data acquisition request to an operator and a camera (including a monitoring camera and a shooting camera) of the virtual production site, and the server acquires the obtained operation video data and forwards the operation video data to the monitoring end device.

Optionally, the operation machine and the camera (including the monitoring camera and the shooting camera) of the virtual production site send the operation video data to the server at regular time, and the server sends the obtained operation video data to the monitoring end device. Optionally, the plurality of hardware devices send real-time operation video data (for example, once every 5 seconds) to the server according to fixed time intervals, and the server sends the obtained operation video data to the monitoring end device.

Step 202, matching the operation state data of the hardware device with the monitoring index of the device type corresponding to the hardware device, and displaying the matching result between the operation state data of the hardware device and the monitoring index.

The monitoring index is used for indicating the operation condition which needs to be met by the hardware equipment corresponding to the equipment type in the operation process, and the matching result is used for indicating the operation fault condition corresponding to the hardware equipment of the virtual film-making shooting site.

Optionally, the operation condition may be a reference operation parameter corresponding to a hardware component of the hardware device during operation; or the reference operation parameter range corresponding to the hardware component in the operation process of the hardware equipment; or the reference running state corresponding to the hardware component in the running process of the hardware device (for example, the reference running state of the camera is a standby state and a shooting state).

The monitoring indicator includes at least one of:

1. the monitoring index refers to index data which is required to be reached by specified running state data in hardware equipment.

The index data can be target data to be achieved; but also the target range that needs to be reached.

Illustratively, the brightness index corresponding to the LED processor is 1900cd/m ² The brightness corresponding to the LED processor needs to be set to 1900cd/m ² The method comprises the steps of carrying out a first treatment on the surface of the Or, if the occupancy rate index of the GPU corresponding to the camera is 97% -100%, the occupancy rate of the camera in the GPU is 97% -100% as a normal condition.

2. The monitoring index refers to a state index of specified running state data in the hardware equipment in normal running.

Schematically, the shooting state index corresponding to the camera is standby or shooting in the middle, which means that the shooting state of the camera is in standby or shooting in the middle and is in normal operation; or the synchronization signal index corresponding to the renderer is 'in synchronization', which indicates that the synchronization signal of the renderer is in normal operation when in the in-synchronization state.

Optionally, the monitoring index is index data preset by a worker; or, the monitoring index is index data determined according to historical operation state data, and illustratively, in the historical operation data of the rendering machine, when the occupancy rate of the GPU is 50%, and the situation of faults is the least, the occupancy rate of the GPU of the rendering machine can be set to be 50%.

Optionally, the monitoring index is stored in a designated memory in the monitoring terminal equipment, and at least two monitoring indexes respectively corresponding to the equipment types are directly taken out from the designated memory; or, if the monitoring index is stored in the server, the monitoring end device may obtain the monitoring index corresponding to at least two device types from the server.

Optionally, the monitoring indexes which need to be matched with the running state data corresponding to the hardware equipment belonging to the same equipment type are the same indexes; or the monitoring indexes which are required to be matched with the running state data corresponding to the hardware equipment belonging to the same type are different indexes.

Illustratively, all LED processors at the shooting site detect them based on the same monitoring index; or, for each LED processor, there is a corresponding monitor index, and the monitor indexes of different LED processors are different. Optionally, setting the monitoring index of the LED processor according to the parameters of the LED display screen corresponding to the LED processor.

Optionally, the running state data of each hardware device is compared with the corresponding monitoring index in turn. Schematically, for a plurality of operation state data of a single hardware device, comparing the operation state data with monitoring indexes corresponding to the operation state data in sequence, and if the operation state data reach the monitoring indexes, indicating that the hardware component corresponding to the operation state data operates normally; and if the running state data does not reach the monitoring index, indicating that the hardware component corresponding to the running state data runs abnormally.

Illustratively, after the hardware equipment is started and before formally starting to work, a worker can detect a plurality of renderers on a shooting site, and sequentially detect whether the display card resolutions of the plurality of renderers are 1920 x 1080, and if the display card resolutions are 1920 x 1080, the detection of the display card resolutions of the plurality of renderers is passed; if the display card resolution of the renderer 1 is 1080×720, displaying that the display card resolution of the renderer 1 is 1080×720 as a matching result corresponding to the plurality of renderers, and indicating that the display card resolution of the renderer 1 is abnormal.

Optionally, if the plurality of hardware devices send the corresponding real-time running state data to the monitoring end device at regular time, the running state data corresponding to the plurality of hardware devices are displayed in real time in the monitoring end device.

Optionally, running state data corresponding to each of the plurality of hardware devices is displayed in the monitoring end device in the form of a statistical table. Referring to fig. 3, an interface diagram of real-time display of operation state data is shown, and as shown in fig. 3, an interface 300 is a real-time operation state data display interface of a renderer, and a frame rate, a synchronization signal, etc. corresponding to the renderer are displayed in the interface 300.

Step 203, displaying a video picture corresponding to the operation video data of the hardware device.

The video picture refers to a work scene picture of the hardware equipment at the virtual film making shooting site.

Optionally, displaying a first video picture corresponding to the operation video data of the hardware device, where the first video picture is a shot picture of the video camera of the virtual production shooting site to the virtual production shooting site.

The first video frame may be a film shot by a camera for shooting at a virtual production shooting site, that is, a fused frame of an LED display screen and a screen front scene shot by the camera. The first video picture can also be a real-time working scene of each hardware device of the scene shot by the monitoring camera of the virtual film-making shooting scene, and a manual operation scene of staff on the scene on each hardware device.

Optionally, the plurality of hardware devices include an operator, the operator is used for receiving software operation in the process of virtual film making shooting, and then a second video picture corresponding to the running video data of the hardware device is displayed, the second video picture is a screen display picture of the operator in the virtual film making shooting scene, and the screen display picture is used for displaying the software operation in the process of virtual film making shooting.

Illustratively, the screen display corresponding to the second video frame is used for displaying the operation process of the control software corresponding to each hardware device, for example: the rendering machine is correspondingly provided with a rendering engine, and the screen display picture of the operating machine can display the operation flow of the rendering task in the rendering engine.

And 204, performing data statistical analysis on the operation performance data of the hardware equipment, and displaying the statistical analysis result of the operation performance data in a chart form.

Optionally, the operation performance data of the hardware is a standard for measuring the quality of the hardware device, and the operation performance data includes: graphics image processing performance data, storage performance data, video processing performance data, and the like. The embodiments of the present application are not limited in this regard.

Alternatively, the number of the operation performance data may be one or more, and the statistical analysis of the operation performance data of the hardware device may be performed and the statistical analysis result of the operation performance data may be displayed in the form of a statistical chart/table.

Illustratively, frame rate data corresponding to the renderer within a period of time is collected, and a line graph of the frame rate change of the renderer within the period of time is displayed.

In summary, according to the device monitoring method based on virtual production provided in the embodiment of the present application, on one hand, by matching the operation state data of multiple hardware devices with the monitoring indexes of the device types corresponding to the hardware devices, the matching result between the operation state data and the monitoring indexes of the hardware devices is displayed in the monitoring end device, and the working scene images of the multiple hardware devices on the virtual production site are displayed, so that the staff can comprehensively manage the multiple hardware devices, monitor the multiple hardware devices remotely, thereby realizing automation of data detection of each device on hardware on the virtual production site, improving the monitoring efficiency of the hardware devices on the virtual production site, and reducing the workload of the on-site staff; on the other hand, by carrying out data statistics analysis on the operation performance data of the hardware equipment, the statistics analysis result of the operation performance data is displayed in a chart form, the performance of the hardware equipment in the operation process is displayed more intuitively, the current operation condition of the hardware equipment is conveniently known in real time by staff, and the possible hardware operation problem is solved in time.

According to the method provided by the embodiment of the application, the running state data corresponding to the hardware devices are displayed in real time in the monitoring terminal device, so that a worker can check the states of the hardware devices in real time, the step of manually checking each parameter of the hardware devices one by one is omitted, and the monitoring efficiency of the hardware devices is improved.

According to the method provided by the embodiment of the application, the shooting picture of the virtual film making shooting site is displayed on the monitoring terminal equipment by the camera of the virtual film making shooting site, on one hand, the staff can be helped to check the problem of the shooting process by acquiring the shooting scene picture of the hardware equipment; on the other hand, the shooting material pictures for making films are obtained, so that staff is helped to judge whether the current shooting material meets shooting requirements or not, and meanwhile, the shooting material pictures are used for backtracking of problems and confirming the effect of the shooting material.

Fig. 4 shows a flowchart of a device monitoring method based on virtual production according to an exemplary embodiment of the present application, and the method is applied to the monitoring end device 110 shown in fig. 1 for illustration, where the method includes:

step 401, acquiring running state data and running video data respectively corresponding to a plurality of hardware devices.

The operation state data corresponding to the hardware equipment are used for indicating the operation condition of each hardware component in the hardware equipment, and each hardware equipment is provided with the operation state data corresponding to each hardware equipment.

Optionally, the plurality of hardware devices includes a renderer, an LED processor, an operator, a camera for shooting, a camera for monitoring, and the like. Referring to fig. 5, an interactive flow chart of a virtual production-based device monitoring method is shown, wherein status data is obtained from a renderer 501, an operator 502, a camera 503 and an LED processor 504 at intervals, and sent to a data server, and the status data is stored by the data server; the monitoring device 505 may request the WEB server to log in, and after the login is successful, may request to obtain the state data stored in the data server through the WEB server.

Illustratively, as shown in fig. 5, at intervals of time, video data is obtained from the manipulator 502 and the camera 503, the video data is sent to a video server, and the video server stores the video data; the monitoring end device 505 may request a network (WEB) server to log in, and after the login is successful, may request to acquire video data stored in the video server through the WEB server.

Step 402, displaying a video picture corresponding to the operation video data of the hardware device.

Optionally, the video pictures include the following three types:

1. and shooting a film shot by a camera at a virtual film-making shooting site, namely, a fusion picture of an LED display screen shot by the camera and a screen front scene.

2. Real-time working scenes of all hardware devices in the field, which are shot by a camera for monitoring the virtual production shooting field, and manual operation scenes of all hardware devices by staff in the field.

3. And the screen display picture of the operating machine of the virtual film making shooting scene is used for displaying the control software operation process corresponding to each hardware device, and the operating machine is used for receiving the software operation in the virtual film making shooting process. For example: the rendering machine is correspondingly provided with a rendering engine, and the screen display picture of the operating machine can display the operation flow of the rendering task in the rendering engine.

For illustration, please refer to fig. 5, after obtaining video data of the camera 501 and the operator 502 from the video server through the WEB server, the video data is returned to the monitoring device 503 for display.

Step 403, displaying running state data corresponding to the plurality of hardware devices in real time.

For illustration, please refer to fig. 5, after obtaining status data from the video server through the WEB server, the status data is obtained from the renderer 504, the camera 501, the operator 502 and the LED processor 505, and then the status data is returned to the monitoring device 503 for real-time display.

In step 404, in response to the running state data of the hardware device not reaching the monitoring index of the device type corresponding to the hardware device, the running state data of the hardware device is highlighted as a matching result.

The monitoring index is used for indicating the operation condition which needs to be met by the hardware equipment corresponding to the equipment type in the operation process.

Optionally, the monitoring index is stored in a designated memory in the monitoring terminal equipment, and at least two monitoring indexes respectively corresponding to the equipment types are directly taken out from the designated memory; or, if the monitoring index is stored in the data server, the monitoring end device may obtain the monitoring index corresponding to at least two device types from the data server.

For illustration, please refer to fig. 3, the synchronization signal index of the renderer is "in synchronization", the state of the synchronization signal of the No. 5 renderer is "synchronization lost", the synchronization signal index is not reached, and the state is abnormal, so the "synchronization lost" state is highlighted as a matching result; meanwhile, the frame rate index of the renderer is 30 frames per second, the frame rate of the No. 3 renderer is 20 frames per second, and the frame rate index is not reached, so that "20" is highlighted as a matching result.

And step 405, displaying early warning information of the operation state data as a matching result in response to the difference between the operation state data of the hardware device and the monitoring index of the device type corresponding to the hardware device being smaller than or equal to a preset difference.

Optionally, the monitoring index may be implemented as a minimum condition that needs to be met by the hardware device during operation, and illustrated by taking the implementation of the hardware device as a camera as an example, and if the difference between the remaining power of the camera and the minimum power required by the camera for operation is less than or equal to a preset difference, the power early warning information is displayed as a matching result.

Schematically, the minimum electric quantity required by the operation of the camera is 20%, the preset difference is 5%, and the residual electric quantity of the camera 1 is 24%, so that early warning information of "the electric quantity of the camera 1 is about to be insufficient" can be displayed as an electric quantity state matching result of the camera 1.

In some optional embodiments, the fault classification may be further performed on the matched operation state data of the hardware device according to the matching situation, where the classification category includes at least one of the following categories:

1. error category.

And classifying the operation state data of the hardware equipment into error types in response to the operation state data of the hardware equipment not reaching the monitoring index of the equipment type corresponding to the hardware equipment.

2. Alert category.

And classifying the operation state data of the hardware equipment into a warning category in response to the difference value between the operation state data of the hardware equipment and the monitoring index of the equipment type corresponding to the hardware equipment being smaller than or equal to a preset difference value.

3. Normal category.

And classifying the operation state data of the hardware equipment into a normal class in response to the operation state data of the hardware equipment reaching a monitoring index of the equipment type corresponding to the hardware equipment. I.e. the hardware equipment has no fault and can normally operate.

Optionally, based on the classification category corresponding to the matched operation state data of the hardware devices, displaying data alarm information corresponding to the plurality of hardware devices respectively, wherein the data alarm information is used for displaying the operation state data of the faults and displaying the category corresponding to the faults.

In some optional embodiments, log data corresponding to each of the plurality of hardware devices is also stored in the monitoring end device, where the log data is used to record running state data of the plurality of hardware devices in a historical time period and matching results corresponding to the running state data.

In some optional embodiments, when the running state data of the hardware device does not reach the monitoring index of the device type corresponding to the hardware device, the running state data of the fault can be analyzed through the video data corresponding to the hardware device, and a fault analysis result of the running state data is obtained.

Optionally, in response to the target running state data of the hardware device in the specified time period not reaching the monitoring index of the device type corresponding to the hardware device, acquiring the target running video data of the hardware device in the specified time period; performing fault detection on the target operation state data based on the target operation video data in a specified time period, and determining a fault reason of the target operation state data; according to the fault reasons, fault repair prompt information is displayed, and the fault repair prompt information is used for providing reference basis for fault repair of the target running state data.

Schematically, if the target running state data hardware device of a certain hardware device in a certain time period does not reach the corresponding monitoring index, it is indicated that the hardware component corresponding to the target running state data has a fault; the monitoring video data of the hardware device in the time period and the software operation video data of the operation machine for controlling the hardware device in the time period can be obtained.

Optionally, inputting the target operation video data and the target operation state data in a specified time period into a fault detection model, wherein the fault detection model is used for predicting a fault reason corresponding to the target operation state data, and is a neural network model obtained by training sample data in advance; and outputting a failure reason prediction result obtained on the target running state data as a failure reason.

The target operation video data comprises a monitoring video of the hardware device in a specified time period and a software operation process video.

Optionally, the fault detection models are in one-to-one correspondence with the hardware devices, that is, the fault detection models corresponding to the renderers are used for predicting operation faults generated by the renderers.

The fault prediction model is a model obtained by training sample operation state data (operation state data corresponding to hardware equipment with faults) and sample monitoring videos, sample software operation process videos and reference fault reasons corresponding to the sample operation state data, the sample monitoring videos, the sample software operation process videos and the sample operation state data are input into the sample fault prediction model in a schematic mode, the prediction fault reasons corresponding to the sample operation state data are obtained through output, and the sample fault prediction model is trained based on the difference between the prediction fault reasons and the reference fault reasons to obtain the fault prediction model.

Optionally, after obtaining the fault cause of the target running state data, determining corresponding fault repair prompt information of the fault cause, and obtaining a fault type library corresponding to the target running state data, wherein the fault type library comprises fault types and fault repair reference information corresponding to the fault types; and matching the fault cause with a fault type library, determining a target fault type matched with the fault cause in the fault type library, and taking fault repair reference information corresponding to the target fault type as fault repair prompt information.

The matching method of the fault cause and the fault type library can schematically calculate the semantic similarity of each fault type in the fault cause and the fault type library through a semantic similarity method, and the fault type with the highest semantic similarity is determined as the target fault type.

And 406, performing rendering data statistical analysis on the rendering data of the rendering machine, and displaying the statistical analysis result of the rendering data in the form of a line graph.

The plurality of hardware devices comprise a renderer, wherein the operation performance data of the renderer comprises rendering data of the renderer; and carrying out rendering data statistical analysis on rendering data corresponding to the rendering machine, and displaying a statistical analysis result of the rendering data in the form of a line graph.

Optionally, the rendering data of the renderer includes a rendering frame rate, a rendering time length, a GPU processing time, and the like of the renderer at the time of rendering.

Illustratively, the rendering frame rate, rendering time length and GPU processing time corresponding to the renderer in a period of time are collected, and a line graph of the rendering frame rate, rendering time length and GPU processing time change in the period of time is displayed.

According to the method provided by the embodiment of the invention, when the running state data of the hardware equipment does not reach the monitoring index corresponding to the hardware equipment, the running state data of the hardware equipment is highlighted, so that a worker can grasp the fault condition of the hardware equipment more efficiently, the cause of the fault is determined, and the monitoring efficiency of the hardware equipment on the shooting site of the virtual film production is further improved.

According to the method provided by the embodiment of the application, when the difference value between the running state data of the hardware equipment and the monitoring index corresponding to the hardware equipment is smaller than or equal to the preset difference value, early warning information of the running state data is displayed, and the state data which possibly has faults is warned, for example: the camera is about to be insufficient in electric quantity and about to be insufficient in hard disk space, so that a worker can prepare a solution in advance before a fault occurs, and the fault solving efficiency of state data is improved.

According to the method provided by the embodiment of the application, the data statistics analysis is carried out on the performance (rendering) data corresponding to the rendering machine, the statistical analysis result of the performance data is displayed in the form of the line diagram, the performance data of the rendering machine is analyzed in the form of the line diagram, a worker can intuitively know whether the frame rate operation of the rendering machine is stable when the rendering task is appointed, the information expression quantity of the operation state data is increased, and the working efficiency of the worker is improved.

According to the method provided by the embodiment of the application, the target operation video data in the specified time period is obtained, and the fault detection is carried out on the target operation state data, so that the fault reason of the target operation state data is determined, and finally, the fault repair prompt information for the fault reason is displayed, so that a reference basis is provided for the fault repair of the target operation state data, and the fault detection and repair efficiency of the hardware equipment by staff is improved.

According to the method provided by the embodiment of the application, the fault reason of the target running state data is predicted through the neural network model (fault prediction model), so that the accuracy of fault detection is improved. And by matching the fault reasons with the fault type library, determining the target fault type matched with the fault reasons in the fault type library, thereby obtaining fault repair prompt information of the fault reasons and improving the accuracy of fault repair.

Fig. 6 shows a flowchart of a device monitoring method based on virtual production according to an exemplary embodiment of the present application, and the method is applied to the monitoring end device 110 shown in fig. 1 for illustration, where the method includes:

step 601, device monitoring operations for a plurality of hardware devices are received.

The hardware devices are devices positioned on the virtual production shooting site and used for building and collecting shooting scenes of the virtual production shooting site, and the hardware devices correspond to at least two device types.

The device monitoring operation is used for detecting running state data corresponding to the hardware devices respectively. Optionally, the device monitoring operation is further used for displaying operation state data corresponding to the plurality of hardware devices respectively.

Step 602, in response to the device monitoring operation, displaying a matching result between the running state data of the hardware device and the monitoring index of the device type corresponding to the hardware device.

The monitoring index is used for indicating the operation condition which needs to be met by the hardware equipment corresponding to the equipment type in the operation process, the matching result is used for indicating the operation fault condition corresponding to the hardware equipment of the virtual film-making shooting site, the operation state data comprise operation performance data, and the operation performance data are used for representing the hardware performance of the hardware equipment in operation.

Optionally, the device monitoring operation may implement a control clicking operation in the monitoring device. Referring to fig. 7, an interface schematic diagram of a hardware status checking module in device monitoring software according to an exemplary embodiment of the present application is shown, where the device monitoring software is a target application installed in a monitoring device, and the device monitoring software is mainly divided into six functional modules, as shown in fig. 7, where the six functional modules are respectively:

The hardware state checking module is used for checking the running state of each hardware device on the shooting site before the shooting of the virtual production is started;

the hardware real-time state module is used for checking the real-time running state of each hardware device on the shooting site in the shooting process of the virtual production;

the data alarm module is used for displaying fault states in the running state data of each hardware device;

the history log module is used for recording log information in the hardware environment of each hardware device;

the on-site video monitoring module comprises three types of video pictures, and a screen display picture of the operating machine, wherein the screen display picture is used for recording a software operation process in a shooting process; shooting a material picture shot by a camera; monitoring working scene pictures of all hardware devices shot by a camera;

the performance data acquisition module is used for acquiring performance data of hardware in the hardware equipment, wherein the performance data of the hardware is a standard for measuring the quality of the hardware equipment, and comprises the following components: graphics image processing performance data, storage performance data, video processing performance data, and the like, to which the embodiments of the present application are not limited.

For illustration, please refer to fig. 7, click on the "hardware status check" control 701, the interface 700 of the hardware status check module may be entered, and the "start detection" control 702 is displayed on the interface 700; clicking the "start detection" control 702 to start detecting each hardware device of the current access device monitoring software, and displaying the detection type, detection item, detection result and detection state in real time in a statistical form manner; as shown in fig. 7, the detection result is highlighted, i.e., representing that the record has failed detection, and the state of the current hardware device is displayed as the detection result (i.e., the matching result).

In fig. 7, when process data of the renderer is detected, it is detected that there are a plurality of window processes in the renderer, and the process data is displayed in a highlighted state as a useless process. Optionally, in fig. 7, each line of data in the table represents not a detection result of a single renderer, but detection results of all renderers accessing the device monitoring system, and optionally, the highlighted detection result may also display that "the x-number renderer has a plurality of window processes", that is, process data represented as the x-number renderer is abnormal data.

Optionally, the device monitoring operation may further implement an operation triggered automatically by the monitoring device according to a preset time interval, where the operation is used to display and detect running state data corresponding to each of the plurality of hardware devices. Optionally, based on a preset time interval, running state data corresponding to each of the plurality of hardware devices is displayed in real time, and a matching result between the running state data of the hardware device and the monitoring index of the device type corresponding to the hardware device is displayed.

For illustration, referring to fig. 8, clicking the "hardware real-time status" control 801 may enter the interface 800 of the hardware status checking module, where running status data corresponding to a plurality of hardware devices are displayed in real time in the interface 800, and at intervals, the interface updates data in the table according to the latest acquired running status data. As shown in fig. 8, the operation state data is highlighted, that is, the operation state data that is highlighted is taken as the matching result corresponding to the data, which represents that the record does not reach the corresponding monitoring index.

In some alternative embodiments, the device monitoring software installed in the monitoring end device is further configured to display data alarm information of the operational status data with the fault, where the data alarm information is used to indicate the fault level, the fault type, and the like of the operational status data.

Optionally, when the running state data fails (i.e. when the running state data does not reach the monitoring index), the monitoring end device sends the data alarm information of the failed running state data to the server, and the server stores and counts the data alarm information; and in response to receiving the data alarm information deriving operation in the monitoring end device, displaying the data alarm information in the monitoring end device.

For illustration, please refer to fig. 9, clicking the "data alert" control 901 may enter the interface 900 of the data alert module, the "export" control 902 is displayed in the interface 900, and clicking the "export" control 902 may export the data alert information, where the data alert information includes: the fault source, fault type, fault level, fault code, fault message, time of occurrence of the fault, etc., optionally, a worker may click on the fault type column in the interface 900, and filter the data alarm information according to the fault type.

Optionally, a "clear" control 903 is displayed in the interface 900, and clicking on the "clear" control 903 can alert the user to clear the data displayed on the current interface.

In some optional embodiments, the device monitoring software installed in the monitoring end device is further configured to display log information in a hardware environment corresponding to each hardware device, where the log information includes not only data alarm information but also normal log content.

For illustration, referring to fig. 10, clicking the "history log" control 1001 may enter the interface 1000 of the history log module, and the log list corresponding to each hardware device is displayed in the interface 1000, as shown in fig. 10, clicking the log3 may view the detailed information of the log 3.

Step 603, displaying a video frame corresponding to the running video data of the hardware device.

The video picture refers to a working scene of the hardware equipment in a virtual film-making shooting site.

Optionally, the video pictures include a first video picture and a second video picture.

Alternatively, the first video frame may be a film shot by a camera for shooting at a virtual production shooting site, that is, a fused frame of an LED display screen and a screen front scene shot by the camera. The first video picture can also be a real-time working scene of each hardware device of the scene shot by the monitoring camera of the virtual film-making shooting scene, and a manual operation scene of staff on the scene on each hardware device.

Optionally, the second video frame is a screen display frame of the operator, where the screen display frame is used to display a control software operation process corresponding to each hardware device, for example: the rendering machine is correspondingly provided with a rendering engine, and the screen display picture of the operating machine can display the operation flow of the rendering task in the rendering engine.

Optionally, in response to receiving a control clicking operation in the monitoring end device, displaying operation video data corresponding to the plurality of hardware devices respectively; or based on a preset time interval, displaying the running video data respectively corresponding to the plurality of hardware devices.

For illustration, referring to fig. 11, clicking the "live video monitoring" control 1101 can enter the interface 1100 of the live video monitoring module, where a plurality of camera frames 1103 for shooting, a screen frame 1104 for operating the camera and a camera frame 1105 for monitoring corresponding to the hardware devices are displayed in real time in the interface 1100, and at intervals, the interface updates the frames according to the latest acquired operation video data.

As shown in fig. 11, the right side of the interface 1100 also displays a history time record, that is, a specified time record in the click history time record, and the screen in the shooting camera screen 1103, the operator screen 1104, and the monitoring camera screen 1105 jumps to a video screen corresponding to the specified time record.

In response to receiving the analysis operation of the operation performance data of the hardware device, a statistical analysis result of the operation performance data is displayed in a graph form in step 604.

The analysis operation is used for carrying out data statistical analysis on the specified running state data.

The above-mentioned operation performance data of the hardware device is used to represent the hardware performance of the hardware device when it is operated, for example: frame rate data of a renderer at the time of rendering, GPU processing time, rendering time, and the like.

For illustration, please refer to fig. 12, clicking the "performance data collection" control 1201 can enter the interface 1200 of the performance data module, the "start collection" control 1202 is displayed in the interface 1200, and clicking the "start collection" control 1202 can start collecting the performance data corresponding to the current time of the renderer, which includes: rendering frame rate, GPU processing time, rendering total duration, etc., and displaying a change curve of the performance data in the form of a statistical graph. Optionally, the collected performance data may also be displayed on the right side of interface 1200 as a statistics.

Optionally, an "end acquisition" control 1203 is displayed in the interface 1200, and clicking on the "end acquisition" control 1202 ends acquiring the performance data corresponding to the renderer.

Optionally, a history collection record is also displayed on the right side of the interface 1200, that is, the designated collection time in the history collection record is clicked, that is, the performance data analysis interface corresponding to the designated collection time is jumped to.

Referring to fig. 13, a block diagram of a device for monitoring equipment based on virtual production according to an exemplary embodiment of the present application is shown, where the device includes:

the obtaining module 1300 is configured to obtain operation state data and operation video data corresponding to a plurality of hardware devices, where the plurality of hardware devices are devices located at a virtual production shooting site, the plurality of hardware devices are used to build and collect shooting scenes of the virtual production shooting site, the plurality of hardware devices correspond to at least two device types, the operation state data includes operation performance data, and the operation performance data is used to represent hardware performance of the hardware devices in operation;

a first display module 1310, configured to match operation state data of a hardware device with a monitoring indicator of a device type corresponding to the hardware device, and display a matching result between the operation state data of the hardware device and the monitoring indicator, where the monitoring indicator is used to indicate an operation condition that needs to be met by the hardware device corresponding to the device type in an operation process, and the matching result is used to indicate an operation fault condition corresponding to the hardware device in the virtual film-making shooting site;

The first display module 1310 is further configured to display a video frame corresponding to the operation video data of the hardware device, where the video frame refers to a work scene frame of the hardware device at the virtual production shooting site;

the first display module 1310 is further configured to perform data statistics analysis on the operation performance data of the hardware device, and display a result of the statistics analysis on the operation performance data in a graph form.

In some optional embodiments, the first display module 1310 is further configured to display, in real time, running state data corresponding to each of the plurality of hardware devices.

In some optional embodiments, the first display module 1310 is further configured to highlight, as the matching result, the running state data of the hardware device in response to the running state data of the hardware device not reaching a monitoring index of a device type corresponding to the hardware device.

In some optional embodiments, the first display module 1310 is further configured to display, as the matching result, early warning information of the running state data, in response to a difference between the running state data of the hardware device and a monitoring indicator of a device type corresponding to the hardware device being less than or equal to a preset difference.

In some optional embodiments, the plurality of hardware devices includes a renderer, and the operational performance data of the renderer includes rendering data of the renderer; the first display module 1310 is further configured to perform a statistical analysis on rendering data of the renderer, and display a result of the statistical analysis on the rendering data in a form of a line graph.

In some optional embodiments, the first display module 1310 is further configured to display a first video frame corresponding to the operation video data of the hardware device, where the first video frame is a shot frame of the virtual production shooting site by a camera of the virtual production shooting site.

In some optional embodiments, the plurality of hardware devices includes an operator, where the operator is configured to receive a software operation during the virtual production shooting process, and the first display module 1310 is further configured to display a second video frame corresponding to the running video data of the hardware device, where the second video frame is a screen display frame of the operator on the virtual production shooting site, and the screen display frame is configured to display a software operation during the virtual production shooting process.

The obtaining module 1300 is further configured to obtain target operation video data of the hardware device in a specified time period, in response to the target operation state data of the hardware device in the specified time period not reaching a monitoring indicator of a device type corresponding to the hardware device; the obtaining module 1300 is further configured to perform fault detection on the target operation state data based on the target operation video data in the specified time period, and determine a cause of the fault of the target operation state data; the first display module 1310 is further configured to display, according to the failure cause, failure repair prompt information, where the failure repair prompt information is used to provide a reference basis for failure repair of the target running state data.

In some optional embodiments, the obtaining module 1300 is further configured to input the target operation video data and the target operation state data in the specified period of time into a fault detection model, where the fault detection model is used for predicting a fault cause corresponding to the target operation state data, and the fault detection model is a neural network model obtained by training sample data in advance; the obtaining module 1300 is further configured to output a failure cause prediction result obtained from the target operation state data as the failure cause.

In some optional embodiments, the obtaining module 1300 is further configured to obtain a fault type library corresponding to the target operating state data, where the fault type library includes a fault type and fault repair reference information corresponding to the fault type; the obtaining module 1300 is further configured to match the fault cause with the fault type library, determine a target fault type matched with the fault cause in the fault type library, and use fault repair reference information corresponding to the target fault type as the fault repair prompt information.

In summary, according to the device monitoring apparatus based on virtual production provided in the embodiment of the present application, on one hand, by matching the operation state data of multiple hardware devices with the monitoring index of the device type corresponding to the hardware devices, the matching result between the operation state data and the monitoring index of the hardware devices is displayed in the monitoring end device, and the working scene images of the multiple hardware devices on the virtual production site are displayed, so that the staff can comprehensively manage the multiple hardware devices, monitor the multiple hardware devices remotely, thereby realizing automation of data detection of each device on hardware on the virtual production site, improving the monitoring efficiency of the hardware devices on the virtual production site, and reducing the workload of the on-site staff; on the other hand, by carrying out data statistics analysis on the operation performance data of the hardware equipment, the statistics analysis result of the operation performance data is displayed in a chart form, the performance of the hardware equipment in the operation process is displayed more intuitively, the current operation condition of the hardware equipment is conveniently known in real time by staff, and the possible hardware operation problem is solved in time.

Referring to fig. 14, there is shown a block diagram of a device for monitoring equipment based on virtual production according to another exemplary embodiment of the present application, the device includes:

the receiving module 1400 is configured to receive device monitoring operations on a plurality of hardware devices, where the plurality of hardware devices are devices located at a virtual production shooting site, the plurality of hardware devices are configured to build and collect shooting scenes of the virtual production shooting site, and the plurality of hardware devices correspond to at least two device types;

the second display module 1410 is configured to respond to the device monitoring operation, and display a matching result between operation state data of a hardware device and a monitoring indicator of a device type corresponding to the hardware device, where the monitoring indicator is used to indicate an operation condition that needs to be met by the hardware device corresponding to the device type in an operation process, and the matching result is used to indicate an operation fault condition corresponding to the hardware device in the virtual film-making shooting site, where the operation state data includes operation performance data, and the operation performance data is used to indicate hardware performance of the hardware device in operation;

the second display module 1410 is further configured to display a video frame corresponding to the operation video data of the hardware device, where the video frame refers to a working scene of the hardware device in the virtual production shooting site;

The second display module 1410 is further configured to graphically display a statistical analysis result of the operation performance data in response to receiving an analysis operation of the operation performance data of the hardware device, where the analysis operation is configured to perform a data statistical analysis on the specified operation state data.

It should be noted that: the device for monitoring equipment based on virtual production provided in the above embodiment is only exemplified by the division of the above functional modules, and in practical application, the above functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the equipment is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the device for monitoring equipment based on virtual production provided in the above embodiment and the device for monitoring equipment based on virtual production are the same conception, and detailed implementation process of the device is shown in the method embodiment, and will not be described herein.

Fig. 15 shows a block diagram of a computer device 1500 provided in an exemplary embodiment of the present application. The computer device 1500 may be: a smart phone, a tablet computer, a dynamic video expert compression standard audio layer 3 player (Moving Picture Experts Group Audio Layer III, MP 3), a dynamic video expert compression standard audio layer 4 (Moving Picture Experts Group Audio Layer IV, MP 4) player, a notebook computer, or a desktop computer. The computer device 1500 may also be referred to as a user device, portable computer device, laptop computer device, desktop computer device, and the like.

In general, the computer device 1500 includes: a processor 1501 and a memory 1502.

The processor 1501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, or the like. The processor 1501 may be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 1501 may also include a main processor, which is a processor for processing data in an awake state, also called a central processor (Central Processing Unit, CPU), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1501 may be integrated with an image processor (Graphics Processing Unit, GPU) for use in connection with rendering and rendering of content to be displayed by the display screen. In some embodiments, the processor 1501 may also include an artificial intelligence (Artificial Intelligence, AI) processor for processing computing operations related to machine learning.

Memory 1502 may include one or more computer-readable storage media, which may be non-transitory. Memory 1502 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 1502 is used to store at least one instruction for execution by processor 1501 to implement the virtual production based device monitoring method provided by the method embodiments herein.

Illustratively, the computer device 1500 also includes other components, and those skilled in the art will appreciate that the structure illustrated in FIG. 15 is not limiting of the computer device 1500, and may include more or fewer components than illustrated, or may combine certain components, or employ a different arrangement of components.

Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be implemented by a program for instructing related hardware, and the program may be stored in a computer readable storage medium, which may be a computer readable storage medium included in the memory of the above embodiments; or may be a computer-readable storage medium, alone, that is not assembled into a computer device. The computer readable storage medium has stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which are loaded and executed by the processor to implement the virtual production-based device monitoring method of any of the above embodiments.

Alternatively, the computer-readable storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), solid state disk (SSD, solid State Drives), or optical disk, etc. The random access memory may include resistive random access memory (ReRAM, resistance Random Access Memory) and dynamic random access memory (DRAM, dynamic Random Access Memory), among others. The foregoing embodiment numbers of the present application are merely for describing, and do not represent advantages or disadvantages of the embodiments.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The foregoing description of the preferred embodiments is merely exemplary in nature and is in no way intended to limit the invention, since it is intended that all modifications, equivalents, improvements, etc. that fall within the spirit and scope of the invention.

Claims

1. A virtual-production-based device monitoring method, the method comprising:

2. The method according to claim 1, wherein the method further comprises:

and displaying running state data corresponding to the hardware devices in real time.

3. The method according to claim 2, wherein the displaying the matching result between the operation state data of the hardware device and the monitoring index includes:

and in response to the running state data of the hardware equipment not reaching the monitoring index of the equipment type corresponding to the hardware equipment, highlighting the running state data of the hardware equipment as the matching result.

4. The method according to claim 2, wherein the displaying the matching result between the operation state data of the hardware device and the monitoring index includes:

and displaying early warning information of the running state data as the matching result in response to the difference value between the running state data of the hardware equipment and the monitoring index of the equipment type corresponding to the hardware equipment being smaller than or equal to a preset difference value.

5. The method of claim 1, wherein the plurality of hardware devices comprises a renderer, and wherein the operational performance data of the renderer comprises rendering data of the renderer;

The data statistical analysis is performed on the operation performance data of the hardware equipment, the statistical analysis result of the operation performance data is displayed in a graph form, and the method comprises the following steps:

and carrying out rendering data statistical analysis on the rendering data of the rendering machine, and displaying a statistical analysis result of the rendering data in the form of a line graph.

6. The method according to claim 1, wherein displaying the video frame corresponding to the operation video data of the hardware device comprises:

and displaying a first video picture corresponding to the operation video data of the hardware equipment, wherein the first video picture is a shooting picture of a camera of the virtual production shooting site to the virtual production shooting site.

7. The method of claim 1, wherein the plurality of hardware devices includes an operator for receiving software operations during the virtual production shot;

the displaying the video picture corresponding to the operation video data of the hardware device includes:

and displaying a second video picture corresponding to the operation video data of the hardware equipment, wherein the second video picture is a screen display picture of an operator of the virtual film-making shooting site, and the screen display picture is used for displaying a software operation process in the virtual film-making shooting process.

8. The method according to any one of claims 1 to 7, further comprising:

acquiring target operation video data of the hardware equipment in a specified time period in response to the target operation state data of the hardware equipment in the specified time period not reaching a monitoring index of the equipment type corresponding to the hardware equipment;

performing fault detection on the target operation state data based on the target operation video data in the specified time period, and determining a fault reason of the target operation state data;

and displaying fault repair prompt information according to the fault cause, wherein the fault repair prompt information is used for providing a reference basis for fault repair of the target running state data.

9. The method of claim 8, wherein the fault detection of the target operating state data based on the operating video data of the hardware device within the specified time period, determining a cause of the fault of the target operating state data, comprises:

inputting the target operation video data and the target operation state data in the specified time period into a fault detection model, wherein the fault detection model is used for predicting a fault reason corresponding to the target operation state data, and is a neural network model obtained through sample data training in advance;

And outputting a failure reason prediction result obtained for the target running state data as the failure reason.

10. The method of claim 8, wherein displaying a fault repair hint based on the fault cause comprises:

acquiring a fault type library corresponding to the target running state data, wherein the fault type library comprises fault types and fault repair reference information corresponding to the fault types;

and matching the fault cause with the fault type library, determining a target fault type matched with the fault cause in the fault type library, and taking fault repair reference information corresponding to the target fault type as the fault repair prompt information.

11. A virtual-production-based device monitoring method, the method comprising:

12. An equipment monitoring device based on virtual film production, characterized in that the device comprises:

13. An equipment monitoring device based on virtual film production, characterized in that the device comprises:

14. A computer device comprising a processor and a memory, wherein the memory has stored therein at least one computer program that is loaded and executed by the processor to implement the virtual production-based device monitoring method of any one of claims 1 to 11.

15. A computer readable storage medium, characterized in that at least one section of a computer program is stored in the computer readable storage medium, which is loaded and executed by a processor to implement the virtual production based device monitoring method according to any of claims 1 to 11.

16. A computer program product comprising a computer program which when executed by a processor implements a virtual production based device monitoring method as claimed in any one of claims 1 to 11.