CN117857513A

CN117857513A - Interactive processing method and device for remote operation and maintenance, terminal equipment and storage medium

Info

Publication number: CN117857513A
Application number: CN202410030929.2A
Authority: CN
Inventors: 魏春城; 梁卓; 刘岸琦; 柴孟强; 田莉
Original assignee: China United Network Communications Group Co Ltd; Unicom Digital Technology Co Ltd
Current assignee: China United Network Communications Group Co Ltd; Unicom Digital Technology Co Ltd
Priority date: 2024-01-09
Filing date: 2024-01-09
Publication date: 2024-04-09

Abstract

The application provides an interactive processing method, device, terminal equipment and storage medium for remote operation and maintenance, which are applied to a first client of operation and maintenance personnel, wherein the method comprises the following steps: initiating a call request to establish connection with an audio-video call of a second client of the expert; receiving a remote interaction signaling sent by a signaling server; after the connection of the audio-video call is established, the second client responds to the communication between expert personnel and operation maintenance personnel through the audio-video call, and then accesses the knowledge base system to generate and send the information to the signaling server; the remote interaction signaling comprises information to be loaded; analyzing the remote interactive signaling to obtain information to be loaded, wherein the information to be loaded comprises a content identifier and a loading mode; pulling interactive content from the knowledge base system according to the content identification; according to the loading mode, the interactive content is displayed, and the working efficiency of remote assistance is improved.

Description

Interactive processing method and device for remote operation and maintenance, terminal equipment and storage medium

Technical Field

The present disclosure relates to the field of remote collaboration technologies, and in particular, to a method and apparatus for interactive processing of remote operation and maintenance, a terminal device, and a storage medium.

Background

Remote collaboration is a method for remotely solving problems through the Internet. With the development of internet technology, the remote collaboration technology can help the operation and maintenance personnel to rapidly solve the field problem under the condition that the operation and maintenance personnel process the field problem.

In the prior art, operation and maintenance personnel and expert personnel perform remote collaboration by using an audio and video technology to finish the instruction of operation and maintenance.

However, in the prior art, remote interaction between the operation and maintenance personnel and the guiding personnel can be realized only through simple audio and video images, and the audio and video communication is limited for the detail problem in the operation and maintenance scene, so that the working efficiency of remote assistance is low.

Disclosure of Invention

The application provides an interactive processing method, device, terminal equipment and storage medium for remote operation and maintenance, which are used for solving the problem of low working efficiency of remote assistance in the prior art.

In a first aspect, the present application provides an interaction processing method for remote operation and maintenance, which is applied to a first client of an operation and maintenance personnel, and includes:

initiating a call request to establish connection with an audio-video call of a second client of the expert;

receiving a remote interaction signaling sent by a signaling server; after the connection of the audio-video call is established, the second client responds to the communication between expert personnel and the operation and maintenance personnel through the audio-video call, and then performs access operation on a knowledge base system to generate and send the remote interactive signaling to the signaling server; the remote interaction signaling comprises information to be loaded;

Analyzing the remote interactive signaling to obtain the information to be loaded, wherein the information to be loaded comprises a content identifier and a loading mode;

pulling interactive content from the knowledge base system according to the content identification;

and displaying the interactive content according to the loading mode.

In one possible design, the audio-video call is displayed at a first display layer of a user interface of the first client; the interactive content is displayed on a second display layer of the user interface of the first client; and the second display layer is positioned above the first display layer and is transparent.

In one possible design, the interactive content is a three-dimensional model; after displaying the interactive content according to the loading mode, the method further comprises: and responding to the rotation operation of the operation and maintenance personnel on the three-dimensional model, and displaying the rotated three-dimensional model with different angles.

In one possible design, after the displaying the interactive content, the method further includes: receiving an interactive mark sent by the signaling server, wherein the interactive mark is generated by the second client in response to the marking operation of the expert, and is sent to the signaling server, and mark position information is recorded in the interactive mark; and displaying the interactive mark according to the mark position information.

In one possible design, the displaying the interactive mark according to the mark position information includes: if the resolution of the first client is different from the resolution of the second client, updating the mark position information according to the resolution of the second client; and displaying the interactive mark according to the updated mark position information.

In a second aspect, the present application provides an interactive processing method for remote operation and maintenance, applied to a second client of expert personnel, including:

receiving a call request sent by a signaling server to establish connection with an audio/video call of a first client of an operation and maintenance person; the call request is generated by the first client in response to the operation of the operation and maintenance personnel;

responding to the access operation of the expert personnel to the knowledge base system after communicating with the operation and maintenance personnel through the audio and video call, and generating a remote interaction signaling;

the remote interaction signaling is sent to the signaling server, so that the signaling server sends the remote interaction signaling to the first client, wherein the remote interaction signaling comprises information to be loaded, the remote interaction signaling is used for indicating the first client to analyze the remote interaction signaling to obtain the information to be loaded, the information to be loaded comprises a content identifier and a loading mode, the interactive content is pulled from the knowledge base system according to the content identifier, and the interactive content is displayed according to the loading mode.

In a third aspect, the present application provides an interaction processing apparatus for remote operation and maintenance, which is applied to a first client of an operation and maintenance personnel, and includes:

the initiation module is used for initiating a call request to establish connection with the audio and video call of the second client of the expert;

the first receiving module is used for receiving the remote interactive signaling sent by the signaling server; after the connection of the audio-video call is established, the second client responds to the communication between expert personnel and the operation and maintenance personnel through the audio-video call, and then performs access operation on a knowledge base system to generate and send the remote interactive signaling to the signaling server; the remote interaction signaling comprises information to be loaded;

the analysis module is used for analyzing the remote interactive signaling to obtain the information to be loaded, wherein the information to be loaded comprises a content identifier and a loading mode;

the pulling module is used for pulling the interactive content from the knowledge base system according to the content identification;

and the display module is used for displaying the interactive content according to the loading mode.

In a fourth aspect, the present application provides an interaction processing device for remote operation and maintenance, which is applied to a second client of expert personnel, and includes:

The first receiving module is used for receiving a call request sent by the signaling server so as to establish connection with the audio-video call of the first client of the operation and maintenance personnel; the call request is generated by the first client in response to the operation of the operation and maintenance personnel;

the generation module is used for responding to the access operation of the knowledge base system after the expert personnel and the operation and maintenance personnel communicate through the audio and video call, and generating a remote interaction signaling;

the remote interaction signaling comprises information to be loaded, the remote interaction signaling is used for indicating the first client to analyze the information to be loaded from the remote interaction signaling, the information to be loaded comprises a content identifier and a loading mode, the interactive content is pulled from the knowledge base system according to the content identifier, and the interactive content is displayed according to the loading mode.

In a fifth aspect, the present application provides a terminal device, including:

at least one processor and memory;

The memory stores computer-executable instructions;

the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the interactive processing method of the remote operation and maintenance as described in the above first aspect or the second aspect and various possible designs of the first aspect or the second aspect.

In a sixth aspect, the present application provides a computer storage medium, where computer execution instructions are stored, when executed by a processor, to implement the interactive processing method for remote operation and maintenance according to the first aspect or the second aspect and the various possible designs of the first aspect or the second aspect.

According to the interactive processing method, the device, the terminal equipment and the storage medium for the remote operation and maintenance, the audio and video call connection is established by initiating a call request to the expert client through the operation and maintenance personnel client, the remote interactive signaling is generated based on the audio and video call connection expert client, the remote interactive signaling is sent to the signaling server, the remote interactive signaling is sent to the operation and maintenance personnel client through the signaling server, the operation and maintenance personnel client directly displays the interactive content by pulling the interactive content to the knowledge base, and compared with the prior art, the interactive content of the operation and maintenance personnel and the expert is displayed more intuitively, the technical guidance of the expert on the operation and maintenance personnel is facilitated, and the working efficiency of remote assistance is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief description will be given below of the drawings that are needed in the embodiments or the prior art descriptions, it being obvious that the drawings in the following description are some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.

Fig. 1 is an application scenario schematic diagram of an interaction processing method of a remote operation and maintenance provided in an embodiment of the present application;

FIG. 2 is a flow chart of an interactive processing method for remote operation and maintenance according to an embodiment of the present application;

fig. 3 is a schematic diagram of a client display interface provided in an embodiment of the present application;

FIG. 4 is a schematic diagram showing interactive marks according to an embodiment of the present disclosure;

FIG. 5 is a flowchart illustrating an interactive processing method for remote operation and maintenance according to another embodiment of the present application;

fig. 6 is a schematic diagram of sending remote interaction signaling according to an embodiment of the present application;

fig. 7 is an interaction flow diagram of an interaction processing method of remote operation and maintenance according to another embodiment of the present application;

FIG. 8 is a schematic structural diagram of an interaction processing device for remote operation and maintenance according to an embodiment of the present application;

FIG. 9 is a schematic structural diagram of an interaction processing device for remote operation and maintenance according to another embodiment of the present application;

fig. 10 is a schematic hardware structure of a terminal device according to an embodiment of the present application.

Detailed Description

For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

Remote collaboration is a method for remotely solving problems through the Internet. With the development of internet technology, the remote collaboration technology can help the operation and maintenance personnel to rapidly solve the field problem under the condition that the operation and maintenance personnel process the field problem. In the prior art, operation and maintenance personnel and expert personnel perform remote collaboration by using an audio and video technology to finish the instruction of operation and maintenance. However, in the prior art, remote interaction between the operation and maintenance personnel and the guiding personnel can be realized only through simple audio and video images, and the audio and video communication is limited for the detail problem in the operation and maintenance scene, so that the working efficiency of remote assistance is low.

In order to solve the above technical problems, the embodiments of the present application provide the following technical ideas: the inventor thinks that on the basis of the audio-video call established by the operation and maintenance personnel and the expert personnel, the expert personnel acquires the content needing to be interacted from the knowledge base system through the audio-video call, and displays the content needing to be interacted to the operation and maintenance personnel, so that the synchronous display of the interactive content by the expert personnel and the operation and maintenance personnel is realized, and the collaboration efficiency of remote operation and maintenance is improved. The following will explain in detail the embodiments.

Fig. 1 is an application scenario schematic diagram of an interactive processing method for remote operation and maintenance provided in an embodiment of the present application. As shown in fig. 1, the scene includes: a first client 101 of an operation and maintenance person, a signaling server 102, a knowledge base system 103, a second client 104 of an expert person and a real-time communication server 105.

The first client 101 of the operation and maintenance personnel sends a call request to the signaling server 102, the signaling server 102 sends the call request to the second client 104 of the expert personnel, the second client of the expert personnel sends a request for creating an audio and video call channel to the real-time communication server 105 according to the call request, the real-time communication server 105 creates an audio and video call channel and sends a channel identification to the signaling server 102, the signaling server 102 sends the channel identification to the first client 101 of the operation and maintenance personnel, and the audio and video call between the operation and maintenance personnel and the expert personnel is established through the channel identification. After the expert and the operation and maintenance personnel establish access to the content of the knowledge base system 103 through audio and video communication, the second client 104 of the expert generates a remote interaction signaling, the remote interaction signaling is sent to the signaling server 102, the signaling server 102 sends the remote interaction signaling to the first client 101 of the operation and maintenance personnel, the first client 101 of the operation and maintenance personnel analyzes the interaction signaling to obtain loading information, the knowledge base system 103 pulls the interaction content in the loading information, and the interaction content is synchronously displayed on the user interfaces of the first client 101 of the operation and maintenance personnel and the second client 104 of the expert.

Example 1

Fig. 2 is a schematic flow chart of an interactive processing method of remote operation and maintenance provided in an embodiment of the present application, and an execution subject of the embodiment may be the first client 101 of the operation and maintenance personnel in the embodiment shown in fig. 1, which is not particularly limited herein. As shown in fig. 2, the method includes:

s201: and initiating a call request to establish a connection with the audio-video call of the second client of the expert.

Specifically, the operation and maintenance personnel initiate a call request to the expert personnel, the call request is sent to the signaling server, the signaling server forwards the call request to a second client of the expert personnel, the second client sends a call channel creation request to the real-time communication server, the real-time communication server creates a call channel and sends a call channel identifier to the signaling server, the signaling server forwards the call channel identifier to a first client of the operation and maintenance personnel, and the first client joins the call channel according to the call channel identifier to realize connection.

In this embodiment, the number of persons in remote collaboration supports interactive operations of not less than two persons.

In this embodiment, the signaling server is responsible for synchronization states and synchronization information between different clients, so as to implement a real-time interaction function.

In this embodiment, the Real-time communication server may be a WebRTC (Web Real-time Communication) website Real-time communication server.

In this embodiment, the real-time communication server is responsible for the audio/video call service between different clients.

In this embodiment, the first client is a client used by an operation and maintenance person, and the second client is a client used by an expert person.

In this embodiment, the first client of the operation and maintenance personnel includes, but is not limited to, a smart phone, a notebook computer, and a tablet computer.

In this embodiment, the second client of the expert person includes, but is not limited to, a smart phone, a computer, and a tablet computer.

In this embodiment, the first client and the second client implement cross-platform deployment using hypertext 5 (HTML 5) technology.

In this embodiment, after receiving the request for creating the talk channel sent by the second client, the real-time communication server creates the talk channel, and generates a corresponding talk channel identifier for the talk channel.

S202: receiving a remote interaction signaling sent by a signaling server; after the connection of the audio-video call is established, the second client responds to the communication between expert personnel and operation maintenance personnel through the audio-video call, and then accesses the knowledge base system to generate and send the information to the signaling server; the remote interaction signaling comprises information to be loaded.

Specifically, an operation and maintenance person obtains information of an operation and maintenance site through a first client, the information is displayed on a second client interface of the expert through an audio and video channel, after the operation and maintenance person communicates with the expert, the second client of the expert generates a remote interactive signaling, the remote interactive signaling is sent to a signaling server, and the signaling server sends the remote interactive signaling to the first client of the operation and maintenance person.

The operation and maintenance personnel acquire information of an operation and maintenance site through the image pickup equipment of the first client.

In this embodiment, the remote interaction signaling includes information to be loaded.

The information to be loaded comprises a content identifier and a loading mode.

S203: and analyzing the remote interactive signaling to obtain information to be loaded, wherein the information to be loaded comprises a content identifier and a loading mode.

In this embodiment, the content identifier is ID (Identity Document) identifier of the content to be displayed.

In this embodiment, the loading mode includes, but is not limited to, window pop-up display, progressive display, and interlaced display.

S204: and pulling the interactive content from the knowledge base system according to the content identification.

In this embodiment, the interactive contents include, but are not limited to, pictures, videos, texts, and three-dimensional models.

In this embodiment, the knowledge base system is responsible for storing and querying knowledge contents of each type, and implementing content pulling according to the ID of querying each knowledge content.

S205: and displaying the interactive content according to the loading mode.

In this embodiment, fig. 3 is a schematic diagram of a client display interface provided in the embodiment of the present application.

In this embodiment, the audio-video call is displayed on a first display layer of the user interface of the first client; the interactive content is displayed on a second display layer of the user interface of the first client; and the second display layer is positioned on the first display layer and is transparent.

The first display layer is a video call layer, and the second display layer is an interactive content layer.

According to the embodiment, the audio-video call connection is established by initiating the call request to the expert client through the operation and maintenance personnel client, the remote interaction signaling is generated based on the audio-video call connection expert client, the remote interaction signaling is sent to the signaling server, the remote interaction signaling is sent to the operation and maintenance personnel client through the signaling server, the operation and maintenance personnel client directly displays the interaction content by pulling the interaction content from the knowledge base, and compared with the prior art, the interactive content of the operation and maintenance personnel and the expert personnel is displayed more intuitively, the technical guidance of the expert personnel to the operation and maintenance personnel is facilitated, and the working efficiency of remote assistance is improved.

In one embodiment of the present application, based on the embodiment of fig. 2, following step S205, the following steps are further included:

s206: and responding to the rotation operation of the operation and maintenance personnel on the three-dimensional model, and displaying the rotated three-dimensional model at different angles.

In this embodiment, if the interactive content is a three-dimensional model, the operation and maintenance personnel can view the model information of the three-dimensional model at 720 degrees at the first client.

Specifically, the operation and maintenance personnel realize the interactive operation of the interactive content by controlling the screen area of the first client.

The operation and maintenance personnel can realize interaction of the interactive contents in a mode of finger clicking, finger touching, finger sliding or mouse dragging.

According to the embodiment, when the interactive content is the three-dimensional model, the operation and maintenance personnel acquire model information of different angles of the three-dimensional model by rotating the three-dimensional model, so that the situation of the three-dimensional model can be more intuitively obtained, the remote assistance of expert personnel on the operation and maintenance personnel is facilitated, and the working efficiency of the remote assistance is improved.

s207: and receiving an interactive mark sent by the signaling server, wherein the interactive mark is generated by the second client in response to the marking operation of expert personnel and is sent to the signaling server, and mark position information is recorded in the interactive mark.

Specifically, expert personnel perform marking operation on the display content at a second client, the second client generates an interactive mark, the interactive mark is sent to a signaling server, and the signaling server sends the interactive mark to the first client.

The interactive mark can be a click picture effect triggered by a user, or a drawing effect of the user on the interactive content.

S208: and displaying the interactive mark according to the mark position information.

Specifically, the first client identifies the position information in the interactive mark, and the interactive mark is displayed corresponding to the position information of the display content of the first client.

The interactive mark can stay in the display picture for not less than one second.

According to the embodiment, the expert initiates the interactive mark at the expert client, the interactive mark is sent to the signaling server, the signaling server forwards the interactive mark to the operation and maintenance personnel client, the operation and maintenance personnel client identifies the position information in the interactive mark and displays the interactive mark, so that the operation of synchronizing to the operation and maintenance personnel interface after the expert initiates the mark is realized, and the working efficiency of remote assistance is improved.

In one embodiment of the present application, step S208 includes:

S2081: if the resolution of the first client is different from the resolution of the second client, the mark position information is updated according to the resolution of the second client.

Fig. 4 is a schematic diagram of displaying interactive marks according to an embodiment of the present application.

Specifically, if the resolution of the first client is different from the resolution of the second client, the first client calculates the relative mark position information of the first client according to the resolution of the second client.

For example, if the resolution of the second client is 1080P and the resolution of the first client is 1440P, the tag information will be displayed at a location according to the 1440P resolution.

S2082: and displaying the interactive mark according to the updated mark position information.

Specifically, the first client displays the interactive mark according to the calculated new mark position information.

According to the embodiment, when the resolutions of the operation and maintenance personnel client and the expert personnel client are different, the operation and maintenance personnel client adjusts the position of the interactive mark according to the resolution of the expert client, so that the position of the interactive mark is displayed more accurately, and the working efficiency of remote assistance is improved.

Example two

Fig. 5 is a schematic flow chart of an interactive processing method of remote operation and maintenance according to another embodiment of the present application, and the execution subject of the embodiment may be the second client 104 of the expert in the embodiment shown in fig. 1, which is not particularly limited herein. As shown in fig. 5, the method includes:

S301: receiving a call request sent by a signaling server to establish connection with an audio/video call of a first client of an operation and maintenance person; wherein the call request is generated by the first client in response to operation of the operation and maintenance personnel.

Specifically, the operation and maintenance personnel send a call request to the signaling server through the first client, the signaling server sends the call request to the second client, the second client sends a call channel creation request to the real-time communication server, the real-time communication server creates a call channel and sends a call channel identifier to the signaling server, and the signaling server forwards the call channel identifier to the first client, so that the audio and video call establishment of the operation and maintenance personnel and expert personnel is realized.

In this embodiment, the audio-video call is displayed on the first display layer of the user interface of the second client; the interactive content is displayed on a second display layer of the user interface of the second client; and the second display layer is positioned on the first display layer and is transparent.

S302: and responding to the access operation of the expert personnel to the knowledge base system after communicating with the operation and maintenance personnel through audio and video communication, and generating a remote interaction signaling.

Specifically, after the expert personnel communicate with the operation and maintenance personnel, the content to be sent to the operation and maintenance personnel is established, and a second client of the expert personnel acquires the content to be sent from the knowledge base system, so that a remote interactive signaling is generated.

S303: and sending the remote interaction signaling to the signaling server so that the signaling server sends the remote interaction signaling to the first client, wherein the remote interaction signaling comprises information to be loaded, the remote interaction signaling is used for indicating the first client to analyze the remote interaction signaling to obtain the information to be loaded, the information to be loaded comprises a content identifier and a loading mode, the interactive content is pulled from the knowledge base system according to the content identifier, and the interactive content is displayed according to the loading mode.

Specifically, the signaling server sends the remote interaction signaling to the first client, and the first client analyzes the remote interaction signaling to display the interaction content on the user interface.

The information to be loaded comprises a content identifier and a loading mode.

Fig. 6 is a schematic diagram of sending interactive signaling according to an embodiment of the present application.

The remote interactive signaling sent by the expert personnel is synchronously displayed on the user interfaces of the first client and the second client.

According to the embodiment, the operation and maintenance personnel initiates a call request to expert personnel to establish audio and video call connection, remote Cheng Hudong signaling is generated, the remote interaction signaling is sent to the signaling server, the signaling server sends the remote interaction signaling to the operation and maintenance personnel client, the operation and maintenance personnel client analyzes the remote interaction signaling, pulls the interaction content to the knowledge base and displays the interaction content, the expert personnel can conduct remote technical guidance on the operation and maintenance personnel conveniently, and the working efficiency of remote assistance is improved.

In one embodiment of the present application, based on the embodiment of fig. 5, the following steps are further included after step S303:

s304: and responding to the rotation operation of the expert on the three-dimensional model, and displaying the rotated three-dimensional model at different angles.

In this embodiment, if the interactive content is a three-dimensional model, the expert may view the model information of the three-dimensional model at 720 degrees at the second client.

According to the embodiment, when the interactive content is the three-dimensional model, the expert acquires the model information of different angles of the three-dimensional model by rotating the three-dimensional model, so that the situation of the three-dimensional model can be more intuitively obtained, the remote operation of the expert can be intuitively acquired by the operation and maintenance personnel, and the working efficiency of remote assistance is improved.

s305: and receiving an interactive mark sent by the signaling server, wherein the interactive mark is generated by the first client in response to the marking operation of the operation and maintenance personnel and is sent to the signaling server, and mark position information is recorded in the interactive mark.

Specifically, the operation and maintenance personnel perform marking operation on the display content at the first client, the first client generates an interactive mark, the interactive mark is sent to the signaling server, and the signaling server sends the interactive mark to the second client.

S306: and displaying the interactive mark according to the mark position information.

Specifically, the second client identifies the position information in the interactive mark, and the interactive mark is displayed corresponding to the position information of the display content of the second client.

According to the embodiment, the interactive mark is initiated by the operation and maintenance personnel at the operation and maintenance personnel client, the interactive mark is sent to the signaling server, the signaling server forwards the interactive mark to the expert personnel client, the expert personnel client identifies the position information in the interactive mark, and the mark is displayed on the interface of the expert personnel client, so that the real-time mark initiated by the operation and maintenance personnel to the expert personnel is realized, the remote technical guidance of the developer on the operation and maintenance personnel is facilitated, and the working efficiency of remote assistance is improved.

In one embodiment of the present application, step S306 includes:

s3061: if the resolution of the first client is different from the resolution of the second client, the mark position information is updated according to the resolution of the first client.

Specifically, if the resolution of the second client is different from the resolution of the first client, the second client calculates the relative mark position information of the second client according to the resolution of the first client.

S3062: and displaying the interactive mark according to the updated mark position information.

Specifically, the second client displays the interactive mark according to the calculated new mark position information.

According to the embodiment, when the resolutions of the operation and maintenance personnel client and the expert personnel client are different, the expert personnel client adjusts the position of the interactive mark according to the resolution of the operation and maintenance personnel client, the position of the interactive mark is displayed more accurately, and the working efficiency of remote assistance is improved.

Example III

Referring to fig. 7, an embodiment of an interaction process with a first client of an operation and maintenance person, a signaling server, a knowledge base system, a second client of an expert person, and a real-time communication server is shown.

Fig. 7 is an interaction flow diagram of an interaction processing method for remote operation and maintenance according to another embodiment of the present application, where in this embodiment, an interaction process executed by a first client of an operation and maintenance person, a signaling server, a knowledge base system, a second client of an expert person, and a real-time communication server is taken as an example, and a process of displaying interactive content by the first client is described. As shown in fig. 7, the details are as follows:

s401: the first client of the operation and maintenance personnel sends a call request to the signaling server.

S402: the signaling server sends a call request to a second client of the expert.

S403: the second client receives the call request and sends a request for creating an audio and video call channel to the real-time communication server.

S404: the real-time communication server creates a channel identification and sends the channel identification to the signaling server.

S405: the signaling server sends a channel identification to the first client.

S406: and the first client joins the audio and video call channel according to the channel identification.

S407: the second client generates a remote interaction signaling, wherein the remote interaction signaling comprises information to be loaded, and sends the remote interaction signaling to the signaling server.

S408: the signaling server sends a remote interactive signaling to the first client.

S409: the first client analyzes the remote interaction signaling to obtain information to be loaded, wherein the information to be loaded comprises a content identifier and a loading mode, and a request for pulling the interactive content is sent to the knowledge base system according to the content identifier.

S410: the knowledge base system pulls the interactive content and sends the interactive content to the first client.

S411: and the first client displays the interactive content according to the loading mode.

Example IV

Fig. 8 is a schematic structural diagram of an interaction processing device for remote operation and maintenance according to an embodiment of the present application. As shown in fig. 8, the interaction processing device 80 for remote operation includes: an initiating module 801, a first receiving module 802, a parsing module 803, a pulling module 804 and a display module 805.

An initiating module 801, configured to initiate a call request to establish a connection with an audio/video call of the second client of the expert.

A first receiving module 802, configured to receive a remote interaction signaling sent by a signaling server; after the connection of the audio-video call is established, the second client responds to the communication between expert personnel and operation maintenance personnel through the audio-video call, and then accesses the knowledge base system to generate and send the information to the signaling server; the remote interaction signaling comprises information to be loaded.

The parsing module 803 is configured to parse the remote interaction signaling to obtain information to be loaded, where the information to be loaded includes a content identifier and a loading mode.

The pulling module 804 is configured to pull the interactive content from the knowledge base system according to the content identifier.

And the display module 805 is configured to display the interactive content according to the loading manner.

In one embodiment of the present application, wherein the audio-video call is displayed at a first display layer of a user interface of the first client; the interactive content is displayed on a second display layer of the user interface of the first client; and the second display layer is positioned on the first display layer and is transparent.

In one embodiment of the present application, the interaction processing device 80 for remote operation and maintenance further includes:

the first display module 806 is configured to display the rotated three-dimensional model at different angles in response to a rotation operation of the three-dimensional model by an operator.

the second receiving module 807 is configured to receive an interaction token sent by the signaling server, where the interaction token is generated by the second client in response to the token operation of the expert, and sent to the signaling server, and the interaction token records the token position information.

And a second display module 808 for displaying the interactive mark according to the mark position information.

In one embodiment of the present application, the second display module 808 includes:

the updating unit 8081 is configured to update the marker location information according to the resolution of the second client if the resolution of the first client is different from the resolution of the second client.

And a display unit 8082 for displaying the interactive mark according to the updated mark position information.

The device provided in this embodiment may be used to implement the technical solution of the foregoing method embodiment, and its implementation principle and technical effects are similar, and this embodiment will not be described herein again.

Fig. 9 is a schematic structural diagram of an interaction processing device for remote operation and maintenance according to another embodiment of the present application. As shown in fig. 9, the interaction processing device 90 for remote operation includes: a first receiving module 901, a generating module 902 and a transmitting module 903.

The first receiving module 901 is configured to receive a call request sent by a signaling server, so as to establish connection with an audio/video call of a first client of an operation and maintenance person; wherein the call request is generated by the first client in response to operation of the operation and maintenance personnel.

The generating module 902 is configured to generate a remote interaction signaling in response to an access operation for the knowledge base system after the expert and the operation and maintenance personnel communicate through an audio/video call.

The sending module 903 is configured to send a remote interaction signaling to the signaling server, so that the signaling server sends the remote interaction signaling to the first client, where the remote interaction signaling includes information to be loaded, and the remote interaction signaling is configured to instruct the first client to parse the remote interaction signaling to obtain the information to be loaded, where the information to be loaded includes a content identifier and a loading manner, pull the interactive content from the knowledge base system according to the content identifier, and display the interactive content according to the loading manner.

In one embodiment of the present application, the audio-video call is displayed on a first display layer of a user interface of the second client; the interactive content is displayed on a second display layer of the user interface of the second client; and the second display layer is positioned on the first display layer and is transparent.

In one embodiment of the present application, the interaction processing device 90 for remote operation and maintenance further includes:

the first display module 904 is configured to display the rotated three-dimensional model at different angles in response to a rotation operation of the three-dimensional model by an expert.

the second receiving module 905 is configured to receive an interaction flag sent by the signaling server, where the interaction flag is generated by the first client in response to a flag operation of an operation and maintenance person and sent to the signaling server, and tag location information is recorded in the interaction flag.

And a second display module 906, configured to display the interactive mark according to the mark position information.

In one embodiment of the present application, the second display module 906 includes:

the updating unit 9061 is configured to update the marker location information according to the resolution of the first client if the resolution of the first client is different from the resolution of the second client.

And a display unit 9062 for displaying the interactive mark according to the updated mark position information.

Example five

Fig. 10 is a schematic hardware structure of a terminal device according to an embodiment of the present application. As shown in fig. 10, the terminal device includes: at least one processor 1001 and memory 1002; the memory stores computer-executable instructions; at least one processor executes computer-executable instructions stored in the memory, causing the at least one processor to perform the interactive processing method of remote operation and maintenance as described above.

Alternatively, the memory 1002 may be separate or integrated with the processor 1001.

When the memory 1002 is provided separately, the terminal device further comprises a bus 1003 for connecting the memory 1002 and the processor 1001.

Example six

The embodiment of the application also provides a computer storage medium, wherein computer execution instructions are stored in the computer storage medium, and when a processor executes the computer execution instructions, the interactive processing method of the remote operation and maintenance is realized.

Example seven

The embodiment of the application also provides a computer program product, which comprises a computer program, and when the computer program is executed by a processor, the interactive processing method of the remote operation and maintenance is realized.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the above-described division of modules is merely a logical function division, and there may be other manners of division in actual implementation, for example, multiple modules may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or modules, which may be in electrical, mechanical, or other forms.

The modules described above as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

In addition, each functional module in each embodiment of the present application may be integrated in one processing unit, or each module may exist alone physically, or two or more modules may be integrated in one unit. The units formed by the modules can be realized in a form of hardware or a form of hardware and software functional units.

The integrated modules, which are implemented in the form of software functional modules, may be stored in a computer readable storage medium. The software functional module is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform some of the steps of the methods described in the embodiments of the present application.

It should be understood that the above processor may be a central processing unit (Central Processing Unit, abbreviated as CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, abbreviated as DSP), application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor for execution, or in a combination of hardware and software modules in a processor for execution.

The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile memory NVM, such as at least one magnetic disk memory, and may also be a U-disk, a removable hard disk, a read-only memory, a magnetic disk or optical disk, etc.

The bus may be an industry standard architecture (Industry Standard Architecture, ISA) bus, an external device interconnect (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, the buses in the drawings of the present application are not limited to only one bus or one type of bus.

The storage medium may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an application specific integrated circuit (Application Specific Integrated Circuits, ASIC for short). It is also possible that the processor and the storage medium reside as discrete components in an electronic device or a master device.

Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the method embodiments described above may be performed by hardware associated with program instructions. The foregoing program may be stored in a computer readable storage medium. The program, when executed, performs steps including the method embodiments described above; and the aforementioned storage medium includes: various media that can store program code, such as ROM, RAM, magnetic or optical disks.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. The interactive processing method for the remote operation and maintenance is characterized by being applied to a first client of an operation and maintenance person and comprising the following steps of:

and displaying the interactive content according to the loading mode.

2. The method of claim 1, wherein the audio-video call is displayed at a first display layer of a user interface of the first client; the interactive content is displayed on a second display layer of the user interface of the first client; and the second display layer is positioned above the first display layer and is transparent.

3. The method of claim 1, wherein the interactive content is a three-dimensional model;

after displaying the interactive content according to the loading mode, the method further comprises:

and responding to the rotation operation of the operation and maintenance personnel on the three-dimensional model, and displaying the rotated three-dimensional model with different angles.

4. The method of claim 1, wherein after displaying the interactive content, further comprising:

receiving an interactive mark sent by the signaling server, wherein the interactive mark is generated by the second client in response to the marking operation of the expert, and is sent to the signaling server, and mark position information is recorded in the interactive mark;

And displaying the interactive mark according to the mark position information.

5. The method of claim 4, wherein displaying the interactive marker according to the marker location information comprises:

if the resolution of the first client is different from the resolution of the second client, updating the mark position information according to the resolution of the second client;

and displaying the interactive mark according to the updated mark position information.

6. The interactive processing method for the remote operation and maintenance is characterized by being applied to a second client of expert personnel and comprising the following steps of:

7. An interactive processing device for remote operation and maintenance, which is applied to a first client of operation and maintenance personnel, comprises:

8. An interactive processing device for remote operation and maintenance, which is applied to a second client of expert personnel, comprising:

9. A terminal device, comprising: at least one processor and memory;

The memory stores computer-executable instructions;

the at least one processor executing computer-executable instructions stored in the memory causes the at least one processor to perform the interactive processing method of remote operation and maintenance as claimed in any one of claims 1 to 5, or claim 6.

10. A computer storage medium having stored therein computer executable instructions which, when executed by a processor, implement the interactive processing method of remote operation and maintenance according to any one of claims 1 to 5, or claim 6.