CN111294568B - Method for realizing video fusion by medical remote control - Google Patents
Method for realizing video fusion by medical remote control Download PDFInfo
- Publication number
- CN111294568B CN111294568B CN202010174550.0A CN202010174550A CN111294568B CN 111294568 B CN111294568 B CN 111294568B CN 202010174550 A CN202010174550 A CN 202010174550A CN 111294568 B CN111294568 B CN 111294568B
- Authority
- CN
- China
- Prior art keywords
- remote
- field
- video
- equipment
- fusion display
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 163
- 238000000034 method Methods 0.000 title claims abstract description 12
- 238000004891 communication Methods 0.000 claims description 13
- 230000008014 freezing Effects 0.000 claims description 12
- 238000007710 freezing Methods 0.000 claims description 12
- 238000007500 overflow downdraw method Methods 0.000 claims description 9
- 241000352333 Amegilla alpha Species 0.000 claims description 5
- 238000012546 transfer Methods 0.000 claims description 4
- 241000903210 Bryconamericus alpha Species 0.000 claims description 3
- 238000012800 visualization Methods 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 241000965481 Darksidea alpha Species 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 239000004984 smart glass Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Selective Calling Equipment (AREA)
Abstract
The invention discloses a method for realizing video fusion by medical remote control, and relates to the technical field of video fusion. The system comprises a field-end video acquisition device, a field-end audio input device, a field-end audio output device, a field-end video fusion display device, a remote-end video acquisition device, a remote-end audio input device, a remote-end audio output device, a remote-end video fusion display device and a server; the field end video fusion display equipment comprises a field end operating system, and the remote end video fusion display equipment comprises a remote end operating system and is used for displaying a video picture fused by the remote end, the field end and the field end. The invention integrates the site-side video and the remote-side video accurately in real time, and realizes the visualization and the accuracy of remote guidance.
Description
Technical Field
The invention relates to the technical field of video fusion, in particular to a method for realizing video fusion through remote control.
Background
In recent years, with the development of technologies such as computer technology, remote guidance systems have also been developed rapidly. At present, most of remote guidance systems in the market are carried out based on a remote video system, remote experts carry out voice guidance according to scene pictures transmitted on site, and due to the fact that the two parties have different understanding on contents and different knowledge structures, the problems that work with certain technical difficulty or complex operation is difficult to solve quickly and accurately only through site video and voice guidance are solved. With the development of augmented reality technology, portable wearable devices such as smart glasses are also added to a remote guidance system, and although field technicians use smart glasses to liberate hands, due to the defect of an image algorithm, accurate fusion of virtual images and real views cannot be achieved, so that guidance of field images and voice still remains. And augmented reality equipment such as intelligent glasses are higher in cost.
In order to solve the above problems, it is necessary to invent a method for remotely controlling and implementing video fusion, in which videos of a remote expert and a field technician are precisely fused in the same scene, guidance opinions (gestures or guidance paths) of the remote expert are fused and superimposed in real time into an operation picture of the field technician, so as to implement visualization and precision of remote guidance.
Disclosure of Invention
The invention aims to solve the technical problem of providing a system and a method for realizing video fusion by fusing remote control of site-side and remote-side videos.
In order to solve the problems, the technical scheme of the invention is as follows:
a system for remotely controlling and realizing video fusion.
The system comprises a field-end video acquisition device, a field-end audio input device, a field-end audio output device, a field-end video fusion display device, a remote-end video acquisition device, a remote-end audio input device, a remote-end audio output device, a remote-end video fusion display device and a server;
the field end video fusion display equipment is respectively in communication connection with the field end video acquisition equipment, the field end audio input equipment and the field end audio output equipment, and the field end video fusion display equipment is in communication connection with the server;
the remote end video fusion display equipment is respectively in communication connection with the remote end video acquisition equipment, the remote end audio input equipment and the remote end audio output equipment, and the remote end video fusion display equipment is in communication connection with the server;
the field end video acquisition equipment is used for acquiring field end video pictures;
the field end audio input equipment is used for acquiring the audio of field operators;
the field end audio output equipment is used for playing audio input by a remote end;
the field end video fusion display equipment comprises a field end operating system and is used for displaying a video picture fused by a field end, a remote end and both ends;
the remote-end video acquisition equipment is used for acquiring remote-end video pictures;
the remote end audio input equipment is used for acquiring the audio of a remote operator;
the remote end audio output equipment is used for playing audio input by a field end;
the remote end video fusion display equipment comprises a remote end operating system and is used for displaying a remote end, a field end and a video picture fused by the remote end, the field end and the field end;
the field end operating system and the remote end operating system respectively comprise a fusion module, and the fusion module is used for fusing videos of the field end and the remote end.
Further, the on-site end video acquisition equipment and the remote end video acquisition equipment are high definition video acquisition equipment, the high definition video acquisition equipment is high definition digtal camera, on-site end audio input equipment and remote end audio input equipment are wireless microphone or wired microphone, on-site end audio output equipment and remote end audio output equipment are multimedia smart loudspeaker boxes.
Further, the field-side operating system and the remote-side operating system respectively include a virtual tool module, the virtual tool module is configured to display a virtual tool that is operable by both sides and synchronously displayed in video pictures displayed by both sides, and the virtual tool is a virtual brush or a virtual arrow.
Furthermore, the field-side operating system and the remote-side operating system respectively comprise a picture freezing module, and the picture freezing module is used for synchronously freezing video pictures displayed by the field-side video fusion display equipment and the remote-side video fusion display equipment according to the operation of field-side personnel or remote-side personnel.
Furthermore, the field-side video fusion display device is electrically connected with the field-side video acquisition device, the field-side audio input device is electrically connected with the field-side audio output device, and the remote-side video fusion display device is electrically connected with the remote-side video acquisition device, the remote-side audio input device and the remote-side audio output device.
Furthermore, the field-side video fusion display device is connected with the server through an internal local area network or the internet, and the remote-side video fusion display device is connected with the server through the internal local area network or the internet.
Furthermore, the system also comprises a field-end movable trolley and a remote-end movable trolley, wherein the field-end video acquisition equipment, the field-end audio input equipment, the field-end audio output equipment and the field-end video fusion display equipment are arranged on the field-end movable trolley, and the remote-end video acquisition equipment, the remote-end audio input equipment, the remote-end audio output equipment and the remote-end video fusion display equipment are arranged on the remote-end movable trolley.
8. A method for realizing video fusion by remote control comprises the following steps:
s1, installing a field-end video acquisition device, a field-end audio input device, a field-end audio output device and a field-end video fusion display device on a field-end movable trolley, installing a remote-end video acquisition device, a remote-end audio input device, a remote-end audio output device and a remote-end video fusion display device on a remote-end movable trolley, connecting the field-end video acquisition device, the field-end audio input device and the field-end audio output device with the field-end video fusion display device, connecting the remote-end video acquisition device, the remote-end audio input device and the remote-end audio output device with the remote-end video fusion display device, and respectively connecting the field-end video fusion display device and the remote-end video fusion display device with a server through a network;
s2, logging in a field end operating system of the field end video fusion display device, logging in a remote end operating system of the remote end video fusion display device, creating a video room in the remote end operating system, transmitting an invitation for establishing an audio and video connection channel through server transfer, inviting a field end to enter the video room, receiving the invitation by the field end operating system, successfully establishing audio and video connection with the remote end operating system through the server transfer, entering the video room, and directly connecting the remote end video room with the field end video room;
s3, the field-end video acquisition device acquires a field-end video picture and outputs the field-end video picture to the field-end video fusion display device, the field-end video fusion display device transmits the field-end video picture to the remote-end video fusion display device, the field-end audio input device acquires the audio of a field operator and outputs the audio of the field operator to the field-end video fusion display device, the field-end video fusion display device transmits the audio of the field operator to the remote-end video fusion display device, and the field-end audio output device plays the audio input by the remote end;
the remote end video acquisition equipment acquires a remote end video picture and outputs the remote end video picture to the remote end video fusion display equipment, the remote end video fusion display equipment transmits the remote end video picture to the field end video fusion display equipment, the remote end audio input equipment acquires the audio of a remote operator and outputs the audio of the remote operator to the remote end video fusion display equipment, the remote end video fusion display equipment transmits the audio of the remote operator to the field end video fusion display equipment, and the remote end audio output equipment plays the audio input by the field end;
s4, fusing a field-end video picture and a remote-end video picture by a fusion module in the field-end operating system, and displaying the fused field-end video picture and the remote-end video picture on field-end video fusion display equipment; a fusion module in the remote end operation system fuses the field end video picture and the remote end video picture and displays the fused pictures on the remote end video fusion display equipment;
s5, an operator at the site end or the remote end can edit the video picture by using a virtual tool or a picture freezing function;
the fused video in step S4 adopts a positive-film-overlap fusion method or a brightness matting and general fusion method.
Further, the positive film bottom-folding fusion method is that positive film bottom-folding fusion is adopted for a field-end video A and a remote-end video B, wherein A is an upper-layer video source, B is a lower-layer video source, one pixel point vector4 a in A is taken, and one pixel point vector 4B in B is taken;
multiplying a by b, taking the linear mixed value to obtain the new value of the pixel point R, drawing the pixel point R on a screen,
R＝vector4(mix(a.rgb,a.rgb*b.rgb,b.alpha),a.alpha)。
compared with the prior art, the invention has the following beneficial effects:
1. the invention integrates the site-side video and the remote-side video accurately in real time, the expert and the site technician quickly establish real-time audio-video connection, the communication is carried out through gestures and voice, and the required actions and contents are visually displayed, so that the site technician can understand and execute the actions and the contents more quickly, and the visualization and the precision of remote guidance are realized;
2. the movable trolley is arranged, so that the equipment is simple and convenient to install and move.
Drawings
The following describes embodiments of the present invention in further detail with reference to the accompanying drawings.
FIG. 1 is a schematic diagram of the system of the present invention;
wherein, 1 is a field-end video acquisition device; 2, field end audio input equipment; 3, field end audio output equipment; 4, on-site video fusion display equipment; 5 is a remote end video acquisition device; 6 is remote audio input equipment; 7 is remote audio output equipment; 8, remote end video fusion display equipment; 9 is a server; and 10 is a movable trolley.
Detailed Description
In order to make the technical means, the original characteristics, the achieved purpose and the efficacy of the invention easy to understand, the invention is further described with reference to the specific drawings.
Example (b):
as shown in fig. 1, a system for remotely controlling to implement video fusion includes a field-side video collection device 1, a field-side audio input device 2, a field-side audio output device 3, a field-side video fusion display device 4, a remote-side video collection device 5, a remote-side audio input device 6, a remote-side audio output device 7, a remote-side video fusion display device 8, and a server 9.
The field-side video fusion display device 4 is respectively in communication connection with the field-side video acquisition device 1, the field-side audio input device 2 and the field-side audio output device 3. Further, the connection mode is electrical connection. The field end video fusion display device 4 is in communication connection with the server 9. Furthermore, the connection mode is internal local area network or internet connection, and the data is encrypted data.
The remote end video fusion display device 8 is respectively in communication connection with the remote end video acquisition device 5, the remote end audio input device 6 and the remote end audio output device 7. Further, the connection mode is electrical connection. The remote end video fusion display device 8 is in communication connection with the server 9; furthermore, the connection mode is internal local area network or internet connection, and the data is encrypted data.
The remote-end video fusion display system is characterized by further comprising a field-end movable trolley 10 and a remote-end movable trolley 10, wherein the field-end video acquisition device 1, the field-end audio input device 2, the field-end audio output device 3 and the field-end video fusion display device 4 are mounted on the field-end movable trolley 10, and the remote-end video acquisition device 5, the remote-end audio input device 6, the remote-end audio output device 7 and the remote-end video fusion display device 8 are mounted on the remote-end movable trolley 10.
The field video acquisition equipment 1 is used for acquiring field video pictures such as surgical fields and the like, acquiring video pictures of equipment such as mirror videos, ultrasound, a DaVinci robot, a surgical field camera and the like, and outputting the video pictures to the field video fusion display equipment 4. Further, the on-site video acquisition device 1 is a high-definition video acquisition device, and the high-definition video acquisition device is a high-definition camera.
The field end audio input device 2 is used for collecting audio of field operators, transmitting the audio to the server 9 and the remote end video fusion display device 8 through the field end video fusion display device 4, and playing the audio through the remote end audio output device 7. The live audio input device 2 is a wireless microphone or a wired microphone or a desktop microphone.
The field end audio output device 3 is used for playing audio input by the remote end. Further, the on-site audio output device 3 is a multimedia smart speaker.
The field-side video fusion display device 4 comprises a field-side operating system, and the field-side video fusion display device 4 is used for displaying a video picture fused by the field side, the remote side and both sides. The field-side video fusion display device 4 can fuse videos collected by the field side and the remote side.
The remote-end video acquisition equipment 5 is used for acquiring remote-end video pictures such as remote expert guide opinions and the like.
The remote-end audio input device 6 is used to capture audio of a remote operator, such as an expert or the like. Further, the remote audio input device 6 is a wireless microphone or a wired microphone or a desktop microphone.
The remote end audio output device 7 is used for playing the audio input by the field end. Further, the remote audio output device 7 is a multimedia smart speaker.
The remote end video fusion display device 8 comprises a remote end operating system, and the remote end video fusion display device 8 is used for displaying a video picture fused by the remote end, the site end and both the site end. The remote end video fusion display device 8 can fuse videos collected by the field end and the remote end.
The server 9 framework adopts multi-service decoupling, serves for multi-level deployment, supports large concurrency and establishes a user database.
The field end operating system and the remote end operating system respectively comprise a fusion module, and the fusion module is used for fusing videos of the field end and videos of the remote end.
The field-end operating system and the remote-end operating system respectively comprise a virtual tool module, the virtual tool module is used for displaying a virtual tool which can be operated by both sides and can be synchronously displayed in video pictures displayed by both sides, and the virtual tool is a virtual brush or a virtual arrow.
The field end operating system and the remote end operating system respectively comprise a picture freezing module, and the picture freezing module is used for synchronously freezing video pictures displayed by the field end video fusion display device 4 and the remote end video fusion display device 8 according to the operation of field end personnel or remote end personnel.
The remote-end expert may point to certain locations in the live-end video using a virtual brush, arrow, or virtual tool on the screen of the remote-end video fusion display device 8, and the live-end video fusion display device 4 will simultaneously display this information.
The field-side operator can point to some position in the field-side video by using a virtual brush, an arrow or a virtual tool on the screen of the field-side video fusion display device 4, and the remote-side video fusion display device 8 can synchronously display the information.
On the remote end video fusion display device 8, the video image collected by the field end can be frozen, and the video on the field end video fusion display device 4 can be synchronously frozen.
The remote-end expert may point at certain locations in the image on the frozen video image using a virtual brush, arrow, or virtual tool, and the live-end video fusion display device 4 will simultaneously display this information. The same applies to the field terminal.
A method for realizing video fusion by remote control comprises the following steps:
s1, a field-end video acquisition device 1, a field-end audio input device 2, a field-end audio output device 3 and a field-end video fusion display device 4 are arranged on a field-end movable trolley 10, installing a remote-end video acquisition device 5, a remote-end audio input device 6, a remote-end audio output device 7 and a remote-end video fusion display device 8 on a remote-end movable trolley 10, connecting a field-end video acquisition device 1, a field-end audio input device 2 and a field-end audio output device 3 with the field-end video fusion display device 4, connecting the remote-end video acquisition device 5, the remote-end audio input device 6 and the remote-end audio output device 7 with the remote-end video fusion display device 8, and respectively connecting the field-end video fusion display device 4 and the remote-end video fusion display device 8 with a server 9 through a network;
s2, logging in a field end operating system of the field end video fusion display device 4, logging in a remote end operating system of the remote end video fusion display device 8, creating a video room in the remote end operating system, transferring the video room through a server 9, sending an invitation for establishing an audio and video connection channel, inviting the field end to enter the video room, accepting the invitation by the field end operating system, successfully establishing audio and video connection with the remote end operating system through the server, entering the video room, directly connecting the remote end video room with the field end video room, and performing point-to-point audio and video transmission without transferring the audio and video through the server 9;
s3, the field-end video acquisition device 1 acquires a field-end video picture and outputs the field-end video picture to the field-end video fusion display device 4, the field-end video fusion display device 4 transmits the field-end video picture to the remote-end video fusion display device 8 through the server 9, the field-end audio input device 2 acquires the audio of a field operator and outputs the audio of the field operator to the field-end video fusion display device 4, the field-end video fusion display device 4 transmits the audio of the field operator to the remote-end video fusion display device 8, and the field-end audio output device 3 plays the audio input by the remote end;
the remote end video acquisition equipment 5 acquires a remote end video picture and outputs the remote end video picture to the remote end video fusion display equipment 8, the remote end video fusion display equipment 8 transmits the remote end video picture to the field end video fusion display equipment 4 through a server 9, the remote end audio input equipment 6 acquires the audio of a remote operator and outputs the audio of the remote operator to the remote end video fusion display equipment 8, the remote end video fusion display equipment 8 transmits the audio of the remote operator to the field end video fusion display equipment 4, and the remote end audio output equipment 7 plays the audio input by the field end;
s4, fusing a field-end video picture and a remote-end video picture by a fusion module in the field-end operating system, and displaying the fused pictures on the field-end video fusion display equipment 4; a fusion module in the remote end operation system fuses the field end video picture and the remote end video picture and displays the fused pictures on the remote end video fusion display device 8;
s5, an operator at the site end or the remote end can edit the video picture by using a virtual tool or a picture freezing function;
and if one of the site end and the remote end finishes audio and video connection, the connection of the site end and the remote end is finished. If it is desired to re-establish continuity, the connection is again established via the server 9.
The fused video in step S4 adopts a positive-film-overlap fusion method or a brightness matting and general fusion method.
The positive film bottom-folding fusion method comprises the steps of adopting positive film bottom-folding fusion to a field-end video A and a remote-end video B, wherein A is an upper-layer video source, B is a lower-layer video source, and assuming that a pixel vector4 a in the video source A is taken and a pixel vector 4B in the video source B is taken; (ii) a
Multiplying a by b, taking the linear mixed value to obtain the new value of the pixel point R, drawing the pixel point R on a screen,
R＝vector4(mix(a.rgb,a.rgb*b.rgb,b.alpha),a.alpha)。
the brightness matting and general fusion method is used for the field end video A and the remote end video B, the brightness matting and general fusion are carried out on the field end video A and the remote end video B, the video A is an upper layer video source, the video B is a lower layer video source, and a pixel vector4 a in the video A source and a pixel vector 4B in the video B source are assumed to be taken; (ii) a
Deducting the white value in a by using the brightness matting to obtain a pixel value d, wherein c is an intermediate variable,
c＝dot(a.rgb,new vector3(0.2125,0.7154,0.0721))
d＝vector4(a.rgb,min(a.alpha,1-smoothstep(0.4,0.6,c)))
then, a common image superposition mode is used for obtaining the pixel point R, and the pixel point R is drawn on a screen
R＝vector4(mix(a.rgb,d.alpha,d.alpha),a.alpha)。
It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (2)
1. A method for realizing video fusion by medical remote control is characterized in that: the system comprises a field-end video acquisition device, a field-end audio input device, a field-end audio output device, a field-end video fusion display device, a remote-end video acquisition device, a remote-end audio input device, a remote-end audio output device, a remote-end video fusion display device and a server;
the field end video fusion display equipment is respectively in communication connection with the field end video acquisition equipment, the field end audio input equipment and the field end audio output equipment, and the field end video fusion display equipment is in communication connection with the server;
the remote end video fusion display equipment is respectively in communication connection with the remote end video acquisition equipment, the remote end audio input equipment and the remote end audio output equipment, and the remote end video fusion display equipment is in communication connection with the server;
the field end video acquisition equipment is used for acquiring field end video pictures;
the field end audio input equipment is used for acquiring the audio of field operators;
the field end audio output equipment is used for playing audio input by a remote end;
the field end video fusion display equipment comprises a field end operating system and is used for displaying a video picture fused by a field end, a remote end and both ends;
the remote-end video acquisition equipment is used for acquiring remote-end video pictures;
the remote end audio input equipment is used for acquiring the audio of a remote operator;
the remote end audio output equipment is used for playing audio input by a field end;
the remote end video fusion display equipment comprises a remote end operating system and is used for displaying a remote end, a field end and a video picture fused by the remote end, the field end and the field end;
the field end operating system and the remote end operating system respectively comprise a fusion module, and the fusion module is used for fusing videos of a field end and a remote end;
the field-end operating system and the remote-end operating system respectively comprise virtual tool modules, the virtual tool modules are used for displaying a virtual tool which can be operated by both sides and can be synchronously displayed in video pictures displayed by both sides, and the virtual tool is a virtual brush or a virtual arrow;
the field-end video acquisition equipment and the remote-end video acquisition equipment are high-definition video acquisition equipment, the high-definition video acquisition equipment is a high-definition camera, the field-end audio input equipment and the remote-end audio input equipment are wireless microphones or wired microphones, and the field-end audio output equipment and the remote-end audio output equipment are multimedia intelligent sound boxes;
the field-end video acquisition device, the field-end audio input device, the field-end audio output device and the field-end video fusion display device are mounted on the field-end movable trolley, and the remote-end video acquisition device, the remote-end audio input device, the remote-end audio output device and the remote-end video fusion display device are mounted on the remote-end movable trolley;
the field end operating system and the remote end operating system respectively comprise a picture freezing module, and the picture freezing module is used for synchronously freezing video pictures displayed by the field end video fusion display equipment and the remote end video fusion display equipment according to the operation of field end personnel or remote end personnel;
the field end video fusion display device is respectively and electrically connected with the field end video acquisition device, the field end audio input device is electrically connected with the field end audio output device, and the remote end video fusion display device is respectively and electrically connected with the remote end video acquisition device, the remote end audio input device and the remote end audio output device;
the field end video fusion display equipment is connected with the server through an internal local area network or the Internet, and the remote end video fusion display equipment is connected with the server through the internal local area network or the Internet;
the method comprises the following steps:
s1, installing a field-end video acquisition device, a field-end audio input device, a field-end audio output device and a field-end video fusion display device on a field-end movable trolley, installing a remote-end video acquisition device, a remote-end audio input device, a remote-end audio output device and a remote-end video fusion display device on a remote-end movable trolley, connecting the field-end video acquisition device, the field-end audio input device and the field-end audio output device with the field-end video fusion display device, connecting the remote-end video acquisition device, the remote-end audio input device and the remote-end audio output device with the remote-end video fusion display device, and respectively connecting the field-end video fusion display device and the remote-end video fusion display device with a server through a network;
s2, logging in a field end operating system of the field end video fusion display device, logging in a remote end operating system of the remote end video fusion display device, creating a video room in the remote end operating system, transmitting an invitation for establishing an audio and video connection channel through server transfer, inviting a field end to enter the video room, receiving the invitation by the field end operating system, successfully establishing audio and video connection with the remote end operating system through the server transfer, entering the video room, and directly connecting the remote end video room with the field end video room;
s3, the field-end video acquisition device acquires a field-end video picture and outputs the field-end video picture to the field-end video fusion display device, the field-end video fusion display device transmits the field-end video picture to the remote-end video fusion display device, the field-end audio input device acquires the audio of a field operator and outputs the audio of the field operator to the field-end video fusion display device, the field-end video fusion display device transmits the audio of the field operator to the remote-end video fusion display device, and the field-end audio output device plays the audio input by the remote end;
the remote end video acquisition equipment acquires a remote end video picture and outputs the remote end video picture to the remote end video fusion display equipment, the remote end video fusion display equipment transmits the remote end video picture to the field end video fusion display equipment, the remote end audio input equipment acquires the audio of a remote operator and outputs the audio of the remote operator to the remote end video fusion display equipment, the remote end video fusion display equipment transmits the audio of the remote operator to the field end video fusion display equipment, and the remote end audio output equipment plays the audio input by the field end;
s4, fusing a field-end video picture and a remote-end video picture by a fusion module in the field-end operating system, and displaying the fused field-end video picture and the remote-end video picture on field-end video fusion display equipment; a fusion module in the remote end operation system fuses the field end video picture and the remote end video picture and displays the fused pictures on the remote end video fusion display equipment;
s5, an operator at the site end or the remote end can edit the video picture by using a virtual tool or a picture freezing function;
wherein, the fusion video in the step S4 adopts a positive film-overlapped fusion method.
2. The method for realizing video fusion by medical remote control according to claim 1, wherein the method comprises the following steps: the positive film bottom-folding fusion method comprises the steps of adopting positive film bottom-folding fusion to a field-end video A and a remote-end video B, wherein A is an upper-layer video source, B is a lower-layer video source, one pixel point vector4 a in A is taken, and one pixel point vector 4B in B is taken;
multiplying a by b, taking the linear mixed value to obtain the new value of the pixel point R, drawing the pixel point R on a screen,
R＝vector4(mix(a.rgb,a.rgb*b.rgb,b.alpha),a.alpha)。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010174550.0A CN111294568B (en) | 2020-03-13 | 2020-03-13 | Method for realizing video fusion by medical remote control |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010174550.0A CN111294568B (en) | 2020-03-13 | 2020-03-13 | Method for realizing video fusion by medical remote control |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111294568A CN111294568A (en) | 2020-06-16 |
| CN111294568B true CN111294568B (en) | 2020-12-04 |
Family
ID=71020953
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010174550.0A Active CN111294568B (en) | 2020-03-13 | 2020-03-13 | Method for realizing video fusion by medical remote control |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111294568B (en) |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101115020B1 (en) * | 2010-08-16 | 2012-03-06 | 주식회사 케이엠티엘 | A method and system for recording, saving and providing surgery video in real time |
| US10363102B2 (en) * | 2011-12-30 | 2019-07-30 | Mako Surgical Corp. | Integrated surgery method |
| CN202515778U (en) * | 2012-01-18 | 2012-11-07 | 深圳市纽泰克电子有限公司 | Movable on-site consultation trolley |
| CN202618804U (en) * | 2012-05-16 | 2012-12-26 | 深圳市纽泰克电子有限公司 | Mobile on-spot consultation trolley with single-screen structure |
| CN202995738U (en) * | 2013-01-11 | 2013-06-12 | 北京蓝卫通科技有限公司 | Multifunctional remote medical movable workstation |
| CN104038722B (en) * | 2013-03-06 | 2018-04-06 | 中兴通讯股份有限公司 | The content interaction method and system of a kind of video conference |
| CN105957045A (en) * | 2016-04-29 | 2016-09-21 | 珠海市魅族科技有限公司 | Picture synthesis method and device |
| CN106339094B (en) * | 2016-09-05 | 2019-02-26 | 山东万腾电子科技有限公司 | Interactive remote expert cooperation examination and repair system and method based on augmented reality |
| CN106650257A (en) * | 2016-12-21 | 2017-05-10 | 深圳市易特科信息技术有限公司 | Spot diagnosis and treatment terminal for remote cure, remote diagnosis and treatment terminal for remote cure and cooperative diagnosis and treatment system |
| CN106603679A (en) * | 2016-12-21 | 2017-04-26 | 深圳市易特科信息技术有限公司 | Internet remote treatment synchronization system and method |
| CN110164530A (en) * | 2019-06-19 | 2019-08-23 | 深圳市医泰天下科技有限公司 | A kind of mobile device of tele-medicine |
| CN110290340A (en) * | 2019-07-02 | 2019-09-27 | 江苏达实久信数字医疗科技有限公司 | A kind of medical operating teaching system and method |
-
2020
- 2020-03-13 CN CN202010174550.0A patent/CN111294568B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111294568A (en) | 2020-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| De Cunha et al. | The MIDSTEP system for ultrasound guided remote telesurgery | |
| CN110349456B (en) | Intelligent control system, remote control terminal and classroom terminal of interactive classroom | |
| CN110333837B (en) | Conference system, communication method and device | |
| JPH11220706A (en) | Video telephone system | |
| WO2020151432A1 (en) | Data processing method and system for intelligent house viewing | |
| EP3253022A1 (en) | System, equipment and method for performing and documenting in real-time a remotely assisted professional procedure | |
| CN107809609A (en) | Video monitoring conference system based on touch apparatus | |
| TW480425B (en) | Remote assist system | |
| CN105933637A (en) | Video communication method and system | |
| Leutert et al. | Projector-based augmented reality for telemaintenance support | |
| CN110830746A (en) | Remote operation assisting system based on video call and augmented reality technology | |
| CN111294568B (en) | Method for realizing video fusion by medical remote control | |
| JP5377854B2 (en) | Dissection sample sharing system | |
| WO2008066705A1 (en) | Image capture apparatus with indicator | |
| WO2005074303A1 (en) | Stereoscopic projection-based display for surgical robot | |
| CN116795464A (en) | Method for realizing remote assistance and related equipment | |
| CN215770538U (en) | Augmented reality system for remote consultation | |
| CN216017008U (en) | Operating room-based data transmission system | |
| CN106231260B (en) | Transformer substation video monitoring method, device and system based on threedimensional model | |
| US20220181001A1 (en) | Operating room control and communication system | |
| JP2920449B2 (en) | Remote technical support system | |
| Dinevski et al. | Video communication in telemedicine | |
| KR20020058189A (en) | Method and apparatus for an operation conference in internet base | |
| CN113470461A (en) | Microscopic interaction method and device for uploading information through image acquisition module | |
| JP2007003844A (en) | Apparatus for operating microscope installed in remote place |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 100082, 9th Floor, Building 1, No. 32 Xizhimen North Street, Haidian District, Beijing 1004A23 Patentee after: Beijing Weizhuo Zhiyuan Medical Technology Co.,Ltd. Country or region after: China Address before: Room 901, Building 4, Hanwei International Zone 4, Fengtai District, Beijing, 100070 Patentee before: VISUAL3D MEDICAL TECHNOLOGY DEVELOPMENT Co.,Ltd. Country or region before: China |
|
| CP03 | Change of name, title or address |