CN115665102A - Intraoperative talking hypermedia recording method - Google Patents
Intraoperative talking hypermedia recording method Download PDFInfo
- Publication number
- CN115665102A CN115665102A CN202211245131.7A CN202211245131A CN115665102A CN 115665102 A CN115665102 A CN 115665102A CN 202211245131 A CN202211245131 A CN 202211245131A CN 115665102 A CN115665102 A CN 115665102A
- Authority
- CN
- China
- Prior art keywords
- video
- data
- audio
- conversation
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention relates to a hypermedia recording method for intraoperative conversation, which comprises the following specific recording steps: data acquisition → data processing → data network transmission → conversation communication and recording, wherein the data acquisition comprises physical audio and video sources output by a plurality of paths of medical imaging equipment and audio acquisition equipment and preoperative examination and inspection information, the data processing is to respectively flow the acquired data into independent flow pipelines, the PC realizes the reading control of different video data streams according to the identification information on each flow pipeline, any path of audio and video signals can be called at any time as the current picture displayed by the PC, the data network transmission is to convert the frame stream of the audio and video original data into network stream by the PC, and the network stream is provided for the conversation terminal machine to read, so that the mutual communication of conversation contents is realized. The invention can encode and transmit various operation data in a video mode, realize multi-channel picture switching and distribution through one-channel video signals, and store conversation contents.
Description
Technical Field
The invention relates to a recording method, in particular to an intraoperative talking hypermedia recording method.
Background
In recent years, the digital reconstruction and upgrade of the traditional operating rooms of hospitals step into the development motorway, and the industry competition is increasingly intense. The information-based breadth of the whole operation department is continuously increased, and new requirements are continuously provided aiming at the customization depth of actual medical user business, for example, in the prior planning of many newly-built digital operating rooms, the functional requirements of conversation of the family members in the operation are provided.
In the current common digital operating room scheme, a camera and a microphone are mostly installed on the wall surface of the operating room, and then a local audio and video is transmitted to a remote family conversation room through a network by an encoding device, and the conversation room realizes full-duplex network communication in the same way.
Although this method can realize contactless network conversation and improve efficiency, the following disadvantages still exist:
(1) The video source is a physical camera signal, the source is single, more operation related information cannot be transmitted, such as preoperative examination and inspection information, and if the operation related information is to be realized, additional equipment is required, and the cost is obviously increased.
(2) The communication mode of the two parties is limited to audio and video, no more interaction mode exists, and for medical care personnel, detailed operation conditions cannot be explained to non-professional patient families.
(3) The process of two parties of a conversation cannot be completely recorded into a single file, and because the network coding is carried out by independent equipment, communication cannot be carried out among network streams, so that at present, only one party of audio and video data file or the respective independent recorded file exists, the content is incomplete when the audio and video data file or the respective independent recorded file needs to be called at the later stage, the cooperativity is poor, and the practical significance is lost.
Disclosure of Invention
The purpose of the invention is: it provides a super media recording method for talking in operation, which not only can encode and transmit various operation data in video mode, but also can implement multi-channel picture switching and distribution by one-channel video signal and can store the talking content.
In order to achieve the purpose, the technical scheme of the invention is as follows: an intraoperative conversation hypermedia recording method, which comprises a PC (personal computer) arranged in an operating room, a collection card device connected with the PC and a conversation terminal arranged in a far-end conversation room, wherein the PC is connected with the conversation terminal through a network, the conversation terminal only collects a video signal source and an audio source, the collection card device is provided with a plurality of collection channels, and the method is characterized in that: a plurality of acquisition channels of the acquisition card equipment are respectively used for accessing a plurality of paths of medical imaging equipment and audio acquisition equipment in an operating room,
the specific recording steps of the conversation in the operation are as follows:
the method comprises the following steps: data acquisition, wherein the data acquisition comprises the following specific steps: physical audio and video sources output by the multi-path medical image equipment and the audio acquisition equipment are sent into a PC (personal computer) through acquisition card equipment, and preoperative examination and inspection information is stored in the PC in a static picture format;
step two: the data processing method comprises the following specific steps: performing audio-video separation logic processing on the multi-channel physical audio and video acquired in the step one in a PC, encapsulating and converting each channel of audio-video signals and respectively flowing into independent flow pipelines, wherein each flow pipeline receives a current real-time audio-video original data frame flow, each frame data is identified by two key attributes of a corresponding memory address and size,
format conversion is carried out on the static picture in the step one, the static picture is consistent with real-time video frame data in a stream pipeline in the internal memory format, reading control of different video data streams is realized by the PC according to identification information on each stream pipeline, and any path of audio and video signals can be taken at any time as a current picture displayed by the PC;
step three: data network transmission, the concrete steps of the data network transmission are: creating a video service engine based on an RTSP (real time streaming protocol) by the PC, receiving the current real-time output audio and video original data frame stream in any stream pipeline in the second step by the video service engine, compressing, coding and converting the audio and video original data frame stream into a network stream, and providing the network stream for a talking terminal to read;
step four: conversation exchange and recording; the conversation communication and recording method comprises the following specific steps: a talking terminal in the far-end talking room receives the network flow output by the PC network, simultaneously forms locally collected audio and video data into a data composite flow through the data processing mode of the second step and transmits the data composite flow to the PC end by using the network, at the moment, the PC and the talking terminal respectively separate the received network flow and restore the audio data and the video data by calling corresponding network flow decoding engines, and respectively output the audio data and the video data on a loudspeaker and a playing display of the PC in the operating room and a loudspeaker and a playing display of the talking terminal, and realize the talking communication between doctors and family members of the far-end talking room through information interaction,
the PC end in the operating room comprises two paths of local and remote audio data and video data, and the two paths of audio data and the two paths of video data are synthesized into one path of signal by the PC end and stored in the PC for subsequent reference and calling.
In the above technical solution, the specific steps of format conversion of the static picture in the second step are as follows: the PC in the operating room accesses a third-party Web service interface to obtain preoperative examination and inspection information, the preoperative examination and inspection information is presented on the PC in a static text data list or image format, the PC copies and stores the preoperative examination and inspection information in a bmp bitmap format, each bitmap takes a memory address and an address length corresponding to the bitmap as a unique identifier, and the bitmap format is consistent with the real-time video frame data in the stream pipeline in structure.
In the above technical solution, in the data processing in the second step, the PC creates a video frame annotation engine, creates a transparent overlay layer in the flow pipeline and places the overlay layer above the real-time video frame, draws a pattern identifier on the transparent overlay layer, renders the drawn pattern identifier into each video frame in real time through the video frame annotation engine, so that the video frame is overlaid with the original video signal into a whole, and finally the PC can select any path of video data signal with the drawn pattern identifier as the current frame displayed by the PC.
In the above technical solution, the drawn pattern identifier is one of a straight line identifier, a free line segment identifier, a circle identifier and a triangle identifier or a combination of several different identifier patterns.
In the above technical solution, the real-time audio/video frames contained in the video service engine in step three are compressed into an audio/video composite RTSP stream through H264 and AAC coding, and a stream pulling address in URL format is provided for a talking terminal in a far-end talking room to decode and play.
In the technical scheme, the two paths of audio data and video data are synthesized into one path of signal by the PC terminal in the fourth step and are stored in the PC in an mp4 video file format, at the moment, a multi-picture synthesis engine is created by the PC, the two paths of video data are subjected to multi-picture combination to form a new path of video source signal, so that the audio and video data at two ends of a talking scene are processed and converted into one path of video source and two paths of audio sources, a mixed video processing engine is created to mix the two paths of audio sources, and a compression coding module is called to write the audio and video data into a standard file to be stored in the PC.
The invention has the positive effects that: after the intraoperative conversation hypermedia recording method is adopted, because a plurality of acquisition channels of the acquisition card device are respectively used for accessing a plurality of paths of medical imaging devices and audio acquisition devices in an operating room, the specific recording steps of intraoperative conversation are as follows: data acquisition → data processing → data network transmission → conversation communication and recording, wherein the data acquisition comprises physical audio and video sources output by a plurality of paths of medical imaging equipment and audio acquisition equipment and preoperative examination and inspection information, the data processing is to flow the acquired data into independent flow pipelines respectively, the PC realizes reading control of different video data streams according to identification information on each flow pipeline, any path of audio and video signals can be called at any time as a current picture displayed by the PC, the data network transmission is to convert frame stream of audio and video original data into network stream by the PC, the network is provided for the conversation terminal to read, the PC and the conversation terminal realize mutual communication of contents, the PC end in the operation room comprises two paths of local and remote audio data and video data, and the PC synthesizes the two paths of audio data and video data into one path of signal and stores the signal in the PC for subsequent reference and calling;
therefore, the invention has the advantages that:
(1) The single video device video picture in the existing conversation scene in the operation is expanded into richer hypermedia resources, and the operation condition can be communicated with the family members of the patient in an all-round way;
(2) During the conversation process, the user can add annotation information to the real-time image source, and synchronize the annotation information to the video in real time and send the video to the opposite side as an auxiliary means for explanation;
(3) The multi-channel video source input is operated by adopting a similar 'director' mode, the terminal switches and controls different signal sources according to the requirements, and only one network video stream is output in the whole process, so that the operation privacy is ensured, the service complexity of software is reduced, the network bandwidth is saved, and the flexibility is improved;
(4) Through a unique audio and video processing framework, the encoding and decoding processes are more flexibly processed and controlled, and therefore the whole-process persistent recording of network audio and video data of both parties is achieved.
Drawings
FIG. 1 is a schematic flow diagram of one embodiment of the present invention.
Detailed Description
The invention is further illustrated, but not limited, by the following examples in connection with the accompanying drawings.
As shown in fig. 1, a hypermedia recording method for intraoperative conversation includes a PC installed in an operating room, an acquisition card device connected to the PC, and a conversation terminal installed in a far-end conversation room, wherein the PC is connected to a conversation terminal network, the conversation terminal only acquires a video signal source and an audio source, the acquisition card device has a plurality of acquisition channels, the plurality of acquisition channels of the acquisition card device are respectively used for accessing a plurality of medical image devices and audio acquisition devices in the operating room,
the specific recording steps of the conversation in the operation are as follows:
the method comprises the following steps: data acquisition, wherein the data acquisition comprises the following specific steps: physical audio and video sources output by the multi-path medical image equipment and the audio acquisition equipment are sent into a PC (personal computer) through acquisition card equipment, and preoperative examination and inspection information is stored in the PC in a static picture format;
step two: data processing, the specific steps of the data processing are as follows: performing audio-video separation logic processing on the multi-channel physical audio and video acquired in the step one in a PC, encapsulating and converting each channel of audio-video signals and respectively flowing into independent flow pipelines, wherein each flow pipeline receives a current real-time audio-video original data frame flow, each frame data is identified by two key attributes of a corresponding memory address and size,
the format of the static picture in the first step is converted, the static picture is kept consistent with the real-time video frame data in the stream pipeline in the internal memory format, the PC realizes the reading control of different video data streams according to the identification information on each stream pipeline, and any path of audio and video signals can be called at any time to be used as the current picture displayed by the PC;
the specific steps of format conversion of the static picture are as follows: the PC in the operating room accesses a third-party Web service interface to obtain preoperative examination and inspection information, the preoperative examination and inspection information is presented on the PC in a static text data list or image format, the PC copies and stores the preoperative examination and inspection information in a bmp bitmap format, each bitmap takes a memory address and an address length corresponding to the bitmap as a unique identifier, and the bitmap format is consistent with the real-time video frame data in a stream pipeline in structure;
the PC machine creates a video picture marking engine, creates a transparent superimposed layer in a flow pipeline and places the transparent superimposed layer above real-time video picture frames, draws pattern marks on the transparent superimposed layer, renders the drawn pattern marks into each video frame in real time through the video picture marking engine so as to enable the video frames and original video signals to be superimposed into a whole, and finally the PC machine can adjust any path to take the video data signals with the drawn pattern marks as the current picture displayed by the PC machine, wherein the drawn pattern marks are one or the combination of several different mark patterns of a straight line mark, a free line mark, a circular mark and a triangular mark;
step three: data network transmission, the concrete steps of the data network transmission are: the PC machine creates a video service engine based on RTSP streaming protocol, the video service engine receives the current real-time output audio and video original data frame stream in any stream pipeline in the second step, compresses, codes and converts the audio and video original data frame stream into network stream, and provides the network stream for a talking terminal to read,
real-time audio and video frames contained in the video service engine are compressed into audio and video composite RTSP (real time streaming protocol) streams through H264 and AAC (audio and video coding), and a stream pulling address in a URL (uniform resource locator) format is provided for a conversation terminal in a far-end conversation room to decode and play;
step four: conversation exchange and recording; the conversation communication and recording method comprises the following specific steps: a talking terminal in the far-end talking room receives the network flow output by the PC network, simultaneously forms locally collected audio and video data into a data composite flow through the data processing mode of the second step and transmits the data composite flow to the PC end by using the network, at the moment, the PC and the talking terminal respectively separate the received network flow and restore the audio data and the video data by calling corresponding network flow decoding engines, and respectively output the audio data and the video data on a loudspeaker and a playing display of the PC in the operating room and a loudspeaker and a playing display of the talking terminal, and realize the talking communication between doctors and family members of the far-end talking room through information interaction,
the PC end in the operating room comprises two paths of local and remote audio data and video data, and the two paths of audio data and the two paths of video data are synthesized into a path of signal by the PC end and stored in the PC for subsequent reference and calling;
the method comprises the following steps that a PC terminal in the fourth step synthesizes two paths of audio data and video data into a path of signal and stores the signal in an mp4 video file format in the PC, a multi-picture synthesis engine is created by the PC at the moment, the two paths of video data are subjected to multi-picture combination to form a new path of video source signal, the audio and video data at two ends of a talking scene are converted into a path of video source and two paths of audio sources through processing, a mixed video processing engine is created to mix the two paths of audio sources, and a compression coding module is called to write the audio and video data into a standard file to be stored in the PC.
Therefore, the specific recording steps of the intraoperative conversation of the invention are as follows: data collection → data processing → data network transmission → conversation communication, recording, wherein, the data collection includes physical audio and video sources output by multi-channel medical imaging equipment and audio collection equipment, and preoperative examination and inspection information, the data processing is to flow the collected data into independent flow pipes respectively, the PC realizes the reading control of different video data streams according to the identification information on each flow pipe, can call any one path of audio and video signals as the current picture displayed by the PC at any time, the data network transmission is to create a video service engine based on RTSP flow protocol by the PC, the PC converts the frame stream of the audio and video original data into network stream by compression coding, and provides the network stream for the terminal machine to read, realizes the mutual communication of conversation contents between the doctor in the far-end conversation in the operating room, and the PC in the operating room contains two paths of audio data and video data at local and far-end, and the PC end synthesizes the two paths of audio data and video data into one path of signal and stores the signal in the PC for the follow-up consultation and call,
therefore, the invention has the advantages that:
(1) The single video device video picture in the existing conversation scene in the operation is expanded into richer hypermedia resources, and the operation condition can be communicated with the family members of the patient in an all-round way;
(2) During the conversation process, the user can add annotation information to the real-time image source, and synchronize the annotation information to the video in real time and send the video to the opposite side as an auxiliary means for explanation;
(3) The multi-channel video source input is operated by adopting a similar 'director' mode, the terminal switches and controls different signal sources according to the requirements, and only one network video stream is output in the whole process, so that the operation privacy is ensured, the service complexity of software is reduced, the network bandwidth is saved, and the flexibility is improved;
(4) Through a unique audio and video processing framework, the encoding and decoding processes are more flexibly processed and controlled, and therefore the whole-process persistent recording of network audio and video data of both parties is achieved.
In light of the foregoing description of the preferred embodiment of the present invention, many modifications and variations will be apparent to those skilled in the art without departing from the spirit and scope of the invention. The technical scope of the present invention is not limited to the content of the specification, and must be determined according to the scope of the claims.
Claims (6)
1. An intraoperative conversation hypermedia recording method, which comprises a PC (personal computer) arranged in an operating room, a collection card device connected with the PC and a conversation terminal arranged in a far-end conversation room, wherein the PC is connected with a conversation terminal network, the conversation terminal only collects a video signal source and an audio source, the collection card device is provided with a plurality of collection channels, and the method is characterized in that: a plurality of acquisition channels of the acquisition card equipment are respectively used for accessing a plurality of paths of medical imaging equipment and audio acquisition equipment in an operating room,
the specific recording steps of the conversation in the operation are as follows:
the method comprises the following steps: data acquisition, wherein the data acquisition comprises the following specific steps: physical audio and video sources output by a plurality of paths of medical image equipment and audio acquisition equipment are sent into a PC (personal computer) through acquisition card equipment, and preoperative examination and inspection information is stored in the PC in a static picture format;
step two: data processing, the specific steps of the data processing are as follows: performing audio and video separation logic processing on the multi-channel physical audio and video acquired in the step one in a PC, packaging and converting each channel of audio and video signals and respectively flowing into independent flow pipelines, wherein a current real-time audio and video original data frame stream is received in each flow pipeline, each frame of data is identified by two key attributes of a corresponding memory address and size,
format conversion is carried out on the static picture in the step one, the static picture is consistent with real-time video frame data in a stream pipeline in the internal memory format, reading control of different video data streams is realized by the PC according to identification information on each stream pipeline, and any path of audio and video signals can be taken at any time as a current picture displayed by the PC;
step three: data network transmission, the concrete steps of the data network transmission are: creating a video service engine based on an RTSP (real time streaming protocol) by the PC, receiving the current real-time output audio and video original data frame stream in any stream pipeline in the second step by the video service engine, compressing, coding and converting the audio and video original data frame stream into a network stream, and providing the network stream for a talking terminal to read;
step four: conversation exchange and recording; the conversation communication and recording method comprises the following specific steps: a talking terminal in the far-end talking room receives the network flow output by the PC network, simultaneously forms locally collected audio and video data into a data composite flow through the data processing mode of the second step and transmits the data composite flow to the PC end by using the network, at the moment, the PC and the talking terminal respectively separate the received network flow and restore the audio data and the video data by calling corresponding network flow decoding engines, and respectively output the audio data and the video data on a loudspeaker and a playing display of the PC in the operating room and a loudspeaker and a playing display of the talking terminal, and realize the talking communication between doctors and family members of the far-end talking room through information interaction,
the PC end in the operating room comprises local and remote audio data and video data, and the PC end synthesizes the two audio data and the video data into a signal and stores the signal in the PC for subsequent reference and calling.
2. The method of claim 1, wherein the method further comprises: the specific steps of format conversion of the static picture in the second step are as follows: the PC in the operating room accesses a third-party Web service interface to obtain preoperative examination and inspection information, the preoperative examination and inspection information is presented on the PC in a static text data list or image format, the PC copies and stores the preoperative examination and inspection information in a bmp bitmap format, each bitmap takes a memory address and an address length corresponding to the bitmap as a unique identifier, and the bitmap format is consistent with the real-time video frame data in the stream pipeline in structure.
3. The intraoperative conversational hypermedia recording method of claim 1, wherein: in the data processing of the second step, the PC creates a video frame marking engine, creates a transparent superposition layer in the flow pipeline and places the transparent superposition layer above the real-time video frame, draws a pattern mark on the transparent superposition layer, renders the drawn pattern mark into each video frame in real time through the video frame marking engine so as to enable the video frame and the original video signal to be superposed into a whole, and finally the PC can adjust any path to take the video data signal with the drawn pattern mark as the current picture displayed by the PC.
4. The method of claim 3, wherein the step of recording the hypermedia of the intraoperative conversation comprises the steps of: the drawn pattern mark is one or the combination of several different mark patterns of a straight line mark, a free line segment mark, a circular mark and a triangular mark.
5. The intraoperative conversational hypermedia recording method of claim 1, wherein: and compressing real-time audio and video frames contained in the video service engine in the third step into audio and video composite RTSP (real time streaming protocol) streams through H264 and AAC (audio and video coding), and providing a stream pulling address in a URL (uniform resource locator) format for decoding and playing by a talking terminal in a far-end talking room.
6. The method of claim 1, wherein the method further comprises: the method comprises the following steps that a PC terminal in the fourth step synthesizes two paths of audio data and video data into a path of signal and stores the signal in an mp4 video file format in the PC, a multi-picture synthesis engine is created by the PC at the moment, the two paths of video data are subjected to multi-picture combination to form a new path of video source signal, the audio and video data at two ends of a talking scene are converted into a path of video source and two paths of audio sources through processing, a mixed video processing engine is created to mix the two paths of audio sources, and a compression coding module is called to write the audio and video data into a standard file to be stored in the PC.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211245131.7A CN115665102A (en) | 2022-10-12 | 2022-10-12 | Intraoperative talking hypermedia recording method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211245131.7A CN115665102A (en) | 2022-10-12 | 2022-10-12 | Intraoperative talking hypermedia recording method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115665102A true CN115665102A (en) | 2023-01-31 |
Family
ID=84987103
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211245131.7A Pending CN115665102A (en) | 2022-10-12 | 2022-10-12 | Intraoperative talking hypermedia recording method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115665102A (en) |
-
2022
- 2022-10-12 CN CN202211245131.7A patent/CN115665102A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| RU2662731C2 (en) | Server node arrangement and method | |
| CN100433826C (en) | Method and system for recording videoconference data | |
| US5390239A (en) | Method for increasing digital data throughput over telephone lines | |
| US7865833B2 (en) | Data processing apparatus, data processing server, data processing system, method of controlling data processing apparatus, method of controlling data processing server, computer program, and computer readable storage medium | |
| CN107637040A (en) | Carry out transmission media data using WEBSOCKET sub-protocols | |
| CN104601863B (en) | A kind of IP matrix systems for recorded broadcast | |
| CN106227492B (en) | Combination and mobile intelligent terminal interconnected method and device | |
| CN102638672A (en) | Automatic video layouts for multi-stream multi-site telepresence conferencing system | |
| US20070274344A1 (en) | Method and system for replacing media stream in a communication process of a terminal | |
| CN103841360A (en) | Distributed video conference achieving method and system, video conference terminal and audio and video integrated device | |
| CN105959613A (en) | Digital conference equipment and system | |
| TW201119391A (en) | Video transmission method, system and device | |
| JP2009044510A (en) | Mobile unit, server, and video reproduction method in mobile unit | |
| CN102104762A (en) | Media recording method, equipment and system of IMS (Internet Management Specification) video conference | |
| JP2016508679A (en) | System, apparatus, and method for sharing a screen having multiple visual components | |
| CN100401736C (en) | Apparatus for identifying a photographer of an image | |
| CN113301295A (en) | Multi-picture video data processing method, device, computer equipment and storage medium | |
| CN102770827A (en) | Method for displaying multimedia content on a screen of a terminal | |
| JP2000148886A (en) | Method and device for processing medical data and medical data processing system | |
| CN108616711A (en) | A kind of exchange method solving information network isolation based on vision signal | |
| CN115665102A (en) | Intraoperative talking hypermedia recording method | |
| CN100448292C (en) | Method and equipment for managing interactions in the MPEG-4 standard | |
| JPS62200994A (en) | Animation communication system | |
| CN115766348A (en) | Multi-protocol video fusion gateway based on Internet of things | |
| CN203070300U (en) | Interactive long-distance ward interrogation system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |