CN114300118A - Auxiliary diagnosis method and device for online remote consultation and electronic equipment - Google Patents
Auxiliary diagnosis method and device for online remote consultation and electronic equipment Download PDFInfo
- Publication number
- CN114300118A CN114300118A CN202111483567.5A CN202111483567A CN114300118A CN 114300118 A CN114300118 A CN 114300118A CN 202111483567 A CN202111483567 A CN 202111483567A CN 114300118 A CN114300118 A CN 114300118A
- Authority
- CN
- China
- Prior art keywords
- auxiliary diagnosis
- ultrasonic image
- cloud node
- diagnosis result
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Ultra Sonic Daignosis Equipment (AREA)
Abstract
The invention relates to the technical field of computers, in particular to an auxiliary diagnosis method, a device and electronic equipment for online remote consultation, which comprises the following steps: the cloud node runs a virtual server and creates a virtual frame cache region in the virtual server; the cloud node runs a web browser in a non-interface browsing mode, the web browser acquires an ultrasonic image through a target website link, and the ultrasonic image is cached in the virtual frame cache region through an X11 protocol; the cloud node feeds the ultrasonic image into an ultrasonic image detection model to obtain an auxiliary diagnosis result; and the cloud node combines the auxiliary diagnosis result with the ultrasonic image to obtain a contour characteristic image of the focus, and the contour characteristic image is used for remote consultation of all parties participating in a consulting room. The invention realizes real-time online consultation by utilizing the cloud nodes, has low delay, flexible deployment scheme and low cost, can perform real-time online feedback on different ultrasonic images, and is favorable for remote consultation of all parties participating in a consulting room.
Description
Technical Field
The invention relates to the technical field of computers, in particular to an auxiliary diagnosis method and device for online remote consultation and electronic equipment.
Background
In the prior art, the auxiliary consultation is mostly offline diagnosis. For example, the ultrasound image is retrieved from the database and then the auxiliary diagnosis is performed, so that the real-time online consultation can be performed, meanwhile, the deployment scheme adopted in the prior art for performing the remote consultation is not flexible enough and the deployment cost is high, the prior art needs to be added with intelligent hardware equipment on site, acquire the real-time ultrasound image directly from the medical equipment, and then output the intelligent diagnosis result to the on-site ultrasound image gateway equipment. If the auxiliary diagnosis and ultrasonic image gateway is integrated in one hardware device, the gateway hardware must be highly configured, and particularly in a dynamic image scene, the requirements on the computing capability of a CPU and a GPU and the capacities of a memory and a video memory are required.
The problem that how to connect with a remote consultation system on line becomes to be solved due to the fact that ultrasonic image acquisition and diagnosis result feedback exist in the prior art, if an ultrasonic image gateway is added to a cloud node, and image data streams on the line and under the line are connected, higher deployment cost, system complexity increase and transmission delay improvement are brought.
The existing remote consultation system lacks a customized open interface, is difficult to be in online butt joint with products of different intelligent equipment manufacturers, and the existing auxiliary diagnosis is mostly static images aiming at a certain ultrasonic image, such as a CT image, and cannot be used for auxiliary diagnosis of images in other forms.
Disclosure of Invention
The invention provides an auxiliary diagnosis method, an auxiliary diagnosis device and electronic equipment for online remote consultation, which realize real-time online consultation by using cloud nodes, can perform real-time online feedback on different ultrasonic images and are beneficial to remote consultation of all parties participating in a consulting room.
An embodiment of the present specification provides an auxiliary diagnosis method for online remote consultation, including:
the cloud node runs a virtual server and creates a virtual frame cache region in the virtual server;
the cloud node runs a web browser in a non-interface browsing mode, the web browser acquires an ultrasonic image through a target website link, and the ultrasonic image is cached in the virtual frame cache region through an X11 protocol;
the cloud node feeds the ultrasonic image into an ultrasonic image detection model to obtain an auxiliary diagnosis result;
and the cloud node combines the auxiliary diagnosis result with the ultrasonic image to obtain a contour characteristic image of the focus, and the contour characteristic image is used for remote consultation of all parties participating in a consulting room.
Preferably, before the cloud node runs the virtual server, the method includes:
medical diagnosis equipment carries out medical detection on a patient to obtain the ultrasonic image;
and the ultrasonic image gateway uploads the ultrasonic image to a real-time audio and video server for remote consultation, so that each party participating in a consulting room can observe the ultrasonic image in real time.
Preferably, the acquiring, by the web browser, the ultrasound image through the target website link includes:
the web browser initiates an ultrasonic imaging request to the real-time audio and video server through the target website link;
and the real-time audio and video server responds to the ultrasonic imaging request and returns the ultrasonic image.
Preferably, the auxiliary diagnosis result includes a position of the suspected lesion, a peripheral frame of the suspected lesion, a contour of the suspected lesion, lesion feature information, and a result of classifying the benign and malignant lesions.
Preferably, the cloud node combines the auxiliary diagnosis result with the ultrasound image to obtain a contour feature image of the lesion, including:
the cloud node creates a virtual camera based on the virtual video equipment, and adds the virtual camera into a corresponding consultation room;
the cloud node draws the auxiliary diagnosis result to a corresponding ultrasonic image to obtain a focus contour feature image;
and the cloud node transmits the outline characteristic image to browsers of other parties participating in the consultation room through the virtual camera.
Preferably, the cloud node combines the auxiliary diagnosis result with the ultrasound image to obtain a contour feature image of the lesion, and further includes:
the cloud node coordinates the auxiliary diagnosis result, and packages the coordinated auxiliary diagnosis result to obtain contour coordinate information;
the cloud node sends the contour coordinate information to an initiating end of a corresponding consultation room in a point-to-point mode based on a transmission control protocol, and the initiating end unpacks the contour coordinate information to restore the auxiliary diagnosis result;
calling a painting brush to draw the outline coordinate information to the ultrasonic image to obtain a painting brush track;
and the initiating end broadcasts the brush pen track to browsers of other parties participating in the consultation room through a medical data channel, and synchronously depicts the outline characteristics of the suspected focus.
Preferably, the cloud node combines the auxiliary diagnosis result with the ultrasound image to obtain a contour feature image of the lesion, and further includes:
the cloud node coordinates the auxiliary diagnosis result, and encapsulates the coordinated auxiliary diagnosis result according to a preset format to obtain an encapsulated auxiliary diagnosis result;
the cloud node sends the packaged auxiliary diagnosis result to the real-time audio and video server based on a transmission control protocol, and the real-time audio and video server unpacks the packaged auxiliary diagnosis result;
and the real-time audio and video server broadcasts the unsealed auxiliary diagnosis result to browsers of other parties participating in the consultation room through the medical data channel, and synchronously depicts the outline characteristics of the suspected focus.
An embodiment of the present specification further provides an auxiliary diagnostic apparatus for online remote consultation, including:
the system comprises a cache region creating module, a frame buffer region creating module and a frame buffer region creating module, wherein the cache region creating module is used for a cloud node to run a virtual server and create a virtual frame cache region in the virtual server;
the ultrasonic image acquisition module is used for operating a first webpage browser by the cloud node in a non-interface browsing mode, the webpage browser acquires an ultrasonic image through a target website link, and the ultrasonic image is cached in the virtual frame cache region through an X11 protocol;
the diagnosis module is used for feeding the ultrasonic image into an ultrasonic image detection model by the cloud node to obtain an auxiliary diagnosis result;
and the image feedback module is used for combining the auxiliary diagnosis result with the ultrasonic image by the cloud node to obtain a contour characteristic image of the focus, and is used for remote consultation of each party participating in a consulting room.
An electronic device, wherein the electronic device comprises:
a processor and a memory storing computer executable instructions that, when executed, cause the processor to perform the method of any of the above.
A computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the method of any of the above.
The invention realizes real-time online consultation by utilizing the cloud nodes, has low delay, flexible deployment scheme and low cost, can perform real-time online feedback on different ultrasonic images, and is favorable for remote consultation of all parties participating in a consulting room.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a schematic diagram illustrating an auxiliary diagnosis method for online remote consultation according to an embodiment of the present disclosure;
fig. 2 is a schematic structural diagram of an auxiliary diagnostic apparatus for online remote consultation according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure;
fig. 4 is a schematic diagram of a computer-readable medium provided in an embodiment of the present specification.
Detailed Description
Exemplary embodiments of the present invention will now be described more fully with reference to the accompanying drawings. The exemplary embodiments, however, may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art. The same reference numerals denote the same or similar elements, components, or parts in the drawings, and thus their repetitive description will be omitted.
Features, structures, characteristics or other details described in a particular embodiment do not preclude the fact that the features, structures, characteristics or other details may be combined in a suitable manner in one or more other embodiments in accordance with the technical idea of the invention.
In describing particular embodiments, the present invention has been described with reference to features, structures, characteristics or other details that are within the purview of one skilled in the art to provide a thorough understanding of the embodiments. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific features, structures, characteristics, or other details.
The diagrams depicted in the figures are exemplary only, and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order depicted. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The term "and/or" and/or "includes all combinations of any one or more of the associated listed items.
Referring to fig. 1, a schematic diagram of an auxiliary diagnosis method for online remote consultation according to an embodiment of the present disclosure includes:
s101: the cloud node runs a virtual server and creates a virtual frame cache region in the virtual server;
in the preferred embodiment of the present invention, the ultrasound image generated by the ultrasound imaging device on line is extracted by using the Virtual display technology, and simultaneously, the cloud node runs a Virtual X11-Server, and a Virtual Frame Buffer (Virtual Frame Buffer) is created in the memory for buffering the ultrasound image.
S102: the cloud node runs a web browser in a non-interface browsing mode, the web browser acquires an ultrasonic image through a target website link, and the ultrasonic image is cached in the virtual frame cache region through an X11 protocol;
in the preferred embodiment of the present invention, the cloud node runs an example of a web browser invisible to the user in a headset mode, which may be any one of applications such as Chrome, Firefox, and Electron. The browser serves as an X11-Client, requests an ultrasonic image from a WebRTC (Web Real-Time Communication ) server through a specified website link, and caches the requested ultrasonic image in a Virtual Frame Buffer through an X11 protocol. The auxiliary diagnostic equipment reads the ultrasonic image cached in the Virtual Frame Buffer, and the video stream content related to the ultrasonic image in the WebRTC server can be obtained. By the aid of the method, the problem of how to acquire the online ultrasonic image is solved, and online auxiliary diagnosis is realized.
S103: the cloud node feeds the ultrasonic image into an ultrasonic image detection model to obtain an auxiliary diagnosis result;
in the preferred embodiment of the present invention, the auxiliary diagnostic function can be run using an existing server/workstation, or deploying a dedicated intelligent diagnostic machine. Machine hardware generally requires the configuration of graphics cards capable of graphics processing, and the GPU acceleration is performed for high throughput operations, and the operating system may be Windows or Linux. And then feeding the real-time ultrasonic image into the trained ultrasonic image detection model to detect the position and the shape of the suspected lesion. The ultrasonic image detection model can be a YoLO (young Only Look one) algorithm model and/or a U-Net algorithm model, the YoLO algorithm model can detect the position and peripheral frames of a suspected focus in the ultrasonic image, the U-Net algorithm model can detect the outline of the suspected focus, and in addition, characteristic information such as aspect ratio and uniformity can be extracted to classify the suspected focus. The auxiliary diagnosis result is obtained in the above way and is used for remote consultation of all parties participating in the subsequent consulting room.
S104: and the cloud node combines the auxiliary diagnosis result with the ultrasonic image to obtain a contour characteristic image of the focus, and the contour characteristic image is used for remote consultation of all parties participating in a consulting room.
In a preferred embodiment of the present invention, the cloud node has three feedback modes for the profile feature of the suspected lesion, the first mode is to draw the profile feature of the suspected lesion to the original ultrasound image and output the profile feature through the virtual camera; secondly, encapsulating the outline coordinate of the suspected focus, sending the outline coordinate information to an initiating end of a corresponding consultation room in a point-to-point mode based on a transmission control protocol, and broadcasting the brush pen track to browsers of other participating parties through a medical data channel by the initiating end; and the third step is that the outline coordinate of the suspected lesion is packaged into a preset format, the coordinated information is sent to the WebRTC server based on a transmission control protocol, and after the packaged outline coordinate of the suspected lesion is received by the WebRTC server, the packaged outline coordinate of the suspected lesion is unpacked and the coordinate information is broadcasted to the browsers of other parties through the medical data channel. Neither of these three approaches requires server-side transcoding, so a WebRTC server of the videoconference server type is equally applicable. By the method, the outline characteristics of the suspected focus can be fed back to all parties participating in the consulting room, and remote consultation of all parties participating in the consulting room is facilitated.
Further, before the cloud node runs a virtual server, the method includes:
medical diagnosis equipment carries out medical detection on a patient to obtain the ultrasonic image;
and the ultrasonic image gateway uploads the ultrasonic image to a real-time audio and video server for remote consultation, so that each party participating in a consulting room can observe the ultrasonic image in real time.
In the preferred embodiment of the invention, an ultrasonic image gateway device is deployed in an ultrasonic image acquisition field to convert an ultrasonic image from off-line to on-line, and the ultrasonic image gateway can also be connected with an audio and video conference device so as to facilitate the communication and interaction between the field and a remote expert or student. Some medical devices and ultrasound image gateways require additional conversion devices. For example, a microscope needs to be installed on a site, a camera of the microscope is required to convert an imaged pathological section from an optical interface to a data interface, and different types of terminal equipment are used at a far end to participate in remote consultation, wherein the terminal equipment can be a desktop computer, a laptop computer, a tablet computer, a Mobile computer and the like.
Further, the web browser obtains the ultrasound image through the target website link, and includes:
the web browser initiates an ultrasonic imaging request to the real-time audio and video server through the target website link;
and the real-time audio and video server responds to the ultrasonic imaging request and returns the ultrasonic image.
In a preferred embodiment of the invention, the web browser initiates an ultrasonic imaging request to the real-time audio/video server through the website link of the audio/video server, when the real-time audio/video server receives the ultrasonic imaging request, the medical equipment detects the patient according to the ultrasonic imaging request and performs ultrasonic imaging processing to obtain an ultrasonic image, and the obtained ultrasonic image is returned to the web browser.
Further, the auxiliary diagnosis result includes the position of the suspected lesion, the peripheral frame of the suspected lesion, the outline of the suspected lesion, feature information of the lesion, and a result of classifying the benign and malignant lesions.
Further, the cloud node combines the auxiliary diagnosis result with the ultrasonic image to obtain a contour feature image of the lesion, including:
the cloud node creates a virtual camera based on the virtual video equipment, and adds the virtual camera into a corresponding consultation room;
the cloud node draws the auxiliary diagnosis result to a corresponding ultrasonic image to obtain a focus contour feature image;
and the cloud node transmits the outline characteristic image to browsers of other parties participating in the consultation room through the virtual camera.
In a preferred embodiment of the invention, the cloud node creates a virtual camera based on the virtual video equipment as a local video source to be added into a corresponding consulting room, and then draws the contour characteristics of the suspected focus to the acquired original ultrasonic image as an input source of the virtual camera, and all parties participating in the consulting room can simultaneously observe and compare two paths of ultrasonic images through respective browsers, wherein one path is the original ultrasonic image, and the other path is the contour characteristic image of the focus. The remote online consultation is realized through the method.
Further, the cloud node combines the auxiliary diagnosis result with the ultrasonic image to obtain a contour feature image of the lesion, and the method further includes:
the cloud node coordinates the auxiliary diagnosis result, and packages the coordinated auxiliary diagnosis result to obtain contour coordinate information;
the cloud node sends the contour coordinate information to an initiating end of a corresponding consultation room in a point-to-point mode based on a transmission control protocol, and the initiating end unpacks the contour coordinate information to restore the auxiliary diagnosis result;
calling a painting brush to draw the outline coordinate information to the ultrasonic image to obtain a painting brush track;
and the initiating end broadcasts the brush pen track to browsers of other parties participating in the consultation room through a medical data channel, and synchronously depicts the outline characteristics of the suspected focus.
In a preferred embodiment of the present invention, the cloud node coordinates the contour features of the suspected lesion, encapsulates the coordinate contour features of the suspected lesion, and sends the encapsulated auxiliary diagnosis result to the intelligent device initiating end of the corresponding consulting room through Socket communication based on the transmission control protocol. After receiving the packaged outline characteristics of the coordinated suspected focus, the initiating end of the intelligent equipment firstly unpacks the outline characteristics of the suspected focus, then configures an original ultrasonic image window into a Canvas mode, calls a painting brush function to paint the focus outline on a corresponding medical image in the original ultrasonic image, finally broadcasts a painting brush track generated in the drawing process to browsers of parties participating in a corresponding consulting room through a medical data channel, and synchronously paints the outline characteristics of the suspected focus. The remote online consultation is realized through the method. The point-to-point mode means that data transmission no longer passes through a server, but data are directly transmitted between network users.
Further, the cloud node combines the auxiliary diagnosis result with the ultrasonic image to obtain a contour feature image of the lesion, and the method further includes:
the cloud node coordinates the auxiliary diagnosis result, and encapsulates the coordinated auxiliary diagnosis result according to a preset format to obtain an encapsulated auxiliary diagnosis result;
the cloud node sends the packaged auxiliary diagnosis result to the real-time audio and video server based on a transmission control protocol, and the real-time audio and video server unpacks the packaged auxiliary diagnosis result;
and the real-time audio and video server broadcasts the unsealed auxiliary diagnosis result to browsers of other parties participating in the consultation room through the medical data channel, and synchronously depicts the outline characteristics of the suspected focus.
In a preferred embodiment of the invention, the cloud node coordinates the outline characteristics of the suspected focus, encapsulates the outline characteristics of the coordinated suspected focus in a preset format, takes a brush pen track format as an example, encapsulates the outline characteristics of the coordinated suspected focus into a brush pen track format, then sends the outline characteristics of the suspected focus encapsulated into the brush pen track format to a real-time audio and video server based on a transmission control protocol, finally decapsulates the outline characteristics of the suspected focus by the real-time audio and video server, broadcasts a brush pen track corresponding to the decapsulated outline characteristics of the suspected focus to browsers of other parties participating in a consultation room through a medical data channel, synchronously depicts the outline characteristics of the suspected focus, and realizes remote online consultation by the above method. The preset formats include a json format, a brush pen track format and the like.
In the preferred embodiment of the invention, the cloud node only requires network bandwidth and hardware operation resources, does not require a human-computer interface (a display screen and a keyboard and a mouse), does not have specific requirements on deployment positions, and can be used as a plug-in for online work by auxiliary diagnosis as long as certain interface specifications are met. Aiming at different types of ultrasonic images, the number of cloud nodes in the system scheme can be more than one, and the number of intelligent plug-ins under the same cloud node can also be more than one. These all provide feasibility and flexibility for intelligent cloud deployment scenarios.
In the aspect of performance measurement, for example, thyroid ultrasound is taken as an example, the resolution of an ultrasound image opened by a Headless browser is 1024x768, and the output frame rate of a virtual camera is configured to be 24 frames per second. The auxiliary diagnosis calls an ultrasonic image detection model of the thyroid ultrasound, such as a YOLO algorithm model or a U-Net algorithm model. Under the scene of no focus detection, the virtual frame buffer acquisition and the virtual camera transmission are considered, and the cloud node time consumption is not more than 10 ms. Under the scene that the suspected focus is detected, the cloud node time consumption is still less than 200ms in consideration of contour drawing. Therefore, the online auxiliary diagnosis has the advantages of cloud deployment, plug-in opening, low delay and full-image intelligent processing. The remote consultation is used as an open platform for accepting a third-party plug-in, online interface specifications are customized, auxiliary diagnosis products provided by different manufacturers can be embedded into an overall solution of a remote consultation system manufacturer in the form of the plug-in, and full-image intelligent processing comprises ultrasonic dynamic images, microscope pathological sections, X-ray digital films and the like.
In a preferred embodiment of the invention, an offline auxiliary diagnosis is initiated: a manufacturer newly develops or updates an auxiliary diagnosis product, one workstation of an office of the manufacturer is used as a cloud node through offline negotiation, auxiliary diagnosis is used as a temporary plug-in for online actual measurement, and a diagnosis result is fed back. Initiating auxiliary diagnosis on line: an auxiliary diagnosis product of a certain manufacturer accords with a customized open interface specification through actual measurement of functions and performances, formally becomes a functional plug-in, an initiator in a consulting room configures page options, and sends an instruction to a cloud node through Socket communication based on a transmission control protocol, and the cloud node monitors the instruction, carries out online operation auxiliary diagnosis and feeds back a diagnosis result. By the aid of the method, the auxiliary diagnosis can be initiated on line and can also be initiated under line.
In the preferred embodiment of the invention, the cloud node (server/workstation) of any WebRTC server can initiate online auxiliary diagnosis, no additional hardware equipment is needed to be added offline, and the deployment scheme is flexible and low in cost.
In a preferred embodiment of the present invention, step one, the medical equipment (ultrasound machine, microscope, X-ray machine, etc.) starts to work, and outputs an ultrasound image; secondly, the ultrasonic image gateway uploads the ultrasonic image to a WebRTC server of the remote consultation system, and all parties participating in a corresponding consulting room can see the real-time ultrasonic image in the remote consultation; thirdly, the cloud node (server/workstation) collects the online ultrasonic image and feeds the online ultrasonic image into an ultrasonic image detection model for assisting diagnosis; fourthly, the cloud node operates an ultrasonic image detection model to detect the outline and other characteristic information of the suspected focus; and step five, the cloud node feeds the auxiliary diagnosis result back to the remote consultation system, and all parties participating in the remote consultation corresponding to the consulting room can synchronously observe the real-time ultrasonic image and the AI auxiliary diagnosis result.
Fig. 2 is a schematic structural diagram of an auxiliary diagnostic apparatus for online remote consultation according to an embodiment of the present disclosure, including:
a cache region creating module 201, configured to run a virtual server by a cloud node, and create a virtual frame cache region in the virtual server;
an ultrasonic image obtaining module 202, configured to run a first web browser by the cloud node in a non-interface browsing manner, where the web browser obtains an ultrasonic image through a target website link, and caches the ultrasonic image in the virtual frame cache region through an X11 protocol;
the diagnosis module 203 is used for the cloud node to feed the ultrasonic image into an ultrasonic image detection model to obtain an auxiliary diagnosis result;
and the image feedback module 204 is used for combining the auxiliary diagnosis result with the ultrasonic image by the cloud node to obtain a contour feature image of the focus, and is used for remote consultation of each party participating in a consulting room.
Further, the ultrasound image acquisition module 202 includes:
the request initiating unit is used for initiating an ultrasonic imaging request to a real-time audio and video server by the webpage browser through the target website link;
and the response unit is used for responding the ultrasonic imaging request by the real-time audio and video server and returning the ultrasonic image.
The functions of the apparatus in the embodiment of the present invention have been described in the above method embodiments, so that reference may be made to the related descriptions in the foregoing embodiments for details that are not described in the present embodiment, and further details are not described herein.
Based on the same inventive concept, the embodiment of the specification further provides the electronic equipment.
In the following, embodiments of the electronic device of the present invention are described, which may be regarded as specific physical implementations for the above-described embodiments of the method and apparatus of the present invention. Details described in the embodiments of the electronic device of the invention should be considered supplementary to the embodiments of the method or apparatus described above; for details which are not disclosed in embodiments of the electronic device of the invention, reference may be made to the above-described embodiments of the method or the apparatus.
Fig. 3 is a schematic structural diagram of an electronic device provided in an embodiment of the present specification. An electronic device 300 according to this embodiment of the invention is described below with reference to fig. 3. The electronic device 300 shown in fig. 3 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 3, electronic device 300 is embodied in the form of a general purpose computing device. The components of electronic device 300 may include, but are not limited to: at least one processing unit 310, at least one memory unit 320, a bus 330 connecting different device components (including the memory unit 320 and the processing unit 310), a display unit 340, and the like.
Wherein the storage unit stores program code executable by the processing unit 310 to cause the processing unit 310 to perform the steps according to various exemplary embodiments of the present invention described in the above-mentioned processing method section of the present specification. For example, the processing unit 310 may perform the steps as shown in fig. 1.
The storage unit 320 may include readable media in the form of volatile storage units, such as a random access memory unit (RAM)3201 and/or a cache storage unit 3202, and may further include a read only memory unit (ROM) 3203.
The storage unit 320 may also include a program/utility 3204 having a set (at least one) of program modules 3205, such program modules 3205 including, but not limited to: an operating device, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 300 may also communicate with one or more external devices 400 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 300, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 300 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 350. Also, the electronic device 300 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 360. Network adapter 360 may communicate with other modules of electronic device 300 via bus 330. It should be appreciated that although not shown in FIG. 3, other hardware and/or software modules may be used in conjunction with electronic device 300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID devices, tape drives, and data backup storage devices, to name a few.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments of the present invention described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a computing device (which can be a personal computer, a server, or a network device, etc.) execute the above-mentioned method according to the present invention. The computer program, when executed by a data processing apparatus, enables the computer readable medium to implement the above-described method of the invention, namely: such as the method shown in fig. 1.
Fig. 4 is a schematic diagram of a computer-readable medium provided in an embodiment of the present disclosure.
A computer program implementing the method shown in fig. 1 may be stored on one or more computer readable media. The computer readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution apparatus, device, or apparatus. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
In summary, the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functionality of some or all of the components in embodiments in accordance with the invention may be implemented in practice using a general purpose data processing device such as a microprocessor or a Digital Signal Processor (DSP). The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
While the foregoing embodiments have described the objects, aspects and advantages of the present invention in further detail, it should be understood that the present invention is not inherently related to any particular computer, virtual machine or electronic device, and various general-purpose machines may be used to implement the present invention. The invention is not to be considered as limited to the specific embodiments thereof, but is to be understood as being modified in all respects, all changes and equivalents that come within the spirit and scope of the invention.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. An auxiliary diagnosis method for online remote consultation, which is characterized by comprising the following steps:
the cloud node runs a virtual server and creates a virtual frame cache region in the virtual server;
the cloud node runs a web browser in a non-interface browsing mode, the web browser acquires an ultrasonic image through a target website link, and the ultrasonic image is cached in the virtual frame cache region through an X11 protocol;
the cloud node feeds the ultrasonic image into an ultrasonic image detection model to obtain an auxiliary diagnosis result;
and the cloud node combines the auxiliary diagnosis result with the ultrasonic image to obtain a contour characteristic image of the focus, and the contour characteristic image is used for remote consultation of all parties participating in a consulting room.
2. The auxiliary diagnostic method for online remote consultation according to claim 1, wherein before the cloud node runs the virtual server, it includes:
medical diagnosis equipment carries out medical detection on a patient to obtain the ultrasonic image;
and the ultrasonic image gateway uploads the ultrasonic image to a real-time audio and video server for remote consultation, so that each party participating in a consulting room can observe the ultrasonic image in real time.
3. The auxiliary diagnostic method for on-line remote consultation according to claim 2, wherein the web browser obtains the ultrasonic image through the link of the target website, and comprises:
the web browser initiates an ultrasonic imaging request to the real-time audio and video server through the target website link;
and the real-time audio and video server responds to the ultrasonic imaging request and returns the ultrasonic image.
4. The method of claim 1, wherein the auxiliary diagnosis result comprises a location of the suspected lesion, a peripheral frame of the suspected lesion, a contour of the suspected lesion, lesion feature information, and a classification result of malignancy and malignancy of the lesion.
5. The on-line remote consultation auxiliary diagnosis method according to claim 4, wherein the cloud node combines the auxiliary diagnosis result with the ultrasonic image to obtain a lesion contour feature image, and the method comprises the following steps:
the cloud node creates a virtual camera based on the virtual video equipment, and adds the virtual camera into a corresponding consultation room;
the cloud node draws the auxiliary diagnosis result to a corresponding ultrasonic image to obtain a focus contour feature image;
and the cloud node transmits the outline characteristic image to browsers of other parties participating in the consultation room through the virtual camera.
6. The on-line remote consultation auxiliary diagnosis method according to claim 5, wherein the cloud node combines the auxiliary diagnosis result with the ultrasound image to obtain a lesion contour feature image, further comprising:
the cloud node coordinates the auxiliary diagnosis result, and packages the coordinated auxiliary diagnosis result to obtain contour coordinate information;
the cloud node sends the contour coordinate information to an initiating end of a corresponding consultation room in a point-to-point mode based on a transmission control protocol, and the initiating end unpacks the contour coordinate information to restore the auxiliary diagnosis result;
calling a painting brush to draw the outline coordinate information to the ultrasonic image to obtain a painting brush track;
and the initiating end broadcasts the brush pen track to browsers of other parties participating in the consultation room through a medical data channel, and synchronously depicts the outline characteristics of the suspected focus.
7. The on-line remote consultation auxiliary diagnosis method according to claim 6, wherein the cloud node combines the auxiliary diagnosis result with the ultrasound image to obtain a lesion contour feature image, further comprising:
the cloud node coordinates the auxiliary diagnosis result, and encapsulates the coordinated auxiliary diagnosis result according to a preset format to obtain an encapsulated auxiliary diagnosis result;
the cloud node sends the packaged auxiliary diagnosis result to the real-time audio and video server based on a transmission control protocol, and the real-time audio and video server unpacks the packaged auxiliary diagnosis result;
and the real-time audio and video server broadcasts the unsealed auxiliary diagnosis result to browsers of other parties participating in the consultation room through the medical data channel, and synchronously depicts the outline characteristics of the suspected focus.
8. An auxiliary diagnosis device for online remote consultation, comprising:
the system comprises a cache region creating module, a frame buffer region creating module and a frame buffer region creating module, wherein the cache region creating module is used for a cloud node to run a virtual server and create a virtual frame cache region in the virtual server;
the ultrasonic image acquisition module is used for operating a first webpage browser by the cloud node in a non-interface browsing mode, the webpage browser acquires an ultrasonic image through a target website link, and the ultrasonic image is cached in the virtual frame cache region through an X11 protocol;
the diagnosis module is used for feeding the ultrasonic image into an ultrasonic image detection model by the cloud node to obtain an auxiliary diagnosis result;
and the image feedback module is used for combining the auxiliary diagnosis result with the ultrasonic image by the cloud node to obtain a contour characteristic image of the focus, and is used for remote consultation of each party participating in a consulting room.
9. An electronic device, wherein the electronic device comprises:
a processor and a memory storing computer-executable instructions that, when executed, cause the processor to perform the method of any of claims 1-7.
10. A computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111483567.5A CN114300118A (en) | 2021-12-07 | 2021-12-07 | Auxiliary diagnosis method and device for online remote consultation and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111483567.5A CN114300118A (en) | 2021-12-07 | 2021-12-07 | Auxiliary diagnosis method and device for online remote consultation and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114300118A true CN114300118A (en) | 2022-04-08 |
Family
ID=80965071
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111483567.5A Pending CN114300118A (en) | 2021-12-07 | 2021-12-07 | Auxiliary diagnosis method and device for online remote consultation and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114300118A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116520815A (en) * | 2023-07-03 | 2023-08-01 | 合众新能源汽车股份有限公司 | Remote diagnosis method, device, computer equipment and storage medium |
-
2021
- 2021-12-07 CN CN202111483567.5A patent/CN114300118A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116520815A (en) * | 2023-07-03 | 2023-08-01 | 合众新能源汽车股份有限公司 | Remote diagnosis method, device, computer equipment and storage medium |
| CN116520815B (en) * | 2023-07-03 | 2023-10-17 | 合众新能源汽车股份有限公司 | Remote diagnosis method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11582284B2 (en) | Optimization of publication of an application to a web browser | |
| US9001229B2 (en) | Multiple sensor input data synthesis | |
| WO2019227905A1 (en) | Method and equipment for performing remote assistance on the basis of augmented reality | |
| US11089349B2 (en) | Apparatus and method for playing back and seeking media in web browser | |
| JP6882868B2 (en) | Image processing equipment, image processing method, system | |
| US20220075491A1 (en) | Remote support service with smart whiteboard | |
| KR101942269B1 (en) | Apparatus and method for playing back and seeking media in web browser | |
| CN103533034A (en) | Method for seamlessly using cloud terminal local camera in VDI (Virtual Device Interface) virtual desktop | |
| WO2019184533A1 (en) | Method and apparatus for remotely configuring control panel of intelligent device | |
| US20230091539A1 (en) | Information processing method, device, system, storage medium, and computer program product | |
| CN114300118A (en) | Auxiliary diagnosis method and device for online remote consultation and electronic equipment | |
| CN104254001A (en) | Remote sharing method, device and terminal | |
| JP2013542649A (en) | Virtual video capture device | |
| US11599974B2 (en) | Joint rolling shutter correction and image deblurring | |
| CN115550687A (en) | Three-dimensional model scene interaction method, system, equipment, device and storage medium | |
| CN114374853A (en) | Content display method and device, computer equipment and storage medium | |
| US20200322317A1 (en) | Systems and methods for establishing secure remote connections to media devices | |
| CN115695383A (en) | Screen sharing method and system, electronic device and android device | |
| CN114666658A (en) | Cloud rendering method, device and system and user terminal | |
| CN114281451A (en) | Method and device for self-adapting screens with different resolutions by application interface and electronic equipment | |
| Chujo et al. | Implementation and Demonstration of Real-time Point Cloud Streaming System using HoloLens | |
| JP2020173776A (en) | Method and device for generating video | |
| JP2016517644A (en) | Wireless docking system for audio-video | |
| CN115134659B (en) | Video editing and configuring method, device, browser, electronic equipment and storage medium | |
| CN113741837B (en) | Information processing method, device, system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |