CN113990457B

CN113990457B - Video image intercepting method, device, chip, surgical robot and system

Info

Publication number: CN113990457B
Application number: CN202111599908.5A
Authority: CN
Inventors: 王迎智; 董先公; 徐高峰
Original assignee: Jixian Artificial Intelligence Co Ltd
Current assignee: Jixian Artificial Intelligence Co Ltd
Priority date: 2021-12-24
Filing date: 2021-12-24
Publication date: 2022-03-04
Anticipated expiration: 2041-12-24
Also published as: WO2023116777A1; CN113990457A

Abstract

The embodiment of the application provides a video image capturing method, a video image capturing device, a chip, a surgical robot and a system, and belongs to the technical field of image processing. Detecting a zone bit of a first memory space and a zone bit of a second memory space in a memory; when the zone bit of the first memory space is a first preset value or the zone bit of the second memory space is a second preset value, sending a screenshot instruction to the second processor; when the second processor detects the screenshot operation of the hardware device, triggering a flag bit of a first memory space to be a first preset value; when the third processor receives a network screenshot request sent by the client, triggering the zone bit of the second memory space to be a second preset value; and setting the zone bit of a fourth memory space in the memory to be a third preset value. By using the video image capturing method provided by the application, the picture of the internal organ of the human body can be obtained in the process of observing the internal organ of the human body.

Description

Video image intercepting method, device, chip, surgical robot and system

Technical Field

The embodiment of the application relates to the technical field of image processing, in particular to a video image capturing method, a video image capturing device, a video image capturing chip, a surgical robot and a surgical system.

Background

The operation robot, such as endoscope, is a detecting instrument integrating traditional optics, ergonomics, precision machinery, modern electronics, mathematics and software into one body, and can enter into stomach or other natural ducts through mouth and body, and the operation robot can be used to see the pathological changes which can not be displayed by X-ray.

At present, a surgical robot only has a detection function, and a user (such as a doctor) can only observe internal parts of a human body, such as stomach, intestine and the like, through the surgical robot to further determine a diseased organ of the human body, but cannot acquire a picture of the diseased organ of the human body.

Disclosure of Invention

The embodiment of the application provides a video image capturing method, a video image capturing device, a video image capturing chip, a surgical robot and a video image capturing system, and aims to solve the problem that a user cannot acquire a picture of a lesion part when using the surgical robot.

A first aspect of an embodiment of the present application provides a video image capturing method, which is applied to a first processor, where the first processor, a second processor, and a third processor share a same memory, and the memory includes a plurality of memory spaces, and the method includes:

detecting a flag bit of a first memory space and a flag bit of a second memory space in the memory;

sending a screenshot instruction to the second processor under the condition that the zone bit of the first memory space is a first preset value or the zone bit of the second memory space is a second preset value, so that the second processor stores the intercepted image in a third memory space after receiving the screenshot instruction; when the second processor detects the screenshot operation of the hardware device, triggering the zone bit of the first memory space to be a first preset value; when the third processor receives a network screenshot request sent by a client, triggering the zone bit of the second memory space to be a second preset value;

and setting a flag bit of a fourth memory space in the memory to a third preset value to instruct the third processor to acquire the image from the third memory space and store the image.

Optionally, after setting a flag bit of a fourth memory space in the memory to a third preset value, the method includes:

and clearing the zone bit of the first memory space and the zone bit of the second memory space.

Optionally, the method further comprises:

after the third processor saves the image, detecting whether the flag bit of the first memory space and the flag bit of the second memory space are cleared;

clearing the zone bit of the first memory space under the condition that the zone bit of the first memory space is not cleared;

and clearing the zone bit of the second memory space under the condition that the zone bit of the second memory space is not cleared.

A second aspect of the present application provides a video image capturing method, which is applied to a second processor, where the second processor shares a same memory with a first processor and a third processor, and the memory includes a plurality of memory spaces, and the method includes:

receiving a screenshot instruction sent by the first processor, and carrying out image interception on the acquired video stream; the screenshot instruction is an instruction sent by the first processor when the flag bit of the first memory space is detected to be a first preset value, or is an instruction sent by the first processor when the flag bit of the second memory space is detected to be a second preset value;

storing the intercepted image in a third memory space, so that the third processor acquires the image from the third memory space and stores the image when detecting that the flag bit in a fourth memory space is a third preset value; after the first processor sends a screenshot instruction, the flag bit of the fourth memory space is triggered to be a third preset value.

Optionally, the flag bit of the first memory space is triggered by the following steps:

detecting whether a hardware device has screenshot operation;

and setting the zone bit of the first memory space to be a first preset value under the condition that the screenshot operation executed on the hardware equipment is detected.

Optionally, the screenshot operation is a screenshot function button; setting the flag bit of the first memory space to a first preset value under the condition that the screenshot operation executed on the hardware equipment is detected, wherein the method comprises the following steps:

when detecting that a screenshot function button on the hardware equipment is triggered, setting a flag bit of the first memory space to a first preset value;

the method further comprises the following steps:

and when detecting that the screenshot function button on the hardware equipment is reset, automatically clearing the zone bit of the first memory space.

A third aspect of the present embodiment provides a video image capturing method, which is applied to a third processor, where the third processor, a second processor, and a first processor share a same memory, and the memory includes multiple memory spaces, and the method includes:

detecting a flag bit of a fourth memory space;

when the flag bit of the fourth memory space is detected to be a third preset value, reading an image in a third memory space, and storing the image;

when detecting the screenshot operation of the hardware equipment, the second processor triggers the flag bit of the first memory space to be a first preset value; when the third processor receives a network screenshot request sent by a client, triggering the zone bit of the second memory space to be a second preset value; and when the first processor sends a screenshot instruction to the second processor under the condition that the flag bit of the first memory space is detected to be a first preset value or the flag bit of the second memory space is detected to be a second preset value, triggering the first processor to set the flag bit of the fourth memory space to be a third preset value.

Optionally, the method further comprises:

and clearing the zone bit of the fourth memory space when the zone bit of the fourth memory space is detected to be a third preset value.

Optionally, saving the image comprises:

and saving the image in a storage device or saving the image in a file system of the image.

Optionally, in a case where the image is saved in its own file system, the method further includes:

sending the storage path and the file name of the image to the client so that the client generates an image downloading request according to the storage path and the file name of the image;

and if an image downloading request sent by a client is received, sending the image to the client.

A fourth aspect of the present embodiment provides a video image capturing method, which is applied to a first processor, a second processor, and a third processor, where the first processor, the second processor, and the third processor share a same memory, and the memory includes a plurality of memory spaces, and the method includes:

when the second processor detects the screenshot operation of the hardware equipment, triggering the zone bit of the first memory space to be a first preset value; when the third processor receives a network screenshot request sent by a client, the flag bit of the second memory space is triggered to be a second preset value;

the first processor detects a zone bit of a first memory space and a zone bit of a second memory space in the memory;

the first processor sends a screenshot instruction to the second processor when detecting that the flag bit in the first memory space is a first preset value or the flag bit in the second memory space is a second preset value, and sets the flag bit in a fourth memory space in the memory to be a third preset value;

the second processor receives the screenshot instruction sent by the first processor, image capture is carried out on the acquired video stream, and the captured image is stored in a third memory space;

and when detecting that the flag bit in the fourth memory space is a third preset value, the third processor acquires the image from the third memory space and stores the image.

Optionally, after the first processor sets a flag bit of a fourth memory space in the memory to a third preset value, the method further includes:

and clearing the zone bit of the first memory space and the zone bit of the second memory space by the first processor.

Optionally, the method further comprises:

and the third processor clears the zone bit of the fourth memory space after detecting that the zone bit of the fourth memory space is a third preset value.

A fifth aspect of the present application provides a video image capturing apparatus, where the apparatus includes a first processor, the first processor shares a same memory with a second processor and a third processor, and the memory includes a plurality of memory spaces, and the apparatus includes:

the detection module is used for detecting the zone bit of the first memory space and the zone bit of the second memory space in the memory;

the instruction sending module is used for sending a screenshot instruction to the second processor under the condition that the zone bit of the first memory space is a first preset value or the zone bit of the second memory space is a second preset value, so that the second processor stores the intercepted image in a third memory space after receiving the screenshot instruction; when the second processor detects the screenshot operation of the hardware device, triggering the zone bit of the first memory space to be a first preset value; when the third processor receives a network screenshot request sent by a client, triggering the zone bit of the second memory space to be a second preset value;

and the setting module is used for setting a flag bit of a fourth memory space in the memory to a third preset value so as to instruct the third processor to acquire the image from the third memory space and store the image.

A sixth aspect of the present embodiment provides a video image capturing apparatus, where the apparatus includes a second processor, the second processor shares a same memory with a first processor and a third processor, and the memory includes a plurality of memory spaces, and the apparatus includes:

the image interception module is used for receiving the screenshot instruction sent by the first processor and intercepting the image of the acquired video stream; the screenshot instruction is an instruction sent by the first processor when the flag bit of the first memory space is detected to be a first preset value, or is an instruction sent by the first processor when the flag bit of the second memory space is detected to be a second preset value;

the storage module is used for storing the intercepted image in a third memory space, so that the third processor acquires the image from the third memory space and stores the image when detecting that the flag bit in a fourth memory space is a third preset value; after the first processor sends a screenshot instruction, the flag bit of the fourth memory space is triggered to be a third preset value.

A seventh aspect of the present embodiment provides a chip, where a first processor, a second processor, and a third processor are integrated on the chip, the first processor, the second processor, and the third processor share a same memory, the memory includes a plurality of memory spaces, and the first processor, the second processor, and the third processor execute the video image capturing method according to the fourth aspect of the present embodiment.

An eighth aspect of the embodiments of the present application provides a surgical robot, including an endoscope and a camera control unit CCU;

wherein the endoscope is used for capturing a video stream;

the CCU is provided with a chip as provided in the seventh aspect of the embodiment of the present application, and performs image capture on a video stream acquired by the endoscope through the chip.

A ninth aspect of the present application provides a video image capturing system, including a storage device, a client, and a surgical robot as provided in the eighth aspect of the present application;

the storage device is used for receiving the image output by the surgical robot;

the client is used for accessing the images saved in the surgical robot.

By adopting the video image capturing method provided by the embodiment of the application, the zone bits are respectively distributed to a plurality of memory spaces. When a user observes that a lesion part is in the body of a patient, screenshot operation can be executed or a network screenshot request is sent to a third processor through a client, the first processor can detect that the user has a screenshot requirement by using a zone bit of a first memory space and a zone bit of a second memory space and sends a screenshot instruction to the second processor, and meanwhile, the first processor sets a zone bit of a fourth memory space; the second processor intercepts the acquired image in the video stream in response to the screenshot instruction, and stores the intercepted image in a third memory space; and the third processor takes out and stores the image stored in the third memory space by using the flag bit of the fourth memory space, so that the user can acquire the image from the position where the third processor stores the image.

Naturally, when the video image capture method provided by the application is applied to the surgical robot, a user can send a capture demand to the surgical robot under the condition that a diseased part is arranged inside the body of a patient, and then the diseased image can be directly obtained from the position stored by the surgical robot.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive exercise.

Fig. 1 is a flowchart illustrating steps of a first video capture method according to an embodiment of the present application;

FIG. 2 is a flowchart illustrating steps of a second video capture method according to an embodiment of the present application;

FIG. 3 is a flowchart illustrating steps of a third video capture method according to an embodiment of the present application;

FIG. 4 is a flowchart illustrating steps of a fourth video capture method according to an embodiment of the present application;

fig. 5 is a block diagram illustrating a structure of a shared memory shared by a first processor, a second processor, and a third processor according to an embodiment of the present application;

FIG. 6 is a logic diagram for a local screenshot request according to an embodiment of the present application;

FIG. 7 is a logic diagram for a remote screenshot request according to an embodiment of the present application;

fig. 8 is a block diagram illustrating a video capture method according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Referring to fig. 1, a video image capturing method provided in an embodiment of the present application is shown and applied to a first processor, and referring to fig. 2, the first processor, a second processor, and a third processor share a same memory, and the memory includes a plurality of memory spaces, where the method includes the following steps:

step S101: and detecting the zone bit of the first memory space and the zone bit of the second memory space in the memory.

In this embodiment, the first processor may be an ARM (advanced RISC machine) processor without an operating system, that is, an ARM bare machine; the second processor can be an FPGA (field Programmable Gate array) chip; the third processor may be an ARM processor provided with an operating system (e.g., Linux system) which is collectively referred to as GNU/Linux.

The flag bit of the first memory space is different from the flag bit of the second memory space. The flag bit of the first memory space is used for identifying whether the second processor detects the screenshot operation of the hardware device. The flag bit of the second memory space is used for identifying whether the third processor receives a network screenshot request sent by a client in the network.

In an example, the flag bit of the first memory space may be bank1, and when the first processor detects that the bank1 changes, it may be determined whether the second processor detects a screenshot operation of the hardware device according to a value of bank 1; the flag bit of the second memory space may be bank2, and when the first processor detects that the bank2 changes, it may be determined whether the third processor receives a network screenshot request sent by a client in the network according to the value of bank 2.

Step S102: sending a screenshot instruction to the second processor under the condition that the zone bit of the first memory space is a first preset value or the zone bit of the second memory space is a second preset value, so that the second processor stores the intercepted image in a third memory space after receiving the screenshot instruction; when the second processor detects the screenshot operation of the hardware device, triggering the zone bit of the first memory space to be a first preset value; and when the third processor receives a network screenshot request sent by a client in a network, triggering the zone bit of the second memory space to be a second preset value.

In the embodiment of the application, when the flag bit of the first memory space is the first preset value, it is described that the second processor has detected the screenshot operation of the hardware device, so as to trigger the flag bit of the first memory space to be the first preset value; under the condition that the flag bit of the second memory space is the second preset value, it is described that the third processor has received the network screenshot request sent by the client, so as to trigger that the flag bit of the second memory space is the second preset value, and the client can send the network screenshot request to the third processor through a Transmission Control Protocol (TCP). At this time, if the first processor detects that the flag bit of the first memory space is the first preset value or the flag bit of the second memory space is the second preset value, a screenshot command is sent to the second processor, and the second processor intercepts a frame of image from the acquired video stream and stores the frame of image in the third memory space.

The third memory space is used for storing the current frame image in the video stream, and the flag bit of the third memory space may be bank 3.

In one example, when the first processor detects that the value of the flag bit of the bank1 is switched to 1, or the first processor detects that the value of the flag bit of the bank2 is switched to 6, a screenshot instruction may be sent to the second processor. Of course, in the embodiment of the present application, the first preset value is not limited to 1, the second preset value is not limited to 6, and the first preset value and the second preset value may also be the remaining setting symbols, which is not described herein again.

As shown in fig. 5, the first processor may read and control a plurality of IP cores of the second processor through an AXI (advanced eXtensible interface) bus, and the first processor may send a screenshot instruction to the second processor through the AXI bus and process data generated by the second processor; the first processor may also communicate with the second processor and the third processor by detecting a change in a value of a flag bit in the shared memory.

The second processor is mainly used for collecting and processing signals of the hardware equipment, for example, detecting whether the hardware equipment executes screenshot operation; the second processor may also collect and process video stream data, such as video streams captured by a surgical robot, such as an endoscope.

Specifically, the second processor has a plurality of IP cores, each of which may operate independently, for example, the IP cores for the second processor to process video stream data include: inverse bayer operation, color correction processing (CCM), gamma correction processing (Gmma), RGB to YcrCb gamut space transformation (RGB 2 YcrCb), image enhancement (Enhance), captured image output (Cap _ vdma), user image overlay (UI _ vdma), chroma resampling (creatample), video stream interface conversion (axi 4-s to DVP), and the like; the IP core of the second processor for processing the signal of the hardware device is provided with: key screenshot operation detection, IIC communication, and the like.

The third processor is mainly responsible for processing network data, for example, receiving a network screenshot request sent by a client in a network; the third processor can also save the image, for example, the image is saved and packaged into an image file and saved in the linux system of the third processor; the third processor may also operate a storage device, such as a USB device; remote login, File Transfer Protocol (FTP) Protocol, and the like may also be implemented.

When the system is used specifically, the second processor acquires a video stream acquired by the surgical robot and intercepts the image of the video stream; when the second processor detects the screenshot operation of the hardware device, triggering the bank1 to be 1; when the third processor receives a network screenshot request sent by the client, the trigger bank2 is 6.

At this time, if the first processor detects that the bank1 is 1 or detects that the bank2 is 6, sending a screenshot instruction to the second processor through the TXI bus; and the second processor receives the current moment of the screenshot instruction, intercepts the current frame image in the acquired video stream and stores the current frame image in the bank 3. Since one frame of image data is 1920 × 1080 pixels, and the RGB data of each pixel occupies 4 bytes, the size of the third memory space occupied by one frame of image data is 1920 × 1080 × 4 bytes.

Step S103: and setting a flag bit of a fourth memory space in the memory to a third preset value to instruct the third processor to acquire the image from the third memory space and store the image.

In the embodiment of the application, after the first processor sends a screenshot instruction to the second processor, the second processor intercepts a current frame image from a collected video stream and stores the current frame image in a third memory space; meanwhile, the first processor sets the flag bit of the fourth memory space to a third preset value to instruct the third processor to acquire the current frame image from the third memory space.

The fourth memory space may be bank4 or bank5, and when the first processor detects that the flag bit of the first memory space is the first preset value, that is, the screenshot operation triggered by the hardware device, the flag bit of the bank4 is set to 1; when the first processor detects that the flag bit of the second memory space is the second preset value, that is, when a network screenshot request sent by the client is received, the flag bit of the bank5 is set to 1.

The second processor is used for carrying out image interception on the acquired video stream and the first processor is used for setting the flag bit of the fourth memory space to be the third preset value, and the processing steps are carried out in parallel, so that the third processor can immediately determine that the current frame image is stored in the third memory space when the second processor finishes storing the intercepted current frame image in the third memory space, and the speed of feeding back the current frame image to the user by the third processor is higher.

In this embodiment of the application, the third processor reads raw data of RGB of the image from the third memory space, and packages the raw data into an image file for storage.

Specifically, at the time of saving, if the image read from the third memory space by the third processor is an image read for a screen capture operation of the hardware device, the image file is saved in the storage device mounted on the third processor, for example, under the mounting path/mnt/USB of the USB device, and the file name format of the image file is automatically set to the current date + the current time + the suffix, in the form of "MM minutes SS seconds · bmp" at MM month DD date HH of YYYY year.

If the image read by the third processor from the third memory space is the image read by the network screenshot request sent by the client by using the TCP protocol, the image file is saved in the linux system in the path of/home/snapshot, and the file name format of the image file is automatically set to be current date + current time + suffix, which is similar to "YYYY year MM, DD, hd, HH, MM, SS, seconds, bmp"; and finally, packaging the storage path and the file name of the image file into a screenshot storage completion message and sending the screenshot storage completion message to the client, and downloading the image file in the storage path by using an FTP (file transfer protocol) after receiving the screenshot storage completion message carrying the storage path and the file name.

Wherein, the hardware equipment can be surgical robots such as endoscopes and the like; the client has an Application program (App) and a browser, and the user can download and save the image file in the path using the App or the browser.

By adopting the video image capturing method provided by the application, the local capturing operation of hardware equipment can be directly responded, for example, a capturing function button on an endoscope and other surgical robots is responded by a user, and an image file is sent to the storage equipment; the image file can also be directly stored in the linux system in response to a network screenshot request remotely sent by a client, for example, in response to a network screenshot request remotely sent by a mobile phone or a computer, so as to be remotely downloaded by a user using the mobile phone or the computer. The image file can be obtained locally or remotely, and the use flexibility is higher.

In this embodiment, after the flag bit of the fourth memory space is set to the third preset value, it indicates that the first processor has successfully sent the screenshot request to the second processor according to the flag bit of the first memory space or the flag bit of the second memory space, and at this time, the flag bit of the first memory space and the flag bit of the second memory space have no function, so that the flag bit of the first memory space and the flag bit of the second memory space can be cleared, and it is avoided that the flag bit of the first memory space and the flag bit of the second memory space are not updated to affect the next screenshot or the screenshot operation is repeatedly performed to cause endless loop.

In the case where the first processor clears the flag bit of the first memory space and the flag bit of the second memory space, a clearing failure may occur. In order to further avoid that the flag bit of the first memory space and the flag bit of the second memory space are not updated to influence the next screenshot, after the third processor stores the image, whether the flag bit of the first memory space and the flag bit of the second memory space are cleared is detected; clearing the zone bit of the first memory space under the condition that the zone bit of the first memory space is not cleared; and clearing the zone bit of the second memory space under the condition that the zone bit of the second memory space is not cleared.

Therefore, after the logic of the whole video image capturing method is completely executed, whether the flag bit of the first memory space and the flag bit of the second memory space are cleared or not can be checked again, and the flag bit of the first memory space and the flag bit of the second memory space are cleared for the second time under the condition that the flag bits are not cleared.

In this embodiment of the application, because a plurality of memory spaces, such as a first memory space, a second memory space, a third memory space, and a fourth memory space, are provided in the shared memory, and each different memory space stores different types of data, it is determined according to the memory space flag bit by setting the memory space flag bit, which type of data is stored in each different memory space, and which logic is executed by the processors, so that the communication interaction among the first processor, the second processor, and the third processor can be realized through the flag bit of each memory space in the shared memory among the processors.

For example, in some scenes, the third memory space may store a history image in the history video stream, and if the third processor detects that the image is stored in the third memory space, the third processor directly takes out the history image from the third memory space and feeds back the history image to the user, which may cause the user to obtain the history image in the history video stream instead of the current frame image required by the user at the current time, and further cause poor user experience.

In order to avoid that a user receives an incorrect historical image, flag bits are respectively set in a plurality of memory spaces in a shared memory. When the flag bit of the first memory space is switched to a first preset value (for example, the value of bank1 is switched to 1), the first processor may determine that the second processor has detected that the hardware device has performed the screenshot operation; when the flag bit of the second memory space is switched to a second preset value (for example, the value of bank2 is switched to 6), the first processor may determine that the third processor has received the screenshot request; the method comprises the steps that a first processor responds to a screenshot operation or a network screenshot request and sends a screenshot instruction to a second processor, so that the second processor stores a current frame image in a video stream in a third memory space, and after the first processor sends the screenshot instruction to the second processor, the first processor switches a flag bit of a fourth memory space to a third preset value (for example, the value of bank4 is switched to 1); when the third processor detects that the flag bit of the fourth memory space is switched to the third preset value, it can be determined that the image is updated in the third memory space, and the image is taken out from the third memory space and sent to the user.

Therefore, in the application, the first processor, the second processor and the third processor determine which logics are executed among the processors or which data are stored in the memory space by sharing the flag bit of each memory space in the memory, and then can feed accurate data back to a user according to the flag bit of the memory space, so that the user experience is improved.

Example two

Referring to fig. 2, a video image capturing method provided in an embodiment of the present application is shown, and is applied to a second processor, where the second processor shares a same memory with a first processor and a third processor, and the memory includes a plurality of memory spaces, and the method includes the following steps:

step S201: receiving a screenshot instruction sent by the first processor, and carrying out image interception on the acquired video stream; the screenshot instruction is an instruction sent by the first processor when the flag bit of the first memory space is detected to be a first preset value, or is an instruction sent by the first processor when the flag bit of the second memory space is detected to be a second preset value.

In this embodiment, the second processor may continuously capture and process the video stream, for example, the surgical robot may continuously capture images of the internal organs of the human body, and the second processor may continuously capture the video stream of the internal organs of the human body from the surgical robot.

When the method is used specifically, the second processor intercepts the current frame image in the acquired video stream at the current moment when the screenshot command sent by the first processor is received. For example, when the second processor receives the screenshot instruction sent by the first processor at "10/14/9/15/25 sec in 2021", the current frame image in the acquired video stream is intercepted at the current time "15/25 sec at 9/10/14/2021".

The flag bit of the first memory space is a first preset value and is triggered by the following steps:

step S2011: and detecting whether a hardware device has screenshot operation.

In this step, the hardware device may be a screenshot function button on the surgical robot; a remote control communicatively coupled to the second processor, etc.

When the surgical robot is used specifically, when a user observes a human body pathological organ through the surgical robot, the user can directly press a screenshot function button or a remote controller on the surgical robot to implement screenshot operation.

Step S2012: and setting the zone bit of the first memory space to be a first preset value under the condition that the screenshot operation executed on the hardware equipment is detected.

During specific application, when it is detected that a screenshot function button on the hardware device is triggered, the flag bit of the first memory space is set to a first preset value.

After the flag bit of the fourth memory space is set to the third preset value by the first processor, the flag bit of the first memory space is actively cleared by the first processor; or clearing the zone bit of the first memory space when the second processor detects that the screenshot function button on the hardware equipment is reset.

Step S202: storing the intercepted image in a third memory space, so that the third processor acquires the image from the third memory space and stores the image when detecting that the flag bit in a fourth memory space is a third preset value; after the first processor sends a screenshot instruction, the flag bit of the fourth memory space is triggered to be a third preset value.

In the embodiment of the application, the first processor intercepts the image after receiving the screenshot instruction sent by the first processor, and stores the intercepted image in the third memory space.

EXAMPLE III

Referring to fig. 3, a video image capturing method provided in an embodiment of the present application is shown, and is applied to a third processor, where the third processor, a second processor and a first processor share a same memory, and the memory includes a plurality of memory spaces, and the method includes:

step S301: and detecting the flag bit of the fourth memory space.

In this embodiment, after the first processor sends the screenshot instruction to the second processor, the flag bit of the fourth memory space is controlled to be the third preset value to indicate that the image stored by the second processor according to the screenshot instruction is stored in the third memory space, so that the third processor detects the flag bit of the fourth memory space to determine whether the image stored according to the screenshot instruction is stored in the third memory space.

Step S302: when the flag bit of the fourth memory space is detected to be a third preset value, reading an image in a third memory space, and storing the image;

In this embodiment, in order to avoid misleading the third processor to assume that the flag bit of the fourth memory space is the third preset value in the next screenshot operation, when the third processor detects that the flag bit of the fourth memory space is the third preset value, the third processor needs to clear the flag bit of the fourth memory space.

The fourth memory space may be bank4 or bank5, and when the first processor detects that the flag bit of the first space is the first preset value, that is, the screenshot operation triggered by the hardware device, the flag bit of the bank4 is set to 1; when the first processor detects that the flag bit of the second memory space is the second preset value, that is, when a network screenshot request sent by the client is received, the flag bit of the bank5 is set to 1.

When the method is used specifically, after the third processor detects that the flag bit of the bank4 is 1, the flag bit of the bank4 is automatically cleared; after the third processor detects that the flag bit of the bank5 is 1, the flag bit of the bank5 is automatically cleared.

When detecting that the flag bit of the fourth memory space is a third preset value, the third processor reads the image from the third memory space and stores the image in the storage device or stores the image in a file system of the third processor.

Specifically, when detecting that the flag bit of the bank4 is 1, the third processor reads an image from the third memory space, packages the image into an image file, and stores the image file in a storage device, such as a USB device, mounted on the third processor.

When detecting that the flag bit in the bank5 is 1, the third processor reads an image from a third memory space, packages the image into an image file and stores the image file in a linux system of the third processor, packages a storage path and a file name of the image file into a screenshot storage completion message, and sends the screenshot storage completion message to a client in the network; and the client generates an image downloading request according to the storage path and the file name of the image file carried in the screenshot storage completion message, sends the image downloading request to the third processor, and feeds back the image corresponding to the image downloading request to the client if the third processor receives the image downloading request.

By adopting the video image capturing method provided by the embodiment of the application, the zone bits are respectively distributed to a plurality of memory spaces. When a user observes that a lesion part is in the body of a patient, screenshot operation can be executed or a network screenshot request is sent to a third processor through a client, the first processor can detect that the user has a screenshot requirement by using a zone bit of a first memory space and a zone bit of a second memory space and sends a screenshot instruction to the second processor, and meanwhile, the first processor sets a zone bit of a fourth memory space; the second processor intercepts the acquired image in the video stream in response to the screenshot instruction, and stores the intercepted image in a third memory space; and the third processor takes out and stores the image stored in the third memory space according to the flag bit of the fourth memory space, so that the user can acquire the image from the position where the image is stored by the third processor.

Example four

Referring to fig. 4, a video image capturing method is shown, which is applied to a first processor, a second processor and a third processor, where the first processor, the second processor and the third processor share a same memory, and the memory includes a plurality of memory spaces, and the method includes:

step S401: when the second processor detects the screenshot operation of the hardware equipment, triggering the zone bit of the first memory space to be a first preset value; and when receiving a network screenshot request sent by a client in a network, the third processor triggers the zone bit of the second memory space to be a second preset value.

In the embodiment of the application, the screenshot requirement of the user can be realized through hardware equipment or a client.

For example, the user may press a screenshot function button on the surgical robot or press a remote controller communicatively connected to the second processor to perform a screenshot operation; the user may also send a web screenshot request to the third processor through a browser or app on the client.

When the second processor detects that the screenshot function button is pressed or the remote controller is pressed, triggering the zone bit of the first memory space to be a first preset value; and when the third processor receives a network screenshot request sent by the client, triggering the zone bit of the second memory space to be a second preset value.

Step S402: and the first processor detects the zone bit of the first memory space and the zone bit of the second memory space in the memory.

In this embodiment of the application, since the flag bit change of the first memory space or the flag bit change of the second memory space both represent that the user has a remote or local screenshot requirement, the first processor needs to detect the flag bit of the first memory space and the flag bit of the second memory space to determine whether the user has the screenshot requirement.

Step S403: and the first processor sends a screenshot instruction to the second processor and sets a flag bit of a fourth memory space in the memory to be a third preset value under the condition that the flag bit of the first memory space is detected to be a first preset value or the flag bit of the second memory space is detected to be a second preset value.

In this embodiment, when the first processor detects that the flag bit of the first memory space is the first preset value, it indicates that the user has a local screenshot requirement, and at this time, a screenshot instruction may be sent to the second processor, so that the second processor stores the captured image in the third memory space, and sets the flag bit of the fourth memory space to the third preset value, so as to represent that the third memory space stores the image captured according to the screenshot instruction.

When the first processor detects that the flag bit of the second memory space is the second preset value, the user has a remote screenshot requirement, and at this time, a screenshot instruction can be sent to the second processor, so that the second processor stores the intercepted image in a third memory space, and the flag bit of the fourth memory space is set to the third preset value, so that the third memory space is represented to store the image intercepted according to the screenshot instruction.

After setting the flag bit of the fourth memory space in the memory to the third preset value, the first processor clears the flag bit of the first memory space and the flag bit of the second memory space.

Step S404: and the second processor receives the screenshot instruction sent by the first processor, intercepts the acquired video stream, and stores the intercepted image in a third memory space.

In the embodiment of the application, after the second processor acquires the screenshot instruction sent by the first processor, the second processor intercepts the current frame image in the acquired video stream and stores the current frame image in the third memory space.

Step S405: and when detecting that the flag bit in the fourth memory space is a third preset value, the third processor acquires the image from the third memory space and stores the image.

In this embodiment, when the third processor detects that the flag bit in the fourth memory space is the third preset value, it indicates that the push-to-screenshot according to the screenshot instruction has been stored in the third memory space, and the image may be obtained from the third memory space and stored.

And the third processor clears the zone bit of the fourth memory space after detecting that the zone bit of the fourth memory space is the third preset value.

In a scene where the user has a local screenshot requirement, referring to fig. 6, the user may press a remote controller communicatively connected to the second processor or a screenshot function button on the surgical robot to perform a screenshot operation; after detecting that the remote controller or the screenshot function button is subjected to screenshot operation, the second processor sets the flag bit of the bank1 to 1; after detecting that the flag bit of the bank1 is converted into 1, the first processor sends a screenshot instruction to the second processor, and sets the flag bit in the bank4 as 1; the second processor intercepts the current frame image in the acquired video stream in response to the screenshot instruction, and stores the intercepted image in the bank 3; after the third processor detects that the flag bit in the bank4 is converted into 1, the current frame image captured according to the screenshot instruction is taken out from the bank3, the current frame image is stored in the USB device, and the user can obtain the current frame image from the USB device.

In a scenario where the user has a remote screenshot requirement, referring to fig. 7, the user may send a screenshot request to the third processor using a browser or app on the client; after receiving the network screenshot request, the third processor sets the flag bit of the bank2 to 6; after detecting that the flag bit of the bank2 is converted into 6, the first processor sends a screenshot instruction to the second processor, and sets the flag bit in the bank5 to be 1; the second processor responds to the screenshot instruction, intercepts the current frame image in the acquired video stream, and stores the intercepted image in the bank 3; after the third processor detects that the flag bit in the bank5 is converted into 1, taking out the current frame image intercepted according to the screenshot instruction from the bank3, packaging the current frame image into an image file and storing the image file in a linux system of the third processor, packaging a storage path and a file name of the image file into a screenshot storage completion message, and sending the screenshot storage completion message to a client through a TCP (transmission control protocol); the client sends an image downloading request to the third processor according to the storage path and the file name of the image file carried in the screenshot storage completion message; the third processor feeds back the image file to the client according to the image downloading request, and a user using the client can obtain the image file.

EXAMPLE five

Based on the same inventive concept, an embodiment of the present application provides a video image capturing apparatus, please refer to fig. 8, where the apparatus includes a first processor, the first processor, a second processor, and a third processor share a same memory, and the memory includes a plurality of memory spaces, and the apparatus includes:

the instruction sending module is used for sending a screenshot instruction to the second processor under the condition that the zone bit of the first memory space is a first preset value or the zone bit of the second memory space is a second preset value, so that the second processor stores the intercepted image in a third memory space after receiving the screenshot instruction; when the second processor detects the screenshot operation of the hardware device, triggering the zone bit of the first memory space to be a first preset value; when the third processor receives a network screenshot request sent by a client in a network, triggering the zone bit of the second memory space to be a second preset value;

Optionally, the apparatus further comprises:

and the first clearing module is used for clearing the zone bit of the first memory space and the zone bit of the second memory space.

Optionally, the apparatus further comprises:

and the clearing detection module is used for detecting whether the zone bit of the first memory space and the zone bit of the second memory space are cleared or not after the third processor stores the image.

A second clearing module, configured to clear the flag bit of the first memory space and the flag bit of the second memory space when the flag bit of the first memory space is not cleared.

EXAMPLE six

Based on the same inventive concept, an embodiment of the present application further provides a video image capturing apparatus, where the apparatus includes a second processor, the second processor shares a same memory with the first processor and the third processor, and the memory includes a plurality of memory spaces, and the apparatus includes:

Optionally, the apparatus further comprises:

the screenshot operation detection module is used for detecting whether the hardware equipment has screenshot operation;

the first setting module is used for setting the zone bit of the first memory space to a first preset value under the condition that the screenshot operation executed on the hardware equipment is detected.

Optionally, the first setting module further includes:

and the second setting module is used for setting the zone bit of the first memory space to be a first preset value when detecting that the screenshot function button on the hardware equipment is triggered.

The device further comprises:

and the automatic reset module is used for automatically clearing the zone bit of the first memory space when detecting that the screenshot function button on the hardware equipment is reset.

EXAMPLE seven

Based on the same inventive concept, an embodiment of the present application further provides a chip, where a first processor, a second processor, and a third processor are integrated on the chip, the first processor, the second processor, and the third processor share a same memory, and the memory includes a plurality of memory spaces, and the first processor, the second processor, and the third processor execute the video image capturing method provided in the fourth embodiment.

Wherein, the chip can be a zynq chip.

Example eight

Based on the same inventive concept, the embodiment of the application also provides a surgical robot, which comprises an endoscope and a camera control unit CCU; wherein the endoscope is used for capturing a video stream; the CCU is provided with the chip provided by the seventh embodiment, and image capturing is performed on the video stream acquired by the endoscope through the chip.

Example nine

Based on the same inventive concept, the embodiment of the application further provides a video image capturing system, which comprises a storage device, a client and the surgical robot provided by the eighth embodiment;

the storage device is used for receiving images output by the surgical robot, and specifically, receiving image files saved by a linux system of a third processor in the surgical robot.

The client is used for accessing the images saved in the surgical robot, and specifically, a client user accesses image files saved in a linux system of a third processor in the surgical robot.

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

As will be appreciated by one of skill in the art, embodiments of the present application may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present application have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the true scope of the embodiments of the application.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.

The video image capturing method, the video image capturing device, the video image capturing chip, the surgical robot and the surgical system provided by the application are introduced in detail, a specific example is applied in the description to explain the principle and the implementation mode of the application, and the description of the embodiment is only used for helping to understand the method and the core idea of the application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A video image intercepting method is applied to a first processor, the first processor, a second processor and a third processor share the same memory, and the memory comprises a plurality of memory spaces, and the method comprises the following steps:

2. The method of claim 1, wherein after setting a flag bit of a fourth memory space in the memory to a third preset value, the method comprises:

3. The method of claim 2, further comprising:

4. A video image intercepting method is applied to a second processor, the second processor, a first processor and a third processor share the same memory, and the memory comprises a plurality of memory spaces, and the method comprises the following steps:

5. The method of claim 4, wherein the flag bit of the first memory space is a first preset value, triggered by the following steps:

detecting whether a hardware device has screenshot operation;

6. The method of claim 5, wherein the screenshot action is a screenshot function button; setting the flag bit of the first memory space to a first preset value under the condition that the screenshot operation executed on the hardware equipment is detected, wherein the method comprises the following steps:

the method further comprises the following steps:

7. A video image intercepting method is applied to a third processor, the third processor, a second processor and a first processor share the same memory, and the memory comprises a plurality of memory spaces, and the method comprises the following steps:

detecting a flag bit of a fourth memory space;

when detecting the screenshot operation of the hardware equipment, the second processor triggers a flag bit of a first memory space to be a first preset value; when the third processor receives a network screenshot request sent by a client, triggering a flag bit of a second memory space to be a second preset value; and when the first processor sends a screenshot instruction to the second processor under the condition that the flag bit of the first memory space is detected to be a first preset value or the flag bit of the second memory space is detected to be a second preset value, triggering the first processor to set the flag bit of the fourth memory space to be a third preset value.

8. The method of claim 7, further comprising:

9. The method of claim 7, wherein saving the image comprises:

10. The method of claim 9, wherein if the image is saved in its own file system, the method further comprises:

and if an image downloading request sent by the client is received, sending the image to the client.

11. A video image intercepting method is applied to a first processor, a second processor and a third processor, wherein the first processor, the second processor and the third processor share the same memory, and the memory comprises a plurality of memory spaces, and the method comprises the following steps:

when the second processor detects the screenshot operation of the hardware equipment, triggering the zone bit of the first memory space to be a first preset value; when the third processor receives a network screenshot request sent by a client, the flag bit of a second memory space is triggered to be a second preset value;

12. The method of claim 11, wherein after the first processor sets a flag bit of a fourth memory space in the memory to a third preset value, the method further comprises:

13. The method of claim 11, further comprising:

14. A video image capturing device, the device comprising a first processor, wherein the first processor shares a same memory with a second processor and a third processor, and the memory comprises a plurality of memory spaces, the device comprising:

15. A video image capturing apparatus, comprising a second processor, wherein the second processor shares a same memory with a first processor and a third processor, and the memory comprises a plurality of memory spaces, the apparatus comprising:

16. A chip, wherein a first processor, a second processor and a third processor are integrated on the chip, the first processor, the second processor and the third processor share a same memory, and the memory includes a plurality of memory spaces, and the first processor, the second processor and the third processor execute the video image capturing method according to any one of claims 11 to 13.

17. A surgical robot, characterized in that it comprises an endoscope and a camera control unit CCU;

wherein the endoscope is used for capturing a video stream;

the CCU is provided with the chip of claim 16, and image capture is performed on a video stream captured by the endoscope by the chip.

18. A video image capture system comprising a storage device, a client, and the surgical robot of claim 17;

the client is used for accessing the images saved in the surgical robot.