CN117158877A

CN117158877A - Image display method of endoscope image pickup system, endoscope image pickup host and system

Info

Publication number: CN117158877A
Application number: CN202311131115.XA
Authority: CN
Inventors: 龚伟枫; 吴晓华
Original assignee: Shenzhen Mindray Bio Medical Electronics Co Ltd; Wuhan Mindray Medical Technology Research Institute Co Ltd
Current assignee: Shenzhen Mindray Bio Medical Electronics Co Ltd; Wuhan Mindray Medical Technology Research Institute Co Ltd
Priority date: 2023-02-28
Filing date: 2023-08-31
Publication date: 2023-12-05
Also published as: CN116138714A

Abstract

The application discloses an image display method of an endoscope image pickup system, an endoscope image pickup host and an endoscope image pickup system, wherein the endoscope image pickup system acquires an image signal of a shooting image through a camera and displays the shooting image in a display area of a display, the endoscope image pickup system detects the position of the center of the endoscope image in the shooting image in real time, if the center of the endoscope image is not in the center area of the shooting image, the endoscope image pickup system carries out correction processing on the shooting image and obtains a target image and displays the target image in the display area of the display, so that the endoscope image which is originally eccentrically displayed can be displayed in the center of the display area, thereby facilitating operation and observation of doctors and improving the efficiency of operations.

Description

Image display method of endoscope image pickup system, endoscope image pickup host and system

The present application claims priority from a chinese patent application filed 2/28 of 2023, application number "2023101957781", application name "image display method of endoscopic camera system and inventive patent application of endoscopic camera host computer, system", which are incorporated herein by reference in their entirety.

Technical Field

The embodiment of the application relates to the technical field of endoscopes, in particular to an image display method of an endoscope image pickup system, an endoscope image pickup host and an endoscope image pickup system.

Background

In recent medical treatment, diagnosis and the like using an endoscope imaging system generally including a camera, an endoscope imaging main body, and a display are widely performed. The camera shoots images through the endoscope, the shot image data are transmitted to the endoscope camera host through the transmission cable, the endoscope camera host processes video information acquired by the camera, and the display displays the video information processed by the camera host, so that the display displays the endoscope images shot by the camera. In the use scene of an endoscope camera system, in order to meet the demands of various operation types, a camera is required to be assembled with endoscopes of different types or different calibers, and because the calibers of the endoscopes are different, the sizes of the endoscope images displayed on a display are different, so that in order to observe the images on the display, a doctor is often required to manually conduct enlargement/reduction adjustment operation, and the sizes of the endoscope images displayed on the display are controlled.

Disclosure of Invention

The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

The embodiment of the application provides an information display method of an endoscope camera host, the endoscope camera host and a system, which can display proper endoscope images on a display, and are convenient for doctors to operate and observe.

In a first aspect, an embodiment of the present application provides an image display method of an endoscopic imaging system, the endoscopic imaging system including a display area, the method including:

the endoscope camera system obtains a shooting image sent by a camera, wherein the shooting image comprises the endoscope image;

the endoscope image pickup system detects the position of the center of the endoscope image in the shot image in real time, and if the center of the endoscope image is not in the center area of the shot image, the endoscope image pickup system performs correction processing on the shot image and obtains a target image, wherein the target image comprises at least a part of the endoscope image;

the endoscope image pickup system outputs a display signal to the display according to the image information of the target image so as to enable the display area to display the target image, wherein the center of the endoscope image displayed in the display area is overlapped with the center of the display area.

In a second aspect, an embodiment of the present application provides an image display method of an endoscopic image pickup system including a display for displaying a display area of an endoscopic image, the method including:

the endoscope image pickup system detects the size of the endoscope image in the photographed image in real time, and if the size of the endoscope image does not meet a preset magnification ratio, the endoscope image pickup system performs magnification processing on the photographed image and obtains a target image, wherein the target image comprises at least a part of the endoscope image;

the endoscope image pickup system outputs a display signal to the display according to the image information of the target image so that the display area displays the target image.

In a third aspect, an embodiment of the present application provides an image display method of an endoscopic image pickup system including a display for displaying a display area of an endoscopic image, the method including:

The endoscope camera system acquires an image signal of a shooting image sent by a camera;

the endoscope camera system enters a first display mode and outputs a first display signal to the display according to the image signal so as to enable the display area to display the shot image, wherein the shot image displayed in the display area comprises an endoscope image, and the center of the endoscope image is deviated from the center of the display area;

the endoscope camera system acquires a scaling operation instruction of a user in a first display mode, and adjusts the first display signal according to the scaling operation instruction so that a picture displayed in the display area is scaled along the center of the display area, and the center of the endoscope image displayed in the display area deviates from the center of the display area after scaling;

the endoscope camera system acquires a centering amplifying operation instruction of a user and enters a second display mode, and in the second display mode, the endoscope camera system determines a target intercepting region containing at least a part of the endoscope image from the photographed image according to the aspect ratio of the display region, and the center of the target intercepting region coincides with the center of the endoscope image;

The endoscope camera system outputs a second display signal to the display according to the image information of the target interception area so as to amplify the endoscope image of the target interception area to the whole display area, wherein the center of the endoscope image displayed in the display area is overlapped with the center of the display area.

In a fourth aspect, an embodiment of the present application provides an image display method of an endoscope image pickup system, the endoscope image pickup system including a display area, the method including:

the endoscope image pickup system detects the position of the center of the endoscope image in the shot image in real time, and if the center of the endoscope image is not in the center area of the shot image, the endoscope image pickup system horizontally centers the shot image and obtains a target image, wherein the target image comprises at least a part of the endoscope image;

the endoscope camera system outputs a display signal to the display according to the image information of the target image so as to enable the display area to display the target image, wherein the center of the endoscope image displayed in the display area coincides with the central axis of the horizontal direction of the display area.

In a fifth aspect, an embodiment of the present application provides an image display method of an endoscopic imaging system, the endoscopic imaging system including a display area, the method including:

the endoscope camera system outputs a first display signal to the display according to the image signal so as to enable the display area to display the shooting image, wherein the shooting image displayed in the display area comprises an endoscope image;

the endoscope camera system acquires a centering amplifying operation instruction of a user, and determines a target interception area containing at least a part of the endoscope image from the photographed image, wherein the center of the target interception area coincides with the center of the endoscope image;

In some embodiments, the endoscopic imaging system acquires a centering zoom operation instruction of a user, and determines a target cut-out area containing at least a part of the endoscopic image from the captured image, including:

the endoscope camera system acquires a centering amplifying operation instruction of a user, and determines a target intercepting region containing at least a part of the endoscope image from the photographed image according to the aspect ratio of the display region.

In a sixth aspect, an embodiment of the present application provides an image display method of an endoscopic imaging system, the endoscopic imaging system including a display area, the method including:

In some embodiments, the determining the target cut-out region of the endoscopic image according to the aspect ratio of the display region includes:

the endoscope imaging system determines an endoscope image region of the endoscope image from the captured image;

the endoscope image pickup system determines the center of the endoscope image area and the scope of the endoscope image according to the endoscope image area;

The endoscopic imaging system determines the target cut-out region based on an aspect ratio of the display region, a center of the endoscopic image region, and an endoscopic image range, wherein the center of the target cut-out region coincides with the center of the endoscopic image region, the aspect ratio of the target cut-out region is the same as the aspect ratio of the display region, and the target cut-out region contains at least a portion of the endoscopic image.

In some embodiments, the endoscopic imaging system determines an endoscopic image area of the endoscopic image from the captured image, comprising:

the endoscope camera system converts the photographed image into a gray scale image;

and the endoscope image pickup system performs threshold segmentation processing on the gray level image according to a preset threshold value to obtain a binary image, wherein the area contained in the binary image is the endoscope image area.

In some embodiments, the endoscopic imaging system determining a center of the endoscopic image area and an endoscopic image range from the endoscopic image area comprises:

the endoscope image pickup system performs distribution statistical calculation on coordinates of each pixel point in the endoscope image area, and determines the center of the endoscope image area and the scope of the endoscope image.

In some embodiments, the endoscopic imaging system performs a distributed statistical calculation of coordinates of each pixel point in the endoscopic image area, determines a center of the endoscopic image area and the endoscopic image range, comprising:

the endoscope image pickup system performs expected distribution calculation on coordinates of each pixel point of the endoscope image in the endoscope image area to obtain coordinates of the center of the endoscope image area;

the endoscope image pickup system calculates a variance distribution of coordinates of each pixel point of the endoscope image in the endoscope image region, and obtains the scope of the endoscope image region.

In some embodiments, the endoscopic imaging system performs a desired distribution calculation on coordinates of each pixel point of the endoscopic image in the endoscopic image area to obtain coordinates of a center of the endoscopic image area, including:

the endoscope image pickup system carries out expected distribution calculation on the coordinates of each row of pixel points in the endoscope image area to obtain a first expected value of the coordinates of each row of pixel points;

the endoscope image pickup system carries out expected distribution calculation on the coordinates of each column of pixel points in the endoscope image area to obtain a second expected value of the coordinates of each column of pixel points;

And the endoscope image pickup system performs statistical calculation on each first expected value and each second expected value to obtain coordinates of the center of the endoscope image area.

In some embodiments, the endoscopic image area is circular, the center of the endoscopic image area is the center of the endoscopic image area, the endoscopic image capturing system determines the center of the endoscopic image area and the endoscopic image range from the endoscopic image area, comprising:

the endoscope image pickup system determines the circle center and the radius of the endoscope image area according to the endoscope image area.

In some embodiments, the target cut-out region is inscribed inside the endoscopic image area or circumscribed outside the endoscopic image area.

In some embodiments, the endoscopic imaging system determining the target cut-out region from an aspect ratio of the display region, a center of the endoscopic image region, and an endoscopic image range comprises:

the endoscope image pickup system judges whether the diameter of the endoscope image is larger than the height of the display area;

if the diameter of the endoscope image is larger than the height of the display area, the endoscope image pickup system determines the target interception area inscribed in the endoscope image according to the circle center of the endoscope image area and the aspect ratio of the display area;

And if the diameter of the endoscope image is smaller than or equal to the height of the display area, the endoscope image pickup system determines the target interception area circumscribed outside the endoscope image according to the circle center of the endoscope image area and the aspect ratio of the display area.

In some embodiments, the endoscopic imaging system outputs a second display signal to the display according to the image information of the target cut-out area to zoom in an endoscopic image of the target cut-out area to the whole display area, including:

the endoscope camera system determines target amplified image frame information according to the image information of the target interception area;

the endoscope camera system carries out interpolation processing on the target amplified image frame information to obtain a plurality of transition image frame information with different amplification degrees;

the endoscope camera system outputs a second display signal to the display according to the transition image frame information and the target amplified image frame information, so that the image of the target intercepting region displayed in the display region is gradually amplified to the whole display region.

In some embodiments, the method further comprises:

The endoscope camera system redetermines the target interception area of the endoscope image from the photographed image every N frames in the second display mode, wherein N is a natural number;

the endoscope camera system updates the second display signal output by the display according to the image information of the redetermined target interception area so as to amplify the endoscope image of the redetermined target interception area to the whole display area.

In some embodiments, the redefining the target cut-out area of the endoscope image from the captured image every N frames includes:

the endoscope image pickup system redetermines an endoscope image area of the endoscope image from the photographed image every N frames;

the endoscope image pickup system redetermines a target center and a target endoscope image range of the endoscope image area according to the endoscope image area;

the endoscopic camera system redetermines the target cut-out region according to the aspect ratio of the display region, the redetermined target center of the endoscopic image region and the target endoscopic image range, wherein the target center of the target cut-out region coincides with the center of the endoscopic image region, the aspect ratio of the target cut-out region is the same as the aspect ratio of the display region, and the target cut-out region contains at least a part of the endoscopic image.

In some embodiments, the endoscopic imaging system redetermines a target center and a target endoscopic image range of the endoscopic image area from the endoscopic image area, comprising:

the endoscope image pickup system redetermines the center of the endoscope image area and the scope of the endoscope image according to the endoscope image area;

the endoscope image pickup system compares the center of the endoscope image area and/or the scope of the endoscope image area before and after the redetermination, if the first threshold condition is exceeded, the center of the endoscope image area and the scope of the endoscope image after the redetermination are used as the target center and the target scope of the endoscope image, and if the first threshold condition is not exceeded, the center of the endoscope image area and the scope of the endoscope image before the redetermination are used as the target center and the target scope of the endoscope image.

In some embodiments, the endoscopic camera system acquires a user's centering zoom operation instruction and enters the second display mode by at least one of:

the endoscope camera system obtains a centering amplifying operation instruction through the operation of a user on a key on the endoscope camera system;

Or the endoscope camera system obtains a centering amplifying operation instruction through the operation of a user on a key on the camera;

or the endoscope camera system acquires a centering amplifying operation instruction through the operation of a user on a key on the display;

or, the endoscope camera system acquires a centering amplifying operation instruction through a voice instruction of a user.

In a seventh aspect, an embodiment of the present application provides an endoscope camera host, including:

a housing;

the communication interface is used for being connected with the endoscope camera in a wired or wireless way;

the display interface is arranged on the shell and is used for being connected with the display;

the processor acquires the image signals of the camera to perform image processing and then outputs the processed image signals to the display to display;

the processor is further configured to execute the image display method of the endoscopic imaging system according to any one of the above embodiments.

In an eighth aspect, an embodiment of the present application provides an endoscope image capturing system, including a light source, a camera, a display, and an endoscope image capturing host according to the sixth aspect, where the endoscope image capturing host is connected to the display through a video cable.

The image display method of the endoscope image pickup system, the endoscope image pickup host and the system provided by the embodiment of the application have at least the following beneficial effects: the endoscope image pickup system acquires an image signal of a shooting image through a camera and displays the shooting image in a display area of a display, and detects the position of the center of the endoscope image in the shooting image in real time, if the center of the endoscope image is not in the center area of the shooting image, the endoscope image pickup system carries out deviation correction processing on the shooting image and obtains a target image and displays the target image in the display area of the display, so that the endoscope image which is originally eccentrically displayed can be displayed in the center of the display area, thereby being convenient for doctors to operate and observe, and improving the efficiency of surgery.

The image display method of the endoscope camera system, the endoscope camera host and the system provided by the application have at least the following beneficial effects: the endoscope camera system obtains an image signal of a shooting image through the camera and displays the shooting image in a display area of the display, the endoscope camera system detects the size of the endoscope image in the shooting image in real time, if the size of the endoscope image does not meet a preset magnification ratio, the endoscope camera system amplifies the shooting image to obtain a target image and displays the target image in the display area of the display, so that the endoscope image which is originally unsuitable in size can be automatically amplified to a proper display effect, thereby being convenient for doctors to operate and observe, and improving the efficiency of surgery.

The image display method of the endoscope camera system, the endoscope camera host and the system provided by the application have at least the following beneficial effects: the endoscope camera system obtains an image signal of a shooting image through a camera and displays the shooting image in a display area of a display; when the center of the endoscope image deviates from the center of the display area, in a first display mode, a user can zoom a picture displayed in the display area along the center of the display area in a manual zooming mode, if the center of the endoscope image displayed in the zoomed display area still deviates from the center of the display, the second display mode can be entered, and the user can enable an endoscope camera system in the second display mode to automatically intercept a target intercepting area from a shooting image according to the aspect ratio of the display area through a centering amplifying operation instruction, and the condition that the center of the target intercepting area coincides with the center of the endoscope image is met, and then the target intercepting area is amplified to the whole display area, and then the center of the target intercepting area coincides with the center of the display area; through the mode, the user can realize automatic amplification and centering of the endoscope image through the centering amplification operation instruction, the problem that the deviation of the axis between the endoscope and the camera is caused to be large in the offset of the shot image displayed on the display area is solved, and therefore operation and observation of a doctor are facilitated, and the operation efficiency is improved.

Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate and do not limit the application.

FIG. 1 is a schematic view of an endoscopic imaging system according to an embodiment of the present application;

FIG. 2 is an overall flow chart of an image display method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a process for obtaining a target image through a target capture area according to an embodiment of the present application;

FIG. 4 is a flow chart of a deskew process for each image frame according to one embodiment of the present application;

FIG. 5 is a flow chart providing a target image by capturing or extracting an endoscopic image according to one embodiment of the present application;

FIG. 6 is a schematic illustration of inscribed circles and circumscribed circles provided by one embodiment of the present application;

FIG. 7 is a schematic diagram of a process for obtaining a target image by extraction according to an embodiment of the present application;

FIG. 8 is a flow chart of a process for providing a deskew function in an on and off state according to one embodiment of the application;

FIG. 9 is a flow chart illustrating a method for setting a correction function according to an embodiment of the present application;

FIG. 10 is a flow chart of an amplification and deskew process according to one embodiment of the application;

FIG. 11 is a schematic diagram of a process for performing two processes of amplifying and rectifying according to an embodiment of the present application;

FIG. 12 is a flow chart of a process for providing a deskew amplifying function in an on and off state according to one embodiment of the present application;

FIG. 13 is a flow chart of determining a target cut-out region based on aspect ratio of a display region and the like, provided by an embodiment of the present application;

FIG. 14 is an overall flow chart of another image display method provided by an embodiment of the present application;

FIG. 15 is a schematic view of a process for obtaining a target image by extraction and magnification according to one embodiment of the present application;

FIG. 16 is a flow chart of a contrast magnification provided by one embodiment of the present application to perform an enlargement process;

FIG. 17 is a flow chart of a direct magnification process according to a preset magnification provided in one embodiment of the present application;

FIG. 18 is a flow chart for adjusting a target magnification according to a history frame provided by one embodiment of the application;

FIG. 19 is another flow chart for adjusting a target magnification according to a history frame provided by one embodiment of the application;

FIG. 20 is a flow chart of a process in which the magnification function provided by one embodiment of the present application is turned on and off;

FIG. 21 is a flow chart illustrating a manner of setting up an amplifying function according to an embodiment of the present application;

FIG. 22 is an overall flowchart of an image display method of an endoscopic imaging system provided by an embodiment of the present application;

FIG. 23 is a general flow chart of an image display method of an endoscopic imaging system provided in one embodiment of the present application;

FIG. 24 is a flow chart for determining a target cut-out area provided by one embodiment of the application;

FIG. 25 is a flowchart of processing a captured image to obtain a binary image according to one embodiment of the present application;

FIG. 26 is a flow chart for determining the center of an endoscopic image area and the range of an endoscopic image provided by one embodiment of the present application;

FIG. 27 is a flow chart for calculating coordinates of a center of an endoscopic image area provided by one embodiment of the present application;

FIG. 28 is a flow chart of a method for determining a target cut-out area cut-out based on a diameter of an endoscopic image, according to an embodiment of the present application;

FIG. 29 is a diagram showing the effect of an object cut area inscribed within an endoscopic image area according to one embodiment of the present application;

FIG. 30 is a diagram showing the effect of a target cut-out area circumscribed to an endoscopic image area provided by an embodiment of the present application;

FIG. 31 is a flow chart of a stepwise enlargement of the target cut area provided by one embodiment of the present application;

FIG. 32 is a flow chart for re-targeting an intercepting region provided by one embodiment of the present application;

FIG. 33 is a flow chart for re-centering a new target cut-out and scope of an endoscopic image provided by one embodiment of the present application;

FIG. 34 is a flow chart showing whether a new target cut-out area is cut out according to the original center and scope of an endoscopic image provided by an embodiment of the present application;

FIG. 35 is another overall flowchart of an image display method of an endoscopic imaging system provided by an embodiment of the present application;

FIG. 36 is an overall flow chart of an image display method provided by one example of the present application;

fig. 37 is a flowchart of a process for converting a current frame image into a gray scale and binarizing the same, provided by an example of the present application;

FIG. 38 is a flow chart for calculating the center coordinates and radius of a circle of an endoscopic image, respectively, provided by an example of the present application;

Fig. 39 is a flowchart illustrating determination of a target cut area after determining a relationship between a radius of a circle and a half height of a display area according to an example of the present application.

Detailed Description

The application will be further described with reference to the drawings and specific examples. The described embodiments should not be taken as limitations of the present application, and all other embodiments that would be obvious to one of ordinary skill in the art without making any inventive effort are intended to be within the scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with one another without conflict.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the application only and is not intended to be limiting of the application.

The endoscope camera system is mainly applied to clinic, is mainly used for observing lesions in a patient and providing images for clinical operations and diagnoses, and generally comprises a camera, an endoscope camera host and a display. The camera transmits shot image data to the endoscope camera host through the transmission cable, the endoscope camera host processes video information acquired by the camera, and the display displays the video information processed by the camera host.

Currently, in the practical use of an endoscope imaging system, a camera needs to be adapted to endoscopes with different calibers so as to meet the requirements of various operations. Therefore, the endoscope camera system is provided with an optical or electronic amplifying function, and can amplify the shooting image shot by the camera to a proper size so as to facilitate a doctor to observe the condition inside a patient. The above-described zoom-in function often requires a doctor or a nurse to perform a manual operation, and adjusts the photographed image to be appropriately presented on a designated display area of the display, which increases the operations of the doctor or the nurse on the endoscopic image pickup system, causing inconvenience to the progress of the operation. On the other hand, although the size of the photographed image can be scaled, in some cases, the axis of the endoscope and the axis of the camera image sensor may deviate, so that the doctor or nurse deviates from the center of the display area after performing the zoom-in/zoom-out adjustment operation on the endoscope image, thereby affecting the doctor's view of the endoscope image.

Based on the above, in order to avoid that an enlarged area is not in the center of a display area when a photographed image of an endoscope photographing system is enlarged, the embodiment of the application provides an image display method of the endoscope photographing system, an endoscope photographing host and a system, wherein after the endoscope photographing host acquires the photographed image sent by a camera, the photographed image is subjected to correction processing and/or enlargement processing in real time according to the position of the center of the endoscope image in the photographed image, so as to obtain a target image containing at least a part of the endoscope image, and the target image is displayed on a display of the endoscope photographing system, so that the center of the target image coincides with the center of the display area of the display; in this way, the endoscope camera can process the endoscope image in real time and rectify/amplify the endoscope image to the display area of the display, thereby realizing automatic centering/amplification, facilitating the operation and observation of doctors and improving the operation efficiency.

Referring to fig. 1, an endoscope camera system 1000 includes, but is not limited to, a light source host 10, a light guide 20, an endoscope 30, an optical bayonet 40, a camera head 50, a camera host 60, a display 70, and a communication cable 80.

One end of the light guide beam 20 is connected to the light source interface of the light source host 10, and the other end is connected to the light source interface of the endoscope 30, and the light guide beam 20 is used for providing a light source for the endoscope so as to illuminate a part to be observed of the observation body.

One end of the camera 50 is connected to an image signal input interface of the camera host 60 through a communication cable 80, and the other end of the camera 50 is used for being clamped on the optical bayonet 40 of the endoscope 30, and is used for acquiring an optical signal of the endoscope 30 for imaging a part 100 to be observed of an observation body, obtaining an image signal, and transmitting the acquired image signal to the camera host 60 for processing. The camera 50 comprises a handle, a plurality of keys are arranged on the handle, and a user can call the functions corresponding to the keys by pressing the keys. The site 100 to be observed of the observation body may be a human body.

The display 70 is connected to the camera host 60 through a communication cable 80 for displaying an image output from the camera host 60. The display 70 may include a liquid crystal display, an LED display, an OLED display, or the like, and may be a display on an electronic device such as a mobile phone, a tablet computer, or a personal computer connected to the camera host 60.

The screen of the display 70 includes one or more subdivided display areas, at least one of which is used for displaying the endoscopic image returned by the camera 50, and the display areas may be provided with a plurality of different display modes, and the display modes of the display areas may be switched by operating a function key on the handle of the camera 50, or the display modes of the display areas may be switched by operating a foot pedal provided by the endoscopic imaging system 1000, or the endoscopic imaging system 1000 may be triggered to perform switching of the display modes of the display areas by a voice command or the like, and so on.

It will be appreciated by those skilled in the art that fig. 1 is merely an example of an endoscopic imaging system 1000 and is not limiting of the endoscopic imaging system 1000, and that the endoscopic imaging system 1000 may include more or fewer components than shown in fig. 1, or may combine certain components, or different components, e.g., the endoscopic imaging system 1000 may further include dilators, smoke control devices, input-output devices, network access devices, etc.

Referring to fig. 2, in one aspect, the present application provides an image display method of an endoscopic imaging system, the endoscopic imaging system including a display area, the method including, but not limited to, the steps of:

In step S101, the endoscopic imaging system acquires a captured image transmitted by the camera, wherein the captured image includes an endoscopic image.

The camera is provided with a lens and an image sensor (such as CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor), and a CCD is taken as an example for follow-up defaults), the camera is provided with an illumination element (such as a light guide window, an LED lamp and the like), light emitted by the illumination element irradiates the body of a patient, reflected light irradiates the image sensor through the lens, and the camera converts the reflected light into an electric signal to obtain a shooting image; the camera transmits the generated captured image to the endoscope image capturing system as an image signal, so that the captured image can be displayed on a designated display area of the display. The endoscope image corresponds to an imaging range in which reflected light irradiates the image sensor after passing through the lens, and the shot image refers to the whole imaging range of the image sensor and comprises the endoscope image; since the shape of the endoscope and the shape of the photosensitive region of the image sensor tend to be different, the region of the captured image from which the endoscope image is divided tends to be dark (i.e., the background region where there is no effective image), and at this time, the peripheral region of the endoscope image is dark in the captured image displayed on the display region. In addition, the matching degree of the lens and the image sensor in endoscopes with different types and different calibers is different, so that the imaging axis and the imaging axis of the image sensor are deviated, the center of the endoscope image is not overlapped with the center of the shot image on the display area, a doctor often needs to focus on the center and the position near the center of the endoscope image, and the deviation of the position of the endoscope image obviously brings trouble to the doctor for observing and adjusting the endoscope image.

In step S102, the endoscope image capturing system detects the position of the center of the endoscope image in the captured image in real time, and if the center of the endoscope image is not in the center area of the captured image, the endoscope image capturing system performs correction processing on the captured image to obtain a target image, wherein the target image includes at least a part of the endoscope image.

The endoscope camera system receives the shooting image returned by the camera, determines whether the center of the endoscope image is at the center of the shooting image according to the shooting image, if the center of the endoscope image deviates from the center area of the shooting image, the shooting image needs to be subjected to deviation correction processing to obtain a target image, the target image comprises the endoscope image with the adjusted center position, and at the moment, the center of the target image is overlapped with the center of the endoscope image. Since there are various ways of matching the lens of the endoscope with the optical lens of the camera, the imaging size of the endoscope image on the photographed image is also different, so that the adjusted endoscope image is not necessarily located in the target image, and there is a possibility that a part of the endoscope image is located outside the target image, but a part of the endoscope image is necessarily located in the target image, for example, a center part is located in the target image. This correction process may not involve the magnification of the endoscopic image, and the center position of the endoscopic image may be directly adjusted to generate the target image. There are various ways of determining the center position of the endoscope image, for example, the pixel point corresponding to the edge of the endoscope image may be determined by a pixel edge search method or the like, and then the pixel point of the center of the endoscope image is determined according to the pixel point of the edge. These determination modes will be described in the following in terms of a plurality of embodiments, and are not particularly limited herein.

In step S103, the endoscope image capturing system outputs a display signal to the display according to the image information of the target image so that the display area displays the target image, wherein the center of the endoscope image displayed in the display area coincides with the center of the display area.

Referring to fig. 3 (a denotes a target cut-out region, B denotes an endoscope image region, C denotes a region where an image is captured, a' denotes a display region of a display), a target image is obtained after the foregoing correction processing, and the endoscope image capturing system outputs a display signal to the display so that the display displays the target image in the display region, wherein the center of the display region coincides with the center of the target image, that is, the center of the endoscope image coincides with the center of the display region. Therefore, the endoscope image pickup system detects the center of the endoscope image in real time according to the shooting image returned by the endoscope camera and corrects the endoscope image when the center of the endoscope image deviates from the center of the shooting image, so that the endoscope image can be centrally displayed in a display area when the target image obtained after correction is displayed on the display. It will be appreciated that the matching of the target image with the display area can be adapted to display the target image in the centre of the display area, e.g. the aspect ratio of the target image is the same as the aspect ratio of the display area, so that when the endoscope image is rectified the resulting target image is virtually the whole of what can be displayed on the display area, wherein the target image can be scaled before being displayed in the display area to fit the width and height of the display area, or the target image can be displayed directly irrespective of the width and height of the display area.

The display area of the display has several forms, the first is that the display area is equal to the effective display area of the whole display panel of the display, at this time, the target image is directly displayed in the center of the screen of the display, the center of the target image coincides with the center of the display, and the aspect ratio of the target image is the same as the aspect ratio of the display; the second is that the effective display area of the whole display panel of the display comprises a display area for displaying the target image and other display areas, such as a user interface like a menu bar, the total area of the display area for displaying the target image and the other display areas is equal to the effective display area of the display panel, the other display areas occupy a part of area, the center of the display area for displaying the target image is not overlapped with the center of the display, and the target image is displayed according to the center of the display area; the third case is that the effective display area of the entire display panel of the display includes a display area for displaying the target image and other display areas, but the other display areas are superimposed on the display area for displaying the target image, when the display area for displaying the target image is equal to the effective display area of the display panel, the center of the target image coincides with the center of the display, and the aspect ratio of the target image is the same as the aspect ratio of the display.

The endoscope image is rectified through the steps and the obtained target image is displayed on the display, so that the endoscope image which is originally eccentrically displayed can be displayed in the center of the display area, a doctor can observe the condition in a patient conveniently by observing the center and the area near the center of the endoscope image displayed by the display in operation, the picture position of the endoscope image does not need to be frequently regulated, and the operation efficiency is improved.

It should be noted that, in the process of capturing the endoscope image through the target capturing area, it is possible to capture an area outside the photosensitive area outside the image sensor (for example, in the case of fig. 3, the upper left and the left are all areas outside the captured image), so that the endoscope image capturing system fills in dark portions in these areas, and the dark portions may be the same as the ground color except for the endoscope image in the captured image or be pure black, or no display signal is output in these areas, so that these areas exhibit default display colors (for example, pure black) of the display device, so that the display effect of the display area is better.

Referring to fig. 4, in some embodiments, the detecting, in real time, the position of the center of the endoscopic image in the captured image by the endoscopic image capturing system in the step S102, and if the center of the endoscopic image does not coincide with the center of the captured image, performing, by the endoscopic image capturing system, correction processing on the captured image to obtain a target image includes:

Step S201, the endoscope image pickup system detects the center position of an endoscope image in each frame of photographed image and determines a first position of the center of the endoscope image in the photographed image;

step S202, comparing the first position with the center of the shot image by the endoscope shooting system, and determining that the shot image of the current frame has abnormal position if the first position is not coincident with the center of the shot image or the distance between the first position and the center of the shot image is greater than a first distance threshold;

in step S203, the endoscope image capturing system performs correction processing on the captured image in which the positional abnormality occurs, to obtain a target image in which the center of the endoscope image coincides with the center of the target image.

The endoscope camera returns the shot images in the mode of image frames, and the endoscope camera system judges the center position of the endoscope image in each frame of shot image according to the image frames. For a frame of shooting image, searching the center position of the endoscope image in the shooting image through a preset algorithm, determining the center position as a first position in the shooting image, for example, searching the edge pixel point of the endoscope image with a circular outline based on the pixel coordinates, and calculating the center coordinates of the endoscope image based on the coordinates of the edge pixel point, wherein the center coordinates represent the coordinates of the center of the endoscope image in the shooting image, namely the first position. And then judging whether the first position is not coincident with the center of the shot image or whether the phase difference distance is larger than a first distance threshold value, and then determining whether to perform deviation correction processing on the shot image according to the judging structure. The center of the shot image is basically known, because the camera and the endoscope are not frequently replaced in the actual working process of the endoscope image pickup system, when the camera and the endoscope are fixed, the imaging area of the camera can be determined by the endoscope image pickup system in the initial stage of working, namely, the height and the width of the shot image can be known, and the center of the shot image can be calculated.

In step S202, it is determined whether the correction is required to have two different judgment conditions. Specifically, if the first position is not coincident with the center of the captured image, it is determined whether the pixel position (for example, represented by coordinates) of the first position is coincident with the pixel position (for example, represented by coordinates) of the center of the captured image, if the first position is coincident with the pixel position, it is considered that correction of the captured image is not required, and the captured image is directly displayed in the display area in the current state, so that the endoscope image can be centrally displayed in the display area; if the two images are not overlapped, the shot image is considered to be required to be rectified, a target image is obtained after rectification, and the target image is displayed in the display area, so that the endoscope image can be centrally displayed in the display area. If the distance between the first position and the center of the captured image is determined to be greater than the first distance threshold, a distance value (e.g., a pixel distance) between the pixel point position at the first position and the pixel point position at the center of the captured image is calculated, and the distance value is compared with the first distance threshold, and if the distance value is smaller than the first distance threshold, the endoscope image is considered to be substantially near the center of the captured image, so that the endoscope image does not deviate significantly from being near the center of the display area when the captured image is displayed in the display area; if the distance value is larger than the first distance threshold value, the endoscope image is considered to be far away from the center of the shot image, correction is needed to be carried out on the shot image, a target image is obtained after correction, and the target image is displayed in the display area so that the endoscope image can be centrally displayed in the display area.

Referring to fig. 5, in some embodiments, the step S102 of performing, by the endoscope camera system, correction processing on the captured image and obtaining a target image includes:

step S301, an endoscope image pickup system determines a target interception area containing at least a part of an endoscope image from the photographed image, and the center of the target interception area coincides with the center of the endoscope image;

step S302, the shot image is intercepted according to the target intercepting area to obtain a target image.

One specific implementation manner of the correction processing is performed according to the above step S301 and step S302. Under the condition that correction processing is required to be carried out on the shot image, the shot image is intercepted according to a target intercepting area, and a target image is obtained through interception. The size of the target intercepting region meets a certain condition, and the center of the target intercepting region is overlapped with the center of the endoscope image before intercepting, so that the intercepted target image naturally enables the endoscope image to be positioned at the center of the target image, namely, the endoscope image is centrally displayed in the display region when the target image is displayed in the display region. Wherein the target cut-out area meets at least one of the following conditions:

The aspect ratio of the target cut-out region is the same as that of the photographed image or the display region;

the width and the length of the target interception area are the same as those of the shot image or the display area;

condition 3, the whole endoscope image is positioned in the target interception area, or one part of the endoscope image is positioned in the target interception area, and the other part of the endoscope image is positioned outside the target interception area;

and 4, the endoscope image is an inscribed circle or an circumscribed circle of the target interception area.

With condition 1, if the aspect ratio of the target cut-out area is determined according to the aspect ratio of the captured image, so that the aspect ratio of the target cut-out area after cutting out the captured image is the same as the aspect ratio of the captured image, the endoscope image does not need to be subjected to stretch deformation, and a circular operation field can be maintained (even in the case where the endoscope image needs to be enlarged), which is convenient for a doctor to observe.

For the condition 2, the condition is similar to the condition 1, but the condition 2 directly sets the width and the length of the target interception area to be the same as those of the photographed image, which is equivalent to the size of the target interception area and the photographed image to be equal, and then the endoscope image in the intercepted target image is restored by 1:1; similarly, if the width and length of the target capturing area are directly set to be the same as those of the display area, which is equivalent to the size of the target capturing area and the display area, the endoscope image in the captured target image can be displayed on the display in a 1:1 size.

With respect to condition 3, since the size of the endoscopic image is affected by the cooperation between the camera and the endoscope, there are two cases in which the endoscopic image is entirely located in the target interception area, so that the intercepted target image can contain a complete endoscopic image, and in which the size of the endoscopic image is large beyond the range of the target interception area, and the intercepted target image can only contain a part of the endoscopic image.

In condition 4, referring to fig. 6 (a indicates a target area and B indicates an endoscopic image area), in the case where the endoscopic image has a circular outline, an endoscopic image is inscribed in the target area or circumscribed with the endoscopic image so that the endoscopic image can be displayed as large as possible. In the case of an endo-endoscopic image, the entire endoscopic image is contained in the target cut-out region, and the endoscopic image is located at the center position of the target cut-out region; in the case of circumscribing an endoscopic image, the target cut-out region includes only a part of the endoscopic image, and the part is displayed centrally, and the part beyond the target cut-out region cannot be displayed in the display region.

The above conditions may be used in combination of one or more, and the combination is not described here alone, and the target cut-out area may be set according to the above conditions and actual needs.

In some embodiments, in the process of detecting the position of the center of the endoscope image in the captured image in real time by the endoscope image capturing system, if the center of the endoscope image is located in the center area of the captured image, the endoscope image capturing system outputs a display signal to the display according to the image information of the captured image so that the display area displays the captured image.

For the case where no correction is required, i.e., the center of the endoscopic image is located in the center region of the captured image (e.g., the condition that no correction is required in step 202 described above is satisfied), the endoscopic image capturing system may not process the captured image, but directly display the captured image on the display region via the display signal, where the endoscopic image is substantially centered or already centered.

Referring to fig. 5 and 7, in some embodiments, the performing, by the endoscope camera system, correction processing on the captured image and obtaining the target image in the foregoing step S102 includes:

step S303, extracting an endoscope image from the shot image, and generating a target image according to the endoscope image, wherein the center of the endoscope image is overlapped with the center of the target image.

In the step S303 of the present embodiment, unlike the above-described method of obtaining the target image by capturing the target capturing area, the endoscope image is extracted from the captured image, the target image is generated according to the extracted endoscope image, the generated target image coincides with the center of the endoscope image, and the display area displays the target image as the center of the endoscope image. The target image in this embodiment may be displayed directly in the display area, or may be enlarged to the width and height of the display area and then displayed in the display area. The endoscope image and the target image are matched in various modes, and the corresponding display effect is achieved under different matching modes. Specifically, the endoscopic image in the target image satisfies one of the following conditions:

The endoscope image is an inscribed circle of the boundary of the target image;

alternatively, a partial image in the endoscope image is located within the target image;

alternatively, the first display ratio of the endoscopic image in the target image is larger than the second display ratio of the endoscopic image in the photographed image, wherein the first display ratio is a display area ratio of the endoscopic image in the target image, and the second display ratio is a display area ratio of the endoscopic image in the photographed image.

The aforementioned step S303 directly extracts an endoscopic image, which may be an inscribed circle of the target image, so that the entire endoscopic image is contained in the target image and is centrally disposed in the target image, and the entire endoscopic image is displayed as a centrally disposed endoscopic image when the target image is displayed in the display area; or the size of the endoscope image is larger and exceeds the size of the target image, for example, the endoscope image is a circumcircle of the target image or the diameter of the endoscope image is larger than the height of the target image, so that only a part of the endoscope image can be displayed in the target image, and the endoscope image is displayed as a part of the endoscope image in the middle when the target image is displayed in the display area; or the endoscope image is enlarged and displayed, that is, the proportion of the endoscope image in the target image (the first display proportion) is larger than the proportion of the endoscope image in the shooting image (the second display proportion), so that the endoscope image is extracted from the shooting image, then the target image is enlarged according to the second display proportion to form the target image, and the target image is displayed in the display area. The first display proportion and the second display proportion are display area occupation ratios, and the display area occupation ratios can be determined through statistical calculation according to a pixel point mode, and are not limited herein.

Referring to fig. 8, in some embodiments, the detecting, in real time, the position of the center of the endoscopic image in the captured image by the endoscopic image capturing system in step S102 includes:

step S401, detecting the starting state of the deviation correcting function by the endoscope camera system;

step S402, if the deviation correcting function is in an enabling state, the endoscope image pickup system detects the position of the center of the endoscope image in the photographed image in real time;

in step S403, if the deviation correcting function is turned off, the endoscope image capturing system outputs a display signal to the display according to the image information of the captured image, so that the captured image is displayed in the display area.

The endoscope image pickup system provides a function switch for providing a doctor with a function of detecting the center of an endoscope image in real time and performing correction, which is called a correction function in this embodiment. When the endoscope camera system detects that the correction function is in an enabled state, executing the steps S102 to S103, detecting the center of the endoscope image in real time and determining whether to correct the correction according to the position of the center, wherein the determination of the center position and the correction process can refer to the description of the steps and are not repeated once; when the endoscope camera system detects that the correction function is in the off state, step S102 and step S103 do not need to be executed, after the shot image returned by the camera is received, the shot image is output to the display, and the endoscope image which is not subjected to correction processing is directly displayed.

Opening or closing the above-described corrective function may be accomplished in a variety of ways. Specifically, referring to fig. 9, the state of the rectification function is set by one of the following steps:

step S501, the endoscope camera system obtains a setting instruction input by a user in a man-machine interaction interface of the endoscope camera system, and sets the state of a deviation correcting function according to the setting instruction;

or,

step S502, the endoscope camera system obtains a key instruction of an operation key of a camera by a user, and sets a state of a deviation correcting function according to the setting instruction.

The endoscope camera system provides a man-machine interaction interface for judging whether the deviation correcting function is started, for example, the man-machine interaction interface provides a specific function key (can be an entity key and/or a virtual key), and a doctor can generate a setting instruction for starting or stopping the deviation correcting function by clicking the function key on the man-machine interaction interface. In addition, the handle of the camera is provided with an operation key, and a doctor can generate a key instruction to send to the endoscope camera system by clicking the operation key on the handle, so that the deviation correcting function is opened or closed. In addition, the doctor can trigger the endoscope image capturing system to open or close the deviation correcting function by means of voice instructions and the like, and the description is not given here.

Referring to fig. 10, in some embodiments, the foregoing detecting, in real time, the position of the center of the endoscopic image in the captured image by the endoscopic image capturing system in step S102, and if the center of the endoscopic image is not in the center area of the captured image, performing the correction processing on the captured image by the endoscopic image capturing system and obtaining the target image includes:

step S601, an endoscope image capturing system detects in real time the position of the center of an endoscope image in a captured image and the size of the endoscope image;

in step S602, if the center of the endoscopic image is not in the center area of the captured image and the size of the endoscopic image does not meet the preset magnification, the endoscopic image capturing system amplifies and rectifies the captured image to obtain the target image.

In the process of correcting deviation, the shot image can be amplified, so that the endoscope image can be amplified for a doctor to observe. Specifically, in the process of detecting the center of the endoscope image in real time, the center position of the endoscope image and the size of the endoscope image are confirmed, and if the magnification of the endoscope image after optical magnification by the camera is still insufficient, digital image processing is required to be performed to enlarge the photographed image so that the endoscope image in the target image obtained after correction of the photographed image is sufficiently large. In the digital image processing process, two processing processes of rectification and amplification are involved, namely, the amplification can be performed first and then, rectification can be performed first, or the amplification can be performed first and then, rectification can be performed, referring to fig. 11 (a represents a target interception area, B represents an endoscope image area, C represents an area where an image is captured, and a' represents a display area of a display), specifically, the process includes one of the following:

The endoscope shooting system amplifies a shooting image according to a preset magnification, and then a target intercepting area containing at least a part of the endoscope image is determined from the shooting image, and the center of the target intercepting area coincides with the center of the endoscope image; intercepting the photographed image according to the target intercepting region to obtain a target image;

or the endoscope camera system sets the size of a target interception area according to a preset magnification, and determines a target interception area containing at least a part of the endoscope image from the photographed image, wherein the area of the target interception area is smaller than that of the photographed image, and the center of the endoscope image is coincident with the center of the target interception area; and intercepting the photographed image according to the target intercepting region to obtain a target image.

The first procedure is to perform the above two procedures, and then perform the correction after performing the amplification, as in the upper part of fig. 11, amplifying the photographed image according to the preset amplification factor, and then intercepting the photographed image according to the target intercepting region, so that at least a part of the endoscope image is intercepted into the target intercepting region, and the center of the target intercepting region coincides with the center of the endoscope image, thereby obtaining the target image centrally displayed by the endoscope image. The second process is to perform the correction first and then perform the amplification, as in the lower part of fig. 11, and set a target interception area according to a preset amplification factor during the correction processing, and intercept the photographed image in the amplified target interception area, so that at least a part of the endoscopic image is intercepted in the target interception area, and the center of the target interception area coincides with the center of the endoscopic image, thereby obtaining a target image centrally displayed by the endoscopic image. In the two processes, the same processing effect can be finally obtained for the same preset magnification and the same frame of shot image.

In the foregoing deviation correcting process, the preset magnification may be set according to different manners, for example, the preset magnification is set by one of the following steps:

mode 1, setting a preset magnification according to a magnification instruction input by a user;

mode 2, determining a preset magnification according to an endoscopic image display size parameter set by a user;

mode 3, setting preset magnification according to the type of the endoscope adopted by the current camera;

mode 4, setting a preset magnification according to the type of the current camera and the type of the endoscope.

Mode 1 is that a user directly sets a magnification, and the user (e.g., doctor or other person) sets a specific value in an endoscope camera system, so that a magnification instruction is generated to assign a preset magnification of the endoscope camera system to the specific value set by the user; mode 2 is that a user inputs display size parameters to determine, wherein the display size parameters include, but are not limited to, the width and the height of a target image, the area occupation ratio of the target image in a display area and the like, for example, the width and the height of the target image defined by the user are equal to the width and the height of the display area, and then the endoscope camera system automatically generates a preset magnification according to the setting of the user and the size of the current shot image; the mode 3 is that the preset magnification is set according to the type of the endoscope installed by the camera, the type of the endoscope is divided by caliber, a user can set the caliber of the endoscope adopted by the current camera by the endoscope camera system, and the endoscope camera system automatically sets the preset magnification according to the caliber; for some endoscopes with smaller caliber (for example, 4mm caliber), in order to display the endoscope image in the display area as large as possible, a larger preset magnification ratio can be set, and for some endoscopes with larger caliber (for example, 10mm caliber), a smaller preset magnification ratio can be set, so that the display content of the display area is prevented from being blurred due to too much magnification of the endoscope image; mode 4 is to determine the preset magnification together according to the type of the camera and the type of the endoscope, and likewise, the type of the camera may be related to the size of the photosensitive element of the camera, the type of the endoscope may be related to the caliber of the endoscope, the size of the photographed image may be related to the size of the photosensitive element, and the size of the endoscopic image may be related to the size of the light entering the endoscope projected on the photosensitive element, so setting the preset magnification in combination with the type of the camera and the type of the endoscope, which may be set by the user in the endoscopic camera system, may be determined by the product id of the endoscope camera automatically recognizing the camera, and the like.

For the various modes of setting the preset magnification, the endoscope image capturing system may have a built-in magnification upper limit (for example, set to 2 times of the original endoscope image, that is, the magnification upper limit is 2), compare the magnification upper limit when the endoscope image capturing system receives the preset magnification set externally, and determine that the preset magnification is smaller than the magnification upper limit, and if the preset magnification is larger than the magnification upper limit, limit the preset magnification to 2 and send a relevant alert to the user.

In some embodiments, the image display method according to the embodiment of the present application further includes:

in the process that the endoscope image pickup system detects the position of the center of the endoscope image in the shot image and the size of the endoscope image in real time, the magnification setting information input by a user is acquired, the preset magnification is updated in real time according to the magnification setting information, and the shot image is amplified and rectified according to the updated preset magnification to obtain a target image.

In the process of correction and amplification processing, a user can adjust the amplification factor in real time, for example, the user sets 1.1 amplification factor to amplify an endoscope image, the endoscope camera system performs correction and amplification processing according to the 1.1 amplification factor, and displays a target image in a display area, if the user finds that the amplification factor is insufficient by observing the display area of the display, the user can adjust the correction and amplification processing in real time by inputting amplification factor setting information, for example, 1.3 amplification factor, and the endoscope camera system updates the target image displayed in the display area in real time based on the 1.3 amplification factor, so that the user can confirm whether the current amplification factor is proper. It is understood that although the magnification is set by the user, normal execution of the correction function is not affected, and after the magnification is changed, the correction function performs the amplification and correction processing according to the magnification to obtain the target image.

Referring to fig. 12, in some embodiments, the detecting, in real time, the position of the center of the endoscopic image in the captured image and the size of the endoscopic image by the endoscopic image capturing system in the aforementioned step S601 includes:

step S701, detecting the starting state of the deviation correcting and amplifying function by the endoscope camera system;

step S702, if the deviation correcting and amplifying function is in an enabling state, the endoscope image pickup system detects the position of the center of the endoscope image in the photographed image and the size of the endoscope image in real time;

in step S703, if the correction and amplification function is turned off, the endoscope image capturing system outputs a display signal to the display according to the image information of the captured image, so that the captured image is displayed in the display area, or the endoscope image capturing system amplifies the captured image according to the preset magnification and outputs a display signal to the display, so that the amplified captured image is displayed in the display area.

Different from the correction function, the endoscope camera system also provides a correction and amplification function, and a user can start the correction and amplification function, so that the endoscope camera system corrects and amplifies a shot image, and a corrected and amplified target image is obtained and displayed in a display area. Under the condition that the deviation correcting and amplifying functions are opened, the endoscope image pickup system detects the position of the center of the endoscope image in the photographed image and the size of the endoscope image in real time according to the step S601; under the condition that the correction function is closed, two processing modes can be adopted, one is that the endoscope camera system directly displays the shot image in the display area of the display, correction and amplification are not carried out on the shot image at all, the other is that the endoscope camera system amplifies the shot image according to the preset amplification factor, and then displays the shot image in the display area of the display, wherein the preset amplification factor can be the amplification factor set by a user before entering the correction amplification function, for example, the amplification factor is 1 in the initial state of the endoscope camera system, after the user adjusts the amplification factor to be 1.2 times, the correction amplification function is started again, the correction amplification function provides the amplification factor of 1.8 times and the endoscope image is centrally displayed, the user can adjust the amplification factor to be 1.4 times as required, then the user exits the correction amplification function, and the endoscope camera system keeps the preset amplification factor set in the correction amplification function, namely 1.4 times, and is used for amplifying the shot image.

Referring to fig. 13, in some embodiments, the determining, by the endoscope image capturing system, a target cut-out region including at least a part of the endoscope image from the captured image in the foregoing step S301 includes:

step S801, an endoscope image capturing system determines an endoscope image area of an endoscope image from captured images;

step S802, an endoscope image pickup system determines the center of an endoscope image area and the scope of the endoscope image according to the endoscope image area;

in step S803, the endoscope camera system determines a target cut-out region according to the aspect ratio of the display region, the center of the endoscope image region, and the scope of the endoscope image, wherein the center of the target cut-out region coincides with the center of the endoscope image region, the aspect ratio of the target cut-out region is the same as the aspect ratio of the display region, and the target cut-out region contains at least a portion of the endoscope image.

The aspect ratio of the target cut-out region may be determined by the aspect ratio of the display region such that the aspect ratio of the target cut-out region is the same as the aspect ratio of the display region, such that the target image cut-out by the target cut-out region can exactly fill the display region, and the target image is maximally displayed. The center of the target intercepting region coincides with the center of the endoscope image, and the width and the height of the target intercepting region can be determined according to the size of the endoscope image region. Specifically, when the endoscope image area does not exceed the boundary of the captured image, the target cut-out area may cut out the endoscope image in an inscribed circle or a circumscribed circle; when the endoscope image area exceeds the boundary of the photographed image, the center of the target cut-out area is overlapped with the center of the endoscope image, and then the endoscope image is cut out in an area with a proper size. In this process, the width and height of the target cut-out region can be adjusted according to the edge of the endoscopic image, so that the endoscopic image pickup system first determines the endoscopic image region, thereby determining the center and edge of the endoscopic image, and thus the size of the target cut-out region can be adjusted based on the edge of the endoscopic image.

In some embodiments, the endoscope image capturing system determines an endoscope image area of the endoscope image from the captured image in the aforementioned step S801, including one of:

the endoscope image pickup system converts the photographed image into a gray image, and performs threshold segmentation processing on the gray image according to a preset threshold value to obtain a binary image, wherein the area contained in the binary image is an endoscope image area;

or the endoscope image pickup system performs expected distribution calculation on the coordinates of each pixel point of the endoscope image in the endoscope image area to obtain the coordinates of the center of the endoscope image area, and performs variance distribution calculation on the coordinates of each pixel point of the endoscope image in the endoscope image area to obtain the scope of the endoscope image area.

The above-mentioned two ways of determining the area of the endoscopic image are given, the first way is to convert the photographed image into a gray-scale image, and then separate the gray-scale image into two parts of the endoscopic image and the non-endoscopic image (the photographed image is not sensitized and appears dark), generally, the area brightness of the endoscopic image is higher than the area brightness of the non-endoscopic image, so after the binarized image processing, the area of the endoscopic image, that is, the area included in the binary image, can be clearly determined. The second is to perform statistical computation on the pixel points, calculate the center coordinates by using the expected distribution of coordinates, calculate the coordinates of the center of the endoscope image area by calculating the coordinates of the pixel points corresponding to the endoscope image, calculate the edge coordinates by using the variance distribution, and calculate the pixel points at the edge of the endoscope image area by calculating the coordinates of the pixel points corresponding to the endoscope image, thereby determining the scope of the endoscope image based on the pixel points. Other ways than the two calculation methods described above may be used, which are not illustrated here.

On the other hand, referring to fig. 14, an embodiment of the present application further provides an image display method of an endoscopic image pickup system, the endoscopic image pickup system including a display area, the method including, but not limited to, the steps of:

in step S901, an endoscopic imaging system acquires a captured image transmitted by a camera, wherein the captured image includes an endoscopic image.

The camera is provided with a lens and an image sensor (such as CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor), and a CCD is taken as an example for follow-up defaults), the camera is provided with an illumination element (such as a light guide window, an LED lamp and the like), light emitted by the illumination element irradiates the body of a patient, reflected light irradiates the image sensor through the lens, and the camera converts the reflected light into an electric signal to obtain a shooting image; the camera transmits the generated captured image to the endoscope image capturing system as an image signal, so that the captured image can be displayed on a designated display area of the display. The endoscope image corresponds to an imaging range in which reflected light irradiates the image sensor after passing through the lens, and the shot image refers to the whole imaging range of the image sensor and comprises the endoscope image; since the shape of the endoscope and the shape of the photosensitive region of the image sensor tend to be different, the region of the captured image from which the endoscope image is divided tends to be dark (i.e., the background region where there is no effective image), and at this time, the peripheral region of the endoscope image is dark in the captured image displayed on the display region. In addition, the matching degree of the lens and the image sensor in the endoscopes with different types and different calibers is different, so that the imaging sizes on the image sensor are different, the imaging sizes of the endoscopes with partial calibers on the image sensor with larger size are smaller, and the endoscope images can not be seen clearly when being directly displayed on a display, and at the moment, a doctor needs to manually amplify the endoscope images, thereby interrupting the operation process and bringing inconvenience to the doctor.

In step S902, the endoscope image capturing system detects the size of the endoscope image in the captured image in real time, and if the size of the endoscope image does not meet the preset magnification, the endoscope image capturing system performs the magnification processing on the captured image to obtain a target image, where the target image includes at least a part of the endoscope image.

The endoscope camera system receives the shooting image returned by the camera, determines the size of the endoscope image in the shooting image according to the shooting image, if the size of the endoscope image is determined to not meet the preset magnification, the shooting image is considered to be required to be amplified so that the endoscope image is also amplified, the amplified shooting image is processed to obtain a target image, the target image comprises the amplified endoscope image, at the moment, the target image is displayed in a display area, and a doctor can see the amplified endoscope image through a display.

In step S903, the endoscope image pickup system outputs a display signal to the display so that the display area displays the target image, based on the image information of the target image.

Referring to fig. 15 (a denotes a target cut-out region, B denotes an endoscopic image region, C denotes a region where an image is captured, a' denotes a display region of a display), a target image is obtained by the above-described enlargement processing, and the endoscopic image pickup system outputs a display signal to the display so that the display displays the target image in the display region, in which the endoscopic image is an enlarged image. Therefore, the endoscope image pickup system detects the size of the endoscope image in the photographed image in real time according to the photographed image returned by the endoscope camera, and amplifies the photographed image under the condition that the preset magnification is not met, so that the endoscope image is also amplified, and the obtained target image can more clearly display the content of the endoscope image in the display area. It will be appreciated that the matching of the target image to the display area can be adapted to display the target image in the display area, e.g. the aspect ratio of the target image is the same as the aspect ratio of the display area, such that when the endoscope image is enlarged, the resulting target image is virtually the entire content that can be displayed on the display area, wherein the target image can be displayed after correction before being displayed in the display area, such that the endoscope image is displayed centrally in the target image, or the target image can be displayed directly irrespective of whether the endoscope image is centered.

By amplifying the endoscope image and displaying the obtained target image on the display through the steps, a doctor can observe the condition in the patient conveniently by observing the amplified endoscope image displayed by the display in the operation, the picture size of the endoscope image is not required to be frequently regulated, and the operation efficiency is improved.

It should be noted that, in the process of capturing the endoscope image through the target capturing area, it is possible to capture an area outside the sightseeing area outside the image sensor (for example, in the case of fig. 3, the upper left and the left side are all areas outside the captured image), so that the endoscope image capturing system fills the dark portions in these areas, and the dark portions may be the same as the ground color except for the endoscope image in the captured image or be pure black, or no display signal is output in these areas, so that these areas exhibit default display colors (for example, pure black) of the display device, so that the display effect of the display area is better.

Referring to fig. 16, in some embodiments, the step S902 of detecting, in real time, the size of the endoscopic image in the captured image by the endoscopic image capturing system, and if the size of the endoscopic image does not satisfy the preset magnification, performing the magnification processing on the captured image by the endoscopic image capturing system to obtain the target image includes:

Step S1001, the endoscope image capturing system detects the boundary of each frame of the endoscope image, and determines the first magnification of the current endoscope image display;

step S1002, comparing the first magnification with a preset magnification, and if the first magnification is different from the preset magnification, or the difference between the first magnification and the preset magnification is greater than a first magnification threshold, performing an amplification process on the captured image by the endoscope camera system, so as to obtain a target image.

The endoscope image pickup system detects each frame of photographed image in real time, determines the area/boundary of the endoscope image in the photographed image, and when the boundary of the endoscope image is determined, can determine the occupation ratio of the area of the endoscope image on the photographed image, thereby determining the first magnification of the endoscope image in the current frame of image. Comparing the first magnification with a preset magnification, and determining whether a method for shooting the image is needed according to a comparison result. There may be two different processing methods, one is that when the first magnification is different from the preset magnification, the first magnification is amplified according to the preset magnification, which includes a case that the first magnification is greater than the preset magnification or less than the preset magnification; the other is to compare the difference between the first magnification and the preset magnification, the difference is compared with the first magnification threshold, if the difference is larger than the first magnification threshold, the magnification is needed according to the preset magnification, the situation is suitable for the situation that in the actual working process, the camera continuously returns the shot images and the front and back shot images slightly change, the problem that the shot images of each frame are amplified or reduced differently to cause frame jumping can be avoided through the processing method, and at the moment, the endoscope images after the display area is amplified are not easy to jump.

In some embodiments, the above-mentioned step S902 of the endoscope image capturing system performs the magnification processing on the captured image and obtains the target image, including one of the following:

the endoscope camera system amplifies a shooting image according to a preset magnification, determines a target intercepting region containing at least a part of the endoscope image from the shooting image, and intercepts the shooting image according to the target intercepting region to obtain a target image;

the endoscope camera system sets the size of a target interception area according to a preset magnification, determines a target interception area containing at least a part of endoscope images from the photographed images, intercepts the photographed images according to the target interception area to obtain target images, wherein the area of the target interception area is smaller than that of the photographed images.

The first process is to amplify the photographed image according to a preset amplification factor, and then intercept the photographed image according to a target intercept area, so that at least a part of the endoscopic image is intercepted into the target intercept area, thereby obtaining a target image for amplifying and displaying the endoscopic image; when the target interception area is matched with the endoscope image, the center of the target interception area is enabled to coincide with the center of the endoscope image, and therefore the display of the endoscope image in the middle of the display area is enlarged. The second process is to firstly execute interception and then execute amplification, firstly set a target interception area according to a preset amplification factor, intercept a shooting image according to the amplified target interception area, intercept at least part of an endoscope image into the target interception area, and then obtain a target image amplified and displayed by the endoscope image. In the two processes, the same processing effect can be finally obtained for the same preset magnification and the same frame of shot image.

Wherein the target interception area satisfies at least one of the following conditions:

in the condition 6, the whole endoscope image is positioned in the target interception area, or one part of the endoscope image is positioned in the target interception area, and the other part of the endoscope image is positioned outside the target interception area;

and 7, the endoscope image is an inscribed circle or an circumscribed circle of the target interception area.

For condition 5, if the aspect ratio of the target cut-out area is determined according to the aspect ratio of the photographed image, so that the aspect ratio of the target cut-out area after cutting out the photographed image is the same as the aspect ratio of the photographed image, the endoscope image does not need to be subjected to stretching deformation, and a circular operation field can be maintained (even in the case where correction of the endoscope image is required), which is convenient for a doctor to observe.

With condition 6, since the size of the endoscopic image is affected by the cooperation between the camera and the endoscope, there are two cases in which the endoscopic image is entirely located in the target interception area, so that the intercepted target image can contain a complete endoscopic image, and in which the size of the endoscopic image is large beyond the range of the target interception area, and the intercepted target image can only contain a part of the endoscopic image.

In condition 7, referring to both the results shown in fig. 15, in the case where the endoscopic image is a circular outline, the endoscopic image is inscribed in the target cut area or circumscribed with the endoscopic image so that the endoscopic image can be displayed as large as possible. In the case of an endo-endoscopic image, the entire endoscopic image is contained in the target cut-out region, and the endoscopic image is enlarged to be tangent to the edge of the target cut-out region; in the case of an external endoscopic image, the target cut-out region includes only a part of the endoscopic image, and the part is displayed centrally, whereas the part beyond the target cut-out region cannot be displayed in the display region.

Referring to fig. 17, in some embodiments, the endoscope image capturing system performs an enlarging process on the captured image and obtains a target image in the foregoing step S902, including:

in step S1003, an endoscopic image is extracted from the captured image, and the extracted endoscopic image is subjected to magnification processing according to a preset magnification ratio, thereby obtaining a target image.

Unlike the above-described determination of the first magnification of the current endoscopic image display by boundary detection, step S1003 of the present embodiment extracts an endoscopic image from the captured image, and performs magnification processing on the extracted endoscopic image according to a preset magnification to generate a target image, so that the endoscopic image in the generated target image is also subjected to magnification processing, and the display of the target image in the display area is the enlarged display of the endoscopic image. The target image in this embodiment may be displayed directly in the display area, or may be displayed in the display area after being subjected to correction processing so that the endoscope image is centered. The endoscope image and the target image are matched in various modes, and the corresponding display effect is achieved under different matching modes. Specifically, the endoscopic image in the target image satisfies one of the following conditions:

the endoscope image is an inscribed circle of the boundary of the target image;

a portion of the image in the endoscopic image is located within the target image.

The aforementioned step S1003 directly extracts an endoscopic image, which may be an inscribed circle of the target image, such that the entire endoscopic image is contained in the target image, the diameter of the endoscopic image is equal to the height of the target image, and the endoscopic image is displayed as a complete endoscopic image when the target image is displayed in the display area (left pattern as shown in fig. 6); or the size of the endoscopic image is larger than the size of the target image, for example, the endoscopic image is a circumscribed circle of the target image or the diameter of the endoscopic image is larger than the height of the target image, etc., which makes it possible to display only a part of the endoscopic image in the target image (for example, the endoscopic image is a circumscribed circle of the target image, i.e., a right-hand pattern shown in fig. 6), and is displayed as a magnified part of the endoscopic image when the target image is displayed in the display area.

Referring to fig. 18, in some embodiments, the endoscope camera system performs an enlarging process on the captured image and obtains a target image in the foregoing step S902, including:

step S1101, an endoscope image capturing system acquires a first target magnification of an endoscope image in a history frame target image;

step S1102, the endoscope image capturing system determines a second target magnification according to the size of the endoscope image in the captured image of the current frame;

step S1103, if the difference between the second target magnification and the first target magnification is greater than the first preset value, the endoscope camera system amplifies the photographed image with the second target magnification to obtain a target image;

in step S1104, if the difference between the second target magnification and the first target magnification is less than or equal to the first preset value, the endoscope camera system amplifies the captured image with the first target magnification to obtain a target image.

The camera of the endoscope camera system periodically returns the shot image, so that in a continuous time period, a plurality of frames of shot images are returned, the endoscope camera system amplifies the shot image according to the steps, and records a first target magnification corresponding to the target image obtained through each amplification, wherein the first target magnification represents a historical frame corresponding magnification before a current frame, and the first target magnification can be a magnification adopted by one frame (a frame before the current frame or one frame in the previous frames) or a plurality of frames in the historical frame. When the endoscope camera system receives the current frame shooting image, determining a second target magnification corresponding to the current frame for magnification processing; in order to prevent the target image from being frequently adjusted in the display area, the target magnification of the current frame is prevented from being different from the first target magnification of the history frame, so that the second target magnification of the current frame is compared with the first target magnification of the history frame in step S1103, if the difference between the second target magnification of the current frame and the first target magnification of the history frame is small, the current frame and the history frame are considered to be basically the same, the difference is small, and the difference may come from algorithm errors (for example, the processes of searching the boundary of the endoscope image, calculating the center of the endoscope image, etc. in the process of performing the magnification processing are performed), the magnification of the current frame is still amplified according to the first target magnification of the history frame, so that the image frame can be processed according to the uniform magnification in most cases, and the content jitter of the display area caused by obtaining the target image with different magnification after the magnification processing is prevented.

Referring to fig. 19, in some embodiments, the endoscope image capturing system performs an enlarging process on the captured image and obtains a target image in the foregoing step S902, including:

step S1201, the endoscope image capturing system acquires a first target magnification of an endoscope image in a history frame target image;

step S1202, calculating the size of an endoscope image in a shooting image of a current frame by an endoscope shooting system, and carrying out confidence analysis on a calculation result of the size of the endoscope image to obtain a target confidence;

step S1203, if the target confidence is higher than a second preset value, the endoscope camera system determines a second target magnification according to the calculation of the size of the endoscope image, and performs magnification processing on the shot image with the second target magnification to obtain a target image;

in step S1204, if the target confidence level is lower than or equal to the second preset value, the endoscope camera system amplifies the captured image with the first target magnification and obtains a target image.

The camera of the endoscope camera system periodically returns the shot image, so that in a continuous time period, a plurality of frames of shot images are returned, the endoscope camera system amplifies the shot image according to the steps, and records a first target magnification corresponding to the target image obtained through each amplification, wherein the first target magnification represents a historical frame corresponding magnification before a current frame, and the first target magnification can be a magnification adopted by one frame (a frame before the current frame or one frame in the previous frames) or a plurality of frames in the historical frame. When the endoscope camera system receives a current frame shooting image, calculating the size of the endoscope image, determining a target confidence coefficient corresponding to a calculation result, and according to the calculation mode of the size of the endoscope image mentioned in the previous step, the target confidence coefficient can be represented by a fitting degree and other results in the calculation process, for example, fitting a circular boundary to the endoscope image, calculating a variance between an edge pixel point of the endoscope image and the fitted boundary to represent the fitting degree, if the fitting degree is poor, indicating that the confidence coefficient is low, considering that an error possibly occurs due to algorithm reasons such as a fitting calculation process, and therefore, amplifying the current frame according to a first target amplification factor of a history frame; if the fitting degree is good, the confidence is high, and the current algorithm processing process is correct, a second target magnification can be determined according to the current frame and the second target magnification is amplified, so that a target image is obtained.

In the foregoing amplifying process, the preset amplifying rate may be set according to different manners, for example, the preset amplifying rate is set by one of the following steps:

mode 5, setting a preset magnification according to a magnification instruction input by a user;

mode 6, setting preset magnification according to the display size parameter of the endoscope image set by the user;

mode 7, setting a preset magnification according to the type of the endoscope adopted by the current camera.

Mode 5 is that a user directly sets a magnification, and the user (e.g., doctor or other personnel) sets a specific value in the endoscope camera system, so that a magnification instruction is generated to assign a preset magnification of the endoscope camera system to the specific value set by the user; mode 6 is that a user inputs display size parameters to determine, wherein the display size parameters include, but are not limited to, the width and the height of a target image, the area occupation ratio of the target image in a display area and the like, for example, the width and the height of the target image are defined to be equal to the width and the height of the display area by the user, and then the endoscope camera system automatically generates a preset magnification according to the setting of the user and the size of the current shot image; mode 7 is to set a preset magnification according to the type of an endoscope installed by a camera, the type of the endoscope is divided by caliber, a user can set the caliber of the endoscope adopted by the current camera by an endoscope camera system, and the endoscope camera system automatically sets the preset magnification according to the caliber; for some endoscopes with smaller caliber (for example, 4mm caliber), in order to display the endoscope image in the display area as large as possible, a larger preset magnification ratio can be set, for some endoscopes with larger caliber (for example, 10mm caliber), a smaller preset magnification ratio can be set, so that the condition that display content of the display area is blurred due to too much magnification of the endoscope image is avoided, or the type of the endoscope is determined according to the size, the area occupation ratio or the optical magnification ratio of the endoscope image in a shooting image.

Specifically, the preset magnification is set according to the type of the endoscope used by the current camera in the foregoing mode 7, including at least one of the following:

if the type of the endoscope is a first type of endoscope, setting a preset magnification to be a third target magnification, wherein the first target magnification is used for magnifying an endoscope image into an inscribed circle of a target image;

if the type of the endoscope is a second type of endoscope, the preset magnification is set to be a fourth target magnification, and the second target magnification is used for magnifying the endoscope image into a circumscribed circle of the target image.

The first type of endoscope can be considered as a smaller bore endoscope, which may be provided with a notch for letting other medical instruments into the patient's lumen to assist in surgery, and for the convenience of doctors in identifying the position of the medical instruments, the endoscope image needs to display the position of the notch, and then the displayed target image should contain the complete endoscope image to show the position of the notch; in this case, the preset magnification of the endoscope camera host is adjusted so that the endoscope image is enlarged to an inscribed circle of the target image, and the target image can contain the entire endoscope image, so that the doctor can see the notch at the endoscope image in the display area, and thus know the position where the medical instrument enters the lumen of the patient. The second type of endoscope can be regarded as a large-caliber endoscope, which is generally used for obtaining a clear surgical field, and in order to obtain a clearer image, the preset magnification of the endoscope image pickup main unit is adjusted so that the endoscope image is enlarged to a circumscribed circle of the target image, thereby displaying the endoscope image enlarged to a sufficiently large size in the display area.

Accordingly, the aforementioned preset magnification is set by the following method:

acquiring the type of a camera;

obtaining the type of an endoscope adopted by a camera;

and determining a preset magnification according to the type of the camera and the type of the endoscope.

Wherein the preset magnification is commonly determined according to the type of the camera and the type of the endoscope, the type of the camera may be related to the size of the photosensitive element of the camera, the type of the endoscope may be related to the caliber of the endoscope, the size of the photographed image may be related to the size of the photosensitive element, the size of the endoscopic image may be related to the size of the light entering the endoscope projected on the photosensitive element, and thus setting the preset magnification in combination with the type of the camera and the type of the endoscope may enable more accurate correction and amplification of the endoscopic image, wherein the type of the camera and the type of the endoscope may be set by a user of the endoscope imaging system, the type of the camera may be determined by the endoscope imaging system automatically recognizing the product id of the camera, etc.

In some embodiments, in the process of detecting the size of the endoscope image in real time by the endoscope image capturing system, the magnification setting information input by the user is acquired, and the preset magnification is updated in real time according to the magnification setting information.

In the course of the magnification processing, the user can adjust the magnification in real time, for example, the user sets a 1.1 magnification to amplify the endoscope image, the endoscope camera system performs the magnification processing according to the 1.1 magnification, and displays the target image in the display area, if the user finds that the magnification is insufficient by observing the display area of the display, the user can adjust the magnification processing in real time by inputting the magnification setting information, for example, inputting a 1.3 magnification, and the endoscope camera system will update the target image displayed in the display area in real time based on the 1.3 magnification, so that the user confirms whether the current magnification is appropriate. It is understood that although the magnification is set by the user, normal execution of the correction function is not affected, and after the magnification is changed, the correction function performs the amplification and correction processing according to the magnification to obtain the target image.

Referring to fig. 20, in some embodiments, the detecting the size of the endoscopic image in the captured image in real time by the endoscopic image capturing system in the aforementioned step S902 includes:

Step S1301, detecting an enabling state of an amplifying function by the endoscope image pickup system;

step S1302, if the amplifying function is in an enabling state, the endoscope image capturing system detects the position of the center of the endoscope image in the captured image and the size of the endoscope image in real time;

in step S1303, if the zoom-in function is turned off, the endoscope image capturing system outputs a display signal to the display according to the image information of the captured image so as to display the captured image in the display area, or the endoscope image capturing system zooms in the captured image according to the preset zoom-in magnification and outputs a display signal to the display so as to display the zoomed-in captured image in the display area.

The endoscope image pickup system provides a function switch for providing a doctor with a function of detecting the center of an endoscope image in real time and performing magnification, which is referred to as a magnification function in this embodiment. When the endoscope image pickup system detects that the amplifying function is in the enabled state, executing the steps S902 to S903, detecting the size of the endoscope image in the photographed image in real time to determine whether to amplify, wherein the amplifying process can refer to the description of the steps and is not repeated once; when the endoscope image pickup system detects that the amplifying function is in the off state, step S902 and step S903 do not need to be executed, after the photographed image returned by the camera is received, the photographed image is output to the display, the endoscope image which is not amplified is directly displayed, or the photographed image is amplified by a preset amplifying rate and then the output target image is output to the display.

Referring to fig. 21, in some embodiments, the state of the amplifying function is set by one of the following steps:

step S1401, the endoscope image pickup system acquires a setting instruction input by a user in a man-machine interaction interface of the image pickup system, and sets a state of an amplifying function according to the setting instruction;

or,

step S1402, the endoscope image capturing system acquires a key instruction of an operation key of the camera by the user, and sets a state of the amplifying function according to the setting instruction.

The endoscope camera system provides a man-machine interaction interface for judging whether the amplifying function is started, for example, the man-machine interaction interface provides a specific function key (which can be a physical key and/or a virtual key), and a doctor can generate a setting instruction for turning on or off the amplifying function by clicking the function key on the man-machine interaction interface. In addition, the handle of the camera is provided with an operation key, and a doctor can generate a key instruction to send to the endoscope camera system by clicking the operation key on the handle, so that the amplifying function is turned on or turned off. In addition, the doctor may trigger the endoscope imaging system to turn on or off the magnifying function by means of a voice command or the like, and will not be described here.

Referring to fig. 22, an embodiment of the present application provides an image display method of an endoscopic image pickup system including a display for displaying an endoscopic image, the method including, but not limited to, the steps of:

step S1501, an endoscope image capturing system acquires a captured image transmitted by a camera, wherein the captured image includes an endoscope image;

the camera is provided with a lens and an image sensor (such as CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor), and a CCD is taken as an example for follow-up defaults), the camera is provided with an illumination element (such as a light guide window, an LED lamp and the like), light emitted by the illumination element irradiates the body of a patient, reflected light irradiates the image sensor through the lens, and the camera converts the reflected light into an electric signal to obtain a shooting image; the camera transmits the generated captured image to the endoscope image capturing system as an image signal, so that the captured image can be displayed on a designated display area of the display. The endoscope image corresponds to an imaging range in which reflected light irradiates the image sensor after passing through the lens, and the shot image refers to the whole imaging range of the image sensor and comprises the endoscope image; since the illumination element and the image sensor tend to be different in size, the area of the captured image from which the endoscopic image is divided tends to be dark (i.e., the background area where there is no effective image), and at this time, the peripheral area of the endoscopic image is dark in the captured image displayed on the display area. In addition, the matching degree of the lens and the image sensor in endoscopes with different types and different calibers is different, so that the imaging axis and the imaging axis of the image sensor are deviated, the center of the endoscope image is not overlapped with the center of the shot image on the display area, a doctor often needs to focus on the center and the position near the center of the endoscope image, and the deviation of the position of the endoscope image obviously brings trouble to the doctor for observing and adjusting the endoscope image.

Step S1502, an endoscope image capturing system detects a position of a center of an endoscope image in a captured image in real time, and if the center of the endoscope image is not in a center area of the captured image, the endoscope image capturing system performs horizontal centering processing on the captured image and obtains a target image, wherein the target image includes at least a part of the endoscope image;

the endoscope camera system receives the shooting image returned by the camera, determines whether the center of the endoscope image is at the center of the shooting image according to the shooting image, if the center of the endoscope image deviates from the center area of the shooting image, the shooting image needs to be subjected to deviation correction processing to obtain a target image, the target image comprises the endoscope image with the adjusted center position, and at the moment, the center of the target image is overlapped with the center of the endoscope image. Because there are various matching modes between the lens of the endoscope and the optical lens of the camera, the imaging size of the endoscope image on the photographed image is different, in some cases, the upper and lower edges of the endoscope image with a circular outline exceed the upper and lower edges of the photosensitive range of the photosensitive element, for example, the upper edge of the endoscope image exceeds the lower edge more, in this case, if the correction processing is performed according to the foregoing methods of step S102 and step S103, the target cut image will be cut to the area above the photosensitive range, and this area is not imaged, so the upper part of the cut target image is a dark part, while the lower part is an endoscope image part, which is obviously unfavorable for the doctor to observe. Therefore, in this step S1502, the horizontal centering process is adopted, and the center deviation correction is not performed. The horizontal centering process makes the upper and lower edges of the target cut-out area coincide with the upper and lower edges of the photographed image, and only translates the endoscopic image, maintaining the look and feel of the endoscopic image in the display area.

In step S1503, the endoscope image capturing system outputs a display signal to the display according to the image information of the target image so that the display area displays the target image, wherein the center of the endoscope image displayed in the display area coincides with the central axis of the display area in the horizontal direction.

Through the above-mentioned horizontal centering adjustment of step S1502, the center of the endoscope image in the target image displayed in the display area coincides with the central axis of the display area in the horizontal direction, so that the endoscope image can be provided to the doctor for centering in the horizontal direction, which is convenient for the doctor to observe.

Referring to fig. 23, an embodiment of the present application provides an image display method of an endoscopic image pickup system including a display for displaying an endoscopic image, the method including, but not limited to, the steps of:

in step S1601, the endoscope image capturing system acquires an image signal of a captured image transmitted by the camera.

In step S1602, the endoscope image capturing system enters a first display mode and outputs a first display signal to a display according to an image signal so that a display area displays a captured image, wherein the captured image displayed in the display area includes an endoscope image whose center is offset from the center of the display area.

In a first display mode, the endoscope camera system receives an image signal transmitted back by the camera, generates a first display signal according to the image signal, and sends the first display signal to the display, and the display displays a shooting image corresponding to the image signal after receiving the first display signal. In this embodiment, the first display signal of the endoscope camera system can adapt to the display size of the display area, if the display sizes of the shot image and the display area are matched, the complete shot image can be directly displayed on the display area, and if the display sizes of the shot image and the display area are not matched (such as different requirements on aspect ratio, resolution, definition, etc.), the shot image can also be displayed on the display area after preliminary processing (such as cutting into corresponding aspect ratio, resolution, definition adjustment, etc.). Note that the above-described preliminary processing does not perform positional adjustment in accordance with the center of the endoscopic image and the center of the display area, and therefore the captured image is directly displayed through the display area in the first display mode, there is still a case where the center of the endoscopic image is deviated from the center of the display area.

Step S1603, in the first display mode, the endoscope image capturing system acquires a zoom operation instruction of the user, and adjusts the first display signal according to the zoom operation instruction, so that the image displayed in the display area is zoomed along the center of the display area, and the center of the endoscope image displayed in the zoomed display area deviates from the center of the display area;

for the case where the center of the endoscopic image is offset from the center of the display area, the endoscopic image pickup system provides a manual zoom function. Specifically, the user enables the manual zoom function through the handle of the endoscope, the endoscope camera host or a key on the display, or enables the endoscope camera host to enable the manual zoom function through a voice command, at this time, the user can manually adjust the zoom scale of the picture displayed in the display area, for example, the user can perform operations such as "zoom in" and "zoom out" on the picture through the handle of the endoscope, the endoscope camera host or the key on the display, or perform operations such as "zoom in" and "zoom out" on the picture through the voice command; the manual zooming in the first display mode is performed along the center of the display area, so that a doctor can observe the endoscope image conveniently to a certain extent, but if the center of the endoscope image deviates from the center of the display area, the center of the endoscope image still cannot coincide with the center of the display area after the manual zooming in the above manner, and therefore the endoscope image can be automatically enlarged and centered in the following second display mode.

Step S1604, the endoscope camera system acquires a user' S centering zoom-in operation instruction and enters a second display mode, in which the endoscope camera system determines a target cut-out area including at least a part of the endoscope image from the photographed image according to the aspect ratio of the display area, the center of the target cut-out area coinciding with the center of the endoscope image;

for the case where the center of the endoscopic image is offset from the center of the display area, the endoscopic image pickup system provides a centered magnification function. Specifically, in the second display mode, the user enables the manual zoom function through a handle of the endoscope, the endoscope camera host or a key on the display, or enables the endoscope camera host to enable the centering amplification function through a voice command, after receiving the centering amplification operation command corresponding to the centering amplification function, the endoscope camera system determines a target intercepting region in the photographed image, the aspect ratio of the target intercepting region is the same as that of the display region, the center of the target intercepting region coincides with the center of the endoscope image, at this moment, the target intercepting region obviously can intercept at least a part of the endoscope image, and the part of the intercepted endoscope image is used for subsequent centering and amplification.

In step S1605, the endoscope image capturing system outputs a second display signal to the display according to the image information of the target cut-out region, so that the endoscope image of the target cut-out region is enlarged to the whole display region, wherein the center of the endoscope image displayed in the display region coincides with the center of the display region.

After the target cut-out area is obtained through the above step S1604, the endoscope image capturing system outputs a second display signal to the display to adjust the picture displayed in the display area, the second display signal is used to enlarge the target cut-out area to be displayed on the whole display area, and since the aspect ratio of the target cut-out area is the same as that of the display area, the picture of the display area can be occupied after the target cut-out area is enlarged, and meanwhile, since the center of the target cut-out area coincides with the center of the endoscope image, after the target cut-out area is enlarged in the display area, the center of the endoscope image naturally coincides with the center of the display area, and a doctor can intuitively see the target cut-out area which is enlarged in the center of the original endoscope image and coincides with the center of the display area, that is, the doctor focuses on the center of the endoscope image and the area nearby the center, can be enlarged and displayed on the display area through the automatic centering of the steps S1604 and S1605, so that the doctor can observe the condition in the patient in the operation without frequently adjusting the scaling ratio and the picture position of the endoscope image, and the efficiency of the operation can be improved. The target intercepting region is displayed in the display region, a dark background region may still exist on the display picture, for example, a circular endoscope image region is located in a rectangular shooting image region, the target intercepting region is rectangular with the same aspect ratio as the display region, and intercepting is performed in a mode of being circumscribed to the circular endoscope image region, so that the target intercepting region itself comprises the background region, and the background region is displayed together; if the target cut-out area is cut out in a manner of inscribing the circular endoscope image area, the whole target cut-out area is the endoscope image area, and the background area is not included at this time.

The display area of the display has several forms, the first is that the display area is equal to the effective display area of the whole display panel of the display, the enlargement of the target interception area is equal to the full screen enlargement, the center of the target interception area coincides with the center of the display, and the aspect ratio of the target interception area is the same as the aspect ratio of the display; the second is that the effective display area of the whole display panel of the display comprises a display area for displaying the target intercepting area and other display areas, such as a user interface like a menu bar, the total area of the display area for displaying the target intercepting area and the other display areas is equal to the effective display area of the display panel, the other display areas occupy a part of area, the center of the display area for displaying the target intercepting area is not coincident with the center of the display, and the center of the display area for displaying the target intercepting area is enlarged in a centering manner; the third case is that the effective display area of the entire display panel of the display includes a display area for displaying the target cut-out area and other display areas, but the other display areas are superimposed on the display area for displaying the target cut-out area, at which time the display area for displaying the target cut-out area is equal to the effective display area of the display panel, the center of the target cut-out area coincides with the center of the display, and the aspect ratio of the target cut-out area is the same as the aspect ratio of the display.

It can be appreciated that the user' S centering operation instruction may be implemented by a single key or a single voice instruction, for example, a certain key is designated (a preset or custom key function) on the handle of the endoscope as a key corresponding to the centering operation instruction, and the doctor may directly execute the automatic centering function of the above step S1604 and step S1605 by pressing the designated key during the operation; in addition, for example, a certain voice command is set in the endoscope camera host, and the doctor sends the voice command in the operation process, and the endoscope camera host directly executes the automatic centering and amplifying functions of the step S1604 and the step S1605 after identifying. In summary, the automatic centering and amplifying function of the embodiment of the present application can be enabled by a simple operation mode, thereby reducing the operation amount of a doctor during an operation.

In addition, the first display mode and the second display mode are two display modes of the endoscope camera system according to the embodiment of the application, and the two display modes can be started simultaneously or can be started independently. For example, the first display mode and the second display mode are simultaneously enabled, when a zoom operation instruction is received, the manual amplification function of step S1603 is performed, and when a center amplification operation instruction is received, the center amplification functions of step S1604 and step S1605 are performed; for another example, the second display mode is independently enabled, and when receiving the centering and amplifying operation instruction, the centering and amplifying functions of step S1604 and step S1605 are performed, and in this mode, the doctor may also actively close the second display mode to enter the first display mode, or give a scaling operation instruction, and the endoscope camera system automatically enables the first display mode and performs the manual amplifying function of step S1603.

Referring to fig. 24, in some embodiments, the determining a target cut area of the endoscope image according to the aspect ratio of the display area in the step S1604 specifically includes the steps of:

step S1701, the endoscope image capturing system determines an endoscope image area of an endoscope image from the captured image;

step S1702, the endoscope image capturing system determines the center of the endoscope image area and the scope of the endoscope image according to the endoscope image area;

in step S1703, the endoscopic image capturing system determines a target cut-out area according to the aspect ratio of the display area, the center of the endoscopic image area, and the endoscopic image range, wherein the center of the target cut-out area coincides with the center of the endoscopic image area, the aspect ratio of the target cut-out area is the same as the aspect ratio of the display area, and the target cut-out area contains at least a part of the endoscopic image.

As can be seen from the foregoing, at least a part of the captured image is an endoscopic image, and the periphery of the endoscopic image may be a dark background area, so that before determining the target cut-out area, it is necessary to determine the scope of the endoscopic image, that is, the endoscopic image area, and then determine the center of the endoscopic image and the scope of the endoscopic image based on the endoscopic image area, and further determine the target cut-out area according to the aspect ratio of the display area. Firstly, the shot image needs to be preprocessed, so that the separation between the endoscope image and the peripheral dark background area is easier, the center and the scope of the separated endoscope image area can be determined through a specific mode (such as pixel statistics, intelligent shape matching and the like), and the target interception area is determined according to the center of the endoscope image area, the scope of the endoscope image and the aspect ratio of the display area. Specifically, the center of the target cut-out area coincides with the center of the endoscope image area, the aspect ratio of the target cut-out area is the same as the aspect ratio of the display area, that is, one area is defined as the target cut-out area with the aspect ratio of the display area as the size ratio centered on the center of the endoscope image area, and the target cut-out area contains at least a part of the endoscope image. For example, the endoscope image is a circular area, the shot image is a rectangular area, the endoscope image is entirely located in the shot image, the circular endoscope image area can be inscribed in the target intercepting area, when the whole display area displays the target intercepting area, the center of the display area is the circular endoscope image area, and the endoscope image area is tangent to the whole edge of the display. The above is only one of the interception modes, and the specific interception mode of the target interception area will be described in detail below.

Referring to fig. 25, in some embodiments, the above-mentioned step S1701 of the endoscopic image capturing system determining an endoscopic image area of an endoscopic image from the captured image specifically includes the steps of:

step S1801, the endoscope image capturing system converts the captured image into a grayscale image;

in step S1802, the endoscopic imaging system performs a threshold segmentation process on the grayscale image according to a preset threshold to obtain a binary image, and the area included in the binary image is an endoscopic image area.

The preprocessing of the photographed image is realized by step S1801 and step S1802, the photographed image is adjusted to a gray level, and then the gray level image is binarized based on a preset threshold value, a dark background area around the periphery of the endoscopic image is removed, and the obtained binary image is an area containing the endoscopic image. The preset threshold can be set according to actual needs, the contrast of the junction between the endoscope image and the dark area is different due to different photosensitive elements and lighting elements, the endoscope image area can be effectively divided by setting the proper preset threshold, and the center of the endoscope image area and the scope of the endoscope image are convenient to calculate subsequently.

In some embodiments, the endoscope image capturing system in the step S1702 determines the center of the endoscope image area and the scope of the endoscope image according to the endoscope image area, specifically, the endoscope image capturing system performs a distribution statistical calculation on coordinates of each pixel point in the endoscope image area to determine the center of the endoscope image area and the scope of the endoscope image. Pixel statistics may be used to determine a particular shape of an endoscopic image region, such as a circular or rectangular endoscopic image region, the center of the endoscopic image region and the scope of the endoscopic image may be calculated according to a pixel statistics algorithm corresponding to a regular shape of the endoscopic image region, and for other irregularly shaped endoscopic image regions, a particular pixel statistics algorithm may be set to determine the center of the endoscopic image region and the scope of the endoscopic image.

As shown in fig. 26, the distribution statistics calculation is performed on the coordinates of each pixel point in the endoscope image region, specifically including the steps of:

step S1901, the endoscope image capturing system performs desired distribution calculation on coordinates of each pixel point of the endoscope image in the endoscope image area, to obtain coordinates of the center of the endoscope image area;

in step S1902, the endoscopic imaging system calculates a variance distribution of coordinates of each pixel point of the endoscopic image in the endoscopic image area, and obtains an endoscopic image range of the endoscopic image area.

For the calculation of the center coordinates, the desired distribution calculation is performed with the coordinates of each pixel point in the endoscope image area, the statistical coordinate values are used for determining the center coordinates of the endoscope image area, for the calculation of the scope of the endoscope image, the variance distribution calculation is performed with the coordinates of each pixel point in the endoscope image area, the statistical coordinate values are used for determining the scope of the endoscope image area, and the calculated scope coordinates of each endoscope image area are arranged to form the scope of the endoscope image. The desired distribution calculation and variance distribution calculation of the pixel points can be implemented in different calculation manners. Referring to fig. 27, in a specific embodiment, the calculation of the desired distribution of the pixel points in step S1901 includes the following steps:

Step S2001, the endoscope image capturing system performs expected distribution calculation on coordinates of each row of pixel points in the endoscope image region, to obtain a first expected value of coordinates of each row of pixel points;

step S2002, the endoscope image pickup system carries out expected distribution calculation on the coordinates of each column of pixel points in the endoscope image area to obtain a second expected value of the coordinates of each column of pixel points;

in step S2003, the endoscope image capturing system performs statistical calculation on each of the first expected values and each of the second expected values to obtain coordinates of the center of the endoscope image region.

The endoscope image capturing system segments an endoscope image area, acquires coordinates of each pixel point in the endoscope image area, and the coordinates of the pixel points are generally represented according to a pixel array corresponding to the resolution of the display (the coordinates of the pixel points can be determined in other self-defined modes), so that the coordinates of each pixel point in the endoscope image area are determined, and the pixel points are calculated in a row and column mode respectively. For each row of pixel points, the ordinate is the same, and expected distribution calculation is carried out on the coordinates of each row of pixel points (for example, the abscissa of each row of pixel points is averaged) to obtain a first expected value of the coordinates of each row of pixel points; similarly, for each column of pixel points, the abscissa is the same, and the coordinates of each column of pixel points are subjected to expected distribution calculation (for example, the ordinate of each column of pixel points is averaged) to obtain a second expected value of the coordinates of each column of pixel points; and carrying out statistical calculation according to the first expected value and the second expected value obtained by calculation, and determining the center coordinates of the endoscope image area. For example, the first expected value represents the center coordinates of the row of pixels, the second expected value represents the center coordinates of the column of pixels, then the center coordinates of the endoscopic image area may be exemplarily calculated by averaging the ordinate of all the first expected values to obtain a first center coordinate, averaging the abscissa of all the second expected values to obtain a second center coordinate, then calculating the variance of the distance between the first center coordinate and the pixel of the endoscopic image area to obtain a first variance, calculating the variance of the distance between the second center coordinate and the pixel of the endoscopic image area to obtain a second variance, comparing the first variance and the second variance, and taking a corresponding center coordinate with a smaller variance value as the coordinate of the center of the endoscopic image area.

In some embodiments, the lens of the endoscope is a circular lens, then the reflected light of the light element in the patient' S body is imaged as a circular spot on the image sensor by the lens, and the sensing area of the image sensor is generally rectangular, then the captured image is rectangular, the endoscope image area is circular, and the center of the endoscope image area is the center of the endoscope image area, then in this embodiment, the endoscope image capturing system determines the center of the endoscope image area and the scope of the endoscope image according to the endoscope image area, specifically the center of the endoscope image area and the radius according to the endoscope image area in step S1702. Likewise, for a circular endoscopic image area, calculating the center of a circle and the endoscopic image range may be performed according to the above-described steps S1901 and S1902, and after calculating the determined center of a circle, determining the radius of the endoscopic image area by calculating the distance between the center of a circle and the endoscopic image range.

Since the endoscope image area is circular, the display area of the display is rectangular, and the target intercepting area is rectangular with the same aspect ratio as the display area of the display, two modes for selecting the target intercepting area can be respectively inscribed in the endoscope image area or circumscribed outside the endoscope image area, and the target intercepting area can be a transverse rectangle or a longitudinal rectangle according to the aspect ratio of the current display area of the display no matter inscribed or circumscribed. Specifically, referring to fig. 28, the step S1703 includes the steps of:

Step S2101, the endoscope image capturing system determines whether the diameter of the endoscope image is larger than the height of the display area;

step S2102, if the diameter of the endoscope image is larger than the height of the display area, the endoscope image pickup system determines a target interception area inscribed in the endoscope image according to the circle center of the endoscope image area and the aspect ratio of the display area;

in step S2103, if the diameter of the endoscopic image is smaller than or equal to the height of the display area, the endoscopic image capturing system determines a target cut-out area circumscribed outside the endoscopic image according to the center of the endoscopic image area and the aspect ratio of the display area.

From the foregoing, it is understood that the target capturing area is used to capture at least a portion of the endoscopic image, and in order to make as many endoscopic images as possible enter the target capturing area, it is determined whether the target capturing area adopts an inscribed or circumscribed manner according to the size of the current endoscopic image. Acquiring the pixel height of a display area of a display, and calculating the diameter or radius of an endoscope image; when the diameter of the endoscope image is larger than the height of the display area, it is indicated that the endoscope image area is relatively large, a part of the image cannot be displayed in the display area of the display, at this time, a target cut-out area is set at the center of the endoscope image area according to the aspect ratio of the display area of the display, and is inscribed inside the endoscope image, as shown in fig. 29 (a represents the target cut-out area, B represents the endoscope image area, C represents the area of the photographed image, and a' represents the display area of the display), at this time, the target cut-out area is enlarged centrally, so that the display area of the display is the part of the endoscope image cut out by the target cut-out area, and the dark background area is not included. When the diameter of the endoscopic image is smaller than the height of the display area, it is indicated that the endoscopic image area has been displayed entirely in the display area of the display, at this time, a target cut-out area is set at the center of the endoscopic image area in accordance with the aspect ratio of the display area of the display, which is circumscribed outside the endoscopic image, as shown in fig. 30 (a represents the target cut-out area, B represents the endoscopic image area, C represents the area of the captured image, and a' represents the display area of the display), at this time, the target cut-out area is centered such that the center portion of the display area of the display displays a circular endoscopic image area, and the rest displays a dark background area.

The above applies to the case where the display area is rectangular in the lateral direction, in which the diameter of the circular endoscopic image area is compared with the height of the display area, and if the display area is rectangular in the longitudinal direction, the diameter of the circular endoscopic image area is compared with the width of the display area. In summary, the diameter of the circular endoscopic image area is compared with the short side of the display area of the rectangular display to determine whether the current endoscopic image is too large, and thus, what way to intercept the target interception area.

Referring to fig. 31, in some embodiments, the endoscope image capturing system outputs a second display signal to the display according to the image information of the target capturing area in the above step S1605, so as to zoom in the endoscope image of the target capturing area to the whole display area, and specifically includes the following steps:

step S2201, the endoscope camera system determines target amplified image frame information according to the image information of the target interception area;

step S2202, the endoscope camera system carries out interpolation processing on the target amplified image frame information to obtain a plurality of transition image frame information with different amplification degrees;

in step S2203, the endoscope image capturing system outputs a second display signal to the display according to the plurality of transition image frame information and the target enlarged image frame information, so that the image of the target cut-out area displayed in the display area is gradually enlarged to the whole display area.

The above-described steps S2201 to S2203 represent a process of gradually enlarging the target cut-out region by means of image frame interpolation. When the size and the position of the target intercepting region are determined, the target intercepting region needs to be amplified, at this time, target amplified image frame information (i.e. interpolated target image frames) is obtained according to the image content (i.e. image information) in the target intercepting region, then the endoscope camera system processes the target amplified image frame information according to a preset interpolation processing mode to obtain a plurality of transition image frame information with different amplifying degrees, and then the plurality of transition image frame information is displayed on the display region according to the amplifying degree sequence. Thus, the physician can see a stepwise enlargement of the target cut-out area on the display area of the display, stepwise from a smaller magnification, until the target cut-out area is enlarged to the entire display area. It can be understood that the interpolation process may obtain a plurality of transition image frame information, for example, one transition image frame information is corresponding to each preset time, for example, 10 transition image frame information is generated, one transition image frame information is displayed every 0.1 second, and the amplification of the whole target cut-out area is completed within about 1 second.

Referring to fig. 32, in some embodiments, the display method of the present application further includes the steps of:

step S2301, the endoscope image capturing system redetermines a target cut-out area of the endoscope image from the captured image every N frames in the second display mode, where N is a natural number;

in step S2302, the endoscope image capturing system updates the second display signal output from the display according to the image information of the newly determined target cut-out area, so that the endoscope image of the newly determined target cut-out area is enlarged to the entire display area.

Since the endoscopic image is transmitted to the endoscopic image pickup host in real time, the endoscopic image pickup system updates the content of the endoscopic image displayed on the display area at intervals of time/every N frames. When in the second display mode, the doctor enables the centering amplification function, and the target interception area needs to be updated at any time. Specifically, the above steps S1604 and S1605 may be regarded as a processing procedure performed on one of the captured images, and when the endoscope image capturing system receives a new captured image after a period of time or after N frames, the processing of steps S1604 and S1605 is performed on the new captured image, and then the new target capture area obtained by the processing is replaced with the original target capture area and displayed on the display area. In the process, a target intercepting region is selected again according to the aspect ratio of the display region, the center of the target intercepting region coincides with the center of the endoscope image, the endoscope camera system updates second display information output to the display and is used for displaying a new target intercepting region on the display region, and real-time updating of the target intercepting region is achieved, namely, video streams of a camera are displayed in the target intercepting region in real time under the centering and amplifying functions.

Referring to fig. 33, in some embodiments, the step S2301 of redefining the target cut area of the endoscope image from the captured image according to the aspect ratio of the display specifically includes the steps of:

step S2401, the endoscope image capturing system redetermines an endoscope image area of the endoscope image from the captured image;

step S2402, the endoscope image capturing system redetermines the target center of the endoscope image area and the target endoscope image range from the endoscope image area;

in step S2403, the endoscope camera system redetermines a target cut-out region based on the aspect ratio of the display region, the target center of the redetermined endoscope image region, and the target endoscope image range, wherein the target center of the target cut-out region coincides with the center of the endoscope image region, the aspect ratio of the target cut-out region is the same as the aspect ratio of the display region, and the target cut-out region contains at least a part of the endoscope image.

When a new captured image is received, the endoscope image capturing system redetermines a region corresponding to the endoscope image, that is, an endoscope image region, in the captured image, and then determines the center of the endoscope image and the scope of the endoscope image based on the endoscope image region, and further determines a target cut-out region according to the aspect ratio of the display region. Similarly, the new captured image may be preprocessed to make it easier to divide between the endoscopic image and the peripheral dark background area, the center and the scope of the divided endoscopic image area may be determined in a specific manner, and the target cut-out area may be determined according to the center of the endoscopic image area, the scope of the endoscopic image and the aspect ratio of the display area. In this way, the target center of the new target cut-out region also coincides with the center of the endoscope image region, and can be enlarged into the entire display region instead of the target cut-out region of the previous frame.

In the process of updating the target center and the target endoscope image range, if the endoscope image pickup system judges that the area of the image does not change greatly, the center of the endoscope image area and the scope of the endoscope image can not be updated, the calculation amount of the system for amplifying and centering is reduced, and the display speed is improved. Specifically, referring to fig. 34, the endoscope image capturing system in step S2402 described above re-determines the target center and the target scope of the endoscope image area from the endoscope image area, specifically including the steps of:

step S2501, the endoscope image capturing system redetermines the center of the endoscope image area and the scope of the endoscope image from the endoscope image area;

in step S2502, the endoscope image capturing system compares the center and/or the scope of the endoscope image area before and after the re-determination, and if the first threshold condition is exceeded, the center and scope of the endoscope image area after the re-determination are set as the target center and the target scope of the endoscope image, and if the first threshold condition is not exceeded, the center and scope of the endoscope image area before the re-determination are set as the target center and the target scope of the endoscope image.

The endoscope image area is redetermined from the new photographed image, the center of the endoscope image area and the scope of the endoscope image are calculated, the new center is compared with the old center, or the new scope of the endoscope image is compared with the old scope of the endoscope image, and the position deviation of the new center and the old center and the position deviation of the new scope of the endoscope image are calculated. The centers of the new and old endoscope image areas can be compared separately, or the ranges of the new and old endoscope image areas can be compared separately, and the ranges need to carry position information, for example, a series of pixels are used for representing the scope of the endoscope image, and the pixels are compared in position, not simply through the length information such as radius. If the scope of the endoscope image area indicates information such as a radius that does not include a position, it is necessary to simultaneously compare the center of the new and old endoscope image areas with the radius to determine whether a positional shift occurs between the new and old endoscope image areas.

When the deviation exceeds a first threshold condition, the position of the new endoscope image area is greatly changed, and the target interception area is required to be selected according to the new center and the scope of the endoscope image; when the deviation does not exceed the first threshold condition, the new endoscope image area is indicated to have no larger change in position (smaller change can be regarded as calculation error), and the target interception area can be selected directly according to the old center and the scope of the endoscope image. The first threshold condition can be set according to actual needs, the comparison between the new center and the old center and the scope of the endoscope image is often the comparison of the pixel point calculation results, if the calculation accuracy of the endoscope image pickup system is higher, the first threshold condition can be set to be stricter, and if the calculation accuracy of the endoscope image pickup system is lower, the first threshold condition can be set to be looser, so that larger calculation errors are tolerated.

In some embodiments, the endoscopic camera system obtains the user's centering zoom operation instruction and enters the second display mode by at least one of:

the method comprises the steps that an endoscope camera system obtains a centering amplifying operation instruction through the operation of a user on a key on the endoscope camera system;

or the endoscope camera system acquires a centering amplifying operation instruction through the operation of a user on a key on the camera;

alternatively, the endoscope image pickup system acquires the centering magnification operation instruction by a voice instruction of the user.

The first condition is that an endoscope camera host of the endoscope camera system is provided with a key corresponding to the centering amplifying operation instruction, a doctor or a nurse can activate the centering amplifying operation instruction by pressing the key on the endoscope camera host, and the endoscope camera system automatically enters a second display mode after receiving the centering amplifying operation instruction; besides setting the key on the endoscope camera host, other control equipment can be externally connected, and the key corresponding to the centering amplifying operation instruction, such as a remote controller, is set on the control equipment. The second condition is that a key corresponding to a centering amplifying operation instruction is arranged at the handle of the endoscope camera system, and if the camera is positioned at the handle (for example, an endoscope adopting a hard tube), a doctor can operate the key on the camera and send out the centering amplifying operation instruction; the third condition is that a key (virtual key or physical key) corresponding to the centering amplifying operation instruction is arranged on a display of the endoscope camera system, a doctor or a nurse can activate the centering amplifying operation instruction by touching or pressing the key on the display, and the endoscope camera system automatically enters a second display mode after receiving the centering amplifying operation instruction; the fourth case is that the endoscope camera system is integrated with a voice recognition module, the voice recognition module is internally provided with a voice command corresponding to the centering amplification operation command, and when the voice recognition module is matched with the voice command sent by the user and corresponds to the centering amplification operation, the centering amplification operation command is generated and the endoscope camera system automatically enters the second display mode.

Referring to fig. 35, an embodiment of the present application provides an image display method of an endoscopic image pickup system including a display area for displaying an endoscopic image, the method including, but not limited to, the steps of:

step S2601, the endoscope image capturing system acquires an image signal of a captured image transmitted by the camera;

step S2602, the endoscope image capturing system outputs a first display signal to the display according to the image signal, so that the display area displays a captured image, wherein the captured image displayed in the display area includes the endoscope image;

step S2603, the endoscope image capturing system acquires a centering zoom operation instruction of the user, and determines a target cut-out region including at least a part of the endoscope image from the captured image, the center of the target cut-out region coinciding with the center of the endoscope image;

in step S2604, the endoscope image capturing system outputs a second display signal to the display according to the image information of the target capturing area, so that the endoscope image of the target capturing area is enlarged to the entire display area, wherein the center of the endoscope image displayed in the display area coincides with the center of the display area.

An endoscopic imaging system according to another embodiment of the present application may be exemplified by the endoscopic imaging system 1000 of fig. 1, in which an endoscopic image corresponding to light reflected by an illumination element in a patient can be generated by a photosensitive element of a camera, and a captured image is output to an endoscopic imaging host, and at least a part of the captured image is an endoscopic image. After receiving the photographed image, the endoscope photographing system generates a first display signal and transmits the first display signal to the display, and the display displays the photographed image according to the first display signal. In this case, there is a possibility that the center of the endoscopic image in the display captured image does not coincide with the center of the display area. In order to facilitate a doctor to check a picture returned by an endoscope, the endoscope camera shooting system provided by the application provides a centering amplifying function, a doctor or a nurse enables a manual zooming function through a handle of the endoscope, an endoscope camera shooting host or a key on a display, or enables the endoscope camera shooting host to enable the centering amplifying function through a voice command, after the endoscope camera shooting system receives a centering amplifying operation command corresponding to the centering amplifying function, a target intercepting area is determined in a shooting image, the center of the target intercepting area coincides with the center of an endoscope image, at the moment, the target intercepting area can obviously intercept at least a part of the endoscope image, and the part of the intercepted endoscope image is used for subsequent centering and amplifying.

In some embodiments, the endoscope camera system acquires the centering zoom operation instruction of the user in step S2603, and determines the target cut-out area including at least a part of the endoscope image from the captured image, specifically, the endoscope camera system acquires the centering zoom operation instruction of the user, and determines the target cut-out area including at least a part of the endoscope image from the captured image according to the aspect ratio of the display area. The endoscope camera system outputs a second display signal to the display to adjust the display area, the second display signal is used for displaying the target intercepting area on the display area, the aspect ratio of the target intercepting area is the same as that of the display area, the target intercepting area can occupy the picture of the display area after being enlarged, meanwhile, the center of the endoscope image naturally coincides with the center of the display area after being enlarged due to the fact that the center of the target intercepting area coincides with the center of the endoscope image, a doctor can intuitively see the target intercepting area which is enlarged by the center of the original endoscope image and coincides with the center of the display area through the display, that is, the doctor focuses on the center of the endoscope image and the area nearby the center, automatic centering enlargement can be achieved through the steps, and therefore the doctor can observe the condition in a patient in an operation conveniently, and the doctor does not need to frequently adjust the scaling and the picture position of the endoscope image, and therefore the operation efficiency can be improved.

It can be appreciated that the user' S centering operation instruction may be implemented by a single key or a single voice instruction, for example, a certain key is designated (a preset or custom key function) on the handle of the endoscope as a key corresponding to the centering operation instruction, and the doctor may directly execute the automatic centering function of the above step S2601 and step S2604 by pressing the designated key during the operation; in addition, for example, a certain voice command is set in the endoscope camera host, and the doctor sends the voice command in the operation process, and the endoscope camera host directly executes the automatic centering and amplifying functions of the step S2601 and the step S2604 after recognition. In summary, the automatic centering and amplifying function of the embodiment of the present application can be enabled by a simple operation mode, thereby reducing the operation amount of a doctor during an operation.

It is to be understood that in the above step S2602, when the display displays the captured image based on the first display signal, there is a possibility that the center of the endoscopic image and the center of the display area have already been overlapped, but the current display or the screen output by the display does not provide the corresponding measurement reference point, so it is difficult for the doctor or the nurse to judge whether the center of the endoscopic image and the center of the display area have already been overlapped. Then, the centering amplification operation instruction can still be executed in step S2603, so that the doctor can ensure that the endoscope image is centered and amplified every time the endoscope is used, even if the screen of the display area is not actually changed after the centering amplification operation instruction is executed.

The embodiment of the application provides an endoscope camera host, which comprises:

a housing;

the processor acquires an image signal of the camera to perform image processing and then outputs the image signal to the display for display;

the processor is further configured to perform the image display method of the endoscopic imaging system of any of the above embodiments.

In another aspect, an embodiment of the present application provides an endoscope camera system, including a light source, a camera, a display, and an endoscope camera host according to any of the foregoing embodiments, where the endoscope camera host is connected to the display through a video cable.

In terms of hardware, the endoscope camera host provides a communication interface and a display interface, wherein the communication interface is used for being connected with a camera of an endoscope, and the display interface is used for being connected with a display to display an endoscope image; when the endoscope is used, a doctor operates the endoscope and obtains the picture in the patient through the camera, and after the endoscope camera host receives the shooting image returned by the camera, a display signal is output to the display through the display interface, so that the display displays the shooting image. After receiving the centering amplifying operation instruction of the user, the endoscope camera host adjusts the output display signal, and displays the target intercepting region in the embodiment of the image display method, thereby realizing centering amplifying.

In summary, in order to adapt to endoscopes with different calibers, when the axis deviation exists between the shot image and the endoscope image by imaging of the camera, in order to realize the rapid and simple centering amplification function, the above embodiment of the present application determines a target intercepting region according to the center of the endoscope image, the scope of the endoscope image and the aspect ratio of the display region, and the center of the target intercepting region coincides with the center of the endoscope image.

The image display method of the present application will be described below by way of a specific example.

Referring to fig. 36 to 39, an endoscope image pickup main unit of the endoscope image pickup system is connected to a display and an endoscope handle provided with a camera. Triggering a key centering and amplifying function by a user, acquiring video current frame data by an endoscope camera system through a camera, extracting a region containing an endoscope image from the current frame data by the endoscope camera system, carrying out coordinate statistics of pixel points on the endoscope image region, and determining the circle center coordinates and the circle radius of the region of the endoscope image; and determining an effective amplifying area, namely a target intercepting area, according to the circle center coordinates and the circle radius, and displaying the amplifying area in a display area to realize a one-key centering amplifying process.

Specifically, the current frame image is converted into a gray level image, then the gray level image is subjected to threshold segmentation to obtain a binary image, and finally the region containing the endoscope image is included according to the binary image.

Carrying out coordinate statistics of pixel points on an image area of the sight glass, and dividing the image area into two parts for calculation, wherein one part is used for calculating the center coordinates of a circle, and the other part is used for calculating the radius of the circle; calculating the expected value of the coordinates of each row of pixel points of the endoscope image area and the expected value of the coordinates of each column of pixel points of the endoscope image area, and then calculating the expected value of the coordinates of each row and each column to obtain the circle center coordinates of the endoscope image area; the circle radius calculation method is to calculate the variance of the coordinates of each row of pixel points in the endoscope image region, and then obtain the circle radius of the endoscope image region by using the calculated variance.

The method comprises the steps of determining an effective amplifying region according to circle center coordinates and a circle radius, specifically determining whether the circle radius exceeds the half height of a display region or judging whether the circle diameter exceeds the height of the display region (the display region is a transverse rectangle), if so, inscribing a target intercepting region in an endoscope image region, and if not, circumscribed the target intercepting region in the endoscope image region, and calculating the amplifying magnification required by the left, right, upper and lower endoscope image ranges of the target intercepting region to reach the width and the height of the display region, and amplifying the target intercepting region to the whole display region according to the amplifying magnification.

The terms "first," "second," "third," "fourth," and the like in the description of the application and in the above figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented, for example, in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.

It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "and/or" for describing the association relationship of the association object, the representation may have three relationships, for example, "a and/or B" may represent: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.

It should be understood that in the description of the embodiments of the present application, plural (or multiple) means two or more, and that greater than, less than, exceeding, etc. are understood to not include the present number, and that greater than, less than, within, etc. are understood to include the present number.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of elements is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

It should also be appreciated that the various embodiments provided by the embodiments of the present application may be arbitrarily combined to achieve different technical effects.

While the preferred embodiment of the present application has been described in detail, the present application is not limited to the above embodiment, and various equivalent modifications and substitutions can be made by those skilled in the art without departing from the spirit of the present application, and these equivalent modifications and substitutions are intended to be included in the scope of the present application as defined in the appended claims.

Claims

1. A method of displaying images of an endoscopic imaging system, the endoscopic imaging system comprising a display, the display comprising a display area, the method comprising:

2. The image display method of an endoscopic image pickup system according to claim 1, wherein the endoscopic image pickup system detects a position of a center of the endoscopic image in the picked-up image in real time, and if the center of the endoscopic image does not coincide with the center of the picked-up image, the endoscopic image pickup system performs a correction process on the picked-up image and obtains a target image, comprising:

the endoscope image pickup system detects the center position of the endoscope image in each frame of the shooting image and determines a first position of the center of the endoscope image in the shooting image;

the endoscope camera system compares the first position with the center of the shot image, and if the first position is not coincident with the center of the shot image or the distance between the first position and the center of the shot image is greater than a first distance threshold, the shot image of the current frame is determined to have abnormal position;

And the endoscope image pickup system performs correction processing on the photographed image with abnormal positions to obtain the target image, wherein the center of the endoscope image is overlapped with the center of the target image in the target image.

3. The image display method according to claim 1, wherein the endoscopic imaging system performs correction processing on the captured image and obtains a target image, comprising:

the endoscope camera system determines a target interception area containing at least a part of the endoscope image from the shot image, and the center of the target interception area coincides with the center of the endoscope image;

and intercepting the photographed image according to the target intercepting region to obtain the target image.

4. The image display method according to claim 3, wherein the target cut-out area satisfies at least one of the following conditions:

the aspect ratio of the target cut-out area is the same as that of the photographed image or the display area;

the width and length of the target intercepting region are the same as those of the photographed image or the display region;

the whole endoscope image is positioned in the target interception area, or one part of the endoscope image is positioned in the target interception area, and the other part of the endoscope image is positioned outside the target interception area;

The endoscope image is an inscribed circle or an circumscribed circle of the target interception area.

5. The image display method of an endoscopic imaging system according to any one of claims 1 to 4, further comprising:

in the process that the endoscope image pickup system detects the position of the center of the endoscope image in the shooting image in real time, if the center of the endoscope image is positioned in the center area of the shooting image, the endoscope image pickup system outputs a display signal to the display according to the image information of the shooting image so as to enable the display area to display the shooting image.

6. The image display method according to claim 1, wherein the endoscopic imaging system performs correction processing on the captured image and obtains a target image, comprising:

and extracting the endoscope image from the shooting image, and generating the target image according to the endoscope image, wherein the center of the endoscope image is coincident with the center of the target image.

7. The image display method according to claim 6, wherein the endoscopic image in the target image satisfies one of the following conditions:

The endoscope image is an inscribed circle of the boundary of the target image;

a portion of the endoscope image is located within the target image;

the first display proportion of the endoscope image in the target image is larger than the second display proportion of the endoscope image in the shooting image, wherein the first display proportion is the display area proportion of the endoscope image in the target image, and the second display proportion is the display area proportion of the endoscope image in the shooting image.

8. The image display method according to any one of claims 1 to 7, wherein the endoscopic imaging system detects a position of a center of the endoscopic image in the captured image in real time, comprising:

the endoscope camera system detects the starting state of the deviation correcting function;

if the deviation correcting function is in an enabling state, the endoscope camera system detects the position of the center of the endoscope image in the shot image in real time;

and if the deviation rectifying function is in a closed state, the endoscope image pickup system outputs a display signal to the display according to the image information of the photographed image so as to enable the display area to display the photographed image.

9. The image display method according to claim 8, wherein the state of the correction function is set by one of:

the endoscope camera system obtains a setting instruction input by a user in a man-machine interaction interface of the endoscope camera system, and sets the state of the deviation correcting function according to the setting instruction;

or the endoscope camera system obtains a key instruction of an operation key of the camera by a user, and sets the state of the deviation correcting function according to the setting instruction.

10. The image display method according to any one of claims 1 to 7, wherein the endoscope image pickup system detects a position of a center of the endoscope image in the photographed image in real time, and if the center of the endoscope image is not in a center area of the photographed image, the endoscope image pickup system performs correction processing on the photographed image and obtains a target image, comprising:

the endoscope image pickup system detects the position of the center of the endoscope image in the photographed image and the size of the endoscope image in real time;

and if the center of the endoscope image is not in the center area of the shot image and the size of the endoscope image does not meet the preset magnification, the endoscope image pickup system amplifies and rectifies the shot image to obtain the target image.

11. The image display method according to claim 10, wherein the image capturing system obtains the target image after amplifying and correcting the captured image, comprising one of:

the endoscope camera system amplifies the shooting image according to the preset magnification, and then a target interception area containing at least a part of the endoscope image is determined from the shooting image, and the center of the target interception area coincides with the center of the endoscope image; intercepting the photographed image according to the target intercepting region to obtain the target image;

or the endoscope camera system sets the size of the target interception area according to the preset magnification, and determines the target interception area containing at least a part of the endoscope image from the photographed image, wherein the area of the target interception area is smaller than that of the photographed image, and the center of the endoscope image coincides with the center of the target interception area; and intercepting the photographed image according to the target intercepting region to obtain the target image.

12. The image display method according to claim 10, wherein the preset magnification is set by one of:

Setting the preset magnification according to a magnification instruction input by a user;

determining the preset magnification according to the display size parameter of the endoscope image set by a user;

setting the preset magnification according to the type of the endoscope adopted by the camera at present;

and setting the preset magnification according to the current type of the camera and the current type of the endoscope.

13. The image display method according to claim 12, wherein the preset magnification is smaller than a magnification upper limit value.

14. The image display method according to claim 10, characterized by further comprising:

and in the process that the endoscope image pickup system detects the position of the center of the endoscope image in the shot image and the size of the endoscope image in real time, acquiring the magnification setting information input by a user, updating the preset magnification in real time according to the magnification setting information, and amplifying and correcting the shot image according to the updated preset magnification to obtain the target image.

15. The image display method according to claim 10, wherein the endoscope image pickup system detects a position of a center of the endoscope image in the photographed image and a size of the endoscope image in real time, comprising:

The endoscope camera system detects the starting state of the deviation correcting and amplifying function;

if the deviation correcting and amplifying function is in an enabling state, the endoscope image pickup system detects the position of the center of the endoscope image in the photographed image and the size of the endoscope image in real time;

and if the deviation rectifying and amplifying function is in a closed state, the endoscope camera system outputs a display signal to the display according to the image information of the shot image so as to enable the display area to display the shot image, or the endoscope camera system amplifies the shot image according to the preset amplification factor and then outputs a display signal to the display so as to enable the display area to display the amplified shot image.

16. The image display method according to any one of claims 3, 4, and 11, wherein the endoscopic imaging system determining a target cut-out region including at least a part of the endoscopic image from the captured image includes:

The endoscopic imaging system determines the target cut-out region based on an aspect ratio of the display region, a center of the endoscopic image region, and the endoscopic image range, wherein the center of the target cut-out region coincides with the center of the endoscopic image region, the aspect ratio of the target cut-out region is the same as the aspect ratio of the display region, and the target cut-out region contains at least a portion of the endoscopic image.

17. The image display method according to claim 16, wherein the endoscopic image capturing system determines an endoscopic image area of the endoscopic image from the captured image, comprising one of:

the endoscope image pickup system converts the photographed image into a gray image, and performs threshold segmentation processing on the gray image according to a preset threshold value to obtain a binary image, wherein the area contained in the binary image is the endoscope image area;

or, the endoscope image capturing system performs desired distribution calculation on coordinates of each pixel point of the endoscope image in the endoscope image area to obtain coordinates of a center of the endoscope image area, and performs variance distribution calculation on coordinates of each pixel point of the endoscope image in the endoscope image area to obtain the scope of the endoscope image area.

18. A method of displaying images of an endoscopic imaging system, the endoscopic imaging system comprising a display, the display comprising a display area, the method comprising:

19. The image display method according to claim 18, wherein the endoscope image capturing system detects a size of the endoscope image in the captured image in real time, and if the size of the endoscope image does not satisfy a preset magnification, the endoscope image capturing system performs a magnification process on the captured image and obtains a target image, including:

The endoscope camera system detects the boundary of each frame of the endoscope image and determines the first magnification of the current display of the endoscope image;

comparing the first magnification with the preset magnification, and if the first magnification is different from the preset magnification or the difference between the first magnification and the preset magnification is larger than a first magnification threshold, amplifying the photographed image by the endoscope camera system to obtain the target image.

20. The image display method according to claim 18, wherein the endoscope image pickup system performs enlargement processing on the captured image and obtains a target image, comprising one of:

the endoscope camera system amplifies the shooting image according to the preset magnification, determines a target intercepting region containing at least a part of the endoscope image from the shooting image, and intercepts the shooting image according to the target intercepting region to obtain the target image;

the endoscope camera system sets the size of the target interception area according to the preset magnification, determines the target interception area containing at least a part of the endoscope image from the photographed image, intercepts the photographed image according to the target interception area to obtain the target image, wherein the area of the target interception area is smaller than that of the photographed image.

21. The image display method according to claim 20, wherein the target cut-out area satisfies at least one of the following conditions:

22. The image display method according to claim 18, wherein the endoscope image pickup system performs enlargement processing on the captured image and obtains a target image, comprising:

and extracting the endoscope image from the shooting image, and amplifying the extracted endoscope image according to the preset amplification factor to obtain the target image.

23. The image display method according to claim 22, wherein the endoscopic image in the target image satisfies one of the following conditions:

the endoscope image is an inscribed circle of the boundary of the target image;

A portion of the endoscopic image is located within the target image.

24. The image display method according to any one of claims 18 to 23, wherein the endoscopic imaging system performs enlargement processing on the captured image and obtains a target image, comprising:

the endoscope camera system acquires a first target magnification of the endoscope image in the target image of the history frame;

the endoscope camera system determines a second target magnification according to the size of the endoscope image in the shooting image of the current frame;

if the difference value between the second target magnification and the first target magnification is larger than a first preset value, the endoscope camera system amplifies the shot image with the second target magnification and obtains the target image;

and if the difference value between the second target magnification and the first target magnification is smaller than or equal to the first preset value, the endoscope camera system amplifies the shooting image with the first target magnification and obtains the target image.

25. The image display method according to any one of claims 18 to 23, wherein the endoscopic imaging system performs enlargement processing on the captured image and obtains a target image, comprising:

the endoscope camera system calculates the size of the endoscope image in the shooting image of the current frame, and carries out confidence analysis on the calculation result of the size of the endoscope image to obtain target confidence;

if the target confidence is higher than a second preset value, the endoscope camera system determines a second target magnification according to the calculation of the size of the endoscope image, and amplifies the shot image with the second target magnification to obtain the target image;

and if the target confidence level is lower than or equal to the second preset value, the endoscope camera system amplifies the shot image with the first target magnification and obtains the target image.

26. The image display method according to any one of claims 18 to 23, wherein the preset magnification is set by one of:

setting the preset magnification according to the display size parameter of the endoscope image set by a user;

And setting the preset magnification according to the type of the endoscope adopted by the camera at present.

27. The image display method according to claim 26, wherein the type of the endoscope is determined by:

the type of the endoscope is determined according to the size, area ratio or optical magnification of the endoscope image in the photographed image.

28. The image display method according to claim 26, wherein the setting of the preset magnification according to the type of endoscope currently employed by the camera includes at least one of:

if the type of the endoscope is a first type of endoscope, setting the preset magnification to be a third target magnification, wherein the first target magnification is used for magnifying the endoscope image into an inscribed circle of the target image;

and if the type of the endoscope is a second type of endoscope, setting the preset magnification to be a fourth target magnification, wherein the second target magnification is used for magnifying the endoscope image into a circumscribed circle of the target image.

29. The image display method according to any one of claims 18 to 23, wherein the preset magnification is set by:

Acquiring the type of the camera;

obtaining the type of an endoscope adopted by the camera;

and determining the preset magnification according to the type of the camera and the type of the endoscope.

30. The image display method according to claim 24, wherein the preset magnification is smaller than a magnification upper limit value.

31. The image display method according to any one of claims 18 to 23, wherein in detecting the size of the endoscopic image in real time by the endoscopic image pickup system, magnification setting information input by a user is acquired, and the preset magnification is updated in real time based on the magnification setting information.

32. The image display method according to claim 18, wherein the endoscope image pickup system detecting a size of the endoscope image in the picked-up image in real time includes:

the endoscope camera system detects the starting state of the amplifying function;

if the amplifying function is in an enabling state, the endoscope image pickup system detects the position of the center of the endoscope image in the photographed image and the size of the endoscope image in real time;

And if the amplifying function is in a closed state, the endoscope camera system outputs a display signal to the display according to the image information of the shot image so as to enable the display area to display the shot image, or the endoscope camera system amplifies the shot image according to the preset amplifying rate and then outputs a display signal to the display so as to enable the display area to display the amplified shot image.

33. The image display method according to claim 32, wherein the state of the zoom-in function is set by one of:

the endoscope camera system obtains a setting instruction input by a user in a man-machine interaction interface of the camera system, and sets the state of the amplifying function according to the setting instruction;

or the endoscope camera system obtains a key instruction of an operation key of the camera by a user, and sets the state of the amplifying function according to the setting instruction.

34. A method of displaying images of an endoscopic imaging system, the endoscopic imaging system comprising a display, the display comprising a display area, the method comprising:

35. A method of displaying images of an endoscopic imaging system, the endoscopic imaging system comprising a display, the display comprising a display area, the method comprising:

36. The image display method of an endoscopic imaging system according to claim 35, wherein: the endoscope camera system obtains a centering amplifying operation instruction of a user, determines a target interception area containing at least a part of the endoscope image from the photographed image, and comprises the following steps:

37. An endoscope camera host, comprising:

a housing;

the processor is further configured to perform the image display method of the endoscopic imaging system of any one of the above claims 1 to 34.

38. An endoscopic camera system comprising a light source, a camera, a display, and the endoscopic camera host of claim 37, the endoscopic camera host being connected to the display by a video cable.