CN114760457B

CN114760457B - Automatic 2D/3D image switching method and system of 3D display system

Info

Publication number: CN114760457B
Application number: CN202011583911.3A
Authority: CN
Inventors: 何建行; 吴伟森; 邵文龙; 薛若雪; 林森伟; 伏继光
Original assignee: Bioisland Laboratory
Current assignee: Bioisland Laboratory
Priority date: 2020-12-28
Filing date: 2020-12-28
Publication date: 2024-03-26
Anticipated expiration: 2040-12-28
Also published as: CN114760457A

Abstract

The embodiment of the disclosure discloses a 2D/3D image automatic switching method and system of a 3D display system, comprising the following steps: acquiring video data; processing the video data to obtain an L image signal and an R image signal; performing time domain analysis on the L image signal and the R image signal, calculating fuzzy values of the L image and the R image, and judging whether the L image and the R image accord with the image normal standard or not based on a comparison result of the fuzzy values and a preset threshold value; processing the L image signal and the R image signal to generate a 3D image output in response to both of the L image and the R image meeting the image normal standard; and responding to one path of the L image or the R image to meet the image normal standard, and forming one path of image signals corresponding to the image meeting the image normal standard into a 2D image for output. So that even if the 3D endoscope lens is partially blocked or stained, the automatic switching of the 2D/3D image can be realized, and the clear 3D image or 2D image can be output.

Description

Automatic 2D/3D image switching method and system of 3D display system

Technical Field

The disclosure relates to the technical field of image processing, in particular to a 2D/3D image automatic switching method and system of a 3D display system.

Background

The 3D display technology divides a display image into two by using parallax between left and right eyes of a person, and provides the left and right images to the left and right eyes respectively in different ways, so that the left and right images are self-mixed into a 3D image in the brain of an observer. Conventionally, 3D display technology is one of the directions of active exploration and pursuit in the industry. The naked eye 3D display technology is also called as a "naked eye multi-viewpoint" technology, so that a realistic 3D effect can be created without wearing external auxiliary tools such as 3D glasses or helmets, and adverse reactions such as malignancy, dizziness, visual fatigue and the like caused by the immersion experience can be effectively avoided, and the naked eye 3D display technology is well seen and touted by various application fields (such as electronic products, advertisement media, demonstration teaching, exhibition and the like).

In the medical field, the application of a 3D or naked eye 3D display system combined with a 3D endoscope can enable the accuracy and efficiency of a plurality of minimally invasive surgeries to be higher, reduce the complexity of the surgeries and enable new doctors to learn the threshold of entering the minimally invasive surgeries, so that the minimally invasive surgeries are widely focused.

However, in performing, for example, a minimally invasive surgery using a 3D endoscope combined with a 3D display system, when an organ tissue is operated on by a high-frequency electric knife, laser, etc., smoke, water vapor, etc. are often generated, which makes the lens of the 3D endoscope easily have problems of partial occlusion, contamination, etc., resulting in blurred images in images output from the 3D endoscope, and thus, the picture of the surgery cannot be normally displayed. According to the past, because of lacking the technology of making automatic identification judgment on the image display content, once the situation occurs, a doctor needs to subjectively judge whether the 3D endoscope has a problem, and whether the 3D endoscope needs to be taken out for purification treatment to remove the blur or the objective lens of the 3D endoscope is replaced, thereby interrupting the operation and delaying the operation time, and increasing the risk of the operation.

In the 3D display system, a 3D endoscope apparatus and a 3D image processing apparatus have been developed, for example, the publication number CN110944566a. Wherein the 3D endoscope device and the 3D image processing device comprise an image synthesis part which is used for carrying out filling synthesis on the L image signal and the R image signal to generate a synthesized image with reduced fuzzy areas when detecting that one image and/or two images have fuzzy areas in the L image signal and the R image signal. However, the above method excessively uses an image processing algorithm, such as a fuzzy region filling synthesis technology, so that the affine transformation of the picture forms a pseudo-depth picture, a distortion effect exists, the interference is brought to the operation, and the time delay of the picture output is increased due to the excessively complex operation algorithm, so that the real-time performance of the system is reduced.

Disclosure of Invention

In order to solve the problems in the related art, embodiments of the present disclosure provide a 2D/3D image automatic switching method, system, electronic device, and readable storage medium for a 3D display system.

In a first aspect, an embodiment of the present disclosure provides a method for automatically switching 2D/3D images of a 3D display system, including:

acquiring video data;

Processing the video data to obtain an L image signal and an R image signal;

performing time domain analysis on the L image signal and the R image signal, calculating fuzzy values of the L image and the R image, and judging whether the L image and the R image accord with image normal standards or not based on a comparison result of the fuzzy values and a preset threshold value;

processing the L image signal and the R image signal to generate a 3D image output in response to both of the L image and the R image meeting an image normal standard; or alternatively

And in response to one path of the L image or the R image accords with the image normal standard, forming one path of image signals corresponding to the image accord with the image normal standard into a 2D image and outputting the 2D image.

With reference to the first aspect, in a first implementation manner of the first aspect, the acquiring video data is implemented as:

and acquiring the video data by adopting a binocular camera.

With reference to the first implementation manner of the first aspect, in a second implementation manner of the first aspect, the processing the video data to obtain an L image signal and an R image signal is implemented as:

the video data is accessed into a processing unit via at least one signal interface, which decodes the video data to obtain the L image signal and the R image signal.

With reference to the second implementation manner of the first aspect, in a third implementation manner of the first aspect, the processing unit decodes the video data, and is implemented as:

the processing unit decodes the video data, and outputs a decoding signal corresponding to the video data which belongs to the highest priority and effectively exists based on priority setting of the signal interface and effectiveness judgment of the video data, so as to obtain a first image signal and a second image signal;

and determining the L image signal and the R image signal by performing matching judgment of LR images on the first image signal and the second image signal.

With reference to the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the determining the L image signal and the R image signal by performing a matching determination of an LR image on the first image signal and the second image signal is implemented as:

graying processing is carried out on the first image signal and the second image signal, so that a first gray image signal and a second gray image signal are obtained;

assuming that the center coordinates of the first gray scale image signal are (X1, Y1), selecting a first matrix E1 by taking the center coordinates of the first gray scale image signal as an origin, and mapping the first matrix E1 to the center point of the second gray scale image signal to obtain a mapping matrix E1', wherein the center coordinates of the mapping matrix E1' are (X1 ', Y1);

Sliding the mapping matrix E1' leftwards and rightwards along the X direction of the second gray level image signal, performing image matching to obtain a matching matrix E2 of the second gray level image signal, and determining the central coordinates (X2, Y2) of the matching matrix E2, wherein Y1 = Y2;

based on the positional relationship of the mapping matrix E1' and the matching matrix E2 in the X direction, the correspondence between the first image signal and the second image signal and the L image signal and the R image signal, respectively, is determined,

under the condition that X1' > X2, determining the first image signal as an L image signal and the second image signal as an R image signal;

under the condition that X1' < X2, the first image signal is determined to be an R image signal, and the second image signal is determined to be an L image signal.

With reference to the first aspect, the second implementation manner of the first aspect, the third implementation manner of the first aspect, and the fourth implementation manner of the first aspect, in a fifth implementation manner of the first aspect, the performing a temporal analysis on the L image signal and the R image signal is implemented as:

performing image FIFO buffer processing on the L image signal or the R image signal to obtain two paths of image signals;

preprocessing one of the two paths of image signals to obtain preprocessed image signals; and

And performing three-level buffer processing on the other image signal in the two paths of image signals to obtain a buffered image signal.

With reference to the fifth implementation manner of the first aspect, in a sixth implementation manner of the first aspect, the preprocessing includes: de-interlacing processing and gamut conversion processing.

With reference to the fifth implementation manner or the sixth implementation manner of the first aspect, in a seventh implementation manner of the first aspect, the disclosure performs three-level buffering processing on the other path of image signal by adopting a mode of interval specific frame buffering, so as to obtain a buffered image signal.

With reference to the fifth implementation manner or the sixth implementation manner of the first aspect, in an eighth implementation manner of the first aspect, the calculating blur values of the L image and the R image is implemented as:

graying the buffered image signal to obtain a graying image, and calculating a fuzzy value of the graying image of each frame, wherein the calculating formula of the fuzzy value s is as follows:

wherein f (x, y) corresponds to the gray value of the coordinate (x, y), f (x+1, y), f (x, y+1), f (x-1, y), f (x, y-1) corresponds to the gray value of the coordinate (x+1, y), (x, y+1), (x-1, y), and (x, y-1) adjacent to the coordinate (x, y) in the up-down, left-right direction, respectively, and M, N represents the row and column size of one frame of image;

Based on the calculated fuzzy value LSi or RSi of each path of image, comparing with a preset threshold SY:

if any one of the fuzzy values LSi or RSi of each path of image is larger than or equal to a preset threshold SY, confirming that the path of image signal is normal, otherwise, confirming that the path of image signal is fuzzy, wherein LSi or RSi is the fuzzy value of any frame of gray-scale image in the L image signal or the R image signal respectively, and the value range of i is a natural number ranging from 1 to 3.

With reference to the eighth implementation manner of the first aspect, in a ninth implementation manner of the first aspect, the disclosure further includes: and in response to the fact that the L image signal and the R image signal are normal in the blur value judgment, performing correlation detection on the L image signal and the R image signal.

With reference to the ninth implementation manner of the first aspect, in a tenth implementation manner of the first aspect, the performing correlation detection on the L image signal and the R image signal is implemented as:

sequentially taking the fuzzy values of each path of image as subtraction operation based on the calculated fuzzy values LSi and RSi of any frame of the gray-scaled image of the L image signal and the R image signal, and dividing the absolute value of the obtained difference value by the maximum value of the two paths of image fuzzy values;

If the calculated result is smaller than the correlation value, confirming that the L image signal and the R image signal both accord with the image normal standard;

if the calculation result is larger than or equal to the correlation value, the path of image signal with the minimum fuzzy value is confirmed to be not in accordance with the normal image standard.

With reference to the ninth implementation manner of the first aspect, in an eleventh implementation manner of the first aspect, the present disclosure further includes:

comparing the fuzzy values LSi of the gray-scale images, judging abnormal frame numbers under the condition that the LSi difference exceeds a specific threshold value, and discarding L paths of corresponding frames of the preprocessed image signals, wherein the value range of i is a natural number ranging from 1 to 3;

comparing the blurred value RSi of the gray-scale image, judging the abnormal frame number under the condition that the LSi difference exceeds a specific threshold value, and discarding the corresponding frame of the R paths of preprocessed image signals, wherein the value range of i is a natural number ranging from 1 to 3.

With reference to the eighth implementation manner, the ninth implementation manner, the tenth implementation manner, and the first implementation manner of the first aspect, in a twelfth implementation manner of the first aspect,

in response to both the L image and the R image meeting an image normal standard, processing the L image signal and the R image signal to generate a 3D image output is implemented as:

And generating a 3D image signal by the L paths of the preprocessed image signals and the R paths of the preprocessed image signals in response to that both the L image and the R image meet the image normal standard, and outputting the 3D image signal.

With reference to the twelfth implementation manner of the first aspect, in a thirteenth implementation manner of the first aspect, before the processing the L image signal and the R image signal to generate the 3D image output in response to both of the L image and the R image conforming to the image normal standard, the method further includes: and performing time sequence synchronization processing on the L image signal and the R image signal.

With reference to the eighth implementation manner, the ninth implementation manner, the tenth implementation manner, and the first implementation manner of the first aspect, in a fourteenth implementation manner of the first aspect,

in response to one path of the L image or the R image meeting the image normal standard, forming one path of image signals corresponding to the image meeting the image normal standard into a 2D image for output, wherein the 2D image is implemented as follows:

and responding to one path of the L image or the R image to meet the image normal standard, and forming the preprocessed image signal of the path meeting the image normal standard into a 2D image for outputting.

With reference to the first aspect, in a first implementation manner, a third implementation manner, a fourth implementation manner, a fifth implementation manner, a sixth implementation manner, and a ninth implementation manner of the first aspect, in a tenth implementation manner or an eleventh implementation manner, in a fifteenth implementation manner of the first aspect, the disclosure sends an alarm to indicate that a problem occurs in an image signal in response to one of an L image or an R image not conforming to an image normal standard.

With reference to the first aspect, in a first implementation manner, a third implementation manner, a fourth implementation manner, a fifth implementation manner, a sixth implementation manner, and a ninth implementation manner of the first aspect, in a tenth implementation manner or an eleventh implementation manner, in a fifteenth implementation manner of the first aspect, the disclosure sends an alarm signal to prompt to process an input signal source in response to any one of an L image and an R image not conforming to an image normal standard.

In a second aspect, in an embodiment of the present disclosure, there is provided a 2D/3D image automatic switching system of a 3D display system, including:

an acquisition module configured to acquire video data;

a processing module configured to process the video data to obtain an L image signal and an R image signal;

The analysis and calculation module is configured to perform time domain analysis on the L image signal and the R image signal and calculate fuzzy values of the L image and the R image;

the judging module is configured to judge whether the L image and the R image accord with the image normal standard or not based on the comparison result of the fuzzy value and the preset threshold value;

a 3D image generation module configured to process the L image signal and the R image signal to generate a 3D image in response to both of the L image and the R image conforming to an image normal standard;

a 2D image generation module configured to generate a 2D image from one path of image signals corresponding to an image conforming to an image normal standard in response to one path of the L image or the R image conforming to the image normal standard; and

and an output module configured to output the 3D image signal or the 2D image signal.

With reference to the second aspect, in a first implementation manner of the second aspect, the processing module includes: a processing unit and a matching module, wherein the processing unit is used for matching the processing unit,

the processing unit is configured to receive the video data and decode the video data to obtain the L image signal and the R image signal.

With reference to the first implementation manner of the second aspect, in a second implementation manner of the second aspect, the processing unit is further configured to decode and output a signal corresponding to an access signal belonging to the highest priority and effectively existing, so as to obtain a first image signal and a second image signal;

The matching module is configured to perform LR image matching judgment on the first image signal and the second image signal, and determine the L image signal and the R image signal.

With reference to the first implementation manner and the second implementation manner of the second aspect, in a third implementation manner of the second aspect, the matching module includes:

the first graying sub-module is configured to carry out graying processing on the first image signal and the second image signal to obtain a first gray image signal and a second gray image signal;

a matching sub-module configured to match the first grayscale image signal with the second grayscale image signal; and

and the judging sub-module is configured to determine the corresponding relation between the first image signal and the second image signal and the L image signal and the R image signal respectively according to the matching relation.

With reference to the second aspect, in a first implementation manner and a second implementation manner of the second aspect, in a fourth implementation manner of the second aspect, the analysis and calculation module includes:

the FIFO buffer module is configured to perform image FIFO buffer processing on the L image signal or the R image signal to obtain two paths of image signals;

The preprocessing module is configured to preprocess one image signal of the two image signals to obtain preprocessed image signals; and

and the multi-level buffer module is configured to perform multi-level buffer processing on the other image signal in the two paths of image signals to obtain a buffered image signal.

With reference to the fourth implementation manner of the second aspect, in a fifth implementation manner of the second aspect, the analysis and calculation module further includes:

a graying second sub-module configured to graying the buffered image signal to obtain a grayed image;

and a calculating sub-module configured to calculate a blur value of the graying image per frame.

With reference to the fifth implementation manner of the second aspect, in a sixth implementation manner of the second aspect, the present disclosure further includes: and the correlation detection module is configured to perform correlation detection on the L image signal and the R image signal in response to the L image signal and the R image signal being normal in the blur value judgment.

With reference to the fifth implementation manner of the second aspect, in a seventh implementation manner of the second aspect, the present disclosure further includes:

and the comparison module is configured to compare the fuzzy value of the gray-scale image.

With reference to the second aspect, in a first implementation manner, a second implementation manner, a fifth implementation manner, a sixth implementation manner, or a seventh implementation manner of the second aspect, in an eighth implementation manner of the second aspect, the disclosure further includes: the first alarm module is configured to send an alarm to indicate that one path of image signals is in question in response to one path of image signals in the L image or the R image not meeting the normal image standard.

With reference to the second aspect, in a first implementation manner, a second implementation manner, a fifth implementation manner, a sixth implementation manner, or a seventh implementation manner of the second aspect, in a ninth implementation manner of the second aspect, the disclosure further includes: and the second alarm module is configured to send an alarm signal to prompt an operator to process the input signal source in response to any one of the L image and the R image not conforming to the image normal standard.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including a memory and a processor, where the memory is configured to store one or more computer instructions, where the one or more computer instructions are executed by the processor to implement the method according to any one of the first aspect, the first to thirteenth implementations of the first aspect.

In a fourth aspect, in an embodiment of the present disclosure, there is provided a computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the method of the first aspect, the first to thirteenth implementation forms.

According to the technical scheme provided by the embodiment of the disclosure, video data are acquired; processing the video data to obtain an L image signal and an R image signal; performing time domain analysis on the L image signal and the R image signal, calculating fuzzy values of the L image and the R image, and judging whether the L image and the R image accord with image normal standards or not based on a comparison result of the fuzzy values and a preset threshold value; processing the L image signal and the R image signal to generate a 3D image output in response to both of the L image and the R image meeting an image normal standard; or in response to one path of the L image or the R image meeting the image normal standard, forming one path of image signals corresponding to the image meeting the image normal standard into a 2D image and outputting the 2D image. On the one hand, even under the condition that the 3D endoscope lens is partially blocked or stained, the 3D display system can realize automatic switching of the 2D/3D image, so that clear 3D image or 2D image can be output or an alarm signal can be sent out, the subjective evaluation process of an observer can be reduced, the accuracy and efficiency of 3D endoscope operation are improved, and the operation risk is reduced. On the other hand, by proposing a fuzzy value judgment theory, whether the image signal meets the image normal standard or not can be accurately judged, and the excessive use of an image processing algorithm is avoided, so that the image meeting the image normal standard can be output in real time in a clear 2D or 3D image mode, and the instantaneity of the system is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

Other features, objects and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:

fig. 1 illustrates a flowchart of a 2D/3D image auto-switching method of a 3D display system according to an embodiment of the present disclosure;

fig. 2 illustrates a flowchart of processing the video data to obtain an L image signal and an R image signal according to an embodiment of the present disclosure;

FIG. 3 illustrates a flowchart of LR image matching determination for a first image signal and a second image signal according to an embodiment of the present disclosure;

fig. 4a shows a schematic diagram of an L image signal and an R image signal according to a gray overlap region definition according to an embodiment of the present disclosure;

fig. 4B shows a schematic diagram of a matching determination that an a-way image signal is an L-way image signal and a B-way image signal is an R-way image signal according to an embodiment of the present disclosure;

fig. 4c shows a schematic diagram of a matching determination that an a-way image signal is an R-way image signal and a B-way image signal is an L-way image signal according to an embodiment of the present disclosure;

FIG. 5 illustrates a flow chart of temporal analysis of L and R image signals according to an embodiment of the present disclosure;

FIG. 6 illustrates a block diagram of a 2D/3D image auto-switching system according to an embodiment of the present disclosure;

fig. 7 is a block diagram showing the configuration of a processing module of the 2D/3D image automatic switching system according to the embodiment shown in fig. 6;

FIG. 8 shows a block diagram of the matching module of the processing module according to the embodiment shown in FIG. 7;

FIG. 9 is a block diagram showing the configuration of an analysis calculation module of the 2D/3D image automatic switching system according to the embodiment shown in FIG. 6;

fig. 10 illustrates a block diagram of a 2D/3D image automatic switching system according to another embodiment of the present disclosure;

FIG. 11 illustrates a block diagram of an electronic device according to an embodiment of the disclosure;

fig. 12 shows a schematic diagram of a computer system suitable for use in implementing the 2D/3D image auto-switching method according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, portions irrelevant to description of the exemplary embodiments are omitted in the drawings.

In this disclosure, it should be understood that terms such as "comprises" or "comprising," etc., are intended to indicate the presence of features, numbers, steps, acts, components, portions, or combinations thereof disclosed in this specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, acts, components, portions, or combinations thereof are present or added.

In addition, it should be noted that, without conflict, the embodiments of the present disclosure and features of the embodiments may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

In the medical field, in performing, for example, a minimally invasive surgery using a 3D endoscope combined with a 3D display system, smoke, water vapor, etc. are often generated when an organ tissue is operated on by a high-frequency electric knife, laser, etc., which makes a 3D endoscope lens easily have problems of partial occlusion, contamination, etc., resulting in a blurred image in an image output from the 3D endoscope, and an operation screen cannot be normally displayed. Due to the lack of a technology for automatically identifying and judging the image display content, once the situation occurs, an operator needs to subjectively judge whether the 3D endoscope has a problem or not, and whether the 3D endoscope needs to be taken out for purification treatment to remove the blur or the objective lens of the 3D endoscope needs to be replaced, so that the operation is interrupted, the operation time is delayed, and the risk of the operation is increased.

In the 3D display system of the prior art, a 3D endoscope apparatus or a 3D image processing apparatus has been developed, for example, the publication number CN110944566a. Because the image processing algorithm is excessively used, for example, a fuzzy region filling synthesis technology is adopted, the picture affine transformation forms a false depth picture, the distortion effect exists, the interference is brought to the operation, and the time delay of picture output is increased due to the excessively complex operation algorithm, so that the real-time performance of the system is reduced.

According to the technical scheme provided by the embodiment of the disclosure, the input video stream data is processed to obtain left and right eye image signals, namely an L image signal and an R image signal, the left and right eye image signals are analyzed by adopting a time domain, the blur values of the left and right eye images (namely the L image and the R image) are calculated, and whether the left and right eye images accord with the normal image standard or not is judged based on the calculated blur values and a preset threshold value, so that whether the 3D image signal, the 2D image signal or the alarm signal is output is judged. Therefore, even if the 3D endoscope lens is partially shielded or stained, the automatic switching output of the 2D/3D image can be realized, the clear 3D image or 2D image is ensured to be displayed for an operation observer, the subjective evaluation process is reduced, the accuracy and the efficiency of the 3D endoscope operation can be improved, and the operation risk is reduced. In addition, the adoption of the fuzzy value judging method is innovatively proposed, so that the image distortion or image output delay caused by excessive use of a complex operation algorithm can be avoided, and the instantaneity of the system is improved.

Fig. 1 illustrates a flowchart of a 2D/3D image auto-switching method of a 3D display system according to an embodiment of the present disclosure.

As shown in fig. 1, the 2D/3D image automatic switching method includes the following steps S101 to S105:

in step S101, video data is acquired;

because the principle of the 3D display technology is to generate two left and right images with vision difference, the left and right eye views can be separated by the light splitting of the parallax image by the display device, so as to reach the left eye and the right eye of a person respectively, and the visual nerves of the brain feel depth and distance through the difference between the visual angles of the two eyes, so that a vivid virtual stereoscopic picture with a spatial depth sense is obtained. In the medical field, the requirement for virtual stereoscopic images is more stringent when using 3D endoscopes for, for example, minimally invasive surgery.

Therefore, according to the technical scheme provided by the embodiment of the disclosure, the binocular camera can be adopted to shoot from different visual angles so as to obtain high-quality video data, so that the manufacturing cost can be considered while the high-quality video stream can be obtained.

In step S102, the video data is processed to obtain an L image signal and an R image signal, the video data is acquired and input to a processing unit through a phase acquisition device, such as a binocular camera, and the video data is decoded by the processing unit to form the L image signal and the R image signal;

In step S103, performing time domain analysis on the L image signal and the R image signal, calculating blur values of the L image and the R image, and determining whether the L image and the R image meet the image normal standard based on a comparison result of the blur values and a preset threshold;

according to the technical scheme provided by the embodiment of the disclosure, the video data is processed by acquiring high-quality video data so as to acquire left-eye and right-eye image signals. Based on the processed (e.g. decoded) image signal, the processed image signal is analyzed in time domain dimension, such as in a certain time sequence (the previous frame and the next frame, the previous 5 frame, the current frame and the next 5 frame, etc.), the blur value of the left and right eye images can be calculated through the comparative analysis of the image information of the previous and the next frames in the time sequence, the blur value is judged and compared with the preset threshold value, and finally whether the left and right eye images meet the image normal standard is determined

In step S104, in response to both of the L image and the R image meeting an image normal standard, processing the L image signal and the R image signal to generate a 3D image output;

if the left eye image and the right eye image both accord with the image normal standard, the backlight and the time sequence control unit of the display device can be correspondingly set according to the principle of pointing to the backlight, and the 3D image is output and displayed. For example, a method of controlling 476 columns of backlight lamps according to binocular different vision areas can be adopted to display 3D images by matching with an optical module based on a directional backlight principle. Here, it will be understood by those skilled in the art that the method for forming the 3D image signal from the L image signal and the R image signal conforming to the normal standard of the image may be selected from the existing technologies known in the art, such as lenticular technology, parallax barrier technology, or MLD technology, etc., and any existing technology in the art capable of achieving the above-mentioned 3D display purpose may be used, and the disclosure will not be described in detail herein.

Alternatively, in step S105, in response to one of the L image or the R image conforming to the image normal standard, one image signal corresponding to the image conforming to the image normal standard is formed into a 2D image output.

If only one of the left and right eye images accords with the image normal standard, forming one image signal (such as an L image signal or an R image signal) corresponding to the image which accords with the image normal standard into a 2D image for outputting and displaying. For example, the control module may automatically control the display unit to switch to the 2D display mode, and output and display the 2D image by controlling the backlight and the timing control unit of the display device. It should be understood by those skilled in the art that the method for forming the 2D image output signal from one of the L image signal or the R image signal conforming to the normal standard of the image may be implemented by using image signal processing techniques existing in the art, and any prior art in the art for achieving the above purpose of 2D image output may be used, which will not be described in detail herein.

According to the technical scheme provided by the embodiment of the disclosure, even under the condition that the 3D endoscope lens is partially blocked or stained, the automatic switching output of the 2D/3D image can be realized, the clear 3D image or 2D image is ensured to be displayed for an operation observer, the subjective evaluation process is reduced, the accuracy and the efficiency of the 3D endoscope operation can be improved, and the operation risk is reduced. In addition, as the fuzzy value judgment theory is innovatively adopted, the simplified judgment process can avoid the distortion of the picture or the delay of the picture output caused by excessive use of a complex operation algorithm, and the real-time performance of the system is improved.

Fig. 2 illustrates a flowchart of processing the video data to obtain an L image signal and an R image signal according to an embodiment of the present disclosure.

As shown in fig. 2, the method of processing the video data to acquire an L image signal and an R image signal may include S201 to S203;

since 3D endoscopes have been widely used in various medical institutions, research centers, or laboratories, video data acquired through a camera device may be accessed into a processing unit through various types of data interfaces, such as HDMI, DVI, SDI and/or RCA interfaces, etc., in consideration of differences in hardware conditions of various application environments. In order to ensure universality and reliability of a signal interface, the video data is accessed into a processing unit through at least one signal interface of HDMI, DVI or SDI.

In step S201, video data may be selectively accessed into a processing unit through at least one signal interface of HDMI, DVI, SDI, or the like, where the processing unit may be implemented by CPU, DSP, FPGA, ASIC or may be implemented by other means, which is not limited in this disclosure.

In step S202, the processing unit may decode the video data accessed by the signal interfaces such as HDMI, DVI, SDI, for example, may call the corresponding signal processing decoder to decode the three video data respectively, so as to obtain the L image signal and the R image signal.

When the above three signals are decoded and output, two or three of the above three signal interfaces may be adopted as the signal interfaces, such as HDMI and DVI, or HDMI and SDI, or HDMI, DVI and SDI, and when decoding and outputting, only one decoded signal corresponding to one signal interface is required to be output, so the priority mode may be preset for the above three signal interfaces according to the difference of the transmission effects of the various signal interfaces, so that the video data of the signal interface corresponding to the highest priority is decoded, for example, the HDMI is set to be the highest priority, the DVI is set to be the next highest priority, and the SDI is set to be the lowest priority.

In addition, in order to avoid abnormity of part of the signal interface caused by human factors (such as non-tightening of the signal interface when the signal interface is plugged or loosening of the connector in the moving or carrying process of the device) or other external factors (such as damage of a data line or a connector of the signal interface), the reliability of video data transmission is ensured by judging the validity of the signal accessed by the signal interface, namely judging whether the signal is valid or not.

In step S203, based on the priority setting order and the signal validity judgment result, a decoded signal corresponding to the access signal belonging to the highest priority and effectively existing is output, and two paths of image signals, i.e., a first image signal and a second image signal, are obtained. Considering that the types of the signal interfaces used are different, the number of the corresponding connectors is also different, in general, the HDMI and the DVI have only one connector, the SDI has two connectors, and two paths of image signals, namely a first image signal and a second image signal, can be formed after video data accessed through the HDMI and the DVI or the SDI connectors are decoded and output.

According to the technical scheme provided by the embodiment of the disclosure, various problems can occur in consideration of a practical application scene, for example, a user does not accurately judge a joint error caused by the position of a joint corresponding interface or connects two joints such as an SDI reversely, or two paths of image signals formed after decoding cannot directly determine which path of image signals corresponds to an L image signal and which path of image signals corresponds to an R image signal due to other factors such as transmission characteristics of various signal joints. Therefore, image signal reality distortion or false alarm is often caused, and in order to solve the above technical problems, the disclosure proposes an LR image matching judgment principle to perform matching judgment on left and right images.

Fig. 3 shows a flowchart of LR image matching determination on a first image signal and a second image signal according to an embodiment of the present disclosure.

As shown in fig. 3, the process of performing LR image matching judgment on the first image signal and the second image signal includes S301 to S304;

firstly, assume that two paths of image signals (namely a first image signal and a second image signal) output through decoding can generate gray overlapping areas in an image display area, and define an L image signal and an R image signal according to the gray overlapping areas, as shown in fig. 4 a;

in step S301, the first image signal and the second image signal are subjected to graying processing, so as to obtain a first gray scale image signal and a second gray scale image signal, which correspond to the a-path image signal and the b-path image signal shown in fig. 4b or fig. 4c, respectively;

in step S302, assuming that the center coordinates of the a-path image signal are (X1, Y1), selecting a first matrix E1, for example, a 5*5 matrix, with the center coordinates (X1, Y1) of the a-path image signal as the origin, and mapping the first matrix E1 to the center point where the b-path image signal is located to obtain a mapping matrix E1', wherein the center coordinates of the mapping matrix E1' are (X1 ', Y2), y1=y2;

In step S303, a spatial two-dimensional sliding method is adopted to slide the mapping matrix E1' left and right along the X direction of the b-path image signal, and after the matching is successful, a matching matrix E2 of the b-path image signal can be obtained, and the central coordinate of the matching matrix E2 is determined to be (X2, Y2), wherein y1=y2;

in step S304, based on the positional relationship between the mapping matrix E1' and the matching matrix E2 along the X direction, the correspondence between the first image signal and the second image signal and the L image signal and the R image signal, respectively, is determined. Wherein,

when X2 falls to the left of X1', it can be determined that the first image signal is an L image signal and the second image signal is an R image signal, as shown in FIG. 4 b;

on the contrary, when X2 falls to the right of X1', it can be determined that the first image signal is an R image signal, the second image signal is an L image signal, as shown in fig. 4c,

namely:

According to the technical scheme of the embodiment of the disclosure, through the LR image judging method, the corresponding relation between the two paths of decoded output image signals and the left-eye image signals and the right-eye image signals can be automatically matched quickly and accurately, so that phenomena such as artificial joint reverse connection, signal disorder distortion caused by individual difference of signal joint characteristics and the like do not need to be considered, great convenience is brought to users, and the degree of automation is improved.

Fig. 5 illustrates a flowchart of temporal analysis of L image signals and R image signals according to an embodiment of the present disclosure.

As shown in fig. 5, the method of performing time domain analysis on the L image signal and the R image signal includes S501 to S503;

in step S501, first, an image FIFO buffer process is performed on an L image signal or an R image signal to obtain two paths of image signals;

in step S502, one of the two image signals is preprocessed, so as to obtain a preprocessed image signal. The preprocessing comprises de-interlacing processing, and the interlaced video is converted into progressive video, so that vertical resolution which is approximately more than 2 times of that of interlaced display can be provided, and the ornamental value of the video is improved; the preprocessing may also include gamut conversion processing, such as converting YUV4:2:2 to GRB888;

in step S503, three-level buffering processing is performed on the other one of the two image signals, so as to obtain a buffered image signal. The manner in which specific frame buffers are spaced apart may be employed herein. Because the 3D display system is mainly applied to slowly-transformed image signals, data buffering can be performed at intervals (for example, 10 frames) so that 3 paths of image data in three-level buffering have obvious change in the dimension of a time domain.

According to still another embodiment of the present disclosure, blur values of the L image and the R image may be calculated based on a blur value calculation formula of the image.

Specifically, the higher the sharpness of the image is, the more high-frequency components in the image are considered. Embodiments of the present disclosure thus employ a neighboring pixel gray scale difference square sum variance algorithm to calculate the blur value of a frame of image. For each pixel point of a frame of image, the gray value difference of the pixel point is obtained, and then the gray value difference is normalized by using the total number of pixels, so that the intensity of the gray change of the image can be represented, wherein the higher the intensity of the gray change is, the clearer the image is, and the lower the intensity of the gray change is, the more the image is blurred.

The embodiment of the disclosure performs graying on the buffered image signal to obtain a grayed image, and calculates a fuzzy value of the grayed image of each frame, wherein a calculation formula of the fuzzy value s is as follows:

comparing the obtained fuzzy value LSi or RSi of any frame image in the left and right images with a preset threshold SY based on calculation, if any fuzzy value in the fuzzy value LSi or RSi of each image is larger than or equal to the preset threshold SY, the image signal of the road can be considered to be normal, otherwise, the image signal of the road is blurred. Namely:

If LSi is more than or equal to SY, judging that the L image is normal, and if LSi is less than SY, judging that the L image is fuzzy;

if RSi is more than or equal to SY, judging that the R image is normal, and if RSi is less than SY, judging that the R image is fuzzy;

wherein, LSi or RSi is the fuzzy value of any frame gray-scale image in L image signal or R image signal, and the value range of i is a natural number of 1-3.

According to the technical scheme provided by the embodiment of the disclosure, whether each path of image signal accords with the normal image standard can be timely and accurately judged, and excessive use of an image processing algorithm can be avoided, so that the image which accords with the normal image standard can be output in real time in a clear 2D or 3D image mode, and the instantaneity of the system is improved.

According to another embodiment of the present disclosure, considering that there is a correlation between the decoded output L image signal and the R image signal, it is possible to further check whether the image signal does meet the image normal standard by detecting the correlation of the two image signals.

According to an embodiment of the present disclosure, in response to the L image signal and the R image signal being both normal in the blur value determination, performing correlation detection on the L image signal and the R image signal is specifically implemented as:

Sequentially taking the fuzzy values of each image as subtraction operation based on the fuzzy values LSi and RSi of any frame of the image after graying of the L image signal and the R image signal obtained by the fuzzy value calculation formula, and dividing the absolute value of the obtained difference value by the maximum value of the two paths of image fuzzy values;

if the calculation result is smaller than the correlation value, for example, the correlation value can be 0.2, confirming that the L image signal and the R image signal accord with the image normal standard;

if the calculation result is greater than or equal to the correlation value, for example, the correlation value can be 0.2, it can be confirmed that the path of image signal with the minimum blur value does not meet the image normal standard.

The above-mentioned correlation detection has a judgment formula as follows:

the L image signal and the R image signal are normal, wherein the value range of i is a natural number ranging from 1 to 3;

one path of image signal of Min (LSi, RSi) is abnormal, wherein the value range of i is a natural number of 1-3.

According to the embodiment of the disclosure, through correlation detection of the L image signal and the R image signal, once the correlation value of the LR two paths of video data deviates greatly, the problem image signal is considered to exist, the abnormal image signal can be automatically identified, at the moment, the normal path of image signal is automatically switched to, effective 2D display is performed, and alarm information is output.

According to the technical scheme of the embodiment of the disclosure, the normal signal and the abnormal signal can be more accurately determined by calculating the correlation value of the two paths of image signals and adding the rechecking detection link of the video data, so that the effectiveness of the image signals is better ensured.

According to another embodiment of the disclosure, in response to the L image signal and the R image signal being normal, comparing the blurred values LSi of the grayscale images, judging an abnormal frame number under the condition that the LSi difference exceeds a specific threshold, and discarding L corresponding frames of the preprocessed image signals, wherein the value range of i is a natural number ranging from 1 to 3; comparing the blurred value RSi of the gray-scale image, judging the abnormal frame number under the condition that the LSi difference exceeds a specific threshold value, and discarding the corresponding frame of the R paths of preprocessed image signals, wherein the value range of i is a natural number ranging from 1 to 3.

Considering that the problem image signal may not be caused by lens offset and the like, but may be a bad block and the like caused by transmission errors of decoded video data, the embodiment of the disclosure may also adopt a mode of comparing frames of each path of buffered image to each other to check abnormal frames caused by data transmission problems, thereby further ensuring display effect.

According to still another embodiment of the present disclosure, before processing the L image signal and the R image signal to generate a 3D image output in response to both of the L image and the R image meeting an image normal standard, timing synchronization processing is further required for the L image signal and the R image signal based on a directional backlight module, and the output 3D image can be ensured to be normal through the timing synchronization operation, otherwise, if there is a frame error in the L image signal and the R image signal, the subsequent synthesized 3D image is not an actual 3D image. The timing synchronization method mentioned in the present disclosure is a known technology for processing image signals, and any technology capable of implementing timing synchronization may be selected, and the disclosure is not described in detail herein.

According to the embodiment of the disclosure, in response to one of the L image or the R image not conforming to the image normal standard, an alarm is issued, and the problem of the image signal of the one is indicated on a screen.

According to the embodiment of the disclosure, in response to either of the L image and the R image not conforming to the image normal standard, an alarm signal is sent to prompt an operator to process the input signal source.

Fig. 6 shows a block diagram of a 2D/3D image automatic switching system according to an embodiment of the present disclosure. Wherein the system may be implemented as part or all of an electronic device by software, hardware, or a combination of both.

As shown in fig. 6, the 2D/3D image automatic switching system 600 includes an acquisition module 601, a processing module 602, an analysis calculation module 603, a judgment module 604, a 3D image generation module 605, a 2D image generation module 606, and an output module 607.

An acquisition module 601 configured to acquire video data using, for example, a binocular camera;

a processing module 602 configured to process the video data to obtain an L image signal and an R image signal;

an analysis calculation module 603 configured to perform temporal analysis on the L image signal and the R image signal, and calculate blur values of the L image and the R image;

a judging module 604, configured to judge whether the L image and the R image meet the image normal standard based on the comparison result of the blur value and the preset threshold;

a 3D image generation module 605 configured to process the L image signal and the R image signal to generate a 3D image in response to both of the L image and the R image conforming to an image normal standard;

a 2D image generation module 606 configured to generate a 2D image from one path of image signals corresponding to an image conforming to an image normal standard in response to one path of the L image or the R image conforming to the image normal standard; and

An output module 607 configured to output a 3D image signal or a 2D image signal generated via the 3D image generation module or the 2D image generation module as described.

According to the technical scheme provided by the embodiment of the disclosure, even if the 3D endoscope lens is partially blocked or stained, the automatic switching output of the 2D/3D image can be realized, the clear 3D image or 2D image is ensured to be displayed for an operation observer, the subjective evaluation process is reduced, the accuracy and efficiency of the 3D endoscope operation can be improved, and the operation risk is reduced. In addition, the calculation analysis module and the judgment module are innovatively provided for analyzing, calculating and judging the fuzzy value of the image so as to automatically determine the image signal which accords with the normal standard of the image, thereby avoiding the picture distortion or the picture output delay caused by excessive use of a complex operation algorithm and improving the real-time performance of the system.

Fig. 7 is a block diagram showing the configuration of a processing module of the 2D/3D image automatic switching system according to the embodiment shown in fig. 6.

According to an embodiment of the present disclosure, the processing module 602 includes: a processing unit 701 and a matching module 702, where the processing unit may be implemented by CPU, DSP, FPGA, ASIC, or may be implemented by other manners, and this disclosure is not limited thereto;

The processing unit 701 is configured to receive the video data accessed by one or more signal interfaces of HDMI, DVI, or SDI, for example, and decode the video data, for example, call a signal processing decoder to decode, to obtain the L image signal and R image signal. The decoded signals of the signal interfaces corresponding to the highest priority are output through the signal processing decoder 702 in a mode of presetting priorities for the three signal interface types, for example, HDMI is set to be the highest priority, DVI is set to be the next highest priority, and SDI is set to be the lowest priority.

In addition, in order to avoid abnormity of part of the signal interfaces caused by human factors (such as loose connection or loose connection in the moving process when the signal interfaces are plugged) or other external factors (such as damage to the data wires or the connectors of the signal interfaces), validity judgment is needed to be carried out on the signals accessed by the signal interfaces, that is, whether the signals really exist effectively or not is judged.

The matching module 702 is configured to determine an L image signal and an R image signal by performing matching judgment of LR images on the first image signal and the second image signal. Therefore, the corresponding relation between the two paths of decoded output image signals and the left-eye image signals and the right-eye image signals can be automatically matched quickly and accurately, phenomena such as signal disorder distortion caused by factors such as artificial joint reverse connection or individual difference of signal joint characteristics are not needed to be considered, great convenience is brought to users, and the degree of automation is improved.

Fig. 8 shows a block diagram of the matching module of the processing module according to the embodiment shown in fig. 7.

As shown in fig. 8, the matching module 702 includes:

a graying first sub-module 801 configured to perform graying processing on the first image signal and the second image signal to obtain a first gray image signal and a second gray image signal;

a matching sub-module 802 configured to match the first grayscale image signal with the second grayscale image signal; and

a judging sub-module 803 configured to determine correspondence between the first image signal and the second image signal and the L image signal and the R image signal, respectively, according to the matching relationship.

The matching module 803 of the embodiment of the present disclosure is configured to implement the matching judgment method described in steps S301 to 303 described above.

According to the above technical solution of the embodiments of the present disclosure, by performing the matching determination on the LR image by the matching sub-module 802 and the determining sub-module 803, the correspondence between the two paths of decoded output image signals and the left-and-right eye image signals can be quickly and accurately matched, so that phenomena such as artificial joint reverse connection or signal disturbance distortion caused by individual differences of signal joint characteristics and the like do not need to be considered, great convenience is brought to users, and the degree of automation is improved.

Fig. 9 is a block diagram showing the structure of an analysis calculation module of the 2D/3D image automatic switching system according to the embodiment shown in fig. 6.

As shown in fig. 9, the analysis calculation module 603 includes:

the FIFO buffer module 901 is configured to perform image FIFO buffer processing on the L image signal or the R image signal, to obtain two paths of image signals;

a preprocessing module 902, configured to perform, for example, de-interlacing processing and/or color gamut conversion processing on one of the two image signals, to obtain a preprocessed image signal; and

the multi-level buffer module 903 is configured to perform multi-level buffer processing on the other image signal of the two image signals, so as to obtain a buffered image signal.

According to a further embodiment of the present disclosure, the analysis calculation module 603 further comprises a graying second sub-module 904 configured to graying the buffered image signal; and a calculation sub-module 905 configured to calculate a blur value of the greyscale image per frame.

In the embodiment of the disclosure, whether each path of image signal accords with the image normal standard can be timely and accurately judged, and excessive use of an image processing algorithm can be avoided, so that the image which accords with the image normal standard can be output in real time in a clear 2D or 3D image mode, and the instantaneity of the system is improved.

Fig. 10 illustrates a block diagram of a 2D/3D image auto-switching system of a 3D display system according to another embodiment of the present disclosure.

As shown in fig. 10, the 2D/3D image automatic switching system 1000 includes an acquisition module 1001, a processing module 1002, an analysis and calculation module 1003, a judgment module 1004, a correlation detection module 1005, a comparison module 1006, a 3D image generation module 1007, a 2D image generation module 1008, and an output module 1009. The configuration and/or function of the other modules except for the correlation detection module 1005 and the comparison module 1006 are identical to those of the embodiment shown in fig. 6, and the disclosure will not be described in detail herein.

The correlation detection module 1005 is configured to perform correlation detection on the L image signal and the R image signal to confirm whether the L image signal and the R image signal conform to an image normal standard in response to both the L image signal and the R image signal being normal in the blur value determination. The correlation detection module 1005 of the present disclosure may automatically execute a determination formula of correlation detection to perform correlation detection determination.

The comparing module 1006 is configured to compare blur values of the grayscale images. In response to the L image signal and the R image signal being normal, the comparing module 1006 compares the blurred values LSi of the grayscale images, determines an abnormal frame number under the condition that the LSi difference exceeds a specific threshold, and discards L corresponding frames of the preprocessed image signals, wherein the value range of i is a natural number ranging from 1 to 3; comparing the blurred value RSi of the gray-scale image, judging the abnormal frame number under the condition that the LSi difference exceeds a specific threshold value, and discarding the corresponding frame of the R paths of preprocessed image signals, wherein the value range of i is a natural number ranging from 1 to 3.

Considering that the problem image signal may not be caused by lens contamination and the like, but may be a bad block and the like caused by transmission errors of decoded video data, the embodiment of the disclosure may further check abnormal frames caused by data transmission problems by comparing frames of each path of buffered image by the comparison module, thereby further ensuring the display effect.

In the embodiment of the disclosure, the correlation detection module 1005 detects the correlation between the L image signal and the R image signal, calculates the correlation value of the LR two paths of video data, and once the correlation value deviates greatly, considers that a problem image signal exists, and can automatically identify an abnormal image signal, at this time, automatically switches to a normal path of image signal, performs effective 2D display, and outputs alarm information. By calculating the correlation value of the two paths of image signals and adding a rechecking detection link of video data, the normal signal and the abnormal signal can be more accurately determined, thereby better ensuring the effectiveness of the image signals.

According to an embodiment of the present disclosure, the 2D/3D image automatic switching system 600/1000 may further include a first alarm module configured to issue an alarm and indicate that a problem occurs in one of the L image or the R image in response to the one of the L image or the R image not conforming to the image normal standard.

And the second alarm module is configured to send an alarm signal to prompt an operator to process the input signal source in response to any one of the L image and the R image not conforming to the image normal standard.

The present disclosure also discloses an electronic device, fig. 11 shows a block diagram of an electronic device according to an embodiment of the present disclosure.

As shown in fig. 11, the electronic device 1100 includes a memory 1101 and a processor 1102, the memory 1101 is configured to store a program for supporting the electronic device to perform the 2D/3D image auto-switching method or the code generating method of the 3D display system of any of the above embodiments, and the processor 1102 is configured to execute the program stored in the memory 1102.

As shown in fig. 12, the computer system 1200 includes a processing device 1201 which can execute various processes in the above embodiments according to a program stored in a Read Only Memory (ROM) 1202 or a program loaded from a storage section 1208 into a Random Access Memory (RAM) 1203. In the RAM1203, various programs and data required for the operation of the system 1200 are also stored. The processing device 1201, the ROM1202, and the RAM1203 are connected to each other through a bus 1204. An input/output (I/O) interface 1205 is also connected to the bus 1204.

The following components are connected to the I/O interface 1205: an input section 1206 including a keyboard, a mouse, and the like; an output portion 1207 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 1208 including a hard disk or the like; and a communication section 1209 including a network interface card such as a LAN card, a modem, or the like. The communication section 1209 performs communication processing via a network such as the internet. The drive 1210 is also connected to the I/O interface 1205 as needed. A removable medium 1211 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on the drive 1210 so that a computer program read out therefrom is installed into the storage section 1208 as needed. The processing device 1201 may be embodied as a processing device such as CPU, GPU, TPU, FPGA, NPU.

In particular, according to embodiments of the present disclosure, the methods described above may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising computer instructions which, when executed by a processor, implement the method steps described above. In such embodiments, the computer program product may be downloaded and installed from a network via the communications portion 1209, and/or installed from the removable media 1211.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units or modules referred to in the embodiments of the present disclosure may be implemented in software or in programmable hardware. The units or modules described may also be provided in a processor, the names of which in some cases do not constitute a limitation of the unit or module itself.

As another aspect, the present disclosure also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or the computer system in the above-described embodiments; or may be a computer-readable storage medium, alone, that is not assembled into a device. The computer-readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the present disclosure.

The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention referred to in this disclosure is not limited to the specific combination of features described above, but encompasses other embodiments in which any combination of features described above or their equivalents is contemplated without departing from the inventive concepts described. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).

Claims

1. A 2D/3D image automatic switching method of a 3D display system, comprising:

Acquiring video data;

processing the video data to obtain an L image signal and an R image signal;

performing time domain analysis on the L image signal and the R image signal, calculating fuzzy values LSi and RSi of gray-scale images of the L image signal and the R image signal, and judging whether each image signal in the L image signal and the R image signal meets the normal image standard or not based on the comparison result of the fuzzy values LSi and RSi and a preset threshold value; wherein, the subscript i is any frame of graying image in the L image signal or the R image signal;

responding to the fact that the L image signal and the R image signal are normal in the fuzzy value judgment, and based on the calculated fuzzy values LSi and RSi, sequentially taking the fuzzy values LSi and RSi to perform subtraction operation, and dividing the absolute value of the obtained difference value by the maximum value in the fuzzy values LSi and RSi; if the calculated result is smaller than the correlation value, confirming that the L image signal and the R image signal both accord with the image normal standard; if the calculation result is larger than or equal to the correlation value, confirming that the path of image signals with minimum fuzzy values LSi and RSi do not accord with the normal image standard;

processing the L image signal and the R image signal to generate a 3D image output in response to both of the L image signal and the R image signal conforming to an image normal standard; or alternatively

And in response to one path of the L image signal or the R image signal meeting the image normal standard, forming one path of image signal corresponding to the image meeting the image normal standard into a 2D image and outputting the 2D image.

2. The 2D/3D image automatic switching method of a 3D display system according to claim 1, wherein the acquiring video data is implemented as:

and acquiring the video data by adopting a binocular camera.

3. The 2D/3D image automatic switching method of a 3D display system according to claim 1 or 2, wherein the processing of the video data to acquire an L image signal and an R image signal is implemented as:

4. A 2D/3D image auto-switching method of a 3D display system according to claim 3, wherein the processing unit decodes the video data, implemented as:

5. The method for automatically switching 2D/3D images of a 3D display system according to claim 4, wherein the determining the L image signal and the R image signal by performing the LR image matching judgment on the first image signal and the second image signal is implemented as:

assuming that the center coordinates of the first gray scale image signal are (X1, Y1), selecting a first matrix E1 by taking the center coordinates of the first gray scale image signal as an origin, and mapping the first matrix E1 to the center point of the second gray scale image signal to obtain a mapping matrix E1 ', wherein the center coordinates of the mapping matrix E1 ' are (X1 ', Y1);

sliding the mapping matrix E1' left and right along the X direction of the second gray level image signal, performing image matching to obtain a matching matrix E2 of the second gray level image signal, and determining the central coordinates (X2, Y2) of the matching matrix E2, wherein Y1 = Y2;

Based on the positional relationship of the mapping matrix E1' and the matching matrix E2 along the X direction, determining the correspondence between the first image signal and the second image signal and the L image signal and the R image signal, respectively, wherein,

under the condition that X1' is more than X2, determining that the first image signal is an L image signal and the second image signal is an R image signal;

under the condition that X1' is less than X2, the first image signal is determined to be an R image signal, and the second image signal is determined to be an L image signal.

6. The method for automatically switching 2D/3D images of a 3D display system according to any one of claims 1 to 2 and 4 to 5, wherein the performing time domain analysis on the L image signal and the R image signal is implemented as:

7. The method for automatically switching 2D/3D images of a 3D display system according to claim 6, wherein the preprocessing comprises: de-interlacing processing and gamut conversion processing.

8. The method for automatically switching 2D/3D images of a 3D display system according to claim 6, wherein the three-level buffer processing is performed on the other image signal by using a specific frame buffer mode at intervals to obtain a buffered image signal.

9. The 2D/3D image automatic switching method of a 3D display system according to claim 6, wherein the calculating of the blur values LSi and RSi of the grayscaled images of the L image signal and the R image signal is implemented as:

wherein, f (x, y) corresponds to the gray value of the coordinate (x, y), M, N respectively represents the row and column size of a frame of image;

calculating the fuzzy values LSi and RSi of the gray-scale images of the L image signal and the R image signal by adopting the calculation formula;

based on the calculated blur values LSi or RSi, comparing with a preset threshold SY:

if any one of the fuzzy values LSi or RSi is greater than or equal to a preset threshold SY, confirming that the image signal is normal, otherwise, confirming that the image signal is fuzzy, wherein the value range of i is 1-3.

10. The 2D/3D image automatic switching method of a 3D display system according to claim 6, further comprising:

comparing the fuzzy values LSi, judging abnormal frame numbers under the condition that the LSi difference exceeds a specific threshold value, and discarding L paths of corresponding frames of the preprocessed image signals, wherein the value range of i is 1-3;

comparing the blurred values RSi, judging abnormal frame numbers under the condition that the RSi difference exceeds a specific threshold value, and discarding R paths of corresponding frames of the preprocessed image signals, wherein the value range of i is 1-3.

11. The method for automatically switching 2D/3D images of a 3D display system according to any one of claims 9 to 10, wherein the processing of the L image signal and the R image signal to generate a 3D image output in response to both of the L image signal and the R image signal conforming to an image normal standard is implemented as:

and generating a 3D image signal by the L paths of the preprocessed image signals and the R paths of the preprocessed image signals in response to that both the L image signals and the R image signals accord with the image normal standard, and outputting the 3D image signal.

12. The method of 2D/3D image auto-switching of a 3D display system of claim 11, wherein the processing of the L image signal and the R image signal to generate a 3D image output in response to both of the L image signal and the R image signal conforming to an image normal standard further comprises: and performing time sequence synchronization processing on the L image signal and the R image signal.

13. The method for automatically switching 2D/3D images of a 3D display system according to any one of claims 9 to 10, wherein in response to one of the L image signal or the R image signal meeting the image normal standard, forming one image signal corresponding to an image meeting the image normal standard into a 2D image output is implemented as:

and responding to one path of the L image signal or the R image signal to meet the image normal standard, and forming one path of the preprocessed image signal meeting the image normal standard into a 2D image for outputting.

14. The method for automatically switching 2D/3D images of a 3D display system according to any one of claims 1, 2, 4 to 5, 7, and 10, wherein an alarm is issued to indicate that a problem occurs in one of the L image signal or the R image signal in response to the one of the L image signal and the R image signal not conforming to the normal image standard.

15. The method for automatically switching 2D/3D images of a 3D display system according to claim 6, wherein an alarm is issued to indicate that one of the L image signal or the R image signal is out of order in response to the one of the L image signal or the R image signal not meeting the image normal standard.

16. The method for automatically switching 2D/3D images of a 3D display system according to any one of claims 1, 2, 4 to 5, 7, and 10, wherein,

And sending an alarm signal to prompt the processing of the input signal source in response to any one of the L image signal and the R image signal not conforming to the normal image standard.

17. The method for automatically switching 2D/3D images for a 3D display system according to claim 6, wherein,

18. A 2D/3D image automatic switching system based on a 3D display system, comprising:

an acquisition module configured to acquire video data;

an analysis calculation module configured to perform time domain analysis on the L image signal and the R image signal, and calculate blur values LSi and RSi of grayscale images of the L image signal and the R image signal;

the judging module is configured to judge whether each path of image signals in the L image signal and the R image signal accords with an image normal standard or not based on the comparison result of the fuzzy values LSi and RSi and a preset threshold value;

the correlation detection module is configured to respond to the fact that the L image signal and the R image signal are normal in the fuzzy value judgment, sequentially take the fuzzy values LSi and RSi as subtraction operation based on the calculated fuzzy values LSi and RSi, and divide the absolute value of the obtained difference value by the maximum value in the fuzzy values LSi and RSi; if the calculated result is smaller than the correlation value, confirming that the L image signal and the R image signal both accord with the image normal standard; if the calculation result is larger than or equal to the correlation value, confirming that the path of image signals with minimum fuzzy values LSi and RSi do not accord with the normal image standard;

A 3D image generation module configured to process the L image signal and the R image signal to generate a 3D image in response to both of the L image signal and the R image signal conforming to an image normal standard;

a 2D image generation module configured to generate a 2D image of one path of image signals corresponding to an image conforming to an image normal standard in response to one path of the L image signal or the R image signal conforming to the image normal standard; and

19. The 2D/3D image auto-switching system of the 3D display system of claim 18, wherein the processing module comprises: a processing unit and a matching module, wherein the processing unit is used for matching the processing unit,

20. The 2D/3D image automatic switching system of a 3D display system according to claim 19, wherein the processing unit is further configured to decode and output a signal corresponding to an access signal belonging to the highest priority and effectively existing, resulting in a first image signal and a second image signal;

21. The 2D/3D image auto-switching system of the 3D display system according to claim 20, wherein the matching module comprises:

a graying first sub-module configured to perform graying processing on the first image signal and the second image signal to obtain a first gray image signal and a second gray image signal;

22. The 2D/3D image automatic switching system of a 3D display system according to any one of claims 18 to 20, wherein the analysis and calculation module includes:

23. The 2D/3D image auto-switching system of a 3D display system of claim 22, wherein the analysis computing module further comprises:

and the calculating sub-module is configured to calculate the blurring value of the grayscale image of each frame to obtain blurring values LSi and RSi of the grayscale images of the L image signal and the R image signal.

24. The 2D/3D image auto-switching system of a 3D display system according to claim 23, further comprising:

and the comparison module is configured to compare the fuzzy value LSi or RSi with a preset threshold SY.

25. The 2D/3D image automatic switching system of a 3D display system according to any one of claims 18 to 20, 23 to 24, further comprising:

the first alarm module is configured to send an alarm to indicate that one of the L image signal or the R image signal is out of order in response to the fact that the one of the L image signal or the R image signal is out of order with the normal image standard.

26. The 2D/3D image automatic switching system of a 3D display system according to any one of claims 18 to 20, 23 to 24, further comprising:

And the second alarm module is configured to send an alarm signal to prompt an operator to process the input signal source in response to any one of the L image signal and the R image signal not conforming to the normal image standard.

27. An electronic device comprising a memory and a processor, the memory to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method of any of claims 1-17.

28. A readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the method of any of claims 1 to 17.