WO2019047896A1

WO2019047896A1 - Image processing method and device

Info

Publication number: WO2019047896A1
Application number: PCT/CN2018/104381
Authority: WO
Inventors: 冯凯
Original assignee: 西安中兴新软件有限责任公司
Priority date: 2017-09-11
Filing date: 2018-09-06
Publication date: 2019-03-14
Also published as: CN107767412A

Abstract

An image processing method, comprising: adjusting a disparity attribute value of an image pixel point in at least one area of an image to be processed, and generating a disparity image of the image to be processed; and generating a three dimensional image according to the image to be processed and the disparity image of the image to be processed.

Description

Image processing method and device

The present application claims the benefit of priority to the benefit of the benefit of the benefit of the benefit of the benefit of the benefit of the benefit of the benefit of the benefit of the benefit of the benefit of the benefit of the benefit of the benefit of the benefit of the benefit of the benefit of the benefit of the disclosure of

Technical field

The present disclosure relates to image processing techniques, for example, to an image processing method and apparatus.

Background technique

With the continuous development of display technology and digital technology, Three-Dimension (3D) display has become a hot spot for display products. There are two main sources of 3D content: one is to capture 3D sources through 3D shooting devices (such as 3D cameras, 3D cameras, etc.); the other is to convert two-dimensional (2D) content into two-dimensional (2D) content. 3D content. For the second method, the entire 2D image is usually processed according to a 3D algorithm to convert into a 3D image, and the details and parts in the image cannot be refined, and the different display effects of the region cannot be realized.

Summary of the invention

The following is an overview of the topics detailed in this document. This Summary is not intended to limit the scope of the claims.

The embodiment of the present application provides an image processing method and device, which can implement adjustment of a three-dimensional display effect.

In a first aspect, an embodiment of the present application provides an image processing method, including: adjusting a disparity attribute value of an image pixel point in at least one region of an image to be processed, and generating a disparity image of the to-be-processed image; The image and the parallax image of the image to be processed generate a three-dimensional image.

In a second aspect, an embodiment of the present application provides an image processing apparatus, including: a parallax image generating module, configured to adjust a parallax attribute value of an image pixel point in at least one region of an image to be processed, and generate a parallax of the image to be processed And a three-dimensional image generating module configured to generate a three-dimensional image according to the image to be processed and the parallax image of the image to be processed.

In a third aspect, an embodiment of the present application provides a terminal, including: a memory, a processor, and an image processing program stored in the memory and executable on the processor, where the image processing program is executed by the processor The steps of the image processing method provided by the above first aspect are implemented.

In a fourth aspect, an embodiment of the present application provides an image processing method, including: determining a selected area of an image to be processed according to an instruction; and adjusting a display effect of the selected area to an outgoing screen or an on-screen effect.

In a fifth aspect, an embodiment of the present application provides a computer readable medium storing an image processing program, where the image processing program is executed by a processor to implement the steps of the image processing method provided by the first aspect or the fourth aspect.

Other aspects will be apparent upon reading and understanding the drawings and detailed description.

DRAWINGS

1 is a schematic diagram of a hardware structure of a terminal for implementing an image processing method according to an embodiment of the present application;

2 is a flowchart of an image processing method according to an embodiment of the present application;

FIG. 3 is an exemplary flowchart of an image processing method according to an embodiment of the present disclosure;

4 is a schematic diagram of an editing interface of 2D to 3D image conversion according to an embodiment of the present application;

FIG. 5 is a schematic diagram of area segmentation of a 2D image according to an embodiment of the present application; FIG.

FIG. 6 is another schematic diagram of region segmentation of a 2D image according to an embodiment of the present application; FIG.

FIG. 7 is a schematic diagram showing a principle of forming a stereoscopic display effect of a 3D image according to an embodiment of the present application; FIG.

FIG. 8 is a schematic diagram of image interleaving according to an embodiment of the present application;

9 is a schematic diagram of displaying a 3D image through a grating according to an embodiment of the present application;

FIG. 10 is a schematic diagram of a voltage applied to a grating cylinder of an embodiment of the present application; FIG.

FIG. 11 is a schematic diagram of an image processing apparatus according to an embodiment of the present application;

FIG. 12 is another schematic diagram of an image processing apparatus according to an embodiment of the present application.

Detailed ways

The embodiments of the present application are described in detail below with reference to the accompanying drawings.

FIG. 1 is a schematic diagram of a hardware structure of a terminal for implementing an image processing method according to an embodiment of the present application. The terminal of this embodiment may include, but is not limited to, a mobile terminal such as a laptop computer, a tablet computer, a mobile phone, a media player, a personal digital assistant (PDA), a projector, and the like, and a digital television (Television, TV). Fixed terminals such as desktop computers. Illustratively, the above terminal can support 3D video and picture capturing and playing functions.

As shown in FIG. 1, the terminal 10 of this embodiment includes a memory 14 and a processor 12. The terminal structure shown in FIG. 1 does not constitute a limitation to the terminal, and the terminal may include more or less components than those illustrated, and the terminal may combine certain components or different component arrangements.

The processor 12 may include, but is not limited to, a processing device such as a Micro Controller Unit (MCU) or a Field-Programmable Gate Array (FPGA). The memory 14 may be provided as a software program and a module for storing application software, such as program instructions or modules corresponding to the image processing method in the embodiment, and the processor 12 executes a plurality of types by executing a software program and a module stored in the memory 14. Functional application and data processing, that is, the image processing method of the present embodiment is implemented. Memory 14 may include high speed random access memory and may also include non-volatile memory such as at least one magnetic storage device, flash memory, or other non-volatile solid state memory. In some examples, memory 14 may include memory remotely located relative to processor 12, which may be coupled to terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

In an embodiment, the terminal 10 may further include a communication unit 16; the communication unit 16 may receive or transmit data via a network. In one example, communication unit 16 can be a Radio Frequency (RF) module configured to communicate with the Internet wirelessly.

In an embodiment, the terminal 10 may further include a display unit configured to display information input by the user or information provided to the user. The display unit may include a display panel, and the display panel may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

Next, the principle of forming a stereoscopic display effect of a 3D image will be described with reference to FIG.

The usual 2D display is to image the left eye image and the right eye image without parallax at the position of the screen, and this display does not have a stereoscopic effect. A stereoscopic effect can be produced when the left eye image and the right eye image are parallaxly imaged at the position of the screen. If the right eye image is located at the right side of the screen where the left eye image is located, then the convergence point of the two (ie, the image point formed in the human brain) will be located behind the screen, resulting in a depression at the screen. The stereo effect is the screen input effect; if the right eye image is located at the left side of the screen where the left eye image is located, the convergence point of the two will be located in front of the screen position, resulting in a prominent position on the screen. The stereo effect is the screen effect.

As shown in Fig. 7, T is the distance between the left and right eyes of the person, and the value of T can be obtained according to the average spacing between the left and right eyes of the person, that is, it is usually constant. f is the distance between the human eye and the screen, f can be fixed or can be changed in real time. If f is changed in real time, the eye tracking function of the front camera of the terminal can be turned on, and the human eye and the screen can be detected in real time. the distance. d is the offset of an image pixel, d=|L _S -R _S |, where L _S represents the position of the image pixel in the left eye image at the screen, and R _S represents the image pixel in the right eye image At the location of the screen. In Fig. 7, L _S 1 and R _S 1 represent the positions of the image pixel points 1 in the left and right eye images on the screen, and L _S 2 and R _S 2 represent the positions of the image pixel points 2 in the left and right eye images on the screen. In FIG. 7, the P1 point is an on-screen imaging point of a 3D image, and the P2 point is an out-of-screen imaging point of a 3D image. M1 represents the parallax attribute value of the image pixel point 1, that is, the vertical distance between the in-screen imaging position and the human eye.

FIG. 2 is a flowchart of an image processing method according to an embodiment of the present application. The image processing method provided by the embodiment is for converting a to-be-processed image into a 3D image including a left-eye image and a right-eye image. The image to be processed may be a 2D image, a left eye image in a 3D image, or a right eye image in a 3D image. In other words, the image processing method provided by this embodiment can be used to convert a 2D image into a 3D image, or to perform editing modification on a 3D image. Moreover, the image processing method provided by this embodiment can also be used to convert 2D video into 3D video, or to edit and modify 3D video. Wherein, by converting any frame 2D image in the 2D video into a 3D image, a 3D video is obtained, thereby presenting a 3D effect during image display or video playback; or by adjusting any frame left eye image in the 3D video , the updated right eye image is obtained, thereby presenting a display effect different from the original 3D video.

As shown in FIG. 2, the image processing method provided in this embodiment includes step S201 and step S202.

In step S201, the disparity attribute value of the image pixel point in at least one region of the image to be processed is adjusted, and a disparity image of the image to be processed is generated.

In step S202, a 3D image is generated based on the image to be processed and the parallax image of the image to be processed.

In an exemplary embodiment, step S201 may include: dividing an image to be processed into at least one region; and determining, for an area of the at least one region, an image pixel point in the region according to a target display effect of the region Parallax attribute value.

Each image pixel may include a plurality of attributes, such as Red Green Blue (RGB) attributes, parallax attributes, and the like. In the present embodiment, the 2D to 3D conversion is realized by adjusting the parallax property of each image pixel, or the 3D display effect of the region is adjusted.

After determining the parallax attribute value of each image pixel in the area according to the target display effect of the area, if the image pixel has the parallax attribute value originally, the current parallax attribute value is used to replace the original parallax attribute value. If the parallax attribute value is not set in the image pixel, the parallax attribute is added to the image pixel, and the value is the currently determined disparity attribute value of the image pixel.

Among them, the target display effect may include one of the following: the screen effect and the screen entry effect. The descriptions of the screen effect and the screen entry effect are as described above, and thus will not be described again. In this embodiment, the target display effects of different regions may be the same or different. However, this application is not limited thereto.

In an embodiment, the image processing method of the embodiment may further include: determining, according to the received instruction or preset configuration information, a region segmentation manner in the image to be processed and a target display effect of the at least one region.

The area division manner of the image to be processed and the target display effect of the area may be set by the user, or may be determined according to a preset configuration. However, this application is not limited thereto.

After the region to be processed is segmented, at least one region may be selected in the segmented region to adjust the target display effect.

In an embodiment, the step S201 may further include: when the target display effect is not set in an area, the original parallax attribute value of each image pixel in the area is kept unchanged; or, each image pixel in the area is updated. The parallax attribute value is a preset value.

Wherein, for an area where the target display effect is not set (for example, an unselected area), if each image pixel of the image to be processed originally has a parallax attribute value, the parallax attribute value of the image pixel point in the area is maintained. If the image pixel of the image to be processed does not have the parallax property value, the parallax property value may be added to the image pixel, and the parallax property value of each image pixel in the region in the image to be processed may be equal to A preset value, such as the distance between the human eye and the screen. Alternatively, if each image pixel of the image to be processed has a parallax attribute value, the parallax attribute value of each image pixel in the area of the image to be processed may be updated to be equal to a preset value, for example, a human eye and a screen. the distance between.

In this embodiment, the parallax attribute values of the selected area of the image to be processed and the image pixel points in the unselected area may be determined in different manners, and the parallax attribute values of the image pixel points in different areas of the image to be processed may be different, so that Different areas produce different display effects.

In an embodiment, determining the disparity attribute value of the image pixel point in the area according to the target display effect of the area may include: determining an offset of each image pixel point in the area according to the target display effect of the area; For each image pixel in the region, the parallax attribute value of the image pixel is determined according to the offset of the image pixel.

In other words, in the present embodiment, the parallax attribute value of the image pixel point in one region can be determined according to the offset of the image pixel point. Wherein, the offset of one image pixel refers to the absolute value of the difference between the position of the image pixel on the display screen in the left eye image and the position of the image pixel on the display screen in the right eye image.

In an embodiment, determining the offset of each image pixel in the region according to the target display effect of the region may include: determining, when the target display effect of the region is set to the on-screen effect, determining each of the regions The offset value d of an image pixel is greater than 0 and less than T; when the target display effect of one region is set to the screen effect, the offset d of each image pixel in the region is determined. The value ranges from greater than T and less than 2T; wherein T is the spacing between the left and right eyes of the person.

In an embodiment, determining the disparity attribute value of the image pixel point according to the offset of the image pixel point may include: according to the offset of the image pixel point, the spacing between the left and right eyes of the image, and the image to be processed. Pixel density, calculating a parallax attribute value of the image pixel point; or calculating a parallax attribute of the image pixel point according to the offset of the image pixel point, the spacing between the left and right eyes of the person, and the distance between the human eye and the screen value.

In an embodiment, calculating the disparity attribute value of the image pixel point according to the offset of the image pixel point, the spacing between the left and right eyes of the person, and the pixel density of the image to be processed may include: calculating an image according to the following formula: The parallax attribute value of the pixel:

Where M is the parallax attribute value of the pixel of the image, d is the offset of the pixel of the image, T is the spacing between the left and right eyes of the person, and PPI is the pixel density.

Among them, the value of T can be obtained according to the average spacing between the left and right eyes of the person. The PPI can be determined according to the image resolution and the screen size, with the resolution being a×b and the screen size being c as an example.

In an embodiment, calculating the disparity attribute value of the image pixel point according to the offset of the image pixel point, the spacing between the left and right eyes of the person, and the distance between the human eye and the screen may include: calculating according to the following formula The parallax attribute value of an image pixel: M = f × T / d.

Where M is the parallax attribute value of the pixel of the image, d is the offset of the pixel of the image, T is the distance between the left and right eyes of the person, and f is the distance between the human eye and the screen.

In an embodiment, when the target display effect of the entire area of the image to be processed is set to the screen effect, it may be determined that the offset of each image pixel in the image to be processed is 1.5T; when all regions of the image to be processed are When the target display effect is set to the on-screen effect, it can be determined that the offset of each image pixel in the image to be processed is 0.5T, where T is the distance between the left and right eyes of the person. However, this application is not limited thereto.

In an embodiment, determining the offset of each image pixel in the region according to the target display effect of the region may include: displaying the target according to the region when the distance between the human eye and the screen has not changed. The effect is to determine an initial parallax attribute value of each image pixel in the area; after the distance between the human eye and the screen is changed, according to the initial parallax attribute value of the image pixel point, the changed human eye and the screen The distance between the person and the distance between the left and right eyes determines the offset of the pixel of the image after the distance between the human eye and the screen changes.

In an embodiment, the distance between the human eye and the screen can be acquired in real time through the eyeball tracking function of the imaging device. When the distance between the human eye and the screen does not change, the initial offset of each image pixel in the region may be determined according to the target display effect of the region; then, for each image pixel in the region Calculating an initial parallax attribute value of the image pixel point according to an initial offset of the pixel of the image, a spacing between the left and right eyes of the person, and a pixel density of the image to be processed; after the distance between the human eye and the screen is changed According to the initial parallax attribute value of the pixel of the image, the distance between the human eye and the screen after the change, and the distance between the left and right eyes of the image, the deviation of the pixel point of the image after the distance between the human eye and the screen is changed is calculated. The shift amount, that is, d=f×T/M0, where d is the offset of the image pixel point after the distance between the human eye and the screen is changed, and M0 is the initial parallax attribute value of the image pixel point, and f is real time. Obtain the distance between the obtained human eye and the screen, T is the distance between the left and right eyes of the person; after that, the offset of the pixel of the image may be changed according to the distance between the human eye and the screen, and the person The distance between the left and right eyes and the pixel density of the image to be processed are calculated as the parallax attribute value of the pixel of the image after the distance between the human eye and the screen is changed.

In an embodiment, determining the disparity attribute value of the image pixel point in the area according to the target display effect of the area may include: determining, according to the target display effect of the area, applying a voltage to the grating cylinder mirror corresponding to the area on the screen. In order to shift the alignment direction of the liquid crystal molecules; according to the voltage, the parallax property value of each image pixel in the region is determined.

Wherein, according to the target display effect of the region of the image to be processed, it may be determined that a voltage is applied to the grating cylinder mirror corresponding to the region on the screen, so that the alignment direction of the liquid crystal molecules is shifted, thereby changing the refractive index of the light. Wherein, the larger the voltage, the larger the refractive index; then, according to the voltage, the parallax attribute value of each image pixel in the region of the image to be processed is determined.

In an embodiment, as shown in FIG. 10, the parallax attribute value of each image pixel in the region of the image to be processed is determined according to the voltage applied to the grating cylinder. As shown in FIG. 10( a ), an indium tin oxide (ITO) layer, that is, a conductive glass layer, can generate an electric field by driving an external voltage, thereby changing the alignment direction of liquid crystal molecules in the liquid crystal layer to change the light. The refractive index can provide a 3D display effect. As shown in FIG. 10(b), in the case where the external voltage is 0, a 2D display effect can be provided.

In an embodiment, step S202 may include: determining an image to be processed as a left eye image, determining a parallax image of the image to be processed as a right eye image; or determining the image to be processed as a right eye image, and the image to be processed The parallax image is determined as the left eye image.

In the present embodiment, according to the original 2D image and the parallax image obtained from the 2D image, a 3D image of the left and right eye format can be generated, thereby realizing conversion of the 2D image to the 3D image. It is also possible to generate a corresponding parallax image according to the left eye image or the right eye image of the original 3D image, thereby generating an updated 3D image, and implementing modified editing of the 3D image to adjust the display effect of the region.

In an embodiment, after the step S202, the method of the embodiment may further include: compressing the left eye image and the right eye image respectively in the first direction; and compressing the compressed left eye image and the right eye image according to the predetermined The format is interleaved to obtain an interlaced image; the interlaced image is stretched in a first direction; and the stretched interlaced image is displayed.

Wherein, the first direction may be a lateral coordinate direction. However, this application is not limited thereto. In other implementations, the first direction can also be a longitudinal coordinate direction.

Wherein, when the first direction is the horizontal coordinate direction, the left eye image and the right eye image are respectively compressed in the lateral coordinate direction, and the left eye image and the right eye image are spaced according to a column of left eye image pixels and a column of right eye image pixels. The order is interleaved to obtain an interlaced image. Then, the interlaced image is stretched in the first direction to achieve a full screen effect.

FIG. 3 is a flowchart of an example of an image processing method according to an embodiment of the present application. The image processing method provided in this embodiment may be applied to a mobile terminal, and the mobile terminal may provide an editing interface for 2D to 3D image conversion, and the editing interface may include: an image editing area and a control button. As shown in FIG. 4, the editing interface includes an image editing area 401, an out-of-screen button 402, and an on-screen button 403. The out button button 402 is set to control the screen effect processing, and the screen button 403 is set to control the screen effect processing. However, this application is not limited thereto. In other embodiments, the control interface may not be provided with a control button, and the user may use the combination key mode to perform on-screen or on-screen effect processing control; or the editing interface may further include: an automatic manual switching button, and an automatic manual switching button. It is set to control the mode of the current image conversion. For example, in the manual mode, the user needs to select an area having a corresponding display effect in the image editing area 401. In the automatic mode, the user can perform area selection without the user, and the mobile terminal can be configured according to the preset configuration information. Check the area.

As shown in FIG. 3, the image processing method provided in this embodiment includes steps S301 to S307.

In step S301, the slice source format and the presentation mode are determined.

The source format may include at least one of the following types: 2D image, 2D video, 3D image, 3D video; the presentation manner may include one of the following: 2D, 3D. Wherein, the 3D image may be a 3D image of a left and right eye format, including a left eye image and a right eye image.

In this embodiment, the slice source format may be determined according to the slice source selected by the user, and the presentation mode is determined according to an instruction input by the user. However, this application is not limited thereto. In other implementation manners, the mobile terminal may preset the configuration information to determine the presentation mode.

It should be noted that, in this embodiment, if the source format is 2D image or 2D video, and the presentation mode is 2D, the 2D image or the 2D video can be normally displayed; if the source format is 2D image or 2D video, the presentation is performed. If the mode is 3D, step S302 to step S307 may be performed to convert the 2D image into a 3D image and then display; if the source format is 2D image or 2D video, and each of the 2D videos may be displayed when the rendering mode is 3D. The frame 2D image is converted into a 3D image, and the 3D video is formed and played; if the source format is 3D image or 3D video, and the rendering mode is 2D, the right eye image can be extracted from the 3D image and processed by image stretching and the like. Display; if the source format is 3D image or 3D video, when the rendering mode is 2D, the right eye image may also be extracted for each frame 3D image in the 3D video, and then played after the image stretching process; if the source format For a 3D image or a 3D video, the presentation mode is 3D, then steps S304 to S307 may be performed to present a 3D display effect. Wherein, if the 3D image or the 3D video provided by the source is an image that has been interlaced, it can be directly displayed or played after the image stretching process.

The following is an example in which the source format is a 2D image and the 3D method is used as an example. In this example, the user selects the area display effect of manually setting the 2D image; after the user selects the 2D image, the 2D image to be processed is displayed in the image editing area of the editing interface. As shown in FIG. 4, the 2D image to be processed in the image editing area 401 is a rabbit.

In step S302, the selected area in the 2D image and the target display effect are determined.

In this step, for the 2D image to be processed, the region segmentation mode and the target display effect of the selected region are determined. For example, in FIG. 6, the target display effect of the selected area of the dotted line frame is the screen output effect, and the target display effect of the selected area of the dotted line frame is the screen entry effect.

In step S303, 3D image conversion is performed based on the determined area.

In an embodiment, the distance f between the human eye and the screen is a fixed value.

When the user selects the screen button 402 in the editing interface shown in FIG. 4 and applies the image editing area 401 to the head area of the rabbit, the area of the smear area can be obtained by marking the area as indicated by the dotted line in FIG. , [20, 20, 40, 40], and add a parallax attribute value to each image pixel in the coordinate area. It should be noted that if each image pixel in the coordinate area has been set with a parallax attribute value, the original parallax attribute value is replaced with a new parallax attribute value.

Among them, the coordinates are represented by [abscissa, ordinate, abscissa, ordinate]. However, the present application does not limit the coordinate determination manner of the area marked by the broken line frame. For example, the coordinates may be determined according to an area intercepted or clicked by the mouse operated by the user in the image editing area 401, or the coordinates of the selected area may be determined according to the touch position of the user on the touch screen.

In this example, the user sets the target display effect of the dotted area in FIG. 5 as the screen-out effect. Therefore, the parallax attribute value of each image pixel in the area is greater than the distance between the human eye and the screen. The parallax attribute value of each pixel in the unselected area (ie, the area outside the dotted line frame in FIG. 5) may be equal to the distance f between the human eye and the screen. At this time, the adjustment range of the offset d of each pixel in the selected area is T<d<2T.

When the user selects the on-screen button 403 in the editing interface shown in FIG. 4 and applies the image editing area 401 to the tail region of the rabbit, the area of the smear area can be obtained by marking the area as shown by the dotted line frame in FIG. For example, [0, 10, 30, 50], and add a disparity attribute value to each image pixel in the coordinate area.

In this example, the user sets the target display effect of the marked area of the dotted line frame in FIG. 6 as the on-screen effect. Therefore, the parallax attribute value of each image pixel in the area is smaller than between the human eye and the screen. Distance f. At this time, the parallax attribute value of each image pixel in the unselected area (ie, the area outside the dotted line and the dotted line frame in FIG. 6) may be equal to the distance f between the human eye and the screen. At this time, the adjustment range of the offset d of each image pixel in the selected area of the dotted line frame is 0<d<T.

In this example, in order to smooth the 3D image transition, the effect of the 3D image can be smoothed by the probability density algorithm, and the parallax attribute value can be calculated according to the following formula:

For example, a 6-inch screen image with a resolution of 1920×1080 is taken as an example, and a pixel density (Pixels Per Inch, PPI) of a 6-inch screen with a resolution of 1920×1080 is equal to 367.

In this example, in the area marked by the dashed box, d=2T of the central image pixel point, and d of the neighboring image pixel point are successively decreased in the range of T<d<2T. The dashed box indicates that the parallax attribute value of each image pixel in the region is calculated according to the corresponding d value using the above formula. In the dotted line marked area, d=T of the central image pixel point, d of the neighboring image pixel point is successively decreased in the range of 0<d<T0. The value of the parallax attribute of each image pixel in the dotted line frame is calculated according to the corresponding d value according to the above formula.

In this example, the parallax of each image pixel in the two regions is updated by adjusting the d in the horizontal direction of the pixel in the region coordinates [20, 20, 40, 40] and the region coordinates [0, 10, 30, 50]. Property value. The parallax attribute value of the image pixel points other than these two areas may be equal to f.

In this example, after updating the parallax attribute value of each image pixel point in the 2D image, a parallax image is generated. The generated parallax image is used as a right eye image, and the 2D image is used as a left eye image to generate a 3D image of the left and right eye format, thereby realizing conversion of the 2D image to the 3D image, and the display effect can be regionalized.

In another example, the distance f between the human eye and the screen can vary in real time. At this point, you can turn on the eye tracking function of the front camera to detect the distance between the human eye and the screen in real time. In this example, the initial disparity attribute value may be determined according to the initial d, using the above-described smoothing disparity attribute value calculation formula; when f changes, the d value may be adjusted according to the following formula: d=f×T/M0; Wherein, M0 is an initial parallax attribute value; after d is updated, the updated M is calculated by using the above-described smoothed disparity attribute value calculation formula. In this example, the d value can be adjusted in real time by the f value, and the screen and screen effects can be controlled by the d value.

In step S304, the converted 3D image is subjected to compression processing.

In this step, the left-eye image and the right-eye image are respectively compressed in the lateral coordinate direction, that is, the horizontal pixels are compressed to half, for example, an image with a resolution of 1920×1080 is compressed to an image with a resolution of 960×1080.

In step S305, image interleaving is performed on the compressed 3D image.

In this step, the left and right eye images in the compressed image processing (Image Signal Processor, ISP) channel are interlaced and merged. As shown in FIG. 8, the left eye image and the right eye image are side by side (side by side). The arrangement is performed to generate a standard interlaced image with a resolution of 1920×1080.

In step S306, the interlaced image is stretched.

In this step, the left eye and right eye data of the interleaved image are stretched back to 1920×1080 resolution.

In step S307, a 3D image is displayed.

In this example, a 3D grating film can be attached to the display surface of the terminal, so that when the display screen plays a 3D image, the left and right eyes of the person can receive different left and right eye image data, as shown in FIG. The frame mark portion is a left eye image, and the white rectangular frame mark portion is a right eye image. However, this application is not limited thereto. In other implementations, when the display screen of the terminal plays a 3D image or a 3D video, the user can watch by wearing the 3D glasses.

It should be noted that, in other implementation manners, after the user selects the 2D image to be processed, the entire 2D image may be processed. For example, when the screen output effect is selected, the d value is increased to 1.5T, and when the screen is selected, In the effect, the d value is reduced to 0.5T, and then the parallax attribute value M of each image pixel in [0, 0, 1080, 1920] is automatically calculated from M = f × T / d, and a parallax image is generated.

In the embodiment shown in FIG. 3, the image processing method of the present embodiment can be used to convert a 2D image into a 3D image, and the display effects of different regions can be different.

In another embodiment, the image processing method of the present embodiment can be used to edit a 3D image. In this embodiment, the left eye image in the original left and right eye format 3D image may be selected as the image to be processed, and the area division is performed in the image to be processed, and the target display effect is determined as an area for the screen or the screen effect; For the area, the parallax attribute value of each image pixel in the area is adjusted, wherein if the image pixel point in the original left eye image is not set with the parallax attribute value, the parallax attribute value may be added to each image pixel point, if the original left The image pixel in the eye image has been set with the parallax attribute value, and can be updated according to the newly determined parallax attribute value. For the calculation of the parallax attribute value of each image pixel in the region, refer to the description of the embodiment shown in FIG. 3, and thus no further details are provided herein. In this embodiment, the parallax attribute value of the image pixel except the selected area remains unchanged or set to a preset value, for example, equal to the distance between the human eye and the screen. In this embodiment, after generating the parallax image of the original left eye image, the original left eye image is still used as the left eye image, and the parallax image of the original left eye image is used as the right eye image to obtain the updated left and right eye format 3D. image. In this way, the modification of the 3D image can be realized, and the display effect of different regions can be modified.

In summary, the image processing method provided by the embodiment enables the user to conveniently convert the 2D image into a 3D image, and can adjust the out-of-screen and on-screen effects of the region as needed to present a 3D effect.

FIG. 11 is a schematic diagram of an image processing apparatus according to an embodiment of the present application. As shown in FIG. 11, the image processing apparatus provided in this embodiment includes a parallax image generating module 1101 and a three-dimensional image generating module 1102.

The parallax image generating module 1101 is configured to adjust a parallax property value of an image pixel point in at least one region of the image to be processed, and generate a parallax image of the image to be processed.

The three-dimensional image generation module 1102 is configured to generate a three-dimensional image according to the image to be processed and the parallax image of the image to be processed.

In an embodiment, the parallax image generating module 1101 may be configured to divide the image to be processed into at least one region; and determine an image pixel in the region according to the target display effect of the region for the region of the at least one region The parallax attribute value of the point.

In an embodiment, the parallax image generating module 1101 may be configured to determine a parallax attribute value of an image pixel point in the area according to a target display effect of the area in the following manner:

Determining an offset of each image pixel in the region according to the target display effect of the region; determining, for each image pixel in the region, a parallax of the pixel of the image according to an offset of the pixel of the image Property value.

In an embodiment, the parallax image generating module 1101 may be configured to determine a disparity attribute value of the image pixel point according to an offset of the image pixel point by: an offset of the pixel point of one image, and a left and right eye of the person The spacing between the pixels and the pixel density of the image to be processed, the parallax property value of the pixel of the image is calculated; or, according to the offset of an image pixel point, the spacing between the left and right eyes of the person, and the distance between the human eye and the screen, The parallax attribute value of the pixel of the image is calculated.

In an embodiment, the parallax image generating module 1101 may be configured to calculate a parallax attribute value of the image pixel point according to an offset of the image pixel point, a spacing between the left and right eyes of the person, and a pixel density of the image to be processed. : Calculate the parallax property value of an image pixel according to the following formula:

In an embodiment, the parallax image generating module 1101 may be configured to calculate the parallax of the image pixel point according to the offset of the image pixel point, the spacing between the left and right eyes of the person, and the distance between the human eye and the screen in the following manner. Attribute value: Calculate the parallax attribute value of an image pixel according to the following formula: M = f × T / d.

In an embodiment, the parallax image generating module 1101 may be configured to determine an offset of each image pixel point in the region according to the target display effect of the region in the following manner: the target display effect in one region is set to the on-screen effect. Determining, the value of the offset d of each image pixel in the region is greater than 0 and less than T; when the target display effect of one region is set to the screen effect, determining each image pixel in the region The offset d ranges from greater than T to less than 2T; wherein T is the spacing between the left and right eyes of the person.

In an embodiment, the parallax image generating module 1001 may further be configured to keep the original parallax property value of each image pixel in the region unchanged when the target display effect is not set in one region; or, update each region in the region. The parallax attribute value of the image pixel is a preset value, for example, the distance between the human eye and the screen.

In an embodiment, the parallax image generating module 1101 may be configured to determine an offset of each image pixel point in the region according to a target display effect of the region in such a manner that the distance between the human eye and the screen does not change. And determining an initial parallax attribute value of the image pixel in the area according to the target display effect of the area; after the distance between the human eye and the screen is changed, according to the initial parallax attribute value of the image pixel point, after the change The distance between the human eye and the screen and the spacing between the left and right eyes of the person determine the offset of the pixel of the image after the distance between the human eye and the screen changes.

In an embodiment, the parallax image generating module 1101 may be configured to determine a parallax attribute value of the image pixel point in the area according to the target display effect of the area by determining the image to the screen according to the target display effect of the area. Applying a voltage to the corresponding grating cylinder of the region to shift the alignment direction of the liquid crystal molecules, thereby changing the refractive index of the light, wherein the larger the voltage, the larger the refractive index; according to the voltage, each image pixel in the region is determined. The parallax attribute value of the point.

In an embodiment, the three-dimensional image generating module 1102 may be configured to generate a three-dimensional image according to the image to be processed and the parallax image of the image to be processed in the following manner: determining the image to be processed as a left-eye image, and distorting the image to be processed The image is determined to be a right eye image; or, the image to be processed is determined as a right eye image, and the parallax image of the image to be processed is determined as a left eye image.

As shown in FIG. 12, the apparatus provided in this embodiment may further include an image compression module 1203, an image interfacing module 1204, an image stretching module 1205, and an image display module 1206.

The image compression module 1203 is configured to compress the left eye image and the right eye image included in the three-dimensional image, respectively, in the first direction.

The image interleaving module 1204 is configured to interleave the compressed left eye image and the right eye image in a predetermined format to obtain an interlaced image.

The image stretching module 1205 is configured to stretch the interlaced image in a first direction.

The image display module 1206 is configured to display the stretched interlaced image.

The image compression module 1203 can compress the left eye image and the right eye image according to the aspect ratio of the 3D screen display of the terminal, so as to perform the next image interleaving. The image interleaving module 1204 can interleave the compressed left eye image and the right eye image in a specific format, respectively. The image stretching module 1205 can stretch the interlaced images in equal proportions according to the aspect ratio of the screen to achieve a full screen effect. The image display module 1206 can adopt a 3D grating technology to apply a 3D grating film on the display screen, so that the left and right eyes of the person receive different left and right eye image data.

For a description of the image processing apparatus provided in this embodiment, reference may be made to the description of the image processing method described above, and thus no further details are provided herein.

The embodiment of the present application further provides an image processing method, including: determining a selected area of an image to be processed according to an instruction; and adjusting a display effect of the selected area to an out screen or an on-screen effect.

In an exemplary embodiment, the selected area may include at least one first selected area and at least one second selected area; the display effect of the first selected area is a screen-out effect, and the display effect of the second selected area is an on-screen effect. For example, the first selected area may include a dotted line frame area in FIG. 6, and the second selected area may include a dotted line frame marked area in FIG.

The description of the image processing method provided in this embodiment can be described with reference to the example shown in FIG. 3, and thus will not be further described herein.

In addition, the embodiment of the present application further provides a computer readable medium storing an image processing program, where the image processing program is executed by a processor to implement the steps of the image processing method.

All or some of the steps, systems, functional blocks or units in the methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules or units mentioned in the above description does not necessarily correspond to the division of physical components. For example, one physical component can have multiple functions, or one function or step can be performed cooperatively by several physical components. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor; some or all of the components may also be implemented as hardware; some or all of the components may also be implemented It is an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer readable medium, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and nonvolatile, implemented in any method or technology for storing information, such as computer readable instructions, data structures, program modules, or other data. , removable and non-removable media. The computer storage medium includes, but is not limited to, a random access memory (RAM), a read only memory (ROM), and an electrically erasable programmable read-only memory (EEPROM). Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical disc storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device Or any other medium that can be used to store the desired information and that can be accessed by the computer. Moreover, it is well known to those skilled in the art that communication media typically includes computer readable instructions, data structures, program modules or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and can include any information delivery media. .

Claims

An image processing method comprising:

Adjusting a disparity attribute value of an image pixel point in at least one region of the image to be processed, and generating a disparity image of the image to be processed;

A three-dimensional image is generated according to the image to be processed and the parallax image of the image to be processed.
The method according to claim 1, wherein the adjusting the disparity attribute value of the image pixel point in the at least one region of the image to be processed and generating the disparity image of the image to be processed comprises:

Dividing the image to be processed into at least one region;

For one of the at least one region, a disparity attribute value of an image pixel point in the region is determined according to a target display effect of the region.
The method according to claim 2, wherein the determining a disparity attribute value of an image pixel point in the area according to a target display effect of the area comprises:

Determining an offset of each image pixel point in the area according to a target display effect of the area; determining, according to an offset of the image pixel point, for each image pixel point in the area The value of the disparity attribute of the image pixel.
The method according to claim 3, wherein the determining the disparity attribute value of the image pixel point according to the offset of the image pixel point comprises:

Calculating a disparity attribute value of the image pixel point according to an offset of the image pixel point, a spacing between the left and right eyes of the person, and a pixel density of the image to be processed; or

The parallax attribute value of the image pixel point is calculated according to the offset of the image pixel point, the spacing between the left and right eyes of the person, and the distance between the human eye and the screen.
The method according to claim 4, wherein said calculating a disparity attribute of said image pixel point based on an offset of said image pixel point, a spacing between left and right eyes of a person, and a pixel density of said image to be processed Values, including:

Calculating the parallax attribute value of the image pixel according to the following formula:

Where M is the parallax attribute value of the image pixel, d is the offset of the image pixel point, T is the spacing between the left and right eyes of the person, and PPI is the pixel density.
The method according to claim 4, wherein said calculating a parallax attribute of said image pixel point according to an offset of said image pixel point, a spacing between a left and right eyes of a person, and a distance between a human eye and a screen Values, including:

Calculating the parallax attribute value of the image pixel according to the following formula:

M=f×T/d;

Where M is the parallax attribute value of the image pixel point, d is the offset of the image pixel point, T is the distance between the left and right eyes of the person, and f is the distance between the human eye and the screen.
The method according to claim 3, wherein the determining an offset of each image pixel in the region according to a target display effect of the region comprises:

When the distance between the human eye and the screen has not changed, determining an initial parallax attribute value of each image pixel point in the area according to the target display effect of the area;

After the distance between the human eye and the screen is changed, the initial parallax attribute value of the image pixel point, the distance between the changed human eye and the screen, and the distance between the left and right eyes of the person are determined in the human eye. The offset of the pixel of the image after the distance between the screens changes.
The method according to claim 3, wherein the determining an offset of each image pixel in the region according to a target display effect of the region comprises:

When the target display effect of the area is set to the on-screen effect, determining that the offset d of each image pixel in the area ranges from greater than 0 and less than T;

When the target display effect of the area is set to the screen effect, determining that the offset d of each image pixel in the area ranges from greater than T and less than 2T;

Where T is the spacing between the left and right eyes of the person.
The method of claim 2, the adjusting the disparity attribute value of the image pixel in the at least one region of the image to be processed, and generating the disparity image of the image to be processed, further comprising:

When the target display effect is not set in the area, the original parallax attribute value of each image pixel in the area is kept unchanged; or the parallax attribute value of each image pixel in the area is updated to a preset value.
The method according to claim 2, wherein the determining a disparity attribute value of an image pixel point in the area according to a target display effect of the area comprises:

Determining, according to the target display effect of the area, applying a voltage to the grating cylinder mirror corresponding to the area on the screen, so as to cause an offset of the alignment direction of the liquid crystal molecules; determining each image pixel point in the area according to the voltage The value of the parallax property.
An image processing apparatus comprising:

a parallax image generating module, configured to adjust a parallax property value of the image pixel point in at least one region of the image to be processed, and generate a parallax image of the image to be processed;

The three-dimensional image generating module is configured to generate a three-dimensional image according to the image to be processed and the parallax image of the image to be processed.
The apparatus according to claim 11, wherein the parallax image generating module is configured to divide the image to be processed into at least one region; and for an area in the at least one region, according to the region to be set The target display effect determines the parallax attribute value of the image pixel point in the area.
The apparatus according to claim 12, wherein the parallax image generating module is configured to determine a parallax attribute value of an image pixel point in the area according to a target display effect set to the area by:

Determining an offset of each image pixel point in the area according to a target display effect of the area; determining, according to an offset of the image pixel point, for each image pixel point in the area The parallax attribute value of the image pixel; or,

Determining, according to the target display effect of the area, applying a voltage to the grating cylinder mirror corresponding to the area on the screen, so as to cause an offset of the alignment direction of the liquid crystal molecules; determining each image pixel point in the area according to the voltage The value of the parallax property.
A terminal comprising: a memory, a processor, and an image processing program stored in the memory and operable on the processor, the image processing program being implemented by the processor when implemented in claims 1 to 10 The steps of the image processing method of any of the above.
A computer readable medium storing an image processing program, the image processing program being executed by a processor to implement the steps of the image processing method according to any one of claims 1 to 10.
An image processing method comprising:

Determining a selected area of the image to be processed according to the instruction;

Adjusting the display effect of the selected area to an on-screen effect or an on-screen effect.
The method according to claim 16, wherein the selected area comprises at least one first selected area and at least one second selected area; the display effect of the first selected area is a screen effect, and the second selected area The display effect is the screen effect.