CN114339071A

CN114339071A - Image processing circuit, image processing method and electronic device

Info

Publication number: CN114339071A
Application number: CN202111636624.9A
Authority: CN
Inventors: 董志祥
Original assignee: Vivo Mobile Communication Co Ltd
Current assignee: Vivo Mobile Communication Co Ltd
Priority date: 2021-12-28
Filing date: 2021-12-28
Publication date: 2022-04-12

Abstract

The application discloses an image processing circuit, an image processing method and electronic equipment, and belongs to the technical field of image processing. The image processing circuit comprises a main control chip and an image processing chip, wherein the main control chip is connected with the image processing chip; the main control chip is used for outputting interface data and first image data according to the acquired first data; the image processing chip is used for carrying out blurring processing on the first image data and outputting second image data; and fusing the second image data with the interface data to output second data.

Description

Image processing circuit, image processing method and electronic device

Technical Field

The present application belongs to the field of image processing technologies, and in particular, to an image processing circuit, an image processing method, and an electronic device.

Background

The camera blurring function is mainly applied to a portrait scene, and the purpose of highlighting a photographing subject is achieved through blurring the background.

The existing photographing blurring technology needs to be realized by a blurring algorithm on software, and when blurring processing is performed on some complex photographing scenes, the processing speed is low, and a good blurring effect is difficult to obtain.

Disclosure of Invention

An object of the embodiments of the present application is to provide an image processing circuit, an image processing method, and an electronic device, which can achieve a better photographing blurring effect by performing hardware blurring through an independent image processing chip.

In a first aspect, an embodiment of the present application provides an image processing circuit, which includes a main control chip and an image processing chip, where the main control chip is connected to the image processing chip; the main control chip is used for outputting interface data and first image data according to the acquired first data; the image processing chip is used for carrying out blurring processing on the first image data and outputting second image data; and fusing the second image data with the interface data to output second data.

In a second aspect, an embodiment of the present application provides an image processing circuit, which is characterized by comprising a main control chip and an image processing chip, wherein the main control chip comprises a separation unit, a first interface and a second interface, and the image processing chip comprises a virtualization unit, a fusion unit, a third interface and a fourth interface; the separation unit is used for separating the acquired first data to obtain interface data and first image data; the separation unit is respectively connected with the first interface and the second interface, the first interface is used for outputting interface data, and the second interface is used for outputting first image data; the first interface is connected with the third interface, the second interface is connected with the fourth interface, the virtualization unit is connected with the fourth interface, and the fusion unit is respectively connected with the third interface and the virtualization unit; the blurring unit is used for blurring the first image data to obtain second image data; the fusion unit is used for fusing the second image data and the interface data to obtain second data.

In a third aspect, an embodiment of the present application provides an image processing method, which is applied to the image processing circuit according to any one of the second aspects, and the method includes: the main control chip outputs interface data and first image data according to the acquired first data; the image processing chip performs blurring processing on the first image data and outputs second image data; and the image processing chip fuses the second image data and the interface data and outputs second data.

In a fourth aspect, an embodiment of the present application provides an electronic device, which includes an image sensor, a display screen, and the image processing circuit according to any one of the first aspect, where the image processing circuit is connected to the image sensor and the display screen, respectively.

In a fifth aspect, the present application provides an electronic device, which includes an image sensor, a display screen, and the image processing circuit according to any one of the second aspects, where the image processing circuit is connected to the image sensor and the display screen, respectively.

In the embodiment of the application, the image processing circuit comprises a main control chip and an image processing chip, wherein the main control chip is connected with the image processing chip; the main control chip is used for outputting interface data and first image data according to the acquired first data; the image processing chip is used for carrying out blurring processing on the first image data and outputting second image data; and fusing the second image data with the interface data to output second data. Therefore, the embodiment of the application can realize better photographing blurring effect by performing hardware blurring through the independent image processing chip; the parallel processing of the main control chip and the image processing chip can be realized, the processing efficiency is greatly improved, and the load power consumption of the main control chip is reduced; due to the time-sharing processing and the improvement of the blurring performance, the continuous snapshot blurring is favorably realized.

Drawings

Fig. 1 is a schematic structural diagram of an image processing circuit provided in this embodiment;

fig. 2 is a schematic diagram of an image blurring processing logic in the single shot blurring preview mode according to the present embodiment;

FIG. 3 is a schematic diagram of the image processing logic of the conventional shooting mode provided by the present embodiment;

fig. 4 is a schematic diagram of an image blurring processing logic in the single shot blurring non-preview mode according to the present embodiment;

fig. 5 is a schematic diagram of an image blurring processing logic in the double-shot blurring preview mode according to the present embodiment;

fig. 6 is a schematic diagram of an image blurring processing logic in the double shot blurring non-preview mode according to the present embodiment;

fig. 7 is a block schematic diagram of another image processing circuit provided in the present embodiment;

fig. 8 is a flowchart of an image processing method provided in the present embodiment;

fig. 9 is a schematic diagram of a hardware structure of another electronic device provided in this embodiment.

Detailed Description

The technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present disclosure.

The terms first, second and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that embodiments of the application may be practiced in sequences other than those illustrated or described herein, and that the terms "first," "second," and the like are generally used herein in a generic sense and do not limit the number of terms, e.g., the first term can be one or more than one. In addition, "and/or" in the specification and claims means at least one of connected objects, a character "/" generally means that a preceding and succeeding related objects are in an "or" relationship.

The image processing circuit provided by the embodiment of the present application is described in detail with reference to the accompanying drawings through specific embodiments and application scenarios thereof.

Referring to fig. 1, an image processing circuit 100 provided in the present embodiment includes a main control chip 110 and an image processing chip 120.

The main control chip 110 is connected to the image processing chip 120; the main control chip 110 is configured to output interface data and first image data according to the acquired first data; the image processing chip 120 is configured to perform blurring processing on the first image data and output second image data; and fusing the second image data with the interface data to output second data.

In detail, the first data may be obtained from image data output by an image sensor of the electronic device. The electronic device may be a smartphone of a user, and the image sensor of the electronic device may be a camera of the smartphone. In detail, when the user turns on the shooting function, the camera may acquire image data and output the image data to the main control chip 110.

In detail, the main control chip 110 may obtain the interface data and the first image data to be blurred by separating the first data, and output the interface data and the first image data to the image processing chip 120 respectively. The image processing chip 120 performs blurring processing on the first image data to obtain second image data, and then the second image data and the interface data may be fused to obtain second data, which may be output to a display screen for displaying.

As shown in fig. 2, fig. 2 shows an image blurring processing logic of the monoscopic blurring preview mode. Referring to fig. 2, the main control chip 110 of the present embodiment may be an AP (Application Processor) main control chip in an electronic device, for example, an ISP IC chip. The main control chip firstly obtains data output by an Image sensor, and the data is processed by an internal Image Signal Processing (ISP) module and a surfefinger module of the main control chip to obtain first Image data. The first image data can be outputted to an image processing chip (or referred to as an image independent processing chip) for blurring processing.

As shown in fig. 2, the image data collected by the image sensor may be transmitted to a DRAM (Dynamic Random Access Memory) of the main control chip through an Interface of MIPI (Mobile Industry Processor Interface) csi (camera Serial Interface) RX0, then output to the ISP module through the DRAM, processed by the ISP module, and then output to the surfefinger module of the frame.

In detail, the ISP module may perform basic effect processing on the input image data and output the processed data.

In detail, in order to realize the display of the first image data on the display panel, interface data necessary for the display operation needs to be acquired.

Referring to fig. 2, the surfefinger module of the frame inside the main control chip may also obtain interface data. The interface data can be output to an image processing chip to be fused with the blurred image data.

In detail, if the camera is in the shooting blurring mode, blurring processing is required, so the main control chip 110 can output the first image data and the interface data to the image processing chip 120 respectively, so as to perform corresponding blurring processing in the image processing chip 120.

The shooting blurring mode may correspond to single blurring or double blurring, preview blurring or non-preview blurring, and a shooting mode or a recording mode. As shown in fig. 2, fig. 2 corresponds to the telephoto preview blurring mode.

Referring to fig. 2, the surfefinger module of the frame inside the main control chip outputs the first image data (IMGAE) and the interface data (UI) to the image processing chip, respectively. Specifically, the interface data is transmitted to the image processing chip through an interface MIPI DSI0 of the main control chip and an interface MIPI DSI RX0 of the image processing chip, and the first image data is transmitted to the image processing chip through an interface MIPI DSI1 of the main control chip and an interface MIPI DSI RX1 of the image processing chip.

In detail, the main control chip 110 may communicate data with the image processing chip 120 through a standard MIPI DSI protocol.

In this embodiment, the image processing chip 120 performs the blurring operation and the fusion operation when the camera is in the shooting blurring mode.

In detail, the camera blurring function is mainly applied to a portrait scene, and the purpose of highlighting the subject of photographing is achieved by blurring the background. The virtual module in the image processing chip 120 may perform blurring effect processing according to the blurring parameter issued by the application, so as to perform blurring processing on the background in the first image data, so as to play a role in highlighting the portrait in the first image data, thereby completing the blurring operation.

For example, when the first image data is taken from image data captured by one camera, depth of field information may be obtained through software analysis in combination with the image data, and then corresponding blurring processing is performed according to the depth of field information, thereby completing background blurring operation.

For another example, when the first image data is taken from image data captured by two cameras, the depth of field information may be calculated by combining the position information of the two cameras and the image data captured by the master camera and the slave camera, and the image captured by the master camera may be background-blurred according to the obtained depth of field information.

As shown in fig. 2, the first image data may be first output to a DRAM of the image processing chip, and then output to a blurring module of the image processing chip via the DRAM for blurring processing, so as to obtain second image data after blurring. And then the blurring module outputs the second image data to a fusion module of the image processing chip so as to be fused with the interface data.

In this embodiment, the interface data and the second image data obtained after the blurring processing are input to the fusion module together for fusion processing, so as to obtain the second data to be displayed. The second data can be further output to the display panel, so that the user can check the blurring effect of the shot image in real time.

As shown in fig. 2, the interface data may be output to the DRAM of the image processing chip first, and then output to the fusion module of the image processing chip through the DRAM.

As shown in fig. 2, the merged data to be displayed is further transmitted to the display panel for preview display through the interface MIPI DSI TX0 of the image processing chip. It can be seen that fig. 2 provides a hardware virtualization scheme based on the ISP IC chip, which can achieve a faster virtualization speed and achieve the purpose of reducing the virtualization power consumption.

In the embodiment, hardware virtualization processing is performed based on the image processing chip to replace the original software virtualization algorithm which needs to run in a CPU (main control chip), so that a faster photographing virtualization function can be realized, and the performance and power consumption problems caused by the fact that the virtualization algorithm occupies CPU resources can be avoided.

Therefore, the embodiment of the application can realize better photographing blurring effect by performing hardware blurring through the independent image processing chip; the parallel processing of the main control chip and the image processing chip can be realized, the processing efficiency is greatly improved, and the load power consumption of the main control chip is reduced; due to the time-sharing processing and the improvement of the blurring performance, the continuous snapshot blurring is favorably realized.

In addition, compared with the existing pure software virtualization scheme, the CPU resource needs to be occupied, so that the user needs to wait for the next photographing after the virtualization is completed.

Moreover, the virtualization effect of the existing pure software virtualization scheme depends on the performance of the used virtualization algorithm on one hand, the better the virtualization algorithm is, the better the final virtualization effect is, and on the other hand, the virtualization effect depends on the performance of the CPU, but the performance and power consumption problems are caused when the CPU runs the virtualization algorithm. In the embodiment, the image processing chip is introduced to convert the virtualization processing of the software into the virtualization processing of the hardware, so that the virtual algorithm software does not need to be used on the CPU, and the dependence on the CPU performance and the influence on the CPU are also obviously reduced. In addition, the embodiment can realize the parallel processing of the CPU and the image processing chip, greatly improves the whole processing efficiency and reduces the load power consumption of the CPU end, and the time-sharing processing and the performance improvement also provide support for virtual continuous snapshot.

As described above, the present embodiment can be based on the image processing chip 120 to realize the image blurring process. In addition to this, the present embodiment can also achieve a regular shooting purpose based on the image processing chip 120.

In detail, when the camera is in a shooting non-blurring mode (i.e., a normal shooting mode), corresponding data to be displayed is obtained according to the first image data and the interface data.

In this embodiment, if the camera is in the normal shooting mode without blurring, the image processing chip 120 may directly process the image data and the interface data in the main control chip 110 as the data to be displayed without blurring or fusing.

At this time, the image processing chip 120 may not be in the blurring processing mode but in the BYPASS mode supporting the conventional shooting mode, and the image processing chip 120 directly outputs the data to be displayed to the display panel in the BYPASS mode without performing data processing, which is equivalent to achieving the effect of directly connecting the main control chip 110 and the display panel.

As shown in fig. 3, fig. 3 shows the image processing logic of the conventional shooting mode. Referring to fig. 3, as for the image blurring processing logic part in the single shot blurring preview mode, the main control chip may obtain image data acquired by the image sensor, and the image data is processed by the ISP module and then output to the surfefinger module. In contrast, in the conventional shooting mode, the image data output by the ISP module is combined with the interface data in the surfefinger module to form data to be displayed.

As shown in fig. 3, the data to be displayed is transmitted to the image processing chip through the interface MIPI DSI0 of the main control chip and the interface MIPI DSI RX0 of the image processing chip, and is directly transmitted to the display panel for preview display through the interface MIPI DSI TX0 of the image processing chip without being processed in the image processing chip, so that the user can view the picture effect of the image shot by the user in real time.

As shown in fig. 3, the ISP module of the main control chip processes the image data to output the processed image data to the surfflag module, so as to support the user to view the shooting effect in real time, and simultaneously, the ISP module may also output the image data to the shooting application program of the electronic device, and the shooting application program may perform encoding and storage processing correspondingly, so as to implement local storage of the image shot by the user, thereby completing a routine shooting process.

Therefore, under the condition that the hardware connection mode in the electronic equipment is not changed, the embodiment can support the realization of the image blurring effect and the conventional shooting effect, and the processing flow of the conventional shooting is simple and has high efficiency.

In detail, in order to ensure that the image processing chip 120 can accurately perform the processing operation conforming to the camera operation mode, the operation mode of the image processing chip 120 may be matched with the camera operation mode. For example, when the camera operation mode is the normal shooting mode, the image processing chip 120 is in the BYPASS mode.

In a feasible implementation manner, the image processing chip 120 may automatically switch its own operation mode as required according to the operation mode of the camera. For example, the image processing chip 120 switches to a mode for executing image blurring processing when the camera is in an image blurring mode to realize an image blurring effect, and switches to a BYPASS mode when the camera is in a normal shooting mode to directly connect the main control chip 110 and the display panel, so as to support a fast preview display of an image shot by a user.

In another possible implementation, the main control chip 110 may control the switching of the operation mode of the image processing chip 120 as needed according to the operation mode of the camera. For example, when the camera is in the image blurring mode, the main control chip 110 controls the image processing chip 120 to switch to a mode for performing image blurring processing so as to achieve an image blurring effect, and when the camera is in the normal shooting mode, the image processing chip 120 is controlled to switch to the BYPASS mode so as to directly connect the main control chip 110 and the display panel, thereby supporting fast preview display of the image shot by the user.

Based on the above, to implement the image blurring as needed, the working mode of the camera may be obtained, for example, the working mode may be a normal shooting mode or a blurring shooting mode, and the blurring shooting mode may specifically be a blurring preview mode or a blurring non-preview mode, and may be a single-shot blurring mode or a double-shot blurring mode.

In this embodiment, for any blurring shooting mode, the image processing chip 120 needs to perform blurring, fusion, and other processing, so that the operation mode can be switched to a mode for performing image blurring preview/non-preview processing. On the contrary, in the normal shooting mode, the image processing chip 120 does not need to perform blurring, merging, and other processing, and thus the operation mode thereof can be switched to the BYPASS mode.

As shown in fig. 2 and fig. 3, the main control chip 110 can control the image processing chip using the interface I2C/SPI to implement on-demand switching of the operation mode of the image processing chip.

As described above, the main control chip 110 can actively control the mode of the image processing chip 120 according to the camera working mode, so as to achieve the purpose of accurate blurring or blurring.

In detail, in the shooting ghosting preview mode, the image processing chip 120 needs to fulfill the ghosting image preview purpose. In the blurring non-preview mode, the image processing chip 120 not only needs to preview the blurring image, but also needs to support local storage of the blurring image.

Based on this, in one embodiment of the present application, the photographing blurring mode includes a photographing blurring non-preview mode. The shooting blurring non-preview mode may be a single-shot blurring non-preview mode or a double-shot blurring non-preview mode. In this way, the main control chip 110 also receives the second image data output by the image processing chip 120.

In this embodiment, since the blurred image data needs to be stored, the image processing chip 120 needs to output the second image data to the main control chip 110 after completing the image blurring process to obtain the second image data. The main control chip 110 may further store the second image data in a memory of the electronic device.

Based on the above, in an embodiment of the present application, the main control chip 110 is further configured to store the second image data.

As shown in fig. 4, fig. 4 illustrates the image blurring processing logic of the monoscopic blurring non-preview mode. Referring to fig. 4, on one hand, the image data obtained after the blurring is fused with the interface data in the fusion module inside the image processing chip, the data to be displayed obtained after the fusion is directly sent to the display panel for preview display, and on the other hand, the image data obtained after the blurring needs to be transmitted to another DRAM of the main control chip through the interface MIPI CSI TX1 of the image processing chip and the interface MIPI CSI RX1 of the main control chip.

Correspondingly, the main control chip stores the received second image data to the memory, so that the local storage of the blurred image is completed.

As shown in fig. 4, after the second image data is written into the DRAM, the main control chip may further output the second image data to a photographing application program of the electronic device, and the photographing application program may correspondingly perform encoding and storage processing on the image data, thereby completing a blurring photographing process.

Taking the blurring photographing as an example, after a blurring photographing process is completed, the camera is switched from the blurring photographing mode to the blurring photographing preview mode, the control image processing chip 120 corresponding to the main control chip 110 is in the working mode corresponding to the blurring photographing preview mode, and based on this, the user can perform the next photographing action.

Therefore, in the embodiment, by adding the image processing chip which also has the driving function of the display panel, the flow of software processing at the CPU end by the blurring algorithm is put into the image processing chip for hardware-level processing, so that the time delay of photographing caused by the running of the blurring algorithm by the CPU can be greatly reduced, and the photographing experience of the user is improved. In addition, because the blurring process is completed in the image processing chip, continuous shooting of blurring photographing can also be realized.

Compared with the single-camera blurring effect, the double-camera blurring effect is relatively better. The embodiment can not only realize the purpose of single-shot blurring, but also support the realization of double-shot blurring.

In this embodiment, when the camera is in a single-camera shooting mode, data output by one of the image sensors of the electronic device is acquired to obtain first image data.

In detail, in the case of single shot blurring, image data acquired by one camera is acquired to obtain first image data accordingly.

As shown in fig. 2 and 4, when the camera is in the single shooting blurring mode, image data acquired by one image sensor is output to the ISP module of the main control chip for processing, the processed data is further output to the surfefinger module, and then the surfefinger module separates the first image data from the interface data and respectively transmits the separated first image data to the image processing chip.

In this embodiment, when the camera is in a dual-camera shooting mode, data output by two image sensors of the electronic device are acquired, so as to obtain first image data.

Based on this, in an embodiment of the present application, the main control chip 110 is further configured to obtain first original image data acquired by a first camera and second original image data acquired by a second camera, and generate the first image data according to the first original image data and the second original image data.

In detail, under the double-shot blurring condition, the image data collected by the two cameras are respectively acquired so as to obtain the first image data.

As shown in fig. 5 and fig. 6, fig. 5 shows the image blurring processing logic of the bi-photography blurring preview mode, fig. 6 shows the image blurring processing logic of the bi-photography blurring non-preview mode, and fig. 6 is compared with fig. 5, and an operation of encoding and storing the blurred image data needs to be additionally performed.

Referring to fig. 5 and 6, when the camera is in the double-shot blurring mode, the image data collected by the two image sensors are respectively output to the ISP module of the main control chip and processed together, the processed data are further output to the surfefinger module, and then the surfefinger module separates the first image data from the interface data and respectively sends the separated first image data to the image processing chip.

In a feasible implementation manner, as shown in fig. 5 and 6, in consideration of the limitation of the number of interfaces, the ISP module may process and combine image data of two cameras into one image according to size information of a main shot image and a sub shot image set by an application, and the surfefinger module further transmits the combined image to the blurring module of the image processing chip through the interfaces MIPI DSI1 and MIPI DSI RX 1. After receiving the merged image, the blurring module may extract the main shot image and the secondary shot image from the merged image according to the size information of the main shot image and the secondary shot image set by the application, further calculate depth-of-field information according to the main shot image, the secondary shot image and the position information of the two cameras, and perform blurring processing on the main shot image according to the depth-of-field information to obtain blurred image data.

In other possible implementations, the image merging operation and the corresponding image splitting operation may not be performed without considering the limitation of the number of interfaces, and the main control chip 110 may send the main shot image and the slave shot image to the image processing chip 120 respectively.

Based on the above, in an embodiment of the present application, the image processing chip 120 is specifically configured to: extracting a first image and a second image from the first image data; acquiring depth-of-field information according to the first image, the second image and the position information of the first camera and the second camera; and performing blurring processing on the first image according to the depth information, and outputting second image data.

Optionally, the first image is an image captured by the main camera, and the second image is an image captured by the sub camera.

Therefore, in the embodiment, by adding the image processing chip which also has the driving function of the display panel, the flow of software processing at the CPU end by the blurring algorithm is put into the image processing chip for hardware-level processing, so that the time delay of photographing caused by the running of the blurring algorithm by the CPU can be greatly reduced, and the photographing experience of the user is improved. In addition, because the calculation of the depth of field and the blurring processing are completed in the image processing chip, continuous shooting of blurring shooting can be realized.

Based on the above, the present embodiment further provides an electronic device, which includes an image sensor, a display screen, and the image processing circuit described in any of the above embodiments, where the image processing circuit is connected to the image sensor and the display screen, respectively. The image sensor may be a camera.

Based on the above and referring to fig. 2 to 6, as shown in fig. 7, an image processing circuit 200 provided in the present embodiment includes a main control chip 210 and an image processing chip 220. The main control chip 210 includes a separation unit 2101, a first interface 2102 and a second interface 2103, and the image processing chip 220 includes a blurring unit 2201, a fusion unit 2202, a third interface 2203 and a fourth interface 2204.

Referring to fig. 2, the main control chip 210 in the present embodiment may be the AP main control chip shown in fig. 2, and the image processing chip 220 may be an image independent processing chip shown in fig. 2.

The separation unit 2101 included in the main control chip 210 may be a surfefinger module shown in fig. 2, the first interface 2102 may be an interface MIPI DSI0 shown in fig. 2, and the second interface 2103 may be an interface MIPI DSI1 shown in fig. 2.

The third interface 2203 included in the image processing chip 220 may be an interface MIPI DSI RX0 shown in fig. 2, and the fourth interface 2204 may be an interface MIPI DSI RX1 shown in fig. 2.

In this embodiment, the separation unit 2101 is configured to separate the acquired first data to obtain interface data and first image data; the separation unit 2101 is connected to the first interface 2102 and the second interface 2103, respectively, where the first interface 2102 is configured to output interface data, and the second interface 2103 is configured to output first image data.

In this embodiment, the first interface 2102 is connected to the third interface 2203, the second interface 2103 is connected to the fourth interface 2204, the virtualization unit 2201 is connected to the fourth interface 2204, and the fusion unit 2202 is connected to the third interface 2203 and the virtualization unit 2201, respectively; the blurring unit 2201 is configured to perform blurring processing on the first image data to obtain second image data; the fusion unit 2202 is configured to fuse the second image data and the interface data to obtain second data.

In an embodiment of the present application, the main control chip 210 further includes a fifth interface, a sixth interface, and a synthesis unit, where the synthesis unit is connected to the fifth interface and the sixth interface respectively.

In this embodiment, the fifth interface may be the interface MIPI CSI RX0 shown in fig. 5, the sixth interface may be the interface MIPI CSI RX1 shown in fig. 5, and the synthesis unit may be the ISP module shown in fig. 5.

In this embodiment, the fifth interface is configured to obtain first original image data collected by a first camera, the sixth interface is configured to obtain second original image data collected by a second camera, and the combining unit is configured to generate the first image data according to the first original image data and the second original image data.

In an embodiment of the present application, the blurring unit 2201 is specifically configured to: extracting a first image and a second image from the first image data; acquiring depth-of-field information according to the first image, the second image and the position information of the first camera and the second camera; and performing blurring processing on the first image according to the depth information.

In an embodiment of the present application, the image processing chip 220 further includes a seventh interface, and the seventh interface is configured to output second image data; the main control chip 210 further includes an eighth interface and a storage unit, the eighth interface is respectively connected to the seventh interface and the storage unit, and the storage unit is configured to store the second image data.

The seventh interface in this embodiment may be the interface MIPI CSI TX1 shown in fig. 4, the eighth interface may be the interface MIPI CSI RX1 shown in fig. 4, and the storage unit may be the DRAM connected to the encoding unit shown in fig. 4.

In an embodiment of the present application, the main control chip 210 further includes a control unit, and the control unit is configured to control the blurring unit 2201 to perform blurring processing on the first image data, and control the fusion unit 2202 to fuse the second image data and the interface data.

In this embodiment, the main control chip 210 may also control other chips, and specifically may control the virtual unit and the fusion unit in the image processing chip 220 to execute corresponding processing.

As shown in fig. 8, an image processing method provided in this embodiment is applied to an image processing circuit according to any embodiment of the present application, and the method may include the following steps 310 and 320:

and step 310, the main control chip outputs interface data and first image data according to the acquired first data.

Step 320, the image processing chip performs blurring processing on the first image data and outputs second image data;

and 330, fusing the second image data and the interface data by the image processing chip, and outputting second data.

In an embodiment of the present application, before the main control chip outputs the interface data and the first image data according to the acquired first data, the method further includes: the main control chip acquires first original image data acquired by a first camera and second original image data acquired by a second camera, and generates the first image data according to the first original image data and the second original image data.

In an embodiment of the present application, the blurring processing the first image data by the image processing chip includes: the image processing chip extracts a first image and a second image from the first image data; the image processing chip acquires depth-of-field information according to the first image, the second image and the position information of the first camera and the second camera; and the image processing chip performs blurring processing on the first image according to the depth of field information.

In an embodiment of the present application, after the outputting the second image data, the method further includes: and the main control chip stores the second image data.

In one embodiment of the present application, the image processing circuit is connected to a display screen; after the outputting the second data, further comprising: and the display screen displays the second data.

The image processing circuit in the embodiments of the present application may be a component in an electronic device, such as an integrated circuit. The electronic device may be a terminal, or may be a device other than a terminal. The electronic Device may be, for example, a Mobile phone, a tablet computer, a notebook computer, a palm top computer, a vehicle-mounted electronic Device, a Mobile Internet Device (MID), an Augmented Reality (AR)/Virtual Reality (VR) Device, a robot, a wearable Device, an ultra-Mobile personal computer (UMPC), a netbook or a Personal Digital Assistant (PDA), and the like, and may also be a server, a Network Attached Storage (NAS), a Personal Computer (PC), a Television (TV), a teller machine, a self-service machine, and the like, and the embodiments of the present application are not particularly limited.

The electronic device in the embodiment of the present application may be an apparatus having an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, and embodiments of the present application are not limited specifically.

The image processing circuit provided in the embodiment of the present application can implement each process implemented in the method embodiment of fig. 8, achieve the same technical effect, and is not described herein again to avoid repetition.

Fig. 9 is a schematic hardware structure diagram of an electronic device 1000 implementing the embodiment of the present application.

The electronic device 1000 includes, but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, a processor 1010, an image processing chip, and the like.

Those skilled in the art will appreciate that the electronic device 1000 may further comprise a power source (e.g., a battery) for supplying power to various components, and the power source may be logically connected to the processor 1010 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system. The electronic device structure shown in fig. 9 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than those shown, or combine some components, or arrange different components, and thus, the description is not repeated here.

Wherein, the processor 1010 is connected with the image processing chip; the processor 1010 is configured to output interface data and first image data according to the acquired first data; the image processing chip is used for carrying out blurring processing on the first image data and outputting second image data; and fusing the second image data with the interface data to output second data.

In the embodiment of the application, hardware blurring is performed through an independent image processing chip, so that a better photographing blurring effect can be realized; the parallel processing of the main control chip and the image processing chip can be realized, the processing efficiency is greatly improved, and the load power consumption of the main control chip is reduced; due to the time-sharing processing and the improvement of the blurring performance, the continuous snapshot blurring is favorably realized.

Optionally, the processor 1010 is further configured to obtain first original image data acquired by a first camera and second original image data acquired by a second camera, and generate the first image data according to the first original image data and the second original image data.

Optionally, the image processing chip is specifically configured to: extracting a first image and a second image from the first image data; acquiring depth-of-field information according to the first image, the second image and the position information of the first camera and the second camera; and performing blurring processing on the first image according to the depth information, and outputting second image data.

Optionally, the processor 1010 is further configured to store the second image data.

Optionally, the processor 1010 includes a separation unit, a first interface, and a second interface, and the image processing chip includes a blurring unit, a fusion unit, a third interface, and a fourth interface; the separation unit is used for separating the acquired first data to obtain interface data and first image data; the separation unit is respectively connected with the first interface and the second interface, the first interface is used for outputting interface data, and the second interface is used for outputting first image data; the first interface is connected with the third interface, the second interface is connected with the fourth interface, the virtualization unit is connected with the fourth interface, and the fusion unit is respectively connected with the third interface and the virtualization unit; the blurring unit is used for blurring the first image data to obtain second image data; the fusion unit is used for fusing the second image data and the interface data to obtain second data.

Optionally, the processor 1010 further includes a fifth interface, a sixth interface, and a combining unit, where the combining unit is connected to the fifth interface and the sixth interface respectively; the fifth interface is used for acquiring first original image data acquired by a first camera, the sixth interface is used for acquiring second original image data acquired by a second camera, and the synthesis unit is used for generating the first image data according to the first original image data and the second original image data.

Optionally, the blurring unit is specifically configured to: extracting a first image and a second image from the first image data; acquiring depth-of-field information according to the first image, the second image and the position information of the first camera and the second camera; and performing blurring processing on the first image according to the depth information.

Optionally, the image processing chip further includes a seventh interface, where the seventh interface is configured to output second image data; the processor 1010 further includes an eighth interface and a storage unit, the eighth interface is respectively connected to the seventh interface and the storage unit, and the storage unit is configured to store the second image data.

Optionally, the processor 1010 further includes a control unit, and the control unit is configured to control the blurring unit to perform blurring processing on the first image data, and control the fusion unit to fuse the second image data with the interface data.

It should be understood that in the embodiment of the present application, the input Unit 1004 may include a Graphics Processing Unit (GPU) 10041 and a microphone 10042, and the Graphics Processing Unit 10041 processes image data of still pictures or videos obtained by an image capturing device (such as a camera) in a video capturing mode or an image capturing mode. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also referred to as a touch screen. The touch panel 10071 may include two parts, a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail herein.

The memory 1009 may be used to store software programs as well as various data. The memory 1009 may mainly include a first storage area storing a program or an instruction and a second storage area storing data, wherein the first storage area may store an operating system, an application program or an instruction (such as a sound playing function, an image playing function, and the like) required for at least one function, and the like. Further, the memory 1009 may include volatile memory or nonvolatile memory, or the memory x09 may include both volatile and nonvolatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. The volatile Memory may be a Random Access Memory (RAM), a Static Random Access Memory (Static RAM, SRAM), a Dynamic Random Access Memory (Dynamic RAM, DRAM), a Synchronous Dynamic Random Access Memory (Synchronous DRAM, SDRAM), a Double Data Rate Synchronous Dynamic Random Access Memory (Double Data Rate SDRAM, ddr SDRAM), an Enhanced Synchronous SDRAM (ESDRAM), a Synchronous Link DRAM (SLDRAM), and a Direct Memory bus RAM (DRRAM). The memory 1009 in the embodiments of the present application includes, but is not limited to, these and any other suitable types of memory.

Processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor, which primarily handles operations related to the operating system, user interface, and applications, and a modem processor, which primarily handles wireless communication signals, such as a baseband processor. It will be appreciated that the modem processor described above may not be integrated into processor 1010.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Further, it should be noted that the scope of the methods and apparatus of the embodiments of the present application is not limited to performing the functions in the order illustrated or discussed, but may include performing the functions in a substantially simultaneous manner or in a reverse order based on the functions involved, e.g., the methods described may be performed in an order different than that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.

While the present embodiments have been described with reference to the accompanying drawings, it is to be understood that the invention is not limited to the precise embodiments described above, which are meant to be illustrative and not restrictive, and that various changes may be made therein by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. An image processing circuit is characterized by comprising a main control chip and an image processing chip, wherein the main control chip is connected with the image processing chip;

the main control chip is used for outputting interface data and first image data according to the acquired first data;

the image processing chip is used for carrying out blurring processing on the first image data and outputting second image data; and fusing the second image data with the interface data to output second data.

2. The image processing circuit of claim 1, wherein the main control chip is further configured to obtain first raw image data collected by a first camera and second raw image data collected by a second camera, and generate the first image data according to the first raw image data and the second raw image data.

3. The image processing circuit of claim 2, wherein the image processing chip is specifically configured to:

extracting a first image and a second image from the first image data;

acquiring depth-of-field information according to the first image, the second image and the position information of the first camera and the second camera;

and performing blurring processing on the first image according to the depth information, and outputting second image data.

4. The image processing circuit of claim 1, wherein the master control chip is further configured to store the second image data.

5. An image processing circuit is characterized by comprising a main control chip and an image processing chip, wherein the main control chip comprises a separation unit, a first interface and a second interface, and the image processing chip comprises a virtualization unit, a fusion unit, a third interface and a fourth interface;

the separation unit is used for separating the acquired first data to obtain interface data and first image data;

the separation unit is respectively connected with the first interface and the second interface, the first interface is used for outputting interface data, and the second interface is used for outputting first image data;

the first interface is connected with the third interface, the second interface is connected with the fourth interface, the virtualization unit is connected with the fourth interface, and the fusion unit is respectively connected with the third interface and the virtualization unit;

the blurring unit is used for blurring the first image data to obtain second image data;

the fusion unit is used for fusing the second image data and the interface data to obtain second data.

6. The image processing circuit according to claim 5, wherein the main control chip further comprises a fifth interface, a sixth interface and a synthesizing unit, and the synthesizing unit is respectively connected to the fifth interface and the sixth interface;

the fifth interface is used for acquiring first original image data acquired by a first camera, the sixth interface is used for acquiring second original image data acquired by a second camera, and the synthesis unit is used for generating the first image data according to the first original image data and the second original image data.

7. The image processing circuit of claim 6, wherein the blurring unit is specifically configured to:

extracting a first image and a second image from the first image data;

and performing blurring processing on the first image according to the depth information.

8. The image processing circuit according to claim 5, wherein the image processing chip further comprises a seventh interface for outputting second image data;

the main control chip further comprises an eighth interface and a storage unit, the eighth interface is respectively connected with the seventh interface and the storage unit, and the storage unit is used for storing the second image data.

9. The image processing circuit according to claim 5, wherein the main control chip further comprises a control unit, and the control unit is configured to control the blurring unit to perform blurring on the first image data and control the fusion unit to fuse the second image data with the interface data.

10. An image processing method applied to the image processing circuit according to any one of claims 5 to 9, the method comprising:

the main control chip outputs interface data and first image data according to the acquired first data;

the image processing chip performs blurring processing on the first image data and outputs second image data;

and the image processing chip fuses the second image data and the interface data and outputs second data.

11. The image processing method according to claim 10, wherein before the main control chip outputs the interface data and the first image data according to the acquired first data, the method further comprises:

the main control chip acquires first original image data acquired by a first camera and second original image data acquired by a second camera, and generates the first image data according to the first original image data and the second original image data.

12. The image processing method of claim 11, wherein the image processing chip performs blurring processing on the first image data, comprising:

the image processing chip extracts a first image and a second image from the first image data;

the image processing chip acquires depth-of-field information according to the first image, the second image and the position information of the first camera and the second camera;

and the image processing chip performs blurring processing on the first image according to the depth of field information.

13. An electronic device comprising an image sensor, a display screen and the image processing circuit of any of claims 1-4, the image processing circuit being connected to the image sensor and the display screen, respectively.

14. An electronic device comprising an image sensor, a display screen and the image processing circuit of any of claims 5-9, the image processing circuit being connected to the image sensor and the display screen, respectively.