CN115134544A - Image processing method, image processing chip, application processing chip and electronic equipment - Google Patents

Abstract

The application discloses an image processing method, an image processing chip, an application processing chip and electronic equipment. The method comprises the following steps: acquiring a shot current image and a shot historical image; dividing the current image into at least a first area image and a second area image, wherein the change range of the first area image relative to the corresponding area image in the historical image is larger than that of the second area image relative to the corresponding area image in the historical image; compressing the first area image according to a first compression algorithm to obtain first compressed data, and compressing the second area image according to a second compression algorithm to obtain second compressed data, wherein the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm; and transmitting the first compressed data and the second compressed data to an application processing chip for processing. The method and the device can improve the flexibility of image compression of the electronic equipment.

Description

Image processing method, image processing chip, application processing chip and electronic device

technical field

The present application belongs to the technical field of image processing, and in particular, relates to an image processing method, an image processing chip, an application processing chip and an electronic device.

Background technique

With the development of technology, users often use electronic devices to capture images, such as taking photos or recording videos. However, in the related art, the electronic device only uses a single compression method to compress the captured image and transmits it to the application processing chip for image processing, so the flexibility of the image processing method of the electronic device is poor.

SUMMARY OF THE INVENTION

The embodiments of the present application provide an image processing method, an image processing chip, an application processing chip and an electronic device, which can improve the flexibility of the image processing method of the electronic device.

In a first aspect, an embodiment of the present application provides an image processing method, which is applied to an image processing chip, wherein the image processing chip is connected to an application processing chip, and the method includes:

Get current and historical images captured;

The current image is divided into at least a first area image and a second area image, and the variation range of the first area image relative to the corresponding area image in the historical image is greater than that of the second area image relative to the corresponding area image in the historical image. The variation range of the regional image;

The first region image is compressed according to a first compression algorithm to obtain first compressed data, and the second region image is compressed according to a second compression algorithm to obtain second compressed data, wherein the compression ratio of the first compression algorithm is smaller than the the compression ratio of the second compression algorithm;

The first compressed data and the second compressed data are transmitted to the application processing chip for processing.

In a second aspect, an embodiment of the present application provides an image processing method, which is applied to an application processing chip, where the application processing chip is connected to the image processing chip, and the method includes:

Receive the first compressed data and the second compressed data sent by the image processing chip, the first compressed data is compressed by the first compression algorithm, the second compressed data is compressed by the second compression algorithm, the first compression algorithm The compression ratio is less than the compression ratio of the second compression algorithm;

Decompressing the first compressed data to obtain a first sub-image, and decompressing the second compressed data to obtain a second sub-image;

Fusion processing is performed on the second sub-image based on the historical image to obtain a third sub-image;

A corresponding image is obtained by synthesizing the first sub-image and the third sub-image.

In a third aspect, an embodiment of the present application provides an image processing chip, wherein the image processing chip is connected to an application processing chip, the image processing chip includes a neural network processor and a first central processing unit, and the neural network processor connected with the first central processing unit through a system bus;

The neural network processor obtains the current image and historical image captured;

The neural network processor divides the current image into at least a first area image and a second area image, and the variation range of the first area image relative to the corresponding area image in the historical image is greater than that of the second area image. the variation range of the corresponding area image in the historical image;

The first central processing unit compresses the first area image according to the first compression algorithm to obtain first compressed data, and compresses the second area image according to the second compression algorithm to obtain second compressed data, the first compression The compression ratio of the algorithm is less than the compression ratio of the second compression algorithm;

The image processing chip transmits the first compressed data and the second compressed data to the application processing chip for processing.

In a fourth aspect, an embodiment of the present application provides an application processing chip, the application processing chip is connected to an image processing chip, and the application processing chip includes a second central processing unit;

The application processing chip receives the first compressed data and the second compressed data sent by the image processing chip, the first compressed data is compressed by the first compression algorithm, and the second compressed data is compressed by the second compression algorithm, so The compression ratio of the first compression algorithm is less than the compression ratio of the second compression algorithm;

The second central processing unit decompresses the first compressed data to obtain a first sub-image, and decompresses the second compressed data to obtain a second sub-image;

The second central processing unit performs fusion processing on the second sub-image based on the historical image to obtain a third sub-image;

The second central processing unit synthesizes the first sub-image and the third sub-image to obtain a corresponding image.

In a fifth aspect, an embodiment of the present application provides an electronic device, which includes the image processing chip described in any embodiment of the present application and the application processing chip described in any embodiment of the present application, the image processing chip and the The application handles the chip connection.

In the embodiment of the present application, the image processing chip compares historical images, divides the current image into at least a first area image and a second area image with different amplitudes of change, and uses different compression algorithms to compress the amplitudes of changes in the images in different areas , a first compression algorithm with relatively small compression is used for the first area image with a large variation range, and a second compression algorithm with relatively large compression is used for the second area image with a small variation range. It can be understood that, for the first area image with a large variation range, the first compression algorithm with relatively small compression is used for compression, which can reduce the compression loss of the first area image. Compressing with the second compression algorithm can reduce the amount of image data transmission, so that the application processing chip can process the image in time, thereby improving the flexibility of the electronic device for image compression.

Description of drawings

The technical solutions of the present application and the beneficial effects thereof will be apparent through the detailed description of the specific embodiments of the present application in conjunction with the accompanying drawings.

FIG. 1 is a first schematic flowchart of an image processing method provided by an embodiment of the present application.

FIG. 2 is a schematic flowchart of a second type of image processing method provided by an embodiment of the present application.

FIG. 3 is a third schematic flowchart of the image processing method provided by the embodiment of the present application.

FIG. 4 is a fourth schematic flowchart of the image processing method provided by the embodiment of the present application.

FIG. 5 is a fifth schematic flowchart of the image processing method provided by the embodiment of the present application.

FIG. 6 is a schematic diagram of a scene of an image processing method provided by an embodiment of the present application.

FIG. 7 is a schematic diagram of another scenario of the image processing method provided by the embodiment of the present application.

FIG. 8 is a schematic structural diagram of an image processing chip and an application processing chip provided by an embodiment of the present application.

FIG. 9 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

Please refer to the drawings, wherein the same component symbols represent the same components, and the principles of the present application are exemplified by being implemented in a suitable computing environment. The following description is based on illustrated specific embodiments of the present application and should not be construed as limiting other specific embodiments of the present application not detailed herein.

It can be understood that, the executive body of the embodiment of the present application may be an electronic device such as a smart phone or a tablet computer.

Please refer to FIG. 1 . FIG. 1 is a first schematic flowchart of an image processing method provided by an embodiment of the present application. The image processing method can be applied to an image processing chip, and the image processing chip and the application processing chip are connected, and the flow of the image processing method can include:

In 101, the captured current image and historical image are acquired.

With the development of technology, users often use electronic devices to capture images, such as taking photos or recording videos. However, in the related art, after the image processing chip acquires the image data collected by the camera, the image processing chip transmits the whole frame of image data to the application processing chip through the MIPI interface for related image processing, and the transmitted image data can be RAW format, YUV format or other formats. The characteristic of the currently applied transmission technology is that the entire frame of image data is directly transmitted, which is easy to cause the problem of excessive transmission pressure. Even if the image data is compressed, the same compression algorithm is used to compress the image data according to the same standard, which is easy to cause image loss. irreparable problems. That is, the electronic device only uses a single compression method to compress the captured image, and then transmits it to the application processing chip for image processing, so the flexibility of the electronic device image processing method is poor.

In the embodiment of the present application, after the camera obtains the image, the image can be transmitted to the image processing chip through the interface, so the image processing chip can obtain the current image and historical image captured, wherein the current image can be the image of the current frame, and the historical image can be It is the image of the previous frame or the image of the adjacent previous frames, such as the images of the first 3 frames, the first 5 frames, etc. It should be noted that the historical image may be one frame of image or multiple frames of images.

It can be understood that, with respect to the image of the previous frame or the images of the adjacent previous frames, only a part of the current image may have undergone major changes, and other parts have undergone minor changes or no changes.

In 102, the current image is divided into at least a first area image and a second area image, and the change range of the first area image relative to the area image corresponding to the historical image is greater than the change range of the second area image relative to the area image corresponding to the historical image.

In this embodiment of the present application, the image processing chip compares the current image with the historical image, and divides the current image into at least a first area image and a second area image. Wherein, the change range of the first area image relative to the area image corresponding to the historical image is greater than the change range of the second area image relative to the area image corresponding to the historical image.

In one embodiment, the image processing chip compares the current image with the historical image, sets the image of the area with a change amplitude greater than a preset amplitude threshold as the first area image, and sets the image of the area with a change amplitude smaller than the preset amplitude threshold as the image of the first area Second area image. The specific preset amplitude threshold can be set as required, for example, the preset amplitude threshold can be set according to the size of the current image, the transmission pressure of the transmission module, etc., which is not limited in this application.

It should be noted that, the present application may divide the variation range of different regions in the current image relative to the historical image into more regions, such as the third region image, the fourth region image, and so on.

In one embodiment, the image processing chip may pre-divide the current image into a plurality of sub-area images of n columns and m rows, compare each sub-area image with the images of the same area in the historical image, and compare the variation range greater than the preset The at least one sub-area image of the amplitude threshold is set as the first area image, and the at least one sub-area image whose variation amplitude is smaller than the preset amplitude threshold is set as the second area image.

It should be noted that, the more sub-regions the image processing chip pre-divides the current image, and the smaller the area of each sub-region, the first region image and the second region image are divided by judging the change range of each sub-region. The results of the division will be more accurate. Therefore, in this embodiment of the present application, a different number of sub-regions can be divided as needed, that is, the values of n and m can be set as needed, and the current image can be pre-divided into multiple sub-regions of uniform size, and can also be divided into multiple sub-regions. For example, the main area and the background area can be identified for the current image, and more sub-areas can be set for the main area with a higher rate of change. A low background area can be set with fewer sub-areas, and it can also be understood that each sub-area set for the background area has a larger area.

In one embodiment, the image processing chip can divide the current image into multiple sub-region images; obtain image feature points of the current image; and obtain the changes of the multiple sub-region images based on the location information of the image feature points in the current image and historical images Amplitude; set at least one sub-area image whose variation range is greater than the preset amplitude threshold as the first area image, and set at least one sub-area image whose variation range is smaller than the preset amplitude threshold as the second area image.

In one embodiment, after the camera acquires the current image, it transmits the current image to the image processing chip, and the image processing chip can identify the image feature points of the current image, which can also be understood as image key points in the current image.

In one embodiment, it can be obtained through autonomous learning of a neural network, for example, CNN feature extraction is used to identify image feature points, and key points of the image are identified, such as the head, shoulder, arm, etc. of the person in the image. Positions of palms, legs, soles of feet, etc., more specifically, such as key points such as the eyes, nose, mouth, ears, eyebrows, etc. of the face in the head. For another example, the outline of an object in an image can be identified, and the distribution of image feature points can be selected according to the outline, such as the upper endpoint, lower endpoint, left endpoint, right endpoint, and middle endpoint as image feature points.

Obtain the coordinate position of the image feature point, for example, the coordinates of the image feature point are (A, B), and then obtain the corresponding coordinate position of the image feature point in the historical image, such as (A', B'), determine the current image and Whether the variation range of the coordinate values of the image feature points contained in each sub-region in the historical image is greater than the preset range threshold, if so, set the sub-region as the first region image, if not, set the sub-region as the second region image area image.

In one embodiment, the image processing chip may also preset the reference point of each sub-region, for example, the middle position of each sub-region is the reference point, and obtain the coordinate position of each reference point, for example, the coordinates of the reference point are (C, D), and then obtain the corresponding coordinate position of the reference point in the historical image, such as (C', D'), to determine whether the coordinate value change range of the reference point of each sub-region in the current image and the historical image is greater than the preset range Threshold, if yes, set it as the first area image, if not, set it as the second area image.

In step 103, the first area image is compressed according to the first compression algorithm to obtain the first compressed data, the second area image is compressed according to the second compression algorithm to obtain the second compressed data, and the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm compression ratio.

In the embodiment of the present application, the image of the first area in the current image has a relatively large change range relative to the historical image. Therefore, for the image of the first area, the image processing chip adopts a first compression algorithm with a smaller compression ratio, and the obtained first compressed data is The image loss is small, so that the image of the first area can be better preserved; wherein, the first compression algorithm can be a lossless compression method, if the lossless compression method is adopted, after decompression, the first compressed data can be completely restored to the original first. A region image does not cause any distortion, lossless compression methods such as: Shannon-Fano coding, Huffman coding, Run-length coding, LZW (Lempel-Ziv-Welch) coding and arithmetic coding.

The second area image in the current image has a small or no change relative to the historical image, such as part of the background area in the current image. Therefore, the image processing chip uses a second compression algorithm with relatively large compression for the second area image. It is understood as lossy compression. Obtaining the second compressed data allows certain image information to be lost during the compression process according to the specific compression ratio. After decompression, the original second region image cannot be completely restored, but the capacity occupied by the image can be reduced, thereby It can reduce the transmission amount of image data and reduce the pressure of image transmission.

In 104, the first compressed data and the second compressed data are transmitted to the application processing chip for processing.

The image processing chip transmits the first compressed data and the second compressed data to the application processing chip for processing.

In the embodiment of the present application, comparing historical images, the image processing chip divides the current image into at least a first area image and a second area image with different amplitudes of change, and uses different compression algorithms for compression according to the amplitudes of changes in the images in different areas , a first compression algorithm with relatively small compression is used for the first area image with a large variation range, and a second compression algorithm with relatively large compression is used for the second area image with a small variation range. It can be understood that, for the first area image with a large variation range, the first compression algorithm with relatively small compression is used for compression, which can reduce the compression loss of the first area image. Compressing with the second compression algorithm can reduce the amount of image data transmission, so that the application processing chip can process the image in time, thereby improving the flexibility of the electronic device for image compression.

Please refer to FIG. 2. FIG. 2 is a second schematic flowchart of an image processing method provided by an embodiment of the present application. The process may include:

In 201, the captured current image and historical image are acquired.

In the embodiment of the present application, after the camera acquires the image, it can transmit the image to the image processing chip. Therefore, the image processing chip can acquire the current image and historical image captured, wherein the current image can be the image of the current frame, and the historical image can be The image of the previous frame or the images of the adjacent previous frames, such as the images of the first 3 frames, the first 5 frames, and so on. It should be noted that the historical image may be one frame of image or multiple frames of images.

It can be understood that, for the image of the previous frame or the image of the adjacent previous frames, only a part of the current image may have a large change, and other parts only have a small change or no change.

In 202, the current image is divided into a first area image, a second area image and a third area image, and the variation range of the first area image relative to the corresponding area image in the historical image is greater than that of the second area image relative to the corresponding area image in the historical image The change range of the second area image relative to the corresponding area image in the historical image is greater than the change range of the third area image relative to the corresponding area image in the historical image.

In this embodiment of the present application, the image processing chip compares the current image with the historical image, and divides the current image into a first area image, a second area image, and a third area image. Wherein, the variation range of the image of the first area relative to the image of the corresponding area of the historical image is greater than that of the image of the second area relative to the image of the corresponding area of the historical image, and the variation range of the image of the second area relative to the image of the corresponding area in the historical image is greater than that of the image of the third area. The variation range of the area image corresponding to the historical image.

In one embodiment, the current image is compared with the historical image, the image of the area with the change amplitude greater than the first preset amplitude threshold is set as the first area image, and the change amplitude is not greater than the first preset amplitude threshold but greater than the first area image. The area image with the two preset amplitude thresholds is set as the second area image, and the area image whose variation amplitude is not greater than the second preset amplitude threshold is set as the third area image. That is, comparing the image of the first area with the image of the second area, the change of the image of the third area relative to the historical image is small or basically unchanged.

It should be noted that the specific first preset amplitude threshold and the second preset amplitude threshold can be set as required, for example, the first preset amplitude threshold and the first preset amplitude threshold can be set according to the size of the current image, the transmission pressure of the transmission module, etc. Two preset amplitude thresholds, which are not limited in this application.

In one embodiment, the current image can be divided into a plurality of sub-region images; the image feature points of the current image are obtained; based on the position information of the image feature points in the current image and the historical image, the variation range of the multiple sub-region images is obtained; At least one sub-area image whose variation amplitude is greater than the first preset amplitude threshold is set as the first area image, and at least one sub-area image whose variation amplitude is not greater than the first preset amplitude threshold but greater than the second preset amplitude threshold is set as the first area image. For the second-region image, at least one sub-region whose variation amplitude is not greater than the second preset amplitude threshold is set as the third-region image.

It should be noted that, the image processing chip can divide multiple sub-regions of different numbers as required, and can divide the current image into multiple sub-regions of uniform size in advance, or can be divided into multiple sub-regions of different sizes.

It should be noted that the more sub-regions the image processing chip pre-divides the current image, and the smaller the area of each sub-region, then the first region image, the second region image and the first region image are divided by judging the change range of each sub-region. The three-region image, the result of its division will be more accurate. Therefore, in this embodiment of the present application, a different number of sub-regions can be divided according to requirements, and the current image can be pre-divided into a plurality of sub-regions of uniform size, and can also be divided into a plurality of sub-regions of different sizes. For example, the main area and the background area can be identified for the current image, and more sub-areas can be set for the main area with a higher rate of change. A low background area can be set with fewer sub-areas, and it can also be understood that each sub-area set for the background area has a larger area.

In one embodiment, after the camera acquires the current image, it transmits the current image to the image processing chip, and the image processing chip can identify the image feature points of the current image, which can also be understood as image key points of objects in the current image. In one embodiment, it can be obtained through autonomous learning of a neural network, for example, CNN feature extraction is used to identify image feature points, and key points of the image are identified, such as the head, shoulder, arm, etc. of the person in the image. Positions of palms, legs, soles of feet, etc., more specifically, such as key points such as the eyes, nose, mouth, ears, eyebrows, etc. of the face in the head. For another example, the outline of an object in an image can be identified, and the distribution of image feature points can be selected according to the outline, such as the upper endpoint, lower endpoint, left endpoint, right endpoint, and middle endpoint as image feature points.

Obtain the coordinate position of the image feature point, for example, the coordinates of the image feature point are (A, B), and then obtain the corresponding coordinate position of the image feature point in the historical image, such as (A', B'), determine the current image and The change range of the coordinate value of the image feature points contained in each sub-region in the historical image, if the change range is greater than the first preset range threshold, the sub-region is set as the first region image, if the change range is not greater than the first preset range If the amplitude threshold is greater than the second preset amplitude threshold, the sub-area is set as the second area image, and if the change amplitude is not greater than the second preset amplitude threshold, the sub-area is set as the third area image.

In one embodiment, the reference point of each sub-area can also be preset, for example, the middle position of each sub-area is the reference point, and the coordinate position of each reference point is obtained, for example, the coordinates of the reference point are (C, D), Then obtain the corresponding coordinate position of the reference point in the historical image, such as (C', D'), and judge the change range of the coordinate value of the reference point of each sub-region in the current image and the historical image. If the amplitude threshold is set, the sub-area is set as the first area image. If the change amplitude is not greater than the first preset amplitude threshold but greater than the second preset amplitude threshold, the sub-area is set as the second area image. If the amplitude is not greater than the second preset amplitude threshold, the sub-area is set as the third area image.

In 203, the first area image is compressed according to the first compression algorithm to obtain the first compressed data, the second area image is compressed according to the second compression algorithm to obtain the second compressed data, and the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm compression ratio.

In the embodiment of the present application, the image of the first area in the current image has a relatively large change range relative to the historical image. Therefore, the image processing chip adopts the first compression algorithm with relatively low compression for the image of the first area, and obtains an image of the first compressed data. The loss is small, so that the first area image can be better preserved; wherein, the first compression algorithm can be a lossless compression method, if the lossless compression method is adopted, after decompression, the first compressed data can be completely restored to the original Regional images without causing any distortion, lossless compression methods such as: Shannon-Fano coding, Huffman coding, Run-length coding, LZW (Lempel-Ziv-Welch) coding and arithmetic coding.

Although the change range of the second area image in the current image relative to the historical image is smaller than that of the first area, it still has a certain change range. Therefore, it is understandable that the image processing chip adopts the second compression algorithm with relatively large compression for the second area image. For lossy compression, obtaining the second compressed data allows certain image information to be lost during the compression process according to the specific compression ratio. After decompression, the original second region image cannot be completely restored, but the capacity occupied by the image can be reduced, so that Reduce the transfer amount of image data.

In 204, position information of the third region image is obtained.

Since the image of the third area has little or no change relative to the image of the corresponding area of the historical image, the image processing chip can directly obtain the location information of the image of the third area, and the application processing chip can obtain the location information of the historical image according to the location information. The corresponding area image does not need to be compressed and transmitted on the third area image.

In 205, the first compressed data, the second compressed data and the location information are transmitted to the application processing chip for processing.

The image processing chip transmits the first compressed data and the second compressed data to the application processing chip for processing. It should be noted that the application processing chip may have cached historical images. Second, the compressed data and historical data are used for further processing of the image.

Please refer to FIG. 3. FIG. 3 is a third schematic flowchart of an image processing method provided by an embodiment of the present application. The image processing method is applied to an application processing chip, and the application processing chip and the image processing chip are connected. The flow may include:

In 301, first compressed data and second compressed data sent by the image processing chip are received, the first compressed data is compressed by a first compression algorithm, the second compressed data is compressed by a second compression algorithm, and the compression ratio of the first compression algorithm is less than The compression ratio of the second compression algorithm.

The application processing chip receives the first compressed data and the second compressed data sent by the image processing chip. The first compressed data is compressed by a first compression algorithm, the second compressed data is compressed by a second compression algorithm, the compression ratio of the first compression algorithm is smaller than the compression ratio of the second compression algorithm, and the first compressed data corresponds to the current image captured by the camera The first area image of the first area image, the second compressed data corresponds to the second area image of the current image, and the change range of the first area image relative to the corresponding area image in the historical image is greater than the change range of the second area image relative to the corresponding area image in the historical image, The historical image is the previous frame or previous frames of the current image.

It can be understood that the image processing chip may be the image processing chip in any one of the above embodiments, and the first compressed data and the second compressed data may be the first compressed data and the second compressed data in any one of the above embodiments. This will not be repeated here.

In 302, decompress the first compressed data to obtain a first sub-image, and decompress the second compressed data to obtain a second sub-image.

The application processing chip decompresses the first compressed data to obtain a first sub-image, and decompresses the second compressed data to obtain a second sub-image. The first sub-image is a sub-image corresponding to the first area image, and the second sub-image is a sub-image corresponding to the second area image.

It can be understood that, the first sub-image and the second sub-image may be the first sub-image and the second sub-image in the foregoing embodiment, and details are not described herein again.

In 303, a third sub-image is obtained by performing fusion processing on the second sub-image based on the historical image.

The application processing chip performs fusion processing on the second sub-image based on the historical image to obtain a third sub-image, and the historical image is an image previously sent by the image processing chip stored in the application processing chip. The historical image and the second sub-image are associated, for example, the historical image may be an image of one frame or several frames before the second sub-image.

In 304, the first sub-image and the third sub-image are synthesized to obtain a corresponding image.

The application processing chip synthesizes the first sub-image and the third sub-image to obtain a corresponding image, and the corresponding image may be a processed image of the current image captured by the camera. For example, the first sub-image is an uncompressed sub-image of the first area image of the current image, the second sub-image is a sub-image of the second area of the current image after lossy compression, and the third sub-image is the second sub-image After the sub-image is fused with the historical image, the obtained third sub-image has less image loss, is closer to the image in the second area, and has higher image quality, while improving the transmission efficiency of the image processing chip and the application processing chip.

Please refer to FIG. 4. FIG. 4 is a fourth schematic flowchart of an image processing method provided by an embodiment of the present application. The image processing method is applied to an application processing chip, and the application processing chip and the image processing chip are connected. The flow may include:

In 401, the first compressed data, the second compressed data and the location information sent by the image processing chip are received.

The application processing chip receives the first compressed data, the second compressed data and the location information sent by the image processing chip. The first compressed data is compressed by a first compression algorithm, the second compressed data is compressed by a second compression algorithm, the compression ratio of the first compression algorithm is smaller than the compression ratio of the second compression algorithm, and the first compressed data corresponds to the current image captured by the camera The first area image of the current image, the second compressed data corresponds to the second area image of the current image, the location information is the third area image corresponding to the current image, and the change range of the first area image relative to the corresponding area image in the historical image is greater than that of the second area image. The change range of the area image relative to the corresponding area image in the historical image, the change range of the second area image relative to the corresponding area image in the historical image is greater than the change range of the third area image relative to the corresponding area image in the historical image, and the historical image is the previous image of the current image. frame or previous frames of images.

It can be understood that the image processing chip may be the image processing chip in any of the foregoing embodiments, and the first compressed data, the second compressed data, and the location information may be the first compressed data and the second compressed data in any of the foregoing embodiments. data, which will not be repeated here.

In 402, decompress the first compressed data to obtain a first sub-image corresponding to the first area image, and decompress the second compressed data to obtain a second sub-image corresponding to the second area image.

After the application processing chip receives the first compressed data and the second compressed data, the application processing chip decompresses the first compressed data to obtain a first sub-image corresponding to the first area image, and decompresses the second compressed data to obtain a corresponding first sub-image. The second sub-image of the two-region image.

Since the first region image adopts the first compression algorithm with relatively low compression, the image loss of the decompressed first sub-image is small. In this embodiment of the present application, a lossless compression method may also be used for the first region image, so that The decompressed first sub-image can completely restore the original first-area image without causing any distortion, that is, the first sub-image obtained by applying the processing chip to decompress the first compressed data may be the first-area image.

Since the second area image adopts the second compression algorithm with relatively large compression, it can also be understood as lossy compression. Therefore, the second area image cannot be completely restored to the original second area image after decompression, and some image information will be lost. That is, the second sub-image has a certain image loss relative to the second region image according to a specific compression ratio.

In 403, a third sub-image is obtained by performing fusion processing on the second sub-image based on the historical image.

It can be understood that the application processing chip may have cached historical images, such as images of the previous frame or images of adjacent previous frames. The variation range of the image of the second area relative to the image of the corresponding area in the historical image is smaller than that of the image of the first area, but still has a certain variation range. After the application processing chip receives the second compressed data, the second sub-image obtained by decompressing the second compressed data has image loss relative to the second region image in the current image.

In the embodiment of the present application, the application processing chip may acquire the image of the area corresponding to the second area image in the current image in the historical image, and fuse the image of the corresponding area with the second sub-image to obtain the third sub-image.

In one embodiment, according to the change process of the historical image, such as several adjacent frames of images, the corresponding area image corresponding to the historical image may be adjusted according to the change process, so that the corresponding area image in the adjusted historical image is the same as the first image. The image changes in the second area are smaller and closer; and the image details lost in the second sub-image after decompression are corrected and enhanced by using the area image corresponding to the adjusted historical image to obtain a third sub-image.

Therefore, even if the second sub-image obtained by decompression has a certain image loss relative to the second region image in the current image, the third sub-image obtained after the second sub-image is enhanced and fused with the historical image is closer to the current image. For the image of the second area in the current image, the image loss is also less, which improves the image quality of the third sub-image, and even has higher image quality than the second area image, for example, it is clearer.

In 404, a corresponding fourth sub-image is obtained based on the historical image and the location information.

Since the third area image has little or no change relative to the historical image, the application processing chip can determine the area image corresponding to the third area image in the historical image according to the position information of the third area image, and assign this area to the image. The image is used as the fourth sub-image corresponding to the third area image.

In 405, the first sub-image, the third sub-image and the fourth sub-image are synthesized to obtain a corresponding image.

Since the first area image adopts the first compression algorithm with less compression, the first sub-image obtained after decompression by the processing chip has less loss than the first area image. If the first area image adopts the lossless compression method, the solution The compressed first sub-image can completely restore the original first-area image without causing any distortion, that is, the first sub-image decompressed by the processing chip can be the first-area image.

The second region image adopts the first compression algorithm with relatively large compression, and the second sub-image obtained after decompression by the processing chip has image loss relative to the second region image. However, in this embodiment of the present application, by combining the second sub-image with After the historical image is enhanced and fused, the image quality of the third sub-image is improved, and the obtained third sub-image is closer to the image of the second area, and the image loss is also smaller.

Compared with the historical image, the variation range of the third area image is very small or even no change. Therefore, in this embodiment of the present application, the area image with the same location information as the third area image in the historical image is used as the fourth sub-image corresponding to the third area image. , it can be understood that the image loss of the fourth sub-image relative to the third region image is also small or no loss.

In this embodiment of the present application, since the image loss of the first sub-image relative to the image of the first area of the current image is small or no loss, the image loss of the third sub-image relative to the image of the second area of the current image is also small, and the image loss of the fourth sub-image is also small. The image loss of the image relative to the image in the third area is also small or no loss, so the application processing chip combines the first sub-image, the third sub-image and the fourth sub-image to obtain an image corresponding to the current image, and the image loss is also relatively low. Small, the application processing chip can perform further image processing based on the image.

It is understandable that if only some of the current image and the historical image have undergone major changes, some have minor changes, and the other part has undergone slight changes or no changes, it is only necessary to use a smaller compression ratio for some of the first region images. To reduce the image loss of the image in the first area, you can compress the image in the second area using a compression method with relatively large compression, thereby reducing the amount of image data transmission. For the image in the third area, no compression transmission is required. , it only needs to send the position information of the image of the third area to the application processing chip, which further reduces the amount of image data transmission and greatly reduces the transmission pressure, so that the application processing chip can receive the image data in real time for processing, and can The image processed by the processing chip has less image loss relative to the current image.

Please refer to FIG. 5. FIG. 5 is a fifth schematic flowchart of an image processing method provided by an embodiment of the present application. The image processing method can be applied to an image processing chip and an application processing chip, and the image processing chip and the application processing chip are connected. Processes can include:

In 501, the captured current image and historical image are acquired.

In the embodiment of the present application, after the camera obtains the image, it can transmit the current image to the image processing chip, so the image processing chip can obtain the current image and historical image captured, wherein the current image can be the image of the current frame, and the historical image can be The image of the previous frame or the images of the adjacent previous frames, such as the images of the first 3 frames, the first 5 frames, and so on. It can be understood that, for the image of the previous frame or the image of the adjacent previous frames, only a part of the current image may have a large change, and other parts only have a small change or no change.

In 502, the current image is divided into at least a first area image and a second area image, and the change range of the first area image relative to the corresponding area image in the historical image is greater than the change range of the second area image relative to the corresponding area image in the historical image.

In this embodiment of the present application, the image processing chip compares the current image with the historical image, and divides the current image into at least a first area image and a second area image. Wherein, the change range of the first area image relative to the area image corresponding to the historical image is greater than the change range of the second area image relative to the area image corresponding to the historical image.

In one embodiment, the image processing chip compares the current image with the historical image, sets the image of the area with a change amplitude greater than a preset amplitude threshold as the first area image, and sets the image of the area with a change amplitude not greater than the preset amplitude threshold as the first area image for the second region image. The specific preset amplitude threshold can be set as required, for example, the preset amplitude threshold can be set according to the size of the current image, the transmission pressure of the transmission module, etc., which is not limited in this application.

In step 503, the first area image is compressed according to the first compression algorithm to obtain the first compressed data, the second area image is compressed according to the second compression algorithm to obtain the second compressed data, and the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm compression ratio.

In the embodiment of the present application, the image of the first area in the current image has a relatively large change range relative to the historical image. Therefore, the image processing chip adopts the first compression algorithm with relatively low compression for the image of the first area, and obtains an image of the first compressed data. The loss is small, so that the first area image can be better preserved; wherein, the first compression algorithm can be a lossless compression method, if the lossless compression method is adopted, after decompression, the first compressed data can be completely restored to the original area image without causing any distortion.

The variation of the second area image in the current image relative to the historical image is small or unchanged. Therefore, the image processing chip adopts the second compression algorithm with relatively large compression for the second area image, which can also be understood as lossy compression, and obtains the second compression algorithm. The compressed data allows a certain amount of image information to be lost during the compression process according to the specific compression ratio. After decompression, the original second region image cannot be completely restored, but it can reduce the amount of image data transmission and reduce the pressure of image transmission.

In 504, the first compressed data and the second compressed data are transmitted to the application processing chip for processing.

The image processing chip transmits the first compressed data and the second compressed data to the application processing chip for processing. It should be noted that the application processing chip may have cached historical images. Second, the compressed data and historical data are used for further processing of the image.

In 505, decompress the first compressed data to obtain a first sub-image corresponding to the first area image, and decompress the second compressed data to obtain a second sub-image corresponding to the second area image.

After the application processing chip receives the first compressed data and the second compressed data sent by the image processing chip, the application processing chip decompresses the first compressed data to obtain a first sub-image corresponding to the image of the first area, and performs the second compression on the second compressed data. Decompression obtains a second sub-image corresponding to the second region image.

Since the first region image adopts the first compression algorithm with relatively low compression, the image loss of the decompressed first sub-image is small. In this embodiment of the present application, a lossless compression method may also be used for the first region image, so that The decompressed first sub-image can completely restore the original first-area image without causing any distortion, that is, the first sub-image obtained by applying the processing chip to decompress the first compressed data may be the first-area image.

Since the second area image adopts the second compression algorithm with relatively large compression, it can also be understood as lossy compression. Therefore, the second area image cannot be completely restored to the original second area image after decompression, and some image information will be lost. That is, the second sub-image has a certain image loss relative to the second region image according to a specific compression ratio.

In 506, a third sub-image is obtained by fusing the second sub-image based on the historical image.

After the application processing chip receives the second compressed data, the second sub-image obtained by decompressing the second compressed data has image loss relative to the second region image in the current image.

In the embodiment of the present application, the application processing chip may acquire the image of the area corresponding to the second area image in the current image in the historical image, and fuse the image of the corresponding area with the second sub-image to obtain the third sub-image. The historical image can be the image of the previous frame or the image of the adjacent previous frames.

The variation of the image of the second area relative to the image of the corresponding area in the historical image is small or does not change. It can be understood that the application processing chip may already have cached historical images. When the application processing chip performs fusion and enhancement processing on the second sub-image, the cached historical images can be directly used without the need for the pre-image processing chip to store the required historical images. The image is transferred again.

It should be noted that, since the image of the second area has a small or no change relative to the image of the corresponding area in the historical image, such as the background area in the current image, the historical image can be the image of the previous few frames, and does not need to be It is updated in real time, and it can be updated every certain number of frames to meet the needs of the application processing chip. For example, historical data is cached every 3 or 5 frames.

In one embodiment, the image details lost in the decompressed second sub-image relative to the second region image can be identified first, and then the lost image details can be corrected and enhanced through the region image corresponding to the historical image to make up for the lossy compression. A third sub-image is obtained due to the resulting image loss. For example, the image details lost in the second sub-image can be compensated by using the corresponding area image in the historical image.

In one embodiment, according to the change process of the historical image, such as several adjacent frames of images, the corresponding area image corresponding to the historical image may be adjusted according to the change process, so that the corresponding area image in the adjusted historical image is the same as the first image. The image changes in the second area are smaller and closer; and the image details lost in the second sub-image after decompression are corrected and enhanced by using the area image corresponding to the adjusted historical image to obtain a third sub-image.

Therefore, even if the second sub-image obtained by decompression has a certain image loss relative to the second region image in the current image, the third sub-image obtained after the second sub-image is enhanced and fused with the historical image is closer to the current image. For the image of the second area in the current image, the image loss is also less, and the image quality of the third sub-image is improved, and even has higher image quality than the image of the second area, for example, the image is clearer.

In one embodiment, the historical image may be multiple frames, the multiple frames of the historical image may be fused with the second sub-image, and the number of frames of the historical image may be controlled as required, so that the obtained third sub-image can obtain more good image quality.

In one embodiment, the third sub-image can also be obtained by partially merging the images in the second area based on the historical images. For example, the background area and the main area of the current image can be identified, and only the images of the second area involve the main body. Fusion enhancement processing is performed on the part of the area, while fusion enhancement processing is not performed on the part of the image in the second area where the background area is designed, so as to reduce the image range of the application processing chip fusion processing and improve the image processing efficiency of the application processing chip.

In one embodiment, even for the first area image with a large variation range, the first area image may be further segmented and processed separately, for example, it is determined that the head area of the person in the image is a high change area, It can be further determined that the eye part image and the nose part image of the face in the head area have less changes. Therefore, after decompressing the first area image to obtain the first sub-image, the changes in the first sub-image can also be calculated. The smaller image part is fused and enhanced using historical images to further improve the quality of the first sub-image.

In 507, the first sub-image and the third sub-image are synthesized to obtain an image corresponding to the current image.

Since the first area image adopts the first compression algorithm with less compression, the first sub-image obtained after decompression by the processing chip has less loss than the first area image. If the first area image adopts the lossless compression method, the solution The compressed first sub-image can completely restore the original first-area image without causing any distortion, that is, the first sub-image decompressed by the processing chip can be the first-area image.

The second area image adopts the first compression algorithm with relatively large compression. The second sub-image obtained after decompression by the processing chip has image loss relative to the second area image, but because the area corresponding to the second area image and the historical image changes The amplitude is small or there is no change. Therefore, in the implementation of this application, after the second sub-image and the historical image are enhanced and fused, the third sub-image obtained has less image loss and is closer to the image of the second area. higher.

In this embodiment of the present application, since the image loss of the first sub-image relative to the image of the first area of the current image is small or no loss, and the image loss of the third sub-image relative to the image of the second area of the current image is also small, the application processing The chip fuses the first sub-image and the third sub-image to obtain an image corresponding to the current image, and its image loss is also small, and further image processing can be performed based on the image.

It is understandable that if only a small part of the current image and the historical image have undergone major changes, but most of the changes have not been sent, it is only necessary to compress a small part of the first area image with a compression method with a small compression ratio to reduce the number of The image loss of one area image, but most of the second area image can be compressed by a compression method with relatively large compression, so the transmission amount of image data can be greatly reduced, so that the application processing chip can receive the image data in time.

In the embodiment of the present application, the image processing chip compares the historical images and divides the current image into at least a first area image and a second area image with different change ranges, and uses different compression algorithms to compress the change ranges of the different area images. The first compression algorithm with relatively small compression is used for the first area image with a large change range, and the second compression algorithm with relatively large compression is used for the second area image with a small change range, which improves the flexibility of the electronic device for image compression.

In the embodiment of the present application, the first area image with a large variation range is compressed by a first compression algorithm with a small compression ratio, such as a lossless compression method, and the first sub-image obtained by applying the processing chip decompression has less compression than the first area image. There is no image loss or no image loss; for the second area image with a small change range, the second compression algorithm with relatively large compression can be used to reduce the transmission amount of image data, but the second sub-image obtained by applying the processing chip decompression is lower than the second sub-image. The region image has a certain image loss, so the application processing chip in this embodiment of the present application also performs fusion and enhancement processing on the second sub-image based on the historical image to obtain a third sub-image, and the third sub-image is closer to the image of the second region. The image corresponding to the current image is obtained by fusing the first sub-image and the third sub-image. The image processing chip not only reduces the amount of image data transmission through the differentiated compression method, so that the application processing chip can receive the image data in time, and can ensure that the obtained image has the same quality as the current image by performing fusion and enhancement processing on the second sub-image. Smaller image loss, higher image quality, or even improved image quality.

It should be noted that, in the differential algorithm processing of the image, the technical solutions in the embodiments of the present application may also be referred to, for example, the noise of the image is processed in sub-regions, and a relatively complex algorithm is used for the areas with many noises in the image. For noise reduction processing, another relatively simple algorithm can be used to perform noise reduction processing on areas with less noise in the image.

Please refer to FIG. 6 to FIG. 7 , FIG. 6 is a schematic diagram of a scene of an image processing method provided by an embodiment of the present application, and FIG. 7 is a schematic diagram of another scene of an image processing method provided by an embodiment of the present application.

The camera application of the electronic device is started and is in a shooting state, and the camera starts to capture images, and the format of the images may be RAW format or other formats.

For example, after the camera obtains the current image, it transmits the current image to the image processing chip. The neural network processor of the image processing chip can identify the image feature points of the current image, and can also be understood as the image key points of the objects in the current image. In one embodiment, it can be obtained through autonomous learning of a neural network, for example, CNN feature extraction is used to identify image feature points, and key points of the image are identified, such as the head, shoulder, arm, etc. of the person in the image. Positions of palms, legs, soles of feet, etc., more specifically, such as key points such as the eyes, nose, mouth, ears, eyebrows, etc. of the face in the head. For another example, the outline of an object in an image can be identified, and the distribution of image feature points can be selected according to the outline, such as the upper endpoint, lower endpoint, left endpoint, right endpoint, and middle endpoint as image feature points.

After the camera obtains the current image, it can transmit the current image to the image processing chip, and the neural network processor of the image processing chip compares the current image with the historical image, and divides the current image into at least the first area image and the second area image. Wherein, the change range of the first area image relative to the area image corresponding to the historical image is greater than the change range of the second area image relative to the area image corresponding to the historical image. Wherein, the historical image may be an image of the previous frame or an image of the adjacent previous frames, such as images of the previous 3 frames, the previous 5 frames, and the like. Historical images can be one frame of images or multiple frames of images.

For example, the neural network processor of the image processing chip can divide the current image into multiple sub-region images; based on the position information of image feature points in the current image and historical images, obtain the variation range of the multiple sub-region images; The at least one sub-area image of the amplitude threshold is set as the first area image, and the at least one sub-area image whose variation amplitude is smaller than the preset amplitude threshold is set as the second area image.

The neural network processor of the image processing chip obtains the coordinate positions of the image feature points, for example, the coordinates of the image feature points are (A, B), and then obtains the corresponding coordinate positions of the image feature points in the historical image, such as (A', B'), judging whether the variation range of the coordinate value of the image feature points contained in each sub-region in the current image and the historical image is greater than the preset range threshold, if so, then the sub-region is set as the first region image, if not, Then the sub-area is set as the second area image.

For example, please refer to Fig. 5, the neural network processor of the image processing chip can divide the current image into 16 sub-regions, determine whether the change range of the sub-region image is within the preset range threshold Areas 3, 6, 7, 8, 11, and 14 are set as the first area image, and the sub-areas 1, 2, 4, 5, 9, 10, 12, 13, 15, and 16 whose change amplitude is not greater than the preset amplitude threshold Set as the second area image.

The first central processing unit of the image processing chip uses a lossless compression method to compress the first area image to obtain first compressed data. After decompression, the first compressed data can completely restore the original first area image without causing any distortion.

The variation of the second area image in the current image relative to the historical image is small or unchanged. Therefore, the first central processing unit of the image processing chip adopts the second compression algorithm with relatively large compression for the second area image, which can also be understood as having Loss compression to obtain the second compressed data. According to the specific compression ratio, a certain amount of image information is allowed to be lost during the compression process. After decompression, the original second region image cannot be completely restored, but it can reduce the amount of image data transmission and reduce image transmission. pressure.

The image processing chip transmits the first compressed data and the second compressed data to the application processing chip. After the application processing chip receives the first compressed data and the second compressed data, the second central processing unit of the application processing chip can process the first compressed data. The first sub-image corresponding to the first area image is obtained by decompressing, and the second sub-image corresponding to the second area image is obtained by decompressing the second compressed data.

Since the first region image adopts the first compression algorithm with relatively low compression, the image loss of the decompressed first sub-image is small. If the lossless compression method is adopted for the first region image, the second central processing of the processing chip is applied. The first sub-image obtained by the device decompressing the first compressed data may be the first region image.

After the application processing chip receives the second compressed data, the second sub-image obtained by decompressing the second compressed data by the second central processing unit of the application processing chip has image loss relative to the second area image in the current image.

In the embodiment of the present application, the second central processing unit of the application processing chip may acquire the image of the area corresponding to the image of the second area in the current image in the historical image, and fuse the image of the corresponding area with the second sub-image to obtain the third sub-image. subimage. For example, you can first identify the image details lost in the decompressed second sub-image relative to the second area image, and then correct and enhance the lost image details through the area image corresponding to the historical image to make up for the image loss caused by lossy compression. Get the third subimage. The image details lost in the second sub-image can be compensated by using the corresponding area image in the historical image, so that the image quality of the third sub-image is improved, and the image loss of the third sub-image is smaller and closer to the second area image.

The second central processing unit of the application processing chip can fuse the first sub-image and the third sub-image to obtain an image corresponding to the current image, and perform further processing on the image, such as adjustment of white balance, contrast, and brightness.

In the present application, the transmission amount of image data is reduced by the differentiated compression method, so that the application processing chip can receive the image data in time, and the second central processing unit of the application processing chip can fuse and enhance the second sub-image according to the historical image. The processing ensures that the image processed by the application processing chip has less image loss than the current image, the image quality is higher, and even the image quality is enhanced.

Please refer to FIG. 8 . FIG. 8 is a schematic structural diagram of an image processing chip and an application processing chip provided by an embodiment of the present application. As shown in FIG. 8 , the image processing chip 400 provided in this embodiment of the present application may include a neural network processor 410 and a first central processing unit 430 , and the neural network processor 410 and the first central processing unit 430 are connected through a system bus 450 . The image processing chip 400 may further include an internal memory 470, and the internal memory 470 is also connected through the system bus 450. The neural network processor 410, the first central processing unit 430 and the like in the image processing chip 400 are used to process the images collected by the camera 500, The processed images are stored in the internal memory 470 through the system bus 450 .

Please continue to refer to FIG. 8 , the image processing chip 400 can be connected to the camera 500 through an interface and to the application processing chip 600 through other interfaces. For example, the camera 500 is connected to the system bus 450 through the third interface 420 to connect to the image processing chip 400, and the application processing chip 600 is connected to the image processing chip 400 through the third interface 420. The second interface 620 is connected to the first interface 440 of the image processing chip 400 , and the first interface 440 is connected to the system bus 450 .

Wherein, the image processing chip 400 can obtain the current image and historical image captured, for example, the image processing chip 400 stores the current image and historical image captured by the camera 500 to the internal memory 470 through the third interface 420 and the system bus 450, and the neural network processes The controller 410 may obtain current images and historical images from the internal memory 470 through the system bus 450 .

The neural network processor 410 can divide the current image into at least a first area image and a second area image, and the change range of the first area image relative to the corresponding area image in the historical image is greater than the change of the second area image relative to the corresponding area image in the historical image. magnitude. The neural network processor 410 may store the first area image and the second area image to the internal memory 470 or transmit to the first central processing unit 430 through the system bus 450 .

After the first central processing unit 430 obtains the first area image and the second area image through the system bus 450, it can compress the first area image according to the first compression algorithm to obtain the first compressed data, and according to the second compression algorithm, compress the second area image. The image is compressed to obtain second compressed data, and the compression ratio of the first compression algorithm is smaller than the compression ratio of the second compression algorithm.

The image processing chip 400 transmits the first compressed data and the second compressed data to the application processing chip 600 for processing. For example, the first 430 may store the first compressed data and the second compressed data in the internal memory 470 through the system bus 450, and then transmit them to the application processing chip 600 through the first interface 440, or the first central processing unit 430 may store the first compressed data The data and the second compressed data are transmitted to the application processing chip 600 through the first interface 440, and the application processing chip 600 can perform subsequent processing according to the first compressed data and the second compressed data.

In some other embodiments, the neural network processor 410 dividing the current image into at least the first area image and the second area image may include: the neural network processor 104 dividing the current image into the first area image, the second area image and the second area image. For the third area image, the change range of the first area image relative to the corresponding area image in the historical image is greater than the change range of the second area image relative to the corresponding area image in the historical image, and the change range of the second area image relative to the corresponding area image in the historical image is greater than The variation range of the third area image relative to the area image corresponding to the historical image;

Before transmitting the first compressed data and the second compressed data to the application processing chip 600, the neural network processor 410 may also acquire the position information of the third region image;

The transmitting the first compressed data and the second compressed data to the application processing chip 600 for processing may include: transmitting the first compressed data, the second compressed data and the location information to the application processing chip 600 for processing.

The image processing chip 400 may also have more functions. For details, please refer to the image processing chip in the above-mentioned embodiment, which will not be repeated here.

The present application further provides an application processing chip, please continue to refer to FIG. 8 , the application processing chip 600 is connected to the image processing chip 400 , and the application processing chip 600 includes a second central processing unit 640 and a second interface 620 . The application processing chip 600 may receive the first compressed data and the second compressed data sent by the image processing chip 400 . For example, the second central processing unit 640 obtains the first compressed data and the second compressed data sent by the image processing chip 400 through the second interface 620 . The first compressed data is compressed by the first compression algorithm, the second compressed data is compressed by the second compression algorithm, the compression ratio of the first compression algorithm is smaller than the compression ratio of the second compression algorithm, and the first compressed data and the second compressed data can be the above The first compressed data and the second compressed data in the embodiment will not be repeated here.

The second central processing unit 640 decompresses the first compressed data to obtain the first sub-image, and decompresses the second compressed data to obtain the second sub-image. The second central processing unit 640 may further decompress the second sub-image based on the historical image. The images are fused to obtain a third sub-image, and the first sub-image and the third sub-image are synthesized to obtain a corresponding image.

In some other embodiments, the application processing chip may also receive the position information sent by the image processing chip; before the second central processing unit of the application processing chip combines the first sub-image and the third sub-image to obtain a corresponding image, the second central processing The processor obtains the fourth sub-image based on the historical image and the position information; the second central processing unit can also synthesize the first sub-image, the third sub-image and the fourth sub-image to obtain a corresponding image.

For the structure and function of the application processing chip 600, reference may be made to the structure and function of the application processing chip in the above-mentioned embodiments, and details are not described herein again.

Embodiments of the present application further provide an electronic device, including a memory, an image processing chip, and an application processing chip. The image processing chip and the application processing chip are used to execute each image processing provided by the embodiments of the present application by calling a computer program stored in the memory. process in the method. The electronic device may be a mobile terminal such as a tablet computer or a smart phone, the image processing chip is connected to the application processing chip, the image processing chip may be the image processing chip in any of the above embodiments, and the application processing chip may be any of the above The application processing chip in the embodiment will not be repeated here.

Please refer to FIG. 9 , which is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device 800 may include components such as a memory 801 , an image processing chip 802 , and an application processing chip 803 . Those skilled in the art can understand that the structure of the electronic device shown in the figures does not constitute a limitation on the electronic device, and may include more or less components than those shown in the figures, or combine some components, or arrange different components.

Memory 801 may be used to store computer programs and data. The computer program stored in the memory 801 contains instructions executable in the processor. A computer program can be composed of various functional modules.

The image processing chip 802 executes various image processing functions and data processing by calling the computer program stored in the memory 801 .

The application processing chip 803 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device, so as to monitor the electronic device as a whole.

In this embodiment, the image processing chip 802 in the electronic device 800 loads the instructions corresponding to the processes of one or more computer programs into the memory 801 according to the following steps, and the image processing chip 802 executes the instructions stored in the memory 801. A computer program in the memory 801, thereby realizing various functions:

Get current and historical images captured;

Divide the current image into at least a first area image and a second area image, and the change range of the first area image relative to the corresponding area image in the historical image is greater than the change range of the second area image relative to the corresponding area image in the historical image;

Compressing the first area image according to the first compression algorithm to obtain the first compressed data, and compressing the second area image according to the second compression algorithm to obtain the second compressed data, the compression ratio of the first compression algorithm is smaller than the compression ratio of the second compression algorithm;

The first compressed data and the second compressed data are transmitted to the application processing chip 803 for processing.

In this embodiment, the application processing chip 803 in the electronic device 800 loads the instructions corresponding to the processes of one or more computer programs into the memory 801 according to the following steps, and is executed by the application processing chip 803 and stored in the memory 801 . A computer program in the memory 801, thereby realizing various functions:

Receive the first compressed data and the second compressed data sent by the image processing chip, the first compressed data is compressed by the first compression algorithm, the second compressed data is compressed by the second compression algorithm, and the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm compression ratio;

Decompressing the first compressed data to obtain a first sub-image, and decompressing the second compressed data to obtain a second sub-image;

Fusion processing is performed on the second sub-image based on the historical image to obtain a third sub-image;

A corresponding image is obtained by synthesizing the first sub-image and the third sub-image.

Please continue to refer to FIG. 9 , the electronic device 800 may further include components such as a camera 804 , a display screen 805 , a control circuit 806 , an input unit 807 , an audio circuit 808 , a sensor 809 , and a power supply 810 .

The camera 804 can be used for video recording and image acquisition. After the image is captured by the lens, the image is processed by the photosensitive component circuit and control component in the camera, and then the image is restored by software, and the image can be seen through the display screen. .

The display screen 805 may be used to display information input by or provided to the user and various graphical user interfaces of the electronic device, which may be composed of images, text, icons, video, and any combination thereof.

The control circuit 806 is electrically connected to the display screen 805 for controlling the display screen 805 to display information.

The input unit 807 can be used to receive input numbers, character information or user characteristic information (eg fingerprint), and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. The input unit 807 may include a fingerprint identification module.

The audio circuit 808 may provide an audio interface between the user and the electronic device through speakers and microphones. Among them, the audio circuit 808 includes a microphone. The microphone is electrically connected to the application processing chip 803 . The microphone is used to receive voice information input by the user.

The sensor 809 is used to collect external environment information. The sensor 809 may include one or more of an ambient brightness sensor, an acceleration sensor, a gyroscope, and the like.

The power supply 810 is used to power various components of the electronic device 800 . In some embodiments, the power supply 810 may be logically connected to the application processing chip 803 through a power management system, so as to implement functions such as managing charging, discharging, and power consumption through the power management system.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the detailed description of the image processing method above, and details are not repeated here.

The electronic device provided in the embodiment of the present application and the image processing method in the above embodiment belong to the same concept, and any method provided in the image processing method embodiment can be executed on the electronic device. For the specific implementation process, please refer to the implementation of the image processing method. For example, it will not be repeated here.

It should be noted that, for the image processing method of the embodiment of the present application, those of ordinary skill in the art can understand that all or part of the process of implementing the image processing method of the embodiment of the present application can be completed by controlling the relevant hardware through a computer program. The program may be stored in a computer-readable storage medium, such as a memory, and executed by at least one image processing chip and an application processing chip, and the execution process may include the flow of an embodiment of the image processing method. The storage medium may be a magnetic disk, an optical disk, a read only memory (ROM, Read Only Memory), a random access memory (RAM, Random Access Memory), and the like.

The image processing method, device, storage medium, and electronic device provided by the embodiments of the present application have been described in detail above, and the principles and implementations of the present application are described with specific examples. The descriptions of the above embodiments are only used for Help to understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation and application scope. In conclusion, the content of this description should not be understood to limit this application.

Claims (10)

1. An image processing method is applied to an image processing chip, and the image processing chip is connected with an application processing chip, and the method is characterized by comprising the following steps:

acquiring a shot current image and a shot historical image;

dividing the current image into at least a first area image and a second area image, wherein the change amplitude of the first area image relative to the corresponding area image in the historical image is larger than the change amplitude of the second area image relative to the corresponding area image in the historical image;

compressing the first area image according to a first compression algorithm to obtain first compressed data, and compressing the second area image according to a second compression algorithm to obtain second compressed data, wherein the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm;

and transmitting the first compressed data and the second compressed data to the application processing chip for processing.

2. The image processing method of claim 1, wherein the dividing the current image into at least a first region image and a second region image comprises:

dividing the current image into a plurality of subarea images;

acquiring image characteristic points of the current image;

acquiring the variation amplitude of a plurality of subarea images based on the position information of the image feature points in the current image and the historical image;

setting at least one sub-area image with the variation amplitude larger than a preset amplitude threshold value as the first area image, and setting at least one sub-area image with the variation amplitude not larger than the preset amplitude threshold value as the second area image.

3. The image processing method of claim 1, wherein the dividing the current image into at least a first region image and a second region image comprises:

dividing the current image into a first area image, a second area image and a third area image, wherein the change amplitude of the first area image relative to the corresponding area image in the historical image is larger than that of the second area image relative to the corresponding area image in the historical image, and the change amplitude of the second area image relative to the corresponding area image in the historical image is larger than that of the third area image relative to the corresponding area image in the historical image;

before the transmitting the first compressed data and the second compressed data to the application processing chip, the method further includes: acquiring position information of the third area image;

transmitting the first compressed data and the second compressed data to the application processing chip for processing, including: and transmitting the first compressed data, the second compressed data and the position information of the third area image to the application processing chip for processing.

4. An image processing method is applied to an application processing chip, and the application processing chip is connected with the image processing chip, and is characterized by comprising the following steps:

receiving first compressed data and second compressed data sent by the image processing chip, wherein the first compressed data are compressed by adopting a first compression algorithm, the second compressed data are compressed by adopting a second compression algorithm, and the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm;

decompressing the first compressed data to obtain a first sub-image, and decompressing the second compressed data to obtain a second sub-image;

performing fusion processing on the second subimage based on the historical image to obtain a third subimage;

and synthesizing the first sub-image and the third sub-image to obtain a corresponding image.

5. The image processing method of claim 4, wherein the receiving the first compressed data and the second compressed data sent by the image processing chip comprises: receiving the first compressed data, the second compressed data and the position information sent by the image processing chip;

decompressing the first compressed data to obtain a first sub-image, and decompressing the second compressed data to obtain a second sub-image;

performing fusion processing on the second sub-image based on the historical image to obtain a third sub-image;

obtaining a fourth sub-image based on the historical image and the position information;

and synthesizing the first sub-image, the third sub-image and the fourth sub-image to obtain a corresponding image.

6. An image processing chip is connected with an application processing chip, and is characterized in that the image processing chip comprises a neural network processor and a first central processing unit, wherein the neural network processor is connected with the first central processing unit through a system bus;

the neural network processor acquires a shot current image and a shot historical image;

the neural network processor divides the current image into at least a first area image and a second area image, and the change amplitude of the first area image relative to the corresponding area image in the historical image is larger than the change amplitude of the second area image relative to the corresponding area image in the historical image;

the first central processing unit compresses the first area image according to a first compression algorithm to obtain first compressed data, and compresses the second area image according to a second compression algorithm to obtain second compressed data, wherein the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm;

and the image processing chip transmits the first compressed data and the second compressed data to the application processing chip for processing.

7. The image processing chip of claim 6, wherein the neural network processor dividing the current image into at least a first region image and a second region image comprises: the neural network processor divides the current image into a first area image, a second area image and a third area image, wherein the change amplitude of the first area image relative to the corresponding area image in the historical image is larger than that of the second area image relative to the corresponding area image in the historical image, and the change amplitude of the second area image relative to the corresponding area image in the historical image is larger than that of the third area image relative to the corresponding area image in the historical image;

before the first compressed data and the second compressed data are transmitted to the application processing chip, the neural network processor acquires position information of the third area image;

the transmitting the first compressed data and the second compressed data to the application processing chip for processing includes: and transmitting the first compressed data, the second compressed data and the position information of the third area image to the application processing chip for processing.

8. An application processing chip is connected with an image processing chip and is characterized by comprising a second central processing unit;

the application processing chip receives first compressed data and second compressed data sent by the image processing chip, the first compressed data are compressed by adopting a first compression algorithm, the second compressed data are compressed by adopting a second compression algorithm, and the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm;

the second central processing unit decompresses the first compressed data to obtain a first sub-image, and decompresses the second compressed data to obtain a second sub-image;

the second central processing unit performs fusion processing on the second sub-image based on the historical image to obtain a third sub-image;

and the second central processing unit synthesizes the first sub-image and the third sub-image to obtain a corresponding image.

9. The application processing chip of claim 8, wherein the application processing chip receiving the first compressed data and the second compressed data sent by the image processing chip comprises: the application processing chip receives the first compressed data, the second compressed data and the position information sent by the image processing chip;

before the second central processor synthesizes the first sub-image and the third sub-image to obtain a corresponding image, the second central processor obtains a fourth sub-image based on the historical image and the position information;

the second central processing unit synthesizes the first sub-image and the third sub-image to obtain a corresponding image, and the method comprises the following steps: and the second central processing unit synthesizes the first sub-image, the third sub-image and the fourth sub-image to obtain a corresponding image.

10. An electronic device comprising the image processing chip of claim 6 or 7 and the application processing chip of claim 8 or 9, the image processing chip and the application processing chip being connected.

Images