CN115134544A

CN115134544A - Image processing method, image processing chip, application processing chip and electronic equipment

Info

Publication number: CN115134544A
Application number: CN202110328025.4A
Authority: CN
Inventors: 朱文波
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2021-03-26
Filing date: 2021-03-26
Publication date: 2022-09-30

Abstract

The application discloses an image processing method, an image processing chip, an application processing chip and electronic equipment. The method comprises the following steps: acquiring a shot current image and a shot historical image; dividing the current image into at least a first area image and a second area image, wherein the change range of the first area image relative to the corresponding area image in the historical image is larger than that of the second area image relative to the corresponding area image in the historical image; compressing the first area image according to a first compression algorithm to obtain first compressed data, and compressing the second area image according to a second compression algorithm to obtain second compressed data, wherein the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm; and transmitting the first compressed data and the second compressed data to an application processing chip for processing. The method and the device can improve the flexibility of image compression of the electronic equipment.

Description

Image processing method, image processing chip, application processing chip and electronic equipment

Technical Field

The present application belongs to the field of image processing technologies, and in particular, to an image processing method, an image processing chip, an application processing chip, and an electronic device.

Background

With the development of technology, users often use electronic devices to take images, such as taking photos or recording videos. However, in the related art, the electronic device only compresses the captured image in a single compression manner and then transmits the compressed image to the application processing chip for image processing, so the flexibility of the image processing method of the electronic device is poor.

Disclosure of Invention

The embodiment of the application provides an image processing method, an image processing chip, an application processing chip and an electronic device, and can improve the flexibility of the image processing method of the electronic device.

In a first aspect, an embodiment of the present application provides an image processing method, which is applied to an image processing chip, where the image processing chip is connected to the application processing chip, and the method includes:

acquiring a shot current image and a shot historical image;

dividing the current image into at least a first area image and a second area image, wherein the change amplitude of the first area image relative to the corresponding area image in the historical image is larger than the change amplitude of the second area image relative to the corresponding area image in the historical image;

compressing the first region image according to a first compression algorithm to obtain first compressed data, and compressing the second region image according to a second compression algorithm to obtain second compressed data, wherein the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm;

and transmitting the first compressed data and the second compressed data to the application processing chip for processing.

In a second aspect, an embodiment of the present application provides an image processing method, which is applied to an application processing chip, where the application processing chip is connected to the image processing chip, and the method includes:

receiving first compressed data and second compressed data sent by the image processing chip, wherein the first compressed data are compressed by adopting a first compression algorithm, the second compressed data are compressed by adopting a second compression algorithm, and the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm;

decompressing the first compressed data to obtain a first sub-image, and decompressing the second compressed data to obtain a second sub-image;

performing fusion processing on the second sub-image based on the historical image to obtain a third sub-image;

and synthesizing the first sub-image and the third sub-image to obtain a corresponding image.

In a third aspect, an embodiment of the present application provides an image processing chip, where the image processing chip is connected to an application processing chip, and the image processing chip includes a neural network processor and a first central processing unit, where the neural network processor is connected to the first central processing unit through a system bus;

the neural network processor acquires a shot current image and a shot historical image;

the neural network processor divides the current image into at least a first area image and a second area image, and the change amplitude of the first area image relative to the corresponding area image in the historical image is larger than that of the second area image relative to the corresponding area image in the historical image;

the first central processing unit compresses the first area image according to a first compression algorithm to obtain first compressed data, and compresses the second area image according to a second compression algorithm to obtain second compressed data, wherein the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm;

and the image processing chip transmits the first compressed data and the second compressed data to the application processing chip for processing.

In a fourth aspect, an embodiment of the present application provides an application processing chip, where the application processing chip is connected to an image processing chip, and the application processing chip includes a second central processing unit;

the application processing chip receives first compressed data and second compressed data sent by the image processing chip, the first compressed data are compressed by adopting a first compression algorithm, the second compressed data are compressed by adopting a second compression algorithm, and the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm;

the second central processing unit decompresses the first compressed data to obtain a first sub-image, and decompresses the second compressed data to obtain a second sub-image;

the second central processing unit performs fusion processing on the second sub-image based on the historical image to obtain a third sub-image;

and the second central processing unit synthesizes the first sub-image and the third sub-image to obtain a corresponding image.

In a fifth aspect, an embodiment of the present application provides an electronic device, which includes the image processing chip according to any embodiment of the present application and the application processing chip according to any embodiment of the present application, where the image processing chip and the application processing chip are connected.

In the embodiment of the application, an image processing chip compares historical images, divides a current image into at least a first area image and a second area image with different change amplitudes, compresses the current image by adopting different compression algorithms according to the change amplitudes of the different area images, adopts a first compression algorithm with smaller compression ratio for the first area image with larger change amplitude, and adopts a second compression algorithm with larger compression ratio for the second area image with smaller change amplitude. It can be understood that, the first compression algorithm with a smaller compression ratio is adopted for the first area image with a large change amplitude to perform compression, so that the compression loss of the first area image can be reduced, and the second compression algorithm with a larger compression ratio is adopted for the second area image with a small change amplitude to perform compression, so that the transmission quantity of image data can be reduced, an application processing chip can process images in time, and the flexibility of image compression of the electronic device is improved.

Drawings

The technical solutions and advantages of the present application will become apparent from the following detailed description of specific embodiments of the present application when taken in conjunction with the accompanying drawings.

Fig. 1 is a schematic flowchart of a first image processing method according to an embodiment of the present application.

Fig. 2 is a schematic flowchart of a second image processing method according to an embodiment of the present application.

Fig. 3 is a schematic flowchart of a third image processing method according to an embodiment of the present application.

Fig. 4 is a fourth flowchart illustrating an image processing method according to an embodiment of the present application.

Fig. 5 is a schematic flowchart of a fifth image processing method according to an embodiment of the present application.

Fig. 6 is a scene schematic diagram of an image processing method according to an embodiment of the present application.

Fig. 7 is a schematic view of another scene of an image processing method according to an embodiment of the present application.

Fig. 8 is a schematic structural diagram of an image processing chip and an application processing chip provided in an embodiment of the present application.

Fig. 9 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.

Detailed Description

Referring to the drawings, wherein like reference numbers refer to like elements, the principles of the present application are illustrated as being implemented in a suitable computing environment. The following description is based on illustrated embodiments of the application and should not be taken as limiting the application with respect to other embodiments that are not detailed herein.

It is understood that the execution subject of the embodiment of the present application may be an electronic device such as a smart phone or a tablet computer.

Referring to fig. 1, fig. 1 is a first flowchart illustrating an image processing method according to an embodiment of the present disclosure. The image processing method can be applied to an image processing chip, the image processing chip is connected with an application processing chip, and the flow of the image processing method can comprise the following steps:

in 101, a captured current image and a history image are acquired.

With the development of technology, users often use electronic devices to take images, such as taking photos or recording videos. However, in the related art, after the image processing chip acquires the image data collected by the camera, the image processing chip transmits the image data of the whole frame to the application processing chip through the MIPI interface for performing the related image processing, and the transmitted image data may be in a RAW format, a YUV format, or another format. The transmission technology applied at present has the characteristics that the problem of overlarge transmission pressure is easily caused by directly transmitting the whole frame of image data, and even if the image data is compressed according to the same standard by using the same compression algorithm, the problem that the image loss cannot be compensated is easily caused. That is, the electronic device only uses a single compression method to compress the shot image and then transmits the compressed image to the application processing chip for image processing, so the flexibility of the image processing method of the electronic device is poor.

In the embodiment of the application, after the camera acquires the image, the image can be transmitted to the image processing chip through the interface, so that the image processing chip can acquire the current image and the historical image, where the current image may be an image of the current frame, and the historical image may be an image of the previous frame or images of adjacent previous frames, such as images of the previous 3 frames and the previous 5 frames. The history image may be one frame image or a plurality of frame images.

It will be appreciated that, for the image of the previous frame or the images of the previous adjacent frames, only a part of the current image may be changed greatly, and the other part may be changed slightly or not.

At 102, the current image is divided into at least a first area image and a second area image, and the change amplitude of the first area image relative to the area image corresponding to the historical image is larger than the change amplitude of the second area image relative to the area image corresponding to the historical image.

In the embodiment of the application, the image processing chip compares the current image with the historical image and divides the current image into at least a first area image and a second area image. The change range of the first area image relative to the area image corresponding to the historical image is larger than the change range of the second area image relative to the area image corresponding to the historical image.

In one embodiment, the image processing chip compares the current image with the historical image, sets an area image with a variation amplitude larger than a preset amplitude threshold value as a first area image, and sets an area image with a variation amplitude smaller than the preset amplitude threshold value as a second area image. The specific preset amplitude threshold may be set as needed, for example, the preset amplitude threshold may be set according to the size of the current image, the transmission pressure of the transmission module, and the like, which is not limited herein.

It should be noted that, in the present application, the variation amplitude of different areas in the current image with respect to the historical image may be divided into more areas, such as a third area image, a fourth area image, and the like.

In one embodiment, the image processing chip may pre-divide the current image into a plurality of sub-region images of n columns and m rows, compare each sub-region image with an image of the same region in the history image, set at least one sub-region image with a variation amplitude larger than a preset amplitude threshold as a first region image, and set at least one sub-region image with a variation amplitude smaller than the preset amplitude threshold as a second region image.

It should be noted that, the more sub-regions that the image processing chip divides the current image in advance, the smaller the area of each sub-region is, and then the result of the division will be more accurate by judging the variation range of each sub-region to divide the first region image and the second region image. Therefore, the embodiment of the application can divide a plurality of sub-regions with different numbers as required, that is, the values of n and m can be set as required, the current image can be pre-divided into a plurality of sub-regions with uniform size, and also can be divided into a plurality of sub-regions with different sizes. For example, a subject region and a background region may be identified for a current image, and more sub-regions may be set for the subject region with a higher change rate, or it may be understood that each sub-region set for the subject region has a smaller area, and fewer sub-regions may be set for the background region with a lower change rate, or it may be understood that each sub-region set for the background region has a larger area.

In one embodiment, the image processing chip may divide the current image into a plurality of subarea images; acquiring image characteristic points of a current image; acquiring the variation amplitude of a plurality of subarea images based on the position information of the image feature points in the current image and the historical image; setting at least one subregion image with the variation amplitude larger than a preset amplitude threshold value as a first region image, and setting at least one subregion image with the variation amplitude smaller than the preset amplitude threshold value as a second region image.

In one embodiment, after the camera acquires the current image, the current image is transmitted to an image processing chip, and the image processing chip can identify image feature points of the current image and can also understand the image feature points as image key points in the current image.

In one embodiment, the image feature points may be identified by autonomous learning of a neural network, for example, CNN feature extraction, and the image key points may be identified, for example, the positions of the head, shoulders, arms, palms, legs, soles, and the like of a person in the image, and more specifically, the positions of the key points such as the eyes, nose, mouth, ears, eyebrows, and the like of the face in the head. For another example, the contour of the object in the image may be identified, and the distribution of the image feature points is selected according to the contour, such as the upper endpoint, the lower endpoint, the left endpoint, the right endpoint, the middle endpoint, and the like, which are set as the image feature points.

Acquiring a coordinate position of an image feature point, for example, the coordinate of the image feature point is (A, B), acquiring a corresponding coordinate position of the image feature point in the history image, for example, (a ', B'), judging whether the change amplitude of the coordinate value of the image feature point included in each sub-region of the current image and the history image is greater than a preset amplitude threshold value, if so, setting the sub-region as a first region image, and if not, setting the sub-region as a second region image.

In an embodiment, the image processing chip may further preset a reference point of each sub-region, for example, a middle position of each sub-region is a reference point, acquire a coordinate position of each reference point, for example, a coordinate of the reference point is (C, D), acquire a corresponding coordinate position of the reference point in the history image, for example, (C ', D'), determine whether a variation range of the coordinate values of the reference point of each sub-region in the current image and the history image is greater than a preset range threshold, if yes, set as the first region image, and if not, set as the second region image.

In 103, the first region image is compressed according to a first compression algorithm to obtain first compressed data, and the second region image is compressed according to a second compression algorithm to obtain second compressed data, wherein the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm.

In the embodiment of the application, the change amplitude of the first area image in the current image relative to the historical image is large, so that for the first area image, the image processing chip adopts a first compression algorithm with small compression ratio, the image loss of the obtained first compressed data is small, and the first area image can be better stored; the first compression algorithm may be a lossless compression method, and if a lossless compression method is adopted, the first compressed data after decompression can completely restore the original first region image without causing any distortion, where the lossless compression method includes: Shannon-Fano coding, Huffman coding, Run-length (Run-length) coding, LZW (Lempel-Ziv-Welch) coding, arithmetic coding, and the like.

The change amplitude of the second area image in the current image relative to the historical image is small or unchanged, for example, a part of background area in the current image, therefore, the image processing chip adopts a second compression algorithm with a large compression ratio for the second area image, which can also be understood as lossy compression, obtains certain image information of the second compressed data, which is allowed to be lost in the compression process according to a specific compression ratio, and can not completely recover the original second area image after decompression, but can reduce the capacity occupied by the image, thereby reducing the transmission quantity of the image data and the pressure of image transmission.

At 104, the first compressed data and the second compressed data are transmitted to an application processing chip for processing.

In the embodiment of the application, compared with historical images, the image processing chip divides the current image into at least a first area image and a second area image with different change amplitudes, and compresses the images by adopting different compression algorithms according to the change amplitudes of the images in different areas, adopts a first compression algorithm with smaller compression ratio according to a first area image with larger change amplitude, and adopts a second compression algorithm with larger compression ratio according to a second area image with smaller change amplitude. It can be understood that, the first compression algorithm with a smaller compression ratio is adopted for the first area image with a large change amplitude to perform compression, so that the compression loss of the first area image can be reduced, and the second compression algorithm with a larger compression ratio is adopted for the second area image with a small change amplitude to perform compression, so that the transmission quantity of image data can be reduced, an application processing chip can process the image in time, and the flexibility of image compression of the electronic equipment is improved.

Referring to fig. 2, fig. 2 is a schematic flow chart of an image processing method according to an embodiment of the present application, where the flow chart may include:

in 201, a current image and a history image of photographing are acquired.

In the embodiment of the application, after the camera acquires the image, the image can be transmitted to the image processing chip, so that the image processing chip can acquire the current image and the historical image which are shot, wherein the current image can be an image of the current frame, and the historical image can be an image of the previous frame or images of adjacent previous frames, such as images of the previous 3 frames and the previous 5 frames. The history image may be a single frame image or a plurality of frame images.

It will be appreciated that the current image may have only a portion that has changed significantly and other portions that have changed little or no change from the previous image or from the next previous image.

In 202, the current image is divided into a first area image, a second area image and a third area image, wherein the change amplitude of the first area image relative to the corresponding area image in the history image is larger than the change amplitude of the second area image relative to the corresponding area image in the history image, and the change amplitude of the second area image relative to the corresponding area image in the history image is larger than the change amplitude of the third area image relative to the corresponding area image in the history image.

In the embodiment of the application, the image processing chip compares the current image with the historical image and divides the current image into the first area image, the second area image and the third area image. The change amplitude of the first area image relative to the area image corresponding to the historical image is larger than that of the second area image relative to the area image corresponding to the historical image, and the change amplitude of the second area image relative to the corresponding area image in the historical image is larger than that of the third area image relative to the area image corresponding to the historical image.

In one embodiment, the current image and the historical image are compared, the area image with the variation amplitude larger than a first preset amplitude threshold value is set as a first area image, the area image with the variation amplitude not larger than the first preset amplitude threshold value but larger than a second preset amplitude threshold value is set as a second area image, and the area image with the variation amplitude not larger than the second preset amplitude threshold value is set as a third area image. That is, compared with the first area image and the second area image, the third area image has little or no change in the change range with respect to the history image.

It should be noted that the specific first preset amplitude threshold and the second preset amplitude threshold may be set as needed, for example, the first preset amplitude threshold and the second preset amplitude threshold may be set according to the size of the current image, the transmission pressure of the transmission module, and the like, which is not limited herein.

In one embodiment, the current image may be divided into a plurality of sub-region images; acquiring image characteristic points of a current image; acquiring the variation amplitude of a plurality of subarea images based on the position information of the image feature points in the current image and the historical image; setting at least one subregion image with the variation amplitude larger than a first preset amplitude threshold value as a first region image, setting at least one subregion image with the variation amplitude not larger than the first preset amplitude threshold value but larger than a second preset amplitude threshold value as a second region image, and setting at least one subregion image with the variation amplitude not larger than the second preset amplitude threshold value as a third region image.

It should be noted that the image processing chip may divide a plurality of sub-regions of different numbers as required, may divide the current image into a plurality of sub-regions of uniform size in advance, and may also divide the current image into a plurality of sub-regions of different sizes.

It should be noted that, the more the image processing chip divides the current image into sub-regions in advance, the smaller the area of each sub-region is, and then the more the change amplitude of each sub-region is determined to divide the first region image, the second region image, and the third region image, the more accurate the division result is. Therefore, the embodiment of the application can divide a plurality of sub-regions with different numbers as required, can divide the current image into a plurality of sub-regions with uniform size in advance, and can also divide the current image into a plurality of sub-regions with different sizes. For example, a main area and a background area may be identified for a current image, and more sub-areas may be set for the main area with a higher rate of change, or it may be understood that each sub-area set for the main area is smaller, and less sub-areas may be set for the background area with a lower rate of change, or it may be understood that each sub-area set for the background area is larger.

In one embodiment, after the camera acquires the current image, the current image is transmitted to the image processing chip, and the image processing chip can identify the image feature points of the current image and can also understand the image feature points as the image key points of the object in the current image. In one embodiment, the image feature points may be identified by autonomous learning of a neural network, for example, CNN feature extraction, and the image key points may be identified, for example, the positions of the head, shoulders, arms, palms, legs, soles, and the like of a person in the image, and more specifically, the positions of the key points such as the eyes, nose, mouth, ears, eyebrows, and the like of the face in the head. For another example, the contour of the object in the image may be identified, and the distribution of the image feature points is selected according to the contour, such as the upper endpoint, the lower endpoint, the left endpoint, the right endpoint, the middle endpoint, and the like, which are set as the image feature points.

Acquiring a coordinate position of an image feature point, for example, the coordinate of the image feature point is (A, B), then acquiring a corresponding coordinate position of the image feature point in the history image, for example, (a ', B'), judging a change amplitude of the coordinate value of the image feature point contained in each sub-region of the current image and the history image, if the change amplitude is greater than a first preset amplitude threshold, setting the sub-region as a first region image, if the change amplitude is not greater than the first preset amplitude threshold but greater than a second preset amplitude threshold, setting the sub-region as a second region image, and if the change amplitude is not greater than the second preset amplitude threshold, setting the sub-region as a third region image.

In an embodiment, a reference point of each sub-region may also be preset, for example, the middle position of each sub-region is a reference point, the coordinate position of each reference point is obtained, for example, the coordinate of the reference point is (C, D), then the corresponding coordinate position of the reference point in the history image is obtained, for example, (C ', D'), the change amplitude of the coordinate value of the reference point of each sub-region in the current image and the history image is determined, if the change amplitude is greater than a first preset amplitude threshold, the sub-region is set as a first region image, if the change amplitude is not greater than the first preset amplitude threshold but greater than a second preset amplitude threshold, the sub-region is set as a second region image, and if the change amplitude is not greater than the second preset amplitude threshold, the sub-region is set as a third region image.

In 203, the first region image is compressed according to a first compression algorithm to obtain first compressed data, and the second region image is compressed according to a second compression algorithm to obtain second compressed data, wherein the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm.

In the embodiment of the application, the change amplitude of the first area image in the current image relative to the historical image is large, so that the image processing chip adopts a first compression algorithm with small compression ratio aiming at the first area image, the image loss of the obtained first compressed data is small, and the first area image can be better stored; the first compression algorithm may be a lossless compression method, and if a lossless compression method is adopted, the first compressed data after decompression can completely restore the original first region image without causing any distortion, where the lossless compression method includes: Shannon-Fano coding, Huffman coding, Run-length (Run-length) coding, LZW (Lempel-Ziv-Welch) coding, arithmetic coding, and the like.

Although the change amplitude of the second area image in the current image relative to the historical image is smaller than that of the first area, the second area image still has a certain change amplitude, therefore, the image processing chip adopts a second compression algorithm with a large compression ratio for the second area image, which can also be understood as lossy compression, obtains certain image information of the second compressed data, which is allowed to be lost in the compression process according to a specific compression ratio, and can not completely recover the original second area image after decompression, but can reduce the capacity occupied by the image, thereby reducing the transmission quantity of the image data.

In 204, position information of the third area image is acquired.

Because the change amplitude of the third area image is very small or even no change relative to the corresponding area image of the historical image, the image processing chip can directly acquire the position information of the third area image, and the application processing chip can acquire the corresponding area image of the historical image according to the position information without compressing and transmitting the third area image.

In 205, the first compressed data, the second compressed data, and the location information are transmitted to an application processing chip for processing.

The image processing chip transmits the first compressed data and the second compressed data to the application processing chip for processing, it should be noted that the application processing chip may already have a history image cached therein, and the application processing chip may further process the image according to the received first compressed data, second compressed data, and history data.

Referring to fig. 3, fig. 3 is a schematic diagram of a third flow of an image processing method according to an embodiment of the present application, where the image processing method is applied to an application processing chip, and the application processing chip is connected to the image processing chip, and the flow may include:

in 301, first compressed data and second compressed data sent by an image processing chip are received, the first compressed data is compressed by using a first compression algorithm, the second compressed data is compressed by using a second compression algorithm, and the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm.

The application processing chip receives the first compressed data and the second compressed data sent by the image processing chip. The first compressed data is compressed by adopting a first compression algorithm, the second compressed data is compressed by adopting a second compression algorithm, the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm, the first compressed data corresponds to a first area image of a current image shot by a camera, the second compressed data corresponds to a second area image of the current image, the change amplitude of the first area image relative to the corresponding area image in the historical image is larger than that of the second area image relative to the corresponding area image in the historical image, and the historical image is the previous frame or previous frames of images of the current image.

It is to be understood that the image processing chip may be the image processing chip in any one of the embodiments described above, and the first compressed data and the second compressed data may be the first compressed data and the second compressed data in any one of the embodiments described above, which is not described herein again.

In 302, the first compressed data is decompressed to obtain a first sub-image, and the second compressed data is decompressed to obtain a second sub-image.

And the application processing chip decompresses the first compressed data to obtain a first sub-image, and decompresses the second compressed data to obtain a second sub-image. The first sub-image is a sub-image corresponding to the first area image, and the second sub-image is a sub-image corresponding to the second area image.

It is understood that the first sub-image and the second sub-image may be the first sub-image and the second sub-image in the above embodiments, and are not described herein again.

In 303, the second sub-image is fused based on the historical image to obtain a third sub-image.

And the application processing chip performs fusion processing on the second subimage based on the historical image to obtain a third subimage, wherein the historical image is an image which is stored by the application processing chip and is sent by the image processing chip before. The history image has an association with the second sub-image, e.g. the history image may be an image of a frame or several frames before the second sub-image.

In 304, the first sub-image and the third sub-image are combined to obtain a corresponding image.

And the application processing chip synthesizes the first sub-image and the third sub-image to obtain a corresponding image, wherein the corresponding image can be an image obtained by processing a current image shot by a camera. For example, the first sub-image is a sub-image of the current image, which is transmitted without compression in the first area image, the second sub-image is a sub-image of the current image, which is subjected to lossy compression in the second area image, and the third sub-image is a sub-image of the current image, which is subjected to fusion processing between the second sub-image and the historical image, so that the obtained third sub-image has less image loss and is closer to the image of the second area, the image quality is higher, and the transmission efficiency of the image processing chip and the application processing chip is improved.

Referring to fig. 4, fig. 4 is a fourth schematic flowchart of an image processing method according to an embodiment of the present disclosure, where the image processing method is applied to an application processing chip, and the application processing chip is connected to the image processing chip, and the flow may include:

in 401, the first compressed data, the second compressed data and the position information sent by the image processing chip are received.

And the application processing chip receives the first compressed data, the second compressed data and the position information sent by the image processing chip. The first compressed data is compressed by adopting a first compression algorithm, the second compressed data is compressed by adopting a second compression algorithm, the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm, the first compressed data corresponds to a first area image of a current image shot by a camera, the second compressed data corresponds to a second area image of the current image, the position information is a third area image corresponding to the current image, the change amplitude of the first area image relative to the corresponding area image in the historical image is larger than that of the second area image relative to the corresponding area image in the historical image, the change amplitude of the second area image relative to the corresponding area image in the historical image is larger than that of the third area image relative to the corresponding area image in the historical image, and the historical image is the previous frame or several previous frames of the current image.

It is to be understood that the image processing chip may be the image processing chip in any one of the embodiments, and the first compressed data, the second compressed data, and the location information may be the first compressed data and the second compressed data in any one of the embodiments, which is not described herein again.

In 402, the first compressed data is decompressed to obtain a first sub-image corresponding to the first region image, and the second compressed data is decompressed to obtain a second sub-image corresponding to the second region image.

After the application processing chip receives the first compressed data and the second compressed data, the application processing chip decompresses the first compressed data to obtain a first sub-image corresponding to the first area image, and decompresses the second compressed data to obtain a second sub-image corresponding to the second area image.

In this embodiment of the present application, a lossless compression method may be further used for the first area image, so that the decompressed first sub-image may completely restore the original first area image without causing any distortion, that is, the first sub-image obtained by decompressing the first compressed data by using the processing chip may be the first area image.

Since the second region image adopts the second compression algorithm with a large compression ratio, which can also be understood as lossy compression, the original second region image cannot be completely recovered after the second compressed data is decompressed, and partial image information is lost, that is, the second sub-image has a certain image loss relative to the second region image according to a specific compression ratio.

In 403, the second sub-image is fused based on the historical image to obtain a third sub-image.

It is understood that the application processing chip may have a history image buffered therein, such as an image of the previous frame or an image of the previous adjacent frame. The image of the second area has a smaller variation range than the image of the first area relative to the image of the corresponding area in the history image, but has a certain variation range. After the application processing chip receives the second compressed data, a second sub-image obtained by decompressing the second compressed data has image loss relative to a second area image in the current image.

In the embodiment of the application, the application processing chip can acquire the image of the area corresponding to the second area image in the current image in the historical image, and perform fusion processing on the image of the corresponding area and the second sub-image to obtain the third sub-image.

In one embodiment, the area image corresponding to the history image may be adjusted according to the change process of the history image, such as the adjacent frames of images, so that the image change of the corresponding area image in the adjusted history image and the image change of the second area are smaller and closer; and then correcting and enhancing the image details lost by the second sub-image after decompression through the area image corresponding to the adjusted historical image to obtain a third sub-image.

Therefore, even if the second sub-image obtained by decompression has a certain image loss relative to the second region image in the current image, after the second sub-image is subjected to enhancement fusion processing with the historical image, the obtained third sub-image is closer to the image of the second region in the current image, the image loss is less, the image quality of the third sub-image is improved, and even the second sub-image has higher image quality relative to the second region image, such as clearer image quality.

At 404, a corresponding fourth sub-image is derived based on the historical image and the location information.

Since the third area image has a small or no change in its variation amplitude with respect to the history image, the application processing chip may determine the area image corresponding to the third area image in the history image according to the position information of the third area image, and use the area image as the fourth sub-image corresponding to the third area image.

In 405, the first sub-image, the third sub-image and the fourth sub-image are synthesized to obtain a corresponding image.

Because the first area image adopts the first compression algorithm with smaller compression ratio, the loss of the first sub-image obtained by the decompression of the application processing chip is smaller relative to the first area image, if the first area image adopts a lossless compression method, the decompressed first sub-image can completely restore the original first area image without causing any distortion, namely, the first sub-image obtained by the decompression of the application processing chip can be the first area image.

The second area image adopts a first compression algorithm with a large compression ratio, and the second sub-image obtained by decompression of the processing chip has image loss relative to the second area image.

Since the third area image has a small or no change in its change amplitude with respect to the history image, in the embodiment of the present application, the area image having the same position information as the third area image in the history image is used as the fourth sub-image corresponding to the third area image, and it can be understood that the image loss of the fourth sub-image with respect to the third area image is also small or no loss.

In the embodiment of the application, because the image loss of the first sub-image relative to the first area image of the current image is small or no loss, the image loss of the third sub-image relative to the second area image of the current image is also small, and the image loss of the fourth sub-image relative to the third area image is also small or no loss, the application processing chip synthesizes the first sub-image, the third sub-image and the fourth sub-image to obtain the image corresponding to the current image, the image loss is also small, and the application processing chip can perform further image processing based on the image.

It can be understood that, if only a part of the current image and the historical image has a large change, a part of the current image and the historical image have a small change, and another part of the current image and the historical image have a small change or no change, only part of the first region image needs to be compressed by a compression method with a small compression ratio to reduce the image loss of the first region image, while the second region image can be compressed by a compression method with a large compression ratio, thereby reducing the transmission quantity of the image data, only sending the position information of the third area image to the application processing chip without compressing and transmitting the third area image, further reducing the transmission quantity of the image data, greatly reducing the transmission pressure, therefore, the application processing chip can receive image data in real time for processing, and the image processed by the application processing chip can be ensured to have smaller image loss compared with the current image.

Referring to fig. 5, fig. 5 is a fifth flowchart illustrating an image processing method according to an embodiment of the present disclosure, where the image processing method can be applied to an image processing chip and an application processing chip, and the image processing chip and the application processing chip are connected. The process can comprise the following steps:

in 501, a current image and a history image of photographing are acquired.

In this embodiment of the application, after the camera acquires the image, the current image may be transmitted to the image processing chip, so that the image processing chip may acquire the current image and the historical image, where the current image may be an image of the current frame, and the historical image may be an image of the previous frame or images of adjacent previous frames, such as images of the previous 3 frames and the previous 5 frames. It will be appreciated that, for the image of the previous frame or the images of the previous adjacent frames, the current image may have only a portion with a large change and other portions with little or no change.

At 502, the current image is divided into at least a first region image and a second region image, and the variation amplitude of the first region image relative to the corresponding region image in the history image is larger than the variation amplitude of the second region image relative to the corresponding region image in the history image.

In the embodiment of the application, the image processing chip compares the current image with the historical image and divides the current image into at least a first area image and a second area image. The change amplitude of the first area image relative to the area image corresponding to the historical image is larger than the change amplitude of the second area image relative to the area image corresponding to the historical image.

In one embodiment, the image processing chip compares the current image with the historical image, sets the area image with the variation amplitude larger than the preset amplitude threshold value as a first area image, and sets the area image with the variation amplitude not larger than the preset amplitude threshold value as a second area image. The specific preset amplitude threshold may be set as needed, for example, the preset amplitude threshold may be set according to the size of the current image, the transmission pressure of the transmission module, and the like, which is not limited herein.

At 503, the first region image is compressed according to a first compression algorithm to obtain first compressed data, and the second region image is compressed according to a second compression algorithm to obtain second compressed data, wherein a compression ratio of the first compression algorithm is smaller than that of the second compression algorithm.

In the embodiment of the application, the change amplitude of the first area image in the current image relative to the historical image is large, so that the image processing chip adopts a first compression algorithm with small compression ratio aiming at the first area image, the image loss of the obtained first compressed data is small, and the first area image can be better stored; the first compression algorithm may be a lossless compression method, and if a lossless compression method is adopted, the first compressed data after decompression can completely restore the original first region image without causing any distortion.

The change amplitude of the second area image in the current image relative to the historical image is small or unchanged, therefore, the image processing chip adopts a second compression algorithm with a large compression ratio for the second area image, which can also be understood as lossy compression, obtains certain image information of the second compressed data lost in the compression process allowed according to a specific compression ratio, and can not completely recover the original second area image after decompression, but can reduce the transmission quantity of the image data and reduce the pressure of image transmission.

At 504, the first compressed data and the second compressed data are transmitted to an application processing chip for processing.

In 505, the first compressed data is decompressed to obtain a first sub-image corresponding to the first region image, and the second compressed data is decompressed to obtain a second sub-image corresponding to the second region image.

After the application processing chip receives the first compressed data and the second compressed data sent by the image processing chip, the application processing chip decompresses the first compressed data to obtain a first sub-image corresponding to the first area image, and decompresses the second compressed data to obtain a second sub-image corresponding to the second area image.

At 506, the second sub-image is fused based on the historical image to obtain a third sub-image.

After the application processing chip receives the second compressed data, a second sub-image obtained by decompressing the second compressed data has image loss relative to a second area image in the current image.

In the embodiment of the application, the application processing chip can acquire the image of the area corresponding to the second area image in the current image in the historical image, and perform fusion processing on the image of the corresponding area and the second sub-image to obtain the third sub-image. The history image may be an image of the previous frame or an image of the adjacent previous frames.

The image of the second region has little or no change in magnitude from the image of the corresponding region in the history image. It can be understood that the application processing chip may already have the history image cached therein, and when the application processing chip performs the fusion enhancement processing on the second sub-image, the application processing chip may directly perform the use of the cached history image without the need of the pre-image processing chip to transmit the required history image again.

It should be noted that, because the image of the second region has a smaller or no change relative to the image of the corresponding region in the history image, such as the background region in the current image, the history image may be an image of the first few frames apart from each other, and the updating is not required in real time, and the requirement of the application processing chip can be met by updating every certain number of frames. For example, the history data is buffered every 3 or 5 frames.

In an embodiment, image details of the decompressed second sub-image lost relative to the second area image may be identified, and then the lost image details are corrected and enhanced by the area image corresponding to the historical image, so as to compensate for the image loss caused by the lossy compression, and obtain a third sub-image, for example, the lost image details of the second sub-image may be compensated by the corresponding area image in the historical image.

Therefore, even if the second sub-image obtained by decompression has a certain image loss relative to the second region image in the current image, after the second sub-image is subjected to enhancement fusion processing with the historical image, the obtained third sub-image is closer to the image of the second region in the current image, the image loss is less, the image quality of the third sub-image is improved, and even the second sub-image has higher image quality relative to the second region image, for example, the image is clearer.

In an implementation manner, the history image may be multiple frames, the fusion processing may be performed on the multiple frames of history images and the second sub-image, and the number of frames of the history image may be controlled as needed, so that the obtained third sub-image may obtain better image quality.

In an embodiment, the image of the second region may also be partially fused based on the historical image to obtain a third sub-image, for example, the background region and the subject region of the current image may be identified, only the portion of the image of the second region related to the subject region is fused and enhanced, and the portion of the image of the second region where the background region is designed is not fused and enhanced, so as to reduce the image range of the fusion processing of the application processing chip and improve the efficiency of the image processing of the application processing chip.

In an embodiment, even for the first region image with a large variation range, the first region image may be further divided and processed separately, for example, it is determined that the head region of a person in the image is a high variation region, and it may be further determined that the eye partial image and the nose partial image of the face in the head region have a small variation range, so that after the first region image is decompressed to obtain the first sub-image, the image portion with a small variation in the first sub-image may be subjected to fusion enhancement processing using the history image, thereby further improving the quality of the first sub-image.

In 507, the first sub-image and the third sub-image are synthesized to obtain an image corresponding to the current image.

Because the first area image adopts the first compression algorithm with smaller compression ratio, the loss of the first sub-image obtained by decompressing by the application processing chip is smaller compared with the first area image, if the first area image adopts a lossless compression method, the decompressed first sub-image can completely restore the original first area image without causing any distortion, namely the first sub-image obtained by decompressing by the application processing chip can be the first area image.

The second area image adopts a first compression algorithm with a large compression ratio, and the second sub-image obtained by decompression of the processing chip has image loss relative to the second area image, but because the area change amplitude of the second area image corresponding to the historical image is small or unchanged, the application implements that after the second sub-image and the historical image are subjected to enhancement fusion processing, the obtained third sub-image has small image loss and is closer to the image of the second area, and the image quality is higher.

In the embodiment of the application, because the image loss of the first sub-image relative to the first area image of the current image is small or no loss, and the image loss of the third sub-image relative to the second area image of the current image is also small, the processing chip is applied to fuse the first sub-image and the third sub-image to obtain the image corresponding to the current image, the image loss is also small, and further image processing can be performed based on the image.

It can be understood that if only a small part of the current image and the historical image has a large change and most of the current image and the historical image have no transmission change, only the small part of the first area image needs to be compressed by a compression method with a small compression ratio to reduce the image loss of the first area image, and most of the second area image needs to be compressed by a compression method with a large compression ratio, so that the transmission amount of image data can be greatly reduced, and the application processing chip can receive the image data in time.

In the embodiment of the application, the image processing chip compares the historical images to divide the current image into at least a first area image and a second area image with different change amplitudes, and compresses the current image by adopting different compression algorithms according to the change amplitudes of the different area images, adopts a first compression algorithm with a smaller compression ratio according to the first area image with a larger change amplitude, and adopts a second compression algorithm with a larger compression ratio according to the second area image with a smaller change amplitude, so that the flexibility of image compression of the electronic equipment is improved.

In the embodiment of the application, a first compression algorithm with a smaller compression ratio, such as a lossless compression method, is adopted for compressing the first area image with a large change amplitude, and the first sub-image obtained by decompression by the processing chip has less image loss or no image loss relative to the first area image; the second compression algorithm with a large compression ratio is adopted for the second area image with a small change amplitude, so that the transmission quantity of image data can be reduced, but the second sub-image obtained by decompression of the processing chip has certain image loss relative to the second area image, so that the processing chip further performs fusion enhancement processing on the second sub-image based on the historical image to obtain a third sub-image, the third sub-image is closer to the image of the second area, and then the first sub-image and the third sub-image are fused to obtain the image corresponding to the current image. The image processing chip reduces the transmission quantity of image data through a differentiated compression mode, so that the application processing chip can receive the image data in time, and the fusion enhancement processing can be carried out on the second sub-image to ensure that the obtained image has smaller image loss relative to the current image, the image quality is higher, and even the image quality is enhanced.

It should be noted that, in the processing of the image differentiation algorithm, reference may also be made to the technical scheme in the embodiment of the present application, for example, the noise of the image is processed in different regions, a relatively complex algorithm is used for the region with more noise in the image to perform noise reduction, and another relatively simple algorithm may be used for the region with less noise in the image to perform noise reduction.

Referring to fig. 6 to 7, fig. 6 is a schematic view of a scene of an image processing method according to an embodiment of the present application, and fig. 7 is a schematic view of another scene of the image processing method according to the embodiment of the present application.

The camera application of the electronic device is started, and is in a shooting state, the camera starts to acquire images, and the format of the images can be a RAW format or other formats.

For example, after the camera acquires the current image, the current image is transmitted to the image processing chip, and the neural network processor of the image processing chip can identify the image feature points of the current image and can also be understood as the image key points of the object in the current image. In one embodiment, the image feature points may be identified by autonomous learning of a neural network, for example, CNN feature extraction, and the key points of the image, for example, the positions of the head, shoulders, arms, palms, legs, soles, etc., of a person in the image, more specifically, the positions of the key points, such as the eyes, nose, mouth, ears, eyebrows, etc., of the face in the head, are identified. For another example, the contour of the object in the image may be identified, and the distribution of the image feature points is selected according to the contour, such as the upper endpoint, the lower endpoint, the left endpoint, the right endpoint, the middle endpoint, and the like, which are set as the image feature points.

After the camera acquires the current image, the current image can be transmitted to an image processing chip, a neural network processor of the image processing chip compares the current image with the historical image, and the current image is divided into at least a first area image and a second area image. The change amplitude of the first area image relative to the area image corresponding to the historical image is larger than the change amplitude of the second area image relative to the area image corresponding to the historical image. The history image may be an image of a previous frame or an image of adjacent previous frames, such as an image of a previous 3 frames, an image of a previous 5 frames, and the like. The history image may be one frame image or a plurality of frame images.

For example, the neural network processor of the image processing chip may divide the current image into a plurality of subarea images; acquiring the variation amplitude of a plurality of subarea images based on the position information of the image feature points in the current image and the historical image; setting at least one subregion image with the variation amplitude larger than a preset amplitude threshold value as a first region image, and setting at least one subregion image with the variation amplitude smaller than the preset amplitude threshold value as a second region image.

The neural network processor of the image processing chip acquires the coordinate position of the image feature point, for example, the coordinate of the image feature point is (A, B), acquires the corresponding coordinate position of the image feature point in the historical image, for example, (A ', B'), judges whether the change amplitude of the coordinate value of the image feature point contained in each subarea of the current image and the historical image is larger than a preset amplitude threshold value, if so, the subarea is set as a first area image, and if not, the subarea is set as a second area image.

For example, referring to fig. 5, the neural network processor of the image processing chip may divide the current image into 16 sub-regions, determine whether the variation amplitude of the sub-region image is within a preset amplitude threshold, set the

sub-regions

3, 6, 7, 8, 11, and 14 whose variation amplitude is greater than the preset amplitude threshold as the first region image, and set the

sub-regions

1, 2, 4, 5, 9, 10, 12, 13, 15, and 16 whose variation amplitude is not greater than the preset amplitude threshold as the second region image.

The first central processing unit of the image processing chip compresses the first area image by adopting a lossless compression method to obtain first compressed data, and the first compressed data after decompression can completely recover the original first area image without causing any distortion.

The change amplitude of the second area image in the current image relative to the historical image is small or unchanged, therefore, the first central processing unit of the image processing chip adopts a second compression algorithm with a large compression ratio for the second area image, which can also be understood as lossy compression, certain image information lost in the compression process is allowed by the obtained second compressed data according to a specific compression ratio, and the original second area image cannot be completely recovered after decompression, but the transmission quantity of the image data can be reduced, and the pressure of image transmission is reduced.

The image processing chip transmits the first compressed data and the second compressed data to the application processing chip, and after the application processing chip receives the first compressed data and the second compressed data, a second central processing unit of the application processing chip can decompress the first compressed data to obtain a first sub-image corresponding to the first area image, and decompress the second compressed data to obtain a second sub-image corresponding to the second area image.

Because the first area image adopts the first compression algorithm with smaller compression ratio, the image loss of the decompressed first sub-image is smaller, and if the lossless compression method is adopted for the first area image, the first sub-image obtained by decompressing the first compressed data by the second central processing unit of the application processing chip may be the first area image.

After the application processing chip receives the second compressed data, the second central processing unit of the application processing chip may decompress the second compressed data to obtain a second sub-image having image loss relative to the second area image in the current image.

In the embodiment of the application processing chip, the second central processing unit can acquire the image of the region corresponding to the second region image in the current image in the historical image, and perform fusion processing on the image of the corresponding region and the second sub-image to obtain the third sub-image. For example, the image details of the decompressed second sub-image lost relative to the second region image may be identified first, and then the lost image details are corrected and enhanced by using the region image corresponding to the history image, so as to compensate for the image loss caused by lossy compression, and obtain a third sub-image. The image details of the second sub-image loss can be compensated by using the corresponding area image in the historical image, so that the image quality of the third sub-image is improved, the image loss of the third sub-image is smaller, and the image loss of the third sub-image is closer to that of the second area image.

The second central processing unit of the application processing chip can fuse the first sub-image and the third sub-image to obtain an image corresponding to the current image, and further process the image, such as adjusting white balance, contrast, brightness, and the like.

According to the image processing method and device, the transmission quantity of the image data is reduced through a differentiated compression mode, the application processing chip can receive the image data in time, the second central processing unit of the application processing chip can perform fusion enhancement processing on the second sub-image according to the historical image, the situation that the image processed by the application processing chip has smaller image loss relative to the current image is guaranteed, the image quality is higher, and even the image quality is enhanced to some extent.

Referring to fig. 8, fig. 8 is a schematic structural diagram of an image processing chip and an application processing chip according to an embodiment of the present disclosure. As shown in fig. 8, the image processing chip 400 provided in the embodiment of the present application may include a neural network processor 410 and a first central processor 430, where the neural network processor 410 and the first central processor 430 are connected by a system bus 450. The image processing chip 400 may further include an internal memory 470, the internal memory 470 is also connected through the system bus 450, and the neural network processor 410, the first central processing unit 430, and the like in the image processing chip 400 are configured to process the image captured by the camera 500 and store the processed image into the internal memory 470 through the system bus 450.

Still referring to fig. 8, the image processing chip 400 may be connected to the camera 500 through an interface and connected to the application processing chip 600 through another interface, for example, the camera 500 is connected to the system bus 450 through the third interface 420 so as to be connected to the image processing chip 400, the application processing chip 600 is connected to the first interface 440 of the image processing chip 400 through the second interface 620, and the first interface 440 is connected to the system bus 450.

For example, the image processing chip 400 stores the current image and the historical image captured by the camera 500 in the internal memory 470 through the third interface 420 and the system bus 450, and the neural network processor 410 may obtain the current image and the historical image from the internal memory 470 through the system bus 450.

The neural network processor 410 may divide the current image into at least a first area image and a second area image, and a variation range of the first area image with respect to a corresponding area image in the history image is larger than a variation range of the second area image with respect to the corresponding area image in the history image. The neural network processor 410 may store the first area image and the second area image to the internal memory 470 or transmit to the first central processor 430 through the system bus 450.

After the first central processing unit 430 obtains the first region image and the second region image through the system bus 450, the first region image may be compressed according to a first compression algorithm to obtain first compressed data, and the second region image may be compressed according to a second compression algorithm to obtain second compressed data, where a compression ratio of the first compression algorithm is smaller than a compression ratio of the second compression algorithm.

The image processing chip 400 transmits the first compressed data and the second compressed data to the application processing chip 600 for processing. For example, the first processor 430 may store the first compressed data and the second compressed data in the internal memory 470 through the system bus 450 and then transmit the first compressed data and the second compressed data to the application processing chip 600 through the first interface 440, or the first central processor 430 may transmit the first compressed data and the second compressed data to the application processing chip 600 through the first interface 440, and the application processing chip 600 may perform subsequent processing according to the first compressed data and the second compressed data.

In some other embodiments, the neural network processor 410 dividing the current image into at least a first region image and a second region image may include: the neural network processor 104 divides the current image into a first area image, a second area image and a third area image, wherein the change amplitude of the first area image relative to the corresponding area image in the historical image is larger than that of the second area image relative to the corresponding area image in the historical image, and the change amplitude of the second area image relative to the corresponding area image in the historical image is larger than that of the third area image relative to the corresponding area image in the historical image;

before transmitting the first compressed data and the second compressed data to the application processing chip 600, the neural network processor 410 may further obtain location information of the third area image;

transmitting the first compressed data and the second compressed data to the application processing chip 600 for processing may include: the first compressed data, the second compressed data and the location information are transmitted to the application processing chip 600 for processing.

The image processing chip 400 may also have more functions, which may specifically refer to the image processing chip in the above embodiments, and details are not described herein.

Referring to fig. 8, the application processing chip 600 is connected to the image processing chip 400, and the application processing chip 600 includes a second central processing unit 640 and a second interface 620. The application processing chip 600 may receive the first compressed data and the second compressed data transmitted by the image processing chip 400. For example, the second central processor 640 obtains the first compressed data and the second compressed data sent by the image processing chip 400 through the second interface 620. The first compressed data is compressed by using a first compression algorithm, the second compressed data is compressed by using a second compression algorithm, a compression ratio of the first compression algorithm is smaller than that of the second compression algorithm, and the first compressed data and the second compressed data may be the first compressed data and the second compressed data in the above embodiments, which is not described herein again.

The second central processing unit 640 decompresses the first compressed data to obtain a first sub-image, decompresses the second compressed data to obtain a second sub-image, and the second central processing unit 640 may further perform fusion processing on the second sub-image based on the historical image to obtain a third sub-image, and synthesize the first sub-image and the third sub-image to obtain a corresponding image.

In other embodiments, the application processing chip may also receive the location information sent by the image processing chip; before the second central processing unit of the application processing chip synthesizes the first sub-image and the third sub-image to obtain a corresponding image, the second central processing unit obtains a fourth sub-image based on the historical image and the position information; the second central processing unit can also synthesize the first sub-image, the third sub-image and the fourth sub-image to obtain corresponding images.

The structure and function of the application processing chip 600 may refer to those of the application processing chips in the above embodiments, and are not described herein again.

The embodiment of the present application further provides an electronic device, which includes a memory, an image processing chip, and an application processing chip, where the image processing chip and the application processing chip are configured to execute the processes in the image processing methods provided in the embodiments of the present application by calling a computer program stored in the memory. The electronic device may be a mobile terminal such as a tablet computer or a smart phone, the image processing chip is connected to the application processing chip, the image processing chip may be the image processing chip in any one of the embodiments, and the application processing chip may be the application processing chip in any one of the embodiments, which is not described herein again.

Referring to fig. 9, fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device 800 may include components such as a memory 801, an image processing chip 802, and an application processing chip 803. Those skilled in the art will appreciate that the electronic device configurations shown in the figures do not constitute limitations of the electronic device, and may include more or fewer components than shown, or some components in combination, or a different arrangement of components.

The memory 801 may be used to store computer programs and data. The memory 801 stores computer programs containing instructions executable in the processor. The computer program may constitute various functional modules.

The image processing chip 802 executes various image processing functions and data processing by calling a computer program stored in the memory 801.

The application processing chip 803 is a control center of the electronic device, and connects various parts of the whole electronic device by using various interfaces and lines, thereby performing overall monitoring on the electronic device.

In this embodiment, the image processing chip 802 in the electronic device 800 loads instructions corresponding to one or more computer program processes into the memory 801, and the image processing chip 802 runs the computer program stored in the memory 801, so as to implement various functions as follows:

acquiring a shot current image and a shot historical image;

compressing the first area image according to a first compression algorithm to obtain first compressed data, and compressing the second area image according to a second compression algorithm to obtain second compressed data, wherein the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm;

the first compressed data and the second compressed data are transmitted to the application processing chip 803 for processing.

In this embodiment, the application processing chip 803 in the electronic device 800 loads instructions corresponding to processes of one or more computer programs into the memory 801 according to the following steps, and the application processing chip 803 runs the computer programs stored in the memory 801, so as to implement various functions:

receiving first compressed data and second compressed data sent by an image processing chip, wherein the first compressed data are compressed by adopting a first compression algorithm, the second compressed data are compressed by adopting a second compression algorithm, and the compression ratio of the first compression algorithm is smaller than that of the second compression algorithm;

performing fusion processing on the second subimage based on the historical image to obtain a third subimage;

With continued reference to fig. 9, the electronic device 800 may further include a camera 804, a display 805, a control circuit 806, an input unit 807, an audio circuit 808, a sensor 809, and a power supply 810.

The camera 804 can be used for video shooting and image acquisition, and processes images by a photosensitive component circuit and a control component in the camera after acquiring the images by means of a lens, and then restores the images by means of software, so that the images can be seen through a display screen.

The display 805 may be used to display information input by or provided to a user as well as various graphical user interfaces of the electronic device, which may be made up of images, text, icons, video, and any combination thereof.

The control circuit 806 is electrically connected to the display 805, and is configured to control the display 805 to display information.

The input unit 807 may be used to receive input numerals, character information, or user characteristic information (e.g., fingerprint), and generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user setting and function control. The input unit 807 may include a fingerprint recognition module.

The audio circuitry 808 may provide an audio interface between the user and the electronic device through a speaker, microphone. Where the audio circuitry 808 includes a microphone. The microphone is electrically connected to the application processing chip 803. The microphone is used for receiving voice information input by a user.

The sensor 809 is used to collect external environment information. The sensors 809 may include one or more of ambient light sensors, acceleration sensors, gyroscopes, etc.

The power supply 810 is used to power the various components of the electronic device 800. In some embodiments, the power supply 810 may be logically connected to the application processing chip 803 through a power management system, so that functions of managing charging, discharging, and power consumption management are implemented through the power management system.

In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and parts that are not described in detail in a certain embodiment may refer to the above detailed description of the image processing method, and are not described again here.

The electronic device provided in the embodiment of the present application and the image processing method in the above embodiment belong to the same concept, and any method provided in the embodiment of the image processing method can be run on the electronic device, and the specific implementation process thereof is described in detail in the embodiment of the image processing method, and is not described herein again.

It should be noted that, for the image processing method in the embodiments of the present application, it can be understood by those skilled in the art that all or part of the processes for implementing the image processing method in the embodiments of the present application can be implemented by controlling the related hardware through a computer program, the computer program can be stored in a computer readable storage medium, such as a memory, and executed by at least one image processing chip and an application processing chip, and the processes such as the processes of the embodiments of the image processing method can be included in the execution process. The storage medium may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like.

The image processing method, the image processing apparatus, the storage medium, and the electronic device provided in the embodiments of the present application are described in detail above, and a specific example is applied in the present application to explain the principles and embodiments of the present application, and the description of the above embodiments is only used to help understand the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. An image processing method is applied to an image processing chip, and the image processing chip is connected with an application processing chip, and the method is characterized by comprising the following steps:

acquiring a shot current image and a shot historical image;

2. The image processing method of claim 1, wherein the dividing the current image into at least a first region image and a second region image comprises:

dividing the current image into a plurality of subarea images;

acquiring image characteristic points of the current image;

acquiring the variation amplitude of a plurality of subarea images based on the position information of the image feature points in the current image and the historical image;

setting at least one sub-area image with the variation amplitude larger than a preset amplitude threshold value as the first area image, and setting at least one sub-area image with the variation amplitude not larger than the preset amplitude threshold value as the second area image.

3. The image processing method of claim 1, wherein the dividing the current image into at least a first region image and a second region image comprises:

dividing the current image into a first area image, a second area image and a third area image, wherein the change amplitude of the first area image relative to the corresponding area image in the historical image is larger than that of the second area image relative to the corresponding area image in the historical image, and the change amplitude of the second area image relative to the corresponding area image in the historical image is larger than that of the third area image relative to the corresponding area image in the historical image;

before the transmitting the first compressed data and the second compressed data to the application processing chip, the method further includes: acquiring position information of the third area image;

transmitting the first compressed data and the second compressed data to the application processing chip for processing, including: and transmitting the first compressed data, the second compressed data and the position information of the third area image to the application processing chip for processing.

4. An image processing method is applied to an application processing chip, and the application processing chip is connected with the image processing chip, and is characterized by comprising the following steps:

5. The image processing method of claim 4, wherein the receiving the first compressed data and the second compressed data sent by the image processing chip comprises: receiving the first compressed data, the second compressed data and the position information sent by the image processing chip;

obtaining a fourth sub-image based on the historical image and the position information;

and synthesizing the first sub-image, the third sub-image and the fourth sub-image to obtain a corresponding image.

6. An image processing chip is connected with an application processing chip, and is characterized in that the image processing chip comprises a neural network processor and a first central processing unit, wherein the neural network processor is connected with the first central processing unit through a system bus;

the neural network processor divides the current image into at least a first area image and a second area image, and the change amplitude of the first area image relative to the corresponding area image in the historical image is larger than the change amplitude of the second area image relative to the corresponding area image in the historical image;

7. The image processing chip of claim 6, wherein the neural network processor dividing the current image into at least a first region image and a second region image comprises: the neural network processor divides the current image into a first area image, a second area image and a third area image, wherein the change amplitude of the first area image relative to the corresponding area image in the historical image is larger than that of the second area image relative to the corresponding area image in the historical image, and the change amplitude of the second area image relative to the corresponding area image in the historical image is larger than that of the third area image relative to the corresponding area image in the historical image;

before the first compressed data and the second compressed data are transmitted to the application processing chip, the neural network processor acquires position information of the third area image;

the transmitting the first compressed data and the second compressed data to the application processing chip for processing includes: and transmitting the first compressed data, the second compressed data and the position information of the third area image to the application processing chip for processing.

8. An application processing chip is connected with an image processing chip and is characterized by comprising a second central processing unit;

9. The application processing chip of claim 8, wherein the application processing chip receiving the first compressed data and the second compressed data sent by the image processing chip comprises: the application processing chip receives the first compressed data, the second compressed data and the position information sent by the image processing chip;

before the second central processor synthesizes the first sub-image and the third sub-image to obtain a corresponding image, the second central processor obtains a fourth sub-image based on the historical image and the position information;

the second central processing unit synthesizes the first sub-image and the third sub-image to obtain a corresponding image, and the method comprises the following steps: and the second central processing unit synthesizes the first sub-image, the third sub-image and the fourth sub-image to obtain a corresponding image.

10. An electronic device comprising the image processing chip of claim 6 or 7 and the application processing chip of claim 8 or 9, the image processing chip and the application processing chip being connected.