CN112153282B - Image processing chip, method, storage medium and electronic device - Google Patents
Image processing chip, method, storage medium and electronic device Download PDFInfo
- Publication number
- CN112153282B CN112153282B CN202010987725.XA CN202010987725A CN112153282B CN 112153282 B CN112153282 B CN 112153282B CN 202010987725 A CN202010987725 A CN 202010987725A CN 112153282 B CN112153282 B CN 112153282B
- Authority
- CN
- China
- Prior art keywords
- image
- hardware
- module
- shake
- distortion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/682—Vibration or motion blur correction
- H04N23/683—Vibration or motion blur correction performed by a processor, e.g. controlling the readout of an image memory
Abstract
The embodiment of the application discloses an image processing architecture, which comprises an application layer and a hardware abstraction layer; the electronic image anti-shake processing module is used for acquiring images and calculating to obtain distortion matrix data according to the acquired images; the hardware abstraction layer comprises a hardware distortion engine module, and the hardware distortion engine module is used for acquiring distortion matrix data and an image of the electronic image anti-shake processing module and carrying out anti-shake processing on the image according to the distortion matrix data. The software code of the image processing architecture, which is relied on when the electronic equipment processes the image, can be conveniently transplanted and maintained.
Description
Technical Field
The present application relates to the field of image technologies, and in particular, to an image processing architecture, an image processing method, a storage medium, and an electronic device.
Background
With the development of the technology, the image processing capability of the electronic device is stronger. Based on this, users often use electronic devices to take images, such as taking photos or recording videos. However, in the related art, the image processing architecture relied on when the electronic device performs image processing is not favorable for the migration and maintenance of software codes.
Disclosure of Invention
The embodiment of the application provides an image processing architecture, an image processing method, a storage medium and an electronic device, wherein software codes of the image processing architecture, which are depended by the electronic device when the electronic device processes images, can be conveniently transplanted and maintained.
In a first aspect, an embodiment of the present application provides an image processing architecture, where the image processing architecture includes an application layer and a hardware abstraction layer;
the application layer comprises an electronic image anti-shake processing module, wherein the electronic image anti-shake processing module is used for acquiring an image and calculating to obtain distorted matrix data according to the acquired image;
the hardware abstraction layer comprises a hardware distortion engine module, and the hardware distortion engine module is used for acquiring distortion matrix data of the electronic image anti-shake processing module and the image, and carrying out anti-shake processing on the image according to the distortion matrix data.
In a second aspect, an embodiment of the present application further provides an image processing method, where the image processing method is applied to the image processing architecture provided in the first aspect of the present application, and the image processing method includes:
acquiring an image;
the electronic image anti-shake processing module of the application layer calculates to obtain distorted matrix data according to the image;
and a hardware distortion engine module of the hardware abstraction layer acquires the distortion matrix data and the image, and carries out anti-shake processing on the image data according to the distortion matrix data.
In a third aspect, embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program runs on a computer, the computer is caused to execute an image processing method according to any embodiment of the present application.
In a fourth aspect, an embodiment of the present application further provides an electronic device, which includes a processor and a memory, where the memory has a computer program, and the processor is configured to execute the image processing method provided in any embodiment of the present application by calling the computer program.
In the embodiment of the application, the electronic image anti-shake processing module in the application layer can call the hardware distortion engine module in the hardware abstraction layer to perform anti-shake image processing, the software code of the application layer and the software code of the hardware abstraction layer are written respectively, if the software code of the application layer or the hardware abstraction layer is modified, two simultaneous modifications are not needed, and only the software code of the application layer or the hardware abstraction layer needs to be modified correspondingly, so that the anti-shake algorithm of the electronic image anti-shake processing module is decoupled from the hardware abstraction layer, and the transplantation and maintenance of the software code of the application layer or the hardware abstraction layer in an image processing framework are facilitated.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a first structural schematic diagram of an image processing architecture according to an embodiment of the present disclosure.
FIG. 2 is a schematic diagram of a second structure of an image processing architecture according to an embodiment of the present application
Fig. 3 is a schematic flowchart of a first image processing method according to an embodiment of the present application.
Fig. 4 is a schematic flowchart of a second image processing method according to an embodiment of the present application.
Fig. 5 is a schematic structural diagram of a first electronic device according to an embodiment of the present application.
Fig. 6 is a schematic structural diagram of a second electronic device according to an embodiment of the present application.
Detailed Description
Referring to the drawings, wherein like reference numbers refer to like elements, the principles of the present application are illustrated as being implemented in a suitable computing environment. The following description is based on illustrated embodiments of the application and should not be taken as limiting the application with respect to other embodiments that are not detailed herein.
With the development of the technology, the image processing capability of the electronic device is stronger. Based on this, users often use electronic devices to take images, such as taking photos or recording videos. However, in the related art, the image processing architecture relied on when the electronic device performs image processing is not favorable for the migration and maintenance of software codes.
Referring to fig. 1, fig. 1 is a first structural schematic diagram of an image processing architecture according to an embodiment of the present disclosure. As shown in fig. 1, the image processing architecture 100 provided by the embodiment of the present application may include an application layer 110 and a hardware abstraction layer 130.
The application layer 110 includes an Electronic Image Stabilization (EIS) processing module 113, where the Electronic Image Stabilization (EIS) processing module 113 is configured to acquire an Image and calculate distortion matrix data according to the acquired Image. The Hardware abstraction layer 130 includes a Hardware Warping Engine module 131 (WPE), and the Hardware Warping Engine module 131 is configured to acquire Warping matrix data of the electronic image anti-shake processing module 113 and the image, and perform anti-shake processing on the image according to the Warping matrix data.
The image processing architecture 100 may perform anti-shake processing on images captured by the camera. Specifically, the electronic image anti-shake processing module 113 of the application layer may obtain an image captured by the camera, then calculate the image according to an algorithm to obtain the distortion matrix data, then transmit the distortion matrix data and the obtained image to the hardware distortion engine module 131 of the hardware abstraction layer 130, where the hardware distortion engine module 131 performs anti-shake processing on the image according to the distortion matrix data to obtain an image after anti-shake processing, and outputs the image after anti-shake processing for further processing or displays the image after anti-shake processing to a user in the application layer 110, so as to provide the image after anti-shake processing for the user.
In this embodiment of the present application, the electronic image anti-shake processing module 113 of the application layer 110 may call the hardware warping engine module 131 of the hardware abstraction layer 130 to perform anti-shake image processing, and the software code of the application layer 110 and the software code of the hardware abstraction layer 130 may be written separately, and if the software code of the application layer 110 or the hardware abstraction layer 130 is modified, two simultaneous modifications are not required, and only the software code of the application layer 110 or the hardware abstraction layer 130 needs to be modified correspondingly, so that the anti-shake algorithm of the electronic image anti-shake processing module 113 is decoupled from the hardware abstraction layer 130, thereby facilitating the transplantation and maintenance of the software code of the application layer 110 or the hardware abstraction layer 130 in the image processing architecture 100.
It is understood that the hardware warping engine module 131 may be a system-on-chip integrated module, which may be used for electronic image anti-shake Processing, and the hardware warping engine module 131 has stronger performance and lower power consumption during image anti-shake Processing compared to a Graphics Processing Unit (GPU). For example, the WPE can save 23mA/s compared with the GPU for anti-shake processing, and the anti-shake processing effect of the WPE is better than that of the GPU. Therefore, in consideration of comprehensive performance and power consumption, the electronic image anti-shake algorithm of the image care architecture performs image anti-shake processing by means of the hardware distortion engine module.
Continuing to refer to fig. 1, the image processing architecture includes a hardware abstraction layer interface definition language interface, and the electronic image anti-shake processing module and the hardware warping engine module communicate data through the hardware abstraction layer interface definition language interface.
Because the electronic image anti-shake processing module and the hardware distortion engine module are respectively positioned on the application layer and the hardware abstraction layer, Inter-Process Communication (IPC) of the electronic image anti-shake processing module and the hardware distortion engine module can transmit data through a hardware abstraction layer interface definition language interface, thereby realizing stable and rapid data transmission.
It is understood that, in some embodiments, referring to fig. 2, electronic image anti-shake processing module 113 may also be located in hardware abstraction layer 130, and may call hardware warping engine module 131 in hardware abstraction layer 130 for image anti-shake processing through calling Application Programming Interface (API), but when electronic image anti-shake processing module 113 and hardware warping engine module 131 are both located in hardware abstraction layer 130, platform codes of electronic image anti-shake processing module 113 and hardware warping engine module 131 are heavily coupled, and workload is large when platform codes need to be transplanted and maintained.
Please refer to fig. 1, wherein the hardware abstraction layer 130 may further include a hardware Multimedia Data Path 133 (MDP), the hardware Multimedia Data Path 133 is configured to receive the image after the anti-shake processing by the hardware warping engine module 131, crop the image after the anti-shake processing to obtain a cropped image, return the cropped image to the hardware warping engine module 131, and the hardware warping engine module 131 returns the cropped image to the electronic image anti-shake processing module 113.
The hardware warping engine module 131 performs anti-shake processing such as deformation and warping on the image according to the acquired warping matrix data, and the hardware multimedia data path 133 cuts the image subjected to anti-shake processing by the hardware warping engine module 131 to obtain a cut image, and transmits the cut image to the hardware warping engine module 131. It can be understood that the hardware multimedia data path 133 may clip the image after the anti-shake processing such as deformation and distortion, for example, the edge of the image after the anti-shake processing such as deformation and distortion is clipped to obtain the processed image with better definition, the hardware multimedia data path 133 may return the processed image to the hardware warping engine module 131, and the hardware warping engine module 131 may transmit the processed image to the electronic image anti-shake processing module 113 through the hardware abstraction interface definition language interface, so that the processed image may be displayed to the user in the application layer through the application layer, and the image after the anti-shake processing may be provided to the user.
It should be noted that, when the user is shooting an image, the user can return two paths of images after cutting through the hardware multimedia data path 133, one path is a preview image, and the other path is a determined shot image; when the user shoots a video, the video can be cut by the hardware multimedia data path 133 and then returned to two paths, one path is a preview video, and the other path is a video after determined recording. The image after shooting or the video after recording is determined to be compared with the preview image or the preview video, and not only needs to be saved, but also needs to be processed. Such as adjusting contrast, night scene optimization, etc.
With reference to fig. 1, the application layer 110 may further include a spatial alignment change module 111, where the spatial alignment change module 111 is configured to receive multiple frames of images obtained by a camera, fuse the multiple frames of images into one image, and transmit the image to the electronic image anti-shake processing module 113.
It can be understood that the Spatial Alignment Transformation (SAT) module 111 may obtain a plurality of frames of images captured by the camera, and then fuse the plurality of frames of images captured by the camera to obtain an image with higher definition. The embodiment of the application is not limited to the number of the cameras, for example, fig. 1 shows 2 cameras, the spatial alignment change module 111 can fuse the multi-frame images shot by a plurality of cameras simultaneously, and also can fuse the multi-frame images shot by one camera, so that the images with higher definition are obtained, and the images with high definition can support the user to zoom the images. For example, after the image is enlarged, the definition of the image fused by the multi-frame images can still meet the requirement of the user, unlike the situation that after the image of a single frame is enlarged, the definition is seriously reduced due to insufficient pixels.
With continued reference to fig. 1, the application layer 110 may further be configured to obtain gyroscope data, and the electronic image anti-shake processing module 113 is configured to calculate the distortion matrix data according to the gyroscope data and the image.
The electronic image anti-shake processing module 113 may acquire gyroscope data and calculate distortion matrix data based on the gyroscope data and the acquired image. It is understood that after the electronic image anti-shake processing module 113 moves up to the application layer 110, the gyroscope data is directly obtained by the program of the application layer 110, and the application layer 110 may directly obtain the gyroscope data through the corresponding interface function. For example, the gyroscope data is acquired through a SensorManager, the sampling frequency can be set to be SensorManager, sensor _ DELAY _ GAME, and the gyroscope data closest to the Preview Image timemap is selected in a SensorEventListener callback function and used as the electronic Image anti-shake processing algorithm processing of the corresponding frame.
It should be noted that the image processing architecture in the embodiment of the present application may be applied to a system-level chip of each manufacturer, such as a Helio series chip of taiwan co-launch technologies ltd (MediaTek).
Referring to fig. 3, fig. 3 is a first flowchart illustrating an image processing method according to an embodiment of the present disclosure. The image processing method can be applied to the image processing architecture provided by the embodiment of the application.
For example, an image processing architecture applied by the image processing method provided by the embodiment of the present application may include an application layer and a hardware abstraction layer. The application layer comprises an electronic image anti-shake processing module, and the electronic image anti-shake processing module is used for acquiring images and calculating distortion matrix data according to the acquired images. The hardware abstraction layer comprises a hardware distortion engine module, and the hardware distortion engine module is used for acquiring distortion matrix data of the electronic image anti-shake processing module and the image, and carrying out anti-shake processing on the image according to the distortion matrix data.
It can be understood that the execution subject of the embodiment of the present application may be an electronic device such as a smart phone or a tablet computer having the image processing architecture. The flow of the image processing method provided by the embodiment of the application can include:
in 101, an image is acquired.
For example, the electronic device may obtain an image to be processed first, where the image to be processed may be an original image acquired by a camera of the electronic device or a processed image. In the embodiment of the present application, the data of the cameras is not limited, and may be 1 or more cameras.
At 102, the electronic image anti-shake processing module of the application layer calculates distortion matrix data according to the image.
The image processing architecture can perform anti-shake processing on the image to be processed. Specifically, the Electronic Image anti-shake processing module of the application layer may acquire an Image to be processed, and then calculate the Image according to an Electronic Image Stabilization (EIS) algorithm to obtain the distortion matrix data.
The electronic image anti-shake algorithm is anti-shake capability realized by providing a software algorithm, and the whole ISO, the shutter and the imaging algorithm are dynamically adjusted to perform fuzzy correction by detecting the shake amplitude of the electronic equipment through an acceleration sensor or a gyroscope.
In 103, a hardware warping engine module of the hardware abstraction layer obtains the warping matrix data and the image, and performs anti-shake processing on the image data according to the warping matrix data.
The electronic anti-shake processing module transmits the distortion matrix data and the acquired image to be processed to a hardware distortion engine module of the hardware abstraction layer, the hardware distortion engine module performs anti-shake processing on the image to be processed according to the distortion matrix data to obtain the image after the anti-shake processing, and further processes the image after the anti-shake processing or displays the image after the anti-shake processing to a user on an application layer to provide the image after the anti-shake processing for the user.
In the embodiment of the application, the electronic image anti-shake processing module of the application layer can call the hardware distortion engine module of the hardware abstraction layer to perform anti-shake image processing, the software code of the application layer and the software code of the hardware abstraction layer can be written respectively, if the software code of the application layer or the hardware abstraction layer is modified, two simultaneous modifications are not needed, and only the software code of the application layer or the hardware abstraction layer needs to be modified correspondingly, so that the anti-shake algorithm of the electronic image anti-shake processing module is decoupled from the hardware abstraction layer, the transplantation and maintenance of the software code of the application layer or the hardware abstraction layer in an image processing framework are facilitated, and the expansibility and the competitiveness of the image processing framework are enhanced.
Referring to fig. 4, fig. 4 is a first flowchart illustrating an image processing method according to an embodiment of the present disclosure. The image processing method can be applied to the image processing architecture provided by the embodiment of the application.
The method comprises the following steps:
in 201, the application layer includes a spatial alignment change module, where the spatial alignment change module receives a plurality of frames of images acquired by the camera, fuses the plurality of frames of images into one image, and transmits the image to the electronic image anti-shake processing module.
The spatial alignment change module can acquire multi-frame images shot by the camera, and then the multi-frame images shot by the camera are fused to obtain an image with higher definition. The multi-frame images shot by the cameras at the same time can be fused, and the multi-frame images shot by one camera can also be fused, so that the images with higher definition can be obtained, and the images with high definition can support the user to zoom the images. For example, after the image is enlarged, the definition of the image fused by the multi-frame images can still meet the requirement of the user, unlike the situation that after the image of a single frame is enlarged, the definition is seriously reduced due to insufficient pixels.
In 202, the application layer is used to obtain gyroscope data, and the electronic image anti-shake processing module of the application layer is used to calculate the distortion matrix data according to the gyroscope data and the image.
The application layer can acquire gyroscope data and transmit the gyroscope data to the electronic image anti-shaking processing module, the space alignment change module transmits the fused image to the electronic image anti-shaking processing module, and the electronic image anti-shaking processing module calculates the distorted matrix data based on the acquired gyroscope data and the image fused by the space alignment module.
Specifically, the Electronic Image anti-shake processing module may calculate the Image according to an Electronic Image Stabilization (EIS) algorithm to obtain the distortion matrix data.
The electronic image anti-shake algorithm is anti-shake capability realized by providing a software algorithm, and the whole ISO, the shutter and the imaging algorithm are dynamically adjusted to perform fuzzy correction by detecting the shake amplitude of the electronic equipment through a gyroscope or an acceleration sensor. It should be noted that the application layer may also acquire data required by other sensors, such as an acceleration sensor.
It can be understood that after the electronic Image anti-shake processing module is moved up to the application layer, the gyroscope data is directly obtained by the application layer program, for example, by using a SensorManager, the sampling frequency can be set to SensorManager.
At 203, a hardware warping engine module of the hardware abstraction layer obtains the warping matrix data and the image through a hardware abstraction interface definition language interface.
The image processing architecture comprises a hardware abstraction layer Interface Definition Language Interface, and an electronic image anti-shake processing module of an application layer and a hardware warping engine module of the hardware abstraction layer can transmit data through a hardware abstraction Interface Definition Language (HIDL). The electronic image anti-shake processing module can transmit the computed image obtained by fusing the distortion matrix data and the space alignment change module to the hardware distortion engine module through a hardware abstract interface definition language interface.
Because the electronic image anti-shake processing module and the hardware distortion engine module are respectively positioned on the application layer and the hardware abstraction layer, Inter-Process Communication (IPC) of the electronic image anti-shake processing module and the hardware distortion engine module can transmit data through a hardware abstraction layer interface definition language interface, thereby realizing stable and rapid data transmission.
At 204, the hardware warp engine module performs anti-shake processing on the image according to the warp matrix data.
The hardware distortion engine module receives the image and the distortion matrix data and carries out anti-shake processing such as deformation and distortion on the image according to the distortion matrix data, so that the problem of deformation caused by image shake is solved.
In 205, the hardware abstraction layer includes a hardware multimedia data path, where the hardware multimedia data path receives the image after the anti-shake processing by the hardware warping engine module, cuts the image after the anti-shake processing to obtain a cut image, and returns the cut image to the hardware warping engine module, and the hardware warping engine module returns the cut image to the electronic image anti-shake processing module.
And the hardware distortion engine module is used for carrying out anti-shake processing such as deformation and distortion on the image according to the acquired distortion matrix data, and the hardware multimedia data channel is used for cutting the image subjected to anti-shake processing by the hardware distortion engine module to obtain a cut image and transmitting the cut image to the hardware distortion engine module. It can be understood that the hardware multimedia data path can cut out the images after the anti-shake processing such as deformation and distortion, for example, cut off the edges of the images after the anti-shake processing such as deformation and distortion to obtain the processed images with better definition, the hardware multimedia data path can return the processed images to the hardware distortion engine module, and the hardware distortion engine module can transmit the processed images to the electronic image anti-shake processing module through the hardware abstraction interface definition language interface, so that the processed images can be displayed to the user through the application layer, and the images after the anti-shake processing can be provided for the user.
It should be noted that, when a user is shooting an image, the user can return two paths of images after cutting through the hardware multimedia data path, one path is a preview image, and the other path is a determined shot image; when a user shoots a video, the user can return two paths of videos after cutting through the hardware multimedia data path, wherein one path of the videos is a preview video, and the other path of the videos is a video after determined recording. The image after shooting or the video after recording is determined to be compared with the preview image or the preview video, and not only needs to be saved, but also needs to be processed. Such as adjusting contrast, night scene optimization, etc.
In the embodiment of the application, the electronic image anti-shake processing module of the application layer can call the hardware distortion engine module of the hardware abstraction layer to perform anti-shake image processing, the software code of the application layer and the software code of the hardware abstraction layer can be written respectively, if the software code of the application layer or the hardware abstraction layer is modified, two simultaneous modifications are not needed, and only the software code of the application layer or the hardware abstraction layer needs to be modified correspondingly, so that the anti-shake algorithm of the electronic image anti-shake processing module is decoupled from the hardware abstraction layer, the transplantation and maintenance of the software code of the application layer or the hardware abstraction layer in an image processing framework are facilitated, and the expansibility and the competitiveness of the image processing framework are enhanced.
The present application also provides a computer-readable storage medium, on which a computer program is stored, which, when executed on a computer, causes the computer to execute the flow in each image processing method provided by the present application.
The embodiment of the present application further provides an electronic device, which includes a memory and a processor, where the processor is configured to execute the processes in the image processing methods provided in the embodiments of the present application by calling the computer program stored in the memory. For example, the electronic device may be a mobile terminal such as a tablet computer or a smart phone. Referring to fig. 5, fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
The camera 401 may be used to acquire image data.
The processor 403 is a control center of the electronic device 400, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or calling a computer program stored in the memory 402 and calling data stored in the memory 402, thereby performing overall monitoring of the electronic device.
The sensors 404 are used to collect environmental information. The sensors 408 may include one or more of acceleration sensors, gyroscopes, and the like.
The electronic device provided by the embodiment of the application can be used for realizing an image processing architecture, wherein the image processing architecture comprises an application layer and a hardware abstraction layer;
the application layer comprises an electronic image anti-shake processing module, wherein the electronic image anti-shake processing module is used for acquiring an image and calculating to obtain distorted matrix data according to the acquired image;
the hardware abstraction layer comprises a hardware distortion engine module, and the hardware distortion engine module is used for acquiring distortion matrix data of the electronic image anti-shake processing module and the image, and carrying out anti-shake processing on the image according to the distortion matrix data.
In one embodiment, the image processing architecture includes a hardware abstraction layer interface definition language interface through which the electronic image anti-shake processing module and the hardware warp engine module communicate data.
In an embodiment, the hardware abstraction layer includes a hardware multimedia data path, the hardware multimedia data path is configured to receive an image after anti-shake processing by the hardware warping engine module, crop the image after anti-shake processing to obtain a cropped image, transmit the cropped image to the hardware warping engine module, and return the cropped image to the electronic image anti-shake processing module by the hardware warping engine module.
In one embodiment, the application layer includes a spatial alignment change module, and the spatial alignment change module is configured to receive multiple frames of images acquired by a camera, fuse the multiple frames of images into one image, and transmit the image to the electronic image anti-shake processing module.
In one embodiment, the application layer is configured to obtain gyroscope data, and the electronic image anti-shake processing module is configured to calculate the distortion matrix data according to the gyroscope data and the image.
Based on the image processing architecture, in the embodiment, the processor 403 in the electronic device 400 loads instructions corresponding to one or more processes of the computer program into the memory 402 according to the following steps, and the processor 403 executes the computer program stored in the memory 402, so as to implement various functions:
acquiring an image;
the electronic image anti-shake processing module of the application layer calculates to obtain distorted matrix data according to the image;
and a hardware distortion engine module of the hardware abstraction layer acquires the distortion matrix data and the image, and carries out anti-shake processing on the image data according to the distortion matrix data.
In some embodiments, please refer to fig. 6, and fig. 6 is a second structural diagram of an electronic device according to an embodiment of the present disclosure. The electronic device 400 further comprises: display 405, control circuit 406, audio circuit 407, power supply 408, and the like. The processor 404 is electrically connected to the control circuit 405, the display 406, the audio circuit 407, the power source 408, and the like.
The display screen 405 may be used to display information entered by or provided to a user as well as various graphical user interfaces of the electronic device, which may be comprised of images, text, icons, video, and any combination thereof.
The control circuit 406 is electrically connected to the display screen 405, and is configured to control the display screen 405 to display information.
The audio circuit 407 may provide an audio interface between the user and the electronic device through a speaker, microphone. Wherein the audio circuit 407 comprises a microphone. The microphone is electrically connected to the processor 404. The microphone is used for receiving voice information input by a user.
Although not shown in the drawings, the electronic device 400 may further include an input module, a bluetooth module, and the like, which are not described in detail herein.
In this embodiment, the processor 403 in the electronic device 400 loads instructions corresponding to one or more processes of the computer program into the memory 402, and the processor 403 executes the computer program stored in the memory 402 according to the following steps, so as to implement various functions:
acquiring an image;
the electronic image anti-shake processing module of the application layer calculates to obtain distorted matrix data according to the image;
and a hardware distortion engine module of the hardware abstraction layer acquires the distortion matrix data and the image, and carries out anti-shake processing on the image data according to the distortion matrix data.
In one embodiment, the application layer includes a spatial alignment change module, and in acquiring an image, the processor 403 executes the spatial alignment change module to receive a plurality of frames of images acquired by a camera, fuse the plurality of frames of images into one image, and transmit the image to the electronic image anti-shake processing module. In one embodiment, in the step of calculating the warp matrix data according to the image by the electronic image anti-shake processing module of the application layer, the processor 403 executes the application layer to obtain gyroscope data, and the electronic image anti-shake processing module of the application layer is used to calculate the warp matrix data according to the gyroscope data and the image.
In one embodiment, in the hardware warping engine module of the hardware abstraction layer acquiring the warping matrix data and the image, and performing image anti-shake processing on the image data according to the warping matrix data, the processor 403 executes the hardware warping engine module of the hardware abstraction layer to acquire the warping matrix data and the image through a hardware abstraction interface definition language interface;
the hardware distortion engine module carries out anti-shake processing on the image according to the distortion matrix data;
and the hardware distortion engine module is used for defining a language interface through the hardware abstract interface and transmitting the processed image to the electronic image anti-shake processing module. In an embodiment, the hardware abstraction layer includes a hardware multimedia data path, and after the hardware warping engine module performs anti-shake processing on the image according to the warping matrix data, the processor 403 executes the hardware multimedia data path to receive the image after the anti-shake processing by the hardware warping engine module, and cuts the image after the anti-shake processing to obtain a cut image, and returns the cut image to the hardware warping engine module, and the hardware warping engine module returns the cut image to the electronic image anti-shake processing module.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and parts that are not described in detail in a certain embodiment may refer to the above detailed description of the device control method, and are not described herein again.
The device control apparatus provided in the embodiment of the present application and the device control method in the above embodiments belong to the same concept, and any method provided in the device control method embodiment may be run on the device control apparatus, and a specific implementation process thereof is described in detail in the device control method embodiment, and is not described herein again.
It should be noted that, for the apparatus control method of the embodiment of the present application, it can be understood by those skilled in the art that all or part of the process for implementing the apparatus control method of the embodiment of the present application can be completed by controlling the relevant hardware through a computer program, the computer program can be stored in a computer readable storage medium, such as a memory, and executed by at least one processor, and during the execution process, the process of the embodiment of the apparatus control method can be included. The storage medium may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like.
In the device control apparatus according to the embodiment of the present application, each functional module may be integrated into one processing chip, each module may exist alone physically, or two or more modules may be integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium, such as a read-only memory, a magnetic or optical disk, or the like.
The above detailed descriptions of the device control method, apparatus, storage medium, and electronic device provided in the embodiments of the present application, and the specific examples applied herein are set forth to explain the principles and implementations of the present application, and the descriptions of the above embodiments are only used to help understand the method and core ideas of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. An image processing chip, characterized in that the image processing chip comprises an application layer and a hardware abstraction layer;
the application layer comprises an electronic image anti-shake processing module, wherein the electronic image anti-shake processing module is used for acquiring an image and calculating to obtain distorted matrix data according to the acquired image;
the hardware abstraction layer comprises a hardware distortion engine module, and the hardware distortion engine module is used for acquiring distortion matrix data of the electronic image anti-shake processing module and the image, and carrying out anti-shake processing on the image according to the distortion matrix data;
the hardware abstraction layer comprises a hardware multimedia data path, the hardware multimedia data path is used for receiving the image after the anti-shake processing of the hardware distortion engine module, cutting the image after the anti-shake processing to obtain a cut image, transmitting the cut image to the hardware distortion engine module, and the hardware distortion engine module returns the cut image to the electronic image anti-shake processing module.
2. The image processing chip of claim 1, wherein the image processing chip comprises a hardware abstraction layer interface definition language interface, the electronic image anti-shake processing module and the hardware warp engine module communicating data through the hardware abstraction layer interface definition language interface.
3. The image processing chip of claim 1, wherein the application layer further comprises a spatial alignment change module, and the spatial alignment change module is configured to receive a plurality of frames of images captured by a camera, fuse the plurality of frames of images into one image, and transmit the image to the electronic image anti-shake processing module.
4. The image processing chip of claim 1, wherein the application layer is configured to obtain gyroscope data, and the electronic image anti-shake processing module is configured to calculate the warp matrix data according to the gyroscope data and the image.
5. An image processing method applied to the image processing chip of claim 1, the image processing method comprising:
acquiring an image;
the electronic image anti-shake processing module of the application layer calculates to obtain distorted matrix data according to the image;
a hardware distortion engine module of the hardware abstraction layer acquires the distortion matrix data and the image, and anti-shake processing is carried out on the image data according to the distortion matrix data;
the hardware abstraction layer comprises a hardware multimedia data path, the hardware multimedia data path receives the image after the anti-shake processing of the hardware distortion engine module, cuts the image after the anti-shake processing to obtain a cut image, returns the cut image to the hardware distortion engine module, and returns the cut image to the electronic image anti-shake processing module.
6. The image processing method of claim 5, wherein the application layer comprises a spatial alignment variation module, and the acquiring the image comprises:
the space alignment change module receives multi-frame images acquired by the camera, fuses the multi-frame images into one image, and transmits the image to the electronic image anti-shake processing module.
7. The image processing method of claim 5, wherein the electronic image anti-shake processing module of the application layer calculating warp matrix data from the image comprises:
the application layer is used for obtaining gyroscope data, and the electronic image anti-shake processing module of the application layer is used for calculating the distortion matrix data according to the gyroscope data and the image.
8. The image processing method of claim 5, wherein the hardware warping engine module of the hardware abstraction layer acquires the warping matrix data and the image, and the image anti-shake processing on the image data according to the warping matrix data comprises:
a hardware warping engine module of the hardware abstraction layer acquires the warping matrix data and the image through a hardware abstraction interface definition language interface;
the hardware distortion engine module carries out anti-shake processing on the image according to the distortion matrix data;
and the hardware distortion engine module is used for defining a language interface through the hardware abstract interface and transmitting the processed image to the electronic image anti-shake processing module.
9. A computer-readable storage medium, on which a computer program is stored, which, when executed on a computer, causes the computer to carry out the method according to any one of claims 5 to 8.
10. An electronic device comprising a memory, a processor, wherein the processor executes the method of any one of claims 5 to 8 by invoking a computer program stored in the memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010987725.XA CN112153282B (en) | 2020-09-18 | 2020-09-18 | Image processing chip, method, storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010987725.XA CN112153282B (en) | 2020-09-18 | 2020-09-18 | Image processing chip, method, storage medium and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112153282A CN112153282A (en) | 2020-12-29 |
| CN112153282B true CN112153282B (en) | 2022-03-01 |
Family
ID=73892390
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010987725.XA Active CN112153282B (en) | 2020-09-18 | 2020-09-18 | Image processing chip, method, storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112153282B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116701288A (en) * | 2022-02-28 | 2023-09-05 | 荣耀终端有限公司 | Streaming media characteristic architecture, processing method, electronic device and readable storage medium |
| CN116320784B (en) * | 2022-10-27 | 2023-11-28 | 荣耀终端有限公司 | Image processing method and device |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111193866A (en) * | 2020-01-08 | 2020-05-22 | Oppo广东移动通信有限公司 | Image processing method, image processor, photographing device and electronic equipment |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9160912B2 (en) * | 2012-06-08 | 2015-10-13 | Apple Inc. | System and method for automatic image capture control in digital imaging |
| US10148880B2 (en) * | 2016-04-04 | 2018-12-04 | Microsoft Technology Licensing, Llc | Method and apparatus for video content stabilization |
| CN107395967A (en) * | 2017-07-20 | 2017-11-24 | 深圳市欧唯科技有限公司 | Image processing method and its system based on more exposure fusions with backtracking algorithm |
| US10462370B2 (en) * | 2017-10-03 | 2019-10-29 | Google Llc | Video stabilization |
| CN110177214B (en) * | 2019-06-28 | 2021-09-24 | Oppo广东移动通信有限公司 | Image processor, image processing method, photographing device and electronic equipment |
| CN110958399B (en) * | 2019-12-09 | 2021-06-29 | Oppo广东移动通信有限公司 | High dynamic range image HDR realization method and related product |
| CN111510630B (en) * | 2020-04-24 | 2021-09-28 | Oppo广东移动通信有限公司 | Image processing method, device and storage medium |
-
2020
- 2020-09-18 CN CN202010987725.XA patent/CN112153282B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111193866A (en) * | 2020-01-08 | 2020-05-22 | Oppo广东移动通信有限公司 | Image processing method, image processor, photographing device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112153282A (en) | 2020-12-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110493538B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| US8780258B2 (en) | Mobile terminal and method for generating an out-of-focus image | |
| CN105227837A (en) | A kind of image combining method and device | |
| CN104683697A (en) | Shooting parameter adjusting method and device | |
| CN112153282B (en) | Image processing chip, method, storage medium and electronic device | |
| US20220130023A1 (en) | Video denoising method and apparatus, terminal, and storage medium | |
| US20220256099A1 (en) | Method for processing video, terminal, and storage medium | |
| CN113411498B (en) | Image shooting method, mobile terminal and storage medium | |
| CN105407295B (en) | Mobile terminal filming apparatus and method | |
| US20200059597A1 (en) | Shooting method and terminal | |
| CN111314606B (en) | Photographing method and device, electronic equipment and storage medium | |
| US20200090309A1 (en) | Method and device for denoising processing, storage medium, and terminal | |
| CN114500821B (en) | Photographing method and device, terminal and storage medium | |
| CN104751488A (en) | Photographing method for moving track of moving object and terminal equipment | |
| CN106375602B (en) | Terminal photographic method and device | |
| CN115359105B (en) | Depth-of-field extended image generation method, device and storage medium | |
| CN113127101A (en) | Application program control method, device, equipment and medium | |
| CN111083235A (en) | Control method and system for shared input and output equipment | |
| CN116757963B (en) | Image processing method, electronic device, chip system and readable storage medium | |
| WO2024040990A1 (en) | Photographing method and electronic device | |
| CN117082295B (en) | Image stream processing method, device and storage medium | |
| CN115802144B (en) | Video shooting method and related equipment | |
| CN111479075B (en) | Photographing terminal and image processing method thereof | |
| CN116069221A (en) | Media content display method and device, electronic equipment and storage medium | |
| CN115696064A (en) | Image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |