CN110086967A - Image processing method, image processor, filming apparatus and electronic equipment - Google Patents
Image processing method, image processor, filming apparatus and electronic equipment Download PDFInfo
- Publication number
- CN110086967A CN110086967A CN201910285175.4A CN201910285175A CN110086967A CN 110086967 A CN110086967 A CN 110086967A CN 201910285175 A CN201910285175 A CN 201910285175A CN 110086967 A CN110086967 A CN 110086967A
- Authority
- CN
- China
- Prior art keywords
- module
- application program
- processing
- program module
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/617—Upgrading or updating of programs or applications for camera control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
Abstract
This application discloses a kind of image processing methods.Image processing method includes: application program module to camera service module request initial data；Camera service module upon receiving a request, sends the requests to hardware abstraction layer；Hardware abstraction layer is sent to application program module through camera service module upon receiving a request, by initial data；Application program module is sent to algorithm post-processing module after receiving initial data, by initial data；Initial data take pictures post-processing and will treated that result data is returned to application program module with algorithm post-processing module.The image processing method of the application embodiment realizes data distribution by algorithm post-processing module, the load of hardware abstraction layer can be reduced, so that system operation is smooth, it is not in Caton phenomenon, and do not need user and fall into a long wait, greatly improve experience of taking pictures.In addition, disclosed herein as well is a kind of image processor, filming apparatus and electronic equipments.
Description
Technical field
This application involves technical field of image processing, more specifically, are related to a kind of image processing method, image procossing
Device, filming apparatus and electronic equipment.
Background technique
With the development of electronic technology, the function of camera is more and more, and module also becomes increasingly complex.When user presses camera
Camera button after, need to wait for a period of time at interface of taking pictures to complete image procossing, in this course, user cannot execute
Other operations, can not exit camera, taking pictures, it is bad to experience.
Summary of the invention
The application embodiment provides a kind of image processing method, image processor, filming apparatus and electronic equipment.
The image processing method of the application embodiment includes: application program module to camera service module request original number
According to；The camera service module sends the request to hardware abstraction layer after receiving the request；The hardware is taken out
As module is after receiving the request, the initial data is sent to the application program mould through the camera service module
Block；The application program module is sent to algorithm post-processing module after receiving the initial data, by the initial data；
The initial data take pictures post-processing and will treated that result data is returned to institute with the algorithm post-processing module
State application program module.
The image processor of the application embodiment includes application program module, camera service module, hardware abstraction layer
With algorithm post-processing module.The application program module is used for the camera service module request initial data.The camera
Service module is used for after receiving the request, sends the request to the hardware abstraction layer.The hardware abstraction
Module is used for after receiving the request, and the initial data is sent to the application program through the camera service module
Module.The application program module is also used to after receiving the initial data, and the initial data is sent to the calculation
Method post-processing module.The algorithm post-processing module is used to take pictures to the initial data post-processing and will treated knot
Fruit data are returned to the application program module.
The filming apparatus of the application embodiment includes above-mentioned image processor and imaging sensor, described image sensing
Device is connect with described image processor.
The electronic equipment of the application embodiment includes above-mentioned filming apparatus and shell, the filming apparatus and the shell
Body combines.
After image processing method, image processor, filming apparatus and the electronic equipment of the application embodiment pass through algorithm
Processing modules implement data distribution, it is possible to reduce the load of hardware abstraction layer, so that system operation is smooth, it is not in Caton
Phenomenon, and the post-processing of taking pictures of algorithm post-processing module can be in running background, not needing user falls into a long wait, and greatly mentions
Experience of taking pictures is risen.
The additional aspect and advantage of presently filed embodiment will be set forth in part in the description, partially will be from following
Description in become obvious, or recognized by the practice of presently filed embodiment.
Detailed description of the invention
The above-mentioned and/or additional aspect and advantage of the application is from combining in description of the following accompanying drawings to embodiment by change
It obtains obviously and is readily appreciated that, in which:
Fig. 1 is the module diagram of the image processor of the application certain embodiments；
Fig. 2 is the flow diagram of the image processing method of the application certain embodiments；
Fig. 3 is the flow diagram of the image processing method of the application certain embodiments；
Fig. 4 is the flow diagram of the image processing method of the application certain embodiments；
Fig. 5 is the flow diagram of the image processing method of the application certain embodiments；
Fig. 6 is the flow diagram of the image processing method of the application certain embodiments；
Fig. 7 is the flow diagram of the image processing method of the application certain embodiments；
Fig. 8 is the module diagram of the filming apparatus of the application certain embodiments；
Fig. 9 is the schematic diagram of the algorithm post-processing module of the application certain embodiments；
Figure 10 is the structural schematic diagram of the electronic equipment of the application certain embodiments；
Figure 11 is the structural schematic diagram of the electronic equipment of the application certain embodiments.
Specific embodiment
Presently filed embodiment is described below in detail, the example of embodiment is shown in the accompanying drawings, same or similar
Label indicates same or similar element or element with the same or similar functions from beginning to end.It is retouched below with reference to attached drawing
The embodiment stated is exemplary, and is only used for explaining the application, and should not be understood as the limitation to the application.
Fig. 1 and Fig. 2 are please referred to, the image processing method of the application embodiment includes:
01: application program module 14 requests initial data to camera service module 18；
02: camera service module 18 upon receiving a request, sends the requests to hardware abstraction layer 12；
03: hardware abstraction layer 12 is sent to application through camera service module 18 upon receiving a request, by initial data
Program module 14；
04: application program module 14 is sent to algorithm post-processing module 16 after receiving initial data, by initial data；
With
05: algorithm post-processing module 16 take pictures post-processing and will treated that result data is returned to initial data
Application program module 14.
Referring to Fig. 1, the image processor 10 of the application embodiment includes application program module (APP) 14, camera clothes
Business module (Camera Service) 18, hardware abstraction layer (Hardware Abstraction Layer, HAL) 12 and algorithm
Post-processing module (Algo Process Service, APS) 16.The image processing method of the application embodiment can be by the application
The image processor 10 of embodiment is realized.For example, application program module 14 can be used for executing the method in 01 and 04, camera clothes
Business module 18 can be used for executing the method in 02, and hardware abstraction layer 12 can be used for executing the method in 03, and algorithm post-processes mould
Block 16 can be used for executing the method in 05.
In other words, application program module 14 can be used for requesting initial data to camera service module 18.Camera service
Module 18 can be used for upon receiving a request, sending the requests to hardware abstraction layer 12.Hardware abstraction layer 12 can be used
In upon receiving a request, initial data is sent to application program module 14 through camera service module 18.Application program module
14 can be also used for after receiving initial data, and initial data is sent to algorithm post-processing module 16.Algorithm post-processes mould
Block 16 can be used for carrying out initial data taking pictures post-processing and will treated that result data is returned to application program module 14.
The image processing method and image processor 10 of the application embodiment realize number by algorithm post-processing module 16
According to shunting, it is possible to reduce the load of hardware abstraction layer 12, so that system operation is smooth, it is not in Caton phenomenon, and algorithm
The post-processing of taking pictures of post-processing module 16 can not need user and fall into a long wait, greatly improve and take pictures in running background
Experience.
Specifically, application program module 14 requests initial data to camera service module 18, and initial data is image data,
Initial data can be the data of the data either yuv format of RAW format；Camera service module 18 upon receiving a request,
Send the requests to hardware abstraction layer 12；Hardware abstraction layer 12 upon receiving a request, by initial data through camera service
Module 18 is sent to application program module 14, i.e. initial data is first sent to camera service module 18 by hardware abstraction layer 12,
Initial data is sent to application program module 14 again by camera service module 18；Application program module 14 is receiving initial data
Afterwards, initial data is sent to algorithm post-processing module 16, at least one image procossing is stored in algorithm post-processing module 16
Algorithm, image processing algorithm include multi-frame processing algorithm, U.S. face Processing Algorithm, virtualization Processing Algorithm, high dynamic range images
(High-Dynamic Range, HDR) Processing Algorithm, filter Processing Algorithm, rotation processing algorithm, watermark processing algorithm etc.；It calculates
Method post-processing module 16 take pictures post-processing and will treated that result data is returned to application program module to initial data
14, it may be assumed that algorithm post-processing module 16 is using the data of above-mentioned image processing algorithm processing RAW format or the number of yuv format
According to, and result data is returned to application program module 14 by treated.
Fig. 1 and Fig. 3 are please referred to, in some embodiments, application program module 14 is requested to camera service module 18
Initial data is multiframe.
Hardware abstraction layer 12 is sent to application program through camera service module 18 upon receiving a request, by initial data
Module 14 (i.e. 03), comprising:
032: hardware abstraction layer 12 upon receiving a request, is successively sent out multiframe initial data through camera service module 18
It send to application program module 14；
Application program module 14 is sent to algorithm post-processing module 16 (i.e. after receiving initial data, by initial data
04), comprising:
042: application program module 14 transmits multiframe initial data to algorithm after receiving multiframe initial data
Post-processing module 16.
Referring to Fig. 1, in some embodiments, the original number that application program module 14 is requested to camera service module 18
According to for multiframe.Hardware abstraction layer 12 can be used for executing the method in 032, and application program module 14 can be used for executing in 042
Method.In other words, hardware abstraction layer 12 can be used for upon receiving a request, successively take multiframe initial data through camera
Business module 18 is sent to application program module 14.Application program module 14 can be used for after receiving multiframe initial data, will
Multiframe initial data is transmitted to algorithm post-processing module 16.
For example, the initial data that application program module 14 is requested to camera service module 18 is 4 frames, camera service module 18
Upon receiving a request, hardware abstraction layer 12 is sent the requests to；Hardware abstraction layer 12 upon receiving a request, firstly,
1st frame initial data is sent to camera service module 18 by hardware abstraction layer 12, and camera service module 18 is by the 1st frame original number
According to being sent to application program module 14；Secondly, the 2nd frame initial data is sent to camera service module by hardware abstraction layer 12
18, the 2nd frame initial data is sent to application program module 14 by camera service module 18；Then, hardware abstraction layer 12 is by the 3rd
Frame initial data is sent to camera service module 18, and the 3rd frame initial data is sent to application program mould by camera service module 18
Block 14；Finally, the 4th frame initial data is sent to camera service module 18 by hardware abstraction layer 12, camera service module 18 will
4th frame initial data is sent to application program module 14.In this course, hardware abstraction layer 12 is by former frame initial data
After being sent to camera service module 18, it can start next frame initial data being sent to camera service module 18, without etc.
Former frame initial data is sent to application program module 14 to camera service module 18, then starts to send out next frame initial data
It send to camera service module 18.It can save in this way and multiframe initial data is sent to application program mould through camera service module 18
Total duration needed for block 14, avoids user from falling into a long wait, and is conducive to promote experience of taking pictures.
Application program module 14 is receiving the 1st frame initial data, the 2nd frame initial data, the 3rd frame initial data and the 4th
After frame initial data, the 1st frame initial data, the 2nd frame initial data, the 3rd frame initial data and the 4th frame initial data are packaged and sent out
It send to algorithm post-processing module 16.It is appreciated that working as the initial data that application program module 14 is requested to camera service module 18
When for multiframe, algorithm post-processing module 16 is usually to need to carry out post-processing of taking pictures to multiframe initial data, and if using journey
Multiframe initial data is successively sent to algorithm post-processing module 16 by sequence module 14, and algorithm post-processing module 16 is only receiving
1 frame initial data perhaps only receives the 1st frame initial data and the 2nd frame initial data or only receives the 1st frame original number
When according to, the 2nd frame initial data and the 3rd frame initial data, algorithm post-processing module 16 can not be completed to multiframe initial data
It takes pictures post-processing.In the application embodiment, application program module 14 transmits multiframe initial data to algorithm
Module 16 is managed, transfer resource can be efficiently utilized, transmit multiframe initial data to algorithm in application program module 14
After post-processing module 16, algorithm post-processing module 16 can carry out post-processing of taking pictures to multiframe initial data, and will be after processing
Result data be returned to the application program module 14.
Certainly, in other embodiments, the function for being packaged multiframe initial data also may be implemented in camera service module 18
Energy.In other words, multiframe initial data upon receiving a request, is successively sent to camera service module by hardware abstraction layer 12
18；Camera service module 18 transmits multiframe initial data to application program module 14；Application program module 14 is receiving
To after packed multiframe initial data, packed multiframe initial data is sent to algorithm post-processing module 16.
Referring to Fig. 1, in some embodiments, carrying out post-processing of taking pictures to initial data in algorithm post-processing module 16
During, application interface can be closed or be exited to application program module 14.
Specifically, during algorithm post-processing module 16 take pictures to initial data post-processing, user can be selected
Application interface is closed or exited to the control application program module 14 of selecting property.In other words, when there is still a need for use using journey by user
When the application interface of sequence module 14, user can also stay open the application interface of application program module 14, and user does not need
Using application program module 14 application interface when, since algorithm post-processing module 16 carries out post-processing of taking pictures to initial data
Process is not related to the work of application program module 14, and application program module 14 can be closed or be exited according to the control of user and answer
Use interface.User is not necessarily to fall into a long wait in the application interface of application program module 14, leads to not execute other operations.
Please refer to Fig. 1 and Fig. 4, in some embodiments, image processing method further include:
06: camera service module 18 sends the frame number suggestion about initial data to application program module 14；
Application program module 14 requests initial data (i.e. 01) to camera service module 18, comprising:
012: application program module 14 requests initial data to camera service module 18 according to frame number suggestion.
Referring to Fig. 1, in some embodiments, camera service module 18 can be used for executing the method in 06, using journey
Sequence module 14 can be used for executing the method in 012.In other words, camera service module 18 can be also used for application program module
14 send the frame number suggestion about initial data.Application program module 14 can be also used for according to frame number suggestion to camera service mould
Block 18 requests initial data.
Specifically, frame number suggestion indicates the specific value, such as 1 frame, 2 frames, 3 frames, 4 frames etc. of frame number.If camera service mould
The frame number suggestion about initial data that block 18 is sent to application program module 14 is 4 frames, then application program module 14 is to camera
Service module 18 requests 4 frame initial data；If camera service module 18 sent to application program module 14 about initial data
Frame number suggestion be 5 frames, then application program module 14 requests 5 frame initial data to camera service module 18.
The available sensitivity of hardware abstraction layer 12 (ISO value) information, the jitter conditions of gyroscope, AR scene detection knot
Fruit (detection scene type, such as personage, animal, landscape etc.) etc., these information are sent to camera and taken by hardware abstraction layer 12
Business module 18, camera service module 18 send to application program module 14 according to these information and build about the frame number of initial data
View.For example, the frame number that camera service module 18 is suggested to application program module 14 can be relatively more when shaking larger,
Post-processing of taking pictures is better achieved；When there's almost no shake, camera service module 18 is suggested to application program module 14
Frame number can be relatively smaller, to reduce volume of transmitted data.In other words, camera service module 18 is to application program module 14
It is recommended that frame number and the degree of jitter of gyroscope be positively correlated.Certainly, hardware abstraction layer 12 can also be directly according to these letters
Breath provides the frame number suggestion about initial data, and camera service module 18 is forwarded to application program module 14 about initial data again
Frame number suggestion.
Please refer to Fig. 1 and Fig. 5, in some embodiments, image processing method further include:
07: camera service module 18 sends to application program module 14 and builds to the take pictures algorithm of post-processing of initial data
View；
08: application program module 14 is arranged pre-defined algorithm according to algorithm suggestion and pre-defined algorithm is sent to hardware abstraction mould
Block 12；
Algorithm post-processing module 16 take pictures post-processing and will treated that result data is returned to answers to initial data
With program module 14 (i.e. 05), comprising:
052: algorithm post-processing module 16 carries out post-processing of taking pictures to initial data according to pre-defined algorithm, and by treated
Result data is returned to application program module 14.
Referring to Fig. 1, in some embodiments, camera service module 18 can be used for executing the method in 07, using journey
Sequence module 14 can be used for executing the method in 08, and algorithm post-processing module 16 can be used for executing the method in 052.In other words,
Camera service module 18, which can be also used for sending to application program module 14, builds the take pictures algorithm of post-processing of initial data
View.Application program module 14 can be also used for that pre-defined algorithm is arranged according to algorithm suggestion and pre-defined algorithm be sent to hardware abstraction
Module 12.Algorithm post-processing module 16 can be also used for carrying out post-processing of taking pictures to initial data according to pre-defined algorithm, and will place
Result data after reason is returned to application program module 14.
Specifically, algorithm suggestion and pre-defined algorithm may each be above-mentioned multi-frame processing algorithm, U.S. face Processing Algorithm, at virtualization
It is adjustment method, HDR Processing Algorithm, filter Processing Algorithm, rotation processing algorithm, one or more in watermark processing algorithm.For example, if
What camera service module 18 was sent to application program module 14 is multiframe to the take pictures algorithm suggestion of post-processing of initial data
Processing Algorithm, then multi-frame processing algorithm is arranged according to algorithm suggestion in application program module 14, and multi-frame processing algorithm is sent to
Hardware abstraction layer 12, algorithm post-processing module 16 carry out post-processing of taking pictures to initial data according to multi-frame processing algorithm, and will
Treated, and result data is returned to application program module 14；If pair that camera service module 18 is sent to application program module 14
The take pictures algorithm suggestion of post-processing of initial data is U.S. face Processing Algorithm and HDR Processing Algorithm, then application program module 14
U.S. face Processing Algorithm and HDR Processing Algorithm are set according to algorithm suggestion, and U.S. face Processing Algorithm and HDR Processing Algorithm are sent to
Hardware abstraction layer 12, algorithm post-processing module 16 clap initial data according to U.S. face Processing Algorithm and HDR Processing Algorithm
According to post-processing, and by treated, result data is returned to application program module 14.
The available sensitivity of hardware abstraction layer 12 (ISO value) information, the jitter conditions of gyroscope, AR scene detection knot
Fruit (detection scene type, such as personage, animal, landscape etc.) etc., these information are sent to camera and taken by hardware abstraction layer 12
Business module 18, after camera service module 18 takes pictures to initial data to the transmission of application program module 14 according to these information
The algorithm suggestion of reason.For example, camera service module 18 suggests multi-frame processing algorithm when the shake that gyroscope detects is larger,
It is shaken with being eliminated according to multi-frame processing；When the scene type of AR scene detection results detection is personage, camera service module 18
It is recommended that U.S. face Processing Algorithm, to carry out U.S. face to personage；When the scene type of AR scene detection results detection is landscape, camera
Service module 18 suggests HDR Processing Algorithm, to form the landscape image of high dynamic range.Certainly, hardware abstraction layer 12 can also
Initial data take pictures the algorithm suggestion of post-processing with directly being provided according to these information, camera service module 18 is again to answering
Initial data take pictures the algorithm suggestion of post-processing with the forwarding of program module 14.
In some embodiments, image processing method further include: camera service module 18 is together to application program module
14 send about the frame number suggestion of initial data and take pictures the algorithm suggestion of post-processing to initial data；Application program module
14 request initial data to camera service module 18 according to frame number suggestion；Application program module 14 is arranged predetermined according to algorithm suggestion
Pre-defined algorithm is simultaneously sent to hardware abstraction layer 12 by algorithm；Algorithm post-processing module 16 according to pre-defined algorithm to initial data into
Capable post-processing of taking pictures, and result data is returned to application program module 14 by treated.
Fig. 1 and Fig. 6 are please referred to, in some embodiments, initial data includes pending data and metadata.Using journey
Sequence module 14 requests initial data (i.e. 01) to camera service module 18, comprising:
014: application program module 14 requests pending data to camera service module 18；
Hardware abstraction layer 12 is sent to application program through camera service module 18 upon receiving a request, by initial data
Module 14 (i.e. 03), comprising:
034: hardware abstraction layer 12 upon receiving a request, by pending data and first number corresponding with pending data
Application program module 14 is sent to according to through camera service module 18；
Application program module 14 is sent to algorithm post-processing module 16 (i.e. after receiving initial data, by initial data
04), comprising:
044: application program module 14 sends out pending data and metadata after receiving pending data and metadata
It send to algorithm post-processing module 16；
Algorithm post-processing module 16 take pictures post-processing and will treated that result data is returned to answers to initial data
With program module 14 (i.e. 05), comprising:
054: algorithm post-processing module 16 take pictures post-processing and will treated to pending data according to metadata
Result data is returned to application program module 14.
Referring to Fig. 1, in some embodiments, initial data includes pending data and metadata.Application program mould
Block 14 can be used for executing the method in 014, and hardware abstraction layer 12 can be used for executing the method in 034, application program module 14
It can be used for executing the method in 044, algorithm post-processing module 16 can be used for executing the method in 054.
In other words, application program module 14 can be used for requesting pending data to camera service module 18.Hardware is taken out
As module 12 can be used for upon receiving a request, pending data and metadata corresponding with pending data are taken through camera
Business module 18 is sent to application program module 14.Application program module 14 can be used for receiving pending data and metadata
Afterwards, pending data and metadata are sent to algorithm post-processing module 16.Algorithm post-processing module 16 can be used for according to member
Data take pictures post-processing and will treated that result data is returned to application program module 14 to pending data.
Specifically, metadata (metadata) is corresponding with pending data.Metadata includes the related letter of pending data
Breath, such as 3a (auto-exposure control AE, auto focus control AF, automatic white balance control AWB) information, pictorial information (picture
Width and height), exposure parameter (aperture size, shutter speed and sensitivity f-number) etc..Algorithm post-processing module 16 can
To realize the post-processing of taking pictures to pending data according to metadata.For example, algorithm post-processing module 16 is according in metadata
ISO value adjusts corresponding brightness of image of pending data etc..
When initial data includes pending data and metadata, application program module 14 is requested to camera service module 18
Pending data；Camera service module 18 upon receiving a request, sends the requests to hardware abstraction layer 12；Hardware abstraction mould
Block 12 is sent through camera service module 18 upon receiving a request, by pending data and metadata corresponding with pending data
To application program module 14；Application program module 14 is after receiving pending data and metadata, by pending data and member
Data are sent to algorithm post-processing module 16；After algorithm post-processing module 16 takes pictures to pending data according to metadata
It manages and incites somebody to action that treated result data is returned to application program module 14.
Further, when the initial data that application program module 14 is requested to camera service module 18 is multiframe, camera
Service module 18 upon receiving a request, sends the requests to hardware abstraction layer 12；Hardware abstraction layer 12 is asked receiving
After asking, successively by multiframe pending data and respectively multiframe metadata corresponding with multiframe pending data through camera service module
18 are sent to application program module 14；Application program module 14, will after receiving multiframe pending data and multiframe metadata
Multiframe pending data and multiframe metadata are transmitted to algorithm post-processing module 16；Algorithm post-processing module 16 is according to multiframe
Metadata take pictures post-processing and will treated that result data is returned to application program module 14 to multiframe pending data.
For example, application program module 14 requests 4 frame pending datas to camera service module 18, camera service module 18 exists
After receiving request, hardware abstraction layer 12 is sent the requests to；Hardware abstraction layer 12 upon receiving a request, firstly, firmly
Part abstract module 12 is by the 1st frame pending data (i.e. pending data 1 in Fig. 1) and the 1st frame metadata (i.e. member in Fig. 1
Data 1) it is sent to camera service module 18, the 1st frame pending data and the 1st frame metadata are sent to by camera service module 18
Application program module 14；Secondly, hardware abstraction layer 12 is by the 2nd frame pending data (i.e. pending data 2 in Fig. 1) and
2 frame metadata (i.e. metadata 2 in Fig. 1) are sent to camera service module 18, and camera service module 18 is by the 2nd frame number to be processed
Application program module 14 is sent to according to the 2nd frame metadata；Then, hardware abstraction layer 12 (schemes the 3rd frame pending data
Pending data 3 in 1) and the 3rd frame metadata (i.e. metadata 3 in Fig. 1) be sent to camera service module 18, camera service
3rd frame pending data and the 3rd frame metadata are sent to application program module 14 by module 18；Finally, hardware abstraction layer 12
4th frame pending data (i.e. pending data 4 in Fig. 1) and the 4th frame metadata (i.e. metadata 4 in Fig. 1) are sent to
4th frame pending data and the 4th frame metadata are sent to application program module by camera service module 18, camera service module 18
14.In this course, former frame pending data and metadata are sent to camera service module 18 by hardware abstraction layer 12
Afterwards, can start next frame pending data and metadata being sent to camera service module 18, without waiting camera service
Former frame pending data and metadata are sent to application program module 14 by module 18, then are started next frame pending data
Camera service module 18 is sent to metadata.It can be saved in this way by multiframe pending data and metadata through camera service mould
Total duration needed for block 18 is sent to application program module 14, avoids user from falling into a long wait, and is conducive to promote experience of taking pictures.
Application program module 14 receive the 1st frame pending data and the 1st frame metadata, the 2nd frame pending data and
After 2nd frame metadata, the 3rd frame pending data and the 3rd frame metadata, the 4th frame pending data and the 4th frame metadata, by the 1st
Frame pending data and the 1st frame metadata, the 2nd frame pending data and the 2nd frame metadata, the 3rd frame pending data and the 3rd frame
Metadata, the 4th frame pending data and the 4th frame metadata are transmitted to algorithm post-processing module 16.It is appreciated that when applying
When the pending data that program module 14 is requested to camera service module 18 is multiframe, algorithm post-processing module 16 is usually to need
Post-processing of taking pictures is carried out to multiframe pending data, and if application program module 14 successively by multiframe pending data and correspondence
Metadata be sent to algorithm post-processing module 16, algorithm post-processing module 16 is only receiving the 1st frame pending data and the 1st
Frame metadata, or only receive the 1st frame pending data and the 1st frame metadata, the 2nd frame pending data and the 2nd frame member number
According to, or only receive the 1st frame pending data and the 1st frame metadata, the 2nd frame pending data and the 2nd frame metadata, the 3rd
When frame pending data and the 3rd frame metadata, algorithm post-processing module 16 can not be completed to take pictures to multiframe pending data
Post-processing, in the application embodiment, multiframe pending data and corresponding metadata are packaged hair by application program module 14
It send to algorithm post-processing module 16, can efficiently utilize transfer resource, in application program module 14 by multiframe pending data
It transmits with corresponding metadata to algorithm post-processing module 16, algorithm post-processing module 16 can be according to multiframe member number
Post-processing of taking pictures is carried out according to multiframe pending data, and result data is returned to the application program module by treated
14。
Fig. 1 and Fig. 7 are please referred to, in some embodiments, the data in hardware abstraction layer 12 are the first picture format.
Wherein: the initial data that application program module 14 is requested to camera service module 18 is also the data of the first picture format.Alternatively,
The initial data that application program module 14 is requested to camera service module 18 is the data of the second picture format, image processing method
Further include:
09: the data of the first picture format are converted to the data of the second picture format by hardware abstraction layer 12.
Referring to Fig. 1, in some embodiments, the data in hardware abstraction layer 12 are the first picture format.Wherein:
The initial data that application program module 14 is requested to camera service module 18 is also the data of the first picture format.Alternatively, using
The initial data that program module 14 is requested to camera service module 18 is the data of the second picture format.Hardware abstraction layer 12 can
For executing the method in 09.In other words, hardware abstraction layer 12 can be also used for the data conversion of the first picture format
For the data of the second picture format.
Specifically, the first picture format can be RAW format, and the second picture format can be yuv format.Hardware abstraction mould
Data in block 12 are the first picture format, i.e. RAW format.
The first situation, the initial data that application program module 14 is requested to camera service module 18 are also RAW format.This
When, raw data format phase that the data in hardware abstraction layer 12 are requested with from application program module 14 to camera service module 18
Together.Hardware abstraction layer 12 is without formatting the data of RAW format.
When application program module 14 requests the initial data of multiframe RAW format to camera service module 18, camera service
Module 18 upon receiving a request, sends the requests to hardware abstraction layer 12；Hardware abstraction layer 12 upon receiving a request,
The data of multiframe RAW format are successively sent to application program module 14 through camera service module 18；Application program module 14 exists
After the data for receiving multiframe RAW format, the data of multiframe RAW format are transmitted to algorithm post-processing module 16；Algorithm
Post-processing module 16 take pictures post-processing and will treated that result data is returned to using journey to the data of multiframe RAW format
Sequence module 14.
When initial data includes pending data and metadata, application program module 14 is requested to camera service module 18
The data of RAW format；Camera service module 18 upon receiving a request, sends the requests to hardware abstraction layer 12；Hardware is taken out
Upon receiving a request as module 12, the data of RAW format and corresponding metadata are sent to through camera service module 18 and are answered
With program module 14；Application program module 14 is after the data and corresponding metadata for receiving RAW format, by RAW format
Data and corresponding metadata are sent to algorithm post-processing module 16；Algorithm post-processing module 16 is according to metadata to RAW format
Data take pictures post-processing and will treated that result data is returned to application program module 14.
Second situation, the initial data that application program module 14 is requested to camera service module 18 are yuv format.This
When, the raw data format that data and application program module 14 in hardware abstraction layer 12 are requested to camera service module 18 is not
Together.Hardware abstraction layer 12 needs the data by RAW format to be converted to the data of yuv format.
When application program module 14 requests the initial data of multiframe yuv format to camera service module 18, camera service
Module 18 upon receiving a request, sends the requests to hardware abstraction layer 12；Hardware abstraction layer 12 upon receiving a request,
The data of multiframe RAW format are first converted to the data of multiframe yuv format, then successively by the data of multiframe yuv format through camera
Service module 18 is sent to application program module 14；Application program module 14, will be more after the data for receiving multiframe yuv format
The data of frame yuv format are transmitted to algorithm post-processing module 16；Data of the algorithm post-processing module 16 to multiframe yuv format
Take pictures post-processing and will treated that result data is returned to application program module 14.
When initial data includes pending data and metadata, application program module 14 is requested to camera service module 18
The data of yuv format；Camera service module 18 upon receiving a request, sends the requests to hardware abstraction layer 12；Hardware is taken out
Upon receiving a request as module 12, the data of RAW format are first converted to the data of yuv format, then by the data of yuv format
Application program module 14 is sent to through camera service module 18 with corresponding metadata；Application program module 14 is receiving YUV
After the data of format and corresponding metadata, the data of yuv format and corresponding metadata are sent to algorithm post-processing module
16；Algorithm post-processing module 16 take pictures post-processing and will treated number of results according to data of the metadata to yuv format
According to being returned to application program module 14.
Fig. 1 and Fig. 8 are please referred to, in some embodiments, the data of the first picture format in hardware abstraction layer 12
Acquisition is directly transmitted by imaging sensor 20.Alternatively, the data of the first picture format stored in hardware abstraction layer 12 are logical
It crosses to be taken pictures to post-process and return through application program module 14 by the processing realization of algorithm post-processing module 16 and obtain.
Referring to Fig. 8, the filming apparatus 100 of the application embodiment includes the image processor of any of the above-described embodiment
10 and imaging sensor 20.Image processor 10 and imaging sensor 20 connect.Imaging sensor 20 includes image acquisition units
(sensor) 22 and RAW image data cell (Image Front-end, IFE) 24, image acquisition units 22 are for receiving light
Image data (RAW image, i.e., the data of aforementioned first picture format) are obtained with acquisition, RAW image data cell 24 is used for will
The image data that image acquisition units 22 acquire is transmitted to image processor 10, wherein RAW image data cell 24 can be to figure
As RAW image that the acquisition of acquisition unit 22 obtains is handled and the RAW image that exports that treated is to image processor 10.
Image processor 10 includes hardware abstraction layer 12, application program module 14, algorithm post-processing module 16 and camera
Service module 18.
Hardware abstraction layer 12 is for receiving RAW image, RAW image being converted to YUV image (i.e. aforementioned second image pane
The data of formula) and transmission RAW image and/or YUV image.Hardware abstraction layer 12 can be connect with imaging sensor 20.Specifically
Ground, hardware abstraction layer 12 may include the cache unit (buffer queue) 122 connecting with imaging sensor 20, RAW turns
After RGB processing unit (Bayer Process Segment, BPS) 124 and the noise reduction and YUV that are connect with application program module 14
Processing unit (Image Process Engine, IPE) 126.Cache unit 122 is for caching from imaging sensor 20
RAW image is simultaneously transferred to algorithm post-processing module 16 by camera service module 18, application program module 14.RAW turns RGB processing
Unit 124 is used to the RAW image from cache unit 122 being converted to RGB image.Noise reduction and YUV post-processing unit 126 are used for
Processing RGB image obtains YUV image and YUV image is transferred to algorithm by camera service module 18, application program module 14
Post-processing module 16.Hardware abstraction layer 12 can also transmit the metadata of image data, and metadata includes 3a (auto-exposure control
AE, auto focus control AF, automatic white balance control AWB) information, pictorial information (such as picture traverse, height), exposure parameter
(aperture size, shutter speed and sensitivity f-number) etc. can use metadata auxiliary and realize and scheme to RAW image and/or YUV
Picture post-processing of taking pictures (for example including the processing of U.S. face, filter processing, rotation processing, watermark processing, virtualization processing, HDR processing,
And at least one of multi-frame processing).In one embodiment, metadata includes sensitivity (ISO) information, is believed according to sensitivity
Breath can be with the brightness of auxiliary adjustment RAW image and/or YUV image, to realize post-processing of taking pictures relevant to brightness is adjusted.
YUV image and transmission RAW image are converted to due to 12 reception RAW images of hardware abstraction layer, by RAW image
And/or YUV image, without carrying out post-processing of taking pictures to RAW image and/or YUV image, the image processing algorithm for post-processing of taking pictures
Without doing process truncation on the algorithm framework of hardware abstraction layer 12 itself, only compatibility need to be done in outside, design difficulty is significantly
Reduce.
In the related art, hardware abstraction layer is created as the side of pipeline (pipeline) by application programming interfaces (API)
Formula is needed in camera starting by the corresponding Working mould of camera since the creation of pipeline needs a large amount of time and memory
All pipelines that formula is used first create, and in order to realize various image processing algorithms, generally require a large amount of pipeline (examples of creation
Such as more than three pipelines), this starting needs that will lead to camera takes a substantial amount of time, and occupies a large amount of memory.This Shen
Please embodiment 12 reception RAW images of hardware abstraction layer, RAW image is converted to YUV image and transmission RAW image
And/or YUV image, without carrying out post-processing of taking pictures to RAW image and/or YUV image, therefore, 12 need of hardware abstraction layer
A small amount of (such as one or two) pipeline is established, without establishing a large amount of pipeline, so as to save memory, and can be with
So that the starting speed of camera becomes faster.
Application program module 14 is connect with hardware abstraction layer 12 by camera service module 18.Application program module 14 can
For generating control instruction according to the input of user and the control instruction being passed through camera service module 18, hardware abstraction layer
12 are sent to imaging sensor 20 is controlled accordingly with the work to imaging sensor 20.Wherein, application program module 14
It can be run with 64 bits (bit), and the static data link library (lib) of the image processing algorithm for post-processing of taking pictures can be with
64 bits are configured to, to improve arithmetic speed.Application program module 14 receives hardware abstraction layer 12 and passes through camera service mould
After RAW image and/or YUV image that block 18 transmits, post-processing of taking pictures can be carried out to RAW and/or YUV image, can also incited somebody to action
RAW and/or YUV image are transmitted to algorithm post-processing module 16 and carry out post-processing of taking pictures.It is, of course, also possible to be application program module
14 carry out some post-processings of taking pictures (such as U.S. face processing, filter processing, rotation processing, watermark processing, virtualization processing etc.), algorithm
Post-processing module 16 carries out other and takes pictures post-processing (such as HDR processing, multi-frame processing etc.).In the application embodiment
In, RAW and/or YUV image are transmitted to algorithm post-processing module 16 and carry out post-processing of taking pictures by application program module 14.
Algorithm post-processing module 16 is connected by application program module 14, camera service module 18 and hardware abstraction layer 12
It connects, at least one image processing algorithm is stored in algorithm post-processing module 16 (for example including U.S. face Processing Algorithm, filter processing
In algorithm, rotation processing algorithm, watermark processing algorithm, virtualization Processing Algorithm, HDR Processing Algorithm and multi-frame processing algorithm extremely
Few one kind), algorithm post-processing module 16 is for using image processing algorithm processing RAW image and/or YUV image to take pictures to realize
Post-processing.Due to that can be realized by algorithm post-processing module 16 to the post-processing that take pictures of RAW image and/or YUV image, thus nothing
Process truncation need to be done on the algorithm framework of hardware abstraction layer 12 itself, only need to do compatibility in outside, design difficulty reduces.And
And realized due to taking pictures post-processing by algorithm post-processing module 16, the function of algorithm post-processing module 16 is more single, more poly-
Coke, it is fast so as to reach transplanting, it is simple and other effects to extend new image processing algorithm.Certainly, if in application program module
14 carry out some post-processings of taking pictures (such as U.S. face processing, filter processing, rotation processing, watermark processing, virtualization processing etc.), algorithm
Post-processing module 16 carry out other take pictures post-processing (such as HDR processing, multi-frame processing etc.) when, in application program module 14
At least one image processing algorithm can also be stored with (to calculate for example including U.S. face Processing Algorithm, filter Processing Algorithm, rotation processing
At least one of method, watermark processing algorithm, virtualization Processing Algorithm, HDR Processing Algorithm and multi-frame processing algorithm), application program
Module 14 is also used to use image processing algorithm processing RAW image and/or YUV image to realize post-processing of taking pictures.Due to RAW
Image and/or YUV image take pictures post-processing and be realized by application program module 14 and algorithm post-processing module 16, without
Process truncation is done on the algorithm framework of hardware abstraction layer 12 itself, only need to do compatibility in outside, Coupling Design difficulty is same
And it greatly reduces.
16 processing RAW images of algorithm post-processing module (such as image processing algorithm is handled for RAW image)
When, hardware abstraction layer 12 can only transmission RAW image (can not need RAW image being converted to YUV image at this time)；It is calculating
When 16 processing YUV images of method post-processing module (such as image processing algorithm is handled for YUV image), hardware abstraction mould
Block 12 can only transmit YUV image；When algorithm post-processing module 16 handles RAW image and YUV image, hardware abstraction layer 12
RAW image and YUV image can be transmitted.
After imaging sensor 20 is once shot (exposure image), photographed data (RAW image) is transferred to hardware and is taken out
As module 12, after algorithm post-processing module 16 receives RAW image corresponding with photographed data and/or YUV image, image is passed
Sensor 20 is able to carry out shooting next time or imaging sensor 20 can close or application program module 14 can be closed or be answered
Application interface can be exited with program module 14.It is realized, will be being clapped by algorithm post-processing module 16 due to taking pictures post-processing
Take the photograph the corresponding RAW image of data and/or after YUV image is transferred to algorithm post-processing module 16, it is only necessary to algorithm post-processing module
16 achieve that post-processing of taking pictures, and imaging sensor 20 and application program module 14 are without participating in post-processing of taking pictures at this time, therefore,
Imaging sensor 20 can be closed or be executed and shoot next time, and application interface can be closed or be exited to application program module 14.Such as
This, filming apparatus 100 can be realized snap, and can close application when algorithm post-processing module 16 take pictures and post-processes
Program module 14 exits Application Program Interface, thus carry out on an electronic device it is some other operation (such as with filming apparatus
100 unrelated operations, for example browse webpage, see video, make a phone call), so that user claps without devoting a tremendous amount of time waiting
According to the completion of post-processing, user-friendly electronic equipment.
Algorithm post-processing module 16 may include coding unit 162, and coding unit 162 is used to YUV image being converted to JPG
Image (or jpeg image etc.).Specifically, algorithm post-processing module 14 processing be YUV image when, coding unit 162 can
Directly to be encoded YUV image to form JPG image, to improve the output speed of image.In algorithm post-processing module
14 processing be RAW image when, processing can be realized the RAW image for post-processing of taking pictures through using journey by algorithm post-processing module 14
Sequence module 14, camera service module 18 are back to hardware abstraction layer 12, such as are back to RAW and turn RGB processing unit 124, RAW
Turn RGB processing unit 124 can be used for by algorithm post-processing module 14 processing realize take pictures post-processing and through application program module
The RAW image of 14 passbacks is converted to RGB image, and RGB image can be converted to YUV figure by noise reduction and YUV post-processing unit 126
Picture, the YUV image can be re-transmitted in the coding unit 162 of algorithm post-processing module 16 to be converted to the YUV image
JPG image.In some embodiments, processing can also be realized the RAW image for post-processing of taking pictures by algorithm post-processing module 14
It is back to cache unit 122 through application program module 14, camera service module 18, the RAW image of passback turns at RGB by RAW
It manages unit 124 and noise reduction and YUV post-processing unit 126 forms YUV image, then be transmitted to coding unit 162 to form JPG figure
Picture.After forming JPG image, algorithm post-processing module 16 can be used for saving JPG image transmitting into memory.
Referring to Fig. 9, algorithm post-processing module 16 includes logical process calling layer 164, algoritic module interface layer 166 and calculates
Method process layer 168.Logical process calling layer 164 with application program module 14 for communicating.Algoritic module interface layer 166 is for tieing up
Protect algorithm interface.Algorithm process layer 168 includes at least one image processing algorithm.Algoritic module interface layer 166 is for passing through calculation
Operation at least one of is registered, nullifies, calls and adjust back to method interface to the image processing algorithm of algorithm process layer 168.
Logical process calling layer 164 may include thread queue, algorithm post-processing module 16 receive RAW image and/or
After the post-processing task of taking pictures of YUV image, can will take pictures post-processes task buffer and handles in thread queue, wherein thread
Queue can cache multiple post-processing tasks of taking pictures, in this way, can realize snap (i.e. snap by logical process calling layer 164
Mechanism).In addition, logical process calling layer 164 also can receive the initialization (init) of the transmission of application program module 14, process
(process) instruction such as, and corresponding instruction and data is saved in thread queue.Logical process calling layer 164 is according to line
Task in journey queue carries out the calling of specific logic (i.e. specific logic calls combination).Logical process calling layer 164 can be with
The thumbnail (thumbnail) that processing obtains is returned into application program module 14 and is shown (i.e. thumbnail echo).
Algoritic module interface layer 166 is for calling algorithm interface, and call instruction can also be saved in thread queue, algorithm
Process layer 168 can parse the figure that the gain of parameter of call instruction needs to call in the call instruction for receiving thread queue
As Processing Algorithm.It, can be new in algorithm process layer 168 when algoritic module interface layer 166 registers image processing algorithm
Increase a kind of image processing algorithm；When algoritic module interface layer 166 is unregistered image processing algorithm, it can be handled with deletion algorithm
A kind of image processing algorithm in layer 168；When algoritic module interface layer 166 is called image processing algorithm, calculation can be called
A kind of image processing algorithm in method process layer 168；It, can be with when algoritic module interface layer 166 adjusts back image processing algorithm
By after algorithm process data and state return to application program module 14.Wherein it is possible to be realized at image using unified interface
The operation such as registration, cancellation, calling, readjustment of adjustment method.Each image processing algorithm in algorithm process layer 168 is all independent
, it so can be convenient and the operation such as registration realized to image processing algorithm, nullified, called, adjust back.
Referring to Fig. 8, camera service module 18 is connect with hardware abstraction layer 12 and application program module 14 respectively.Camera
Service module 18 RAW image and/or YUV image are packaged and by after encapsulation RAW image and/or YUV image be transmitted to
Application program module 14 and the RAW image that application program module 14 returns is transmitted to hardware abstraction layer 12.In this way, passing through
Camera service module 18 is packaged image, and the efficiency of image transmitting can be improved, and can be improved the peace of image transmitting
Quan Xing.The data transmitted between hardware abstraction layer 12 and application program module 14 are needed through camera service module 18.For example,
When hardware abstraction layer 12 transmits RAW image and/or YUV image to application program module 14, hardware abstraction layer 12 is first by RAW
Image and/or YUV image are transmitted to camera service module 18, and camera service module 18 carries out RAW image and/or YUV image
Encapsulate and by after encapsulation RAW image and/or YUV image be transmitted to application program module 14.In another example hardware abstraction layer 12
When transmitting metadata to application program module 14, metadata is first transmitted to camera service module 18, phase by hardware abstraction layer 12
Machine service module 18 is packaged metadata and the metadata after encapsulation is transmitted to application program module 14.In another example firmly
When part abstract module 12 transmits frame number suggestion to application program module 14, frame number suggestion is first transmitted to phase by hardware abstraction layer 12
Machine service module 18, camera service module 18 are packaged frame number suggestion and the frame number suggestion after encapsulation are transmitted to using journey
Sequence module 14.In another example hardware abstraction layer 12 is first when hardware abstraction layer 12 is suggested to 14 transmission algorithm of application program module
Algorithm suggestion is transmitted to camera service module 18, camera service module 18 is packaged algorithm suggestion and by the calculation after encapsulation
Method suggestion is transmitted to application program module 14.Certainly, in some embodiments, hardware abstraction layer 12 can believe sensitivity
Breath, the jitter conditions of gyroscope, AR scene detection results etc. are transmitted to camera service module 18, and camera service module 18 is according to sense
Luminosity information, the jitter conditions of gyroscope, AR scene detection results etc. obtain frame number suggestion and/or algorithm suggestion, then by frame number
It is recommended that and/or algorithm suggestion be transmitted to application program module 14.
Figure 10 and Figure 11 are please referred to, the electronic equipment 1000 of the application embodiment includes the bat of any of the above-described embodiment
Device 100 and shell 200 are taken the photograph, filming apparatus 100 is combined with shell 200.Shell 200 can be used as the function of electronic equipment 1000
The installation carrier of element.Shell 200 can provide the protection such as dust-proof, shatter-resistant, waterproof for function element, and function element can be aobvious
Display screen, filming apparatus 100, receiver etc..Wherein, in one embodiment, shell 200 includes main body 210 and movable support
220, movable support 220 can move under the drive of the drive relative to main body 210, such as movable support 220 can phase
For main 210 bodies sliding, to slide into main body 210 (such as state of Figure 10) or skid off (such as state of Figure 11) from main body 210.
Partial function element may be mounted in main body 210, and another part function element (such as filming apparatus 100) may be mounted at can
On dynamic bracket 220, the movement of movable support 220 can drive another part function element to retract in main body 210 or from main body 210
It stretches out.In another embodiment, acquisition window is offered on shell 200, filming apparatus 100 is directed at installation with acquisition window
So that filming apparatus 100 can receive ambient by acquisition window to form image or the setting of filming apparatus 100 aobvious
Below display screen, filming apparatus 100 receives the ambient across display screen to form image.
Although presently filed embodiment has been shown and described above, it is to be understood that above embodiment is
Illustratively, it should not be understood as the limitation to the application, those skilled in the art within the scope of application can be right
Above embodiment is changed, modifies, replacement and variant.
Claims (16)
1. a kind of image processing method characterized by comprising
Application program module is to camera service module request initial data；
The camera service module sends the request to hardware abstraction layer after receiving the request；
The hardware abstraction layer is sent to after receiving the request, by the initial data through the camera service module
The application program module；
The application program module is sent to algorithm post-processing mould after receiving the initial data, by the initial data
Block；With
The algorithm post-processing module take pictures post-processing and will treated that result data is returned to the initial data
The application program module.
2. image processing method according to claim 1, which is characterized in that the application program module takes to the camera
The initial data of business module request is multiframe；
The hardware abstraction layer is sent to after receiving the request, by the initial data through the camera service module
The application program module, comprising:
The hardware abstraction layer is after receiving the request, successively by initial data described in multiframe through the camera service mould
Block is sent to the application program module；
The application program module is sent to algorithm post-processing mould after receiving the initial data, by the initial data
Block, comprising:
The application program module transmits initial data described in multiframe to institute after receiving initial data described in multiframe
State algorithm post-processing module.
3. image processing method according to claim 1, which is characterized in that in the algorithm post-processing module to the original
Beginning data take pictures during post-processing, and application interface can be closed or be exited to the application program module.
4. image processing method according to claim 1, which is characterized in that described image processing method further include:
The camera service module sends the frame number suggestion about the initial data to the application program module；
The application program module is to camera service module request initial data, comprising:
The application program module is according to the frame number suggestion to initial data described in the camera service module request.
5. image processing method according to claim 1, which is characterized in that described image processing method further include:
The camera service module sends to the application program module and take pictures the algorithm of post-processing to the initial data
It is recommended that；
The application program module is arranged pre-defined algorithm according to the algorithm suggestion and is sent to the pre-defined algorithm described hard
Part abstract module；
The algorithm post-processing module take pictures post-processing and will treated that result data is returned to the initial data
The application program module, comprising:
The algorithm post-processing module carries out post-processing of taking pictures to the initial data according to the pre-defined algorithm, and will be after processing
The result data be returned to the application program module.
6. image processing method according to claim 1, which is characterized in that the initial data include pending data and
Metadata；The application program module is to camera service module request initial data, comprising:
The application program module is to pending data described in the camera service module request；
The hardware abstraction layer is sent to after receiving the request, by the initial data through the camera service module
The application program module, comprising:
The hardware abstraction layer is after receiving the request, by the pending data and corresponding with the pending data
The metadata be sent to the application program module through the camera service module；
The application program module is sent to algorithm post-processing mould after receiving the initial data, by the initial data
Block, comprising:
The application program module is after receiving the pending data and the metadata, by the pending data and institute
It states metadata and is sent to algorithm post-processing module；
The algorithm post-processing module take pictures post-processing and will treated that result data is returned to the initial data
The application program module, comprising:
The algorithm post-processing module take pictures post-processing and will be after processing to the pending data according to the metadata
Result data be returned to the application program module.
7. image processing method according to claim 1, which is characterized in that data in the hardware abstraction layer are the
One picture format；Wherein:
The application program module to the initial data of the camera service module request be also the first image format
Data；Or
The application program module to the initial data of the camera service module request be the second picture format data,
Described image processing method further include:
The data of the first image format are converted to the data of second picture format by the hardware abstraction layer.
8. a kind of image processor, which is characterized in that including application program module, camera service module, hardware abstraction layer and
Algorithm post-processing module；
The application program module is used for the camera service module request initial data；
The camera service module is used for after receiving the request, sends the request to the hardware abstraction layer；
The hardware abstraction layer is used for after receiving the request, and the initial data is sent out through the camera service module
It send to the application program module；
The application program module is also used to after receiving the initial data, and the initial data is sent to the algorithm
Post-processing module；With
The algorithm post-processing module is used to take pictures to the initial data post-processing and will treated that result data passes
Back to the application program module.
9. image processor according to claim 8, which is characterized in that the application program module is to the camera service
The initial data of module request is multiframe；
The hardware abstraction layer is used for after receiving the request, successively takes initial data described in multiframe through the camera
Business module is sent to the application program module；
The application program module is used for after receiving initial data described in multiframe, and initial data described in multiframe is transmitted
To the algorithm post-processing module.
10. image processor according to claim 8, which is characterized in that in the algorithm post-processing module to the original
Beginning data take pictures during post-processing, and application interface can be closed or be exited to the application program module.
11. image processor according to claim 8, which is characterized in that
The camera service module is also used to send the frame number suggestion about the initial data to the application program module；
The application program module is also used to according to the frame number suggestion to initial data described in the camera service module request.
12. image processor according to claim 8, which is characterized in that
The camera service module, which is also used to send to the application program module, carries out post-processing of taking pictures to the initial data
Algorithm suggestion；
The application program module is also used to that pre-defined algorithm is arranged according to the algorithm suggestion and is sent to the pre-defined algorithm
The hardware abstraction layer；
The algorithm post-processing module is also used to carry out post-processing of taking pictures to the initial data according to the pre-defined algorithm, and will
Treated, and the result data is returned to the application program module.
13. image processor according to claim 8, which is characterized in that the initial data include pending data and
Metadata；
The application program module is used for pending data described in the camera service module request；
The hardware abstraction layer be used for after receiving the request, by the pending data and with the pending data
The corresponding metadata is sent to the application program module through the camera service module；
The application program module is used for after receiving the pending data and the metadata, by the pending data
Algorithm post-processing module is sent to the metadata；
The algorithm post-processing module is used to take pictures post-processing and will place to the pending data according to the metadata
Result data after reason is returned to the application program module.
14. image processor according to claim 8, which is characterized in that data in the hardware abstraction layer are the
One picture format；Wherein:
The application program module to the initial data of the camera service module request be also the first image format
Data；Or
The application program module to the initial data of the camera service module request be the second picture format data,
The hardware abstraction layer is also used to be converted to the data of the first image format the number of second picture format
According to.
15. a kind of filming apparatus characterized by comprising
Image processor described in claim 8-14 any one；With
Imaging sensor, described image sensor are connect with described image processor.
16. a kind of electronic equipment characterized by comprising
Filming apparatus described in claim 15, and
Shell, the filming apparatus is in conjunction with the shell.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910285175.4A CN110086967B (en) | 2019-04-10 | 2019-04-10 | Image processing method, image processor, photographing device and electronic equipment |
| PCT/CN2020/079534 WO2020207200A1 (en) | 2019-04-10 | 2020-03-16 | Image processing method, image processor, photography device, and electronic apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910285175.4A CN110086967B (en) | 2019-04-10 | 2019-04-10 | Image processing method, image processor, photographing device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110086967A true CN110086967A (en) | 2019-08-02 |
| CN110086967B CN110086967B (en) | 2021-02-05 |
Family
ID=67414749
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910285175.4A Active CN110086967B (en) | 2019-04-10 | 2019-04-10 | Image processing method, image processor, photographing device and electronic equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110086967B (en) |
| WO (1) | WO2020207200A1 (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110401800A (en) * | 2019-08-26 | 2019-11-01 | Oppo广东移动通信有限公司 | Image processing method, image processor, filming apparatus and electronic equipment |
| CN110933314A (en) * | 2019-12-09 | 2020-03-27 | Oppo广东移动通信有限公司 | Focus-following shooting method and related product |
| CN110941821A (en) * | 2019-12-09 | 2020-03-31 | Oppo广东移动通信有限公司 | Data processing method, device and storage medium |
| CN110941344A (en) * | 2019-12-09 | 2020-03-31 | Oppo广东移动通信有限公司 | Method for obtaining gazing point data and related device |
| CN110942047A (en) * | 2019-12-09 | 2020-03-31 | Oppo广东移动通信有限公司 | Application optimization method and related product |
| CN110958390A (en) * | 2019-12-09 | 2020-04-03 | Oppo广东移动通信有限公司 | Image processing method and related device |
| CN110971830A (en) * | 2019-12-09 | 2020-04-07 | Oppo广东移动通信有限公司 | Anti-shake method for video shooting and related device |
| CN110996012A (en) * | 2019-12-23 | 2020-04-10 | Oppo广东移动通信有限公司 | Continuous shooting processing method, image processor, shooting device and electronic equipment |
| CN110995994A (en) * | 2019-12-09 | 2020-04-10 | 上海瑾盛通信科技有限公司 | Image shooting method and related device |
| CN110996089A (en) * | 2019-12-09 | 2020-04-10 | 上海瑾盛通信科技有限公司 | Volume calculation method and related device |
| CN111225153A (en) * | 2020-01-21 | 2020-06-02 | Oppo广东移动通信有限公司 | Image data processing method, image data processing device and mobile terminal |
| CN111314606A (en) * | 2020-02-21 | 2020-06-19 | Oppo广东移动通信有限公司 | Photographing method and device, electronic equipment and storage medium |
| WO2020207200A1 (en) * | 2019-04-10 | 2020-10-15 | Oppo广东移动通信有限公司 | Image processing method, image processor, photography device, and electronic apparatus |
| WO2020207192A1 (en) * | 2019-04-10 | 2020-10-15 | Oppo广东移动通信有限公司 | Image processor, image processing method, photography apparatus, and electronic device |
| CN112291475A (en) * | 2020-10-30 | 2021-01-29 | 维沃移动通信（杭州）有限公司 | Photographing method and device and electronic equipment |
| CN113554718A (en) * | 2020-04-24 | 2021-10-26 | Oppo（重庆）智能科技有限公司 | Image encoding method, storage medium, and electronic device |
| CN113645409A (en) * | 2021-08-16 | 2021-11-12 | 展讯通信（上海）有限公司 | Photographing processing method and device, photographing method, device and system and terminal equipment |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114125277B (en) * | 2021-11-12 | 2024-02-09 | 北京达佳互联信息技术有限公司 | Method and equipment for controlling video shooting special effects of application |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070209056A1 (en) * | 2006-03-02 | 2007-09-06 | Canon Kabushiki Kaisha | Image transmission apparatus |
| CN102799469A (en) * | 2012-05-17 | 2012-11-28 | 江苏中科梦兰电子科技有限公司 | Method for calling photographing module of android application software |
| CN103294493A (en) * | 2012-02-28 | 2013-09-11 | 联想(北京)有限公司 | Method and device for initializing peripheral equipment, and camera |
| CN104471510A (en) * | 2012-07-11 | 2015-03-25 | 谷歌公司 | Abstract camera pipeline for uniform cross-device control of image capture and processing |
| CN105827957A (en) * | 2016-03-16 | 2016-08-03 | 上海斐讯数据通信技术有限公司 | Image processing system and method |
| WO2016149894A1 (en) * | 2015-03-23 | 2016-09-29 | Intel Corporation | Workload scheduler for computing devices with camera |
| CN108833803A (en) * | 2018-09-20 | 2018-11-16 | Oppo广东移动通信有限公司 | Imaging method, device and electronic equipment |
| CN109101352A (en) * | 2018-08-30 | 2018-12-28 | Oppo广东移动通信有限公司 | Algorithm framework, algorithm call method, device, storage medium and mobile terminal |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6642956B1 (en) * | 1998-05-29 | 2003-11-04 | Agilent Technologies, Inc. | Digital image processor for a digital camera |
| JP2005284411A (en) * | 2004-03-26 | 2005-10-13 | Fuji Photo Film Co Ltd | Method, apparatus, system and program for image processing |
| CN105827940A (en) * | 2015-09-24 | 2016-08-03 | 维沃移动通信有限公司 | Photographing method and electronic device |
| CN105827971B (en) * | 2016-03-31 | 2019-01-11 | 维沃移动通信有限公司 | A kind of image processing method and mobile terminal |
| CN110086967B (en) * | 2019-04-10 | 2021-02-05 | Oppo广东移动通信有限公司 | Image processing method, image processor, photographing device and electronic equipment |
-
2019
- 2019-04-10 CN CN201910285175.4A patent/CN110086967B/en active Active
-
2020
- 2020-03-16 WO PCT/CN2020/079534 patent/WO2020207200A1/en active Application Filing
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070209056A1 (en) * | 2006-03-02 | 2007-09-06 | Canon Kabushiki Kaisha | Image transmission apparatus |
| CN103294493A (en) * | 2012-02-28 | 2013-09-11 | 联想(北京)有限公司 | Method and device for initializing peripheral equipment, and camera |
| CN102799469A (en) * | 2012-05-17 | 2012-11-28 | 江苏中科梦兰电子科技有限公司 | Method for calling photographing module of android application software |
| CN104471510A (en) * | 2012-07-11 | 2015-03-25 | 谷歌公司 | Abstract camera pipeline for uniform cross-device control of image capture and processing |
| WO2016149894A1 (en) * | 2015-03-23 | 2016-09-29 | Intel Corporation | Workload scheduler for computing devices with camera |
| CN105827957A (en) * | 2016-03-16 | 2016-08-03 | 上海斐讯数据通信技术有限公司 | Image processing system and method |
| CN109101352A (en) * | 2018-08-30 | 2018-12-28 | Oppo广东移动通信有限公司 | Algorithm framework, algorithm call method, device, storage medium and mobile terminal |
| CN108833803A (en) * | 2018-09-20 | 2018-11-16 | Oppo广东移动通信有限公司 | Imaging method, device and electronic equipment |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020207192A1 (en) * | 2019-04-10 | 2020-10-15 | Oppo广东移动通信有限公司 | Image processor, image processing method, photography apparatus, and electronic device |
| WO2020207200A1 (en) * | 2019-04-10 | 2020-10-15 | Oppo广东移动通信有限公司 | Image processing method, image processor, photography device, and electronic apparatus |
| CN110401800A (en) * | 2019-08-26 | 2019-11-01 | Oppo广东移动通信有限公司 | Image processing method, image processor, filming apparatus and electronic equipment |
| CN110971830A (en) * | 2019-12-09 | 2020-04-07 | Oppo广东移动通信有限公司 | Anti-shake method for video shooting and related device |
| CN110941821A (en) * | 2019-12-09 | 2020-03-31 | Oppo广东移动通信有限公司 | Data processing method, device and storage medium |
| CN110958390A (en) * | 2019-12-09 | 2020-04-03 | Oppo广东移动通信有限公司 | Image processing method and related device |
| CN110941344A (en) * | 2019-12-09 | 2020-03-31 | Oppo广东移动通信有限公司 | Method for obtaining gazing point data and related device |
| CN110971830B (en) * | 2019-12-09 | 2021-11-02 | Oppo广东移动通信有限公司 | Anti-shake method for video shooting and related device |
| CN110995994A (en) * | 2019-12-09 | 2020-04-10 | 上海瑾盛通信科技有限公司 | Image shooting method and related device |
| CN110996089A (en) * | 2019-12-09 | 2020-04-10 | 上海瑾盛通信科技有限公司 | Volume calculation method and related device |
| CN110942047A (en) * | 2019-12-09 | 2020-03-31 | Oppo广东移动通信有限公司 | Application optimization method and related product |
| CN110958390B (en) * | 2019-12-09 | 2021-07-20 | Oppo广东移动通信有限公司 | Image processing method and related device |
| CN110996089B (en) * | 2019-12-09 | 2021-11-30 | 上海瑾盛通信科技有限公司 | Volume calculation method and related device |
| CN110933314A (en) * | 2019-12-09 | 2020-03-27 | Oppo广东移动通信有限公司 | Focus-following shooting method and related product |
| CN110941344B (en) * | 2019-12-09 | 2022-03-15 | Oppo广东移动通信有限公司 | Method for obtaining gazing point data and related device |
| WO2021115113A1 (en) * | 2019-12-09 | 2021-06-17 | Oppo广东移动通信有限公司 | Data processing method and device, and storage medium |
| CN110933314B (en) * | 2019-12-09 | 2021-07-09 | Oppo广东移动通信有限公司 | Focus-following shooting method and related product |
| CN110996012A (en) * | 2019-12-23 | 2020-04-10 | Oppo广东移动通信有限公司 | Continuous shooting processing method, image processor, shooting device and electronic equipment |
| CN111225153B (en) * | 2020-01-21 | 2021-08-06 | Oppo广东移动通信有限公司 | Image data processing method, image data processing device and mobile terminal |
| CN111225153A (en) * | 2020-01-21 | 2020-06-02 | Oppo广东移动通信有限公司 | Image data processing method, image data processing device and mobile terminal |
| CN111314606A (en) * | 2020-02-21 | 2020-06-19 | Oppo广东移动通信有限公司 | Photographing method and device, electronic equipment and storage medium |
| CN113554718A (en) * | 2020-04-24 | 2021-10-26 | Oppo（重庆）智能科技有限公司 | Image encoding method, storage medium, and electronic device |
| CN112291475A (en) * | 2020-10-30 | 2021-01-29 | 维沃移动通信（杭州）有限公司 | Photographing method and device and electronic equipment |
| CN112291475B (en) * | 2020-10-30 | 2022-04-12 | 维沃移动通信（杭州）有限公司 | Photographing method and device and electronic equipment |
| CN113645409A (en) * | 2021-08-16 | 2021-11-12 | 展讯通信（上海）有限公司 | Photographing processing method and device, photographing method, device and system and terminal equipment |
| CN113645409B (en) * | 2021-08-16 | 2022-08-19 | 展讯通信（上海）有限公司 | Photographing processing method and device, photographing method, device and system and terminal equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110086967B (en) | 2021-02-05 |
| WO2020207200A1 (en) | 2020-10-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110086967A (en) | Image processing method, image processor, filming apparatus and electronic equipment | |
| CN109963083A (en) | Image processor, image processing method, filming apparatus and electronic equipment | |
| CN110177215A (en) | Image processing method, image processor, filming apparatus and electronic equipment | |
| US9723159B2 (en) | RAW camera peripheral for handheld mobile unit | |
| CN110290288A (en) | Image processor, image processing method, filming apparatus and electronic equipment | |
| CN110121022A (en) | Control method, filming apparatus and the electronic equipment of filming apparatus | |
| CN110276718A (en) | Image processing method, image processor, filming apparatus and electronic equipment | |
| WO2020207192A1 (en) | Image processor, image processing method, photography apparatus, and electronic device | |
| CN110266951A (en) | Image processor, image processing method, filming apparatus and electronic equipment | |
| CN110278373A (en) | Image processor, image processing method, filming apparatus and electronic equipment | |
| US9538083B2 (en) | Motion blur avoidance | |
| US11070742B2 (en) | Optimized exposure temporal smoothing for time-lapse mode | |
| CN111147695B (en) | Image processing method, image processor, shooting device and electronic equipment | |
| CN110177214A (en) | Image processor, image processing method, camera arrangement and electronic equipment | |
| CN110996012B (en) | Continuous shooting processing method, image processor, shooting device and electronic equipment | |
| CN111510629A (en) | Data display method, image processor, photographing device and electronic equipment | |
| CN111193867A (en) | Image processing method, image processor, photographing device and electronic equipment | |
| CN110418061A (en) | Image processing method, image processor, camera arrangement and electronic equipment | |
| KR102058860B1 (en) | Apparatus and mehod for processing a image in camera device | |
| CN110602359B (en) | Image processing method, image processor, photographing device and electronic equipment | |
| CN111491101B (en) | Image processor, image processing method, photographing device, and electronic apparatus | |
| CN110049254A (en) | Image processing method, device, storage medium and electronic equipment | |
| IES20140134A2 (en) | Improved smartphone imaging using an external peripheral | |
| IES86519B2 (en) | Camera peripheral with external camera controls and lens mount for smartphone or hand-held device | |
| IES86520B2 (en) | RAW camera peripheral for smartphone or hand-held device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |