CN109286753A - Image processing method, system and computer readable storage medium - Google Patents
Image processing method, system and computer readable storage medium Download PDFInfo
- Publication number
- CN109286753A CN109286753A CN201811145822.3A CN201811145822A CN109286753A CN 109286753 A CN109286753 A CN 109286753A CN 201811145822 A CN201811145822 A CN 201811145822A CN 109286753 A CN109286753 A CN 109286753A
- Authority
- CN
- China
- Prior art keywords
- data
- image
- optimization
- image data
- chip
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/646—Circuits for processing colour signals for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
The present invention provides image processing method, system and computer readable storage mediums.Image processing method, comprising the following steps: S100: imaging sensor acquires image data；S200: the initial data of image data is sent to picture processing chip by imaging sensor, by digital signal processing module to original data processing, forms optimization image data；S300: optimization image data is sent to image sensing processing module and handled optimization image data by digital signal processing module, forms strengthens view data；S400: picture processing chip will optimize image data and/or strengthens view data and be back to central processing element, by central processing element record optimization image data and/or strengthens view data to storing equipment；S500: picture processing chip will optimize image data and/or strengthens view data is sent to display module, display optimization image and/or enhancing image, so that handling independently of the CPU of intelligent terminal for the Intelligent treatment and optimization of image, reducing cpu load.
Description
Technical field
The present invention relates to field of intelligent control more particularly to a kind of image processing methods, system and computer-readable storage
Medium.
Background technique
Such as the mobile terminal devices such as mobile phone, tablet computer increasingly enter into people's lives.Therefore in such movement
In terminal, also integrate it is multi-functional just like shooting, camera shooting, positioning, illumination etc., with can be complete after facilitating people to hold a mobile terminal
At more than one piece demand.When being shot and being imaged using mobile terminal, the resolution ratio and pixel of camera module are higher and higher, so that
Mobile terminal imaging modalities achieved in the function are more close to the camera apparatus of profession.Specifically, mobile terminal
The imaging modality of equipment, there are mainly two types of:
1.CPU is built-in with image sensing processing module:
As shown in Figure 1, it is built-in with an image sensing processing module ISP in the central processing unit CPU of mobile terminal,
Data will be sent to ISP by mipi interface by camera shooting group, and ISP can be different giving after data processing according to different applications
Module.Data processing unit DPU such as is given by data path 1, display screen LCD is given after DPU processing and shows, for imaging
The image pickup preview of module；Video processing unit VPU such as is given by data path 2, VPU is stored the data to after being encoded
It stores in equipment, the recording function for photographing module；For another example picture compression device jpeg is given by data path 3
Encoder, jpeg encoder data compression at jpeg file after, store the data to storage equipment in, for imaging
The camera function of module；.
Image sensing processing module is equipped with outside 2.CPU
As shown in Fig. 2, it is externally connected with an image sensing processing module in CPU, and by the way of internal ISPbypass,
Other than built-in ISPbypass, other are equal data flow with mode one.
Internal image due in currently existing scheme, being all made of CPU incudes and processing function, the inside function by CPU
The limitation of energy, it is limited for the processing of image data, more professional and careful adjustment can not be carried out to shooting image, meanwhile,
The data processing load of CPU is aggravated, so that the speed of service of mobile terminal is slack-off.
Therefore, it is necessary to a kind of novel image processing methods and system, image processing flow independently can be gone out CPU, sufficiently
Using external hardware, the display effect of image quality and mobile terminal is improved.
Summary of the invention
In order to overcome the above technical defects, the purpose of the present invention is to provide a kind of image processing method, system and calculating
Machine readable storage medium storing program for executing, so that handled independently of the CPU of intelligent terminal for the Intelligent treatment and optimization of image, it is negative to reduce CPU
It carries.
The invention discloses a kind of image processing methods, comprising the following steps:
S100: the image data of imaging sensor acquisition reference object；
S200: the initial data of described image data is sent to picture processing chip by imaging sensor, by described image
The digital signal processing module of chip is handled to the original data processing, forms optimization image data；
S300: the optimization image data is sent to the figure of described image processing chip by the digital signal processing module
The optimization image data is handled as sensing processing module, forms strengthens view data；
S400: described image handles chip and the optimization image data and/or strengthens view data is back to an intelligence
Central processing element in terminal records the optimization image data and/or strengthens view data by the central processing element
To a storage equipment；
S500: described image handles chip and the optimization image data and/or strengthens view data is sent to a display
Module, by the corresponding optimization image data of display module display and/or strengthens view data optimization image and/
Or enhancing image.
Preferably, the step S200 includes:
S210: described image data are sent to image procossing with raw data format by camera interface by imaging sensor
Chip；
S220: described image handles the image sensing module of chip to the original data processing, forms optimization picture number
According to.
Preferably, the step S220 includes:
S221: described image sensing module will extract the image frame in the initial data；
S222: synthesizing the image frame or grafting mends frame to the image frame, forms multiframe composograph data.
Preferably, the step S220 includes:
S221 ': described image sensing module judges the vision environment of the initial data；
S222 ': vision environment described in needle handles the initial data rendering, forms optimization image data.
Preferably, the step S300 includes:
S310: the digital signal processing module forwards the initial data to central processing element；
S320: the intelligent object of the central processing element handles the initial data rendering, forms optimization picture number
According to；
S330: the intelligent object sends the optimization image data to processing module.
Preferably, described image processing method further include:
S600: the frame per second data of described image data are sent to picture processing chip by imaging sensor；
S700: described image handles chip and makees interleave compensation to the frame per second data, forms interleave data, the interleave number
It is matched according to the refresh rate of the display module of the intelligent terminal；
S800: compressing the interleave data, and stores to storage equipment；
S900: described image handles chip and the interleave data is sent to display module, is shown by the display module
The interleave image of corresponding interleave data.
Preferably, the frame per second of the frame per second data is more than or equal to 60fps.
The invention also discloses a kind of image processing system, including be set in a terminal display module, imaging sensor,
Picture processing chip and central processing element；
Described image processing chip is connect with the display module, imaging sensor and central processing element respectively；
The image data of described image sensor acquisition reference object is simultaneously sent to picture processing chip；
The digital signal processing module of described image processing chip handles the initial data of described image data, shape
At optimization image data；
The digital signal processing module passes the image that the optimization image data is sent to described image processing chip
Sense processing module handles the optimization image data, forms strengthens view data；
The optimization image data and/or strengthens view data are back in an intelligent terminal by described image processing chip
Central processing element, the optimization image data and/or strengthens view data are recorded by the central processing element and deposited to one
Equipment is stored up, and the optimization image data and/or strengthens view data are sent to a display module by described image processing chip,
By optimization image and/or the increasing of the corresponding optimization image data of display module display and/or strengthens view data
Strong image.
Preferably, the digital signal processing module carries out grafting to the initial data and mends frame, forms multiframe composite diagram
As data；Or
The digital signal processing module renders the initial data and contributes, and forms optimization image data.
The present invention discloses a kind of computer readable storage medium again, is stored thereon with computer program, the computer
The step of image processing method as described above is realized when program is executed by processor.
After above-mentioned technical proposal, compared with prior art, have the advantages that
1. CPU will be partitioned into for the processing of image data, do not handled, is turned complete by independent picture processing chip by CPU
At, reduction cpu load, speed up processing；
2. making full use of external hardware, the display effect of image quality and mobile terminal is improved.
Detailed description of the invention
Fig. 1 is the system structure diagram for realizing shooting image procossing in the prior art；
Fig. 2 is the system structure diagram for realizing shooting image procossing in the prior art；
Fig. 3 is the flow diagram for meeting image processing method in one embodiment of the invention；
Fig. 4 is the flow diagram for meeting image processing method in first preferred embodiment of the invention；
Fig. 5 is the flow diagram for meeting image processing method in second preferred embodiment of the invention；
Fig. 6 is the flow diagram for meeting image processing method in third preferred embodiment of the invention；
Fig. 7 is the flow diagram for meeting image processing method in one embodiment of the present invention；
Fig. 8 is the structural schematic diagram for meeting image processing system in one embodiment of the present invention.
Specific embodiment
Below in conjunction with attached drawing, the advantages of the present invention are further explained with specific embodiment.
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to
When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment
Described in embodiment do not represent all implementations consistent with this disclosure.On the contrary, they be only with it is such as appended
The example of the consistent device and method of some aspects be described in detail in claims, the disclosure.
It is only to be not intended to be limiting the disclosure merely for for the purpose of describing particular embodiments in the term that the disclosure uses.
The "an" of the singular used in disclosure and the accompanying claims book, " described " and "the" are also intended to including majority
Form, unless the context clearly indicates other meaning.It is also understood that term "and/or" used herein refers to and wraps
It may be combined containing one or more associated any or all of project listed.
It will be appreciated that though various information, but this may be described using term first, second, third, etc. in the disclosure
A little information should not necessarily be limited by these terms.These terms are only used to for same type of information being distinguished from each other out.For example, not departing from
In the case where disclosure range, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as
One information.Depending on context, word as used in this " if " can be construed to " ... when " or " when ...
When " or " in response to determination "
In the description of the present invention, it is to be understood that, term " longitudinal direction ", " transverse direction ", "upper", "lower", "front", "rear",
The orientation or positional relationship of the instructions such as "left", "right", "vertical", "horizontal", "top", "bottom" "inner", "outside" is based on attached drawing institute
The orientation or positional relationship shown, is merely for convenience of description of the present invention and simplification of the description, rather than the dress of indication or suggestion meaning
It sets or element must have a particular orientation, be constructed and operated in a specific orientation, therefore should not be understood as to limit of the invention
System.
In the description of the present invention, unless otherwise specified and limited, it should be noted that term " installation ", " connected ",
" connection " shall be understood in a broad sense, for example, it may be mechanical connection or electrical connection, the connection being also possible to inside two elements can
, can also indirectly connected through an intermediary, for the ordinary skill in the art to be to be connected directly, it can basis
Concrete condition understands the concrete meaning of above-mentioned term.
In subsequent description, it is only using the suffix for indicating such as " module ", " component " or " unit " of element
Be conducive to explanation of the invention, itself there is no specific meanings.Therefore, " module " can mixedly make with " component "
With.
Refering to Fig. 3, for the flow diagram for meeting image processing method in one embodiment of the present invention.In the embodiment
In, image processing method the following steps are included:
S100: the image data of imaging sensor acquisition reference object
To form initial pictures to be processed, using an imaging sensor, such as photography and vedio recording mould group, lens module, prism
Mould group etc. carries out a reference object such as to clap photo, video, record motion frame Image Acquisition, to form original picture number
According to.
It is understood that the hardware configuration due to different images sensor is different, acquired image data, it may be possible to
Do not make the image data of the non-photo fix type of any modification, it is also possible to carried out such as U.S. face, cutting, optimization, at rendering initial stage
The image data of reason.No matter whether the image data in the step has dealt with, will make in subsequent step further
Optimization.
S200: the initial data of image data is sent to picture processing chip by imaging sensor, by picture processing chip
Digital signal processing module to original data processing, form optimization image data
It is formed by image data in the step s 100, will be sent to by imaging sensor and be electrically connected with the imaging sensor
A picture processing chip (IPIC), the electric connection mode of imaging sensor and picture processing chip can be integrated in same
Two independent image sensing modules and image processing module on printed circuit board are also possible to two connected with conducting wire solely
Vertical device is also possible to that two independent devices being wirelessly connected in same local area network or mutually are arranged in.Image procossing core
Piece IPIC will carry out such as interleave, intelligent optimization (AI processing) precision processing after receiving image data to it, to form one
Optimize image data.The difference for optimizing processing capacity of the image data based on picture processing chip IPIC can carry out in various degree
Image optimization, in the present embodiment, picture processing chip as separately from other any modules, mould group, unit, equipment device
Part optimizes processing to image data in a manner of independently operated, on the one hand can reduce image processing function to other equipment
Load pressure, on the other hand can specially can be dedicated, not by the integrated equipment of castrating partial function to image procossing, increase at image
The efficiency and effect of reason.
S300: digital signal processing module passes the image that the optimization image data is sent to described image processing chip
Sense processing module handles the optimization image data, forms strengthens view data
After completing the first round optimization for original image, digital signal processing module is sent to figure for image data is optimized
As the image sensing processing module of processing chip, optimization image data is carried out such as denoising, stain by image sensing processing module
The processing such as correction, forms strengthens view data.As second wheel optimization, the first round have determined that meet reference object type,
After the works such as scene, time optimize, the further picture sense of strengthens view data, to call whole energy of picture processing chip
Power completes the maximization to image and handles and progress greatly.
S400: picture processing chip will optimize image data and/or strengthens view data is back in an intelligent terminal
Central processing element, by central processing element record optimization image data and/or strengthens view data to a storage equipment
The optimization image data and/or strengthens view data formed after being disposed will be back to one by picture processing chip
Central processing element (end CPU or AP).The central processing element is arranged in an intelligent terminal, and is hung on outside picture processing chip
Central processing element may also be arranged in same intelligent terminal, but be independently arranged with central processing element, and the two distribution is set
Meter.Picture processing chip can design (but not integrating) or image procossing core on same printed circuit board with central processing element
Piece is connect with central processing element by conducting wire, or the wireless protocols by supporting mutually are wirelessly connected.By central processing core
Piece and picture processing chip are independent, central processing element can focus in intelligent terminal to the instruction of each component pass on and under
Hair, and the function of image procossing, then be partitioned into central processing element, is individually completed by plug-in picture processing chip, in reduction
While centre processing chip load, the standard requirement of central processing element can also be simplified, internal original image processing capacity can
Compression simplifies, and works for the optimization processing of the image data of outside acquisition, transfers to more specialized, the higher figure of optimization degree
As processing chip completion, image processing speed can be accelerated.Simultaneously as image data is no longer pass through central processing element, for
The display and load of image data after being disposed also will faster.
After central processing element receives optimization image data and/or strengthens view data, by record optimization image data
And/or strengthens view data to one storage equipment, such as the memory of intelligent terminal, external connection storage apparatus (USB flash disk, shifting of intelligent terminal
Dynamic hard disk, the other equipment etc. of OTG connection).It in this step, is the processed optimization image data and/or increasing finished of storage
Strong image data, central processing element execute the movement of a forwarding record, do not make any variation for strengthens view data itself,
Reduce the operation load of central processing element.
It is understood that the difference of model and configuration for picture processing chip, parts of images handles chip in shape
At optimization image data when, may by complete together for initial data picture enhancing, under this embodiment, no longer will be performed
The step of strengthens view data is formed in S300 (or can be described as executing together)；For another example in section Example, step S300 is skipped,
Optimization image data is directly forwarded to storage equipment, can also realize technical solution of the present invention.
S500: picture processing chip will optimize image data and/or strengthens view data is sent to a display module, by showing
Show the optimization image and/or enhancing image of the corresponding optimization image data of module display and/or strengthens view data.
Step S400 and step S500 can be executed either synchronously or asynchronously, i.e., in central processing element record optimization image data
And/or strengthens view data to storage equipment while, before or after etc. any times, picture processing chip will optimize image
Data and/or strengthens view data are sent to a display module, display screen, the touch screen specifically such as installed on intelligent terminal
Display module, received by display module and show corresponding optimization image data and/or strengthens view data optimization image and/or
Enhance image, i.e., carries out intelligent optimization, rendering, color-temperature regulating, light filling, AR enhancing, information on the basis of initial image data
Enhancing image or scene rendering, the Object identifying, frame per second compensation, blood-shot eye illness formed after every image procossings such as identification, U.S. face, cutting
The optimization image of removal etc..
Optimization and enhancing two-step pretreatment are made to image by above-mentioned individual picture processing chip, and directly by treated
Enhancing image is sent to display module all the way and shows, is sent to storage equipment storage, the transmission of image after handling with raising all the way
Speed, and load and display speed in user side.Effect of the photographing module in AI processing can be additionally promoted, and is promoted fast
The video recording effect of fast motion picture.
In one preferred embodiment, step S200 includes:
S210: image data is sent to picture processing chip with raw data format by camera interface by imaging sensor
Imaging sensor Image sensor is sent the raw data format of image data by camera interface CAMIF
To picture processing chip.CAMIF is the first part of video front-end (VFE) hardware, and main task is synchronous
Sensor sends the row being related in data procedures, field sync signal.In addition it also has image zooming-out and image mirrors ability.
In order to accurately synchronous with external camera sensor, CAMIF hardware must provide two programmable interrupt lines, one with
Control the opening of shutter, another be used to control flash of light.CAMIF hardware input equipment includes PCLK, HSYNC, VSYNC, one
A pixel enables gate and 12bit data line.Input interface type is controlled by adsp.When powering on, CAMIF module is
It closes, data will not be grabbed, until adsp setting enables as enable, turn-on data capture.Data bit is enabled when being arranged
Afterwards, CAMIF can just start to collect data until next frame starts.Similar, when adps is set as disable, CAMIF
It can stop working until next frame data are sent completely, it is ensured that all things have all been handled.CAMIF hardware provides pair
The function of camera sensor data resampling, before CAMIF output data, it is complete that adsp, which independent can be controlled,
Output image still does double sampling.
Different from imaging sensor in the prior art by the compressed data of reference object, i.e. the data of yuv format are sent to
CPU is dealt with, and imaging sensor directly transmits that data volume is larger, the higher initial data Raw of resolution ratio in the present embodiment
Data, the i.e. such as raw data format of bayer format to picture processing chip.In the prior art, due to by CPU to picture number
According to processing, and CPU does not have processing capacity to Raw Data and therefore even if raw data format is sent to CPU, can not yet
Processing.And after picture processing chip of the invention is utilized, it can just be improved at data source directly to raw data format processing
Picture quality.
S220: the image sensing module of picture processing chip forms optimization image data to original data processing.
In different embodiments, picture processing chip for image data or forms the processing mode of optimization image data not
Together, specifically, it will be described in detail by the introduction of following different embodiments.
Embodiment one
Refering to Fig. 4, in this embodiment, S220 includes:
S221: image sensing module will extract the image frame in initial data；
S222: synthesis image frame or grafting mend frame to the image frame, form multiframe composograph data.
Image sensing module will obtain in shooting to reference object under synchronization or closely at the time of under
To the image frame at many corresponding each moment, due to the limitation of shooting, the clear pixel of each image frame is different.Cause
This, in this embodiment, image sensing module will extract the image frame in initial data, and image frame is sent to image procossing
The image processing unit IPU of chip is screened, and image processing unit makees multiframe conjunction according to the clear situation of each image frame
At that is, using clearly pixel replaces the pixel obscured in an image frame in other image frames, to be mended by synthesis or grafting
The mode of frame improves the defect of initial data, forms multiframe composograph data.
Multiframe composograph data can be back to central processing element, use image compression module by central processing element
Compression of images, and write storage device are carried out to multiframe composograph data.
Embodiment two
Refering to Fig. 5, in this embodiment, S220 includes:
S221 ': image sensing module judges the vision environment of initial data；
S222 ': initial data rendering is handled for vision environment, forms optimization image data.
For the AI Intelligent treatment of initial data, rarely needed shooting function will be helped or not will use the use of shooting function
Shoot same profession, correct image of finding a view in family.Specifically, imaging sensor will judge the vision environment of initial data, such as
Reference object is in the indisposed of certain sight spot, the bright dark degree of indoor environment, the type of reference object, reference object and ambient enviroment
Degree etc., will be judged by experience and algorithm.After the completion of judgement, initial data rendering will be handled, such as according to ambient enviroment
Add filter automatically to initial data, to initial data automatic cutting, by reference object be placed on position the most suitable, from
Dynamic virtualization background, adjust automatically frame parameter (such as colour temperature, brightness, contrast), to form optimization original image.
The result of above-mentioned AI Intelligent treatment will be sent to the image sensing processing module ISP inside IPIC and continue to locate
Reason, further enhances.
Embodiment three
Refering to Fig. 6, in another preferred embodiment, step S300 includes:
S310: digital signal processing module forwards initial data to central processing element；
S320: the intelligent object of central processing element handles initial data rendering, forms optimization image data；
S330: intelligent object sends optimization image data to processing module.
For configuration is poor or is no longer appropriate for the intelligent terminal for increasing picture processing chip, at imaging sensor,
Raw data format is compressed, such as RGB, yuv format are formed, and be forwarded directly to central processing element, by central processing
Chip carries out AI processing as described above, and executes compression of images.
In another preferred embodiment, tool will be used for the image data for cooperatively forming high quality, high effect refering to Fig. 7
There is the display module of high refresh rate (frame per second for the frame per second data that image is shown is more than or equal to 60fps), therefore, for such display
Module, image processing method further include:
S600: the frame per second data of image data are sent to picture processing chip by imaging sensor, such as pass through CamIF interface
It is sent to IPIC；
S700: picture processing chip makees interleave compensation to frame per second data, forms interleave data, interleave data and intelligent terminal
Display module refresh rate matching；
S800: compression interleave data, and store to storage equipment, the storage equipment can it is identical as storage equipment above or
Difference before storage, can first be compressed by the video processing unit VPU of IPIC, to accelerate storage speed；
S900: interleave data are sent to display module by picture processing chip, show corresponding interleave data by display module
Interleave image.
By the image data of above-mentioned high refresh rate, even on different manufacturers, different configuration of display module, all may be used
Show image non-dazzing, without dizziness effect, the impression and experience of user side, which will be promoted, arrives most preferably, such as with 240 frames, 120
The image quality of the displays such as frame, 60 frames improves.
In addition, can be executed according to following methods when forming strengthens view data:
To video image enhancement
Firstly, the image sensing processing module ISP of picture processing chip IPIC will carry out as described above pre- to image
Processing, after pretreatment, retransmits the image processing unit IPU of pretreated image data to picture processing chip IPIC
Make image enhancement, the strengthens view data formed after image enhancement processing will be back to central processing element.
Central processing element carries out Video coding after receiving strengthens view data, and coding protocol can be International Telecommunication Association
H.261, H.263, the MPEG system of the M-JPEG of motor rest motion picture expert group version and Motion Picture Experts Group of International Organization for standardization
Column standard etc., is formed with video data after coding.Since the video data is based on the enhancing formed after picture processing chip processing
Image data, therefore each frame for being formed by video data is the image frame for having executed optimization processing, each frame synthesis
The more colored sense on picture of video data afterwards.
Finally formed video data is recorded by central processing element to storage device memory storage, and subsequent calls are checked altogether.
IPU another way is connect with display module, specifically, enhancing image is sent to data processing unit DPU, by DPU
It increases transparency interactive interface, and is sent to display module loaded and displayed.
The embodiment is to carry out answering for video capture using camera assembly or photographing module based on technical solution of the present invention
With increasing external storage interface (USB, TF etc.) by picture processing chip, when video recording can be stored directly in picture processing chip
In external storage equipment.
Picture is enhanced
The image sensing processing module ISP of picture processing chip IPIC will carry out pretreatment as described above to image,
After pretreatment, the image processing unit IPU for retransmiting pretreated image data to picture processing chip IPIC maps
Image intensifying, the strengthens view data formed after image enhancement processing will be back to central processing element.
Central processing element will compress strengthens view data, such as jpeg encoder jpeg encoder, form pressure
Contract drawing picture.JPEG cataloged procedure needs the rgb format in strengthens view data being converted to yuv format, first in logger computer
When image, most commonly using RGB (red, green, blue) color component come the information that saves colors, such as uncompressed 24
BMP image just uses rgb space to save image.One pixel 24, every 8 save a kind of color intensity (0-255), such as
Red saves as 0xFF0000.YUV is a kind of colour coding method used by eurovision system, China's radio and television
Generally use such methods.Wherein " Y " indicates brightness (Luminance or Luma), that is, grayscale value；And " U " and " V " table
That show is then coloration (Chrominance or Chroma).Color TV uses yuv space precisely in order to being solved with luminance signal Y
The compatibling problem of colour television set and black and white television set makes black and white television set that can also receive colour TV signal.It is converted by RGB
When for YUV, it is usually taken such as Y=0.299R+0.587G+0.114B, U=-0.147R-0.289G+0.436B, V=0.615R-
The mode of 0.515G-0.100B, R=Y+1.14V, G=Y-0.39U-0.58V, B=Y+2.03U carry out.Afterwards by image 8*8 points
Block is sampled image by certain sample format, common format has 4:4 after original image is converted to yuv format:
4,4:2:2 and 4:2:0.After the completion of sampling, image is divided into MCU by 8*8 (pixel).Then just carry out discrete cosine transform
(DCT), discrete cosine transform (Discrete Cosine Transform) is that numeric code rate compression needs a common change
Change coding method.Containing only cosine term in the Fourier transformation of any continuous real symmetric function, thus cosine transform with pay a vertical leaf
Transformation equally has specific physical significance.DCT is that general image is first divided into N*N block of pixels, then to N*N block of pixels one by one into
Row dct transform.Since the high fdrequency component of most of images is smaller, it is often zero corresponding to the coefficient of image high fdrequency component, in addition
Human eye is less sensitive to the distortion of radio-frequency component, so available thicker quantization.Therefore, the numeric code rate for transmitting transformation coefficient is big
It is big to be less than numeric code rate used in transmission image pixel.Sample value is returned to by inverse discrete cosine transformation behind arrival receiving end, although meeting
There is certain distortion, but human eye is acceptable.Picture signal is decomposed into flip-flop；And from low to high
Various cosinusoidal components；And DCT coefficient shows only the fixed portion size of original image signal shared by this kind of ingredient；Obviously, restore figure
As information can be expressed as such a matrix form: F (n)=C (n) * E (n), E (n) is a substrate in formula, and C (n) is DCT
Coefficient, F (n) are then picture signals.If considering further that the variation in vertical direction, it is necessary to a two-dimensional substrate, i.e.,
The substrate will not only reflect the variation of horizontal direction frequency；And to reflect the variation of vertical spatial frequency；Picture corresponding to 8*8
Plain block；Spatial base is image, also commonly referred to as primary image as composed by 64 pixel values.They are called parent map
It seem any different size of group for being expressed as 64 coefficients as block because in the inverse transform of discrete cosine transform
It closes.Since primary image is equivalent to the single coefficient in transform domain, then any pixel can also be regarded as by 64 different width
The combination of the primary image of degree.This is having the same from the combination for the harmonic wave that any signal can resolve into fundamental wave and different amplitudes
Physical significance.Quantizing process is the process of an amplitude discretization by signal, and discrete signal becomes digital letter after quantization
Number.Since HVS is more sensitive to low frequency signal, so the low frequency part to signal uses relatively short quantization step, to signal
High frequency section use relatively long quantization step.Clearly image and higher relatively can be obtained to a certain extent in this way
Compression ratio.Zigzag coding (zigzag scan) is executed later, and the data after quantization are read by zigzag.Finally, using row
Journey length coding (RLE) encodes ac coefficient (AC), and so-called run length coding, RLC refers to that a code can indicate simultaneously
The value and front of code have several zero.The advantages of Z-shaped is read thus has been played, because Z-shaped is read, even zero chance has occurred
Compare it is more, especially to the end, if being all zero, after reading last number, as long as providing " block end " (EOB) code, so that it may
To terminate output, therefore save many code rates.
Finally formed compression image is recorded by central processing element to storage device memory storage, and subsequent calls are checked altogether.
The embodiment is to be carried out image taking based on technical solution of the present invention using camera assembly or photographing module, taken pictures
Application, by picture processing chip increase external storage interface (USB, TF etc.), image procossing can be stored directly in after taking pictures
In the storage equipment of chip exterior.
Preview image is enhanced
After central processing element is connect with picture processing chip, central processing element is by preconfigured for image preview
The interactive interface gui data of preview interface picture processing chip is sent to by DSI interface.DSI defines one and is located at place
The HSSI High-Speed Serial Interface between device and display module is managed, points four layers, PHY layer, Management layers of Lane, Low Level
Protocol layers, Application layers respectively correspond D-PHY, DSI, DCS specification, and wherein PHY defines transmission medium, defeated
Enter/output circuit and with clock and signaling mechanism, Management layers of Lane send and collect data flow to every lane, Low
Protocol layers of Level define how framing and parsing and error detection etc., the high-rise coding of Application layers of description
With parsing data flow, and the interactive interface data include the transparence information of interactive interface, i.e. Alpha information.
Picture processing chip is integrated interactive interface data and strengthens view data by Alphablending, so that increasing
Strong image is shown by display interface of interactive interface, that is, shows an integral image.The integral image will be sent to display by DPU
Module is to show.
The embodiment is mentioning when carrying out image taking using camera assembly or photographing module based on technical solution of the present invention
For the application of preview function to user, by the cooperation with interactive interface, provides a user the vision for being more easily accepted by and being accustomed to and see
Sense.
Refering to Fig. 8, the invention also discloses a kind of image processing systems, including display module, the figure being set in a terminal
As sensor, picture processing chip and central processing element；Picture processing chip respectively with display module, imaging sensor and in
Centre processing chip connection；The image data of imaging sensor acquisition reference object is simultaneously sent to picture processing chip；Image procossing
The digital signal processing module of chip handles the initial data of image data, forms optimization image data；Digital signal
Processing module will optimize image data and be sent to the image sensing processing module of picture processing chip to optimization image data progress
Processing forms strengthens view data；Picture processing chip will optimize image data and/or strengthens view data is back to an intelligence
Central processing element in terminal, by central processing element record optimization image data and/or strengthens view data to a storage
Equipment, and picture processing chip will optimize image data and/or strengthens view data is sent to a display module, by display module
The optimization image and/or enhancing image of the corresponding optimization image data of display and/or strengthens view data.
Preferably, digital signal processing module carries out grafting to initial data and mends frame, forms multiframe composograph data；Or
Digital signal processing module handles initial data rendering, forms optimization image data.
In addition, being based on above-mentioned image processing method, computer readable storage medium can be installed in intelligent terminal, deposited thereon
Computer program is contained, when the computer program is executed by processor, realizes the step of image processing method as described above
Suddenly.
Intelligent terminal can be implemented in a variety of manners.For example, terminal described in the present invention may include such as moving
Phone, smart phone, laptop, PDA (personal digital assistant), PAD (tablet computer), PMP (put by portable multimedia broadcasting
Device), the fixed terminal of the intelligent terminal of navigation device etc. and such as number TV, desktop computer etc..Hereinafter it is assumed that eventually
End is intelligent terminal.However, it will be understood by those skilled in the art that other than the element for being used in particular for mobile purpose, root
It can also apply to the terminal of fixed type according to the construction of embodiments of the present invention.
It should be noted that the embodiment of the present invention has preferable implementation, and not the present invention is made any type of
Limitation, any one skilled in the art change or are modified to possibly also with the technology contents of the disclosure above equivalent effective
Embodiment, as long as without departing from the content of technical solution of the present invention, it is to the above embodiments according to the technical essence of the invention
Any modification or equivalent variations and modification, all of which are still within the scope of the technical scheme of the invention.
Claims (10)
1. a kind of image processing method, which comprises the following steps:
S100: the image data of imaging sensor acquisition reference object；
S200: the initial data of described image data is sent to picture processing chip by imaging sensor, is handled by described image
The digital signal processing module of chip forms optimization image data to the original data processing；
S300: the digital signal processing module passes the image that the optimization image data is sent to described image processing chip
Sense processing module handles the optimization image data, forms strengthens view data；
S400: described image handles chip and the optimization image data and/or strengthens view data is back to an intelligent terminal
Interior central processing element records the optimization image data and/or strengthens view data to one by the central processing element
Store equipment；
S500: described image handles chip and the optimization image data and/or strengthens view data is sent to a display module,
By the optimization image and/or enhancing figure of the corresponding optimization image data of display module display and/or strengthens view data
Picture.
2. image processing method as described in claim 1, which is characterized in that
The step S200 includes:
S210: described image data are sent to image procossing core with raw data format by camera interface by imaging sensor
Piece；
S220: described image handles the image sensing module of chip to the original data processing, forms optimization image data.
3. image processing method as claimed in claim 2, which is characterized in that
The step S220 includes:
S221: described image sensing module will extract the image frame in the initial data；
S222: synthesizing the image frame or grafting mends frame to the image frame, forms multiframe composograph data.
4. image processing method as claimed in claim 2, which is characterized in that
The step S220 includes:
S221 ': described image sensing module judges the vision environment of the initial data；
S222 ': being handled the initial data rendering for the vision environment, forms optimization image data.
5. image processing method as claimed in claim 2, which is characterized in that
The step S300 includes:
S310: the digital signal processing module forwards the initial data to central processing element；
S320: the intelligent object of the central processing element handles the initial data rendering, forms optimization image data；
S330: the intelligent object sends the optimization image data to processing module.
6. image processing method as described in claim 1, which is characterized in that
Described image processing method further include:
S600: the frame per second data of described image data are sent to picture processing chip by imaging sensor；
S700: described image handles chip and makees interleave compensation to the frame per second data, forms interleave data, the interleave data with
The refresh rate of the display module of the intelligent terminal matches；
S800: compressing the interleave data, and stores to storage equipment；
S900: described image handles chip and the interleave data is sent to display module, is shown and is corresponded to by the display module
The interleave image of interleave data.
7. image processing method as claimed in claim 6, which is characterized in that
The frame per second of the frame per second data is more than or equal to 60fps.
8. a kind of image processing system, which is characterized in that including display module, the imaging sensor, image being set in a terminal
Handle chip and central processing element；
Described image processing chip is connect with the display module, imaging sensor and central processing element respectively；
The image data of described image sensor acquisition reference object is simultaneously sent to picture processing chip；
The digital signal processing module of described image processing chip handles the initial data of described image data, is formed excellent
Change image data；
The optimization image data is sent at the image sensing of described image processing chip by the digital signal processing module
Reason module handles the optimization image data, forms strengthens view data；
Described image is handled during the optimization image data and/or strengthens view data be back in an intelligent terminal by chip
Processing chip in centre records the optimization image data and/or strengthens view data to a storage by the central processing element and sets
It is standby, and the optimization image data and/or strengthens view data are sent to a display module by described image processing chip, by institute
State the optimization image and/or enhancing figure of the corresponding optimization image data of display module display and/or strengthens view data
Picture.
9. image processing system as claimed in claim 8, which is characterized in that
The digital signal processing module carries out grafting to the initial data and mends frame, forms multiframe composograph data；Or
The digital signal processing module handles the initial data rendering, forms optimization image data.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program
The step of image processing methods as described in any item such as claim 1-7 are realized when being executed by processor.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811145822.3A CN109286753A (en) | 2018-09-29 | 2018-09-29 | Image processing method, system and computer readable storage medium |
| PCT/CN2019/107194 WO2020063507A1 (en) | 2018-09-29 | 2019-09-23 | Image processing method and system, and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811145822.3A CN109286753A (en) | 2018-09-29 | 2018-09-29 | Image processing method, system and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109286753A true CN109286753A (en) | 2019-01-29 |
Family
ID=65182139
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811145822.3A Pending CN109286753A (en) | 2018-09-29 | 2018-09-29 | Image processing method, system and computer readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109286753A (en) |
| WO (1) | WO2020063507A1 (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110379358A (en) * | 2019-07-04 | 2019-10-25 | 南京宇丰晔禾信息科技有限公司 | LED display image broadcast control method, device |
| WO2020063507A1 (en) * | 2018-09-29 | 2020-04-02 | 上海众链科技有限公司 | Image processing method and system, and computer readable storage medium |
| WO2020063506A1 (en) * | 2018-09-29 | 2020-04-02 | 上海众链科技有限公司 | Smart terminal, image processing method and computer-readable storage medium |
| WO2020063505A1 (en) * | 2018-09-29 | 2020-04-02 | 上海众链科技有限公司 | Image processing method, system, and computer readable storage medium |
| CN113744139A (en) * | 2020-05-29 | 2021-12-03 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN113810593A (en) * | 2020-06-15 | 2021-12-17 | Oppo广东移动通信有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
| CN113873143A (en) * | 2020-06-30 | 2021-12-31 | Oppo广东移动通信有限公司 | Multimedia processing chip and electronic equipment |
| CN113873142A (en) * | 2020-06-30 | 2021-12-31 | Oppo广东移动通信有限公司 | Multimedia processing chip, electronic device and dynamic image processing method |
| WO2022000347A1 (en) * | 2020-06-30 | 2022-01-06 | 深圳市大疆创新科技有限公司 | Image processing method, display processing apparatus, and computer readable storage medium |
| CN114025079A (en) * | 2021-09-29 | 2022-02-08 | 大连中科创达软件有限公司 | Method, device and system for processing image quality optimization parameters |
| CN114071036A (en) * | 2020-08-10 | 2022-02-18 | Oppo广东移动通信有限公司 | Multimedia processing chip, electronic device and dynamic image data processing method |
| CN114286003A (en) * | 2021-12-28 | 2022-04-05 | 维沃移动通信有限公司 | Shooting method, shooting device and electronic equipment |
| CN114285957A (en) * | 2021-12-28 | 2022-04-05 | 维沃移动通信有限公司 | Image processing circuit and data transmission method |
| CN114339071A (en) * | 2021-12-28 | 2022-04-12 | 维沃移动通信有限公司 | Image processing circuit, image processing method and electronic device |
| CN114338955A (en) * | 2021-12-28 | 2022-04-12 | 维沃移动通信有限公司 | Image processing circuit, image processing method, image processing device, electronic equipment and chip |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101192136A (en) * | 2006-11-30 | 2008-06-04 | 北京思比科微电子技术有限公司 | Image transfer method and apparatus based on USB mode |
| CN102231098A (en) * | 2008-01-10 | 2011-11-02 | Lg电子株式会社 | Data processing unit with multi-graphic controller and method for processing data using the same |
| CN102883167A (en) * | 2012-09-19 | 2013-01-16 | 旗瀚科技有限公司 | Video image data processing method and system |
| CN203675199U (en) * | 2013-12-31 | 2014-06-25 | 冠捷显示科技（厦门）有限公司 | Television set capable of upgrading software and hardware performance |
| US20160295127A1 (en) * | 2015-04-02 | 2016-10-06 | Ultracker Technology Co., Ltd. | Real-time image stitching apparatus and real-time image stitching method |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104869381B (en) * | 2014-02-25 | 2017-07-25 | 炬芯（珠海）科技有限公司 | A kind of image processing system, method and device |
| US20150350504A1 (en) * | 2014-06-03 | 2015-12-03 | 2P & M Holdings, LLC | RAW Camera Peripheral for Handheld Mobile Unit |
| CN205375584U (en) * | 2016-01-04 | 2016-07-06 | 临沂大学 | Independent image acquisition system in computer |
| CN106815077B (en) * | 2016-12-30 | 2020-03-17 | Oppo广东移动通信有限公司 | Data processing method based on intelligent device architecture and intelligent device |
| CN109167916A (en) * | 2018-09-29 | 2019-01-08 | 南昌黑鲨科技有限公司 | intelligent terminal, image processing method and computer readable storage medium |
| CN109242757A (en) * | 2018-09-29 | 2019-01-18 | 南昌黑鲨科技有限公司 | intelligent terminal, image processing method and computer readable storage medium |
| CN109167915A (en) * | 2018-09-29 | 2019-01-08 | 南昌黑鲨科技有限公司 | Image processing method, system and computer readable storage medium |
| CN109286753A (en) * | 2018-09-29 | 2019-01-29 | 南昌黑鲨科技有限公司 | Image processing method, system and computer readable storage medium |
-
2018
- 2018-09-29 CN CN201811145822.3A patent/CN109286753A/en active Pending
-
2019
- 2019-09-23 WO PCT/CN2019/107194 patent/WO2020063507A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101192136A (en) * | 2006-11-30 | 2008-06-04 | 北京思比科微电子技术有限公司 | Image transfer method and apparatus based on USB mode |
| CN102231098A (en) * | 2008-01-10 | 2011-11-02 | Lg电子株式会社 | Data processing unit with multi-graphic controller and method for processing data using the same |
| CN102883167A (en) * | 2012-09-19 | 2013-01-16 | 旗瀚科技有限公司 | Video image data processing method and system |
| CN203675199U (en) * | 2013-12-31 | 2014-06-25 | 冠捷显示科技（厦门）有限公司 | Television set capable of upgrading software and hardware performance |
| US20160295127A1 (en) * | 2015-04-02 | 2016-10-06 | Ultracker Technology Co., Ltd. | Real-time image stitching apparatus and real-time image stitching method |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020063507A1 (en) * | 2018-09-29 | 2020-04-02 | 上海众链科技有限公司 | Image processing method and system, and computer readable storage medium |
| WO2020063506A1 (en) * | 2018-09-29 | 2020-04-02 | 上海众链科技有限公司 | Smart terminal, image processing method and computer-readable storage medium |
| WO2020063505A1 (en) * | 2018-09-29 | 2020-04-02 | 上海众链科技有限公司 | Image processing method, system, and computer readable storage medium |
| CN110379358A (en) * | 2019-07-04 | 2019-10-25 | 南京宇丰晔禾信息科技有限公司 | LED display image broadcast control method, device |
| CN110379358B (en) * | 2019-07-04 | 2021-03-30 | 南京宇丰晔禾信息科技有限公司 | LED display screen image playing and controlling method and device |
| CN113744139A (en) * | 2020-05-29 | 2021-12-03 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN113810593A (en) * | 2020-06-15 | 2021-12-17 | Oppo广东移动通信有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
| CN113873142A (en) * | 2020-06-30 | 2021-12-31 | Oppo广东移动通信有限公司 | Multimedia processing chip, electronic device and dynamic image processing method |
| CN113873143A (en) * | 2020-06-30 | 2021-12-31 | Oppo广东移动通信有限公司 | Multimedia processing chip and electronic equipment |
| WO2022000347A1 (en) * | 2020-06-30 | 2022-01-06 | 深圳市大疆创新科技有限公司 | Image processing method, display processing apparatus, and computer readable storage medium |
| CN113873143B (en) * | 2020-06-30 | 2023-07-25 | Oppo广东移动通信有限公司 | Multimedia processing chip and electronic equipment |
| CN114071036A (en) * | 2020-08-10 | 2022-02-18 | Oppo广东移动通信有限公司 | Multimedia processing chip, electronic device and dynamic image data processing method |
| CN114025079A (en) * | 2021-09-29 | 2022-02-08 | 大连中科创达软件有限公司 | Method, device and system for processing image quality optimization parameters |
| CN114025079B (en) * | 2021-09-29 | 2024-02-13 | 大连中科创达软件有限公司 | Image quality optimization parameter processing method, device and system |
| CN114286003A (en) * | 2021-12-28 | 2022-04-05 | 维沃移动通信有限公司 | Shooting method, shooting device and electronic equipment |
| CN114285957A (en) * | 2021-12-28 | 2022-04-05 | 维沃移动通信有限公司 | Image processing circuit and data transmission method |
| CN114339071A (en) * | 2021-12-28 | 2022-04-12 | 维沃移动通信有限公司 | Image processing circuit, image processing method and electronic device |
| CN114338955A (en) * | 2021-12-28 | 2022-04-12 | 维沃移动通信有限公司 | Image processing circuit, image processing method, image processing device, electronic equipment and chip |
| WO2023125659A1 (en) * | 2021-12-28 | 2023-07-06 | 维沃移动通信有限公司 | Image processing circuit and data transmission method |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020063507A1 (en) | 2020-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109286753A (en) | Image processing method, system and computer readable storage medium | |
| CN109242757A (en) | intelligent terminal, image processing method and computer readable storage medium | |
| CN109167915A (en) | Image processing method, system and computer readable storage medium | |
| CN109167916A (en) | intelligent terminal, image processing method and computer readable storage medium | |
| US7483486B2 (en) | Method and apparatus for encoding high dynamic range video | |
| CN115242992B (en) | Video processing method, device, electronic equipment and storage medium | |
| CN113810641B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN105491358B (en) | A kind of image processing method and device, terminal | |
| US9197813B2 (en) | Method and apparatus for obtaining a digital image | |
| CN113810642B (en) | Video processing method and device, electronic equipment and storage medium | |
| US8514254B2 (en) | Apparatus and method for processing digital images | |
| CN113824914B (en) | Video processing method and device, electronic equipment and storage medium | |
| JP2002016870A (en) | Imaging apparatus | |
| US10674163B2 (en) | Color space compression | |
| WO2023016040A1 (en) | Video processing method and apparatus, electronic device, and storage medium | |
| EP4287604A1 (en) | Video processing method and apparatus, electronic device, and storage medium | |
| CN115706767B (en) | Video processing method, device, electronic equipment and storage medium | |
| CN115706766B (en) | Video processing method, device, electronic equipment and storage medium | |
| CN115706863B (en) | Video processing method, device, electronic equipment and storage medium | |
| US20080048961A1 (en) | Method for improving image quality differences on an LCD due to different viewing modules | |
| JP3882270B2 (en) | Imaging device and storage medium | |
| JPH01191596A (en) | Orthogonal transform coding device for image signal | |
| JP2002152775A (en) | Method and device for encoding | |
| JP2001331163A (en) | Color image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190129 |
|
| RJ01 | Rejection of invention patent application after publication |