CN106572290B - A kind of processing method and mobile terminal of image data - Google Patents
A kind of processing method and mobile terminal of image data Download PDFInfo
- Publication number
- CN106572290B CN106572290B CN201610921869.9A CN201610921869A CN106572290B CN 106572290 B CN106572290 B CN 106572290B CN 201610921869 A CN201610921869 A CN 201610921869A CN 106572290 B CN106572290 B CN 106572290B
- Authority
- CN
- China
- Prior art keywords
- electric
- image
- signal
- buffer area
- image signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The embodiment of the invention provides a kind of processing method of image data and mobile terminal, the mobile terminal includes camera module, is arranged and the camera module is provided with imaging sensor, which comprises acquisition incident ray；Described image sensor, which incudes incident ray and converts, forms electric image signal；Described image electric signal is exported, includes RGB image electric signal and IR electric image signal in described image electric signal；For described image electric signal, the first buffer area and second buffering area are set；The RGB image electric signal is saved in the first buffer area, and, the IR electric image signal is saved in the second buffer area；RGB image electric signal is obtained from first buffer area, and, IR electric image signal is obtained from second buffer area；Target image is obtained according to the RGB image electric signal and the IR electric image signal.The embodiment of the present invention is to solve the problems, such as that dual camera excessively occupies mobile terminal space.
Description
Technical field
The present invention relates to fields of communication technology, and the processing method and a kind of movement more particularly to a kind of image data are eventually
End.
Background technique
With the continuous development of electronic products, the mobile terminal with shooting function is more more and more universal, and user can be at any time
It is shot everywhere with it.In order to obtain better shooting effect, it is equipped on some mobile terminals on the market at present double
Rear camera.
Double rear cameras refer to that there are two camera modules for installation on the mobile terminal back side.Two camera modules can
To improve the shooting effect of mobile terminal.However, being executed since two camera modules in double rear cameras are independent mutually
It is independent of each other when shooting work, in other words, two camera modules respectively need the component of complete set, such as camera lens, figure
As sensor (sensor), microscope base, motor, substrate etc..It is appreciated that the camera module of this structure is on mobile terminals
The space occupied can be increased, and the certain functions of may result in mobile terminal are restricted, or damage mobile terminal
It is beautiful.
Summary of the invention
The embodiment of the present invention provides the processing method and mobile terminal of a kind of image data, is excessively accounted for solving dual camera
The problem of with mobile terminal space.
In a first aspect, providing a kind of method of the processing of image data, the method is applied to mobile terminal, the side
Method includes:
Acquire incident ray；
Described image sensor, which incudes incident ray and converts, forms electric image signal；
Described image electric signal is exported, includes RGB image electric signal and IR electric image signal in described image electric signal；
For described image electric signal, the first buffer area and second buffering area are set；
The RGB image electric signal is saved in the first buffer area, and, the IR electric image signal is saved in
Two buffer areas；
RGB image electric signal is obtained from first buffer area, and, IR image is obtained from second buffer area
Electric signal；
Target image is obtained according to the RGB image electric signal and the IR electric image signal.
Second aspect provides a kind of mobile terminal, and the mobile terminal includes camera module, in the camera mould
Group is provided with imaging sensor, and the mobile terminal includes:
Incident ray acquisition module, for acquiring incident ray；
Electric image signal obtains module, incudes incident ray for described image sensor and conversion forms image telecommunications
Number；
Electric image signal output module includes RGB image in described image electric signal for exporting described image electric signal
Electric signal and IR electric image signal；
Buffer area setup module, for the first buffer area and second buffering area to be arranged for described image electric signal；
Electric image signal preserving module, for the RGB image electric signal to be saved in the first buffer area, and, by institute
It states IR electric image signal and is saved in the second buffer area；
Electric image signal obtains module, for the acquisition RGB image electric signal from first buffer area, and, from institute
State acquisition IR electric image signal in the second buffer area；
Target image generation module, for obtaining target according to the RGB image electric signal and the IR electric image signal
Image.
In this way, in the embodiment of the present invention, when incident ray projects camera module, image sensor senses incident light
Line and convert form electric image signal, be then output in image signal process again；And it is additionally provided with the first buffer area and
Distinguish if the image signal process electric image signal may separate out RGB image electric signal and IR electric image signal two buffer areas
RGB image electric signal and IR electric image signal are saved in the first buffer area and second buffering area；When required, it can be based on
Can individually using or combination target image obtained using rgb image data and IR image data.Using the embodiment of the present invention,
Camera module there are two being equivalent in same camera module, can independently export two-way image data.In addition, of the invention
Embodiment can realize function possessed by dual camera mould group by a camera module, and this simple structure is not
First dual camera mould group the space occupied on mobile terminals is only effectively saved, and dual camera mould can also be retained simultaneously
Function possessed by group.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below by institute in the description to the embodiment of the present invention
Attached drawing to be used is needed to be briefly described, it should be apparent that, the accompanying drawings in the following description is only some implementations of the invention
Example, for those of ordinary skill in the art, without any creative labor, can also be according to these attached drawings
Obtain other attached drawings.
Fig. 1 is a kind of step flow chart of the processing method embodiment of image data of the embodiment of the present invention one；
Fig. 2 is a kind of step flow chart of the processing method embodiment of image data of the embodiment of the present invention two；
Fig. 3 is a kind of structural block diagram of mobile terminal of the embodiment of the present invention three；
Fig. 3 a is a kind of one of the structural block diagram of mobile terminal of the embodiment of the present invention three；
Fig. 3 b is the two of the structural block diagram of a kind of mobile terminal of the embodiment of the present invention three；
Fig. 3 c is the three of the structural block diagram of a kind of mobile terminal of the embodiment of the present invention three；
Fig. 3 d is the four of the structural block diagram of a kind of mobile terminal of the embodiment of the present invention three；
Fig. 4 is a kind of block diagram of mobile terminal of the embodiment of the present invention four；
Fig. 5 is a kind of structural schematic diagram of mobile terminal of the embodiment of the present invention five.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair
Embodiment in bright, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, shall fall within the protection scope of the present invention.
Embodiment one
Fig. 1 is please referred to, is a kind of step flow chart of the processing method of image data of the embodiment of the present invention one.
A kind of processing method of image data provided in an embodiment of the present invention is applied to mobile terminal, the mobile terminal
May include camera module, can be set imaging sensor in the camera module, this method can specifically include as
Lower step:
Step 101, incident ray is acquired.
It in the concrete realization, include camera lens in camera module, when light is reflected and is incident on from subject
When camera module, the incident ray of the camera lens acquisition subject in the camera module is used for subsequent processing operation.
Step 102, described image sensor, which incudes incident ray and converts, forms electric image signal.
In embodiments of the present invention, incident ray can be converted to image telecommunications by the imaging sensor in camera module
Number.
Imaging sensor, also referred to as photosensitive element are a kind of equipment for converting optical signals into electronic signal, are the present invention
One of the important component of camera module that embodiment provides.Imaging sensor provided in an embodiment of the present invention includes CCD
(Charge Coupled Device, charge coupled cell) and CMOS (Complementary Metal-Oxide
Semiconductor, metal oxide semiconductor device) two major classes.Specifically, above-mentioned CCD provides good picture quality, resists
Flexibility when ability of making an uproar and camera module design；But it needs to increase external circuit and makes becoming large-sized, being complicated for system
Property improve, needed in circuit design it is more flexible, to promote the performance of CCD camera mould group as far as possible.Above-mentioned CCD is more suitable for
It is such as astronomical, high-definition in the application field very high and less strict to cost control to camera module performance requirement
Medical X-ray image, other need time exposure and the scientific application stringent to picture noise requirement.
Above-mentioned CMOS is that have using contemporary large-scale semiconductive integrated circuit production technology come the imaging sensor produced
The features such as high yield rate, integrated level height, small power consumption, low price.Level of the CMOS in terms of image quality can be with CCD technology phase
Trial of strength, and they are more suitably applied to require space small, small in size, low in energy consumption and are not special to picture noise and quality requirement
High occasion.Such as most of industrial detection application for having filler lighting, security protection security personnel application and most of consumption-orientation business numbers
Code camera applications.
The imaging sensor of camera module on mobile terminals mainly uses CMOS as imaging sensor at present, below
CMOS is introduced.
CMOS is a kind of typical solid state image sensor, usually by pixel array, line driver, row driver, timing
A few part compositions such as control logic, A/D converter, data/address bus output interface, control interface, this several part is usually all integrated
On same silicon wafer.
CMOS basic functional principle are as follows: firstly, extraneous incident ray irradiates pixel array, photoelectric effect occurs, in picture
Corresponding charge is generated in the pixel unit of pixel array to be converted to electric signal.The row selection logic unit of CMOS is according to need
It wants, gates corresponding row pixel unit.Electric image signal in row pixel unit is transmitted by the signal bus of respective column
To corresponding analogy signal processing unit and A/D converter, led to after being converted into digital picture electric signal by output accordingly
Road output.Row selection logic unit therein can progressively scan pixel array can also interlacing scan.Row selection logic unit
The window abstraction function that image may be implemented is used cooperatively with column selection logic unit.
Certainly, it can also be passed in embodiments of the present invention using other images in addition to CMOS and ccd image sensor
Sensor, as long as can be realized the acquisition of electric image signal, the embodiments of the present invention are not limited thereto.
Step 103, described image electric signal is exported, includes RGB image electric signal and IR image in described image electric signal
Electric signal.
After described image sensor incudes incident ray and conversion forms electric image signal, which is exported
To the ISP (Image Signal Processing, electric image signal processor) of mobile terminal, ISP will based on preset algorithm from
RGB image electric signal and IR electric image signal are isolated in electric image signal.
Step 104, the first buffer area and second buffering area are set for described image electric signal.
In embodiments of the present invention, described image sensor is correspondingly provided with two buffer areas for electric image signal,
Including the first buffer area for storing RGB image electric signal and the second buffer area for storing IR electric image signal.Certainly,
If only forming a kind of electric image signal, a buffer area can also be only set.
Step 105, the RGB image electric signal is saved in the first buffer area, and, by the IR electric image signal
It is saved in the second buffer area.
Specifically, RGB image electric signal can be saved in the first buffer area, and IR electric image signal is saved in
Second buffer area can not be subject to this according to the practical preservation for carrying out image data, the embodiment of the present invention in a particular application
Limitation.
Step 106, RGB image electric signal is obtained from first buffer area, and, it is obtained from second buffer area
Take IR electric image signal.
When needing image data, RGB image electric signal can be obtained from the first buffer area, alternatively, from the second caching
IR electric image signal is obtained in area, alternatively, obtaining in RGB image electric signal and the second buffer area from the first buffer area simultaneously
Obtain IR electric image signal.
Step 107, target image is obtained according to the RGB image electric signal and the IR electric image signal.
It, can be according to reality for obtained RGB image electric signal and IR electric image signal in concrete application of the invention
The corresponding electric image signal of border demand selection individually carries out handling or synthesis is handled, to obtain required target image.
Based on obtained target image, such as normal image shooting may be implemented, with aware space and the depth of field, iris, make 3D rendering
Etc. functions.
In this way, in the embodiment of the present invention, when incident ray projects camera module, image sensor senses incident light
Line and convert form electric image signal, be then output in image signal process again；And it is additionally provided with the first buffer area and
Distinguish if the image signal process electric image signal may separate out RGB image electric signal and IR electric image signal two buffer areas
RGB image electric signal and IR electric image signal are saved in the first buffer area and second buffering area；When required, it can be based on
Can individually using or combination target image obtained using rgb image data and IR image data.Using the embodiment of the present invention,
Camera module there are two being equivalent in same camera module, can independently export two-way image data.In addition, of the invention
Embodiment can realize function possessed by dual camera mould group by a camera module, and this simple structure is not only
First dual camera mould group the space occupied on mobile terminals is effectively saved, and dual camera mould group can also be retained simultaneously
Possessed function.
Embodiment two
Referring to figure 2., be the embodiment of the present invention two a kind of image data processing method step flow chart.
A kind of processing method of image data provided in an embodiment of the present invention is applied to mobile terminal, the mobile terminal
It may include camera module, be arranged and imaging sensor can be set in the camera module, this method specifically can wrap
Include following steps:
Step 201, incident ray is acquired.
Step 202, described image sensor, which incudes incident ray and converts, forms electric image signal.
Step 203, described image electric signal is exported, includes RGB image electric signal and IR image in described image electric signal
Electric signal.
Since the step 201-203 of embodiment two is similar with embodiment of the method one, therefore the present embodiment is for step 201-203
Description in not detailed place, may refer to the related description in preceding method embodiment one, i will not repeat them here.
Step 204, the first buffer area and second buffering area are set for the type amount and data capacity of electric image signal.
In one preferred embodiment of the invention, the step 204 may include following sub-step:
Determine the number of species that described image electric signal includes, preferably RGB image electric signal and IR electric image signal this
Two kinds；
Determine the data capacity of every kind of electric image signal；
According to the data volume of default first rule and the RGB image electric signal, the first buffer area is set；
According to the data volume of default Second Rule and the IR electric image signal, the second buffer area is set.
It in embodiments of the present invention, can be according to two pieces of buffer areas (buffer) of the rule setting set, for example, by it
In a buffer area be set as consistent with the pixel size of original image electric signal, another is set as original image electric signal
The size of pixel 1/4.
Step 205, the RGB image electric signal is saved in the first buffer area, and, the IR electric image signal is protected
It is stored to the second buffer area.
In a kind of concrete application, the data volume as shared by usual RGB image electric signal is larger, so can be used compared with
Big buffer area stores RGB image electric signal, stores IR electric image signal using lesser buffer area.
In one preferred embodiment of the invention, the RGB image electric signal may include R component, G component and B points
Amount, wherein the R component, G component and B component refer to: under RGB mode, every kind of RGB ingredient uses (white to 255 from 0 (black)
Color) value indicate different degrees of color value；The step 205 may include following sub-step:
Judge in described image electric signal whether to include IR electric image signal line by line；
If in electric image signal not including IR electric image signal, the electric image signal of the row is saved in the first caching
Area；
If in electric image signal including IR electric image signal, the IR electric image signal of the row is substituted using G component, and will
The electric image signal of the row is saved in the first buffer area after substitution；And the IR electric image signal of the row is saved in second and is delayed
Rush area.
In one preferred embodiment of the invention, the IR electric image signal that the row is substituted using G component, and will replace
The step of electric image signal of the row is saved in the first buffer area after generation may include:
From the G component extracted in described image electric signal around the IR electric image signal；
Average G component is calculated based on the G component around described；
The IR electric image signal of the row is substituted using the average G component；
The row electric image signal after substitution is saved in the first buffer area.
In the concrete realization, electric image signal processor is with behavior unit recipient electric image signal.At electric image signal
It, will be whole if the row being currently received does not have IR electric image signal when reason device receives the image data of imaging sensor conversion
Row electric image signal is put in larger buffer area；If the row being currently received has IR electric image signal, IR electric image signal
It can be replaced with G component, and be saved in larger buffer area.Wherein, the intensity of G component can be by 4 G components around electric image signal
The average G component being averagely calculated.Meanwhile also IR electric image signal being saved in compared in minibuffer area, so
Two individual rgb image datas and IR image data have just been obtained, such as:
Step 206, RGB image electric signal is obtained from first buffer area, and, it is obtained from second buffer area
Take IR electric image signal.
In the concrete realization, RGB image electric signal, IR image telecommunications can be extracted from buffer area according to actual needs
Number or RGB image electric signal and IR electric image signal；Then it is calculated again based on extracted electric signal.
Step 207, colored target RGB image is obtained based on the RGB image electric signal, or is based on the IR image
Electric signal obtains the target IR image of black and white, or the combination based on the RGB image electric signal and the IR electric image signal
Obtain target mixed image.
The camera module of the embodiment of the present invention can collect two-way electric image signal, i.e. RGB image electric signal and IR
Electric image signal, the camera module of an embodiment of the present invention are able to achieve the function of two cameras, have the advantage that including
There are apparent shooting, shallow depth of field shooting and more preferably low light performance etc..
Specifically, the first function of camera module of the embodiment of the present invention is that the shallow depth of field is supported to take pictures, and realizes that background is empty
Change, prominent photograph subject；Second function is that have better performance under low luminous environment.Camera mould through the embodiment of the present invention
Group can analyze which pixel to scene reduction degree it is relatively high, this can help mobile terminal effectively to inhibit noise, thus
So that image frame possesses more preferably image quality reduction.
For example, the embodiment of the present invention obtains conventional color image using RGB image electric signal, due to eliminating IR image
Electric signal, so that rubescent will not be distorted according to the image of RGB image electric signal；Black and white can also be obtained using IR electric image signal
Image, the characteristic based on infrared ray, resulting black white image can be with aware spaces and the depth of field；Further, acceptable while benefit
Mixed image is obtained with RGB image electric signal and IR electric image signal, in conjunction with the image of the two characteristic, can be and be able to reflect sky
Between information 3D rendering.
Certainly, RGB image electric signal and IR electric image signal are isolated for the camera module of the embodiment of the present invention,
It can be applied according to actual needs, the embodiments of the present invention are not limited thereto.
In embodiments of the present invention, the image telecommunications exported by imaging sensor in the camera module of mobile terminal
Number, transfer to electric image signal processor to isolate RGB image electric signal and IR electric image signal, so that in a camera module
It can realize the function of dual camera.
Embodiment three
Referring to figure 3., be the embodiment of the present invention three a kind of mobile terminal 300 structural block diagram.
A kind of mobile terminal provided in an embodiment of the present invention, the mobile terminal include camera module, in the camera shooting
Head mould group is provided with imaging sensor, and the mobile terminal includes:
Incident ray acquisition module 301, for acquiring incident ray.
Electric image signal obtains module 302, incudes incident ray for described image sensor and conversion forms image electricity
Signal.
Electric image signal output module 303 includes RGB in described image electric signal for exporting described image electric signal
Electric image signal and IR electric image signal.
Buffer area setup module 304, for the first buffer area and second buffering area to be arranged for described image electric signal.
Electric image signal preserving module 305, for the RGB image electric signal to be saved in the first buffer area, and, it will
The IR electric image signal is saved in the second buffer area.
Electric image signal obtains module 306, for the acquisition RGB image electric signal from first buffer area, and, from
IR electric image signal is obtained in second buffer area.
Target image generation module 307, for obtaining mesh according to the RGB image electric signal and the IR electric image signal
Logo image.
Referring to shown in Fig. 3 a, in a kind of mobile terminal 300 of the embodiment of the present invention, the buffer area setup module 304 is wrapped
It includes number of species and determines submodule 3041, data capacity determines submodule 3042, and submodule 3043 and the is arranged in the first buffer area
Submodule 3044 is arranged in two buffer areas.
Number of species determine submodule 3041, the number of species for including for determining described image electric signal, the type
Quantity includes RGB image electric signal and IR electric image signal；
Data capacity determines submodule 3042, for determining the data capacity of every kind of electric image signal；
Submodule 3043 is arranged in first buffer area, for the number according to default first rule and the RGB image electric signal
According to capacity, the first buffer area is set；
Submodule 3044 is arranged in second buffer area, for the data according to default Second Rule and the IR electric image signal
The second buffer area is arranged in capacity.
Referring to shown in Fig. 3 b, in a kind of mobile terminal 300 of the embodiment of the present invention, described image electric signal preserving module
305 include judging submodule 3051, and first, which saves submodule 3052 and second, saves submodule 3053.
Judging submodule 3051, for judging in described image electric signal whether to include IR electric image signal line by line；
First saves submodule 3052, if for not including IR electric image signal in electric image signal, by the figure of the row
As electric signal is saved in the first buffer area；
Second saves submodule 3053, if being substituted for including IR electric image signal in electric image signal using G component
The IR electric image signal of the row, and the electric image signal of the row after substitution is saved in the first buffer area；And by the institute of the row
It states IR electric image signal and is saved in second buffering area.
Referring to shown in Fig. 3 c, in a kind of mobile terminal 300 of the embodiment of the present invention, described second, which saves submodule 3053, is wrapped
G component extraction unit 30531 is included, average G component calculation unit 30532, IR electric image signal substituting unit 30533 and first delays
Memory cell 43053.
G component extraction unit 30531, for from the G extracted in described image electric signal around the IR electric image signal
Component；
Average G component calculation unit 30532 calculates average G component for the G component based on the surrounding；
IR electric image signal substituting unit 30533, for substituting the IR image telecommunications of the row using the average G component
Number；
First cache unit 43053, for the row electric image signal after substitution to be saved in the first buffer area.
Referring to shown in Fig. 3 d, in a kind of mobile terminal 300 of the embodiment of the present invention, the target image generation module 307
Submodule 3071 is obtained including target RGB image, target IR image obtains submodule 3072 and target mixed image obtains submodule
Block 3073.
Target RGB image obtains submodule 3071, for obtaining colored target RGB based on the RGB image electric signal
Image；
Target IR image obtains submodule 3072, and the target IR for being obtained black and white based on the IR electric image signal is schemed
Picture；
Target mixed image obtains submodule 3073, for being based on the RGB image electric signal and the IR image telecommunications
Number combination obtain target mixed image.
For 300 embodiment of mobile terminal, since it is basically similar to the method embodiment, so the comparison of description is simple
Single, the relevent part can refer to the partial explaination of embodiments of method.
In this way, when incident ray projects camera module, can be based on imaging sensor will in the embodiment of the present invention
Incident ray is converted to electric image signal, is then output to again in electric image signal processing, and be additionally provided with the first buffer area
And second buffering area, if electric image signal handles the electric image signal and isolates RGB image electric signal and IR electric image signal,
RGB image electric signal and IR electric image signal are saved in the first buffer area and second buffering area respectively, it when required, can be with
Target image is obtained based on rgb image data and IR image data.Using the embodiment of the present invention, in same camera module
Camera module there are two being equivalent to, can independently export two-way image data.In addition, the embodiment of the present invention passes through a camera shooting
Head mould group can realize that dual camera mould group, this simple structure not only effectively save first dual camera mould group in movement
The space occupied in terminal, and function possessed by dual camera mould group can also be retained simultaneously.
Example IV
Fig. 4 is the block diagram of the mobile terminal 700 of another embodiment of the present invention.
Mobile terminal 700 shown in Fig. 4 includes: at least one processor 701, memory 702, at least one network interface
704 and other users interface 703.Various components in mobile terminal 700 are coupled by bus system 705.It is understood that
Bus system 705 is for realizing the connection communication between these components.Bus system 705 is also wrapped in addition to including data/address bus
Include power bus, control bus and status signal bus in addition.But for the sake of clear explanation, various buses are all marked in Fig. 4
For bus system 705.
Wherein, user interface 703 may include display, keyboard or pointing device (for example, mouse, trace ball
(trackball), touch-sensitive plate or touch screen etc..
It is appreciated that the memory 702 in the embodiment of the present invention can be volatile memory or nonvolatile memory,
It or may include both volatile and non-volatile memories.Wherein, nonvolatile memory can be read-only memory (Read-
OnlyMemory, ROM), programmable read only memory (ProgrammableROM, PROM), Erasable Programmable Read Only Memory EPROM
(ErasablePROM, EPROM), electrically erasable programmable read-only memory (ElectricallyEPROM, EEPROM) dodge
It deposits.Volatile memory can be random access memory (RandomAccessMemory, RAM), and it is slow to be used as external high speed
It deposits.By exemplary but be not restricted explanation, the RAM of many forms is available, such as static random access memory
(StaticRAM, SRAM), dynamic random access memory (DynamicRAM, DRAM), Synchronous Dynamic Random Access Memory
(SynchronousDRAM, SDRAM), double data speed synchronous dynamic RAM (DoubleDataRate
SDRAM, DDRSDRAM), enhanced Synchronous Dynamic Random Access Memory (Enhanced SDRAM, ESDRAM), synchronized links
Dynamic random access memory (SynchlinkDRAM, SLDRAM) and direct rambus random access memory
(DirectRambusRAM, DRRAM).The memory 702 of the system and method for description of the embodiment of the present invention is intended to include but unlimited
In the memory of these and any other suitable type.
In some embodiments, memory 702 stores following element, executable modules or data structures, or
Their subset of person or their superset: operating system 7021 and application program 7022.
Wherein, operating system 7021 include various system programs, such as ccf layer, core library layer, driving layer etc., are used for
Realize various basic businesses and the hardware based task of processing.Application program 7022 includes various application programs, such as media
Player (MediaPlayer), browser (Browser) etc., for realizing various applied business.Realize embodiment of the present invention side
The program of method may be embodied in application program 7022.
In embodiments of the present invention, by the program or instruction of calling memory 702 to store, specifically, can be application
The program or instruction stored in program 7022, processor 701 is for acquiring incident ray；Described image sensor incudes incident light
Line and convert form electric image signal；Described image electric signal is exported, includes RGB image electric signal in described image electric signal
With IR electric image signal；For described image electric signal, the first buffer area and second buffering area are set；By the RGB image telecommunications
Number it is saved in the first buffer area, and, the IR electric image signal is saved in the second buffer area；From first buffer area
RGB image electric signal is obtained, and, IR electric image signal is obtained from second buffer area；According to the RGB image telecommunications
Number and the IR electric image signal obtain target image.
The method that the embodiments of the present invention disclose can be applied in processor 701, or be realized by processor 701.
Processor 701 may be a kind of IC chip, the processing capacity with signal.During realization, the above method it is each
Step can be completed by the integrated logic circuit of the hardware in processor 701 or the instruction of software form.Above-mentioned processing
Device 701 can be general processor, digital signal processor (DigitalSignalProcessor, DSP), specific integrated circuit
(ApplicationSpecific IntegratedCircuit, ASIC), ready-made programmable gate array
(FieldProgrammableGateArray, FPGA) either other programmable logic device, discrete gate or transistor logic
Device, discrete hardware components.It may be implemented or execute disclosed each method, step and the logical box in the embodiment of the present invention
Figure.General processor can be microprocessor or the processor is also possible to any conventional processor etc..In conjunction with the present invention
The step of method disclosed in embodiment, can be embodied directly in hardware decoding processor and execute completion, or use decoding processor
In hardware and software module combination execute completion.Software module can be located at random access memory, and flash memory, read-only memory can
In the storage medium of this fields such as program read-only memory or electrically erasable programmable memory, register maturation.The storage
Medium is located at memory 702, and processor 701 reads the information in memory 702, and the step of the above method is completed in conjunction with its hardware
Suddenly.
It is understood that the embodiment of the present invention description these embodiments can with hardware, software, firmware, middleware,
Microcode or combinations thereof is realized.For hardware realization, processing unit be may be implemented in one or more specific integrated circuit (App
LicationSpecificIntegratedCircuits, ASIC), digital signal processor
(DigitalSignalProcessing, DSP), digital signal processing appts (DSPDevice, DSPD), programmable logic device
(ProgrammableLogicDevice, PLD), field programmable gate array (Field-ProgrammableGateArray,
FPGA), general processor, controller, microcontroller, microprocessor, other electronics lists for executing herein described function
In member or combinations thereof.
For software implementations, can by execute the embodiment of the present invention described in function module (such as process, function etc.) come
Realize technology described in the embodiment of the present invention.Software code is storable in memory and is executed by processor.Memory can
With portion realizes in the processor or outside the processor.
Optionally, processor 701 is also used to: determining the number of species that described image electric signal includes, the number of species
Including RGB image electric signal and IR electric image signal；Determine the data capacity of every kind of electric image signal；According to default first rule
The first buffer area is set with the data capacity of the RGB image electric signal；According to default Second Rule and the IR image telecommunications
Number data capacity be arranged the second buffer area.
Optionally, whether processor 701 is also used to: judging in described image electric signal to include IR electric image signal line by line；
If in electric image signal not including IR electric image signal, the electric image signal of the row is saved in the first buffer area；If image
Include IR electric image signal in electric signal, then the IR electric image signal of the row is substituted using G component, and by the figure of the row after substitution
As electric signal is saved in the first buffer area；And the IR electric image signal of the row is saved in second buffering area.
Optionally, processor 701 is also used to: from the G extracted around the IR electric image signal in described image electric signal
Component；Average G component is calculated based on the G component around described；The IR image telecommunications of the row is substituted using the average G component
Number；The row electric image signal after substitution is saved in the first buffer area.
Optionally, processor 701 is also used to: obtaining colored target RGB image based on the RGB image electric signal；Base
The target IR image of black and white is obtained in the IR electric image signal；And it is based on the RGB image electric signal and the IR image
The combination of electric signal obtains target mixed image.
Mobile terminal 700 can be realized each process that mobile terminal is realized in previous embodiment, to avoid repeating, here
It repeats no more.
In this way, in the embodiment of the present invention, when incident ray projects camera module, image sensor senses incident light
Line and convert form electric image signal, be then output in image signal process again；And it is additionally provided with the first buffer area and
Two buffer areas are divided if the image signal process electric image signal may separate out RGB image electric signal and IR electric image signal
RGB image electric signal and IR electric image signal the first buffer area and second buffering area are not saved in；It when required, can be with base
In can individually using or combination target image obtained using rgb image data and IR image data.Implement using the present invention
Example is equivalent to there are two camera module in same camera module, can independently export two-way image data.In addition, this
Inventive embodiments can realize function possessed by dual camera mould group, this simple structure by a camera module
First dual camera mould group the space occupied on mobile terminals is not only effectively saved, and dual camera can also be retained simultaneously
Function possessed by mould group.
Embodiment five
Fig. 5 is the structural schematic diagram of the mobile terminal of another embodiment of the present invention.Specifically, the mobile terminal in Fig. 5
800 can be mobile phone, tablet computer, personal digital assistant (PersonalDigital Assistant, PDA) or vehicle-mounted computer
Deng.
Mobile terminal 800 in Fig. 5 includes radio frequency (RadioFrequency, RF) circuit 810, memory 820, input list
Member 830, display unit 840, processor 860, voicefrequency circuit 870, WiFi (WirelessFidelity) module 880 and power supply
890。
Wherein, input unit 830 can be used for receiving the number or character information of user's input, and generation and mobile terminal
The related signal input of 800 user setting and function control.Specifically, in the embodiment of the present invention, which can
To include touch panel 831.Touch panel 831, also referred to as touch screen collect the touch operation of user on it or nearby
(for example user uses the operations of any suitable object or attachment on touch panel 831 such as finger, stylus), and according to preparatory
The formula of setting drives corresponding attachment device.Optionally, touch panel 831 may include touch detecting apparatus and touch controller
Two parts.Wherein, the touch orientation of touch detecting apparatus detection user, and touch operation bring signal is detected, by signal
Send touch controller to；Touch controller receives touch information from touch detecting apparatus, and is converted into contact coordinate,
The processor 860 is given again, and can be received order that processor 860 is sent and be executed.Furthermore, it is possible to using resistance-type,
The multiple types such as condenser type, infrared ray and surface acoustic wave realize touch panel 831.In addition to touch panel 831, input unit
830 can also include other input equipments 832, other input equipments 832 can include but is not limited to physical keyboard, function key
One of (such as volume control button, switch key etc.), trace ball, mouse, operating stick etc. are a variety of.
Wherein, display unit 840 can be used for showing information input by user or be supplied to the information and movement of user
The various menu interfaces of terminal 800.Display unit 840 may include display panel 841, optionally, can use LCD or organic hair
The forms such as optical diode (OrganicLight-EmittingDiode, OLED) configure display panel 841.
It should be noted that touch panel 831 can cover display panel 841, touch display screen is formed, when the touch display screen is examined
After measuring touch operation on it or nearby, processor 860 is sent to determine the type of touch event, is followed by subsequent processing device
860 provide corresponding visual output according to the type of touch event in touch display screen.
Touch display screen includes Application Program Interface viewing area and common control viewing area.The Application Program Interface viewing area
And arrangement mode of the common control viewing area does not limit, can be arranged above and below, left-right situs etc. can distinguish two it is aobvious
Show the arrangement mode in area.The Application Program Interface viewing area is displayed for the interface of application program.Each interface can be with
The interface elements such as the icon comprising at least one application program and/or widget desktop control.The Application Program Interface viewing area
Or the empty interface not comprising any content.This commonly uses control viewing area for showing the higher control of utilization rate, for example,
Application icons such as button, interface number, scroll bar, phone directory icon etc. are set.
Wherein processor 860 is the control centre of mobile terminal 800, utilizes various interfaces and connection whole mobile phone
Various pieces, by running or executing the software program and/or module that are stored in first memory 821, and calling storage
Data in second memory 822 execute the various functions and processing data of mobile terminal 800, thus to mobile terminal 800
Carry out integral monitoring.Optionally, processor 860 may include one or more processing units.
In embodiments of the present invention, by call store the first memory 821 in software program and/or module and/
Or the data in the second memory 822, processor 860 is for acquiring incident ray；Described image sensor incudes incident light
Line and convert form electric image signal；Export described image electric signal, include in described image electric signal RGB image electric signal and
IR electric image signal；For described image electric signal, the first buffer area and second buffering area are set；By the RGB image telecommunications
Number it is saved in the first buffer area, and, the IR electric image signal is saved in the second buffer area；From first buffer area
RGB image electric signal is obtained, and, IR electric image signal is obtained from second buffer area；According to the RGB image telecommunications
Number and the IR electric image signal obtain target image.
The processor 860 is also used to: determining that the number of species that described image electric signal includes, the number of species include
RGB image electric signal and IR electric image signal；Determine the data capacity of every kind of electric image signal；According to default first rule and institute
The first buffer area is arranged in the data capacity for stating RGB image electric signal；According to default Second Rule and the IR electric image signal
The second buffer area is arranged in data capacity.
Whether the processor 860 is also used to: judging in described image electric signal to include IR electric image signal line by line；If figure
As not including IR electric image signal in electric signal, then the electric image signal of the row is saved in the first buffer area；If image telecommunications
Include IR electric image signal in number, then substitutes the IR electric image signal of the row using G component, and the image of the row after substitution is electric
Signal is saved in the first buffer area；And the IR electric image signal of the row is saved in second buffering area.
The processor 860 is also used to: from the G extracted around the IR electric image signal in described image electric signal points
Amount；Average G component is calculated based on the G component around described；The IR image telecommunications of the row is substituted using the average G component
Number；The row electric image signal after substitution is saved in the first buffer area.
The processor 860 is also used to: obtaining colored target RGB image based on the RGB image electric signal；Based on institute
It states IR electric image signal and obtains the target IR image of black and white；And it is based on the RGB image electric signal and the IR image telecommunications
Number combination obtain target mixed image.
For 800 embodiment of mobile terminal, since it is basically similar to the method embodiment, so the comparison of description is simple
Single, the relevent part can refer to the partial explaination of embodiments of method.
In this way, in the embodiment of the present invention, when incident ray projects camera module, image sensor senses incident light
Line and convert form electric image signal, be then output in image signal process again；And it is additionally provided with the first buffer area and
Distinguish if the image signal process electric image signal may separate out RGB image electric signal and IR electric image signal two buffer areas
RGB image electric signal and IR electric image signal are saved in the first buffer area and second buffering area；It when required, can be with base
In can individually using or combination target image obtained using rgb image data and IR image data.Implement using the present invention
Example is equivalent to there are two camera module in same camera module, can independently export two-way image data.In addition, this
Inventive embodiments can realize function possessed by dual camera mould group, this simple structure by a camera module
First dual camera mould group the space occupied on mobile terminals is not only effectively saved, and dual camera can also be retained simultaneously
Function possessed by mould group.
Those of ordinary skill in the art may be aware that the embodiment in conjunction with disclosed in the embodiment of the present invention describe it is each
Exemplary unit and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond the scope of this invention.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description,
The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In embodiment provided herein, it should be understood that disclosed device and method can pass through others
Mode is realized.For example, the apparatus embodiments described above are merely exemplary, for example, the division of the unit, only
A kind of logical function partition, there may be another division manner in actual implementation, for example, multiple units or components can combine or
Person is desirably integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutual
Between coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or communication link of device or unit
It connects, can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.
It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product
It is stored in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially in other words
The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a
People's computer, server or network equipment etc.) it performs all or part of the steps of the method described in the various embodiments of the present invention.
And storage medium above-mentioned includes: that USB flash disk, mobile hard disk, ROM, RAM, magnetic or disk etc. are various can store program code
Medium.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain
Lid is within protection scope of the present invention.Therefore, protection scope of the present invention should be subject to the protection scope in claims.
Claims (8)
1. a kind of processing method of image data is applied to mobile terminal, which is characterized in that the mobile terminal includes camera
Mould group is provided with imaging sensor in the camera module, which comprises
Acquire incident ray；
Described image sensor, which incudes incident ray and converts, forms electric image signal；
Described image electric signal is exported, includes RGB image electric signal and IR electric image signal in described image electric signal, it is described
RGB image electric signal includes G component；
For described image electric signal, the first buffer area and the second buffer area are set；
The RGB image electric signal is saved in the first buffer area, and, the IR electric image signal is saved in the second caching
Area；
RGB image electric signal is obtained from first buffer area, and, IR image telecommunications is obtained from second buffer area
Number；
Target image is obtained according to the RGB image electric signal and the IR electric image signal；
Wherein, described that the RGB image electric signal is saved in the first buffer area, and, the IR electric image signal is saved
Include: to the step of the second buffer area
Judge in described image electric signal whether to include IR electric image signal line by line；
If in electric image signal not including IR electric image signal, the electric image signal of the row is saved in the first buffer area；
If in electric image signal including IR electric image signal, the IR electric image signal of the row is substituted using G component, and will substitution
The electric image signal of the row is saved in the first buffer area afterwards；And the IR electric image signal of the row is saved in second and is delayed
Deposit area.
2. the method according to claim 1, wherein described be arranged the first buffer area and for electric image signal
The step of two buffer areas includes:
Determine that the number of species that described image electric signal includes, the number of species include RGB image electric signal and IR image electricity
Two kinds of signal；
Determine the data capacity of every kind of electric image signal；
According to the data capacity of default first rule and the RGB image electric signal, the first buffer area is set；
According to default Second Rule and the data capacity of the IR electric image signal, the second buffer area is set.
3. according to the method described in claim 2, it is characterized in that, the IR electric image signal for substituting the row using G component,
And the step of electric image signal of the row after substitution is saved in the first buffer area, includes:
From the G component extracted in described image electric signal around the IR electric image signal；
Average G component is calculated based on the G component around described；
The IR electric image signal of the row is substituted using the average G component；
The row electric image signal after substitution is saved in the first buffer area.
4. the method according to claim 1, wherein described according to RGB image electric signal and IR image electricity
Signal obtains the step of target image and includes:
Colored target RGB image is obtained based on the RGB image electric signal；
The target IR image of black and white is obtained based on the IR electric image signal；
And the combination based on the RGB image electric signal and the IR electric image signal obtains target mixed image.
5. a kind of mobile terminal, which is characterized in that the mobile terminal includes camera module, is arranged in the camera module
There is imaging sensor, the mobile terminal includes:
Incident ray acquisition module, for acquiring incident ray；
Electric image signal obtains module, incudes incident ray for described image sensor and conversion forms electric image signal；
Electric image signal output module includes RGB image telecommunications in described image electric signal for exporting described image electric signal
Number and IR electric image signal, the RGB image electric signal includes G component；
Buffer area setup module, for the first buffer area and the second buffer area to be arranged for described image electric signal；
Electric image signal preserving module, for the RGB image electric signal to be saved in the first buffer area, and, by the IR
Electric image signal is saved in the second buffer area；
Electric image signal obtains module, for obtaining RGB image electric signal from first buffer area, and, from described the
IR electric image signal is obtained in two buffer areas；
Target image generation module, for obtaining target image according to the RGB image electric signal and the IR electric image signal；
Wherein, described image electric signal preserving module includes:
Judging submodule, for judging in described image electric signal whether to include IR electric image signal line by line；
First saves submodule, if for not including IR electric image signal in electric image signal, by the electric image signal of the row
It is saved in the first buffer area；
Second saves submodule, if substituting the IR of the row using G component for including IR electric image signal in electric image signal
Electric image signal, and the electric image signal of the row after substitution is saved in the first buffer area；And by the IR image of the row
Electric signal is saved in the second buffer area.
6. mobile terminal according to claim 5, which is characterized in that the buffer area setup module includes:
Number of species determine submodule, the number of species for including for determining described image electric signal, and the number of species include
RGB image electric signal and two kinds of IR electric image signal；
Data capacity determines submodule, for determining the data capacity of every kind of electric image signal；
Submodule is arranged in first buffer area, for setting according to the data capacity of default first rule and the RGB image electric signal
Set the first buffer area；
Submodule is arranged in second buffer area, for according to default Second Rule and the setting of the data capacity of the IR electric image signal
Second buffer area.
7. mobile terminal according to claim 5, which is characterized in that described second, which saves submodule, includes:
G component extraction unit, for from the G component extracted in described image electric signal around the IR electric image signal；
Average G component calculation unit calculates average G component for the G component based on the surrounding；
IR electric image signal substituting unit, for substituting the IR electric image signal of the row using the average G component；
First cache unit, for the row electric image signal after substitution to be saved in the first buffer area.
8. mobile terminal according to claim 5, which is characterized in that the target image generation module includes:
Target RGB image obtains submodule, for obtaining colored target RGB image based on the RGB image electric signal；
Target IR image obtains submodule, for obtaining the target IR image of black and white based on the IR electric image signal；
Target mixed image obtains submodule, for the combination based on the RGB image electric signal and the IR electric image signal
Obtain target mixed image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610921869.9A CN106572290B (en) | 2016-10-21 | 2016-10-21 | A kind of processing method and mobile terminal of image data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610921869.9A CN106572290B (en) | 2016-10-21 | 2016-10-21 | A kind of processing method and mobile terminal of image data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106572290A CN106572290A (en) | 2017-04-19 |
| CN106572290B true CN106572290B (en) | 2019-08-20 |
Family
ID=58533394
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610921869.9A Active CN106572290B (en) | 2016-10-21 | 2016-10-21 | A kind of processing method and mobile terminal of image data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106572290B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107707744A (en) * | 2017-09-22 | 2018-02-16 | 惠州Tcl移动通信有限公司 | Photo storage method, mobile terminal and the storage medium of a kind of mobile terminal |
| KR102590315B1 (en) | 2018-05-28 | 2023-10-16 | 삼성전자주식회사 | Organic photoelectric device and image sensor including the same |
| TWI768725B (en) * | 2020-09-16 | 2022-06-21 | 瑞昱半導體股份有限公司 | Separate display system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104463112A (en) * | 2014-11-27 | 2015-03-25 | 深圳市科葩信息技术有限公司 | Method and system for carrying out biological recognition through RGB+IR image sensor |
| CN105763848A (en) * | 2016-03-03 | 2016-07-13 | 浙江宇视科技有限公司 | Fish-eye camera rear end access method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9996726B2 (en) * | 2013-08-02 | 2018-06-12 | Qualcomm Incorporated | Feature identification using an RGB-NIR camera pair |
-
2016
- 2016-10-21 CN CN201610921869.9A patent/CN106572290B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104463112A (en) * | 2014-11-27 | 2015-03-25 | 深圳市科葩信息技术有限公司 | Method and system for carrying out biological recognition through RGB+IR image sensor |
| CN105763848A (en) * | 2016-03-03 | 2016-07-13 | 浙江宇视科技有限公司 | Fish-eye camera rear end access method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106572290A (en) | 2017-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105827964B (en) | A kind of image processing method and mobile terminal | |
| CN107205120B (en) | A kind of processing method and mobile terminal of image | |
| CN105827754B (en) | A kind of generation method and mobile terminal of high dynamic range images | |
| CN105827970B (en) | A kind of image processing method and mobile terminal | |
| CN106101545B (en) | A kind of image processing method and mobile terminal | |
| CN107222680A (en) | The image pickup method and mobile terminal of a kind of panoramic picture | |
| CN106791450B (en) | A kind of photographic method and mobile terminal | |
| US10270988B2 (en) | Method for generating high-dynamic range image, camera device, terminal and imaging method | |
| CN105474626B (en) | Photographic device | |
| CN110505411A (en) | Image capturing method, device, storage medium and electronic equipment | |
| CN106572290B (en) | A kind of processing method and mobile terminal of image data | |
| CN106060406A (en) | Photographing method and mobile terminal | |
| CN106060422B (en) | A kind of image exposure method and mobile terminal | |
| CN106060386A (en) | Preview image generation method and mobile terminal | |
| CN106454085B (en) | A kind of image processing method and mobile terminal | |
| CN107079083A (en) | A kind of photographic method, camera arrangement and terminal | |
| CN109547662A (en) | Nonmodal video and static frame-grab | |
| CN107465874B (en) | A kind of dark current processing method and mobile terminal | |
| CN106101666B (en) | A kind of method and mobile terminal of image color reservation | |
| CN108391060A (en) | A kind of image processing method, image processing apparatus and terminal | |
| CN107277351A (en) | The processing method and mobile terminal of a kind of view data | |
| CN106973233A (en) | A kind of camera, mobile terminal and processing method of taking pictures | |
| CN105979155A (en) | Photographing method and mobile terminal | |
| CN106657726A (en) | Image processing method of camera module set and mobile terminal | |
| US20190098213A1 (en) | Mobile terminal facilitating image capture mode switching, and method therefor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |