CN108924426A - Image processing method and device, electronic equipment, computer readable storage medium - Google Patents

Image processing method and device, electronic equipment, computer readable storage medium Download PDF

Info

Publication number
CN108924426A
CN108924426A CN201810867083.2A CN201810867083A CN108924426A CN 108924426 A CN108924426 A CN 108924426A CN 201810867083 A CN201810867083 A CN 201810867083A CN 108924426 A CN108924426 A CN 108924426A
Authority
CN
China
Prior art keywords
temperature
image
light source
initial
target component
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810867083.2A
Other languages
Chinese (zh)
Other versions
CN108924426B (en
Inventor
郭子青
周海涛
欧锦荣
谭筱
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN201810867083.2A priority Critical patent/CN108924426B/en
Publication of CN108924426A publication Critical patent/CN108924426A/en
Priority to PCT/CN2019/075374 priority patent/WO2020024575A1/en
Priority to TW108120554A priority patent/TWI708192B/en
Priority to EP19186497.4A priority patent/EP3605393A1/en
Priority to US16/523,292 priority patent/US11218650B2/en
Application granted granted Critical
Publication of CN108924426B publication Critical patent/CN108924426B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/62Control of parameters via user interfaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161Detection; Localisation; Normalisation
    • G06V40/166Detection; Localisation; Normalisation using acquisition arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172Classification, e.g. identification
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof

Abstract

This application involves a kind of image processing methods and device, electronic equipment, computer readable storage medium.The method includes:The temperature of light source emitter is detected, when the Current Temperatures of light source emitter and the temperature gap of initial temperature are more than threshold value, according to initial temperature target component corresponding with temperature gap acquisition, preview image is corrected according to target component.Image is corrected since corresponding parameter can be obtained according to the temperature of light source emitter, it is possible to reduce the deviation of the image of acquisition.

Description

Image processing method and device, electronic equipment, computer readable storage medium
Technical field
This application involves field of computer technology, more particularly to a kind of image processing method and device, electronic equipment, meter Calculation machine readable storage medium storing program for executing.
Background technique
With the development of computer technology, depth image technology and infrared image technology be widely used in recognition of face, The scenes such as human-computer interaction, image beautification.Electronic equipment issues corresponding light source by light source emitters such as floodlight or color-changing lamps It projects on object, Laser video camera head obtains corresponding image according to the light being reflected back.However, there is acquisition in traditional technology Image there are problems that deviation.
Summary of the invention
The embodiment of the present application provides a kind of image processing method and device, electronic equipment, computer readable storage medium, can To reduce the image deviations of acquisition.
A kind of image processing method, including:
The temperature of light source emitter is detected;
When the Current Temperatures of the light source emitter and the temperature gap of initial temperature are more than threshold value, according to described initial Temperature target component corresponding with temperature gap acquisition;
Preview image is corrected according to the target component.
A kind of image processing apparatus, including:
Temperature detecting module is detected for the temperature to light source emitter;
Parameter acquisition module, for being more than threshold when the Current Temperatures of the light source emitter and the temperature gap of initial temperature When value, according to initial temperature target component corresponding with temperature gap acquisition;
Image correction module, for being corrected according to the target component to preview image.
A kind of electronic equipment, including memory and processor store computer program, the calculating in the memory When machine program is executed by the processor, so that the processor executes following steps:
The temperature of light source emitter is detected;
When the Current Temperatures of the light source emitter and the temperature gap of initial temperature are more than threshold value, according to described initial Temperature target component corresponding with temperature gap acquisition;
Preview image is corrected according to the target component.
A kind of computer readable storage medium, is stored thereon with computer program, and the computer program is held by processor Following steps are realized when row:
The temperature of light source emitter is detected;
When the Current Temperatures of the light source emitter and the temperature gap of initial temperature are more than threshold value, according to described initial Temperature target component corresponding with temperature gap acquisition;
Preview image is corrected according to the target component.
Above-mentioned image processing method and device, electronic equipment, computer readable storage medium, by light source emitter Temperature is detected, when the Current Temperatures of light source emitter and the temperature gap of initial temperature are more than threshold value, according to initial temperature Target component corresponding with temperature gap acquisition is spent, preview image is corrected according to target component.Due to can be according to light The temperature of source transmitter obtains corresponding parameter and is corrected to image, it is possible to reduce the deviation of image.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments of application for those of ordinary skill in the art without creative efforts, can be with It obtains other drawings based on these drawings.
Fig. 1 is the application scenario diagram of image processing method in one embodiment;
Fig. 2 is the flow chart of image processing method in one embodiment;
Fig. 3 is the flow chart of image processing method in another embodiment;
Fig. 4 is the flow chart of image processing method in another embodiment;
Fig. 5 is the flow chart of image processing method in one embodiment;
Fig. 6 is the flow chart of image processing method in another embodiment;
Fig. 7 is the structural block diagram of image processing apparatus in one embodiment;
Fig. 8 is the schematic diagram of internal structure of electronic equipment in one embodiment;
Fig. 9 is the schematic diagram of image processing circuit in one embodiment.
Specific embodiment
It is with reference to the accompanying drawings and embodiments, right in order to which the objects, technical solutions and advantages of the application are more clearly understood The application is further elaborated.It should be appreciated that specific embodiment described herein is only used to explain the application, and It is not used in restriction the application.
It is appreciated that term " first " used in this application, " second " etc. can be used to describe various elements herein, But these elements should not be limited by these terms.These terms are only used to distinguish the first element from the other element.Citing comes It says, in the case where not departing from scope of the present application, the first client can be known as the second client, and similarly, can incite somebody to action Second client is known as the first client.The first client and the second client both client, but it is not same visitor Family end.
Fig. 1 is the application scenario diagram of image processing method in one embodiment.As shown in Figure 1, electronic equipment 10 may include Camera module 110, second processor 120, first processor 130.Above-mentioned second processor 120 can be CPU (Central Processing Unit, central processing unit) module.Above-mentioned first processor 130 can be MCU (Microcontroller Unit, microcontroller) module 130 etc..Wherein, first processor 130 is connected to second processor 120 and camera module 110 Between, above-mentioned first processor 130 can control Laser video camera head 112 in camera module 110, floodlight 114 and color-changing lamp 118, above-mentioned second processor 120 can control RGB camera 116 in camera module 110.
In camera module 110 include Laser video camera head 112, floodlight 114, RGB (Red/Green/Blue, it is red green/ Blue color mode) camera 116 and color-changing lamp 118.Above-mentioned Laser video camera head 112 is infrared camera, for obtaining infrared figure Picture.Above-mentioned floodlight 114 is the point light source that infrared light can occur;Above-mentioned color-changing lamp 118 is the point light source that infrared laser can occur And for figuratum point light source.Wherein, when floodlight 114 emits point light source, Laser video camera head 112 can be according to being reflected back Light obtains infrared image.When color-changing lamp 118 emits point light source, Laser video camera head 112 can be obtained according to the light being reflected back Speckle image.Above-mentioned speckle image be color-changing lamp 118 emit reflected with figuratum point light source after pattern deformation occurs Image.
First processor 130 includes PWM (Pulse Width Modulation, pulse width modulation) module 132, SPI/ I2C (Serial Peripheral Interface/Inter-Integrated Circuit, Serial Peripheral Interface (SPI)/two-way two Line synchronous serial interface) interface 134, RAM (Random Access Memory, random access memory) module 136 and depth Spend engine 138.Above-mentioned PWM module 132 can emit pulse to camera module, control floodlight 114 or color-changing lamp 118 is opened, So that Laser video camera head 112 can collect infrared image or speckle image.Above-mentioned SPI/I2C interface 134 is for receiving at second Manage the face acquisition instructions that device 120 is sent.Above-mentioned depth engine 138 can handle speckle image to obtain depth parallax figure.
When second processor 120 receives the data acquisition request of application program, for example, when application program needs to carry out When face unlock, face payment, face acquisition can be sent to first processor 130 by the CPU core operated under TEE environment Instruction.After first processor 130 receives face acquisition instructions, impulse wave can be emitted by PWM module 132 and control camera Floodlight 114 is opened and by radium-shine in the acquisition of Laser video camera head 112 infrared image, control camera module 110 in mould group 110 Lamp 118 is opened and acquires speckle image by Laser video camera head 112.Camera module 110 can by collected infrared image and Speckle image is sent to first processor 130.First processor 130 can be handled to obtain infrared to the infrared image received Disparity map;The speckle image received is handled to obtain speckle disparity map or depth parallax figure.Wherein, first processor 130 pairs of above-mentioned infrared images and speckle image carry out processing and refer to and be corrected to infrared image or speckle image, removal camera shooting Influence of the inside and outside parameter to image in head mould group 110.Wherein, first processor 130 can be set to different modes, different mode The image of output is different.When first processor 130 is set as speckle chart-pattern, first processor 130 is to speckle image processing Speckle disparity map is obtained, target speckle image can be obtained according to above-mentioned speckle disparity map;When first processor 130 is set as depth When chart-pattern, first processor 130 obtains depth parallax figure to speckle image processing, and depth can be obtained according to above-mentioned depth parallax figure Image is spent, above-mentioned depth image refers to the image with depth information.First processor 130 can by above-mentioned infrared disparity map and dissipate Spot disparity map is sent to second processor 120, and first processor 130 can also send above-mentioned infrared disparity map and depth parallax figure To second processor 120.Second processor 120 can obtain Infrared Targets image, according to above-mentioned depth according to above-mentioned infrared disparity map It spends disparity map and obtains depth image.Further, second processor 120 can be carried out according to Infrared Targets image, depth image Recognition of face, face matching, In vivo detection and the depth information for obtaining the face detected.
Communicate between first processor 130 and second processor 120 is by fixed safe interface, to ensure to transmit The safety of data.As shown in Figure 1, the data that second processor 120 is sent to first processor 130 are serially outer by safety If interface or bidirectional two-line synchronous serial interface (SECURE SPI/I2C) 140, first processor 130 is sent to second processing The data of device 120 are by safety moving Industry Processor Interface (SECURE MIPIMobile, Industry Processor Interface)150。
In one embodiment, electronic equipment is to light source emitter, that is, floodlight 114 in camera module 110 and radium-shine The temperature of lamp 118 is detected, when the Current Temperatures of light source emitter and the temperature gap of initial temperature are more than threshold value, first Processor 130 can be according to initial temperature target component corresponding with temperature gap acquisition, according to target component to preview image It is corrected.
Electronic equipment can be mobile phone, tablet computer, personal digital assistant or wearable device etc. in the embodiment of the present application.
Fig. 2 is the flow chart of image processing method in one embodiment.Image processing method in the present embodiment, with operation It is described on electronic equipment in Fig. 1.As shown in Fig. 2, the image processing method includes step 202 to step 206.
Step 202, the temperature of light source emitter is detected.
Light source emitter refers to the device that light source is generated in camera of electronic equipment mould group.Specifically, light source emitter can To include floodlight and color-changing lamp.Floodlight is the point light source that infrared light can occur;Color-changing lamp is the point that infrared laser can occur Light source and to be reflected back when Laser video camera head can emit point light source according to floodlight in camera module with figuratum point light source Light obtain infrared image;The light being reflected back when point light source figuratum according to color-changing lamp transmitting band obtains speckle image. Electronic equipment can be detected by temperature of the built-in temperature sensor to light source emitter.Specifically, electronic equipment can be with Preset interval time is detected every temperature of the preset interval time to light source emitter, can also persistently be emitted light source The temperature of device is detected, can also when it is more than preset duration that light source emitter is using the time to the temperature of light source emitter into Row detection etc..
Step 204, when the Current Temperatures of light source emitter and the temperature gap of initial temperature are more than threshold value, according to initial Temperature target component corresponding with temperature gap acquisition.
The Current Temperatures of light source emitter refer to the temperature of the light source emitter of electronic equipment real-time detection.Initial temperature can To be the temperature of light source emitter when enabling light source emitter, the calibrating parameters used when may also mean that camera collection image Corresponding temperature.Calibrating parameters refer to that the three-dimensional geometry position for determining space object surface point is corresponding in the picture with it The parameter of correlation between point.Each camera when leaving the factory, can all configure corresponding calibrating parameters.Calibrating parameters Including internal reference, outer ginseng and distortion parameter.Threshold value can be configured according to the demand of practical application.For example, Celsius when using Degree is come when measuring, threshold value can be without being limited thereto for 3 degree, 4 degree, 5 degree, 6 degree etc..
Target component refers to the compensating parameter determined according to initial temperature and Current Temperatures and the temperature gap of initial temperature, The deviation for the image that camera is acquired according to former calibrating parameters caused by for reducing because of temperature change.Specifically, target component Including at least one compensating parameter in internal reference, outer ginseng, distortion parameter.When camera is dual camera, electronic equipment can be with Obtain the corresponding target component of two cameras.For example, camera can be RGB camera, in Laser video camera head extremely It is one few.Electronic equipment can preset different temperatures and the corresponding target component of temperature gap.Specifically, electronic equipment can be Different temperatures difference prestores different target components in different initial temperatures.For example, when initial temperature is 20 degree, it is respectively small In minus 6 degree, less than minus 3 degree, greater than 3 degree, the temperature gap greater than 6 degree, different target components is set;When temperature is 25 degree, Different target component etc. respectively is set less than 10 degree, less than minus 5 degree, greater than 5 degree, the temperature gap greater than 10 degree.Electronics Equipment can also prestore the difference for the temperature range that the corresponding target component of different temperatures difference is i.e. different in different temperatures gradient Temperature gap prestores different target components etc..
When the Current Temperatures of light source emitter and the temperature gap of initial temperature are more than threshold value, electronic equipment can be according to first Beginning temperature searches with temperature gap and obtains corresponding target component.
Step 206, preview image is corrected according to target component.
Preview image refers to what electronic equipment was generated by the picture of camera real-time capture current scene.Preview image can With real-time exhibition on the display screen of electronic equipment, detection processing can also be carried out by processor in electronic equipment.Specifically, electric Camera is corrected preview image according to target component in sub- equipment, and the preview image after can making correction is more in line with very Real environment information.When the temperature of camera module changes, can principle point location to camera, focal length size cause shadow It rings, so that camera is according between the former calibrating parameters image acquired and true environmental information, there are deviations, and then influence deep It spends image detection accuracy or causes the parallax between infrared image and RGB image.
Image processing method in the embodiment of the present application is detected by the temperature to light source emitter, when light source is sent out When the Current Temperatures of emitter and the temperature gap of initial temperature are more than threshold value, obtained according to initial temperature and temperature gap corresponding Target component, camera are corrected preview image according to target component, can to avoid due to temperature influence depth image examine It surveys accuracy or causes the parallax between infrared image and RGB image, reduce the deviation between image and true environment information.
In one embodiment, according to initial temperature mesh corresponding with temperature gap acquisition in the image processing method provided After marking parameter, further include:Initial temperature is updated according to the Current Temperatures of light source emitter.
Specifically, electronic equipment can be after obtaining corresponding target component according to initial temperature and temperature gap, by light The Current Temperatures of source transmitter are updated initial temperature as initial temperature.To which electronic equipment can be persistently to light source The temperature of transmitter is detected, when the Current Temperatures of light source emitter and the temperature gap of updated initial temperature are more than threshold When value, obtains updated initial temperature target component corresponding with temperature gap and preview image is corrected, it is possible to reduce Acquire the deviation of image.
As shown in figure 3, in one embodiment, the image processing method provided includes step 302 to step 310.Wherein:
Step 302, the starting to camera is received to request.
Camera may include front camera and rear camera.Specifically, camera can also be dual camera.Camera shooting The starting request of head can be what user was generated by the button clicked on display screen, is also possible to user and is set by pressing electronics What the control on standby or touch screen generated.Electronic equipment can receive the request of the starting to camera.
Step 304, according to the initial parameter and the corresponding initial temperature of initial parameter of starting request camera.
The initial parameter of camera can be used calibrating parameters when last camera collection image.In a reality It applies in example, camera can configure different calibrating parameters according to temperature when leaving the factory or provide the temperature value of determining calibrating parameters, After initial parameter of the electronic equipment according to starting request camera, it can be obtained according to the initial parameter corresponding initial Temperature;Electronic equipment can also obtain camera according to the initial temperature according to the initial temperature of starting request camera Initial parameter of the calibrating parameters as camera, and obtain the initial temperature.
Step 306, the temperature of light source emitter is detected.
Step 308, when the Current Temperatures of light source emitter and the temperature gap of initial temperature are more than threshold value, according to initial Temperature target component corresponding with temperature gap acquisition.
Step 310, preview image is corrected according to target component.
Electronic equipment can receive the request of the starting to camera, according to the initial parameter of the starting request camera And the corresponding initial temperature of initial parameter, the temperature of light source emitter is detected, when light source emitter Current Temperatures with When the temperature gap of initial temperature is more than threshold value, initial temperature target component corresponding with temperature gap is obtained, is joined according to target Several pairs of preview images are corrected, and obtain corresponding target component due to that can carry out detection to temperature when camera starts, Preview image is corrected, it is possible to reduce the deviation of acquisition image is caused because of temperature change.
As shown in figure 4, in one embodiment, the image processing method provided may include step 402 to step 410. Wherein:
Step 402, the 3D process instruction to preview image is received.
3D (3Dimensions) process instruction can be what user was generated by the button clicked on display screen, can also be with It is that user is generated by the control on pressing touch screen, the available 3D process instruction to preview image of electronic equipment.3D Processing refers to the processing that three dimensions are carried out to image, that is, has three dimensions of length.Specifically, electronic equipment can pass through The depth information of object or person face in depth image or infrared image detection image carries out 3D processing to image.For example, 3D is handled It can be and U.S. face processing is carried out to image, electronic equipment can determine the area for needing to carry out U.S. face according to the depth information of face Domain keeps the U.S. face effect of image more preferable;3D processing is also possible to 3D face modeling processing, i.e., according to the face foundation pair in image 3D faceform answered etc..Electronic equipment can receive the 3D process instruction to preview image.Electronic equipment is being received to preview After the 3D process instruction of image, corresponding 3D processing is carried out to preview image.
Step 404, the corresponding initial parameter of preview image is obtained according to 3D process instruction and initial parameter is corresponding initial Temperature.
It is pre- that electronic equipment can obtain the corresponding initial parameter of preview image, that is, camera acquires this according to 3D process instruction Look at image when used calibrating parameters, corresponding initial temperature is obtained according to the initial parameter.
Step 406, the temperature of light source emitter is detected.
Step 408, when the Current Temperatures of light source emitter and the temperature gap of initial temperature are more than threshold value, according to initial Temperature target component corresponding with temperature gap acquisition.
Step 410, preview image is corrected according to target component.
Electronic equipment can receive the 3D process instruction to preview image, obtain preview image pair according to the 3D process instruction The corresponding initial temperature of initial parameter and initial parameter answered, detects the temperature of light source emitter, works as light source emitter Current Temperatures and the temperature gap of initial temperature when being more than threshold value, corresponding target is obtained according to initial temperature and temperature gap Parameter is corrected preview image.Since it is corresponding that detection acquisition can be carried out to temperature when carrying out 3D processing to image Target component is corrected preview image, carries out corresponding 3D processing to the image after correction further according to 3D process instruction, can To reduce the deviation of acquisition image, the accuracy of 3D processing is improved.
As shown in figure 5, in one embodiment, the image processing method provided may include step 502 to step 510. Wherein:
Step 502, the facial image of acquisition is matched with facial image is prestored.
It prestores facial image and refers to the preset face of electronic equipment, prestoring facial image is usually holding for electronic equipment The corresponding facial image of the user that the facial image for the person of having or other persons of being held allow to operate electronic equipment.It prestores Facial image can be one or more, and electronic equipment corresponding to the setting of different application scenarios can prestore facial image. Electronic equipment can acquire facial image in the case where carrying out the application scenarios such as face unlock or face payment, and obtain the application scenarios It is corresponding to prestore facial image, the facial image of acquisition is matched with facial image is prestored.
Step 504, when facial image and when prestoring facial image it fails to match, the corresponding initial parameter of facial image is obtained And the corresponding initial temperature of initial parameter.
Matching threshold can be set in electronic equipment, when facial image is lower than matching threshold with the matching degree for prestoring facial image When, then it fails to match for judgement.Electronic equipment can facial image and it is all prestore facial image it fails to match when, obtain people The corresponding initial parameter of face image and the corresponding initial temperature of initial parameter.Electronic equipment is in facial image and prestores facial image When it fails to match, facial image and each matching degree for prestoring facial image are analyzed, for example, electronic equipment can be Other of facial image and a matching degree for prestoring facial image and the facial image prestore the difference of the matching degree of facial image When value is more than preset value, the corresponding initial parameter of facial image initial temperature corresponding with initial parameter is obtained.For example, when matching When threshold value is 80%, preset value is 30%, prestore facial image A, B, C if it exists, facial image D and prestore facial image A, B, The matching degree of C is respectively 75%, 40%, 25%, then the matching degree of facial image D and A and the matching degree of D and B, C are more than pre- If value 30%, then electronic equipment judges facial image D and the difference of the matching degree and matching threshold that prestore facial image A may be Due to caused by the deviation of the facial image of acquisition, and then the corresponding initial parameter of acquisition facial image and initial parameter are corresponding Initial temperature.
Step 506, the temperature of light source emitter is detected.
Step 508, when the Current Temperatures of light source emitter and the temperature gap of initial temperature are more than threshold value, according to initial Temperature target component corresponding with temperature gap acquisition.
Step 510, facial image is corrected according to target component.
Electronic equipment can be corrected facial image according to target component, by the facial image after correction and prestore people Face image is matched.Due to that can be detected when carrying out face verification to the temperature of light source emitter, when light source emits When the Current Temperatures of device and the temperature gap of initial temperature are more than threshold value, according to initial temperature mesh corresponding with temperature gap acquisition Parameter is marked, facial image is corrected, so that electronic equipment is according to the facial image after correction and prestores facial image progress Matching, it is possible to reduce the deviation of the facial image of acquisition improves the matched accuracy of face.
In one embodiment, it in the image processing method provided when facial image is with prestoring facial image and matching, obtains The process of the corresponding initial parameter of facial image and the corresponding initial temperature of initial parameter is taken to further include:When facial image with prestore When the facial image number that it fails to match is more than preset times, obtains the corresponding initial parameter of facial image and initial parameter is corresponding Initial temperature.
Preset times can be set according to practical application request, it is not limited here.For example, preset times can be 2 times, 3 times, 4 times, it is 5 inferior, it is without being limited thereto.When facial image and the facial image number that it fails to match is prestored more than preset times When, the corresponding initial parameter of the available facial image of electronic equipment and the corresponding initial temperature of initial parameter emit light source The temperature of device is detected, when the Current Temperatures of light source emitter and the temperature gap of initial temperature are more than threshold value, according to first Beginning temperature target component corresponding with temperature gap acquisition, is corrected facial image according to target component, so that electronics is set It is standby to match the facial image after correction with facial image is prestored, it can be to avoid the people of acquisition caused by due to temperature The deviation of face image improves the matched accuracy of face.
As shown in fig. 6, in one embodiment, the image processing method provided may include step 602 to step 606. Wherein:
Step 602, the temperature of light source emitter is detected.
Step 604, when the duration that the Current Temperatures of light source emitter and the temperature gap of initial temperature are more than threshold value When more than preset duration, according to initial temperature target component corresponding with temperature gap acquisition.
Preset duration can be set according to practical application request.For example, preset duration can be 3 seconds, 4 seconds, 5 seconds etc. It is not limited in this.When the duration that the Current Temperatures of light source emitter and the temperature gap of initial temperature are more than threshold value is more than pre- If when duration, specifically, average value and the initial temperature acquisition pair of the temperature gap in the available preset duration of electronic equipment The target component answered;Also the most temperature gap of frequency of occurrence and initial temperature obtain electronic equipment in available preset duration Corresponding target component;It is corresponding with initial temperature acquisition that electronic equipment can also obtain temperature gap last in preset duration Target component etc..For example, when initial temperature is 20 degree, threshold value is 3 degree, if electronic equipment detects the current of light source emitter Variation of the temperature in preset duration is 23 degree, 24 degree, 25 degree, 25 degree, 25 degree, 24 degree, 26 degree, then electronic equipment can basis The target component corresponding with 20 degree of acquisitions of initial temperature of the most temperature gap of frequency of occurrence i.e. 5 degree, can also be according to temperature difference The average value of value i.e. 4.6 degree target component corresponding with 20 degree of acquisitions of initial temperature, can also obtain the last temperature of preset duration Spend difference i.e. 6 degree target component corresponding with 20 degree of acquisitions of initial temperature.When the Current Temperatures and initial temperature of light source emitter Temperature gap when being more than that the duration of threshold value is more than preset duration, electronic equipment is obtained according to initial temperature and temperature gap The mode of corresponding target component can there are many, can specifically be set according to practical application request, it is not limited here.
Step 606, preview image is corrected according to target component.
Electronic equipment detects the temperature of light source emitter, when the Current Temperatures and initial temperature of light source emitter When temperature gap is more than that the duration of threshold value is more than preset duration, according to initial temperature target corresponding with temperature gap acquisition Parameter is corrected preview image, it is possible to reduce the deviation of the image of acquisition.
It should be understood that although each step in the flow chart of Fig. 2-6 is successively shown according to the instruction of arrow, These steps are not that the inevitable sequence according to arrow instruction successively executes.Unless expressly stating otherwise herein, these steps Execution there is no stringent sequences to limit, these steps can execute in other order.Moreover, at least one in Fig. 2-6 Part steps may include that perhaps these sub-steps of multiple stages or stage are not necessarily in synchronization to multiple sub-steps Completion is executed, but can be executed at different times, the execution sequence in these sub-steps or stage is also not necessarily successively It carries out, but can be at least part of the sub-step or stage of other steps or other steps in turn or alternately It executes.
Fig. 7 is the structural block diagram of the image processing apparatus of one embodiment.As shown in fig. 7, the image processing apparatus includes: Temperature detecting module 720, parameter acquisition module 740, image correction module 760.Wherein:
Temperature detecting module 720 is detected for the temperature to light source emitter.
Parameter acquisition module 740, for being more than threshold when the Current Temperatures of light source emitter and the temperature gap of initial temperature When value, corresponding target component is obtained according to initial temperature and temperature gap.
Image correction module 760, for being corrected according to target component to preview image.
In one embodiment, the image processing apparatus provided can also include initial temperature determining module 780, initial temperature Degree determining module 780 is for being updated the initial temperature according to the Current Temperatures of the light source emitter.
In one embodiment, initial temperature determining module 780 can be also used for receiving the request of the starting to camera;Root According to the initial parameter and the corresponding initial temperature of initial parameter of starting request camera.
In one embodiment, initial temperature determining module 780 can be also used for receiving and refer to the 3D processing of preview image It enables;The corresponding initial parameter of preview image and the corresponding initial temperature of initial parameter are obtained according to 3D process instruction.
In one embodiment, initial temperature determining module 780 can be also used for the facial image of acquisition and prestore people Face image is matched;When facial image and when prestoring facial image it fails to match, the corresponding initial parameter of facial image is obtained And the corresponding initial temperature of initial parameter;Image correction module 760 can be also used for carrying out facial image according to target component Correction.
In one embodiment, initial temperature determining module 780 can be also used for when facial image and prestore facial image When the number that it fails to match is more than preset times, the corresponding initial parameter of facial image and the corresponding initial temperature of initial parameter are obtained Degree.
In one embodiment, parameter acquisition module 740 can be also used for when the Current Temperatures of light source emitter and initial It is corresponding with temperature gap acquisition according to initial temperature when the temperature gap of temperature is more than that the duration of threshold value is more than preset duration Target component.
Image processing apparatus provided by the embodiments of the present application can detect the temperature of light source emitter, work as light source It is corresponding with temperature gap acquisition according to initial temperature when the Current Temperatures of transmitter and the temperature gap of initial temperature are more than threshold value Target component, preview image is handled according to target component, it is possible to reduce the deviation of image.
The division of modules is only used for for example, in other embodiments, can will scheme in above-mentioned image processing apparatus As processing unit is divided into different modules as required, to complete all or part of function of above-mentioned image processing apparatus.
Specific about image processing apparatus limits the restriction that may refer to above for image processing method, herein not It repeats again.Modules in above-mentioned image processing apparatus can be realized fully or partially through software, hardware and combinations thereof.On Stating each module can be embedded in the form of hardware or independently of in the processor in computer equipment, can also store in a software form In memory in computer equipment, the corresponding operation of the above modules is executed in order to which processor calls.
Fig. 8 is the schematic diagram of internal structure of electronic equipment in one embodiment.As shown in figure 8, the electronic equipment includes logical Cross processor, memory and the network interface of system bus connection.Wherein, which is used to provide calculating and control ability, Support the operation of entire electronic equipment.Memory for storing data, program etc., at least one computer journey is stored on memory Sequence, the computer program can be executed by processor, to realize the image suitable for electronic equipment provided in the embodiment of the present application Processing method.Memory may include non-volatile memory medium and built-in storage.Non-volatile memory medium is stored with operation system System and computer program.The computer program can be performed by processor, for realizing provided by following each embodiment A kind of image processing method.Built-in storage provides cache for the operating system computer program in non-volatile memory medium Running environment.Network interface can be Ethernet card or wireless network card etc., for being communicated with external electronic equipment.It should Electronic equipment can be mobile phone, tablet computer or personal digital assistant or wearable device etc..
Realizing for the modules in image processing apparatus provided in the embodiment of the present application can be the shape of computer program Formula.The computer program can be run in terminal or server.The program module that the computer program is constituted is storable in terminal Or on the memory of server.When the computer program is executed by processor, method described in the embodiment of the present application is realized Step.
The embodiment of the present application also provides a kind of computer readable storage mediums.One or more is executable comprising computer The non-volatile computer readable storage medium storing program for executing of instruction, when the computer executable instructions are executed by one or more processors When, so that the step of processor executes image processing method.
A kind of computer program product comprising instruction, when run on a computer, so that computer executes image Processing method.
The embodiment of the present application also provides a kind of electronic equipment.It include image processing circuit in above-mentioned electronic equipment, at image Reason circuit can use hardware and or software component realization, it may include define ISP (Image Signal Processing, figure As signal processing) the various processing units of pipeline.Fig. 9 is the schematic diagram of image processing circuit in one embodiment.Such as Fig. 9 institute Show, for purposes of illustration only, only showing the various aspects of image processing techniques relevant to the embodiment of the present application.
As shown in figure 9, image processing circuit includes ISP processor 940 and control logic device 950.Imaging device 910 captures Image data handled first by ISP processor 940, ISP processor 940 to image data analyzed with capture can be used for really The image statistics of fixed and/or imaging device 910 one or more control parameters.Imaging device 910 may include having one The camera of a or multiple lens 912 and imaging sensor 914.Imaging sensor 914 may include colour filter array (such as Bayer filter), imaging sensor 914 can obtain the luminous intensity captured with each imaging pixel of imaging sensor 914 and wavelength Information, and the one group of raw image data that can be handled by ISP processor 940 is provided.Sensor 920 (such as gyroscope) can be based on biography The parameter (such as stabilization parameter) of the image procossing of acquisition is supplied to ISP processor 940 by 920 interface type of sensor.Sensor 920 Interface can use SMIA (Standard Mobile Imaging Architecture, Standard Mobile Imager framework) interface, The combination of other serial or parallel camera interfaces or above-mentioned interface.
In addition, raw image data can also be sent to sensor 920 by imaging sensor 914, sensor 920 can be based on biography Raw image data is supplied to ISP processor 940 to 920 interface type of sensor or sensor 920 deposits raw image data It stores up in video memory 930.
ISP processor 940 handles raw image data pixel by pixel in various formats.For example, each image pixel can Bit depth with 8,10,12 or 14 bits, ISP processor 940 can carry out raw image data at one or more images Reason operation, statistical information of the collection about image data.Wherein, image processing operations can be by identical or different bit depth precision It carries out.
ISP processor 940 can also receive image data from video memory 930.For example, 920 interface of sensor will be original Image data is sent to video memory 930, and the raw image data in video memory 930 is available to ISP processor 940 It is for processing.Video memory 930 can be independent special in a part, storage equipment or electronic equipment of memory device It with memory, and may include DMA (Direct Memory Access, direct direct memory access (DMA)) feature.
When receiving from 914 interface of imaging sensor or from 920 interface of sensor or from video memory 930 When raw image data, ISP processor 940 can carry out one or more image processing operations, such as time-domain filtering.Treated schemes As data can be transmitted to video memory 930, to carry out other processing before shown.ISP processor 940 is from image Memory 930 receives processing data, and carries out in original domain and in RGB and YCbCr color space to the processing data Image real time transfer.Treated that image data may be output to display 970 for ISP processor 940, for user's viewing and/or It is further processed by graphics engine or GPU (Graphics Processing Unit, graphics processor).In addition, ISP processor 940 output also can be transmitted to video memory 930, and display 970 can read image data from video memory 930.? In one embodiment, video memory 930 can be configured to realize one or more frame buffers.In addition, ISP processor 940 Output can be transmitted to encoder/decoder 960, so as to encoding/decoding image data.The image data of coding can be saved, And it is decompressed before being shown in 970 equipment of display.Encoder/decoder 960 can be real by CPU or GPU or coprocessor It is existing.
The statistical data that ISP processor 940 determines, which can be transmitted, gives control logic device Unit 950.For example, statistical data can wrap Include the image sensings such as automatic exposure, automatic white balance, automatic focusing, flicker detection, black level compensation, 912 shadow correction of lens 914 statistical information of device.Control logic device 950 may include the processor and/or micro-control for executing one or more routines (such as firmware) Device processed, one or more routines can statistical data based on the received, determine the control parameter and ISP processor of imaging device 910 940 control parameter.For example, the control parameter of imaging device 910 may include 920 control parameter of sensor (such as gain, exposure The time of integration, stabilization parameter of control etc.), camera flash control parameter, 912 control parameter of lens (such as focus or zoom With focal length) or these parameters combination.ISP control parameter may include for automatic white balance and color adjustment (for example, in RGB During processing) 912 shadow correction parameter of gain level and color correction matrix and lens.
In the embodiment of the present application, the application is realized when which executes the computer program of storage on a memory In embodiment the step of image processing method.
Any reference to memory, storage, database or other media used in this application may include non-volatile And/or volatile memory.Suitable nonvolatile memory may include read-only memory (ROM), programming ROM (PROM), Electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.Volatile memory may include arbitrary access Memory (RAM), it is used as external cache.By way of illustration and not limitation, RAM is available in many forms, such as It is static RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic ram (DRDRAM) and memory bus dynamic ram (RDRAM).
The several embodiments of the application above described embodiment only expresses, the description thereof is more specific and detailed, but simultaneously The limitation to the application the scope of the patents therefore cannot be interpreted as.It should be pointed out that for those of ordinary skill in the art For, without departing from the concept of this application, various modifications and improvements can be made, these belong to the guarantor of the application Protect range.Therefore, the scope of protection shall be subject to the appended claims for the application patent.

Claims (10)

1. a kind of image processing method, which is characterized in that including:
The temperature of light source emitter is detected;
When the Current Temperatures of the light source emitter and the temperature gap of initial temperature are more than threshold value, according to the initial temperature Target component corresponding with temperature gap acquisition;
Preview image is corrected according to the target component.
2. the method according to claim 1, wherein described obtain according to the initial temperature and the temperature gap After taking corresponding target component, further include:
The initial temperature is updated according to the Current Temperatures of the light source emitter.
3. the method according to claim 1, wherein before the temperature to light source emitter detects, Including:
The starting to camera is received to request;
According to the initial parameter and the corresponding initial temperature of the initial parameter of camera described in the starting request.
4. the method according to claim 1, wherein before the temperature to light source emitter detects, Including:
Receive the 3D process instruction to preview image;
The corresponding initial parameter of preview image is obtained according to the 3D process instruction and the initial parameter is corresponding initial Temperature.
5. the method according to claim 1, wherein the method also includes:
The facial image of acquisition is matched with facial image is prestored;
When the facial image and when prestoring facial image it fails to match, the corresponding initial parameter of the facial image and institute are obtained State the corresponding initial temperature of initial parameter;
The temperature of light source emitter is detected;
When the Current Temperatures of the light source emitter and the temperature gap of initial temperature are more than threshold value, according to the initial temperature Target component corresponding with temperature gap acquisition;
The facial image is corrected according to the target component.
6. according to the method described in claim 5, it is characterized in that, the method also includes:
When the facial image is with the facial image number that it fails to match is prestored more than preset times, the face figure is obtained As corresponding initial parameter and the corresponding initial temperature of the initial parameter.
7. the method according to claim 1, wherein Current Temperatures when the light source emitter and initial When the temperature gap of temperature is more than threshold value, according to initial temperature target component corresponding with temperature gap acquisition, packet It includes:
When the duration that the Current Temperatures of the light source emitter and the temperature gap of initial temperature are more than threshold value is more than default When duration, according to initial temperature target component corresponding with temperature gap acquisition.
8. a kind of image processing apparatus, which is characterized in that including:
Temperature detecting module is detected for the temperature to light source emitter;
Parameter acquisition module, for being more than threshold value when the Current Temperatures of the light source emitter and the temperature gap of initial temperature When, according to initial temperature target component corresponding with temperature gap acquisition;
Image correction module, for being corrected according to the target component to preview image.
9. a kind of electronic equipment, including memory and processor, computer program, the computer are stored in the memory When program is executed by the processor, so that the processor executes the image procossing as described in any one of claims 1 to 7 The step of method.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program The step of method as described in any one of claims 1 to 7 is realized when being executed by processor.
CN201810867083.2A 2018-08-01 2018-08-01 Image processing method and device, electronic equipment and computer readable storage medium Active CN108924426B (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
CN201810867083.2A CN108924426B (en) 2018-08-01 2018-08-01 Image processing method and device, electronic equipment and computer readable storage medium
PCT/CN2019/075374 WO2020024575A1 (en) 2018-08-01 2019-02-18 Image processing method and apparatus, electronic device, and computer-readable storage medium
TW108120554A TWI708192B (en) 2018-08-01 2019-06-13 Image processing method, electronic device, and computer-readable storage medium
EP19186497.4A EP3605393A1 (en) 2018-08-01 2019-07-16 Image correction due to deviations caused by temperature changes of camera light emitter
US16/523,292 US11218650B2 (en) 2018-08-01 2019-07-26 Image processing method, electronic device, and computer-readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810867083.2A CN108924426B (en) 2018-08-01 2018-08-01 Image processing method and device, electronic equipment and computer readable storage medium

Publications (2)

Publication Number Publication Date
CN108924426A true CN108924426A (en) 2018-11-30
CN108924426B CN108924426B (en) 2020-01-10

Family

ID=64394135

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810867083.2A Active CN108924426B (en) 2018-08-01 2018-08-01 Image processing method and device, electronic equipment and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN108924426B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109863491A (en) * 2019-01-22 2019-06-07 深圳市汇顶科技股份有限公司 Living creature characteristic recognition system, method and terminal device
CN110544335A (en) * 2019-08-30 2019-12-06 北京市商汤科技开发有限公司 Object recognition system and method, electronic device, and storage medium
WO2020024575A1 (en) * 2018-08-01 2020-02-06 Oppo广东移动通信有限公司 Image processing method and apparatus, electronic device, and computer-readable storage medium
CN111339854A (en) * 2020-02-14 2020-06-26 北京猎户星空科技有限公司 Temperature measurement method, temperature measurement device, robot and computer storage medium
CN113078964A (en) * 2021-04-06 2021-07-06 昆山迈致治具科技有限公司 Parameter correction method, device, computer equipment and storage medium
CN113436088A (en) * 2021-06-09 2021-09-24 浙江兆晟科技股份有限公司 Real-time suppression method and system for thermal window effect of infrared image

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6628337B1 (en) * 2001-04-18 2003-09-30 Eastman Kodak Company Electronic camera with internal temperature increase controlled
CN102262335A (en) * 2010-05-28 2011-11-30 株式会社日立制作所 Imaging device
CN102542299A (en) * 2011-12-07 2012-07-04 惠州Tcl移动通信有限公司 Face recognition method, device and mobile terminal capable of recognizing face
CN103098110A (en) * 2010-03-17 2013-05-08 本田技研工业株式会社 Vehicle surroundings monitoring device
CN103424954A (en) * 2012-05-18 2013-12-04 佳能株式会社 Lens apparatus and image pickup system
CN107942463A (en) * 2017-11-02 2018-04-20 广东欧珀移动通信有限公司 Focusing method, device, computer equipment and readable storage medium storing program for executing

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6628337B1 (en) * 2001-04-18 2003-09-30 Eastman Kodak Company Electronic camera with internal temperature increase controlled
CN103098110A (en) * 2010-03-17 2013-05-08 本田技研工业株式会社 Vehicle surroundings monitoring device
CN102262335A (en) * 2010-05-28 2011-11-30 株式会社日立制作所 Imaging device
CN102542299A (en) * 2011-12-07 2012-07-04 惠州Tcl移动通信有限公司 Face recognition method, device and mobile terminal capable of recognizing face
CN103424954A (en) * 2012-05-18 2013-12-04 佳能株式会社 Lens apparatus and image pickup system
CN107942463A (en) * 2017-11-02 2018-04-20 广东欧珀移动通信有限公司 Focusing method, device, computer equipment and readable storage medium storing program for executing

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020024575A1 (en) * 2018-08-01 2020-02-06 Oppo广东移动通信有限公司 Image processing method and apparatus, electronic device, and computer-readable storage medium
US11218650B2 (en) 2018-08-01 2022-01-04 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Image processing method, electronic device, and computer-readable storage medium
CN109863491A (en) * 2019-01-22 2019-06-07 深圳市汇顶科技股份有限公司 Living creature characteristic recognition system, method and terminal device
CN109863491B (en) * 2019-01-22 2023-10-27 深圳市汇顶科技股份有限公司 Biometric identification system, method and terminal equipment
CN110544335A (en) * 2019-08-30 2019-12-06 北京市商汤科技开发有限公司 Object recognition system and method, electronic device, and storage medium
CN111339854A (en) * 2020-02-14 2020-06-26 北京猎户星空科技有限公司 Temperature measurement method, temperature measurement device, robot and computer storage medium
CN111339854B (en) * 2020-02-14 2023-06-30 北京猎户星空科技有限公司 Temperature measuring method, temperature measuring device, robot and computer storage medium
CN113078964A (en) * 2021-04-06 2021-07-06 昆山迈致治具科技有限公司 Parameter correction method, device, computer equipment and storage medium
CN113436088A (en) * 2021-06-09 2021-09-24 浙江兆晟科技股份有限公司 Real-time suppression method and system for thermal window effect of infrared image
CN113436088B (en) * 2021-06-09 2022-07-26 浙江兆晟科技股份有限公司 Real-time suppression method and system for thermal window effect of infrared image

Also Published As

Publication number Publication date
CN108924426B (en) 2020-01-10

Similar Documents

Publication Publication Date Title
CN108924426A (en) Image processing method and device, electronic equipment, computer readable storage medium
CN108716983B (en) Optical element detection method and device, electronic equipment, storage medium
JP7139452B2 (en) Image processing method, computer readable storage medium, and electronic device
CN108055452A (en) Image processing method, device and equipment
TWI709110B (en) Camera calibration method and apparatus, electronic device
CN108965721A (en) The control method and device of camera module, electronic equipment
JP2015197745A (en) Image processing apparatus, imaging apparatus, image processing method, and program
CN108716982A (en) Optical element detection method, device, electronic equipment and storage medium
CN109040745B (en) Camera self-calibration method and device, electronic equipment and computer storage medium
CN107948617B (en) Image processing method, image processing device, computer-readable storage medium and computer equipment
CN109712192A (en) Camera module scaling method, device, electronic equipment and computer readable storage medium
CN108600740A (en) Optical element detection method, device, electronic equipment and storage medium
CN108024054A (en) Image processing method, device and equipment
CN109327626A (en) Image-pickup method, device, electronic equipment and computer readable storage medium
CN108063891B (en) Image processing method, image processing device, computer-readable storage medium and computer equipment
CN109040746B (en) Camera calibration method and apparatus, electronic equipment, computer readable storage medium
CN110121031A (en) Image-pickup method and device, electronic equipment, computer readable storage medium
TWI708192B (en) Image processing method, electronic device, and computer-readable storage medium
CN109559353A (en) Camera module scaling method, device, electronic equipment and computer readable storage medium
CN109151303A (en) Image processing method and device, electronic equipment, computer readable storage medium
CN109584312A (en) Camera calibration method, device, electronic equipment and computer readable storage medium
CN108805025A (en) Laser output control method and device, electronic equipment, storage medium
CN109120846A (en) Image processing method and device, electronic equipment, computer readable storage medium
CN109068060A (en) Image processing method and device, terminal device, computer readable storage medium
CN108760245A (en) Optical element detection method and device, electronic equipment, readable storage medium storing program for executing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant