CN103491230B - Can the mobile terminal of automatic regulating volume and font and Automatic adjustment method thereof - Google Patents

Can the mobile terminal of automatic regulating volume and font and Automatic adjustment method thereof Download PDF

Info

Publication number
CN103491230B
CN103491230B CN201310397909.0A CN201310397909A CN103491230B CN 103491230 B CN103491230 B CN 103491230B CN 201310397909 A CN201310397909 A CN 201310397909A CN 103491230 B CN103491230 B CN 103491230B
Authority
CN
China
Prior art keywords
data
distance
range
volume
judge
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201310397909.0A
Other languages
Chinese (zh)
Other versions
CN103491230A (en
Inventor
朱磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Semiconductor China R&D Co Ltd
Samsung Electronics Co Ltd
Original Assignee
Samsung Semiconductor China R&D Co Ltd
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Semiconductor China R&D Co Ltd, Samsung Electronics Co Ltd filed Critical Samsung Semiconductor China R&D Co Ltd
Priority to CN201310397909.0A priority Critical patent/CN103491230B/en
Publication of CN103491230A publication Critical patent/CN103491230A/en
Application granted granted Critical
Publication of CN103491230B publication Critical patent/CN103491230B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The present invention propose a kind of can the mobile terminal of automatic regulating volume and font and Automatic adjustment method thereof.Mobile terminal has at least two distance sensor module and a camera module.This Automatic adjustment method comprises: the human eye or the relative position of people's ear in mobile terminal screen that obtain the range data between human body and mobile terminal and the camera module identification of measuring from first, second distance sensor module, described first distance sensor module is positioned near mobile terminal receiver position, second distance sensor assembly is positioned at bottom mobile terminal front, and camera module is positioned near the first distance sensor module position; Calculate the azimuth-range of face or people's ear relative movement terminal receiver; Automatically volume and the font size of mobile terminal is regulated again according to the control law preset.Use the mobile terminal according to automatic regulating volume of the present invention and font, volume and font size can be regulated accurately.

Description

Can the mobile terminal of automatic regulating volume and font and Automatic adjustment method thereof
Technical field
The present invention relates to a kind of can the mobile terminal of automatic regulating volume and font and Automatic adjustment method thereof, particularly relate to a kind of can by accurate location the mobile terminal of automatic regulating volume and font and Automatic adjustment method thereof.
Background technology
Along with the progress of society, portable terminal has become more and more universal, people use portable terminal to converse in daily life in a large number, carry out text event detection or carry out Entertainment, viewing video etc., but portable terminal only has fixing volume and word size usually, this is for the people of custom with different use portable terminals, or in different places (such as, on bed, chair is first-class) use the people of portable terminal, or for the people of different motion state (motion or static), the size of volume and word and user can be produced and occur unmatched situation to the distance of portable terminal, such as, to can not hear clearly phone, or the e-book do not seen in Chu's portable terminal or note, or the eyesight of font too macrolesion user.This situation can cause great inconvenience to user.
According to this needs, people have carried out a series of improvement to portable terminal.
Such as, there is a kind of volume adjusting method of mobile phone, it can utilize the infrared ray sensor being arranged on the earphone portions of mobile phone to measure the distance with user's ear when voice call, and according to distance adjustment the tinkle of bells of its mensuration and the volume of call connecting sound.It is made up of following several stages: the stage measuring user place distance with an infrared ray sensor, the stage of the reference range that more described mensuration Distance geometry has set, if described mensuration distance is less than the reference range set, then the volume rank exported by loud speaker is transformed to volume rank that user set and exports.
In addition, have also appeared a kind of method regulating the font size of mancarried device, it utilizes a range cells near receiver to measure the distance of portable video unit objects in front to portable video unit, match corresponding font size information from the distance preset with font size relation information according to the distance recorded, thus regulate the font size of the Word message for display.
But the present inventor, through carefully studying, finds that the equipment of prior art exists following problem:
Only the distance value between the object utilizing range sensor measurement to obtain and mobile terminal is as the foundation regulating volume and font, but the head of people has and faces mobile terminal, inclined plane to mobile terminal or head with the multiple situation of askew attitude facing to mobile terminal, and range sensor is measured usually be all in head part in the measurable space angular range of range sensor to the distance between the nearest point of range sensor and range sensor, therefore measured distance value inhuman ear or the real distance between human eye and mobile terminal, and the volume that people can hear and people's ear and other positions of non-head are to the distance dependent of mobile terminal, the size of the font that people can see clearly and eye and other positions of non-head are to the distance dependent of mobile terminal, therefore the adjustment of volume and font there will be larger error in different situations.
And the inadequate hommization of the adjustment of said apparatus, easily occur that meaningless mistake regulates.
In addition, misoperation is there will be when there are other objects in the measuring range of range sensor, such as, use portable terminal browsing page as user or see e-book, during video clip, finger whips sometimes before range cells, or around head part, sometimes have other foreign material (lamp holder of such as desk lamp), then now range cells can will be pointed, lamp holders etc. are mistakened as the head doing user, thus portable terminal can judge user from terminal very close to, thus display font (may also have volume) is adjusted very little, user is caused not see shown content (or not hearing play content).In addition, when people is with the orientation tilted very much viewing mobile terminal, also likely there is the situation of the range sensor measurement near receiver less than the head of people.
Summary of the invention
In order to improve prior art only utilize the range sensor be arranged near receiver measure distance signal between user and mobile terminal as adjustment font and volume Problems existing, the present invention proposes a kind of utilization and be arranged on multiple range sensor of different key positions and camera obtains human eye or the azimuth-range signal between people's ear and mobile terminal screen realizes the volume of mobile terminal and the automatic adjustment of font, to improve Consumer's Experience.
To achieve these goals, the method that the present invention proposes is applied to the mobile terminal with at least two distance sensor module and a camera module, the method comprises: obtain from first, range data between the human body that second distance sensor assembly is measured and mobile terminal and the human eye of camera module identification or the relative position of people's ear in mobile terminal screen, described first distance sensor module is positioned near mobile terminal receiver position, second distance sensor assembly is positioned at bottom mobile terminal front, camera module is positioned near the first distance sensor module position, calculate the distance and bearing of face or people's ear relative movement terminal receiver, automatically volume and the font size of mobile terminal is regulated again according to the control law preset.
According to an aspect of the present invention, a kind of portable terminal is provided, comprises: the first range sensor, for measuring the first range data between user and mobile phone; Second distance transducer, for measuring the second distance data between user and mobile phone; Camera, there is face identification functions, camera measures the bearing data of face in screen, processor, by judge the first range data and second distance data whether exceed setting measuring range and by identifying that the facial contour in the image of catching judges whether the image of catching is normal facial contour, determine the first range data and second distance data whether effective; Wherein, processor obtains the eyes of people and/or the ear final distance to loud speaker or receiver according to being judged as effective first range data and/or being judged as effective second distance data and described bearing data, then obtains with this finally apart from corresponding volume and/or displaying ratio.
Described portable terminal also comprises memory, and memory stores volume and the corresponding relation of described final distance and/or the corresponding relation of displaying ratio and described final distance.
Processor obtains corresponding volume and/or displaying ratio to the final distance of loud speaker or receiver from memory according to the eyes of people and/or ear.
Described portable terminal also comprises judge module, the volume that judge module receiving processor obtains and/or displaying ratio, the volume that decision processor obtains and/or the displaying ratio volume current with portable terminal and/or displaying ratio whether identical, if different, then trigger volume and change event and/or displaying ratio change event, automatically to regulate volume and/or the displaying ratio of portable terminal.
Processor judges that whether the volume that obtains and/or the displaying ratio volume current with portable terminal and/or displaying ratio be identical, if different, then automatically regulates volume and/or the displaying ratio of portable terminal.
Around the receiver that first range sensor and camera are positioned at portable terminal or loud speaker, second distance transducer is positioned at around the below of display screen or the bottom button of portable terminal.
Except the first range sensor and second distance transducer, portable terminal also comprises the range sensor being positioned at other positions, and the valid data in the range data that portable terminal records according to all range sensors regulate volume and/or the displaying ratio of portable terminal automatically.
Described portable terminal also comprises the camera being positioned at other positions, and portable terminal regulates volume and/or the displaying ratio of portable terminal automatically according to the bearing data that all cameras record.
According to a further aspect in the invention, a kind of automatic volume of adjustment portable terminal and/or the method for displaying ratio are provided, comprise: the first range data between user and mobile phone measured by the first range sensor; Second distance data between second distance sensor measurement user and mobile phone; Camera measures the bearing data of face in screen, and judge that whether the image of catching is normal facial contour and by judging whether the first range data and second distance data exceed the measuring range of setting by the facial contour identifying in the image of catching, determine the first range data and second distance data whether effective; Portable terminal obtains the eyes of people and/or the ear final distance to loud speaker or receiver according to being judged as effective first range data and/or being judged as effective second distance data and described bearing data, then obtain with this finally apart from corresponding volume and/or displaying ratio, and the volume of portable terminal and/or displaying ratio are adjusted to described corresponding volume and/or displaying ratio.
According to a further aspect in the invention, a kind of automatic volume of adjustment portable terminal and/or the method for displaying ratio are provided, comprise: the first range sensor is set and enters mode of operation, gather the first range data D1, camera is set and enters mode of operation, gathering the image catching portable terminal screen front in the time during D1 or contiguous, judge that whether the integrity degree of the facial contour in the image of catching is lower than predetermined ratio, when lower than predetermined ratio, judge that the first range data D1 is invalid, terminate the automatic adjustment of volume and/or displaying ratio, when being more than or equal to predetermined ratio, whether there is in the presumptive area of the image that judgement is caught the shape outside facial contour, if have the shape outside facial contour, then judge that the first range data D1 is invalid, enter step S206, if do not have the shape outside facial contour, then judge whether the first range data D1 exceeds the measuring range of setting, when exceeding the measuring range of setting, enter step S206, when not exceeding the measuring range of setting, then judge that the first range data D1 is effective, enter step S209, step S206: arrange second distance transducer and enter mode of operation, gathers second distance data D2, whether there is in the presumptive area of the image that judgement is caught the shape outside facial contour, if have the shape outside facial contour, then judge that second distance data D2 is invalid, terminate the automatic adjustment of volume and/or displaying ratio, if do not have the shape outside facial contour, then judge whether second distance data D2 exceeds the measuring range of setting, when exceeding the measuring range of setting, terminate the automatic adjustment of this volume and/or displaying ratio, when not exceeding the measuring range of setting, then judge that second distance data D2 is effective, enters step S209, step S209: by face identification functions identification people's ear or the orientation C of human eye in mobile terminal screen of camera, determine the ear of people or the eyes final distance D3 to loud speaker or receiver according to the valid data in D1 and D2 and orientation C, and then obtain corresponding level of sound volume R and/or displaying ratio P, the level of sound volume of portable terminal is adjusted to obtained level of sound volume R and/or the displaying ratio of the pattern in portable terminal or word is adjusted to obtained displaying ratio P.
First range sensor is positioned at around the receiver of portable terminal.
According to a further aspect in the invention, a kind of automatic volume of adjustment portable terminal and/or the method for displaying ratio are provided, comprise: the first range sensor is set and enters mode of operation, gather the first range data D1, camera is set and enters mode of operation, gathering the image catching portable terminal screen front in the time during D1 or contiguous, judge that whether the integrity degree of the facial contour in the image of catching is lower than predetermined ratio, when lower than predetermined ratio, judge that the first range data D1 is invalid, when being more than or equal to predetermined ratio, whether there is in the presumptive area of the image that judgement is caught the shape outside facial contour, if have the shape outside facial contour, then judge that the first range data D1 is invalid, if do not have the shape outside facial contour, then judge whether the first range data D1 exceeds the measuring range of setting, when exceeding the measuring range of setting, judge that the first range data D1 is invalid, when not exceeding the measuring range of setting, then judge that the first range data D1 is effective, second distance transducer is set and enters mode of operation, gather second distance data D2, judge that whether the integrity degree of the facial contour in the image of catching is lower than predetermined ratio, when lower than predetermined ratio, judge that second distance data D2 is invalid, when being more than or equal to predetermined ratio, whether there is in the presumptive area of the image that judgement is caught the shape outside facial contour, if have the shape outside facial contour, then judge that second distance data D2 is invalid, if do not have the shape outside facial contour, then judge whether second distance data D2 exceeds the measuring range of setting, when exceeding the measuring range of setting, judge that second distance data D2 is invalid, when not exceeding the measuring range of setting, then judge that second distance data D2 is effective, if the first range data D1, second distance data D2 are all invalid, then terminate this adjustment work, by face identification functions identification people's ear or the orientation C of human eye in mobile terminal screen of camera, determine the ear of people or the eyes final distance D3 to loud speaker or receiver according to the valid data in the first range data D1 and second distance data D2 and orientation C, and then obtain corresponding level of sound volume R and/or displaying ratio P, the level of sound volume of portable terminal is adjusted to obtained level of sound volume R and/or the displaying ratio of the pattern in portable terminal or word is adjusted to obtained displaying ratio P.
The described automatic volume of adjustment portable terminal and/or the method for displaying ratio comprise: after the data not having trigger event or acquire range sensor and camera, range sensor and camera are set to sleep pattern, to reduce power consumption.
The described automatic volume of adjustment portable terminal and/or the method for displaying ratio comprise: judge that face is in side face pattern or positive face pattern by camera, if be in side face pattern, then survey calculation ear is to the final distance of loud speaker or receiver, if be in positive face pattern, then survey calculation two ear line centers are to the final distance of loud speaker or receiver.
Judge the first range data D1 that the first range sensor gathers whether effectively time institute's reference presumptive area with judging that whether effective the presumptive area of second distance data D2 time institute's reference that second distance transducer gathers be different.
Use the mobile terminal according to automatic regulating volume of the present invention and font, volume and font size can be regulated accurately, greatly reduce the frequency of misoperation.
Accompanying drawing explanation
By below in conjunction with exemplarily illustrating the description that the accompanying drawing of an example carries out, above and other object of the present invention and feature will become apparent, wherein:
Fig. 1 diagrammatically illustrates the structure of mobile terminal according to an embodiment of the invention.
Fig. 2 is the indicative flowchart of the method for the volume automatically regulating mobile terminal according to an embodiment of the invention.
Fig. 3 is the indicative flowchart of the method for the font size automatically regulating mobile terminal according to an embodiment of the invention.
Embodiment
Below, embodiments of the invention are described in detail with reference to accompanying drawing.
Fig. 1 diagrammatically illustrates the structure of mobile terminal 10 according to an embodiment of the invention.Certainly, here only to have the mobile terminal of communication function, also can be other mancarried devices not possessing communication function, such as, e-book, game machine etc.Mobile terminal 10 comprises at least two range sensors 102 and 104, and a camera 106.
Specifically, mobile terminal 10 comprises: the first range sensor 102, for measuring the first range data D1 between user and mobile phone; Second distance transducer 104, for measuring the second distance data D2 between user and mobile phone; Camera 106, there is face identification functions, camera measures the bearing data of face in screen (such as, the relative position of the facial contour of catching that screen shows in screen and angle), and identify whether the image of catching is normal facial contour, whether effectively (will describe in detail) to judge the first range data and second distance data below; Wherein, mobile terminal 10, according to being judged as effective first range data and/or being judged as effective second distance data and described bearing data obtains the eyes of people and/or the position data of ear, then obtains the volume corresponding with this position data and/or displaying ratio.Described displaying ratio can be the displaying ratio (font size) of word, also can be the displaying ratio of pattern or icon etc.
Described range sensor adopts existing techniques in realizing, can be ultrasonic distance transducer, infrared sensor etc., for measuring the range data between user and mobile phone.Described camera is front-facing camera, and it should have face identification functions, for identifying the relative position of face in screen, specifically, may be used for the eyes and the orientation of ear in screen that identify people.
Mobile terminal 10 can comprise memory, and memory stores volume and the corresponding relation of distance and/or the corresponding relation of displaying ratio and distance, specifically, is the displaying ratio of distance level scale, level of sound volume, character or pattern and the corresponding relation between them.Mobile terminal 10 also can comprise CPU, CPU gathers and is judged as effective data and bearing data in the first range data and second distance data, calculate the eyes of people and/or the position data of ear, then obtain corresponding volume and/or displaying ratio according to this position data from memory.Mobile terminal 10 also can comprise judge module, it receives volume and/or the displaying ratio of CPU acquisition, judge that whether the volume that CPU obtains and/or the displaying ratio volume current with portable terminal and/or displaying ratio be identical, if different, then trigger volume and change event and/or displaying ratio change event, automatically to regulate volume and/or the displaying ratio of portable terminal.
Certainly, judge module also can not independently exist, and it can be a part of CPU.
Preferably, the first described range sensor 102 and camera 106 can be positioned at receiver 101 around, described second distance transducer 104 can be positioned at the below of display screen 103 or button 105 around, further preferably, multiple range sensor can be set near receiver 101, or in other positions (such as, second distance sensor proximity) of mobile terminal 10, multiple camera is set, to improve the accuracy of measurement.
Fig. 2 is the indicative flowchart of the method for the volume automatically regulating mobile terminal according to an embodiment of the invention.
The method is applied to the mobile terminal comprising at least two range sensors and a camera.In the present embodiment, comprise the first range sensor 102, second distance transducer 104 and camera 106 for one and receive calls at user's loud speaker or receiver or watch video, browse network, to play games etc. time use loud speaker (that is, loud speaker or receiver enable and under state that earphone is not enabled) mobile terminal automatic regulating volume illustrate.The method comprises the following steps:
S201, two range sensors are set and camera enters sleep pattern.This step is optional step, and preferably adopt this step, this is the power consumption in order to save mobile terminal, the initial condition of two range sensors and camera can be set to low power sleep mode.Certainly, also this initial setting up can not be carried out.
S202, judge whether trigger event.Trigger event is mainly used in notice mobile terminal and starts automatic sound-volume adjusting function.Trigger event mainly can manually boot automatic sound-volume adjusting function for user, or user starts phone functions, or user starts to start loud speaker etc., but is not limited to these.When judging there is no trigger event, then can continue to get back to step S201; When there being trigger event, then enter step S203.If the initial default state of two range sensors and camera is not low power sleep mode, if that is, there is no step S201, then step S202 can be saved.
S203, the first range sensor is set enters mode of operation, gather range data D1.That is, trigger the first range sensor and gather current distance signal.
S203b, camera is set enters mode of operation, during image data D1 or catch the image in mobile terminal front in the contiguous time.
S204, the first range sensor is set enters sleep pattern.This step is also optional, when power consumption saved by needs, arranges the sleep pattern of transducer.
S205a-S205c, judge that whether data D1 is effective.This point three levels, the first two level be used for the image of catching in determining step S203b be whether normal facial contour (namely, first level " judges whether the departure degree of head part and mobile terminal exceeds predetermined value " and the second level " judges whether to exist between the first range sensor and the head of user other foreign matter "), the 3rd level judges whether the distance between mobile terminal and user exceeds the measuring range of setting.
In the determining step S205a of the first level, if the integrity degree of the facial contour of catching is lower than predetermined ratio, such as, only photograph 60% of facial contour, then illustrate that head part and mobile terminal depart from (exceeding predetermined value) comparatively far away, the data that range sensor records are not probably that face reflects the data produced, and now directly judge data invalid, flow process can directly jump to " end ", terminates this automatic tuning work.If determine that in step S205a the integrity degree of the facial contour of catching is more than or equal to predetermined ratio, then enter the judgement (step S205b) of the second level, there is shape outside facial contour in the presumptive area of caught image (such as, the shape of finger or the truncated conical shape of lamp holder) time, then probably there is foreign matter, decision data D1 is invalid, when data D1 is invalid, enters step S206.The area of described presumptive area and position are rule of thumb, the screen size of mobile terminal, camera be in the position of mobile terminal and coverage setting.When whether the data D1 judging that according to the shape had outside facial contour in the presumptive area of image the first range sensor gathers is effective, described presumptive area is positioned at the middle and upper part of caught image; When whether the data D2 judging that according to the shape had outside facial contour in the presumptive area of image second distance transducer gathers is effective (step S206b below will discuss in detail), then presumptive area is now positioned at the middle and lower part of caught image, and this is all that those skilled in the art can set.
If do not have the shape outside facial contour in the presumptive area of the image of catching, then enter into the judgement (step S205c) of the 3rd level, judge whether the distance between mobile terminal and user exceeds the measuring range of setting, when exceeding the measuring range of setting, can the data-signal that records of determine sensor be invalid, the signal of what it recorded may be reflections off objects such as metope, now enters step S206; When the distance recorded does not exceed the measuring range of setting, then decision data D1 is effective, enters step S209.
It should be noted that, step S203b above preferably performs between S203 and S205a.Certain S203b also can perform with S203 simultaneously (because captured image may also will use when second distance sensor measurement distance, it is not best for therefore while the first range sensor image data, taking facial image) or perform when performing step S205b or S205c.Step S203b can also perform after S205c, but must ensure the time of implementation that can not be later than step S206.
S206, second distance transducer is set enters mode of operation, gather range data D2.When the automatic sound-volume adjusting function of mobile terminal triggers, preferentially using the distance signal of the first range sensor as the basis for estimation of control law.In the embodiment illustrated in figure 2, only have when data D1 is invalid, just gather the range data of second distance transducer.In another embodiment, even if data D1 is effective, also can gather range data D2, thus utilize data D1 come with data D2 together with determine the position of people's ear.That is, even if the judged result being now equivalent to the S205c of Fig. 2 is "No", flow process also still forwards S206 to.
S207, second distance transducer is set enters sleep pattern.This is optional, when power consumption saved by needs, arranges the sleep pattern of transducer.
S208b-S208c, judge that whether data D2 is effective, it also carries out the judgement of two levels, namely, first as described in step S205b, whether there is shape outside facial contour (such as in the presumptive area judging caught image in step S208b, the shape of finger or the truncated conical shape of lamp holder), wherein, presumptive area is positioned at the middle and lower part (this is that those skilled in the art can set) of caught image.When having the shape outside facial contour, data D2 is invalid, terminates this time to regulate volume work.When not having the shape outside facial contour, then entering step S208c, judging whether the distance between mobile terminal and user exceeds the measuring range of setting, if exceeded, then judge that data D2 is invalid, terminate this time to regulate volume work; If do not exceeded, then judge that data D2 is effective, enter step S209.In another embodiment described (not illustrating in Fig. 2), when data D2 is invalid and D1 is effective, still enter step S209, if D2 and D1 is all invalid, then terminate this time to regulate volume work.
S209, for caught image, people's ear relative position and quantity of people's ear of photographing in mobile terminal screen is identified by face identification functions identification multi-group data, and then determine the relative position C of people's ear and the first range sensor and/or second distance transducer, and confirm that current face is in side face pattern or positive face pattern, such as then think when recognizing in image and only there is people's helix exterior feature and be in side face pattern, when can recognize in image there is two people's ears time, then think and be in positive face pattern.When being in side face pattern, determine the relative position of this people's ear and camera; When being in positive face pattern, then determine the position at two ear line centers and the relative position C of camera.Because the relative position of camera and range sensor is determined, so taken people's ear or the relative position C of two ear line centers and the first range sensor and/or second distance transducer finally can be obtained.
S210, camera is set enters sleep pattern.This is also optional, when power consumption saved by needs, arranges the sleep pattern of camera.
S211, people's ear or two the ear line centers 3 D stereo coordinate relative to the first range sensor and/or second distance transducer can be determined according to the relative position C of data D1 and/or D2 and ear and camera, thus finally calculate people's ear or two the ear line centers distance D3 to loud speaker (or receiver), and then obtain corresponding level of sound volume R.In order to realize the relative stability that volume exports, volume can be exported in system and being divided into multiple grade, selecting different level of sound volume along with the change of the distance between mobile terminal and user.
S212, judge whether the current level of sound volume r of mobile terminal equals level of sound volume R;
When both are identical, then show that the distance between mobile terminal and user does not change or change very little, and then terminate this time to regulate volume work; When both are different, then enter step S213.
S213, triggering volume change event.Now show that the distance between mobile terminal and user there occurs certain change, need the output size changing volume.
In one embodiment, mobile terminal preferentially obtains the range data of the first range sensor to regulate volume, only has when first sensor data invalid, just adopts the range data of second distance transducer; In another embodiment, mobile terminal considers the range data of two range sensors to judge the position of last ear.Alternatively, the distance signal of more range sensor also can be adopted in system as basis for estimation, regulate volume more accurately.In this example, camera mainly identifies the relative position of people's ear, and avoids the various chaff interference such as staff, desk lamp to cause range sensor to carry out misoperation, can regulate volume more accurately like this.
Figure 3 shows that embodiment of the present invention mobile terminal regulates a kind of schematic flow sheet of the method for display font automatically.The method is applied to the mobile terminal comprising at least two range sensors and a camera.In the present embodiment, the mobile terminal specifically comprising the first range sensor 102, second distance transducer 104 and camera 106 with regulates font size to illustrate automatically.The method comprises:
S301, two range sensors are set and camera enters sleep pattern.As previously mentioned, this is optional.
S302, judge whether trigger event.Trigger event is mainly used in notice mobile terminal and starts adjustment display font function automatically.Trigger event mainly can regulate font functionalities for user manually boots automatically, or user's opening document, or user starts browsing page etc., but is not limited to these.When judging there is no trigger event, then can continue to get back to step S301; When there being trigger event, then enter step S303.If the initial default state of two range sensors and camera is not low power sleep mode, if that is, there is no step S301, then step S202 can be saved.
S303, the first range sensor is set enters mode of operation, gather range data D1.That is, trigger the first range sensor and gather current distance signal.
S303b, camera is set enters mode of operation, catch facial contour image.
S304, the first range sensor is set enters sleep pattern.This is optional.
S305a-S305c, judge that whether data D1 is effective.This point three levels, the first two level whether be the image of catching in determining step S303b be normal facial contour (namely, first level " judges whether the departure degree of head part and mobile terminal exceeds predetermined value " and the second level " judges whether to exist between the first range sensor and the head of user other foreign matter "), the 3rd level judges whether the distance between mobile terminal and user exceeds the measuring range of setting.
In the determining step S305a of the first level, if the integrity degree of the facial contour of catching is lower than predetermined ratio, such as, only photograph 60% of facial contour, then illustrate that head part and mobile terminal depart from (exceeding predetermined value) comparatively far away, the data that range sensor records are not probably that face reflects the data produced, and now directly judge data invalid, flow process can directly jump to " end ", terminates this automatic tuning work.If determine that in step S305a the integrity degree of the facial contour of catching is more than or equal to predetermined ratio, then enter the judgement (step S305b) of the second level, there is shape outside facial contour in the presumptive area of caught image (such as, the shape of finger or the truncated conical shape of lamp holder) time, then probably there is foreign matter, decision data D1 is invalid, when data D1 is invalid, enters step S306.As mentioned above, judge data D1 that the first range sensor gathers whether effectively time institute's reference presumptive area with judging that whether effective the presumptive area of data D2 time institute's reference that second distance transducer gathers be different.
If do not have the shape outside facial contour in the presumptive area of the image of catching, then enter into the judgement (step S305c) of the 3rd level, judge whether the distance between mobile terminal and user exceeds the measuring range of setting, when exceeding the measuring range of setting, can the data-signal that records of determine sensor be invalid, the signal of what it recorded may be reflections off objects such as metope, now enters step S306; When the distance recorded does not exceed the measuring range of setting, then decision data D1 is effective, enters step S309.
It should be noted that, step S303b above preferably performs between S303 and S305a.Certain S303b also can perform with S303 simultaneously (because captured image may also will use when second distance sensor measurement distance, it is not best for therefore while the first range sensor image data, taking facial image) or perform when performing step S305b or S305c.Step S303b can also perform after S305c, but must ensure the time of implementation that can not be later than step S306.
S306, second distance transducer is set enters mode of operation, gather range data D2.Trigger second distance transducer and gather current distance signal.In the embodiment of shown in Fig. 3, only have when data D1 is invalid, just gather the range data of second distance transducer.In another embodiment, even if data D1 is effective, also can gather range data D2, thus utilize data D1 come with data D2 together with determine the position of eye.That is, even if the judged result being now equivalent to the S305c of Fig. 3 is "No", flow process also still forwards S306 to.
S307, second distance transducer is set enters sleep pattern; Optionally.
S308b-S308c, judge that whether data D2 is effective, it also carries out the judgement of two levels, namely, first as described in step S305b, whether there is shape outside facial contour (such as in the presumptive area judging caught image in step S308b, the shape of finger or the truncated conical shape of lamp holder), wherein, presumptive area is positioned at the middle and lower part (this is that those skilled in the art can set) of caught image.When having the shape outside facial contour, data D2 is invalid, terminates this time to regulate volume work.When not having the shape outside facial contour, then entering step S308c, judging whether the distance between mobile terminal and user exceeds the measuring range of setting, if exceeded, then judge that data D2 is invalid, terminate this time to regulate volume work; If do not exceeded, then judge that data D2 is effective, enter step S309.In another embodiment described (not illustrating in Fig. 3), when data D2 is invalid and D1 is effective, still enter step S309, if D2 and D1 is all invalid, then terminate this time to regulate volume work.
S309, by face identification functions identification human eye relative position in mobile terminal screen, and then determine the position at two eyes line centers and the relative position C of the first range sensor and/or second distance transducer.Wherein, camera determines the relative position of two eyes line centers and camera by identification multi-group data based on face recognition technology, because the relative position of camera and range sensor is determined, so the relative position C of two eyes line centers and the first range sensor and/or second distance transducer finally can be obtained.
S310, camera is set enters sleep pattern.This is optional.
S311, the 3 D stereo coordinate of two eyes line centers relative to the first range sensor and/or second distance transducer can be determined according to the relative position C of data D1 and/or D2 and ear and camera, thus finally calculate the distance D3 of two eyes line centers to loud speaker (or receiver), and then obtain corresponding level of sound volume R.In another embodiment recited above, in order to judge the distance between user and screen more accurately, gather the range data D1 of the first and second range sensors and D2 and the mean value calculating both as the current distance between user and mobile terminal, again it is revised, and then determine positional distance D3 according to relative position C.
S312, obtain corresponding displaying ratio P according to D3.Can preset displaying ratio scope in system, as reduce in scale has 10% ~ 90%, magnification ratio is by 110% ~ 200% etc.The displaying ratio P of current the best is determined according to range data D3.
S313, judge whether displaying ratio P equals the current font displaying ratio p1 of mobile terminal.When both are identical, then show that the distance between mobile terminal and user does not change or change very little, and then terminate this time to regulate font work; When both are different, then enter step S314.
S314, triggering font displaying ratio change event.Now show that the distance between mobile terminal and user there occurs certain change, need the display size changing font.
By the present embodiment, mobile terminal, by two range data, judges the distance between mobile terminal and user more accurately, and corresponding automatic adjustment.
In addition, the invention is not restricted to the size that automatically can regulate font, similarly, can regulate the displaying contents such as picture, icon or pattern in screen size.
In sum, the present invention obtains human eye or the distance and bearing data between people's ear and mobile terminal by range sensor and camera, judge the best volume that these data are corresponding and font size according to the control law preset again, thus realize the volume and the font size that automatically regulate mobile terminal.
For the object promoted the understanding of principle of the present invention, the preferred embodiment shown in accompanying drawing is illustrated, and has employed specific language to describe these embodiments.But this specific language is not intended to limit the scope of the invention, the present invention should be construed as comprising all embodiments that usually there will be for those of ordinary skills.In addition, except not element is specifically described as " requisite " or " key ", otherwise element or module is not had to be absolutely necessary to enforcement of the present invention.
Although described exemplary embodiment of the present invention in detail above, there is common practise person in the technical field of the invention without departing from the spirit and scope of the present invention, various amendments, retouching and modification can have been made to embodiments of the invention.But should be appreciated that In the view of those skilled in the art, these amendments, retouching and modification will fall in the spirit and scope of the exemplary embodiment of the present invention that claim limits.
Finally, unless to point out here or in addition and the obvious contradiction of context, otherwise methodical step described herein can perform with the order of any appropriate.

Claims (12)

1. a portable terminal, comprising:
First range sensor, for measuring the first range data between user and mobile phone;
Second distance transducer, for measuring the second distance data between user and mobile phone;
Camera, has face identification functions, and camera measures the bearing data of face in screen,
Processor, judge that whether the integrity degree of the facial contour in the image that camera is caught is lower than predetermined ratio, when described integrity degree is more than or equal to predetermined ratio, whether there is in the presumptive area of the image that judgement is caught the shape outside facial contour, when not having the shape outside facial contour, judge whether the first range data and second distance data exceed the measuring range of setting, if the first range data and the second data distance do not exceed the measuring range of setting, then determine the first range data and second distance data effective;
Wherein, processor obtains the eyes of people and/or the ear final distance to loud speaker or receiver according to being judged as effective first range data and/or being judged as effective second distance data and described bearing data, then obtains with this finally apart from corresponding volume and/or displaying ratio.
2. portable terminal according to claim 1, also comprises:
Memory, memory stores volume and the corresponding relation of described final distance and/or the corresponding relation of displaying ratio and described final distance,
Wherein, processor obtains corresponding volume and/or displaying ratio to the final distance of loud speaker or receiver from memory according to the eyes of people and/or ear.
3. portable terminal according to claim 2, also comprises:
Judge module, the volume that receiving processor obtains and/or displaying ratio, the volume that decision processor obtains and/or the displaying ratio volume current with portable terminal and/or displaying ratio whether identical, if different, then trigger volume and change event and/or displaying ratio change event, automatically to regulate volume and/or the displaying ratio of portable terminal.
4. portable terminal according to claim 2, wherein,
Processor judges that whether the volume that obtains and/or the displaying ratio volume current with portable terminal and/or displaying ratio be identical, if different, then automatically regulates volume and/or the displaying ratio of portable terminal.
5. the portable terminal according to any one of claim 1-4, wherein,
Around the receiver that first range sensor and camera are positioned at portable terminal or loud speaker, second distance transducer is positioned at around the below of display screen or the bottom button of portable terminal.
6. the portable terminal according to any one of claim 1-4, wherein,
Except the first range sensor and second distance transducer, portable terminal also comprises the range sensor being positioned at other positions, and the valid data in the range data that portable terminal records according to all range sensors regulate volume and/or the displaying ratio of portable terminal automatically.
7. automatically regulate the volume of portable terminal and/or a method for displaying ratio, comprising:
The first range data between user and mobile phone measured by first range sensor;
Second distance data between second distance sensor measurement user and mobile phone;
Camera measures the bearing data of face in screen, and whether the integrity degree of the facial contour in judging the image that camera is caught is lower than predetermined ratio, when described integrity degree is more than or equal to predetermined ratio, whether there is in the presumptive area of the image that judgement is caught the shape outside facial contour, when not having the shape outside facial contour, judge whether the first range data and second distance data exceed the measuring range of setting, if the first range data and the second data distance do not exceed the measuring range of setting, then determine the first range data and second distance data effective,
Portable terminal obtains the eyes of people and/or the ear final distance to loud speaker or receiver according to being judged as effective first range data and/or being judged as effective second distance data and described bearing data, then obtain with this finally apart from corresponding volume and/or displaying ratio, and the volume of portable terminal and/or displaying ratio are adjusted to described corresponding volume and/or displaying ratio.
8. method according to claim 7, comprising:
Judge that face is in side face pattern or positive face pattern by camera, if be in side face pattern, then survey calculation ear is to the final distance of loud speaker or receiver, if be in positive face pattern, then survey calculation two ear line centers are to the final distance of loud speaker or receiver.
9. automatically regulate the volume of portable terminal and/or a method for displaying ratio, comprising:
First range sensor is set and enters mode of operation, gather the first range data D1;
Camera is set and enters mode of operation, gathering the image catching portable terminal screen front in the time during D1 or contiguous;
Judge that whether the integrity degree of the facial contour in the image of catching is lower than predetermined ratio, when lower than predetermined ratio, judge that the first range data D1 is invalid, terminate the automatic adjustment of volume and/or displaying ratio, when being more than or equal to predetermined ratio, whether there is in the presumptive area of the image that judgement is caught the shape outside facial contour, if have the shape outside facial contour, then judge that the first range data D1 is invalid, enter step S206, if do not have the shape outside facial contour, then judge whether the first range data D1 exceeds the measuring range of setting, when exceeding the measuring range of setting, enter step S206, when not exceeding the measuring range of setting, then judge that the first range data D1 is effective, enter step S209,
Step S206: arrange second distance transducer and enter mode of operation, gathers second distance data D2;
Whether there is in the presumptive area of the image that judgement is caught the shape outside facial contour, if have the shape outside facial contour, then judge that second distance data D2 is invalid, terminate the automatic adjustment of volume and/or displaying ratio, if do not have the shape outside facial contour, then judge whether second distance data D2 exceeds the measuring range of setting, when exceeding the measuring range of setting, terminate the automatic adjustment of this volume and/or displaying ratio, when not exceeding the measuring range of setting, then judge that second distance data D2 is effective, enters step S209;
Step S209: by face identification functions identification people's ear or the orientation C of human eye in mobile terminal screen of camera;
Determine the ear of people or the eyes final distance D3 to loud speaker or receiver according to the valid data in D1 and D2 and orientation C, and then obtain corresponding level of sound volume R and/or displaying ratio P;
The level of sound volume of portable terminal is adjusted to obtained level of sound volume R and/or the displaying ratio of the pattern in portable terminal or word is adjusted to obtained displaying ratio P.
10. method according to claim 9, comprising:
Judge that face is in side face pattern or positive face pattern by camera, if be in side face pattern, then survey calculation ear is to the final distance of loud speaker or receiver, if be in positive face pattern, then survey calculation two ear line centers are to the final distance of loud speaker or receiver.
11. 1 kinds automatically regulate the volume of portable terminal and/or the method for displaying ratio, comprising:
First range sensor is set and enters mode of operation, gather the first range data D1;
Camera is set and enters mode of operation, gathering the image catching portable terminal screen front in the time during D1 or contiguous;
Judge that whether the integrity degree of the facial contour in the image of catching is lower than predetermined ratio, when lower than predetermined ratio, judge that the first range data D1 is invalid, when being more than or equal to predetermined ratio, whether there is in the presumptive area of the image that judgement is caught the shape outside facial contour, if have the shape outside facial contour, then judge that the first range data D1 is invalid, if do not have the shape outside facial contour, then judge whether the first range data D1 exceeds the measuring range of setting, when exceeding the measuring range of setting, judge that the first range data D1 is invalid, when not exceeding the measuring range of setting, then judge that the first range data D1 is effective,
Second distance transducer is set and enters mode of operation, gather second distance data D2;
Judge that whether the integrity degree of the facial contour in the image of catching is lower than predetermined ratio, when lower than predetermined ratio, judge that second distance data D2 is invalid, when being more than or equal to predetermined ratio, whether there is in the presumptive area of the image that judgement is caught the shape outside facial contour, if have the shape outside facial contour, then judge that second distance data D2 is invalid, if do not have the shape outside facial contour, then judge whether second distance data D2 exceeds the measuring range of setting, when exceeding the measuring range of setting, judge that second distance data D2 is invalid, when not exceeding the measuring range of setting, then judge that second distance data D2 is effective, if the first range data D1, second distance data D2 are all invalid, then terminate this adjustment work,
By face identification functions identification people's ear or the orientation C of human eye in mobile terminal screen of camera;
Determine the ear of people or the eyes final distance D3 to loud speaker or receiver according to the valid data in the first range data D1 and second distance data D2 and orientation C, and then obtain corresponding level of sound volume R and/or displaying ratio P;
The level of sound volume of portable terminal is adjusted to obtained level of sound volume R and/or the displaying ratio of the pattern in portable terminal or word is adjusted to obtained displaying ratio P.
12. automatic volumes of adjustment portable terminal according to any one of claim 11 and/or the method for displaying ratio, comprising:
Judge that face is in side face pattern or positive face pattern by camera, if be in side face pattern, then survey calculation ear is to the final distance of loud speaker or receiver, if be in positive face pattern, then survey calculation two ear line centers are to the final distance of loud speaker or receiver.
CN201310397909.0A 2013-09-04 2013-09-04 Can the mobile terminal of automatic regulating volume and font and Automatic adjustment method thereof Active CN103491230B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310397909.0A CN103491230B (en) 2013-09-04 2013-09-04 Can the mobile terminal of automatic regulating volume and font and Automatic adjustment method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310397909.0A CN103491230B (en) 2013-09-04 2013-09-04 Can the mobile terminal of automatic regulating volume and font and Automatic adjustment method thereof

Publications (2)

Publication Number Publication Date
CN103491230A CN103491230A (en) 2014-01-01
CN103491230B true CN103491230B (en) 2016-01-27

Family

ID=49831181

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310397909.0A Active CN103491230B (en) 2013-09-04 2013-09-04 Can the mobile terminal of automatic regulating volume and font and Automatic adjustment method thereof

Country Status (1)

Country Link
CN (1) CN103491230B (en)

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3140998A4 (en) * 2014-05-05 2017-10-25 Harman International Industries, Incorporated Speaker
CN105338156A (en) * 2014-08-07 2016-02-17 中兴通讯股份有限公司 Mode judging device and method, hearing protection device and method and terminal
CN105450832A (en) * 2014-08-19 2016-03-30 富强 Personalized mobile phone system
CN104468912B (en) * 2014-12-08 2018-01-16 上海斐讯数据通信技术有限公司 A kind of control method and its display terminal of mobile terminal screen brightness
CN104536560B (en) * 2014-12-10 2017-10-27 广东欧珀移动通信有限公司 A kind of method and device for adjusting terminal font size
CN104571885B (en) * 2014-12-11 2019-04-23 深圳市金立通信设备有限公司 A kind of terminal
CN104575552A (en) * 2014-12-11 2015-04-29 深圳市金立通信设备有限公司 Method for regulating playing progress
US9489172B2 (en) * 2015-02-26 2016-11-08 Motorola Mobility Llc Method and apparatus for voice control user interface with discreet operating mode
CN104683696A (en) * 2015-03-10 2015-06-03 山东超越数控电子有限公司 Method for realizing fast and accurate self-snapshooting of camera based on ultrasonic measurement
CN105183160B (en) * 2015-09-01 2018-08-10 联想(北京)有限公司 A kind of information processing method and electronic equipment
CN105159641B (en) * 2015-09-02 2020-03-24 联想(北京)有限公司 Information processing method and flexible electronic equipment
CN105657126B (en) * 2015-12-31 2018-12-25 宇龙计算机通信科技(深圳)有限公司 Method for controlling volume, sound volume control device and terminal for terminal
GB2548157A (en) * 2016-03-11 2017-09-13 Sony Europe Ltd Apparartus, method and computer program
CN106529449A (en) * 2016-11-03 2017-03-22 英华达(上海)科技有限公司 Method for automatically adjusting the proportion of displayed image and its display apparatus
CN106375605A (en) * 2016-11-18 2017-02-01 惠州Tcl移动通信有限公司 Phone answering/hanging-up method, phone answering/hanging-up system based on double distance sensors, and mobile terminal
CN106534982A (en) * 2016-11-28 2017-03-22 南京九致信息科技有限公司 Television apparatus and volume balance control method
CN106791058A (en) * 2016-12-09 2017-05-31 广西大学 A kind of method and apparatus based on user distance adjustment display effect
CN106686493B (en) * 2017-01-23 2019-08-27 瑞声科技(南京)有限公司 The method and system of adjust automatically sound quality
CN106954143B (en) * 2017-03-02 2018-11-02 瑞声科技(南京)有限公司 Manually adjust the method and electronic equipment of sound quality
CN106897713A (en) * 2017-03-13 2017-06-27 宇龙计算机通信科技(深圳)有限公司 A kind of method and mobile terminal for waking up mobile terminal screen
CN107483659B (en) * 2017-07-13 2019-12-10 Oppo广东移动通信有限公司 Control method and device based on proximity sensor and mobile terminal
CN107506171B (en) * 2017-08-22 2021-09-28 深圳传音控股股份有限公司 Audio playing device and sound effect adjusting method thereof
CN108243277A (en) * 2017-12-12 2018-07-03 惠州Tcl移动通信有限公司 Mobile terminal and its control method, storage device
CN108062197A (en) * 2017-12-21 2018-05-22 西安钛克韦尔信息科技有限公司 A kind of screen adjustment method and mobile terminal
CN107995559B (en) * 2017-12-26 2020-12-01 上海展扬通信技术有限公司 Audio control terminal and method
CN109067998B (en) * 2018-09-14 2020-12-25 奇酷互联网络科技(深圳)有限公司 Sound adjusting method, terminal equipment and device with storage function
CN110418023B (en) * 2019-07-31 2021-08-17 Oppo广东移动通信有限公司 Ring processing method, device, mobile terminal and storage medium
CN111107214A (en) * 2019-12-09 2020-05-05 西安易朴通讯技术有限公司 Volume adjusting method and device
CN111327749A (en) * 2020-02-28 2020-06-23 维沃移动通信有限公司 Volume control method and electronic equipment
CN112004151B (en) * 2020-08-20 2023-10-03 深圳前海茂佳软件科技有限公司 Control method of television equipment, television equipment and readable storage medium
CN112904986A (en) * 2021-03-24 2021-06-04 重庆电子工程职业学院 Computer auxiliary support
CN113965641B (en) * 2021-09-16 2023-03-28 Oppo广东移动通信有限公司 Volume adjusting method and device, terminal and computer readable storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101969509A (en) * 2010-11-02 2011-02-09 华为终端有限公司 Method for automatically adjusting call volume and communication terminal
CN102752438A (en) * 2011-04-20 2012-10-24 中兴通讯股份有限公司 Method and device for automatically regulating terminal interface display
CN103220432A (en) * 2013-05-10 2013-07-24 上海华勤通讯技术有限公司 Mobile terminal and call on-off control method thereof

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7589631B2 (en) * 2006-06-09 2009-09-15 Sony Ericsson Mobile Communications Ab Adaptive audio signals in portable communications devices
US9131060B2 (en) * 2010-12-16 2015-09-08 Google Technology Holdings LLC System and method for adapting an attribute magnification for a mobile communication device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101969509A (en) * 2010-11-02 2011-02-09 华为终端有限公司 Method for automatically adjusting call volume and communication terminal
CN102752438A (en) * 2011-04-20 2012-10-24 中兴通讯股份有限公司 Method and device for automatically regulating terminal interface display
CN103220432A (en) * 2013-05-10 2013-07-24 上海华勤通讯技术有限公司 Mobile terminal and call on-off control method thereof

Also Published As

Publication number Publication date
CN103491230A (en) 2014-01-01

Similar Documents

Publication Publication Date Title
CN103491230B (en) Can the mobile terminal of automatic regulating volume and font and Automatic adjustment method thereof
CN109361865B (en) Shooting method and terminal
CN109381165B (en) Skin detection method and mobile terminal
CN106203254B (en) Method and device for adjusting photographing direction
WO2020020134A1 (en) Photographing method and mobile terminal
CN108848313B (en) Multi-person photographing method, terminal and storage medium
CN107864336B (en) A kind of image processing method, mobile terminal
CN110933468A (en) Playing method, playing device, electronic equipment and medium
CN108881544B (en) Photographing method and mobile terminal
CN108012026B (en) Eyesight protection method and mobile terminal
CN108174103A (en) A kind of shooting reminding method and mobile terminal
CN109658886A (en) A kind of control method and terminal of display screen
CN110300267A (en) Photographic method and terminal device
CN109474787A (en) A kind of photographic method, terminal device and storage medium
CN109819167A (en) A kind of image processing method, device and mobile terminal
CN108762877A (en) A kind of control method and mobile terminal of interface of mobile terminal
CN109688325A (en) A kind of image display method and terminal device
CN107277364B (en) Shooting method, mobile terminal and computer readable storage medium
CN108345657B (en) Picture screening method and mobile terminal
CN109005337A (en) A kind of photographic method and terminal
CN109743448A (en) Reminding method and terminal device
CN109547700A (en) Photographic method and terminal
CN110837557B (en) Abstract generation method, device, equipment and medium
CN110740270B (en) Camera adjusting method and device, electronic equipment and storage medium
CN112100528A (en) Method, device, equipment and medium for training search result scoring model

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant