CN103303224B - Vehicle-mounted equipment gesture control system and usage method thereof - Google Patents

Vehicle-mounted equipment gesture control system and usage method thereof Download PDF

Info

Publication number
CN103303224B
CN103303224B CN201310240524.3A CN201310240524A CN103303224B CN 103303224 B CN103303224 B CN 103303224B CN 201310240524 A CN201310240524 A CN 201310240524A CN 103303224 B CN103303224 B CN 103303224B
Authority
CN
China
Prior art keywords
manipulation
mobile unit
control
camera
equipment
Prior art date
Application number
CN201310240524.3A
Other languages
Chinese (zh)
Other versions
CN103303224A (en
Inventor
汝正阳
张向文
蔡其瑾
朱喜燕
许勇
党选举
张进
李靖
古昂
Original Assignee
桂林电子科技大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 桂林电子科技大学 filed Critical 桂林电子科技大学
Priority to CN201310240524.3A priority Critical patent/CN103303224B/en
Publication of CN103303224A publication Critical patent/CN103303224A/en
Application granted granted Critical
Publication of CN103303224B publication Critical patent/CN103303224B/en

Links

Abstract

The invention discloses a vehicle-mounted equipment gesture control system and a usage method thereof. A CPU (central processing unit) at a vehicle-mounted platform part of the system is connected with an electronic control unit of vehicle-mounted equipment through a CAN (controller area network); the CPU is connected with a projector; a DSP (digital signal processor) at a gesture recognition part of the system is connected with two sets of identical and parallelly arranged infrared cameras; and the CPU is connected with the DSP which is connected with an infrared gain. The usage method comprises the steps as follows: the CPU acquires manipulation information of each vehicle-mounted equipment through the CAN and generates a manipulation map, the manipulation map is projected on a piece of front windshield glass, a finger moves in front of a lens, a related area of the manipulation map is discolored/ deformed to display options, and the finger clicks the determination in the air; the equipment required to be manipulated is determined by the DSP according to the finger position and a moving signal which are captured by the cameras; and the CPU controls corresponding equipment states or takes information according to the manipulation signal. A driver manipulates the automotive electronic equipment without transferring eyesight to an operation panel, different gestures are not required to be memorized, and the manipulation is convenient, simple and safe; and besides, CANs of various automobiles are compatible with the system, and the mounting and the use are facilitated.

Description

Mobile unit gestural control system and using method thereof

Technical field

The present invention relates to a kind of automobile equipment electronic control system, particularly a kind of mobile unit gestural control system and using method thereof.

Background technology

Along with the fast development of automotive technology, various new equipment is applied on automobile, such as electronic stability program of automobile (ESP), electronic induction braking control system (SBC), electronic braking force distributes (EBD), auxiliary braking system (BAS), safety air bag (SRS), car radar collision avoidance system, comprise the Vehicle Electronic Control speed change system of electronic automatic transmission (EAT) and toric transmission (CVT), automatically controlled automatic air condition (A/C), CCS cruise control system (CCS), Electronic control suspension (ECS), electronic contro lled power steering system (EPS), central door lock and guard against theft and alarm system, In-vehicle networking technology etc., also have other electronic control packages, as electronic automobile meter, auto-navigation system, car audio system, automobile travel recorder and system for monitoring pressure in tyre etc. all adopt electron steering.

In order to control in numerous equipment de-sign on automobile automotive environment control unit local area network (LAN) CAN, (English is Controller Area Network, conventional abbreviation CAN represents), with exchange message between the electronic control package (ECU) of each mobile unit, define electronic control network of automobile.Such as: in engine management system, gearbox control, instrument equipment, electronics trunk system, all embed CAN.

The electronic control unit of various kinds of equipment, concentrates in operator's compartment mostly, is dispersed with various control button around central operation panel and bearing circle.While automobile possesses and selects more equipment, the operation of chaufeur is just more complicated.When needs high concentration spirit observation road surface ahead is driven, the operation push-button wanted required for searching of diverting attention, also needs during operation to touch with hand or pin to control, causes the very big potential safety hazard of the vehicle of running at high speed undoubtedly.

Manipulate various equipment for the ease of chaufeur, occurred the touch-screen concentrating all kinds of control button, although convenient in use touch-screen domination set, the sight line of chaufeur still inevitably leaves front road conditions.

Occur in recent years much using gesture for this reason and manipulated the scheme of in-vehicle device, as the Chinese invention patent " gesture switch " that the patent No. is ZL200310102916.X, application number is the Chinese invention patent application " a kind of support the virtual console gesture control method of the automobile of blind operation and device " of 201210312798.4, application number be 201220433136.8 Chinese invention patent " utilize Gesture Recognition to control the device of car electrics ", application number is the Chinese invention patent application " a kind of vehicle-mounted Android platform multi-point gesture control method " of 201210197697.7, application number be 201120302339.9 Chinese invention patent " utilize electronic machine and the mobile unit of gesture operation ".These schemes arrange certain camera watch region, with different hand shape or moving direction for identifying gesture, controls corresponding mobile unit.Gesture manipulation can allow chaufeur under the state of vehicle driving posture maintaining its face forward, stretches one's arm, by gesture, operation information is inputed to control object.Shortcoming is the various shape move modes that chaufeur needs the gesture of memory operation distinct device, and gesture is nonstandard there will be maloperation.

Summary of the invention

In order to solve the problem, the invention provides a kind of mobile unit gestural control system, comprise vehicular platform part and gesture identification part, the center processor of vehicular platform part connects projector, the manipulation figure of mobile unit is produced in front windshield projection, 2 cameras of gesture identification part are connected with digital signal processor, and digital signal processor is connected with the center processor of vehicular platform part.

Another object of the present invention is to provide the using method of above-mentioned mobile unit gestural control system, and obtain pointing the position on corresponding manipulation figure and sensing by camera, vehicular platform part sends operation information to corresponding equipment according to this signal.

Mobile unit gestural control system provided by the invention, comprise vehicular platform part, the center processor of vehicular platform part is connected through network-bus with the electronic control unit local area network (CAN) of mobile unit, and the electronic control unit local area network of mobile unit is connected with electronic control units several arbitrarily in the various kinds of equipment such as each Door Lock, each vehicle window skylight, left and right back mirror, radio receiver, DAB broadcasting, Reverse Sensor image, electronic navigator, height of seat, seat heating and air-conditioning.The center processor of vehicular platform part connects projector, shield glass is projecting glass, projector is at the manipulation figure of front windshield projection mobile unit, native system also comprises gesture identification part, gesture identification part comprises the digital signal processor camera identical with 2, the signal incoming digital signal treater that 2 cameras export.The center processor of vehicular platform part is connected with the digital signal processor of gesture identification part through I2C bus (twin wire universal serial bus Inter-IntegratedCircuit) or wireless telecommunications.Described camera is infrared pick-up head, and 2 camera parallel arranged, camera lens are towards identical.The distance of 2 cam lens center-points is 40mm ~ 100mm.

Digital signal processor is connected with the mouth of 2 cameras through Serial Peripheral Interface (SPI) (SPISerial Peripheral Interface).

Described camera refreshing frequency is more than or equal to 40 frames per second.

For strengthening drop shadow effect, shield glass posts reflective membrane.

Described digital signal processor connects infrared multiplier (-icator).Infrared multiplier (-icator) is connected with the infrared emitting lamp of camera, on the finger of the infrared radiation that infrared emitting lamp sends before camera lens, the infrared signal of camera collection finger reflection, digital signal processor is according to the finger infrared signal intensity of camera collection, adjusting pulsewidth modulation (PWM) dutycycle, control infrared multiplier (-icator), thus the infrared ray controlling the transmitting of infrared emitting lamp is strong and weak, controls the power of camera collection signal.

The center processor of vehicular platform part is also connected to screen, the manipulation figure of the generation of simultaneous display projector, is convenient to regulate use.

The gesture identification part of this mobile unit gestural control system is installed in control capsule, control capsule has the camera aperture of 2 cameras and the perforation hole of infrared emitting lamp.Control capsule is placed in by operator's saddle, and the finger of chaufeur manipulates above the cam lens hole of control capsule.Vehicular platform part is installed on below front windshield.

The using method of mobile unit gestural control system of the present invention is as follows:

The manipulation information that the center processor of vehicular platform part obtains each mobile unit from the electronic control unit local area network of mobile unit produces manipulation figure, sends into projector and is projected on front windshield by manipulation figure.

Chaufeur finger moves in the camera lens front of camera, with relevant control region variable color and/or the current option of distortion display successively of the movement of finger, the manipulation figure of shield glass projection, when color change interval is required controlling equipment, points aerial click and determine this option.The position pointed before 2 camera head monitor cameras and motion, digital signal processor receives the signal of 2 cameras, adopt triangulation, the path of motion of finger is gone out according to the spatial coordinates calculation of 2 camera same times, particular location on the manipulation figure corresponding by finger and sensing, determine that chaufeur needs the equipment of manipulation, relevant manipulation signal is sent to the center processor of vehicular platform part, center processor changes running state by the corresponding equipment of this manipulation signal control, or shows from corresponding equipment gathering information at front windshield.

During conventional driving, vehicular platform system acceptance is from the information of each mobile unit of CAN vehicle electronic control unit local area network, and the item description that default display is schemed in manipulation are the control selection key of main driving states information and each mobile unit.

Chaufeur need not be bowed, as long as lift a hand, moveable finger, can select the relevant control region of the manipulation figure of shield glass projection.

The camera of gesture identification part captures finger position and sensing, determine its device that will manipulate or search for information about, and this information is sent to vehicular platform, control to adjust the running state of corresponding equipment, or transfer relevant information and show on front windshield.Such as chaufeur selects the radio receiver on manipulation figure, and center processor can open vehicle-mounted radio, and demonstrates wireless frequency-modulation chart picture, so that chaufeur channel selection; When chaufeur selects oil mass, car internal and external temperature on manipulation figure, manipulation instruction is sent to center processor by signal processor, namely center processor by electronic control unit local area network manipulation relevant device, transfers current oil scale and car internal and external temperature meter information displaying on manipulation figure.

Compared with prior art, the beneficial effect of mobile unit gestural control system of the present invention and using method thereof is: 1, chaufeur is without the need to diverting sb.'s attention viewing, touching the guidance panel of mobile unit, more convenient, the safety of gesture manipulation, neither affect traffic safety, make again chaufeur cosily experience convenience and the enjoyment of various equipment on car; On the manipulation figure of 2, projection, the control area of each equipment is obvious, and driver need not remember various different gesture, brand-new every blank operation interactive mode, manipulates convenient and simple; 3, the control area net(CAN) network of compatible existing automobile, is convenient to Installation and Debugging on various automobile and uses.

Accompanying drawing explanation

Fig. 1 is this mobile unit gestural control system embodiment integral structure schematic diagram;

Fig. 2 is this mobile unit gestural control system embodiment using state schematic diagram.

Detailed description of the invention

Mobile unit gestural control system embodiment

This mobile unit gestural control system embodiment as shown in Figure 1, comprise vehicular platform part and gesture identification part, the center processor of vehicular platform part is connected through network-bus with the electronic control unit local area network (LAN) CAN of mobile unit, and this routine CAN is connected to the electronic control unit of Door Lock, vehicle window, skylight, left and right back mirror, radio receiver, DAB broadcasting, Reverse Sensor image, electronic navigator, height of seat, seat heating and air-conditioning.The center processor of vehicular platform part connects projector and touch-screen, and shield glass is projecting glass, and projector is at the manipulation figure of front windshield projection mobile unit.Gesture identification part comprises the digital signal processor DSP camera A identical with 2 and camera B.This number of cases word signal processor is connected to infrared multiplier (-icator), and digital signal processor DSP adjusting pulsewidth modulation duty cycle controls infrared multiplier (-icator).Camera A and camera B VT are through Serial Peripheral Interface (SPI) incoming digital signal treater.

As shown in Figure 2,2 camera parallel arranged, camera lens upward, are convenient to finger manipulation.The distance of 2 cam lens center-points is 50mm.

The center processor of this routine vehicular platform part connects wireless communication module, and the digital signal processor DSP of gesture identification part connects DSP wireless module, and vehicular platform part is connected through wireless telecommunications with gesture identification part.

This routine camera refreshing frequency is 40 frames per second.

The gesture identification part of this routine mobile unit gestural control system is installed in control capsule, and the control capsule of this example is cuboid box, and on control capsule, coverboard has the camera aperture of 2 cameras and the perforation hole of infrared emitting lamp.Control capsule is placed in below bearing circle, by gear-change hand lever, and the finger of chaufeur manipulates above the cam lens hole of control capsule.This routine vehicular platform part is installed on below front windshield, on engine case.

This routine shield glass posts reflective membrane.

The using method embodiment of mobile unit gestural control system

This example adopts above-mentioned mobile unit gestural control system embodiment.

The manipulation information that the center processor of vehicular platform part obtains each mobile unit from the electronic control unit local area network of mobile unit produces manipulation figure, sends into projector and is projected on front windshield by manipulation figure.As shown in Figure 2, the finger of chaufeur moves in the camera lens front of camera, with the relevant control region variable color amplify the current option of display successively of the movement of finger, the manipulation figure of shield glass projection, when variable color magnification region is required controlling equipment, points aerial click and determine this option.

The position of 2 camera head monitor fingers and motion, digital signal processor receives the signal of 2 cameras, adopt triangulation, the path of motion of finger is gone out according to the spatial coordinates calculation of 2 camera same times, particular location on the manipulation figure corresponding by finger and sensing, determine that chaufeur needs the equipment of manipulation, relevant manipulation signal is sent to the center processor of vehicular platform part, center processor changes running state by the corresponding equipment of this manipulation signal control, or shows from corresponding equipment gathering information at front windshield.

During conventional driving, namely when chaufeur does not send gesture instruction, vehicular platform system acceptance is from the information of each mobile unit of CAN vehicle electronic control unit local area network, and the control selection key that default display items display content is speed per hour, oil mass, car internal and external temperature, navigation information and each mobile unit is schemed in manipulation.

Above-described embodiment, be only the specific case further described object of the present invention, technical scheme and beneficial effect, the present invention is not defined in this.All make within scope of disclosure of the present invention any amendment, equivalent replacement, improvement etc., be all included within protection scope of the present invention.

Claims (10)

1. mobile unit gestural control system, comprise vehicular platform part, the center processor of vehicular platform part is connected through network-bus with the electronic control unit local area network of mobile unit, and the electronic control unit local area network of mobile unit to be connected with in each Door Lock, each vehicle window skylight, left and right back mirror, radio receiver, DAB broadcasting, Reverse Sensor image, electronic navigator, height of seat, seat heating and air-conditioning any one or a few electronic control unit; It is characterized in that:
The center processor of described vehicular platform part connects projector, shield glass is projecting glass, projector is at the manipulation figure of front windshield projection mobile unit, native system also comprises gesture identification part, gesture identification part comprises the digital signal processor camera identical with 2, the vision signal incoming digital signal treater that 2 cameras export; The center processor of vehicular platform part is connected with the digital signal processor of gesture identification part through I2C bus or wireless telecommunications, and described camera is infrared pick-up head, and 2 camera parallel arranged, camera lens are towards identical.
2. mobile unit gestural control system according to claim 1, is characterized in that:
The distance of described 2 cam lens center-points is 40mm ~ 100mm.
3. mobile unit gestural control system according to claim 1, is characterized in that:
Described digital signal processor is connected through the mouth of Serial Peripheral Interface (SPI) with 2 cameras.
4. mobile unit gestural control system according to claim 1, is characterized in that:
Described camera refreshing frequency is more than or equal to 40 frames per second.
5. mobile unit gestural control system according to claim 1, is characterized in that:
Described shield glass posts reflective membrane.
6. mobile unit gestural control system according to claim 1, is characterized in that:
Described digital signal processor connects infrared multiplier (-icator).
7. mobile unit gestural control system according to claim 1, is characterized in that:
The center processor of described vehicular platform part is also connected to screen.
8. mobile unit gestural control system according to claim 6, is characterized in that:
The gesture identification part of described mobile unit gestural control system is installed in control capsule, and control capsule has the camera aperture of 2 cameras and the perforation hole of infrared emitting lamp, control capsule is placed in by operator's saddle; Vehicular platform part is arranged on below front windshield.
9. the using method of mobile unit gestural control system according to any one of claim 1 to 8, is characterized in that:
The manipulation information that the center processor of vehicular platform part obtains each mobile unit from the electronic control unit local area network of mobile unit produces manipulation figure, sends into projector and is projected on front windshield by manipulation figure;
Chaufeur finger moves in camera front, with relevant control region variable color and/or the current option of distortion display successively of the movement of finger, the manipulation figure of shield glass projection, when color change interval is required controlling equipment, points aerial click and determine this option;
The position that 2 camera head monitors are pointed before camera and motion, digital signal processor receives the signal of 2 cameras, adopt triangulation, the path of motion of finger is gone out according to the spatial coordinates calculation of 2 camera same times, particular location on the manipulation figure corresponding by finger and sensing, determine that chaufeur needs the equipment of manipulation, relevant manipulation signal is sent to the center processor of vehicular platform part, center processor changes running state by the corresponding equipment of this manipulation signal control, or show from corresponding equipment gathering information at front windshield.
10. the using method of mobile unit gestural control system according to claim 9, is characterized in that:
During conventional driving, vehicular platform system acceptance is from the information of each mobile unit of CAN vehicle electronic control unit local area network, and the item description that default display is schemed in manipulation are the control selection key of main driving states information and each mobile unit.
CN201310240524.3A 2013-06-18 2013-06-18 Vehicle-mounted equipment gesture control system and usage method thereof CN103303224B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310240524.3A CN103303224B (en) 2013-06-18 2013-06-18 Vehicle-mounted equipment gesture control system and usage method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310240524.3A CN103303224B (en) 2013-06-18 2013-06-18 Vehicle-mounted equipment gesture control system and usage method thereof

Publications (2)

Publication Number Publication Date
CN103303224A CN103303224A (en) 2013-09-18
CN103303224B true CN103303224B (en) 2015-04-15

Family

ID=49129080

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310240524.3A CN103303224B (en) 2013-06-18 2013-06-18 Vehicle-mounted equipment gesture control system and usage method thereof

Country Status (1)

Country Link
CN (1) CN103303224B (en)

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101550604B1 (en) * 2013-09-25 2015-09-07 현대자동차 주식회사 Vehicle operation device
CN103552517A (en) * 2013-11-20 2014-02-05 江南大学 Driver gesture control language
CN103631442A (en) * 2013-12-12 2014-03-12 惠州市华阳多媒体电子有限公司 Touch control system of head-up display device
DE102014202834A1 (en) * 2014-02-17 2015-09-03 Volkswagen Aktiengesellschaft User interface and method for contactless operation of a hardware-designed control element in a 3D gesture mode
CN104866084A (en) 2014-02-25 2015-08-26 中兴通讯股份有限公司 Methods, device and system for gesture recognition
CN103914152B (en) * 2014-04-11 2017-06-09 周光磊 Multi-point touch and the recognition methods and system that catch gesture motion in three dimensions
CN104063929A (en) * 2014-06-11 2014-09-24 江门市联创发展有限公司 Electronic door lock realizing unlocking through mobile terminal
CN104834244A (en) * 2014-11-05 2015-08-12 电子科技大学 Gesture switch and gesture control method
GB2534152B (en) * 2015-01-14 2017-12-20 Jaguar Land Rover Ltd Apparatus and method for controlling operation of a component
CN104731326A (en) * 2015-03-19 2015-06-24 广东好帮手电子科技股份有限公司 Vehicle-mounted rearview mirror navigator gesture recognition system and method thereof
DE102015006613A1 (en) * 2015-05-21 2016-11-24 Audi Ag Operating system and method for operating an operating system for a motor vehicle
CN106289299A (en) * 2015-05-22 2017-01-04 深圳市凯立德科技股份有限公司 A kind of navigator and operational approach thereof
CN105133954B (en) * 2015-06-25 2018-09-18 安恒世通(北京)网络科技有限公司 A kind of intelligent lock
CN106394439B (en) * 2015-07-28 2019-12-20 比亚迪股份有限公司 Control method and system of automobile
CN105049534A (en) * 2015-09-01 2015-11-11 南通希尔顿博世流体设备有限公司 Vehicle-mounted system
CN105049535A (en) * 2015-09-01 2015-11-11 南通希尔顿博世流体设备有限公司 Vehicle-mounted system having function of environment detection
CN105446553A (en) * 2015-12-30 2016-03-30 江西好帮手电子科技有限公司 Noncontact gesture recognition car DVD operation method and system
CN106120242A (en) * 2016-07-29 2016-11-16 无锡飞翎电子有限公司 Washing machine and control device and control method
CN106354259A (en) * 2016-08-30 2017-01-25 同济大学 Automobile HUD gesture-interaction-eye-movement-assisting system and device based on Soli and Tobii
CN106501808A (en) * 2016-09-14 2017-03-15 芜湖扬展新材料科技服务有限公司 A kind of robot communication data handling system based on DSP
CN107483818A (en) * 2017-08-15 2017-12-15 上海蔚来汽车有限公司 Automobile and vehicle-mounted image-taking system, image processing system, correlation technique
CN107719122A (en) * 2017-09-14 2018-02-23 中国第汽车股份有限公司 A kind of new line actuation means and method applied to automobile

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7142197B2 (en) * 2002-10-31 2006-11-28 Microsoft Corporation Universal computing device
JP2012121386A (en) * 2010-12-06 2012-06-28 Fujitsu Ten Ltd On-board system
JP2012243007A (en) * 2011-05-18 2012-12-10 Toshiba Corp Image display device and image area selection method using the same
US8957868B2 (en) * 2011-06-03 2015-02-17 Microsoft Corporation Multi-touch text input
US20130155237A1 (en) * 2011-12-16 2013-06-20 Microsoft Corporation Interacting with a mobile device within a vehicle using gestures
CN202904252U (en) * 2012-08-29 2013-04-24 杨尧任 Device using gesture recognition technology to control automotive electric appliance
CN102897091A (en) * 2012-09-22 2013-01-30 浙江中呼科技有限公司 Vehicle-mounted image display device, mobile vehicle-mounted display device and monitoring device

Also Published As

Publication number Publication date
CN103303224A (en) 2013-09-18

Similar Documents

Publication Publication Date Title
US10177986B2 (en) Universal console chassis for the car
US9956910B2 (en) Audible notification systems and methods for autonomous vehicles
CN106027749B (en) Vehicle, mobile terminal and control method thereof
CN106314152B (en) Vehicle parking assistance device and vehicle with it
CN106143283B (en) Vehicular illumination device and vehicle including it
KR101994698B1 (en) User interface appartus for vehicle and vehicle
EP3095635B1 (en) Display apparatus and method for controlling the same
KR101750876B1 (en) Display apparatus for vehicle and Vehicle
US10527450B2 (en) Apparatus and method transitioning between driving states during navigation for highly automated vechicle
US9666079B2 (en) Systems and methods for driver assistance
US10181266B2 (en) System and method to provide driving assistance
US9442484B2 (en) Apparatus for switching driving modes of vehicle and method of switching between modes of vehicle
EP3070700B1 (en) Systems and methods for prioritized driver alerts
US10565870B2 (en) System and method for driving assistance along a path
US10315566B2 (en) Vehicle control device mounted on vehicle and method for controlling the vehicle
US20170228126A1 (en) Vehicle user interface apparatus and vehicle
KR101741433B1 (en) Driver assistance apparatus and control method for the same
US8979159B2 (en) Configurable hardware unit for car systems
KR101895485B1 (en) Drive assistance appratus and method for controlling the same
US20150352953A1 (en) Vehicle control system with mobile device interface
US20170140651A1 (en) Driver assistance apparatus and vehicle including the same
US10431069B2 (en) Apparatus, method and mobile terminal for providing object loss prevention service in vehicle
JP6773046B2 (en) Driving support device, driving support method, and moving object
EP3184365B1 (en) Display device for vehicle and control method thereof
US20170315556A1 (en) Vehicle control system, vehicle control method, and vehicle control program

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
GR01 Patent grant
C14 Grant of patent or utility model
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20150415

Termination date: 20160618

CF01 Termination of patent right due to non-payment of annual fee