EP3552508B1 - Smart helmet having remote control, and remote control method thereof - Google Patents

Smart helmet having remote control, and remote control method thereof Download PDF

Info

Publication number
EP3552508B1
EP3552508B1 EP17878836.0A EP17878836A EP3552508B1 EP 3552508 B1 EP3552508 B1 EP 3552508B1 EP 17878836 A EP17878836 A EP 17878836A EP 3552508 B1 EP3552508 B1 EP 3552508B1
Authority
EP
European Patent Office
Prior art keywords
module
remote control
microphone
helmet
wireless communication
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP17878836.0A
Other languages
German (de)
French (fr)
Other versions
EP3552508A4 (en
EP3552508A1 (en
Inventor
Bo Zheng
Yongzheng Ye
Zhiming Li
Xiangling Yi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Qianhai Livall IoT Technology Co Ltd
Original Assignee
Shenzhen Qianhai Livall IoT Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Qianhai Livall IoT Technology Co Ltd filed Critical Shenzhen Qianhai Livall IoT Technology Co Ltd
Publication of EP3552508A1 publication Critical patent/EP3552508A1/en
Publication of EP3552508A4 publication Critical patent/EP3552508A4/en
Application granted granted Critical
Publication of EP3552508B1 publication Critical patent/EP3552508B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A42HEADWEAR
    • A42BHATS; HEAD COVERINGS
    • A42B3/00Helmets; Helmet covers ; Other protective head coverings
    • A42B3/04Parts, details or accessories of helmets
    • AHUMAN NECESSITIES
    • A42HEADWEAR
    • A42BHATS; HEAD COVERINGS
    • A42B3/00Helmets; Helmet covers ; Other protective head coverings
    • A42B3/04Parts, details or accessories of helmets
    • A42B3/30Mounting radio sets or communication systems

Definitions

  • the present invention relates to a smart helmet having a remote control and a remote control method thereof.
  • US 2012/0198603 discloses a smart helmet with a remote control.
  • An object of the present invention is to solve a problem that it is not convenient to communicate or control a helmet to perform recording and inter-conversation in sports in the prior art.
  • the object of the present invention is achieved with a following technical solution.
  • a smart helmet having a remote control comprises: a remote control and a helmet, wherein the remote control comprises a button and a first wireless communication module, the first wireless communication module is configured to send to the helmet a connection request, wherein the connection request carries identification information as to whether the remote control carries a microphone, and the first wireless communication module is configured to send a first operation instruction triggered by the button; and the helmet includes a second wireless communication module, a center control module, a first microphone module, and a power supply, the power supply is configured to provide electricity to the second wireless communication module, the center control module, and the first microphone module, the second wireless communication module is configured to communicate with the first wireless communication module in response to the connection request, the center control module is configured to recognize the identification information and store a first recognition result representing that the remote control does not carry a microphone or a second recognition result representing that the remote control carries a microphone, recognize the first operation instruction according to the first recognition result or the second recognition result, and generate a second operation instruction configured to activate the first microphone module to perform recording, or
  • the remote control further has a second microphone module, and the second microphone module is configured to perform recording in response to the third operation instruction.
  • the helmet further comprises a Bluetooth module and a Bluetooth speaker
  • the Bluetooth module is configured to receive the microphone data of the first microphone module or of the remote control and send the microphone data to a mobile terminal connected with the helmet, and receive a recording of other communication party
  • the Bluetooth speaker is configured to broadcast the recording of the other communication party.
  • the helmet further comprises an audio amplification module which is connected with the Bluetooth module and the Bluetooth speaker, and is configured to perform power amplification for the voice of the other party, and the Bluetooth speaker is a Bluetooth trumpet.
  • the helmet further comprises a judgment module and at least one execution module, wherein the judgment module is configured to judge whether a corresponding operation needs a microphone according to the first operation instruction, and generate a first result representing that "no microphone is needed” or a second result representing that "microphone is needed", and return the first result or the second result to the center control module, the center control module is further configured to generate, in response to the first result, a fourth operation instruction sent to the corresponding execution module, or recognize, in response to the second result, the first operation instruction according to the first recognition result or the second recognition result.
  • a wireless communication protocol used by the first wireless communication module and the second wireless communication module comprising: a Bluetooth communication protocol, an ANT+ communication protocol, an infrared communication protocol, or a 433 communication protocol.
  • the remote control is installed on a ski pole or on a handlebar of a bicycle.
  • a remote control method for controlling to the smart helmet having a remote control including: sending a connection request to a second wireless communication module by a first wireless communication module, wherein the connection request carries identification information as to whether a remote control carries a microphone; establishing a connection with the first wireless communication module by the second wireless communication module in response to the connection request, and recognizing, by a center control module, the identification information and storing, by the center control module, a first recognition result representing that the remote control does not carry a microphone or a second recognition result representing that the remote control carries a microphone; generating a first operation instruction by the remote control in response to a button triggering action, and sending the first operation instruction to the second wireless communication module via the first wireless communication module; and recognizing the first operation instruction by the center control module according to the first recognition result or the second recognition result, and generating a second operation instruction for activating a first microphone module for recording, or generating a third operation instruction sent to the remote control so as to receive microphone data of the remote control.
  • the method further includes: receiving the microphone data of the first microphone module or of the remote control by the Bluetooth module and sending the microphone data to a mobile terminal connected with the helmet, and sending the microphone data to a mobile terminal and a helmet of other communication party by the mobile terminal; receiving a recording of the other communication party by the Bluetooth module; and broadcasting the recording of the other communication party by the Bluetooth speaker.
  • the method further includes, before the recognizing the first operation instruction by the center control module according to the first recognition result or the second recognition result, the method further includes: judging, by the judgment module, whether a corresponding operation needs a microphone according to the first operation instruction, and generating a first result representing that "no microphone is needed” or a second result representing that "microphone is needed", and returning the first result or the second result to the center control module; and generating, by the center control module, a fourth operation instruction sent to a corresponding execution module in response to the first result, or implementing, in response to the second result, a step of recognizing the first operation instruction according to the first recognition result or the second recognition result.
  • the smart helmet having a remote control and the remote control method thereof provided in the present invention control the recording through the remote control, and are suitable for sport events such as skiing and the like where it is unadvisable to take out a mobile phone for communication, without increasing the equipment burden or affecting the progress of sports, moreover, the remote control carrying a microphone or not can cooperate with the smart helmet to realize recording, with strong compatibility.
  • a smart helmet 100 not forming part of the present invention includes a remote control 20 and a helmet 30.
  • the remote control 20 includes a button 21 and a first wireless communication module 24.
  • the first wireless communication module 24 is configured to send a connection request and a first operation instruction triggered by the button 21 to the helmet 30, wherein the connection request carries identification information as to whether the remote control 20 carries a microphone.
  • the remote control 20 is disposed separately from the helmet 30.
  • the remote control 20 is installed on sports equipment, for example, installed on a ski pole or on a handlebar of a bicycle, located at a position where a finger can easily touch, and when a user needs to perform inter-conversation with a companion, the user can conveniently press down the button of the remote control to send a recording control signal.
  • the helmet 30 includes a second wireless communication module 31, a center control module 32, a first microphone module 34, and a power supply 39.
  • the power supply 39 is configured to provide electricity to the second wireless communication module 31, the center control module 32, and the first microphone module 34.
  • the first microphone module 34 is located at any position where a user voice can be recorded.
  • the first microphone module 34 is mounted on the helmet 30, for example, the first microphone module 34 can be mounted at a front end of the helmet 30, closer to a user voice source.
  • the second wireless communication module 31 is configured to communicate with the first wireless communication module 24, wherein a wireless communication protocol used includes: a Bluetooth communication protocol, an ANT+ communication protocol, an infrared communication protocol, and a 433 communication protocol.
  • a wireless communication protocol used includes: a Bluetooth communication protocol, an ANT+ communication protocol, an infrared communication protocol, and a 433 communication protocol.
  • the ANT+ communication protocol is used.
  • the second wireless communication module 31 responds to the connection request from the first wireless communication module 24 and connects, and sends the connection request or the identification information in the connection request to the center control module 32.
  • the center control module 32 is configured to recognize the identification information and store a first recognition result representing that the remote control 20 carries a microphone or a second recognition result representing that the remote control 20 does not carry a microphone, thereafter, recognize a first operation instruction according to the first recognition result or the second recognition result, and generate a second operation instruction sent to the remote control 20 so as to receive microphone data of the remote control, or generate a third operation instruction for activating the first microphone module 34 for recording.
  • the remote control 20 does not carry a microphone.
  • the connection request sent by the first wireless communication module 24 is sent to the second wireless communication module 31, then a connection is established between the helmet 30 and the remote control 20, and the center control module 32 recognizes the connection request and stores the first recognition result representing that the remote control does not carry a microphone.
  • the first operation instruction triggered by the button 21 includes a signal for activating the microphone, and the center control module 32 recognizes the first operation instruction according to the first recognition result, and generates the second operation instruction for activating the first microphone module 34 to perform recording.
  • the first microphone module 34 responds to the second operation instruction so as to record the user voice. Thereafter, the first microphone module 34 sends the microphone data to the Bluetooth module.
  • the helmet is controlled by the remote control.
  • a smart helmet 200 having a remote control not forming part of the present invention includes the remote control 20 and the helmet 30, wherein the remote control 20 includes the button 21 and the first wireless communication module 24.
  • the helmet 30 includes the second wireless communication module 31, the center control module 32, the first microphone module 34, and the power supply 39.
  • a difference lies in that the remote control 20 further has a second microphone module 22.
  • the button 21 is connected with the second microphone module 22, and the second microphone module 22 is connected with the first wireless communication module 24.
  • a recording signal triggered by the button 21 is sent to the second microphone module 22, and the second microphone module 22 records the user voice, including steps such as collecting, recording, compressing, and encoding.
  • a compressed voice can be transmitted at a relatively higher speed with suitable quality.
  • the center control module 32 recognizes that the remote control 20 has a microphone, thus stores a second recognition result representing that the remote control carries a microphone, then recognizes a first operation instruction according to the second recognition result, wherein the first operation instruction includes a signal of being ready to send recording data, thereby the center control module 32 generates a third operation instruction sent to the remote control 20 so as to receive the microphone data of the remote control, and after the third operation instruction is sent, the remote control 20 sends the microphone data to the second wireless communication module 31, wherein the microphone data can be firstly stored locally (in the remote control) and then sent, and also can be sent while recording, thus, the microphone data of the remote control 20 is sent to the helmet 30.
  • the manner of firstly storing and then sending is used, which can reduce data loss.
  • the second wireless communication module 31 sends the microphone data received to the Bluetooth module.
  • the helmet for recording, getting ready for performing inter-conversation, the helmet is controlled by the remote control, moreover, no matter the remote control carries the microphone or not, the smart helmet can receive the microphone data.
  • the helmet 30 comprises the second wireless communication module 31, the center control module 32, the first microphone module 34, and the power supply 39.
  • the helmet 30 further includes a Bluetooth module 36 and a Bluetooth speaker 38.
  • the mobile phone In sport events where it is inconvenient to use a mobile phone, the mobile phone only can be carried with oneself, for example, put in a pocket.
  • the mobile phone When a Bluetooth function of the mobile phone is turned on, the mobile phone can establish a connection relationship with the helmet 30 also having a Bluetooth function, and the Bluetooth module 36 sends microphone data received thereby to a mobile terminal 41 of a user via Bluetooth.
  • the mobile terminal 41 sends the microphone data to a server via a mobile network, wherein the server can be provided by a service provider.
  • the server sends the voice to a target mobile terminal 42 of a communication object, after receiving voice information, the target mobile terminal 42 decodes the voice information, and then the voice information is directly broadcast on the target mobile terminal 42, or preferably, considering that it is also inconvenient for the communication object who is doing the same or similar sports to answer the voice with a mobile phone, the voice information continues to be sent to a Bluetooth speaker channel of the target mobile terminal 42, and then is sent to the Bluetooth speaker 38 of a helmet 44 of the communication object via Bluetooth to be broadcast.
  • the Bluetooth speaker 38 of the helmet 30 or the helmet 44 can be a Bluetooth headphone, i.e. in-ear headphone, and preferably, the Bluetooth speaker 38 is a Bluetooth speaker built in an outer casing of the helmet 30, close to ear, thus both voice of the other side in the inter-conversation and external sounds can be heard, thus being safer for use.
  • the Bluetooth speaker 38 is a Bluetooth speaker, in order drive the Bluetooth speaker, the helmet 30 further comprises an audio amplification module 37 configured to perform power amplification for the voice.
  • the helmet is controlled by the remote control for inter-conversation
  • the smart helmet is suitable for sport events such as skiing and riding where it is unadvisable to take out a mobile phone for communication, and facilitates the user in performing inter-conversation with companions, without increasing the equipment burden or affecting the progress of sports.
  • the remote control can be used to perform recording, for example, in skiing, inter-conversation can be performed by lifting up a ski pole without affecting the skiing, or in riding, inter-conversation can be performed by speaking towards a handlebar equipped with the remote control; when there is no microphone on the remote control, the microphone module carried by the helmet itself is activated, therefore, the remote control carrying a microphone or not can cooperate with the smart helmet to realize recording, with strong compatibility.
  • a smart helmet 400 having a remote control comprises a helmet 50 and a remote control 60, wherein the helmet 50 comprises a second wireless communication module 531, a center control module 532, a first microphone module 534, and a power supply 539.
  • the remote control 60 comprises a button 621 and a first wireless communication module 624, and possibly has a second microphone module 622.
  • a Bluetooth module and a Bluetooth speaker may be further included.
  • the helmet 50 thereof further includes a judgment module 502 and at least one execution module 506, wherein the judgment module 502 is connected with the center control module 532.
  • the remote control 60 has a plurality buttons or has one button with different pressing manners representing different functions, for example, representing photographing, camera shooting, and inter-conversation, correspondingly, apart from the first microphone module 534, the helmet 50 further comprises at least one execution module corresponding to photographing and camera shooting. In the present embodiment, there is correspondingly one execution module 506.
  • the judgment module 502 judges whether an operation corresponding to the first operation instruction needs a microphone. If the corresponding operation is photographing or camera shooting, a first result representing that "no microphone is needed" is generated, then the first result is returned to the center control module 532, and the center control module 532 generates, in response to the first result, a fourth operation instruction sent to the corresponding execution module 506, to order the execution module 506 to perform photographing or camera shooting; if the corresponding operation is inter-conversation, a second result representing that "microphone is needed” is generated, then the second result is returned to the center control module 504, and the center control module 504 implements, in response to the second result, a step of recognizing the first operation instruction according to a first recognition result or a second recognition result, then a second operation instruction or a third operation instruction is executed according to whether the remote control 60 has a microphone.
  • the first recognition result or the second recognition result is recognized by the center control module 504 and stored locally in advance.
  • the smart helmet provided in the present embodiment further can process more remote control tasks, with stronger processing capabilities, and convenient use.
  • a remote control method includes following steps:
  • the remote control method provided in the present example can enable the smart helmet to conveniently control recording. Further, the following steps may be further included:
  • the Bluetooth module receives the microphone data of the first microphone module or of the remote control and sends the microphone data to a mobile terminal connected with the helmet, and the microphone data is sent to a mobile terminal and a helmet of other communication party by the mobile terminal; the Bluetooth module receives a recording of the other communication party; and the Bluetooth speaker broadcasts the recording of the other communication party.
  • the Bluetooth module After receiving the microphone data, the Bluetooth module sends the microphone data to the mobile terminal via Bluetooth, the mobile terminal sends the microphone data to a mobile terminal of a communication object via a mobile network or the like, then the microphone data is decoded by a mobile terminal of the communication object, and sent to the Bluetooth speaker of the helmet via a Bluetooth headphone channel of the mobile terminal to be broadcast, so as to achieve inter-conversation.
  • the remote control method provided in the present embodiment further can include following steps: the judgment module judges whether a corresponding operation needs a microphone according to the first operation instruction, and generates a first result representing that "no microphone is needed” or a second result representing that "microphone is needed", and returns the first result or the second result to the center control module; and the center control module generates a fourth operation instruction sent to a corresponding execution module in response to the first result, or implements, in response to the second result, a step of recognizing the first operation instruction according to the first recognition result or the second recognition result.
  • the smart helmet further can judge whether an operation thereof needs a microphone, wherein if no microphone is needed, a corresponding operation is directly executed, and if needed, it is recognized whether the remote control has a microphone thereon.
  • the remote control method provided in the present embodiment is suitable for sport events such as skiing and riding where it is unadvisable to take out a mobile phone for communication, and facilitates the user in performing inter-conversation with companions, without increasing the burden of equipment;
  • a second microphone module is installed on the remote control, it is convenient for the user to use the remote control to perform recording, for example, in skiing, inter-conversation can be performed by lifting up a ski pole without affecting the skiing, or in riding, inter-conversation can be performed by speaking towards a handlebar equipped with the remote control.
  • the helmet detects that there is no microphone on the remote control the microphone module carried by the helmet itself is activated, therefore, the remote control carrying a microphone or not or a multi-functional remote control can cooperate with the smart helmet to realize recording, with strong compatibility.

Landscapes

  • Helmets And Other Head Coverings (AREA)

Description

    Technical Field
  • The present invention relates to a smart helmet having a remote control and a remote control method thereof.
  • Background
  • When doing outdoor sports in cold environments, for example, skiing and climbing Snow Mountains, etc., communication capacity of mobile phone is reduced due to a too low environment temperature, thus it is necessary to provide a device suitable for communication during outdoor sports. Users who do outdoor sports often wear helmets, therefore, it is expected to achieve a communication function by setting up a speaker and a microphone on the helmet, but the helmet is worn on the head, then it is quite inconvenient to operate the helmet with a hand in sports for performing inter-conversation, moreover, it is extremely easy to cause riding accidents.
  • US 2012/0198603 discloses a smart helmet with a remote control.
  • Summary
  • An object of the present invention is to solve a problem that it is not convenient to communicate or control a helmet to perform recording and inter-conversation in sports in the prior art.
  • The object of the present invention is achieved with a following technical solution.
  • A smart helmet having a remote control comprises: a remote control and a helmet, wherein the remote control comprises a button and a first wireless communication module, the first wireless communication module is configured to send to the helmet a connection request, wherein the connection request carries identification information as to whether the remote control carries a microphone, and the first wireless communication module is configured to send a first operation instruction triggered by the button; and the helmet includes a second wireless communication module, a center control module, a first microphone module, and a power supply, the power supply is configured to provide electricity to the second wireless communication module, the center control module, and the first microphone module, the second wireless communication module is configured to communicate with the first wireless communication module in response to the connection request, the center control module is configured to recognize the identification information and store a first recognition result representing that the remote control does not carry a microphone or a second recognition result representing that the remote control carries a microphone, recognize the first operation instruction according to the first recognition result or the second recognition result, and generate a second operation instruction configured to activate the first microphone module to perform recording, or generate a third operation instruction sent to the remote control so as to receive microphone data of the remote control.
  • In one embodiment, the remote control further has a second microphone module, and the second microphone module is configured to perform recording in response to the third operation instruction.
  • In an embodiment, the helmet further comprises a Bluetooth module and a Bluetooth speaker, the Bluetooth module is configured to receive the microphone data of the first microphone module or of the remote control and send the microphone data to a mobile terminal connected with the helmet, and receive a recording of other communication party, and the Bluetooth speaker is configured to broadcast the recording of the other communication party.
  • In an embodiment, the helmet further comprises an audio amplification module which is connected with the Bluetooth module and the Bluetooth speaker, and is configured to perform power amplification for the voice of the other party, and the Bluetooth speaker is a Bluetooth trumpet.
  • The helmet further comprises a judgment module and at least one execution module, wherein the judgment module is configured to judge whether a corresponding operation needs a microphone according to the first operation instruction, and generate a first result representing that "no microphone is needed" or a second result representing that "microphone is needed", and return the first result or the second result to the center control module, the center control module is further configured to generate, in response to the first result, a fourth operation instruction sent to the corresponding execution module, or recognize, in response to the second result, the first operation instruction according to the first recognition result or the second recognition result.
  • In an embodiment, a wireless communication protocol used by the first wireless communication module and the second wireless communication module comprising: a Bluetooth communication protocol, an ANT+ communication protocol, an infrared communication protocol, or a 433 communication protocol.
  • In an embodiment, the remote control is installed on a ski pole or on a handlebar of a bicycle.
  • A remote control method for controlling to the smart helmet having a remote control including: sending a connection request to a second wireless communication module by a first wireless communication module, wherein the connection request carries identification information as to whether a remote control carries a microphone; establishing a connection with the first wireless communication module by the second wireless communication module in response to the connection request, and recognizing, by a center control module, the identification information and storing, by the center control module, a first recognition result representing that the remote control does not carry a microphone or a second recognition result representing that the remote control carries a microphone; generating a first operation instruction by the remote control in response to a button triggering action, and sending the first operation instruction to the second wireless communication module via the first wireless communication module; and recognizing the first operation instruction by the center control module according to the first recognition result or the second recognition result, and generating a second operation instruction for activating a first microphone module for recording, or generating a third operation instruction sent to the remote control so as to receive microphone data of the remote control.
  • In an embodiment, the method further includes: receiving the microphone data of the first microphone module or of the remote control by the Bluetooth module and sending the microphone data to a mobile terminal connected with the helmet, and sending the microphone data to a mobile terminal and a helmet of other communication party by the mobile terminal; receiving a recording of the other communication party by the Bluetooth module; and broadcasting the recording of the other communication party by the Bluetooth speaker.
  • The method further includes, before the recognizing the first operation instruction by the center control module according to the first recognition result or the second recognition result, the method further includes: judging, by the judgment module, whether a corresponding operation needs a microphone according to the first operation instruction, and generating a first result representing that "no microphone is needed" or a second result representing that "microphone is needed", and returning the first result or the second result to the center control module; and generating, by the center control module, a fourth operation instruction sent to a corresponding execution module in response to the first result, or implementing, in response to the second result, a step of recognizing the first operation instruction according to the first recognition result or the second recognition result.
  • Compared with the prior art, the smart helmet having a remote control and the remote control method thereof provided in the present invention control the recording through the remote control, and are suitable for sport events such as skiing and the like where it is unadvisable to take out a mobile phone for communication, without increasing the equipment burden or affecting the progress of sports, moreover, the remote control carrying a microphone or not can cooperate with the smart helmet to realize recording, with strong compatibility.
  • The above description is only an overview of the technical solutions of the present invention, and in order to be capable of understanding the technical means of the present invention for implementation according to the contents in the specification, and in order to be capable of making the above and other objects, features, and advantages of the present invention more apparent and understandable, preferred examples are particularly illustrated below in cooperation with drawings to make detailed description below.
  • Brief Description of the Drawings
    • FIG. 1 is a structural schematic diagram of a smart helmet having a remote control not forming part of the present invention;
    • FIG. 2 is a structural schematic diagram of the smart helmet having a remote control not forming part of the present invention;
    • FIG. 3 is a structural schematic diagram of the smart helmet having a remote control not forming part of the present invention;
    • FIG. 4 is a schematic diagram of interaction between the smart helmet provided in the previous example and a mobile terminal;
    • FIG. 5 is a structural schematic diagram of the smart helmet having a remote control according to the present invention;
    • FIG. 6 is a flow diagram of a remote control method; and
    • FIG. 7 is a flow diagram of the remote control method in an implementation of the present invention.
    Detailed Description of the Embodiments
  • The technical solutions in embodiments of the present invention will be described clearly and completely below in combination with drawings in the examples of the present invention. Apparently, embodiments described are merely some of the examples of the present invention, rather than all of embodiments.
  • Please referring to FIG. 1, a smart helmet 100 not forming part of the present invention includes a remote control 20 and a helmet 30.
  • The remote control 20 includes a button 21 and a first wireless communication module 24. The first wireless communication module 24 is configured to send a connection request and a first operation instruction triggered by the button 21 to the helmet 30, wherein the connection request carries identification information as to whether the remote control 20 carries a microphone.
  • The remote control 20 is disposed separately from the helmet 30. Preferably, the remote control 20 is installed on sports equipment, for example, installed on a ski pole or on a handlebar of a bicycle, located at a position where a finger can easily touch, and when a user needs to perform inter-conversation with a companion, the user can conveniently press down the button of the remote control to send a recording control signal. The helmet 30 includes a second wireless communication module 31, a center control module 32, a first microphone module 34, and a power supply 39. The power supply 39 is configured to provide electricity to the second wireless communication module 31, the center control module 32, and the first microphone module 34. The first microphone module 34 is located at any position where a user voice can be recorded. In the present embodiment, the first microphone module 34 is mounted on the helmet 30, for example, the first microphone module 34 can be mounted at a front end of the helmet 30, closer to a user voice source.
  • The second wireless communication module 31 is configured to communicate with the first wireless communication module 24, wherein a wireless communication protocol used includes: a Bluetooth communication protocol, an ANT+ communication protocol, an infrared communication protocol, and a 433 communication protocol. Preferably, the ANT+ communication protocol is used.
  • The second wireless communication module 31 responds to the connection request from the first wireless communication module 24 and connects, and sends the connection request or the identification information in the connection request to the center control module 32. The center control module 32 is configured to recognize the identification information and store a first recognition result representing that the remote control 20 carries a microphone or a second recognition result representing that the remote control 20 does not carry a microphone, thereafter, recognize a first operation instruction according to the first recognition result or the second recognition result, and generate a second operation instruction sent to the remote control 20 so as to receive microphone data of the remote control, or generate a third operation instruction for activating the first microphone module 34 for recording.
  • In the present example, the remote control 20 does not carry a microphone. The connection request sent by the first wireless communication module 24 is sent to the second wireless communication module 31, then a connection is established between the helmet 30 and the remote control 20, and the center control module 32 recognizes the connection request and stores the first recognition result representing that the remote control does not carry a microphone. The first operation instruction triggered by the button 21 includes a signal for activating the microphone, and the center control module 32 recognizes the first operation instruction according to the first recognition result, and generates the second operation instruction for activating the first microphone module 34 to perform recording. The first microphone module 34 responds to the second operation instruction so as to record the user voice. Thereafter, the first microphone module 34 sends the microphone data to the Bluetooth module.
  • To sum up, for the smart helmet provided in the present example, for recording, getting ready for performing inter-conversation, the helmet is controlled by the remote control.
  • Please referring to FIG. 2, substantially the same as the smart helmet 100, a smart helmet 200 having a remote control not forming part of the present invention includes the remote control 20 and the helmet 30, wherein the remote control 20 includes the button 21 and the first wireless communication module 24. The helmet 30 includes the second wireless communication module 31, the center control module 32, the first microphone module 34, and the power supply 39. A difference lies in that the remote control 20 further has a second microphone module 22. The button 21 is connected with the second microphone module 22, and the second microphone module 22 is connected with the first wireless communication module 24.
  • A recording signal triggered by the button 21 is sent to the second microphone module 22, and the second microphone module 22 records the user voice, including steps such as collecting, recording, compressing, and encoding. A compressed voice can be transmitted at a relatively higher speed with suitable quality.
  • According to connection information sent by the remote control 20, the center control module 32 recognizes that the remote control 20 has a microphone, thus stores a second recognition result representing that the remote control carries a microphone, then recognizes a first operation instruction according to the second recognition result, wherein the first operation instruction includes a signal of being ready to send recording data, thereby the center control module 32 generates a third operation instruction sent to the remote control 20 so as to receive the microphone data of the remote control, and after the third operation instruction is sent, the remote control 20 sends the microphone data to the second wireless communication module 31, wherein the microphone data can be firstly stored locally (in the remote control) and then sent, and also can be sent while recording, thus, the microphone data of the remote control 20 is sent to the helmet 30. Preferably, the manner of firstly storing and then sending is used, which can reduce data loss.
  • Thereafter, the second wireless communication module 31 sends the microphone data received to the Bluetooth module.
  • To sum up, for the smart helmet provided in the present example, for recording, getting ready for performing inter-conversation, the helmet is controlled by the remote control, moreover, no matter the remote control carries the microphone or not, the smart helmet can receive the microphone data.
  • Please referring to FIG. 3 and FIG. 4, substantially the same as the smart helmet 100 or 200, a smart helmet 300 not forming part of the present invention comprises the remote control 20 and the helmet 30, wherein the remote control 20 includes the button 21 and the first wireless communication module 24, and possibly has the second microphone module 22. The helmet 30 comprises the second wireless communication module 31, the center control module 32, the first microphone module 34, and the power supply 39. A difference lies in that the helmet 30 further includes a Bluetooth module 36 and a Bluetooth speaker 38.
  • In sport events where it is inconvenient to use a mobile phone, the mobile phone only can be carried with oneself, for example, put in a pocket. When a Bluetooth function of the mobile phone is turned on, the mobile phone can establish a connection relationship with the helmet 30 also having a Bluetooth function, and the Bluetooth module 36 sends microphone data received thereby to a mobile terminal 41 of a user via Bluetooth. The mobile terminal 41 sends the microphone data to a server via a mobile network, wherein the server can be provided by a service provider. The server sends the voice to a target mobile terminal 42 of a communication object, after receiving voice information, the target mobile terminal 42 decodes the voice information, and then the voice information is directly broadcast on the target mobile terminal 42, or preferably, considering that it is also inconvenient for the communication object who is doing the same or similar sports to answer the voice with a mobile phone, the voice information continues to be sent to a Bluetooth speaker channel of the target mobile terminal 42, and then is sent to the Bluetooth speaker 38 of a helmet 44 of the communication object via Bluetooth to be broadcast.
  • The Bluetooth speaker 38 of the helmet 30 or the helmet 44 can be a Bluetooth headphone, i.e. in-ear headphone, and preferably, the Bluetooth speaker 38 is a Bluetooth speaker built in an outer casing of the helmet 30, close to ear, thus both voice of the other side in the inter-conversation and external sounds can be heard, thus being safer for use. When the Bluetooth speaker 38 is a Bluetooth speaker, in order drive the Bluetooth speaker, the helmet 30 further comprises an audio amplification module 37 configured to perform power amplification for the voice.
  • To sum up, for the smart helmet provided in the present example, the helmet is controlled by the remote control for inter-conversation, the smart helmet is suitable for sport events such as skiing and riding where it is unadvisable to take out a mobile phone for communication, and facilitates the user in performing inter-conversation with companions, without increasing the equipment burden or affecting the progress of sports. When there is a microphone on the remote control, the remote control can be used to perform recording, for example, in skiing, inter-conversation can be performed by lifting up a ski pole without affecting the skiing, or in riding, inter-conversation can be performed by speaking towards a handlebar equipped with the remote control; when there is no microphone on the remote control, the microphone module carried by the helmet itself is activated, therefore, the remote control carrying a microphone or not can cooperate with the smart helmet to realize recording, with strong compatibility.
  • Please referring to FIG. 5, substantially the same as the smart helmet 100 or the smart helmet 200, a smart helmet 400 having a remote control according to the present invention comprises a helmet 50 and a remote control 60, wherein the helmet 50 comprises a second wireless communication module 531, a center control module 532, a first microphone module 534, and a power supply 539. The remote control 60 comprises a button 621 and a first wireless communication module 624, and possibly has a second microphone module 622. Of course, a Bluetooth module and a Bluetooth speaker may be further included. A difference lies in that the helmet 50 thereof further includes a judgment module 502 and at least one execution module 506, wherein the judgment module 502 is connected with the center control module 532.
  • The remote control 60 has a plurality buttons or has one button with different pressing manners representing different functions, for example, representing photographing, camera shooting, and inter-conversation, correspondingly, apart from the first microphone module 534, the helmet 50 further comprises at least one execution module corresponding to photographing and camera shooting. In the present embodiment, there is correspondingly one execution module 506.
  • The judgment module 502 judges whether an operation corresponding to the first operation instruction needs a microphone. If the corresponding operation is photographing or camera shooting, a first result representing that "no microphone is needed" is generated, then the first result is returned to the center control module 532, and the center control module 532 generates, in response to the first result, a fourth operation instruction sent to the corresponding execution module 506, to order the execution module 506 to perform photographing or camera shooting; if the corresponding operation is inter-conversation, a second result representing that "microphone is needed" is generated, then the second result is returned to the center control module 504, and the center control module 504 implements, in response to the second result, a step of recognizing the first operation instruction according to a first recognition result or a second recognition result, then a second operation instruction or a third operation instruction is executed according to whether the remote control 60 has a microphone. The first recognition result or the second recognition result is recognized by the center control module 504 and stored locally in advance.
  • To sum up, apart from inter-conversation, the smart helmet provided in the present embodiment further can process more remote control tasks, with stronger processing capabilities, and convenient use.
  • Please referring to FIG. 6, a remote control method includes following steps:
    • Step S1, a first wireless communication module sends a connection request to a second wireless communication module, wherein the connection request carries identification information as to whether a remote control carries a microphone;
    • Step S2, the second wireless communication module establishes connection with the first wireless communication module in response to the connection request, and a center control module recognizes the identification information and stores a first recognition result representing that the remote control does not carry a microphone or a second recognition result representing that the remote control carries a microphone;
    • Step S3, the remote control generates a first operation instruction in response to a button triggering action, and sends the first operation instruction to the second wireless communication module via the first wireless communication module; and
    • Step S4, the center control module recognizes the first operation instruction according to the first recognition result or the second recognition result, and generates a second operation instruction for activating a first microphone module for recording, or generates a third operation instruction sent to the remote control so as to receive microphone data of the remote control.
  • The remote control method provided in the present example can enable the smart helmet to conveniently control recording. Further, the following steps may be further included:
    The Bluetooth module receives the microphone data of the first microphone module or of the remote control and sends the microphone data to a mobile terminal connected with the helmet, and the microphone data is sent to a mobile terminal and a helmet of other communication party by the mobile terminal; the Bluetooth module receives a recording of the other communication party; and the Bluetooth speaker broadcasts the recording of the other communication party.
  • After receiving the microphone data, the Bluetooth module sends the microphone data to the mobile terminal via Bluetooth, the mobile terminal sends the microphone data to a mobile terminal of a communication object via a mobile network or the like, then the microphone data is decoded by a mobile terminal of the communication object, and sent to the Bluetooth speaker of the helmet via a Bluetooth headphone channel of the mobile terminal to be broadcast, so as to achieve inter-conversation.
  • Further, please referring to FIG. 7, the remote control method provided in the present embodiment further can include following steps: the judgment module judges whether a corresponding operation needs a microphone according to the first operation instruction, and generates a first result representing that "no microphone is needed" or a second result representing that "microphone is needed", and returns the first result or the second result to the center control module; and the center control module generates a fourth operation instruction sent to a corresponding execution module in response to the first result, or implements, in response to the second result, a step of recognizing the first operation instruction according to the first recognition result or the second recognition result.
  • For the remote control having a plurality of remote control functions, the smart helmet further can judge whether an operation thereof needs a microphone, wherein if no microphone is needed, a corresponding operation is directly executed, and if needed, it is recognized whether the remote control has a microphone thereon.
  • To sum up, the remote control method provided in the present embodiment is suitable for sport events such as skiing and riding where it is unadvisable to take out a mobile phone for communication, and facilitates the user in performing inter-conversation with companions, without increasing the burden of equipment; when a second microphone module is installed on the remote control, it is convenient for the user to use the remote control to perform recording, for example, in skiing, inter-conversation can be performed by lifting up a ski pole without affecting the skiing, or in riding, inter-conversation can be performed by speaking towards a handlebar equipped with the remote control. When the helmet detects that there is no microphone on the remote control, the microphone module carried by the helmet itself is activated, therefore, the remote control carrying a microphone or not or a multi-functional remote control can cooperate with the smart helmet to realize recording, with strong compatibility.
  • The above examples merely express several embodiments of the present invention, and the description thereof is relatively concrete and detailed, but it should not be thereby construed as limitation on the scope of the present patent for invention. It should be indicated that for a person ordinarily skilled in the art, several alterations and improvements can be further made without departing from the scope of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present patent for invention should be based on the attached claims.

Claims (8)

  1. A remote control method for controlling the smart helmet (400) having a remote control (60), characterized in that the method including:
    sending a connection request to a second wireless communication module (531) by a first wireless communication module (624), wherein the connection request carries identification information as to whether a remote control (60) carries a microphone;
    establishing a connection with the first wireless communication module (624) by the second wireless communication module (531) in response to the connection request, and
    recognizing, by a center control module (532), the identification information and storing a first recognition result representing that the remote control (60) does not carry a microphone or a second recognition result representing that the remote control (60) carries a microphone (622);
    generating a first operation instruction by the remote control (60) in response to a button triggering action, and sending the first operation instruction to the second wireless communication module (531) via the first wireless communication module (624); and
    recognizing the first operation instruction by the center control module (532) according to the first recognition result or the second recognition result, and generating a second operation instruction for activating a first microphone module (534) for
    recording, or generating a third operation instruction sent to the remote control (60) so as to receive microphone data of the remote control (60), before the recognizing the first operation instruction by the center control module (532)
    according to the first recognition result or the second recognition result, the method further including: judging, by the judgment module (502), whether a
    corresponding operation needs a microphone according to the first operation instruction, and generating a first result representing that "no microphone is needed" or a second result representing that "microphone is needed", and returning the first result or the second result to the center control module (532); and generating, by the center control module (532), a fourth operation instruction sent to a corresponding execution module (506) in response to the first result, or implementing, in response to the second result, a step of recognizing the first operation instruction according to the first recognition result or the second recognition result.
  2. The remote control method according to claim 1, characterized in that the method further including:
    receiving the microphone data of the first microphone module (534) or of the remote
    control by the Bluetooth module and sending the microphone data to a mobile terminal connected with the helmet, and sending the microphone data to a mobile terminal and a helmet of other communication party by the mobile terminal;
    receiving a recording of the other communication party by the Bluetooth module; and broadcasting the recording of the other communication party by the Bluetooth speaker.
  3. A smart helmet (400) having a remote control (60), comprising: a remote control (60) and a helmet (50), wherein the remote control (60) comprising a button (621) and a first wireless communication module (624) and the helmet (50) comprising a second wireless communication module (531), a center control module (532), a first microphone module (534), a judgment module (502), at least one execution module (506) and a power supply (539) the power supply (539) being configured to provide electricity to the second wireless communication module (531), the center control module (532), the first microphone module (534), the judgment module (502) and the at least one execution module (506) characterized in that the smart helmet (400) is configured to perform the remote control method according to one of claims 1 to 2.
  4. The smart helmet (400) having a remote control according to claim 3, characterized in that the remote control (60) further having a second microphone module (622), and the second microphone module being configured to perform recording in response to the third operation instruction.
  5. The smart helmet having a remote control according to claim 3, characterized in that the helmet further comprising a Bluetooth module and a Bluetooth speaker, the Bluetooth module being configured to receive the microphone data of the first microphone module or of the remote control and sending the microphone data to a mobile terminal connected with the helmet, and receiving a recording of other communication party, and the Bluetooth speaker being configured to broadcast the recording of the other communication party.
  6. The smart helmet having a remote control according to claim 3, characterized in that the helmet further comprising an audio amplification module which is connected with the Bluetooth module and the Bluetooth speaker, and being configured to perform power amplification for the voice of the other party, and the Bluetooth speaker is a Bluetooth speaker.
  7. The smart helmet having a remote control according to claim 3, characterized in that a wireless communication protocol used by the first wireless communication module and the second wireless communication module comprising: a Bluetooth communication protocol, an ANT+ communication protocol, an infrared communication protocol, or a 433 communication protocol.
  8. The smart helmet having a remote control according to claim 3, characterized in that the remote control being installed on a ski pole or on a handlebar of a bicycle.
EP17878836.0A 2016-12-07 2017-11-06 Smart helmet having remote control, and remote control method thereof Active EP3552508B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201611115437.5A CN106490751B (en) 2016-12-07 2016-12-07 Intelligent helmet and its remote control method with remote controler
PCT/CN2017/109453 WO2018103483A1 (en) 2016-12-07 2017-11-06 Smart helmet having remote control, and remote control method thereof

Publications (3)

Publication Number Publication Date
EP3552508A1 EP3552508A1 (en) 2019-10-16
EP3552508A4 EP3552508A4 (en) 2020-07-22
EP3552508B1 true EP3552508B1 (en) 2021-11-03

Family

ID=58330634

Family Applications (1)

Application Number Title Priority Date Filing Date
EP17878836.0A Active EP3552508B1 (en) 2016-12-07 2017-11-06 Smart helmet having remote control, and remote control method thereof

Country Status (4)

Country Link
EP (1) EP3552508B1 (en)
CN (1) CN106490751B (en)
ES (1) ES2900531T3 (en)
WO (1) WO2018103483A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106490751B (en) * 2016-12-07 2019-03-08 深圳前海零距物联网科技有限公司 Intelligent helmet and its remote control method with remote controler
CN107951117A (en) * 2017-12-31 2018-04-24 天津天健消防设备有限公司 The fire-fighting call helmet
CN110798238B (en) * 2019-11-07 2023-08-15 奥维通信股份有限公司 Be used for individual soldier's helmet-type laser receiving system
CN112841798B (en) * 2021-01-11 2023-02-10 浙江农林大学暨阳学院 Electronic anti-theft helmet

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5438702A (en) * 1993-12-28 1995-08-01 Jackson; Reed B. Bicycle helmet communication device
FR2867593B3 (en) * 2004-03-15 2006-02-24 Christine Muller REMOTE CONTROL DEVICE FOR RADIO APPARATUS HAVING SCREEN AND KEYBOARD
US8001623B2 (en) * 2005-05-26 2011-08-23 Gertsch Jeffrey H Electronic helmet
FR2921629A1 (en) * 2007-09-28 2009-04-03 Parrot Sa HANDS-FREE TELEPHONE SET FOR MOTORCYCLE, HANDLED REMOTE CONTROL
US8380128B2 (en) * 2010-11-30 2013-02-19 Motorola Solutions, Inc. User interface for a communication system
CN104157119B (en) * 2014-08-06 2018-01-16 朱亚光 Digiplex intercommunication integration external member
CN205093657U (en) * 2015-10-27 2016-03-23 舒超 Multi -function smart helmet
CN205693658U (en) * 2016-06-06 2016-11-16 深圳市绿鹰科技发展有限公司 Integrally wireless talkback communication system
CN206443280U (en) * 2016-12-07 2017-08-29 深圳前海零距物联网科技有限公司 Intelligent helmet with remote control
CN106490751B (en) * 2016-12-07 2019-03-08 深圳前海零距物联网科技有限公司 Intelligent helmet and its remote control method with remote controler

Also Published As

Publication number Publication date
EP3552508A4 (en) 2020-07-22
ES2900531T3 (en) 2022-03-17
CN106490751B (en) 2019-03-08
WO2018103483A1 (en) 2018-06-14
CN106490751A (en) 2017-03-15
EP3552508A1 (en) 2019-10-16

Similar Documents

Publication Publication Date Title
EP3552508B1 (en) Smart helmet having remote control, and remote control method thereof
US11805350B2 (en) Point-to-multipoint data transmission method and device
CN113382397B (en) Bluetooth connection method, device and system
CN110138937B (en) Call method, device and system
US11653398B2 (en) Bluetooth connection method and device
KR102102647B1 (en) Wireless receiver and method for controlling the same
CN108710615B (en) Translation method and related equipment
CN107978316A (en) The method and device of control terminal
US9167086B1 (en) Bluetooth headset with audio recording capabilities
CN104484037A (en) Method for intelligent control by virtue of wearable device and wearable device
US20100039493A1 (en) Mobile video headset
CN108595003A (en) Function control method and relevant device
EP2773552A1 (en) Mobile device for group activities having broadcasting of emergency messages capability
CN115002934A (en) Audio service processing system, electronic equipment and Bluetooth headset
KR101718466B1 (en) Wearable necklace having the functions of taking a picture and voice recording by using bluetooth with smart phone and method thereof
KR101048523B1 (en) A multi-functional walkie-talkie
CN108683997A (en) A kind of guard method of terminal, terminal and computer readable storage medium
CN110543231A (en) electronic device control method and related equipment
CN205378218U (en) Wearable auxiliary hearing aid system and portable communication device
WO2021009513A1 (en) Headset system for establishing a call session
CN206443280U (en) Intelligent helmet with remote control
CN110174837B (en) Shooting control method in call process and smart watch
JP2018037963A (en) Wearable camera system and video recording control method
CN113438102A (en) Network configuration system, network configuration method, electronic device and routing device
CN104601926A (en) Information interaction system between people and animals

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20190603

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602017048908

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: A42B0003040000

Ipc: A42B0003300000

A4 Supplementary search report drawn up and despatched

Effective date: 20200623

RIC1 Information provided on ipc code assigned before grant

Ipc: A42B 3/30 20060101AFI20200617BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20210602

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1443186

Country of ref document: AT

Kind code of ref document: T

Effective date: 20211115

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602017048908

Country of ref document: DE

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20211103

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1443186

Country of ref document: AT

Kind code of ref document: T

Effective date: 20211103

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2900531

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20220317

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220203

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220303

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220303

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220203

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220204

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211106

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211130

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20211130

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602017048908

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211130

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211130

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20220804

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211106

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20171106

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20231120

Year of fee payment: 7

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20231218

Year of fee payment: 7

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20231124

Year of fee payment: 7

Ref country code: DE

Payment date: 20231107

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211103