EP3729390B1 - Système comprenant des dispositifs d'alarme - Google Patents

Système comprenant des dispositifs d'alarme Download PDF

Info

Publication number
EP3729390B1
EP3729390B1 EP18830496.8A EP18830496A EP3729390B1 EP 3729390 B1 EP3729390 B1 EP 3729390B1 EP 18830496 A EP18830496 A EP 18830496A EP 3729390 B1 EP3729390 B1 EP 3729390B1
Authority
EP
European Patent Office
Prior art keywords
alarm
fish farm
sound
aquatic vehicle
operations
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP18830496.8A
Other languages
German (de)
English (en)
Other versions
EP3729390A1 (fr
Inventor
Pasi Karppinen
Ilmo Aronen
Knut-Olof Lerche
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Raisioaqua Oy
Original Assignee
Raisioaqua Oy
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Raisioaqua Oy filed Critical Raisioaqua Oy
Priority to EP21214120.4A priority Critical patent/EP3989193A1/fr
Publication of EP3729390A1 publication Critical patent/EP3729390A1/fr
Application granted granted Critical
Publication of EP3729390B1 publication Critical patent/EP3729390B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/16—Actuation by interference with mechanical vibrations in air or other fluid
    • G08B13/1654—Actuation by interference with mechanical vibrations in air or other fluid using passive vibration detection systems
    • G08B13/1672—Actuation by interference with mechanical vibrations in air or other fluid using passive vibration detection systems using sonic detecting means, e.g. a microphone operating in the audio frequency range
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/16—Actuation by interference with mechanical vibrations in air or other fluid
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B29/00—Checking or monitoring of signalling or alarm systems; Prevention or correction of operating errors, e.g. preventing unauthorised operation
    • G08B29/18—Prevention or correction of operating errors
    • G08B29/185—Signal analysis techniques for reducing or preventing false alarms or for enhancing the reliability of the system
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19678—User interface
    • G08B13/19684—Portable terminal, e.g. mobile phone, used for viewing video remotely

Definitions

  • the present invention relates to fish farming.
  • WO 94/00007 discloses a fish farm cage security system includes sound-emitting devices attached to the cage for detecting tampering with one or more of the nets forming the cage. Detector means are provided for detecting the sound emitted by the sound-emitting devices and to ambient sounds. The detectors may be responsive not only to the sounds emitted by the bells but also to other sounds generated in the area of the cages, such as those made by boats.
  • JP S62 116283 A discloses a system for monitoring a fish farm, the system comprising one or more alarm devices operatively connected to receiving means for acoustic signals, wherein the system is configured to determine whether an acoustic signal received by the receiving means corresponds to the sound of an aquatic vehicle approaching the fish farm.
  • a system for monitoring a fish farm comprising one or more alarm devices operatively connected to receiving means for acoustic signals, said system being configured to determine whether an acoustic signal received by the receiving means corresponds to the sound of an aquatic vehicle approaching the fish farm; and cause, by at least one of the alarm devices, one or more alarm operations, when the received acoustic signal comprises a sound indicative of the aquatic vehicle stopping at the fish farm.
  • a fish farm comprising a system according to the first aspect.
  • a system comprising one or more alarm devices operatively connected to receiving means for acoustic signals.
  • An acoustic signal received by the receiving means is determined to correspond to an aquatic vehicle approaching a fish farm.
  • the received acoustic signal comprises a stopping sound of the aquatic vehicle at the fish farm.
  • one or more alarm operations are performed. In this way alarm operations of the fish farm may be controlled on the basis of the stopping sound of the aquatic vehicle, whereby security and monitoring of the fish farm may be provided.
  • FIGURE 1 illustrates a fish farm 106 in accordance with at least some embodiments of the present invention.
  • the fish farm may comprise one or more floating cage structures, i.e. fish farm cages, at sea or lake.
  • a fish farm cage 103 may comprise a floating walkway from which suspended nets form an enclosure for the fish inside the cage. Accordingly the cage structures are mostly under the water surface 110.
  • the cage structures may extend downwards from the water surface and in some cases all the way to the bottom 112.
  • the fish farm may be visited by an aquatic vehicle and/or aquatic vehicles may pass by the fish farm. Examples of the aquatic vehicles comprise a motorboat, a rowboat, a kayak, a canoe or a sailing boat.
  • the fish farm may comprise a system comprising one or more alarm devices operatively connected to receiving means for acoustic signals, said system configured to cause one or more functionalities in accordance with at least some embodiments described herein.
  • the system 2. comprises receiving means 104 for acoustic signals.
  • the receiving means are capable of receiving acoustic signals under water or in the air, in the area of the cage and/or around the cage. In this way acoustic signals are received for determining, whether an acoustic signal received by the receiving means 104 corresponds to an aquatic vehicle 105 approaching the fish farm 106.
  • the system further comprises one or more alarm devices 102.
  • the alarm devices are operatively connected to the receiving means 104. In this way, one or more alarm operations are performed, when the received acoustic signal comprises a stopping sound of the aquatic vehicle 105 at the fish farm 106. Accordingly, alarm operations of the fish farm are controlled on the basis of the stopping sound of the aquatic vehicle, whereby security and monitoring of the fish farm are provided.
  • Visits of aquatic vehicles to the fish farm may comprise authorized and unauthorized visits. Typically only authorized persons are allowed to visit the fish farm. Detecting authorized and/or unauthorized visits provide that activity of the fish farming may be monitored. More particularly, detected visits may be logged to a history for later analysis. The unauthorized visits to the fish farm are a risk to fish farming for various reasons and their detection serves for supporting production yield of the fish farm. In one example, unauthorized visits may cause contamination of the fish stock, if contaminated fish farming equipment is brought into contact with the fish stock. In another example, the farming process such as the feeding times of the fish may be disturbed. In yet another example, unauthorized visitors may steal fish from the fish farm. Moreover, aquatic vehicles that approach the fish farm may be detected by the fish, whereby movement of the fish is increased, which may lead to injuries and stress of the fish.
  • the alarm devices 102 and the receiving means 104 may be positioned to various locations at the fish farm. Examples of the locations comprise the bottom 112 of the sea or lake, portion of the cage structure above the surface. Moreover, the alarm devices may be installed to a pole such that they are significantly above the cage structure to enable improved coverage for the alarm operations performed by the alarm devices. Similarly, the receiving means may be installed to a pole, whereby unobstructed reception of acoustic signals may be provided.
  • the alarm devices 102 may comprise one or more of a sound output device and spotlight that may be local to the fish farm.
  • the alarm devices may alternatively or additionally comprise devices that are external to the fish farm and which can be moved to different locations. Accordingly, the alarm devices may be local to the fish farm or remote from the fish farm.
  • Such devices may comprise computing devices for example mobile devices and handheld devices, such as smart phones.
  • Examples of the sound output devices comprise a loudspeaker, a bullhorn and a siren.
  • an alarm operation comprises turning on one or more spotlights directed to one or more locations at the fish farm. In this way specific locations may be enlightened by the spotlights, whereby unauthorized visitors at those locations may be detected more efficiently.
  • one or more video recorders may be installed at the fish farm to record video from the locations that are enlightened by the spotlights, whereby video of the unauthorized visitors may be recorded.
  • the video recorders may be enabled with night vision, whereby the video recorders may record video without the spotlights being turned on. Accordingly, an alarm operation may comprise recording video by one or more video recorders installed at the fish farm.
  • one or more still cameras may be installed at the fish farm to take still photos from the locations that are enlightened by the spotlights, whereby photos of the unauthorized visitors may be taken.
  • the still cameras may be enabled with night vision, whereby the still cameras may take still photos without the spotlights being turned on. Accordingly, an alarm operation may comprise taking still photos by one or more still cameras installed at the fish farm.
  • an alarm operation comprises causing an alarm sound and/or audio playback by a sound output device.
  • the alarm sound may notify the unauthorized visitor about the alarm.
  • the audio playback may be an audio recording of speech that asks the unauthorized visitor to move away from the fish farm.
  • a neighbourhood of the fish farm 106 may comprise an area around the fish farm, where acoustic signals originating from locations of the neighbourhood are perceivable to the receiving means of the fish farm.
  • the neighbourhood of the fish farm comprises preferably an area around the fish farm, where the acoustic signals originating from locations of the neighbourhood may be classified into at least one class of a plurality of classes, said plurality of classes comprising at least one class of acoustic signals, where alarm operations are not performed and at least one other class of acoustic signals, where alarm operations are performed.
  • classes of acoustic signals where alarm operations are not performed comprise at least one of a class indicating that an acoustic signal fails to correspond to an aquatic vehicle and a class indicating an acoustic signal that corresponds to an aquatic vehicle passing the fish farm without stopping at the fish farm 106.
  • classes of acoustic signals where alarm operations are performed comprise a class indicating that an acoustic signal corresponds to a stopping sound of an aquatic vehicle at the fish farm.
  • classes of acoustic signals are configured to the system by a machine-learning process, whereby classification of received acoustic signals may be machine-controlled.
  • the class of acoustic signals, where alarm operations are not performed may comprise acoustic signals that are not generated by an aquatic vehicle. It should be appreciated that the class of acoustic signals, where alarm operations are not performed, may further comprise acoustic signals generated by aquatic vehicles that bypass the fish farm. Accordingly, acoustic signals belonging to this class may first have increasing amplitude after which the amplitude begins to decrease, without any stopping sounds there between. The increasing amplitude followed by decreasing amplitude may be used to determine that the aquatic vehicle has passed the fish farm without stopping at the fish farm.
  • Improved accuracy for classifying acoustic signals into this class may be provided, when received acoustic signals are classified into this class provided the decreasing amplitude of the acoustic signal follows within a specific time frame. If the decreasing amplitude does not follow within the specific time, the acoustic signal may be classified into another class, for example to the class where alarm operations are performed, since a lack of satisfying the condition that the decreasing amplitude would follow within the specific time may be interpreted as a stopping sound of the aquatic vehicle.
  • the class of acoustic signals may comprise acoustic signals that are generated by an aquatic vehicle. Accordingly, acoustic signals belonging to this class may have increasing amplitude and comprise one or more stopping sounds.
  • the stopping sounds may comprise a clutch sound, mooring sound, impact sound, sound of fixing to the fish farm and/or a lack of sound indicating the aquatic vehicle moving away from the fish farm. The lack of sound may be determined after a specific time has passed after detecting the sound of the aquatic vehicle.
  • FIGURE 2 illustrates a method in accordance with at least some embodiments of the present invention.
  • the method may be performed at the fish farm described with FIGURE 1 .
  • Phase 202 comprises determining an acoustic signal received by the receiving means to correspond to an aquatic vehicle approaching a fish farm.
  • Phase 204 comprises causing, by at least one of the alarm devices, one or more alarm operations, when the received acoustic signal comprises a stopping sound of the aquatic vehicle at the fish farm.
  • the method provides that alarm operations of the fish farm are controlled on the basis of the stopping sound of the aquatic vehicle, whereby security and monitoring of the fish farm may be provided.
  • phase 202 may comprise that the received acoustic signal is classified into at least one class of a plurality of classes configured to the system by a machine-learning process, said plurality of classes comprising at least one class of acoustic signals, where alarm operations are not performed and at least one other class of acoustic signals, where alarm operations are performed.
  • the received acoustic signal may be processed by a classification determined by the machine-learning process, for example a support vector machines (SVM) process.
  • SVM support vector machines
  • phase 202 may comprise that the classification of the received acoustic signal may be updated based on user input for turning alarm operations on or a user input for turning alarm operations off for the received acoustic signal.
  • the user input may be obtained via a user interface on a computing device, for example.
  • phase 204 may comprise that the alarm operation are caused, when the stopping sound indicates that the aquatic vehicle is sufficiently close for getting off the vehicle to the fish farm. In this way the, unnecessary alarm operations may be avoided.
  • the location of the aquatic vehicle may be determined on the basis of the characteristics of the received acoustic signal.
  • the characteristics may comprise amplitude, power and/or frequency characteristics. Alternatively or additionally, one or more changes of the characteristics may be used to determine location of the aquatic vehicle.
  • the aquatic vehicle may be determined to be sufficiently close for getting of the vehicle, when the power of the acoustic signal is above a threshold level.
  • the aquatic vehicle may be determined to be a stopping sound and to be stopped at the fish farm for getting of the vehicle.
  • the change of the frequency characteristics may be determined, when the acoustic signal first comprises Doppler frequencies caused by the Doppler effect, when the aquatic vehicle approaches the fish farm, and thereafter the Doppler frequencies are removed.
  • the aquatic vehicle may be sufficiently close for getting off the vehicle when the distance between the aquatic vehicle and the fish farm is less than 2m.
  • the minimum distance is, when the aquatic vehicle has arrived at the fish farm and touches the fish farm.
  • the aquatic vehicle that arrives at the fish farms makes an impact with the fish farm.
  • the alarm operations may be initiated in a sequence. Particularly, if the alarm operations may be performed by different types of devices, the alarm operations may be initiated one type of alarm devices at a time. Consequently, more than one, at least two, a part or all the alarm operations may be performed at the same time.
  • the alarm devices may comprise a spot light, a sound output device, and a computing device such as a smart phone of the fish farmer.
  • the first alarm operation may comprise turning on the spot light
  • the second alarm operation may comprise causing an alarm sound
  • the third alarm operation may comprise causing an audio playback
  • the fourth alarm operation may comprise displaying a notification indicating an alarm on the user interface of the computing device.
  • the notification indicating an alarm on the user interface may be displayed in connection with any of the other alarm operations instead of being performed separately.
  • the notification may be provided with a sound played by the computing device, to aid the user to observe the notification.
  • a stopping sound comprises a clutch sound, a mooring sound, an impact sound, a sound of fixing to the fish farm and/or a lack of sound indicating the aquatic vehicle moving away from the fish farm.
  • the lack of sound may be determined after a specific time has passed after detecting the sound of the aquatic vehicle.
  • the acoustic signals are determined using increasing amplitude and decreasing amplitude of the acoustic signals, however, alternatively or additionally other properties of the received acoustic signal may be utilized, for example properties derivable based on the Doppler effect.
  • a clutch sound may be generated by a clutch of the aquatic vehicle.
  • the aquatic vehicle may be a motorized aquatic vehicle comprising an engine.
  • the engine may be a diesel engine or a petrol engine. Accordingly, when an acoustic signal generated by the aquatic vehicle is received and the aquatic vehicle is approaching the fish farm and the acoustic signal comprises a clutch sound, it may be determined that the aquatic vehicle is stopping at the fish farm.
  • characteristics of the clutch sound comprise a series of tapping sounds for e.g. 0,1 seconds, re-occurrence of the tapping sounds and duration of the received acoustic signal. In an example, re-occurrence of the tapping sounds within a specific time period may be used to determine that the clutch is used repeatedly, when the aquatic vehicle is stopping at the fish cage.
  • a mooring sound may be caused by mooring operations performed at the aquatic vehicle.
  • the mooring operations comprise an impact sound between the aquatic vehicle and the fish farm and fixing the aquatic vehicle to the fish farm.
  • the aquatic vehicle may be fixed to the fish farm by ropes and/or chains. Accordingly, fixing the aquatic vehicle may cause acoustic signals originating from the chains and/or ropes, when the chains and/or ropes touch the fish farm and/or the aquatic vehicle.
  • FIGURE 3 illustrates a system in accordance with at least some embodiments of the present invention.
  • the system is described with reference to examples of units illustrated in the FIGURE 3 . However, it should be appreciated that also other implementations of the system may be viable.
  • the system comprises one or more alarm devices 302 and one or more receiving means 304 for acoustic signals.
  • the alarm devices are operatively connected to the receiving means to perform the method described with FIGURE 2 . In this way alarm operations of the fish farm are controlled on the basis of the stopping sound of the aquatic vehicle, whereby security and monitoring of the fish farm are provided.
  • the system may be installed to the fish farm described with FIGURE 1 . Examples of the alarm device comprise a spot light, a sound output device, and a computing device such as a smart phone of the fish farmer.
  • the system may comprise a data connection unit (DCU) 310 configured to communicate information indicating an alarm over a data connection to a remote system (RS) 312.
  • DCU data connection unit
  • RS remote system
  • the remote system may obtain data, e.g. information indicating an alarm, and perform one or more operations for supporting security and/or monitoring of the fish farm.
  • the remote system and/or devices connected to the remote system may serve for an alarm device for example for alarming users that are remote from the fish farm.
  • the remote system may be in a fixed location ashore or the remote system may be a mobile system, mobile device or handheld device.
  • wireless connections between the system and the remote system may be preferred.
  • wired connections may be preferred, if wireless connections are not available and a wired connection would be available.
  • An example of the system comprises a controller (CNTL) 306 and a memory (M) 308 that are connected to the alarm device, receiving means and the data connection unit by wired or wireless connections.
  • the data connection unit may be connected to the remote system by wired or wireless connection to provide the remote system information indicating an alarm to enable the remote system to perform one or more operations for supporting security and/or monitoring of the fish farm.
  • the connections may be capable of communications of data between at least part of the units of the system.
  • the data may comprise one or more of: acoustic signals, video, still images, data indicating one or more acoustic signals, information indicating an alarm and information indicating an alarm operation.
  • the data indicating an acoustic signal may comprise an acoustic signal received by the receiving means and which has been digitized and/or classified.
  • the information indicating an alarm may indicate that a received acoustic signal comprises a stopping sound of the aquatic vehicle at the fish farm.
  • the information indicating the alarm may further indicate one or more alarm operations caused or to be caused, when the received acoustic signal comprises a stopping sound of the aquatic vehicle at the fish farm.
  • information indicating an alarm operation may indicate an alarm operation to be performed, an alarm operation that has been performed and/or an alarm operation that is being performed.
  • the alarm operation may comprise transmitting information indicating an alarm to a computing device, turning on one or more spotlights, recording video, taking still photos, causing an alarm sound by a sound output device and/or causing audio playback by a sound output device.
  • the information indicating an alarm operation may serve for information indicating an alarm.
  • the controller 306 may be a processor capable of executing computer program code and/or instructions to cause one or more functionalities described herein in various embodiments.
  • the computer program code and/or instructions may be stored in the memory 308 and the computer program code and/or instructions may be accessed by the processor to retrieve the computer program code and/or instructions for executing the computer program code and/or instructions.
  • the alarm devices, receiving means, data connection unit and remote system may include one or more similar processors and memory for execution of functionalities described herein.
  • connections between units devices and apparatuses comprise wired connections for data transfer such as computer buses.
  • the units may be connected by wireless connections of which examples comprise data connections provided by wireless communications networks.
  • wireless communications networks comprise mobile communications networks based on GSM, IS-95, UMTS, CDMA2000 or LTE technology, and IoT networks based on NB-IoT, SIGFOX, LoRaWAN.
  • the receiving means 304 may comprise means for receiving acoustic signals carried by the air and/or water from a source of the acoustic signal to the fish farm.
  • the receiving means may comprise one or more video cameras, still cameras, microphones and/or hydrophones.
  • the receiving means is at least configured to receive acoustic signals from an aquatic vehicle. Further receiving means such as video cameras and still camera may provide that the source of the acoustic signal may be verified by a user on the basis of video and/or still images.
  • the remote system 312 may be a network service provided by one or more network elements located in a communications network.
  • the remote system may be accessible to computing devices and/or the system over data connections.
  • the network service may be a single server computer or a cloud computing service.
  • the remote system may be a computing device connected to the communications network.
  • the system may send a message to the remote system. Examples of messages comprise Short Message Service and Internet Protocol-based messaging services such as Facebook Messenger, WhatsApp, Viber, WeChat and Line, available on smart phones with data connections.
  • the message may serve for a notification indicating the alarm on a user interface of the remote system.
  • the system and the remote system may essentially operate as terminals of a mobile communications network. Terminals of the mobile communications network may be smartphones that are capable or executing third-party applications and have touch screen displays for providing graphical user interfaces.
  • the receiving means 304 may be an acoustic positioning system capable of positioning aquatic vehicles approaching the fish farm.
  • the acoustic positioning system may be an underwater positioning system or a surface positioning system.
  • the acoustic positioning system may perform tracking and positioning of the aquatic vehicle by means of acoustic distance and/or direction measurements, and subsequent position triangulation.
  • the underwater positioning system may comprise hydrophones that receive underwater acoustic signals for tracking and positioning of the aquatic vehicle.
  • the surface positioning system may comprise microphones that receive acoustic signals in the air for tracking and positioning of the aquatic vehicle.
  • At least one alarm operation may be caused, when a stopping sound is determined from an aquatic vehicle located at at least one location at the fish farm. Accordingly, the stopping sound from the location may be used to trigger one or more alarm operations.
  • the location may be determined for example by the receiving means that is an acoustic positioning system as described above.
  • the receiving means that is an acoustic positioning system as described above.
  • one or more alarm operations may be caused. In this way reception sensitivity of the acoustics signals may be tuned to the predefined locations.
  • the path taken by the aquatic vehicle may be tracked by the acoustic positioning system to ascertain it is approaching the fish farm, whereby unnecessary alarm operations after reception of the stopping sound may be avoided.
  • FIGURE 4a illustrates configuring a system for processing acoustic signals in accordance with at least some embodiments of the present invention.
  • the system may be the system described with FIGURE 3 , which is now referred together with FIGURE 4a .
  • the system may have a training mode and a production mode, whereby the system may be first configured in the training mode before the mode is switched to the production mode after configuration of the system is performed. It should be appreciated that functionalities of the training mode may be performed also after an initial configuration of the system for improving the processing capabilities of the acoustic signals.
  • Configuring the system may comprise receiving 402 input data.
  • the input data may be received from the receiving means 304.
  • the input data may be processed by a machine-learning process to determine parameters for classifying input data.
  • the machine-learning process may be a support vector machines (SVM) process for example.
  • SVM support vector machines
  • Features may be extracted 406 from the received input data and the extracted features may be fed to a learning model 408.
  • the extracted features may comprise characteristics of the received acoustic signal, e.g. amplitude, power and/or frequency characteristics of the acoustic signal. Alternatively or additionally the characteristics may comprise changes of the amplitude, power and/or frequency.
  • the extracted features may define a stopping sound comprising a clutch sound, a mooring sound, impact sound, sound of fixing to the fish farm and/or a lack of sound indicating the aquatic vehicle moving away from the fish farm.
  • a clutch sound may comprise a series of tapping sounds for e.g.
  • re-occurrence of the tapping sounds and duration of the received acoustic signal may be used to determine that the clutch is used repeatedly, when the aquatic vehicle is stopping at the fish cage.
  • the input data may be evaluated 404 also by a user and the user may provide a result of the evaluation as input to the learning model 408.
  • the input of the user may be provided via a user interface provide on a computing device, for example.
  • the learning model may determine 410 parameters for classifying input data on the basis of the evaluation result from the user and the extracted features.
  • the parameters may be used for classifying input data to at least one class of more than one, for example two, three, four or more classes. Examples of the classes are given below with FIGURE 4b . After the parameters have been determined the system has been configured and the system may be switched to the production mode.
  • the input data or at least part of the input data may be transmitted by the data connection unit 310 to a remote system 312, for example a computing device operated by the user.
  • the computing device may comprise a user interface via which the user may evaluate the input data and provide his/her evaluation result.
  • the evaluation result of the user may be transmitted from the computing device to the data connection unit of the apparatus to be used as input to the learning model 408.
  • the input data may comprise one or more video, still images, acoustic signals and/or data indicating one or more acoustic signals, whereby the system may be configured to cause one or more alarm operations, when the received acoustic signal comprises a stopping sound of the aquatic vehicle at the fish farm.
  • the input data evaluated 404 by the user may be different than the input data fed to feature extraction 406.
  • the input data to the feature extraction may comprise acoustic signals and/or data indicating one or more acoustic signals
  • the input data evaluated 404 by the user may comprise video and/or still images.
  • the user may also evaluate a representation of the acoustic signals and/or data indicating one or more acoustic signals. In this way the capabilities of the user may facilitate interpretation of the input data, when the result of the evaluation of the user is fed as input to the learning model 408.
  • FIGURE 4b illustrate an example operation of a system configured for processing acoustic signals in accordance with at least some embodiments of the present invention.
  • the system may be the system described with FIGURE 3 .
  • the system may have a training mode and a production mode, whereby the system may be first configured in the training mode, for example in accordance with FIGURE 4a , and the mode may be switched to the production mode, after the system is configured.
  • the FIGURE 4b is explained with reference to FIGURE 4a and FIGURE 3 .
  • input data and feature extraction may be performed similar to described with FIGURE 4a .
  • the extracted features may be classified 412 on the basis of parameters 410 for classifying input data.
  • the extracted features may comprise characteristics of the received acoustic signal, e.g. amplitude, power and/or frequency characteristics of the acoustic signal. Alternatively or additionally the characteristics may comprise changes of the amplitude, power and/or frequency.
  • the extracted features may define a stopping sound comprising a clutch sound, a mooring sound, impact sound, sound of fixing to the fish farm and/or a lack of sound indicating the aquatic vehicle moving away from the fish farm.
  • a clutch sound may comprise a series of tapping sounds for e.g.
  • re-occurrence of the tapping sounds and duration of the received acoustic signal may be used to determine that the clutch is used repeatedly, when the aquatic vehicle is stopping at the fish cage.
  • the classification 412 may output a result indicating at least one class 414, 416, 418 of more than one, for example two, three, four or more classes.
  • the classes may comprise at least one class indicating that the input data comprises an acoustic signal that fails to correspond to an aquatic vehicle.
  • the classes may further comprise at least one class indicating that the input data comprises an acoustic signal that corresponds to a stopping sound of an aquatic vehicle at the fish farm.
  • the classes may further comprise at least one class indicating that the input data comprises an acoustic signal that corresponds to an aquatic vehicle passing the fish farm.
  • FIGURE 5 illustrates a user interface for alarm operations in accordance with at least some embodiments of the present invention.
  • the user interface 500 may be provided on a computing device.
  • the user interface may be caused to display a notification 506 indicating an alarm on the user interface in response to the computing device receiving information indicating an alarm operation.
  • the notification may be provided with a sound to aid the user to observe the notification. Examples of the notification comprise a graphical element on the user interface, video and/or still image.
  • the alarm operation may be caused in response to a received wave at the fish cage corresponding to at least one type of waves, for example as described with FIGURE 2 .
  • the user interface may be provided on a remote system 312 described with FIGURE 3 .
  • the user interface 500 may comprise one or more elements capable of receiving a user input from the user of the computing device.
  • the elements may comprise an element 502 for turning alarm operations on and/or an element 504 for turning alarm operations off.
  • the notification 506 may be an in-app notification or a push notification.
  • An in-app notification may be also referred to as in-app messages.
  • the in-app notifications may be displayed on the user interface during use of a specific app, i.e. application. Accordingly, the in-app notification may be displayed directly within the app itself executed in the computing device.
  • the notification 506 may be a push notification.
  • the push notification may be displayed on the user interface even if a specific application is not being displayed. Accordingly, the push notification may be a notification by one application being executed in the computing device in the background and the push notification may be displayed on top of a home screen of the computing device or on top of a user interface view of another application. In this way the push notification may serve for alerting the user of the device even if the application is executed in the background.
  • the notification 506 may comprise one or more video and/or still image.
  • the video and/or still image may be received from a receiving means, e.g. video camera and alarm device, of the system described with FIGURE 3 .
  • the user may evaluate the video and/or still images for determining whether the alarm indicated by the notification should be turned on or off.
  • the user may select the element 502 for turning alarm operations on or the element 504 for turning alarm operations off in accordance with the result of the user's evaluation of the video and/or still images.
  • the evaluation of the user may be utilized in configuring the system in accordance with FIGURE 4a .
  • selection of the user to turn the alarm on or off on the basis of the video and/or still images may be fed to the learning model 408 for determining 410 new or updated parameters for classifying acoustic signals.
  • the system may be configured to classify received acoustic signals based on notifications indicating alarm operations particularly after an initial configuration of the system.
  • the new or updated parameters may be applied in accordance with the operation described with FIGURE 4b .
  • Configuration of the system after initial configuration provides that the system may achieve a relatively high accuracy for causing alarm operations, whereby the system may be adapted to provide reliable monitoring for a specific fish farm and user.
  • notification may comprise messages of Short Message Service and Internet Protocol-based messaging services such as Facebook Messenger, WhatsApp, Viber, WeChat and Line, available on smart phones with data connections.
  • a computing device may be configured for communications of messages according to one or more of the above messaging services.
  • a received acoustic signal may be classified to a class, where an alarm operation is performed.
  • the alarm operation may comprise displaying a notification 506 indicating the alarm on the user interface.
  • a user input received in response to the alarm operation may be used for configuring a system for processing acoustic signals, for example as described with FIGURE 4a .
  • the user input may be received by an element 504 for turning alarm operations off.
  • the received acoustic signal may be received 402 as input data in the process of FIGURE 4a and the user input received in response to the alarm operation may be fed to the learning model 408 for determining 410 new or updated parameters for classifying acoustic signals.
  • the new or updated parameters are determined such that the received acoustic signal does not cause an alarm operation.
  • the determined 410 parameters may be applied in classifying the acoustic signals in accordance with the operation described with FIGURE 4b . Accordingly, the updated parameters may cause that the received acoustic signal is classified into a class, where alarm operations are not performed. In this way the system may be configured to turn off alarm operations for specific received acoustic signals.
  • a notification 506 indicating the acoustic signal may be displayed on the user interface. Then a user input received in response the notification may be used for configuring a system for processing acoustic signals.
  • the user input received by an element 502 for turning alarm operations on. Accordingly, the received acoustic signal may be received 402 as input data in the process of FIGURE 4a and the user input received in response to the notification may be fed to the learning model 408 for determining 410 new or updated parameters for classifying acoustic signals.
  • the new or updated parameters are determined 410 such that the received acoustic signal causes an alarm operation.
  • the determined parameters may be applied in classifying the acoustic signals in accordance with the operation described with FIGURE 4b .
  • the updated parameters may cause that the received acoustic signal is classified into a class, where alarm operations are performed. In this way the system may be configured to turn on alarm operations for received acoustic signals.
  • the user input may be received by the user operating the user interface 500.
  • the user interface may comprise various forms of possibilities to receive user input, comprising, a button, a user interface element or a location on a touch screen.
  • the user input may cause that classification of the acoustic signals is updated such that alarm operations for specific received acoustic signals are turned on or off.
  • the notification may be an in-app notification or a push notification.
  • An in-app notification may be also referred to as in-app messages.
  • the in-app notifications may be displayed on the user interface during use of a specific app, i.e. application. Accordingly, the in-app notification may be displayed directly within the app itself executed in the computing device.
  • the notification may be a push notification.
  • the push notification may be displayed don the user interface even if a specific application is not being displayed.
  • the push notification may be a notification by one application being executed in the computing device in in the background and the push notification may be displayed on top of a home screen of the computing device or on top of a user interface view of another application. In this way the push notification may serve for alerting the user of the device even if the application is executed in the background.
  • the present invention can be applied in monitoring a fish farm.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Multimedia (AREA)
  • Emergency Alarm Devices (AREA)
  • Traffic Control Systems (AREA)
  • Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)

Claims (15)

  1. Système de surveillance d'une ferme piscicole, le système comprenant un ou plusieurs dispositifs d'alarme connectés fonctionnellement à un moyen de réception de signaux acoustiques, dans lequel
    ledit système est configuré pour déterminer (202) si un signal acoustique reçu par le moyen de réception correspond au son d'un véhicule aquatique s'approchant de la ferme piscicole et déclencher (204), par au moins l'un des dispositifs d'alarme, une ou plusieurs opérations d'alarme, lorsque le signal acoustique reçu comprend un son indicatif du véhicule aquatique s'arrêtant au niveau de la ferme piscicole.
  2. Système selon la revendication 1, dans lequel le signal acoustique reçu est classé dans au moins une classe d'une pluralité de classes configurées pour le système par un processus d'apprentissage automatique, ladite pluralité de classes comprenant au moins une classe de signaux acoustiques, où des opérations d'alarme ne sont pas exécutées et au moins une autre classe de signaux acoustiques, où des opérations d'alarme sont exécutées.
  3. Système selon la revendication 2, dans lequel la classification est mise à jour sur la base d'une entrée d'utilisateur pour activer des opérations d'alarme ou d'une entrée d'utilisateur pour désactiver des opérations d'alarme.
  4. Système selon la revendication 1, 2 ou 3, dans lequel le son indiquant que le véhicule aquatique s'arrête au niveau de la ferme piscicole comprend un son d'embrayage, un son d'amarrage, un son de fixation à la jetée et/ou une absence de son indiquant que le véhicule aquatique s'éloigne de la ferme piscicole.
  5. Système selon la revendication 1, 2, 3 ou 4, dans lequel l'opération d'alarme est déclenchée, lorsque le son indiquant que le véhicule aquatique s'arrête au niveau de la ferme piscicole indique que le véhicule aquatique est suffisamment proche pour descendre du véhicule vers la ferme piscicole.
  6. Système selon l'une quelconque de la revendication 1 à 5, dans lequel le moyen de réception comprend une ou plusieurs caméras vidéo, caméras fixes, microphones et/ou hydrophones.
  7. Système selon l'une quelconque de la revendication 1 à 6, dans lequel le moyen de réception est un système de positionnement acoustique capable de positionner des véhicules aquatiques s'approchant de la ferme piscicole.
  8. Système selon l'une quelconque des revendications 1 à 7, dans lequel les opérations d'alarme sont déclenchées, lorsque le son indiquant que le véhicule aquatique s'arrête au niveau de la ferme piscicole est déterminé à partir du véhicule aquatique situé au niveau d'au moins un emplacement au niveau de la ferme piscicole.
  9. Système selon la revendication 2 et l'une quelconque de la revendication 3 à 8, dans lequel la classification des signaux acoustiques reçus est commandée par machine et la classification commandée par machine est mise à jour sur la base d'une entrée d'utilisateur.
  10. Système selon l'une quelconque de la revendication 1 à 9, comprenant une unité de connexion de données configurée pour communiquer des informations indiquant une alarme sur une connexion de données à un système distant.
  11. Système selon l'une quelconque de la revendication 1 à 10, dans lequel les opérations d'alarme comprennent la transmission d'informations indiquant une alarme à un dispositif informatique comprenant une interface utilisateur, dans lequel les informations indiquant l'alarme déclenchent l'affichage d'une notification indiquant l'alarme sur l'interface utilisateur.
  12. Système selon l'une quelconque des revendications 1 à 11, dans lequel les opérations d'alarme comprennent l'allumage d'un ou de plusieurs projecteurs dirigés vers un ou plusieurs emplacements au niveau de la ferme piscicole.
  13. Système selon l'une quelconque des revendications 1 à 12, dans lequel les opérations d'alarme comprennent le déclenchement d'un son d'alarme et/ou d'une lecture audio par un dispositif de sortie sonore.
  14. Système selon l'une quelconque des revendications 1 à 13, dans lequel le véhicule aquatique est un bateau à moteur, une barque, un kayak, un canoë ou un bateau à voile.
  15. Ferme piscicole caractérisée en ce que la ferme piscicole comprend un système selon l'une quelconque des revendications 1 à 14.
EP18830496.8A 2017-12-22 2018-12-18 Système comprenant des dispositifs d'alarme Active EP3729390B1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP21214120.4A EP3989193A1 (fr) 2017-12-22 2018-12-18 Système comprenant des dispositifs d'alarme

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FI20176169A FI127510B (en) 2017-12-22 2017-12-22 Systems comprising alarm devices
PCT/FI2018/050941 WO2019122523A1 (fr) 2017-12-22 2018-12-18 Système comprenant des dispositifs d'alarme

Related Child Applications (2)

Application Number Title Priority Date Filing Date
EP21214120.4A Division-Into EP3989193A1 (fr) 2017-12-22 2018-12-18 Système comprenant des dispositifs d'alarme
EP21214120.4A Division EP3989193A1 (fr) 2017-12-22 2018-12-18 Système comprenant des dispositifs d'alarme

Publications (2)

Publication Number Publication Date
EP3729390A1 EP3729390A1 (fr) 2020-10-28
EP3729390B1 true EP3729390B1 (fr) 2022-02-02

Family

ID=63112836

Family Applications (2)

Application Number Title Priority Date Filing Date
EP18830496.8A Active EP3729390B1 (fr) 2017-12-22 2018-12-18 Système comprenant des dispositifs d'alarme
EP21214120.4A Withdrawn EP3989193A1 (fr) 2017-12-22 2018-12-18 Système comprenant des dispositifs d'alarme

Family Applications After (1)

Application Number Title Priority Date Filing Date
EP21214120.4A Withdrawn EP3989193A1 (fr) 2017-12-22 2018-12-18 Système comprenant des dispositifs d'alarme

Country Status (4)

Country Link
EP (2) EP3729390B1 (fr)
EA (1) EA038889B1 (fr)
FI (1) FI127510B (fr)
WO (1) WO2019122523A1 (fr)

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS62116283A (ja) * 1985-11-15 1987-05-27 Nec Corp 音源位置標定装置
GB9213715D0 (en) 1992-06-27 1992-08-12 Jackman Stephen E Fish farm cage security system
MY165896A (en) * 2009-11-03 2018-05-18 Stevens Institute Of Technology Passive acoustic underwater intruder detection system
US8489065B2 (en) * 2011-05-03 2013-07-16 Robert M Green Mobile device controller application for any security system
KR102058918B1 (ko) * 2012-12-14 2019-12-26 삼성전자주식회사 홈 모니터링 방법 및 장치
GB201518050D0 (en) * 2015-10-12 2015-11-25 Binatone Electronics Internat Ltd Home monitoring and control systems

Also Published As

Publication number Publication date
EA202091564A1 (ru) 2020-12-02
FI127510B (en) 2018-08-15
EA038889B1 (ru) 2021-11-02
EP3729390A1 (fr) 2020-10-28
WO2019122523A1 (fr) 2019-06-27
EP3989193A1 (fr) 2022-04-27

Similar Documents

Publication Publication Date Title
US9049352B2 (en) Pool monitor systems and methods
US9769435B2 (en) Monitoring systems and methods
KR102195897B1 (ko) 음향 사건 검출 장치, 그 동작 방법 및 그 동작 방법을 컴퓨터에서 실행시키기 위한 프로그램을 기록한 컴퓨터 판독 가능 기록 매체
US11990992B2 (en) System and method for triggering an alarm during a sensor jamming attack
CN109416867B (zh) 机器人、机器人系统和记录介质
US10555393B1 (en) Face recognition systems with external stimulus
US9055202B1 (en) Doorbell communication systems and methods
JP6163017B2 (ja) 携帯端末、及び、危険報知システム
KR20190098105A (ko) 스마트 홈 감시 장치 및 방법
US20140111336A1 (en) Method and system for awareness detection
KR20170091466A (ko) 물고기의 종류를 감지하는 스마트 낚시찌
CN110209281B (zh) 对运动信号进行处理的方法、电子设备和介质
US20180141627A1 (en) Systems and methods for handling a man overboard situation
JP2017133901A (ja) 監視装置、および監視方法、並びにプログラム
JP6169747B2 (ja) 携帯端末
EP3729390B1 (fr) Système comprenant des dispositifs d'alarme
US11004324B1 (en) Pool alarm
JP5371103B2 (ja) 通報システム、通報装置、通報制御方法、ならびに、プログラム
US10798237B2 (en) Automated individual security
JP2015133099A (ja) 養殖場監視システム
CN110807896A (zh) 安全监护方法、装置及系统
US20240101059A1 (en) System and method for campsite monitoring using vehicle sensors
WO2024123710A1 (fr) Systèmes et procédés de sécurité de surveillance et de protection de piscine
JP2021162481A (ja) 情報処理装置、情報処理方法、プログラムおよびシステム
JP2023149905A (ja) サーバ、処理方法およびプログラム

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200626

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20210812

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 1466787

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220215

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602018030422

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20220202

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1466787

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220202

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220602

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220502

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220502

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220503

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220602

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602018030422

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20221103

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602018030422

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20221218

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20221231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221231

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221218

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221218

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230701

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221231

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220202