WO2024068142A1 - Ski goggles, wireless communication terminal, and method of controlling a display of ski goggles - Google Patents

Ski goggles, wireless communication terminal, and method of controlling a display of ski goggles Download PDF

Info

Publication number
WO2024068142A1
WO2024068142A1 PCT/EP2023/073082 EP2023073082W WO2024068142A1 WO 2024068142 A1 WO2024068142 A1 WO 2024068142A1 EP 2023073082 W EP2023073082 W EP 2023073082W WO 2024068142 A1 WO2024068142 A1 WO 2024068142A1
Authority
WO
WIPO (PCT)
Prior art keywords
ski goggles
display
ski
output
information
Prior art date
Application number
PCT/EP2023/073082
Other languages
French (fr)
Inventor
Aiping GAO
Amir MELZER
Original Assignee
Ostloong Innovations Ag
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ostloong Innovations Ag filed Critical Ostloong Innovations Ag
Publication of WO2024068142A1 publication Critical patent/WO2024068142A1/en

Links

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0101Head-up displays characterised by optical features
    • G02B2027/014Head-up displays characterised by optical features comprising information/image processing systems
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0101Head-up displays characterised by optical features
    • G02B2027/0141Head-up displays characterised by optical features characterised by the informative content of the display

Definitions

  • the present invention relates to ski goggles according to the preamble of independent claim 1 and more particularly to ski goggles that comprise a display.
  • ski goggles can be used for providing information to a wearer by means of the display and implementing augmented reality functions in ski goggles.
  • CN 112 782 870 A and CN 214 252 800 U disclose ski goggles which comprise a goggle frame, a lens, and components for audio functions, including a microphone and speakers.
  • the ski goggles also comprise receiving and transmitting antennas.
  • CN 215 117 044 U discloses adaptive ski goggles which comprise a photosensitive sensor and a lens that is responsive to voltage changes.
  • US 2011/0227820 A1 discloses an interactive head-mounted eyepiece with an integrated processor for handling content for display and an integrated image source for introducing the content to an optical assembly through which the wearer views a surrounding environment and the displayed content.
  • the displayed content may comprise a local advertisement wherein the location of the eyepiece is determined by an integrated location sensor.
  • the eyepiece comprises arm portions disposed on opposite sides of the eyepiece.
  • CN 209 167 686 U which serves as basis for the preamble of claim 1 , discloses augmented reality (AR) ski goggles.
  • the AR ski goggles comprise an optical display that forms a wind shielding sheet.
  • the AR ski goggles also comprise a frame structure and a communication module.
  • ski goggles do not adequately address the need for ski goggles having a simple and reliable construction that provides comfort to the wearer while accommodating the wearer’s desire for enhancing the real-world view seen through a goggle lens with additional information.
  • WO 2011/044680 A1 and WO 2013/000075 A1 disclose ski goggles comprising a display.
  • the display is arranged to extend adjacent a lens of the ski goggles and blocks part of a real-world field of view that would otherwise be visible through the ski goggles. This may adversely affect user experience.
  • the display blocks safetyrelevant features of the real-word view (such as an obstacle on a skiing slope or a recess in the skiing slope).
  • ski goggles there is a need for improved ski goggles.
  • ski goggles having a simple and reliable construction that provides comfort to the wearer while accommodating the wearer’s desire for enhancing the real-world view seen through a goggle lens with additional information.
  • a communication terminal configured to interact with such ski goggles and control methods suitable for such ski goggles.
  • the invention are ski goggles comprising: a lens; a lens frame configured to retain the lens; a functional body comprising a cavity, wherein the functional body further comprises at least one wireless interface circuit and at least one integrated circuit arranged within the cavity; and a battery body comprising one or several batteries for supplying power to the functional body.
  • the battery body and the functional body are attached to the lens frame on opposite sides of the lens frame.
  • the functional body comprises a display extending along part of the lens in a space defined by the lens frame and the lens, the display being transparent or semitransparent to output information as an overlay to a real-world view visible through the lens.
  • the functional body comprises at least one sensor arranged within the cavity.
  • the at least one integrated circuit is configured to process a sensor output of the at least one sensor and use the processed sensor output to control the display to output the information and cause a data transmission by the at least one wireless interface circuit.
  • the ski goggles enhance the wearer’s experience by providing augmented reality (AR) functionality.
  • Information that is derived at least from a sensor output is used for controlling a display, making it possible to blend information relating to the environment of the ski goggles with a real-world view visible through the lens.
  • the arrangement of the battery body and functional body facilitate comfort to the wearer by providing a design that balances weight distribution.
  • the balancing also contributes to enhanced safety by mitigating the risk of inadvertent dislodgment of the ski goggles during continued use (which might otherwise occur when the ski goggles are subject to force pulses, as is the case when skiing on uneven terrain).
  • a transparent or semitransparent display that blends information relating to the environment of the ski goggles with a real-world view visible through the lens as an overlay, safety and comfort are enhanced further.
  • the display does not block part of the real-world view that would otherwise be visible through the lens.
  • a functional body that comprises such a display, robust connections of the display to the circuitry that processes the sensor output is attained.
  • the risk of malfunction of the display is mitigated as compared to arrangement in which the display is attached by means of a mounting structure that projects from a portion of the lens frame other than the functional body on a side of the lens frame. In the latter case, the additional connections required to extend along the lens frame and the mounting structure can increase the risk of display malfunction.
  • the processed sensor data By using the processed sensor data to not only control the display to output the information but also to cause a data transmission, safety and comfort increased further. Critical situations or other situations warranting the data transmission may be identified by the circuitry of the functional body and may be used to trigger the data transmission.
  • the at least one integrated circuit is configured to use data received via the wireless interface circuit to control the display.
  • information received via the wireless interface circuit can be overlaid onto the real-world view.
  • the information received via the wireless interface circuit can be communication-related information (such as information on incoming calls, voice messages, video messages, or text messages) and/or social networking information (such as information on status updates).
  • the information received via the wireless interface circuit may also include information related to skiing, such as avalanche warnings, slope difficulty rating, and/or snow condition information, which may be retrieved by the ski goggles using the processed sensor output (e.g., a geolocation derived from a Global Navigation Satellite System, GNSS, e.g., derived from a global positioning system, GPS, sensor output).
  • GNSS Global Navigation Satellite System
  • GPS global positioning system
  • the at least one sensor is configured to determine a three- dimensional, 3D, orientation and/or 3D acceleration of the functional body, wherein the at least one integrated circuit is configured to use the sensor output to determine a velocity of the ski goggles as the information.
  • the data transmission may comprise the transmission of the determined velocity.
  • information of particular interest to the wearer such as a velocity during skiing, may be output as overlay on the real-world view.
  • the at least one sensor comprises a global navigation satellite system (GNSS) receiver and the at least one integrated circuit is configured to process a GNSS receiver output.
  • GNSS global navigation satellite system
  • Such a configuration facilitates the determination of geolocation-related information, such as an altitude, for displaying.
  • This configuration also facilitates the retrieval of information related to the geolocation, such as weather conditions, slope difficulty, avalanche risk rating, by querying information services to which the ski goggles may be communicatively interfaced via a cellular or other wireless network.
  • the at least one integrated circuit is configured to use the processed GNSS receiver output to control the display to output the processed GNSS receiver output.
  • This allows location-dependent information to be displayed for viewing.
  • the location-dependent information may include an altitude which may be of particular interest to the wearer of the ski goggles.
  • the at least one integrated circuit is additionally or alternatively configured to use the processed GNSS receiver output to control the at least one wireless interface circuit such that the data transmission includes the processed GNSS receiver output.
  • location-dependent information such as geotagged status information, geo-tagged audio and/or video data, geo-tagged social networking information
  • the at least one integrated circuit is additionally or alternatively configured to use the processed GNSS receiver output to control the at least one wireless interface circuit to perform a data retrieval.
  • servers such as weather forecast, avalanche service, or other servers
  • the processed GNSS receiver output comprises altitude information.
  • the at least one sensor may comprise at least one microelectromechanical system operative as one, several, or all of: an inertial measurement unit, a magnetometer, a barometer.
  • the sensor at least one sensor may comprise different microelectromechanical systems (MEMS) sensors as IMU (inertial measurement unit), magnetometer and barometer [0033]
  • MEMS microelectromechanical systems
  • the at least one integrated circuit is configured to use the processed sensor output to perform safety-related functions.
  • the safety-related functions comprise controlling the display to output a hazard warning or risk assessment.
  • the hazard warning or risk assessment may comprise an avalanche risk and/or a slope difficulty rating retrieved via the at least one wireless interface circuit using the processed sensor output.
  • the safety-related functions comprise controlling the at least one wireless interface circuit to transmit an emergency signal.
  • the at least one integrated circuit may perform emergency detection based on the processed sensor output.
  • the ski goggles comprise a camera arranged on the lens frame, the at least one integrated circuit being configured to receive image frames captured by the camera.
  • the camera has a camera objective and the lens has a recess or aperture into which the camera objective projects.
  • the camera objective may be flush with a surface of the lens facing away from the wearer’s face in use.
  • the integrated circuit is configured to control the wireless interface circuit such that the data transmission includes at least some of the image frames.
  • This configuration facilitates the sharing of visual data, for remote analysis, sharing, storing, or other purposes.
  • the integrated circuit is configured to process the image frames and use the processed image frames to control the display to output the information.
  • This configuration facilitates the integration of displayed information into the real-world view visible through the lens.
  • information derived from the captured image frames may be generated and output.
  • the ski goggles comprise an audio interface.
  • the audio interface may comprise several transducers.
  • the audio interface may comprise one or several microphones and one or several speakers. Drive circuit(s) for the microphones and/or speakers may be integrated into the functional body.
  • the ski goggles may comprise a strap for securing the ski goggles to a head.
  • the functional body may comprise a first passage for passing the strap through the first passage along the functional body.
  • the battery body may comprise a second passage for passing the strap through the second passage along the battery body.
  • the lens frame may comprise an opening for allowing the display of the functional body to enter the space defined by the lens frame and the lens.
  • This configuration provides a simple way of arranging the display in the interior of the space defined by the lens frame and the lens, without requiring control electronics or other display control components to be arranged on the lens frame.
  • the lens frame may comprise conductors (such as wires or conductive traces) that provide a conductive path between the battery or batteries of the battery body and the functional body.
  • conductors such as wires or conductive traces
  • This configuration allows the functional body to be powered while facilitating the implementation of a weight balanced ski goggle design.
  • the battery body and functional body may be movably attached to the lens frame.
  • This configuration facilitates adaptation of the ski goggles to different head sizes and/or head shapes and/or helmet sizes and/or helmet shapes.
  • the battery body and functional body may be attached to the lens frame via a soft linking material.
  • the soft linking material may implement a living hinge.
  • the battery body and functional body may be attached to the lens frame via a living hinge, which may be implemented by a soft linking material.
  • This configuration facilitates adaptation of the ski goggles to different head sizes and/or head shapes and/or helmet sizes and/or helmet shapes.
  • the functional body and/or the battery body may respectively comprise one or several mechanically actuable buttons or keys that are arranged on an end of the functional body and/or battery body adjacent the lens.
  • the mechanically actuable allow the wearer to perform various frequently used activities (such as accepting a call, speaker volume adjustment, dialing, etc.) to be performed in a simple and intuitive manner.
  • the lens may be attached to the lens frame in a reversibly releasable manner.
  • the lens may be configured such that it attenuates but does not block the blue wavelength range.
  • a wireless communication terminal comprising a wireless communication chipset configured to perform bi-directional data communication with the ski goggles according to any aspect or embodiment; a terminal display; and at least one processor configured to process the data transmission from the ski goggles and use the processed data transmission to perform activity tracking and control the terminal display to output results of the activity tracking.
  • the wireless communication terminal facilitates monitoring of a skiing activity by virtue of its communicative coupling with the ski goggles and by virtue of its activity tracking function
  • the wireless communication terminal can also serve as a communication hub configured to relay data, audio, and/or video communication to and from the ski goggles.
  • the activity tracking comprises using the processed data transmission to track an altitude and/or a velocity of the ski goggles as a function of time.
  • the wearer of the ski goggles and/or another user of the communication terminal is/are enabled to review the results of the activity tracking.
  • the wireless communication terminal is configured to use the processed data transmission to retrieve, via a cellular network or wireless local area network, an avalanche risk and/or a slope difficulty rating for a geolocation defined by the data transmission from the ski goggles and transmit the avalanche risk and/or the slope difficulty rating to the ski goggles.
  • the wireless communication terminal assists the ski goggles in retrieving safety-related information, while obviating the need for the ski goggles to have a cellular communication interface circuit.
  • the wireless communication terminal is configured to relay audio and/or video data to and from the ski goggles.
  • the wireless communication terminal can act as a communication hub for communication to and/or from the ski goggles.
  • the mobile communication terminal and/or the ski goggles may be operative to enable a usage mode, in particular a skiing mode, to be input.
  • the usage mode may be selected from a pre-defined group, e.g., a pre-defined group comprising or consisting of skiing on designated slopes, skiing on pathways that are not limited to designated slopes, and skiing in a ski park. [0075] This allows operation to be tailored to the scenario in which the user uses the ski goggles.
  • the mobile communication terminal and/or the ski goggles may be operative to use the processed sensor data in a manner which is dependent on the usage mode input by the user. This may comprise aggregating different physical parameters, generated from the processed sensor output, depending on which usage mode has been input by the user.
  • the mobile communication terminal and/or the ski goggles may be operative to output different possible selections for the usage mode.
  • the mobile communication terminal and/or the ski goggles may be operative to enable a skill level to be input.
  • the skill level may be selected from a pre-defined group, e.g., beginner, intermediate, advanced.
  • the mobile communication terminal and/or the ski goggles may be operative to perform a navigation task.
  • the navigation task may comprise a route search and/or route guidance.
  • a target of the route search and/or route guidance may have stationary geo-coordinates (such as a point of interest) or may be movable (such as another skier).
  • the mobile communication terminal and/or the ski goggles may be operative to retrieve the coordinates of a movable target from other ski goggles according to an embodiment.
  • the user of the ski goggles receives assistance for the task of finding a good route, which is particularly challenging when the target of the route search or route guidance is movable.
  • the mobile communication terminal and/or the ski goggles may be operative to perform the navigation task based on the input usage mode and/or skill level. [0085] This configuration allows the user's skills and desired usage (on designated slopes/off designated slopes) to be taken into consideration when performing the navigation task.
  • the mobile communication terminal and/or the ski goggles may be operative to aggregate activity-related data based on the processed sensor output.
  • This configuration allows relevant physical parameters to be aggregated over a usage session (e.g., during a skiing day).
  • the mobile communication terminal and/or the ski goggles may be operative to generate output that depends on the aggregated activity-related data.
  • the mobile communication terminal and/or the ski goggles may be operative such that the data that are being aggregated and/or the generated output is dependent on the input usage mode and/or skill level.
  • the mobile communication terminal and/or the ski goggles may be operative to enable groups of users to be defined.
  • This configuration enables a set of trusted users to be established with which communication and/or activity-related data (such as position information or information derived therefrom) may be shared.
  • the mobile communication terminal and/or the ski goggles may be operative to enable communication functions among the users belonging to a group.
  • This configuration enables a set of trusted users to be established with which communication exchange may be implemented.
  • the mobile communication terminal and/or the ski goggles may be operative to enable sharing of data (such as text or voice communication, images, and/or activity- related data).
  • data such as text or voice communication, images, and/or activity- related data.
  • the mobile communication terminal and/or the ski goggles may be operative to restrict access to the shared data, e.g., to member of a previously defined group.
  • the mobile communication terminal and/or the ski goggles may be operative to enable generation of an information panel (which may include any one or any combination of text or voice communication, images, activity-related data).
  • an information panel which may include any one or any combination of text or voice communication, images, activity-related data.
  • This configuration facilitates information sharing.
  • the mobile communication terminal and/or the ski goggles may be operative to identify persons and/or landmarks in proximity to the user of the ski goggles and to output the information on the identified persons and/or landmarks via the display of the ski goggles or via the mobile communication terminal.
  • the mobile communication terminal and/or the ski goggles may be operative to execute Al-based processing for performing at least one of its functions (such as identification of POIs, identification of terrain features, identification of designated ski slopes, navigation function).
  • Al-based processing for performing at least one of its functions (such as identification of POIs, identification of terrain features, identification of designated ski slopes, navigation function).
  • the processing may be implemented in a data-driven manner.
  • This configuration increases reliability and, thus, safety.
  • the Al-based processing may comprise executing at least one Al model, wherein the at least one Al model has an input operative to receive the processed sensor data and at least one image captured by a camera of the ski goggles.
  • the processing may be implemented in a data-driven manner. This configuration increases reliability and, thus, safety.
  • the input of the at least one Al model may be operative to receive GNSS- based data and the at least one image.
  • the processing may be implemented in a data-driven manner.
  • This configuration increases reliability and, thus, safety.
  • the at least one Al model may have an output operative to provide a classification result (e.g., for a slope difficulty level, an avalanche risk, presence or absence of a designated slope) and/or an identification result (e.g., for a contact recognized based on image recognition techniques).
  • the classification result may comprise a pixelwise classification of areas identified to be hazardous.
  • the mobile communication terminal and/or the ski goggles may be operative to generate output for displaying via the display of the ski goggles and/or via the mobile communication result, based on the Al model output.
  • the results obtained by using a data-driven processing implemented by the Al model may be harnessed for ski goggle operation and/or wireless communication terminal operation.
  • the at least one Al model may comprise a recurrent neural network (RNN) and/or a convolutional neural network (CNN).
  • RNN recurrent neural network
  • CNN convolutional neural network
  • Such Al models are particularly suitable for processing the combination of images and other processed sensor output.
  • the Al-based processing may comprise executing at least one Al model, wherein the at least one Al model has an input operative to receive a time series of the processed sensor data.
  • This configuration allows time-series data to be processed to identify situations that are of relevance to user safety. Safety and user comfort are enhanced thereby.
  • the input of the at least one Al model may be operative to receive a time series of velocity and/or acceleration values obtained from the processed sensor output. [00118] Such time-series data are particularly relevant to identify potentially hazardous situations that are of relevance to user safety. Safety and user comfort are enhanced thereby.
  • the at least one Al model may have an output operative to provide a classification result that may comprise safety-related information, such as a classification result on whether the user of the ski goggles is in a hazardous situation (e.g., an accident or avalanche situation that warrants automatic triggering of an emergency signal transmission).
  • a classification result may comprise safety-related information, such as a classification result on whether the user of the ski goggles is in a hazardous situation (e.g., an accident or avalanche situation that warrants automatic triggering of an emergency signal transmission).
  • This configuration allows time-series data to be processed to identify situations that are of relevance to user safety. Safety and user comfort are enhanced thereby.
  • the at least one Al model may comprise gated recurrent units (GRUs) and/or long short term memory (LSTM) cells.
  • GRUs gated recurrent units
  • LSTM long short term memory
  • Such structures are particularly suitable for processing time series data.
  • the mobile communication terminal and/or the ski goggles may be operative to generate the data transmission based on the Al model output.
  • data communication may be triggered automatically, based on a result of processing that is performed by the Al model, i.e. , in a data-driven manner.
  • machine- readable instruction code comprising instructions which, when executed by at least one processing circuit, cause the at least one processing circuit to process data captured by the ski goggles according to an aspect or embodiment and generate output based on a result of the processing.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to enable a usage mode, in particular a skiing mode, to be input.
  • the usage mode may be selected from a pre-defined group, e.g., a pre-defined group comprising or consisting of skiing on designated slopes, skiing on pathways that are not limited to designated slopes, and skiing in a ski park. [00127] This allows operation to be tailored to the scenario in which the user uses the ski goggles.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to use the processed sensor data in a manner which is dependent on the usage mode input by the user. This may comprise aggregating different physical parameters, generated from the processed sensor output, depending on which usage mode has been input by the user.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to output different possible selections for the usage mode.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to enable a skill level to be input.
  • the skill level may be selected from a pre-defined group, e.g., beginner, intermediate, advanced.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to perform a navigation task.
  • the navigation task may comprise a route search and/or route guidance.
  • a target of the route search and/or route guidance may have stationary geo-coordinates (such as a point of interest) or may be movable (such as another skier).
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to retrieve the coordinates of a movable target from other ski goggles according to an embodiment. [00135] Thereby, the user of the ski goggles receives assistance for the task of finding a good route, which is particularly challenging when the target of the route search or route guidance is movable.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to perform the navigation task based on the input usage mode and/or skill level.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to aggregate activity-related data based on the processed sensor output.
  • This configuration allows relevant physical parameters to be aggregated over a usage session (e.g., during a skiing day).
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to generate output that depends on the aggregated activity-related data.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit such that the data that are being aggregated and/or the generated output is dependent on the input usage mode and/or skill level.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to enable groups of users to be defined. [00145] This configuration enables a set of trusted users to be established with which communication and/or activity-related data (such as position information or information derived therefrom) may be shared.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to enable communication functions among the users belonging to a group.
  • This configuration enables a set of trusted users to be established with which communication exchange may be implemented.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to enable sharing of data (such as text or voice communication, images, and/or activity-related data).
  • This configuration enables a set of trusted users to be established with which communication data may be shared.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to restrict access to the shared data, e.g., to member of a previously defined group.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to enable generation of an information panel (which may include any one or any combination of text or voice communication, images, activity-related data).
  • This configuration facilitates information sharing.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to identify persons and/or landmarks in proximity to the user of the ski goggles and to output the information on the identified persons and/or landmarks via the display of the ski goggles or via the mobile communication terminal. [00155] Thereby, information is automatically determined which is of particular relevance for the functions performed by the mobile communication terminal and/or the ski goggles.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to execute Al-based processing for performing at least one of its functions (such as identification of POIs, identification of terrain features, identification of designated ski slopes, navigation function).
  • the processing may be implemented in a data-driven manner.
  • This configuration increases reliability and, thus, safety.
  • the Al-based processing may comprise executing at least one Al model, wherein the at least one Al model has an input operative to receive the processed sensor data and at least one image captured by a camera of the ski goggles.
  • the processing may be implemented in a data-driven manner.
  • This configuration increases reliability and, thus, safety.
  • the input of the at least one Al model may be operative to receive GNSS- based data and the at least one image.
  • the at least one Al model may have an output operative to provide a classification result (e.g., for a slope difficulty level, an avalanche risk, presence or absence of a designated slope) and/or an identification result (e.g., for a contact recognized based on image recognition techniques).
  • the classification result may comprise a pixelwise classification of areas identified to be hazardous.
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to generate output for displaying via the display of the ski goggles and/or via the mobile communication result, based on the Al model output.
  • the results obtained by using a data-driven processing implemented by the Al model may be harnessed for ski goggle operation and/or wireless communication terminal operation.
  • the at least one Al model may comprise a recurrent neural network (RNN) and/or a convolutional neural network (CNN).
  • RNN recurrent neural network
  • CNN convolutional neural network
  • Such Al models are particularly suitable for processing the combination of images and other processed sensor output.
  • the Al-based processing may comprise executing at least one Al model, wherein the at least one Al model has an input operative to receive a time series of the processed sensor data.
  • This configuration allows time-series data to be processed to identify situations that are of relevance to user safety. Safety and user comfort are enhanced thereby.
  • the input of the at least one Al model may be operative to receive a time series of velocity and/or acceleration values obtained from the processed sensor output.
  • Such time-series data are particularly relevant to identify potentially hazardous situations that are of relevance to user safety. Safety and user comfort are enhanced thereby.
  • the at least one Al model may have an output operative to provide a classification result that may comprise safety-related information, such as a classification result on whether the user of the ski goggles is in a hazardous situation (e.g., an accident or avalanche situation that warrants automatic triggering of an emergency signal transmission).
  • a classification result may comprise safety-related information, such as a classification result on whether the user of the ski goggles is in a hazardous situation (e.g., an accident or avalanche situation that warrants automatic triggering of an emergency signal transmission).
  • This configuration allows time-series data to be processed to identify situations that are of relevance to user safety. Safety and user comfort are enhanced thereby.
  • the at least one Al model may comprise gated recurrent units (GRUs) and/or long short term memory (LSTM) cells.
  • GRUs gated recurrent units
  • LSTM long short term memory
  • the machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to generate the data transmission based on the Al model output.
  • data communication may be triggered automatically, based on a result of processing that is performed by the Al model, i.e. , in a data-driven manner.
  • a non-transitory storage medium having stored thereon the machine-readable instruction code according to an aspect or embodiment of the invention.
  • a system comprising the wireless communication terminal according to an embodiment and the ski goggles according to an embodiment.
  • a method of controlling a display of ski goggles wherein the display is comprised by a functional body attached to a lens frame of the ski goggles that retains a lens, the display being transparent or semitransparent and extending in a space defined by the lens frame and the lens.
  • the method comprises the following steps: receiving, by at least one integrated circuit arranged in a cavity of the functional body, a sensor output of at least one sensor arranged in the cavity of the functional body; processing, by the at least one integrated circuit, the sensor output; and sing, by the at least one integrated circuit, the processed sensor output to control the display to output information as an overlay to a real-world view visible through the lens.
  • the method enhances the wearer’s experience by providing augmented reality (AR) functionality.
  • Information that is derived at least from a sensor output is used for controlling a display, making it possible to blend information relating to the environment of the ski goggles with a real-world view visible through the lens.
  • the arrangement of the battery body and functional body facilitate comfort to the wearer by providing a design that balances weight distribution.
  • the ski goggles may be the ski goggles according to any embodiment disclosed herein.
  • the method may be performed automatically by the ski goggles or system according to any embodiment disclosed herein.
  • Another aspect of the invention is directed to the use of the ski goggles described herein together with a functional application provided on the wireless communication terminal described herein for tracking and/or displaying on the ski goggles display and/or on the terminal display weather information, device information, ski mode choice information, live map navigation information, reference point information, area recognition information, character recognition information, sports data information, SOS emergency call information and/or danger warning information (i.e. as specified further below).
  • ski goggles, communication terminal, system, and method according to the invention are described in more detail hereinbelow by way of an exemplary embodiment and with reference to the attached drawings, in which:
  • Fig. 1 shows a perspective view of ski goggles according to the invention
  • Fig. 2 shows another perspective view of the ski goggles of Fig. 1 ;
  • Fig. 3 shows yet another perspective view of the ski goggles of Fig. 1 ;
  • Fig. 4 shows a detailed view of a light waveguide display (AR display) of the inventive ski goggles
  • Fig. 5 shows a block diagram of a functional body of the ski goggles of Fig. 1 ;
  • Fig. 6 shows a system comprising the ski goggles of Fig. 1 and a user terminal according to the invention
  • Figs. 7-18 show as screenshots from the AR display the functionalities of an APP configured for the wireless communication terminal for use with the inventive ski goggles for activity tracking and information displaying on particularly the AR display;
  • Figs. 19-33 show respective screenshots from the APP on the terminal display
  • Fig. 34 shows a system comprising ski goggles according to the invention
  • Fig. 35 shows an artificial intelligence-based processing of data captured using the ski goggles according to the invention
  • Fig. 36 shows another artificial intelligence-based processing of data captured using the ski goggles according to the invention
  • Fig. 37 shows another artificial intelligence-based processing of data captured using the ski goggles according to the invention.
  • Fig. 38 shows a flow chart of a method performed by the system of Fig. 34;
  • Fig. 39 shows a block diagram of a specific implementation of the functional body of the ski goggles of Fig. 1 ;
  • Fig. 40 shows a block diagram of another specific implementation of the functional body of the ski goggles of Fig. 1 ;
  • Fig. 41 shows a block diagram of a yet another specific implementation of the functional body of the ski goggles of Fig. 1 .
  • APP is used to refer to machine-readable instruction code, without implying any limitations to a particular operating system and/or computing framework.
  • the APP may be executed on a mobile device, in particular a mobile communication terminal, without being limited thereto.
  • Fig. 1 , Fig. 2, and Fig. 3 are perspective views of ski goggles 10 according to the invention.
  • the ski goggles 10 comprise a lens 11 , a lens frame 20 that retains the lens 11 , a functional body 30, and a battery body 40.
  • the functional body 30 may be a functional unit.
  • the battery body 40 may be a battery unit. Any reference to a functional/battery “body” may be considered to encompass or be a reference to a functional/battery “unit.”
  • the functional/battery unit may respectively be a component that is attached to the lens 11 on a minor side thereof as an integral unit during manufacture. I.e. , the functional/batter unit may be formed and only then attached to the lens 11 .
  • the battery body 40 includes one or several batteries.
  • the battery or batteries may include one or several rechargeable batteries, in particular lithium ion batteries.
  • a battery housing 42 of the battery body 40 may be configured to be coupled to a connector portion 43 of the battery body 40 in a reversibly releasable manner.
  • the battery housing 42 may be the housing of the one or several batteries. Conductive pathways are provided in the lens frame 20 to supply components of the functional body 40 with energy.
  • the functional body 30 comprises one or several sensors.
  • the one or several sensors may include sensors configured to sense an orientation, position, and/or acceleration of the ski glasses 10.
  • the one or more sensors comprise a sensor (e.g., a six-axis gyroscope) configured to sense a three-dimensional orientation of the ski goggles 10 and a three-dimensional acceleration of the ski goggles 10.
  • the three-dimensional orientation may be defined by three angles (e.g., three Euler angles) defining the orientation relative to a reference frame.
  • the three-dimensional acceleration may be defined by three acceleration values defining the acceleration along, e.g., three coordinate axes.
  • the functional body 30 comprises one or several integrated circuit(s) (IC(s)).
  • the one or several IC(s) may comprise one or several application specific integrated circuit(s) (ASIC(s)), controller(s), processor(s), field programmable gate array(s) (FPGA(s)), or combinations thereof.
  • ASIC(s) application specific integrated circuit
  • controller(s) controller(s)
  • processor(s) field programmable gate array
  • FPGA(s) field programmable gate array
  • the IC(s) are configured to control a display 51 of the functional body 30.
  • the display 51 extends through a lateral passage of the frame 20 into a space defined by the frame 20 and the lens 11.
  • the display 51 is arranged such that it remains spaced from a contact area defined by an abutment rim 24 on the frame 20.
  • the display 51 is a transparent or semitransparent display. By virtue of this configuration, information output via the display 51 are visible as an overlay to a real- world scene that can be seen through the lens 11 , as illustrated in Fig. 4.
  • the IC(s) control the display 51 to output information.
  • the information depends at least on sensor output of the one or several sensors, processed by the one or several IC(s).
  • the information may include information relevant for skiing, such as a velocity, snow condition information, slope difficulty rating, etc.
  • This information may include data derived by processing the sensor output and/or data retrieved via a communication interface circuit of the functional body 40.
  • velocity information may be derived by the IC(s) by integrating accelerometer measurements.
  • Altitude may be derived by the IC(s) by processing a global navigation satellite system (GNSS) receiver output.
  • GNSS global navigation satellite system
  • Snow condition information may be derived from a temperature sensor output or may be obtained by querying a remote server, using a location derived from a GNSS receiver output.
  • Slope difficulty rating may be obtained by querying a remote server, using a location derived from a GNSS receiver output. All these determination steps may be performed automatically by the ski goggles 10 and, more particularly, using the processing capability of the one or several IC(s).
  • the functional body 30 may comprise one or several wireless interface circuit(s).
  • the functional body 30 may comprise a Bluetooth (BT) and/or wireless local area network (WLAN) interface, e.g., an IEEE 802.11 compliant interface.
  • the wireless interface circuit(s) may be configured to transmit, under the control of the IC(s), results of processing a sensor output to a wireless communication terminal.
  • the wireless interface circuit(s) may be configured to retrieve, using the results of the sensor output processing (e.g., a location derived thereby), information for displaying via the display 51.
  • the ski goggles 10 comprise transducers that allow the ski goggles 10 to play audio signals and to receive and process voice signals. For illustration, microphones and speakers may be integrated into the ski goggles 10.
  • the wireless communication interface(s) of the ski goggles 10 allow audio and/or video signals to be relayed between the ski goggles 10 and a wireless communication terminal. This improves convenience for accepting incoming calls and/or establishing outgoing calls while skiing.
  • a camera having a camera objective 15 may be provided on the lens frame 20.
  • the lens 11 has an aperture or recess 14 that is positioned such that, when the lens 11 is retained by the lens frame 20, the camera objective 15 is positioned at the aperture or recess 14, leaving the camera objective 15 unobstructed by the lens.
  • the lens frame 20 has image data paths that allow the IC(s) of the functional body to receive image frames captured by the camera. At least some of the image frames may be transmitted to a wireless communication terminal via the wireless interface circuit(s).
  • the battery body 40 and functional body 30 are provided on opposing lateral sides of the lens frame 20.
  • the functional body 30 is attached to a first side 21 of the lens frame 20.
  • the battery body 40 is attached to a second side 22 of the lens frame 20, which is opposite the first side 21.
  • the first side and the second side 22 may be the minor (i.e. , shorter) sides of the lens frame 20 that extend to the left and right when the goggles are worn on a head with horizontal facing direction.
  • the attachment of the functional body 30 to the lens frame 20 may allow movement (e.g., slight tilting or slight displacement) of the functional body 30 relative to the lens frame 20.
  • the attachment of the battery body 40 to the lens frame 20 may allow movement (e.g., slight tilting or slight displacement) of the battery body 40 relative to the lens frame 20.
  • Different head sizes and/or head shapes and/or helmet sizes and/or helmet shapes can be more easily accommodated thereby, without adverse effects on wearer comfort.
  • the soft linking material may be a plastic material.
  • the soft linking material may be a polymeric material.
  • the linking material may implement a living hinge that accommodates the relative movement of the functional body 30 and/or battery body 40 relative to the lens frame 20.
  • Attachment of the functional body 30 and of the battery body 40 to the lens frame 20 may be attained by means of injection molding, ultrasonic welding, or other techniques.
  • the lens frame 20 may have first and second attachment features on the first and second sides 21 , 22, respectively, to facilitate attachment of the functional body 30 and battery body 40.
  • the lens frame 20 may have a stepped recess on which the functional body 30 can be attached to the lens frame 20. Suitable mating engagement means ensure the desired degree of movability to accommodate different head sizes and/or shapes.
  • the coupling of the lens frame 20 to the battery body 40 ensures a conductive pathway from an output terminal of the battery body 40 to the conductors of the frame that supply power to the functional body 30 and camera.
  • the functional body 30 is coupled to the lens frame 20 so as to ensure a conductive pathway between the battery body 40 and the electronic components of the functional body 30.
  • the functional body 30 is coupled to the lens frame 20 so as to ensure a data communication pathway between the camera and the IC(s) 52 of the functional body 30.
  • the functional body 30 may comprise a guide portion 39 for guiding a strap 13 of the ski goggles on and along the functional body 30.
  • the battery body 40 may comprise another guide portion 49 for guiding the strap 13 of the ski goggles 10 on and along the battery body 40.
  • the guide portions 39, 49 may respectively define a passage that allows the strap 13 to pass therethrough.
  • the functional body 30 may comprise one or several hard keys 31 disposed on an outer surface thereof, e.g., at a forward end of the outer surface adjacent the lens 11. Additionally or alternatively, the battery body 40 may comprise one or several hard keys 41 disposed on an outer surface thereof, e.g., at a forward end of the outer surface adjacent the lens 11.
  • the hard keys 31 , 41 may be assigned various frequently used functions, such as accepting an incoming call, dialing, etc.
  • the lens 11 can be retained by the lens frame 20 in a reversibly removable manner.
  • the lens 11 can be temporarily removed for, e.g., cleaning, without affecting any of the electrical connections, and can be re-attached subsequently.
  • the lens 11 can be exchanged, e.g. when needed for repair purposes.
  • the lens 11 may be configured to improve vision in snow, thereby making it easier to perform obstacle detection.
  • the lens 11 may enhance colors brightness and contrast.
  • the lens 11 may also be configured to automatically change color with sunlight intensity. Contrary to standard blue blocker filters, the lens 11 presents a specific spectral curve that acts as a blue attenuator but still allows some blue light to reach the eye, e.g., in a spectral range between 380-500 nm. This makes it easier for the wearer to read snow irregularities like bumps and hollows. Skiing and snowboarding become safer.
  • Fig. 5 is a functional block diagram of a functional body 30. It is understood that additional and/or alternative components may be used in other embodiments.
  • the functional body comprises the display 51 protruding from a wall 32.
  • the wall 32 may define a housing having a cavity 33 in which various components are housing. These components include:
  • - processor(s) or other IC(s) 52 for performing the processing and control functions described herein; one or several wireless interface circuit(s) 54, 55, such as a Bluetooth chipset 54 and/or a chipset 55 configured for communication in accordance with an IEEE 802.11 standard;
  • a gyroscope 58 which may be a three-axis gyroscope or a so-called six-axis gyroscope operative to sense both 3D orientation and 3D acceleration
  • a GNSS receiver 57 a GNSS receiver 57
  • additional interface circuits such as a display driver circuit 53 for driving the display 51 , a camera interface 56 for receiving image frames from the camera, and/or an audio circuit 59 for controlling speakers and processing microphone output signals.
  • the display 51 may be or may comprise an optical waveguide display.
  • a main body of the optical waveguide display may be embedded in or arranged on a circuit board of the functional body 30.
  • the display 51 extends over part of the lens 11 , so as to selectively display information as an overlay over part of a field of view.
  • the display driver circuit 51 may be or may comprise a light engine.
  • the light engine may be operative to generate optical signals that are being guided along the optical waveguide display until they information is output as an overlay to the real-world view. Integration of the light engine into the functional body 30 facilitates balancing of the ski goggles (and, thus, safety and comfort) and prevents non-transparent components from blocking part of the user’s field of view otherwise visible through the lens 11 .
  • the display 51 is independent of and separate from the lens 11. In particular, when the lens 11 is exchanged or temporarily removed, it is not required to disconnect any electrical connection to the display.
  • the lens 11 does not have to include any conductive traces interfaced with the battery body 40 and/or functional body 30.
  • the invention provides augmented reality (AR) intelligent ski goggles 10.
  • the AR ski goggle includes: the lens frame 20, the battery body 30 connected to the lens frame 20, and the functional body 30.
  • the functional body 30 comprises a light waveguide display module 51 , a camera module, a communication module, a GPS module, a further sensor module, a voice module, and at least one chip.
  • the light waveguide display 51 is positioned within the circumference of the lens frame 11 and connected to a circuit board of the functional body 30.
  • the camera is arranged so that the camera objective 15 is located at an upper end of the lens 11 in a laterally central position.
  • the wiring for the camera is hidden in the lens structure 20 and connected to the functional body 30.
  • the communication module 54, 55, GPS module 57, sensor unit 58, and voice module 59 are communicatively connected to the at least one chip 52.
  • the AR ski goggles 10 integrate several different interactive devices and data acquisition devices to achieve many different intelligent functions, which improve functionality for the wearer while skiing.
  • buttons 31 , 41 can be arranged on each side of the lens frame 20 for operation and use. In an embodiment, these buttons provide the following functions:
  • SOS button When a user presses this SOS button for more than a threshold time (e.g., 5 seconds), an SOS signal will broadcast to other users within a certain range (e.g., 5 km) who wear the goggles. Additionally or alternatively, an emergency number is called. This function allows the user to be found by the nearest people and professional rescue team.
  • a threshold time e.g., 5 seconds
  • an SOS signal will broadcast to other users within a certain range (e.g., 5 km) who wear the goggles. Additionally or alternatively, an emergency number is called. This function allows the user to be found by the nearest people and professional rescue team.
  • chat group button The user can create or join a chat group in app, when the user presses this chat group button, it allows the user to talk to a ski team, akin to dedicated portable communication terminals.
  • the various components of the ski goggles 10 may have the following functions: communication components 54, 55:
  • a Bluetooth module 54 e.g., Bluetooth 4.2 module
  • a WiFi e.g., a WiFi 2.4Ghz
  • a WiFi e.g., a WiFi 2.4Ghz
  • GPS module 57 used for location determination and map use for positioning
  • - sensor module 58 a six-axis gyroscope or another inertial measurement unit (IMU) (e.g., a Micro-Electro-Mechanical-Systems (MEMS)-IMU) can be used to determine the user's posture and glasses state judgment
  • IMU inertial measurement unit
  • MEMS Micro-Electro-Mechanical-Systems
  • - camera module with camera and camera interface 56 for camera image frame capture (e.g., high-definition images);
  • - display 51 Array light waveguide configured to display usage data and for data interaction
  • - voice module 59 for voice interaction.
  • - noise-resistant speaker integrated into, e.g., functional body 30 and/or battery body 40: for audio output.
  • Fig. 6 is a block diagram of a system 60.
  • the system 60 comprises the ski goggles 10 according to the invention.
  • the system 60 comprises a wireless communication terminal 70.
  • the wireless communication terminal 70 may be a cellphone, without being limited thereto.
  • the wireless communication terminal 70 may be configured for bi-directional communication with the ski goggles 10.
  • the wireless communication terminal 70 may be configured to serve as a communication hub, facilitating retrieval of skiing related data (such as avalanche risk, snow condition information, slope difficulty rating, and/or weather conditions) from one or several servers 81 , 82 via a wide area network (WAN) 80.
  • the communication terminal 70 may also enable audio, video, and/or data communication between the wearer of the ski goggles and other mobile communication terminals 78, 79 of a cellular network.
  • the communication terminal 70 comprises a wireless communication chipset 74 (which may include a Bluetooth and/or WiFi chip) configured to perform bi-directional data communication with the ski goggles 10.
  • the wireless communication terminal 70 comprises a terminal display 72.
  • the wireless communication terminal 70 comprises at least one processor 73 configured to process the data transmission from the ski goggles 10 and use the processed data transmission to perform activity tracking. Results of the activity tracking may be output via the terminal display 72.
  • the processor(s) 73 may execute machine-readable instruction code, which may be embodied in a non-transitory storage medium.
  • the machine-readable instruction code may cause the processor(s) 73 to perform the activity tracking and/or other functions.
  • the following functions may be provided by the communication terminal 70:
  • the user can review a ski or snowboarding performance via the machine-readable instruction code.
  • the user’s skill score can be ranked in, e.g., social networking scenarios and can be used for online (e.g., cloud-based) ski competition scenarios.
  • map information navigation on ski slope and interesting points, like cable lift, restaurant and friend location can be displayed.
  • the GPS module of the ski goggles provides location information.
  • a navigation routine can be executed to determine a route to the destination by calculating the shortest distance according to the wearer’s level of skill. For illustration, the navigation routine will prevent the wearer of the ski goggles from being guided to a slope that has a difficulty rating (e.g., black slope) that exceeds the wearer’s skill level.
  • the machine-readable instruction code can allow users to create a team and team chat room, it allows the team to talk through the chat room directly without calling via phone. This is particularly useful for a ski instructor and his/her student group, or for parents and their children.
  • the machine-readable instruction code can allow other users to watch the video and photo online instantly.
  • the machine-readable instruction code can allow users to publish his/her ski trip that can be viewed by other users on the event calendar, and the other users can select to join his ski plan.
  • - SOS and security The machine-readable instruction code collects the information of avalanche risk and gives notice when the user steps into the risky area. The machine-readable instruction code alerts the user in avalanche accidents. An SOS broadcast will send an emergency alert and the SOS location to other users within a certain range (e.g., 5 km), and will also call the emergency number to obtain professional help.
  • ski goggles disclosed herein as compared to conventional ski goggles comprising a display, such as those of WO 2011/044680 A1 and WO 2013/000075 A1 .
  • the ski goggles disclosed herein provide a transparent or semitransparent display.
  • the display does not block part of the field of view visible through the lens. This enhances safety and comfort.
  • the display of the present ski goggles is comprised by the functional body.
  • ski goggles are operative such that the processed sensor output is not only used to control the provision of the information via the display, but also to control the wireless interface circuit. This increases safety and comfort, as the data transmission is based (at least) on the processed sensor output, allowing it to be initiated automatically and/or to include information critical in safety-relevant applications.
  • the ski goggles 10 have additional, fewer, or alternative wireless interface circuits than described in association with the drawings;
  • the ski goggles 10 have additional or alternative sensors (e.g., in which the ski goggles 10 do not have an acceleration sensor but rely on position sensing for determining velocity).
  • Fig. 7 is an illustration of a screenshot from a light waveguide display (which is one implementation of an AR display) of the inventive AR ski goggles showing general information available in connection with the respective APP.
  • general information e.g. weather information (e.g. sunshine or cloudiness or rain/snow and the temperature), device information (e.g. date and time, user information, WI-FI Bluetooth strength, battery level, altitude and compass data).
  • the ski goggles 10 may be operative such that the display 51 is controlled to output choices of different usage modes.
  • the ski goggles 10 may be operative to enable a selection of the usage modes, e.g., by means of hard keys provided on the ski goggles and/or by means of voice input.
  • the ski goggles 10 may be operative such that different monitoring functions are performed, using camera images and/or sensor output captured by the ski goggles 10, depending on which usage mode is selected.
  • Fig. 8 shows a screenshot that represents choices available for a skiing mode, i.e. piste (i.e. regular pistes), backcountry (i.e. touring off-piste) or park (i.e. snowpark with ski or snowboard jump facilities).
  • the ski goggles 10 and/or the wireless communication terminal 70 may be operative to control a user interface (e.g., the display 51 and/or a user interface of the wireless communication terminal 70) to provide information on the available usage modes and/or enable a selection thereof.
  • the skiing goggles 10 and/or the wireless communication terminal 70 may be operative to record parameters (such as location traces and/or velocity traces and/or other parameters) derived from the processed sensor output, with the parameters that are being recorded being dependent on the selected usage mode. At least some of the parameters may be shown on the display 51 as overlay on the real-world view.
  • the skiing goggles 10 and/or the wireless communication terminal 70 may be operative to perform route guidance and/or navigation functions, with the display 51 being controlled to output information relating to route guidance or navigation.
  • the route guidance or navigation may be performed in relation to objects that are geo-stationary (such as landmarks or other points of interest (POIs)) or in relation to objects that can move themselves.
  • POIs points of interest
  • An example for the latter scenario comprises navigation or route guidance for directing a user of the ski goggles 10 to another person who has agreed to share his/her geocoordinates with the user of the ski goggles 10.
  • the routing and navigation may be performed in a manner which takes into account the routes accessible to a skier, optionally also the selected skiing mode and/or skill levels.
  • the routing and navigation may be implemented in such a manner that a user of the ski goggles 10 is guided to a target (which may be stationary, such as a landmark or POI, or movable, such as a fellow skier) in a manner which ensures that the pathway does not require skiing on pathways not compatible with the user’s skill level and/or which respect the selected usage mode.
  • routing and navigation may be implemented in such a manner that the user of the ski goggles 10 is not guided along pathways that are not in accordance with the selected usage mode. Routing and navigation may be performed using techniques known from the field of vehicle navigation (but applied to a directed graph network that represents pathways accessible to skiers), such as Dijkstra or A* route search techniques. Conformity with the selected usage mode and/or user skill levels can be ensured by either hard avoid options (eliminating parts of the network representing the skiing paths) and/or by imposing high costs (in the sense of a cost function optimized in the route search) for graph links that are not in conformity with the selected usage mode and/or user skill level.
  • Figs. 9 to 14 are screenshots that relate to a map navigation functions.
  • GNSS e.g., the GPS
  • the other sensors such as gyroscope and/or MEMS IMU
  • Fig. 9 shows an exemplary display graphics for navigation in real time, immersive navigation in real time and speed in real time.
  • the ski goggles may be operative to update the output information (i.e. , the content of what is being displayed) in an ongoing basis (i.e., continually, e.g., with a repetition interval that may be fixed or variable).
  • the real time position of the ski comrades and their current speed can be shown on the map, e.g., on the a right or left AR side portion of the display 51.
  • the real time positions may be displayed using their GNSS (e.g., GPS)- based locations, which may be communicatively shared between the various ski goggles, either directly or using communication terminals.
  • GNSS e.g., GPS
  • Fig. 11 is a screenshot that shows how the ski goggles 10 are operative to assist the user to navigate to the position of a friend (in the sense of a contact who has agreed to share GNSS coordinates with the user of the ski goggles 10).
  • the ski goggles 10 enable the user to actuate a navigation symbol (on the AR display or the terminal display) in order to create a live navigation in order to follow a track of the ski comrade (middle and lower screen shots).
  • the navigation may be coordinated with the skiing skills of the ski driver in order to create a track suited for him/her which e.g. does not guide a beginner to pistes for advanced ski drivers.
  • the ski goggles 10 may also allow the user to choose the telephone or video call symbol on the display 51 of the ski goggles or of the wireless communication terminal 70 in order to make live telephone or video calls.
  • Fig. 12 is a screenshot that shows how points of interest (POIs), such as reference points or other landmarks, are displayed.
  • the ski goggles 10 and/or the wireless communication terminal 70 may allow the user to choose on the AR or the APP map (i.e. on the wireless communication terminal - which may generally display the same information as the AR display) a point of interest, such as a restaurant, a cable car or a first aid station in order to show the location and the opening times of the point of reference as well as a real time navigation track to the respective destination.
  • a point of interest such as a restaurant, a cable car or a first aid station in order to show the location and the opening times of the point of reference as well as a real time navigation track to the respective destination.
  • Fig. 13 is a screenshot that shows the functionality of area recognition.
  • the user of the ski goggles 10 directs the camera of the ski goggles 10 to a landmark, information regarding the landmark may be shown on the display.
  • the display For example, if the user of the ski goggles 10 directs the camera onto a mountain, the name of the mountain will be shown on the AR display 51 of the ski goggles and/or the display of the communication terminal 60.
  • the ski goggles 10 and/or the display of the communication terminal 60 may be operative to display the name of the latter on the AR display 51 and/or the display of the communication terminal.
  • GNSS e.g., GPS
  • Al artificial intelligence
  • the ski goggles 10, optionally in combination with the wireless communication terminal 70 and/or a computing system remote from the ski goggles 10, may be operative to perform image processing tasks.
  • Al-based techniques may be used. For illustration, person recognition and/or landmark recognition and/or slope difficulty determination may be performed, using GNSS sensor data (such as GNSS-based coordinates) and/or image data as input of the processing.
  • the output of the processing may specify an identifier for a person and/or landmark and/or slope difficulty, which may be used to control the display 51.
  • direct communication between ski goggles 10 according to the invention may be used to identify users in proximity to each other and display information relating to such users on the display 51 .
  • Fig. 14 is a screenshot that shows the functionality of AR related recognition of users of ski goggles 10.
  • GNSS e.g., GPS
  • the recognition from ski goggles to ski goggles the user names or other identifiers of other users may be identified and displayed.
  • the ski goggles 10 may be operative to collect and aggregate data relating to the activity performed.
  • the parameters collected and aggregated may depend on the selected usage mode if the ski goggles 10 allow different usage modes to be selected.
  • the ski goggles 10 may use the processed sensor output to determine the activity- related parameters.
  • the parameters may be stored in the ski goggles 10 and/or the wireless communication terminal 70 and/or a computing system separate therefrom.
  • Figs. 15 and 16 are screenshots that are directed to such a further functionality of displaying sports data.
  • the ski goggles 10 is provided with a sensor which enables to track and collect the sports data of the user of the ski goggles 10. Such data may comprise, for example, speed, maximum speed, altimeters, skiing kilometers and so forth.
  • the ski goggles 10 may be operative to transmit any one or any combination of these parameter to the AR display after each skiing activity (e.g., downhill along a ski slope).
  • the sports-related movement data may be divided with respect to the ski mode into piste, backcountry and park. I.e., the aggregation may be performed separately for each usage mode.
  • Fig. 16 illustrates the skiing data in the park mode, i.e. displaying for example a jump length, a jump height, a flight time and a flip (in degrees).
  • the type of parameters that are recorded may be dependent on the selected usage mode.
  • the ski goggles 10 and/or APP executed on the wireless communication terminal 70 may be operative to perform safety-relevant functions.
  • the ski goggles 10 and/or APP executed on the wireless communication terminal 70 may be operative to determine, responsive to a user input and/or responsive to the processed sensor output, whether a safety-relevant function (such as triggering an emergency call) is to be performed.
  • the safety-related functions may comprise the provision of danger warnings.
  • Fig. 17 is a screenshot that is directed to a functionality of an SOS emergency call. If the user presses an SOS button on the ski goggles (or on the AR display or the terminal display) and holds down the latter, an emergency call is sent to persons in the vicinity of a predetermined distance (e.g. 3 kilometers) wearing the same ski goggles and who can then decide whether it is necessary to call for help. Alternatively or additionally, the emergency call functionality may be triggered automatically responsive to the processed sensor output indicating that a situation (such as accident and/or avalanche) has occurred that may have caused the user of the ski goggles 10 to be unconscious.
  • a situation such as accident and/or avalanche
  • Fig. 18 is a screenshot that is directed to the functionality of displaying danger warnings on particularly the AR display of the ski goggles 10 or the display of the wireless communication terminal 70.
  • a danger indicator is shown as e.g. color (e.g. red)-marked area if the user of the ski goggles 10 is near to cliffs, woods or other potentially dangerous areas.
  • the lower screenshot represents a warning regarding speed reduction on the AR display when the ski driver is too fast or near to a crowded area. Crowded areas may be identified using Al-based image processing techniques (as will be explained in more detail below) or by processing GNSS traces of several ski goggles.
  • the ski goggles 10 and/or the wireless communication terminal 70 may enable different usage modes and/or different skill levels to be specified by the user of the ski goggles 10.
  • the ski goggles 10 and/or the wireless communication terminal 70 may enable the user of the ski goggles 10 to make a selection from a pre-defined set of usage modes and a pre-defined set of skill levels.
  • the ski goggles 10 and/or the wireless communication terminal 70 may use this information for, e.g., determining avoid and/or prefer options for routing and navigation, for determining which parameters are to be recorded and/or aggregated, or for other purposes.
  • Fig. 19 is a screenshot that shows a display graphics enabling the ski mode and ski level choice for the ski goggles 10 when recording skiing data (first functional module).
  • the user of the ski goggles 10 is enabled to choose between the three ski modes piste, backcountry and park, in order to record the performance of the ski driver in different ski parameters as for example speed, driving time, altitude, turns and/or other.
  • the three modes have different functional/technical requirements for the ski driver.
  • the user of the ski goggles 10 is enabled to choose between the levels beginner, intermediate and expert.
  • the three modes cause different data (e.g., different physical parameters derived from the processed sensor output) to be recorded.
  • the mode piste (cf. Fig. 20 on the right), data may be recorded that may be provided as a total summary of a ski day.
  • the data in this mode may comprise any one or any combination of: a total ski distance (e.g. in km, a duration (on the mountain), a total ski time (e.g. in h), an average ski speed (e.g. in km/h), a number of runs, a biggest slope and a highest altitude.
  • the ski goggles 10 and/or wireless communication terminal 70 may determine any one or any combination of: the ski time, the ski distance (e.g. in km), the fastest speed (e.g. in km/h), the altitude and the slope.
  • the ski goggles 10 and/or wireless communication terminal 70 may record the recorded the ski time, the ski distance (e.g. in km), the fastest speed (e.g. in km/h), the altitude and the slope.
  • the ski goggles 10 and/or wireless communication terminal 70 may determine and record, for each run, any one or any combination of the following: the average ski speed (e.g. in km/h), a length of a jump (e.g. in m), a height of a jump (e.g. in m), a time in the air (e.g. in s) and a number of spins (e.g. 360°, 720°, 1080° etc.). Further, for the total summary of a ski day, the ski goggles 10 and/or wireless communication terminal 70 may determine and record a longest jump distance, a maximum spin numbers, a longest time in the air and a highest jump).
  • the average ski speed e.g. in km/h
  • a length of a jump e.g. in m
  • a height of a jump e.g. in m
  • a time in the air e.g. in s
  • a number of spins e.g. 360°, 720
  • the ski goggles 10 and/or wireless communication terminal 70 may determine and record any one or any combination of: the ski time, the ski distance (e.g. in km), the fastest speed (e.g. in km/h), the average speed (e.g. in km/h), the altitude and a vertical drop (altitude difference from top to bottom).
  • the machine-readable instruction code (also referred to APP herein) executed by the wireless communication terminal 70 and/or the ski goggles 10 may also enable sharing of information among different users of ski goggles having the configuration disclosed herein, and/or among different users of wireless communication terminals 60 executing the machine-readable instruction code.
  • the machine-readable instruction code may be operative to allow groups of users to be defined via the ski goggles 10 and/or the wireless communication terminal 70.
  • the machine-readable instruction code may be operative to enable, when executed by the wireless communication terminal 70 and/or the ski goggles, transmission of data derived from the processed sensor output (such as GNSS-based position information and/or skiing activity-related data) to share such data among selected users of the ski goggles, e.g., users assigned to a same group.
  • the machine-readable instruction code may additionally or alternatively be operative to enable, when executed by the wireless communication terminal 70 and/or the ski goggles, communication between the users assigned to the same group.
  • the communication enabled by the machine-readable instruction code may comprise one, several, or all of: text message communication, voice communication, image data transmission, video data transmission, transmission of activity-related data.
  • the machine-readable instruction code may be operative to enable, when executed by the wireless communication terminal 70 and/or the ski goggles, the generation of information based on the processed sensor data and camera images captured by the ski goggles 10 and provision of the information for retrieval by users of the same group and/or users that do not necessarily have to belong the same group.
  • the APP can offer the possibility of performing data processing related to skiing in groups (and may have a functional module for the group-related activities) in a further mode sessions (cf. Fig. 21 ).
  • friends may be gathered who want to go skiing in a same skiing resort on a same day.
  • the machine-readable instruction code may enable a user or several users to choose a date in order to start a group specifying one, several, or all of: a time, a location, a maximum number of skiers/members, a (required) ski level and whether the group shall be open to the public (cf. Figs. 22 and 23).
  • the machine-readable instruction code may enable other users to view the group and allow them to join if they decide to do so.
  • the machine-readable instruction code may enable an initiator to also invite friends into the group (cf. Fig. 23 on the right).
  • the machine-readable instruction code may enable the invited friends to choose whether they want to join the group or not, when they see the group information.
  • the machine-readable instruction code may be operative to transfer the formed group to a next communication function, namely a chat group.
  • the machine-readable instruction code may enable the user to decide to join the group of another user.
  • the users may see the groups of other users with which they have made friends and may decide whether they want to join these groups (cf. Fig. 24).
  • the machine-readable instruction code may enable the group administrator who has the authorization to invite users and exclude users to determine which user is allowed to join which user group.
  • the machine-readable instruction code may enable the group administrator to set a maximum size of the group (cf. Fig. 25).
  • the machine-readable instruction code when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative such that a group formation is automatically performed in a communication module and in the community module of the APP (cf. Fig. 26).
  • the community module may comprise two parts, namely friends and chat group. When a group has successfully been formed in the module sessions or if a user joins another group, that user will see the chat group in the community.
  • the group chat function is implemented in the chat group. There are two possibilities to implement the chat, namely via the APP on the wireless communication terminal 70 and/or via the ski goggles 10.
  • the machine-readable instruction code when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative such that when chatting via the APP a user is enabled to send a voice message directly from the chat group. This may be performed responsive to a command, e.g., by tactile command (e.g., by pressing down and holding down the microphone logo and clicking onto other users in order to play a user’s voice) or via another input modality, such as a voice command.
  • a command e.g., by tactile command (e.g., by pressing down and holding down the microphone logo and clicking onto other users in order to play a user’s voice) or via another input modality, such as a voice command.
  • the user wears the ski goggles 10 and is enabled to hear the voice directly via the speakers of the ski goggles 10.
  • a button for the voice communication in the chatroom (group chat button).
  • the user presses down and holds down the group chat button in order to speak and the other members of the chat room may receive and hear the message.
  • Other ways of activating this functions are possible, e.g., by way of voice command.
  • the machine-readable instruction code when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative such that users in the chat groups may send photographs (cf. Fig. 27). This may be done in such a manner that sharing of photographs is enabled. Other data sharing may be supported as well, such as sharing activity-related data.
  • the machine-readable instruction code when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to perform map- related functions.
  • the map-related functions may comprise any one or any combination of: identifying paths accessible to a user of the ski goggles (based on, e.g., map data, crowdsourced data from other ski goggles users, the processed sensor output and/or images as processed by an Al model); performing navigation tasks, such as route search and/or route guidance; performing POI identification; performing landmark identification, without being limited thereto.
  • Al-based processing techniques may be used in this context.
  • the machine-readable instruction code when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to provide a further functionality that is implemented as a map navigation module (i.e., a further functional module).
  • the machine-readable instruction code when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative such that a ski track and mountain map and navigation are generated.
  • a ski track and mountain map and navigation By using the current ski resort's data of snowcast and mountain map data, optionally in combination with crowdsourced terrain data processed with an Al program, such information may be generated. Satellite and/or aerial imagery may be used by the machine-readable instruction code for performing or assisting in the localization.
  • the machine- readable instruction code when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to build up a unique ski tracks map and provide corresponding navigation.
  • GNSS e.g., GPS
  • the speed and the displacement may be determined.
  • the navigation takes the level of the skier into consideration and creates a suitable track for the skier without guiding, for example a beginner onto the intermediate piste.
  • the current position of the user’s ski friends and their current driving speed may be displayed on the map generated by the machine-readable instruction code (cf. Fig. 28).
  • navigation tasks may comprise navigation to a movable target, such as a location of another skier who has agreed to share his/her location with the user of the ski goggles.
  • the machine-readable instruction code when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to enable the user of the ski goggles 10 to search the current position of this friend and click onto the navigation logo when the position is found in order to create a live navigation and to follow the track to the position of this friend (cf. Fig. 29).
  • the map-related functions may also comprise displaying POIs, such as points of orientation or other landmarks.
  • the machine-readable instruction code when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to enable the user to choose, via the ski goggles 10 or an APP-generated map on the wireless communication terminal 10, such a POI, as for example a restaurant, a cable car or a first aid station in order to display the location and the opening hours of the point of orientation as well as real time navigation track to the respective destination (cf. Fig. 30).
  • Fig. 31 shows an exemplary navigation track to a restaurant chosen by the user.
  • the machine-readable instruction code when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to enable the type of information provided by the display 51 of the ski goggles 10 to be configured.
  • the machine-readable instruction code may allow the user to set which information is to be displayed via the display 51.
  • the user in the configuration mode, the user may be enabled to freely choose the AR display settings in the machine-readable instruction code executed on the wireless communication terminal 70 in order to get the most suited and comfortable amount of information in order not to impede the joy of skiing (cf. Fig. 32).
  • Such a configuration also is useful for enhanced safety.
  • the machine-readable instruction code when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to enable a user to aggregate and share information.
  • an information board or information wall may be generated and provided for access. . This function allows users of the ski goggles 10 to upload ski photos into the personal information, to leave messages, to record voice protocols and get into contact with other ski comrades/friends (cf. Fig. 33).
  • artificial intelligence (Al)-based processing may be applied to at least part of the data captured using the ski goggles 10.
  • the Al-based processing may be implemented in the ski goggles 10, the wireless communication terminal 70, or a computing resource 81 , 82 separate therefrom.
  • the Al-based processing may be performed for various purposes, such as for performing a navigation function, object identification, communication functions, and/or for safety-relevant functions.
  • the Al-based processing may comprise a classifier to classify terrain, possible dangers, and difficulties (e.g., slope difficulty levels).
  • Fig. 34 shows a system comprising the ski goggles 10 and at least one processing circuit 90. While the at least one processing circuit 90 is shown separately from the ski goggles 10 for clarity, the at least one processing circuit 90 may comprise one or several processing circuits that are incorporated into the functional body of the ski goggles 10. Alternatively or additionally, the at least one processing circuit 90 may comprise one or several processing circuits that are incorporated into the wireless communication terminal 70 or a computing resource 81 , 82 separate from the ski goggles 10 and the wireless communication terminal 70.
  • the at least one processing circuit 90 is operative to execute one or several Al models 91 .
  • the one or several Al models 91 may comprise an Al model operative to receive data captured using the ski goggles 92.
  • the input data 92 to the one or several Al models 91 may comprise one or several images, sensor output of one or several sensors of the ski goggles 10, or processed sensor output generated by the ski goggles 10 or separately therefrom in a preprocessing step.
  • the one or several Al models 91 may have an input layer or other input operative to receive the data captured by the ski goggles 10 or data obtained therefrom by preprocessing.
  • the Al model input 102 may be based on data captured by the camera and other sensors of the ski goggles 10.
  • the one or several Al models 90 may comprise classifiers or other Al models.
  • An Al model output 93 of the one or several Al models 91 may be provided to an output generation 94.
  • the output generation 94 may generate output for provision via a communication interface and/or display (such as the display 51 of the ski goggles or the terminal display of the wireless communication terminal 70) and/or audio device.
  • the output generation 94 may provide the output 95 to a GUI controller 96. Alternatively or additionally, the output generation 94 may provide the output 95 to a communication interface control.
  • the GUI controller 96 may cause the output to be displayed via the display 51 of the ski goggles or the terminal display of the wireless communication terminal 70.
  • the at least one processing circuit 90 may comprise one or several integrated circuits, such as integrated semiconductor circuits (e.g., processor(s), controller(s), FPGAs, ASICs) and/or circuits comprising quantum gates.
  • integrated semiconductor circuits e.g., processor(s), controller(s), FPGAs, ASICs
  • quantum gates e.g., processor(s), controller(s), FPGAs, ASICs
  • any reference to artificial intelligence or machine learning as used herein also encompasses quantum machine learning techniques.
  • the at least one Al model may comprise a recurrent neural network (RNN) or a convolutional neural network (CNN).
  • the RNN or CNN may have an input layer operative to receive image data captured by the ski goggles 10 and/or other sensor data (such as the processed sensor output or the sensor output prior to processing).
  • the RNN or CNN may be operative to perform a classification or identification task.
  • the RNN or CNN may be operative to identify potentially hazardous areas within an image.
  • the output may be used to provide, as overlay to the real-world view or an image, a marking designating a hazardous area or an area otherwise unsuitable for the user of the ski goggles.
  • the RNN or CNN may be operative to identify an avalanche risk based on the image and other sensor data.
  • the RNN or CNN may be operative to identify people shown in images, which is useful for communication functions.
  • Fig. 35 shows an Al model 100 that may be executed by the at least one processing circuit 90.
  • the Al model 100 has an input layer 101 , an output layer 102, and several hidden layers 103.
  • the Al model 100 is operative to process inputs 104, 105 received by the input layer 101 to generate the output 106 provided by the output layer.
  • the input of the Al model 100 comprises GNSS-based data 104 and image data 105.
  • the output 106 may comprise a classification relevant for safety- related functions, such as an avalanche risk, a terrain difficulty (such as a skiing slope difficulty level), or other data that is useful in the various functions disclosed herein.
  • Fig. 36 shows the Al model 100 that may be executed by the at least one processing circuit 90 when applied to image data 105 showing a person.
  • the Al model output layer 106 may in this case provide an output 106 including an identifier for that person.
  • the Al model 100 may operate as an object recognition module.
  • the Al model 100 may be operative to process time-series data. This may be particularly useful for situation analysis, e.g., for identifying hazardous situations.
  • LSTM cells and/or GRUs may be stacked in the Al model 100 to process the time series data.
  • Fig. 37 shows an implementation of a LSTM cell 110 that may be used in an Al model 91 operative to process time-series data. Such processing may be useful for, e.g., identifying safety-critical situations.
  • a stack of LSTM cells 110 may be provided in the Al model 91 .
  • C designates a cell state.
  • the parameter h designates a hidden state.
  • the parameter x designates an input.
  • the subscript respectively designates the time.
  • the subscript t designates the time of the input processed by the cell illustrated in Fig. 37.
  • the subscript t - 1 designates the preceding time.
  • parameters of the LSTM cell or of several LSTM cells arranged in a LSTM cell stack may be trained.
  • the training may comprise training of a forget gate (designated by ft), an input gate (designated by it) and an output gate (designated by ot).
  • Training the various gates may comprise training the weight parameters and bias parameters of each of the gates of the LSTM cell.
  • the training of the at least one Al model 91 may be performed using training data.
  • the training data is training data obtained from one or several cameras and/or additional sensors, as provided in the ski goggles 10.
  • the training data may be split into a training set (which may include, e.g., 70% of the training data), into a validation set (which may include, e.g., 15% of the training data), and into a test set (which may include, e.g., 15% of the training data).
  • Data augmentation may be performed, to account for the fact that critical slope situations (such as avalanche risk situations or cliffs) are inevitably rare compared to safe situations.
  • the output of the at least one Al model 91 may be used in various ways, e.g. for controlling a head-up display or an audio device to provide output that depends on the output 93 of the at least one Al model 91 .
  • Fig. 38 is a flow chart of a method 120, comprising the training 121 of the Al model(s), the storing 122 of the Al model(s) in a storage or memory device for use by at least one processing circuit, using 123 the Al model to process data captured using the ski goggles 10, and controlling 124 an interface (e.g., the display 51) to provide an output that depends on the Al-based processing.
  • the output may comprise visual and/or audio output and/or data output via a data interface.
  • the output that is provided may comprise any or any combination of the following:
  • Al-based image processing may be applied for this purpose.
  • POI common friend
  • the identifier may be determined by utilized by GNSS location sharing between the different ski goggles (the current location and direction of view is known) and/or object recognition, which may be implemented in an Al-based manner;
  • Al-based techniques may be used, for example, for one or several of the following:
  • - data fusion e.g., fusing image data with other sensor data or fusing different types of sensor data
  • Additional processing techniques that do not involve Al-based processing may be used in addition to or instead of Al-based processing.
  • processing may comprise data fusion techniques, data sharing, use of aerial or satellite images that are retrievable from servers, use of image data for generating a world model of a surrounding of the skier, and similar techniques.
  • the ski goggles 10 may also provide a digital ski coach functionality. Feedback can be provided based on sensor measurements included in the ski goggles 10 and/or in other wearables (such as ski boots or other ski outfit) to provide feedback, optionally including suggestions for performance improvement.
  • Communication techniques such as Bluetooth Low Energy (BLE) beacons or other wireless communication techniques may be used to establish relative spatial proximity of users of different ski goggles 10.
  • BLE Bluetooth Low Energy
  • Such data may be fused which image processing results, which may be based on one or several trained neural networks, for example.
  • the various processing techniques may be used to generate output for displaying as an overlay to a real-world view via the display 51 .
  • Fig. 39 shows a particularly preferred embodiment of the functional body 30 of the ski goggles 10 in which the display 51 comprises or is an optical waveguide display.
  • the display driver 53 comprises an electrooptical transducer operative to generate electromagnetic signals (e.g., in the visible range of the electromagnetic spectrum) and provide them to the optical waveguide display.
  • the display driver 53 may comprise a light engine 130 operative to generate light at one or, preferably, several different wavelengths and to supply the signals to the optical waveguide display.
  • the optical waveguide display may be operative to guide the generated optical signals for generating the optical output that is visible on the optical waveguide display as an overlay to the real-world view.
  • Fig. 40 and Fig. 41 show a particularly preferred embodiment of the functional body 30 of the ski goggles 10 in which the at least one sensor comprises one or several microelectromechanical systems (MEMS) sensors 128 (Fig. 40) or a sensor set 129 comprising a gyroscope, inertial measurement unit (IMU), barometer, and magnetometer (Fig. 41 ).
  • MEMS sensors 128 in Fig. 40 may be operative to implement an IMU, barometer, and magnetometer.
  • the display driver 53 in the functional body configurations of Fig. 40 and Fig. 41 may comprise an electrooptical transducer operative to generate electromagnetic signals (e.g., in the visible range of the electromagnetic spectrum) and provide them to the optical waveguide display.
  • the display driver 53 may comprise a light engine 130 operative to generate light at one or, preferably, several different wavelengths and to supply the signals to an optical waveguide display 51 .
  • the sensor combinations disclosed in association with Fig. 40 and Fig. 41 are robust in field use in skiing applications and provide sensor data of particular interest to skiers, such as: acceleration, orientation of the ski goggles in space, altitude.
  • This sensor data may be processed to generate information for provision via the display 51 , a communication terminal 70 coupled to the ski goggles 10 and/or for otherwise monitoring the activity of the wearer of the ski goggles.
  • a computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.
  • a computer program can be a computer program product stored on a computer readable medium which computer program product can have computer executable program code adapted to be executed to implement a specific method such as the method according to the invention.
  • a computer program can also be a data structure product or a signal for embodying a specific method such as the method according to the invention.

Abstract

Ski goggles (10) comprise a lens frame (20) configured to retain a lens (11), a functional body (30), and a battery body (40) attached to opposite sides of the lens frame (20). The functional body (30) comprises a display (51), at least one wireless interface circuit, at least one sensor, and at least one integrated circuit. The display (51) extends along part of the lens (11) in a space defined by the lens frame (20) and the lens (11). The display (51) is transparent or semitransparent to output information as an overlay to a real-world view visible through the lens (11). The at least one integrated circuit is configured to process a sensor output of the at least one sensor and use the processed sensor output to control the display to output the information and cause a data transmission by the at least one wireless interface circuit.

Description

DESCRIPTION
Title
SKI GOGGLES, WIRELESS COMMUNICATION TERMINAL, AND METHOD OF CONTROLLING A DISPLAY OF SKI GOGGLES
Technical Field
[0001] The present invention relates to ski goggles according to the preamble of independent claim 1 and more particularly to ski goggles that comprise a display.
[0002] Such ski goggles can be used for providing information to a wearer by means of the display and implementing augmented reality functions in ski goggles.
Background Art
[0003] CN 112 782 870 A and CN 214 252 800 U disclose ski goggles which comprise a goggle frame, a lens, and components for audio functions, including a microphone and speakers. The ski goggles also comprise receiving and transmitting antennas.
[0004] CN 215 117 044 U discloses adaptive ski goggles which comprise a photosensitive sensor and a lens that is responsive to voltage changes.
[0005] US 2011/0227820 A1 discloses an interactive head-mounted eyepiece with an integrated processor for handling content for display and an integrated image source for introducing the content to an optical assembly through which the wearer views a surrounding environment and the displayed content. The displayed content may comprise a local advertisement wherein the location of the eyepiece is determined by an integrated location sensor. The eyepiece comprises arm portions disposed on opposite sides of the eyepiece.
[0006] CN 209 167 686 U, which serves as basis for the preamble of claim 1 , discloses augmented reality (AR) ski goggles. The AR ski goggles comprise an optical display that forms a wind shielding sheet. The AR ski goggles also comprise a frame structure and a communication module.
[0007] Conventional ski goggles do not adequately address the need for ski goggles having a simple and reliable construction that provides comfort to the wearer while accommodating the wearer’s desire for enhancing the real-world view seen through a goggle lens with additional information.
[0008] WO 2011/044680 A1 and WO 2013/000075 A1 disclose ski goggles comprising a display. The display is arranged to extend adjacent a lens of the ski goggles and blocks part of a real-world field of view that would otherwise be visible through the ski goggles. This may adversely affect user experience. When the display blocks safetyrelevant features of the real-word view (such as an obstacle on a skiing slope or a recess in the skiing slope).
[0009] Thus, it would be desirable to provide ski goggles that further improve safety.
[0010] Therefore, there is a need for improved ski goggles. There is in particular a need for ski goggles having a simple and reliable construction that provides comfort to the wearer while accommodating the wearer’s desire for enhancing the real-world view seen through a goggle lens with additional information. There is also a need for a communication terminal configured to interact with such ski goggles and control methods suitable for such ski goggles.
Disclosure of the Invention
[0011] According to the invention this need is settled by ski goggles as defined by the features of independent claim 1 , by a wireless communication terminal as defined by the features of independent claim 14, by a method as defined by independent claim 18, by a use as defined by independent claim 19, and by machine-readable instruction code as defined by claim 20. Preferred embodiments are subject of the dependent claims.
[0012] In one aspect, the invention are ski goggles comprising: a lens; a lens frame configured to retain the lens; a functional body comprising a cavity, wherein the functional body further comprises at least one wireless interface circuit and at least one integrated circuit arranged within the cavity; and a battery body comprising one or several batteries for supplying power to the functional body. The battery body and the functional body are attached to the lens frame on opposite sides of the lens frame. The functional body comprises a display extending along part of the lens in a space defined by the lens frame and the lens, the display being transparent or semitransparent to output information as an overlay to a real-world view visible through the lens. The functional body comprises at least one sensor arranged within the cavity. The at least one integrated circuit is configured to process a sensor output of the at least one sensor and use the processed sensor output to control the display to output the information and cause a data transmission by the at least one wireless interface circuit.
[0013] The ski goggles enhance the wearer’s experience by providing augmented reality (AR) functionality. Information that is derived at least from a sensor output is used for controlling a display, making it possible to blend information relating to the environment of the ski goggles with a real-world view visible through the lens. The arrangement of the battery body and functional body facilitate comfort to the wearer by providing a design that balances weight distribution. The balancing also contributes to enhanced safety by mitigating the risk of inadvertent dislodgment of the ski goggles during continued use (which might otherwise occur when the ski goggles are subject to force pulses, as is the case when skiing on uneven terrain).
[0014] By using a transparent or semitransparent display that blends information relating to the environment of the ski goggles with a real-world view visible through the lens as an overlay, safety and comfort are enhanced further. The display does not block part of the real-world view that would otherwise be visible through the lens. By using a functional body that comprises such a display, robust connections of the display to the circuitry that processes the sensor output is attained. The risk of malfunction of the display is mitigated as compared to arrangement in which the display is attached by means of a mounting structure that projects from a portion of the lens frame other than the functional body on a side of the lens frame. In the latter case, the additional connections required to extend along the lens frame and the mounting structure can increase the risk of display malfunction.
[0015] By using the processed sensor data to not only control the display to output the information but also to cause a data transmission, safety and comfort increased further. Critical situations or other situations warranting the data transmission may be identified by the circuitry of the functional body and may be used to trigger the data transmission. [0016] Preferably, the at least one integrated circuit is configured to use data received via the wireless interface circuit to control the display.
[0017] Thereby, information received via the wireless interface circuit can be overlaid onto the real-world view. The information received via the wireless interface circuit can be communication-related information (such as information on incoming calls, voice messages, video messages, or text messages) and/or social networking information (such as information on status updates). The information received via the wireless interface circuit may also include information related to skiing, such as avalanche warnings, slope difficulty rating, and/or snow condition information, which may be retrieved by the ski goggles using the processed sensor output (e.g., a geolocation derived from a Global Navigation Satellite System, GNSS, e.g., derived from a global positioning system, GPS, sensor output).
[0018] Preferably, the at least one sensor is configured to determine a three- dimensional, 3D, orientation and/or 3D acceleration of the functional body, wherein the at least one integrated circuit is configured to use the sensor output to determine a velocity of the ski goggles as the information. The data transmission may comprise the transmission of the determined velocity.
[0019] Thereby, information of particular interest to the wearer, such as a velocity during skiing, may be output as overlay on the real-world view.
[0020] Preferably, the at least one sensor comprises a global navigation satellite system (GNSS) receiver and the at least one integrated circuit is configured to process a GNSS receiver output.
[0021] Such a configuration facilitates the determination of geolocation-related information, such as an altitude, for displaying. This configuration also facilitates the retrieval of information related to the geolocation, such as weather conditions, slope difficulty, avalanche risk rating, by querying information services to which the ski goggles may be communicatively interfaced via a cellular or other wireless network.
[0022] Preferably, the at least one integrated circuit is configured to use the processed GNSS receiver output to control the display to output the processed GNSS receiver output. [0023] This allows location-dependent information to be displayed for viewing. The location-dependent information may include an altitude which may be of particular interest to the wearer of the ski goggles.
[0024] Preferably, the at least one integrated circuit is additionally or alternatively configured to use the processed GNSS receiver output to control the at least one wireless interface circuit such that the data transmission includes the processed GNSS receiver output.
[0025] This facilitates the provision of location-dependent information (such as geotagged status information, geo-tagged audio and/or video data, geo-tagged social networking information) via a cellular or other wireless network.
[0026] Preferably, the at least one integrated circuit is additionally or alternatively configured to use the processed GNSS receiver output to control the at least one wireless interface circuit to perform a data retrieval.
[0027] This facilitates the retrieval of location-dependent information from one or several servers (such as weather forecast, avalanche service, or other servers) via a cellular or other wireless network.
[0028] Preferably, the processed GNSS receiver output comprises altitude information.
[0029] This allows altitude information, which is of particular interest for ski goggle wearers, to be displayed as overlay on the real-world view and/or transmitted to a communication terminal for activity tracking.
[0030] Preferably, the at least one sensor may comprise at least one microelectromechanical system operative as one, several, or all of: an inertial measurement unit, a magnetometer, a barometer.
[0031] This allows information of particular interest for ski goggle wearers to be determined and used for display control and/or data communication purposes.
[0032] Preferably, the sensor at least one sensor may comprise different microelectromechanical systems (MEMS) sensors as IMU (inertial measurement unit), magnetometer and barometer [0033] This allows information of particular interest for ski goggle wearers to be determined and used for display control and/or data communication purposes.
[0034] Preferably, the at least one integrated circuit is configured to use the processed sensor output to perform safety-related functions.
[0035] Thereby, safety of the wearer of the ski goggles is improved.
[0036] Preferably, the safety-related functions comprise controlling the display to output a hazard warning or risk assessment. The hazard warning or risk assessment may comprise an avalanche risk and/or a slope difficulty rating retrieved via the at least one wireless interface circuit using the processed sensor output.
[0037] Thereby, safety of the wearer of the ski goggles is improved.
[0038] Preferably, the safety-related functions comprise controlling the at least one wireless interface circuit to transmit an emergency signal.
[0039] Thereby, safety of the wearer of the ski goggles is improved in case of an emergency. The at least one integrated circuit may perform emergency detection based on the processed sensor output.
[0040] Preferably, the ski goggles comprise a camera arranged on the lens frame, the at least one integrated circuit being configured to receive image frames captured by the camera.
[0041] This facilitates positioning of information on the display relative to the real-world view and allows the transmission or recording of image frames (such as a video sequence).
[0042] The camera has a camera objective and the lens has a recess or aperture into which the camera objective projects. The camera objective may be flush with a surface of the lens facing away from the wearer’s face in use.
[0043] This configuration facilitates image capture without interference by the lens.
[0044] Preferably, the integrated circuit is configured to control the wireless interface circuit such that the data transmission includes at least some of the image frames. [0045] This configuration facilitates the sharing of visual data, for remote analysis, sharing, storing, or other purposes.
[0046] Preferably, the integrated circuit is configured to process the image frames and use the processed image frames to control the display to output the information.
[0047] This configuration facilitates the integration of displayed information into the real-world view visible through the lens. Alternatively or additionally, information derived from the captured image frames may be generated and output.
[0048] Preferably, the ski goggles comprise an audio interface. The audio interface may comprise several transducers. The audio interface may comprise one or several microphones and one or several speakers. Drive circuit(s) for the microphones and/or speakers may be integrated into the functional body.
[0049] This configuration allows the ski goggles to be used for various communication functions during skiing.
[0050] The ski goggles may comprise a strap for securing the ski goggles to a head. The functional body may comprise a first passage for passing the strap through the first passage along the functional body. The battery body may comprise a second passage for passing the strap through the second passage along the battery body.
[0051] This configuration allows the strap to be guided by the functional body and battery body, providing increased wearer comfort.
[0052] The lens frame may comprise an opening for allowing the display of the functional body to enter the space defined by the lens frame and the lens.
[0053] This configuration provides a simple way of arranging the display in the interior of the space defined by the lens frame and the lens, without requiring control electronics or other display control components to be arranged on the lens frame.
[0054] The lens frame may comprise conductors (such as wires or conductive traces) that provide a conductive path between the battery or batteries of the battery body and the functional body.
[0055] This configuration allows the functional body to be powered while facilitating the implementation of a weight balanced ski goggle design. [0056] The battery body and functional body may be movably attached to the lens frame.
[0057] This configuration facilitates adaptation of the ski goggles to different head sizes and/or head shapes and/or helmet sizes and/or helmet shapes.
[0058] The battery body and functional body may be attached to the lens frame via a soft linking material. The soft linking material may implement a living hinge. The battery body and functional body may be attached to the lens frame via a living hinge, which may be implemented by a soft linking material.
[0059] This configuration facilitates adaptation of the ski goggles to different head sizes and/or head shapes and/or helmet sizes and/or helmet shapes.
[0060] The functional body and/or the battery body may respectively comprise one or several mechanically actuable buttons or keys that are arranged on an end of the functional body and/or battery body adjacent the lens.
[0061] The mechanically actuable allow the wearer to perform various frequently used activities (such as accepting a call, speaker volume adjustment, dialing, etc.) to be performed in a simple and intuitive manner.
[0062] The lens may be attached to the lens frame in a reversibly releasable manner.
[0063] This facilitates exchange of the lens when needed, e.g., for cleaning or other purposes.
[0064] The lens may be configured such that it attenuates but does not block the blue wavelength range.
[0065] This enhances safety by making it easier for the wearer of the ski goggles to see surface features in a snow-covered terrain.
[0066] According to another aspect of the invention, there is provided a wireless communication terminal. The wireless communication terminal comprises a wireless communication chipset configured to perform bi-directional data communication with the ski goggles according to any aspect or embodiment; a terminal display; and at least one processor configured to process the data transmission from the ski goggles and use the processed data transmission to perform activity tracking and control the terminal display to output results of the activity tracking.
[0067] The wireless communication terminal facilitates monitoring of a skiing activity by virtue of its communicative coupling with the ski goggles and by virtue of its activity tracking function The wireless communication terminal can also serve as a communication hub configured to relay data, audio, and/or video communication to and from the ski goggles.
[0068] Preferably, the activity tracking comprises using the processed data transmission to track an altitude and/or a velocity of the ski goggles as a function of time.
[0069] Thereby, the wearer of the ski goggles and/or another user of the communication terminal is/are enabled to review the results of the activity tracking.
[0070] Preferably, the wireless communication terminal is configured to use the processed data transmission to retrieve, via a cellular network or wireless local area network, an avalanche risk and/or a slope difficulty rating for a geolocation defined by the data transmission from the ski goggles and transmit the avalanche risk and/or the slope difficulty rating to the ski goggles.
[0071] Thereby, the wireless communication terminal assists the ski goggles in retrieving safety-related information, while obviating the need for the ski goggles to have a cellular communication interface circuit.
[0072] Preferably, the wireless communication terminal is configured to relay audio and/or video data to and from the ski goggles.
[0073] Thereby, the wireless communication terminal can act as a communication hub for communication to and/or from the ski goggles.
[0074] The mobile communication terminal and/or the ski goggles may be operative to enable a usage mode, in particular a skiing mode, to be input. The usage mode may be selected from a pre-defined group, e.g., a pre-defined group comprising or consisting of skiing on designated slopes, skiing on pathways that are not limited to designated slopes, and skiing in a ski park. [0075] This allows operation to be tailored to the scenario in which the user uses the ski goggles.
[0076] The mobile communication terminal and/or the ski goggles may be operative to use the processed sensor data in a manner which is dependent on the usage mode input by the user. This may comprise aggregating different physical parameters, generated from the processed sensor output, depending on which usage mode has been input by the user.
[0077] This allows the use of the processed sensor data to be tailored to the needs of the usage mode.
[0078] The mobile communication terminal and/or the ski goggles may be operative to output different possible selections for the usage mode.
[0079] This facilitates inputting of the usage mode.
[0080] The mobile communication terminal and/or the ski goggles may be operative to enable a skill level to be input. The skill level may be selected from a pre-defined group, e.g., beginner, intermediate, advanced.
[0081] This facilitates inputting of the skill level, which is relevant for safety-relevant functions.
[0082] The mobile communication terminal and/or the ski goggles may be operative to perform a navigation task. The navigation task may comprise a route search and/or route guidance. A target of the route search and/or route guidance may have stationary geo-coordinates (such as a point of interest) or may be movable (such as another skier). The mobile communication terminal and/or the ski goggles may be operative to retrieve the coordinates of a movable target from other ski goggles according to an embodiment.
[0083] Thereby, the user of the ski goggles receives assistance for the task of finding a good route, which is particularly challenging when the target of the route search or route guidance is movable.
[0084] The mobile communication terminal and/or the ski goggles may be operative to perform the navigation task based on the input usage mode and/or skill level. [0085] This configuration allows the user's skills and desired usage (on designated slopes/off designated slopes) to be taken into consideration when performing the navigation task.
[0086] The mobile communication terminal and/or the ski goggles may be operative to aggregate activity-related data based on the processed sensor output.
[0087] This configuration allows relevant physical parameters to be aggregated over a usage session (e.g., during a skiing day).
[0088] The mobile communication terminal and/or the ski goggles may be operative to generate output that depends on the aggregated activity-related data.
[0089] This configuration allows physical parameters relevant for the user to be automatically collected.
[0090] The mobile communication terminal and/or the ski goggles may be operative such that the data that are being aggregated and/or the generated output is dependent on the input usage mode and/or skill level.
[0091] This configuration allows physical parameters relevant for the user to be automatically collected.
[0092] The mobile communication terminal and/or the ski goggles may be operative to enable groups of users to be defined.
[0093] This configuration enables a set of trusted users to be established with which communication and/or activity-related data (such as position information or information derived therefrom) may be shared.
[0094] The mobile communication terminal and/or the ski goggles may be operative to enable communication functions among the users belonging to a group.
[0095] This configuration enables a set of trusted users to be established with which communication exchange may be implemented.
[0096] The mobile communication terminal and/or the ski goggles may be operative to enable sharing of data (such as text or voice communication, images, and/or activity- related data). [0097] This configuration enables a set of trusted users to be established with which communication data may be shared.
[0098] The mobile communication terminal and/or the ski goggles may be operative to restrict access to the shared data, e.g., to member of a previously defined group.
[0099] This configuration enhances privacy.
[00100] The mobile communication terminal and/or the ski goggles may be operative to enable generation of an information panel (which may include any one or any combination of text or voice communication, images, activity-related data).
[00101] This configuration facilitates information sharing.
[00102] The mobile communication terminal and/or the ski goggles may be operative to identify persons and/or landmarks in proximity to the user of the ski goggles and to output the information on the identified persons and/or landmarks via the display of the ski goggles or via the mobile communication terminal.
[00103] Thereby, information is automatically determined which is of particular relevance for the functions performed by the mobile communication terminal and/or the ski goggles.
[00104] The mobile communication terminal and/or the ski goggles may be operative to execute Al-based processing for performing at least one of its functions (such as identification of POIs, identification of terrain features, identification of designated ski slopes, navigation function).
[00105] Thereby, the processing may be implemented in a data-driven manner. This configuration increases reliability and, thus, safety.
[00106] The Al-based processing may comprise executing at least one Al model, wherein the at least one Al model has an input operative to receive the processed sensor data and at least one image captured by a camera of the ski goggles.
[00107] Thereby, the processing may be implemented in a data-driven manner. This configuration increases reliability and, thus, safety. [00108] The input of the at least one Al model may be operative to receive GNSS- based data and the at least one image.
[00109] Thereby, the processing may be implemented in a data-driven manner. This configuration increases reliability and, thus, safety.
[00110] The at least one Al model may have an output operative to provide a classification result (e.g., for a slope difficulty level, an avalanche risk, presence or absence of a designated slope) and/or an identification result (e.g., for a contact recognized based on image recognition techniques). The classification result may comprise a pixelwise classification of areas identified to be hazardous.
[00111] Thereby, processing results of particular relevance for user safety and comfort are obtained in a reliable manner.
[00112] The mobile communication terminal and/or the ski goggles may be operative to generate output for displaying via the display of the ski goggles and/or via the mobile communication result, based on the Al model output.
[00113] Thereby, the results obtained by using a data-driven processing implemented by the Al model may be harnessed for ski goggle operation and/or wireless communication terminal operation.
[00114] The at least one Al model may comprise a recurrent neural network (RNN) and/or a convolutional neural network (CNN). Such Al models are particularly suitable for processing the combination of images and other processed sensor output.
[00115] The Al-based processing may comprise executing at least one Al model, wherein the at least one Al model has an input operative to receive a time series of the processed sensor data.
[00116] This configuration allows time-series data to be processed to identify situations that are of relevance to user safety. Safety and user comfort are enhanced thereby.
[00117] The input of the at least one Al model may be operative to receive a time series of velocity and/or acceleration values obtained from the processed sensor output. [00118] Such time-series data are particularly relevant to identify potentially hazardous situations that are of relevance to user safety. Safety and user comfort are enhanced thereby.
[00119] The at least one Al model may have an output operative to provide a classification result that may comprise safety-related information, such as a classification result on whether the user of the ski goggles is in a hazardous situation (e.g., an accident or avalanche situation that warrants automatic triggering of an emergency signal transmission).
[00120] This configuration allows time-series data to be processed to identify situations that are of relevance to user safety. Safety and user comfort are enhanced thereby.
[00121] The at least one Al model may comprise gated recurrent units (GRUs) and/or long short term memory (LSTM) cells.
[00122] Such structures are particularly suitable for processing time series data.
[00123] The mobile communication terminal and/or the ski goggles may be operative to generate the data transmission based on the Al model output.
[00124] Thereby, data communication may be triggered automatically, based on a result of processing that is performed by the Al model, i.e. , in a data-driven manner.
[00125] According to another aspect of the invention, there is provided machine- readable instruction code comprising instructions which, when executed by at least one processing circuit, cause the at least one processing circuit to process data captured by the ski goggles according to an aspect or embodiment and generate output based on a result of the processing.
[00126] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to enable a usage mode, in particular a skiing mode, to be input. The usage mode may be selected from a pre-defined group, e.g., a pre-defined group comprising or consisting of skiing on designated slopes, skiing on pathways that are not limited to designated slopes, and skiing in a ski park. [00127] This allows operation to be tailored to the scenario in which the user uses the ski goggles.
[00128] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to use the processed sensor data in a manner which is dependent on the usage mode input by the user. This may comprise aggregating different physical parameters, generated from the processed sensor output, depending on which usage mode has been input by the user.
[00129] This allows the use of the processed sensor data to be tailored to the needs of the usage mode.
[00130] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to output different possible selections for the usage mode.
[00131] This facilitates inputting of the usage mode.
[00132] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to enable a skill level to be input. The skill level may be selected from a pre-defined group, e.g., beginner, intermediate, advanced.
[00133] This facilitates inputting of the skill level, which is relevant for safety-relevant functions.
[00134] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to perform a navigation task. The navigation task may comprise a route search and/or route guidance. A target of the route search and/or route guidance may have stationary geo-coordinates (such as a point of interest) or may be movable (such as another skier). The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to retrieve the coordinates of a movable target from other ski goggles according to an embodiment. [00135] Thereby, the user of the ski goggles receives assistance for the task of finding a good route, which is particularly challenging when the target of the route search or route guidance is movable.
[00136] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to perform the navigation task based on the input usage mode and/or skill level.
[00137] This configuration allows the user's skills and desired usage (on designated slopes/off designated slopes) to be taken into consideration when performing the navigation task.
[00138] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to aggregate activity-related data based on the processed sensor output.
[00139] This configuration allows relevant physical parameters to be aggregated over a usage session (e.g., during a skiing day).
[00140] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to generate output that depends on the aggregated activity-related data.
[00141] This configuration allows physical parameters relevant for the user to be automatically collected.
[00142] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit such that the data that are being aggregated and/or the generated output is dependent on the input usage mode and/or skill level.
[00143] This configuration allows physical parameters relevant for the user to be automatically collected.
[00144] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to enable groups of users to be defined. [00145] This configuration enables a set of trusted users to be established with which communication and/or activity-related data (such as position information or information derived therefrom) may be shared.
[00146] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to enable communication functions among the users belonging to a group.
[00147] This configuration enables a set of trusted users to be established with which communication exchange may be implemented.
[00148] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to enable sharing of data (such as text or voice communication, images, and/or activity- related data).
[00149] This configuration enables a set of trusted users to be established with which communication data may be shared.
[00150] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to restrict access to the shared data, e.g., to member of a previously defined group.
[00151] This configuration enhances privacy.
[00152] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to enable generation of an information panel (which may include any one or any combination of text or voice communication, images, activity-related data).
[00153] This configuration facilitates information sharing.
[00154] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to identify persons and/or landmarks in proximity to the user of the ski goggles and to output the information on the identified persons and/or landmarks via the display of the ski goggles or via the mobile communication terminal. [00155] Thereby, information is automatically determined which is of particular relevance for the functions performed by the mobile communication terminal and/or the ski goggles.
[00156] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to execute Al-based processing for performing at least one of its functions (such as identification of POIs, identification of terrain features, identification of designated ski slopes, navigation function).
[00157] Thereby, the processing may be implemented in a data-driven manner. This configuration increases reliability and, thus, safety.
[00158] The Al-based processing may comprise executing at least one Al model, wherein the at least one Al model has an input operative to receive the processed sensor data and at least one image captured by a camera of the ski goggles.
[00159] Thereby, the processing may be implemented in a data-driven manner. This configuration increases reliability and, thus, safety.
[00160] The input of the at least one Al model may be operative to receive GNSS- based data and the at least one image.
[00161] Thereby, the processing may be implemented in a data-driven manner. This configuration increases reliability and, thus, safety.
[00162] The at least one Al model may have an output operative to provide a classification result (e.g., for a slope difficulty level, an avalanche risk, presence or absence of a designated slope) and/or an identification result (e.g., for a contact recognized based on image recognition techniques). The classification result may comprise a pixelwise classification of areas identified to be hazardous.
[00163] Thereby, processing results of particular relevance for user safety and comfort are obtained in a reliable manner.
[00164] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to generate output for displaying via the display of the ski goggles and/or via the mobile communication result, based on the Al model output.
[00165] Thereby, the results obtained by using a data-driven processing implemented by the Al model may be harnessed for ski goggle operation and/or wireless communication terminal operation.
[00166] The at least one Al model may comprise a recurrent neural network (RNN) and/or a convolutional neural network (CNN). Such Al models are particularly suitable for processing the combination of images and other processed sensor output.
[00167] The Al-based processing may comprise executing at least one Al model, wherein the at least one Al model has an input operative to receive a time series of the processed sensor data.
[00168] This configuration allows time-series data to be processed to identify situations that are of relevance to user safety. Safety and user comfort are enhanced thereby.
[00169] The input of the at least one Al model may be operative to receive a time series of velocity and/or acceleration values obtained from the processed sensor output.
[00170] Such time-series data are particularly relevant to identify potentially hazardous situations that are of relevance to user safety. Safety and user comfort are enhanced thereby.
[00171] The at least one Al model may have an output operative to provide a classification result that may comprise safety-related information, such as a classification result on whether the user of the ski goggles is in a hazardous situation (e.g., an accident or avalanche situation that warrants automatic triggering of an emergency signal transmission).
[00172] This configuration allows time-series data to be processed to identify situations that are of relevance to user safety. Safety and user comfort are enhanced thereby.
[00173] The at least one Al model may comprise gated recurrent units (GRUs) and/or long short term memory (LSTM) cells.
[00174] Such structures are particularly suitable for processing time series data. [00175] The machine-readable instruction code may comprise instructions which, when executed by the at least one processing circuit, cause the at least one processing circuit to generate the data transmission based on the Al model output.
[00176] Thereby, data communication may be triggered automatically, based on a result of processing that is performed by the Al model, i.e. , in a data-driven manner.
[00177] According to another aspect of the invention, there is provided a non-transitory storage medium having stored thereon the machine-readable instruction code according to an aspect or embodiment of the invention.
[00178] According to another aspect of the invention, there is provided a system, comprising the wireless communication terminal according to an embodiment and the ski goggles according to an embodiment.
[00179] According to another aspect of the invention, there is provided a method of controlling a display of ski goggles, wherein the display is comprised by a functional body attached to a lens frame of the ski goggles that retains a lens, the display being transparent or semitransparent and extending in a space defined by the lens frame and the lens. The method comprises the following steps: receiving, by at least one integrated circuit arranged in a cavity of the functional body, a sensor output of at least one sensor arranged in the cavity of the functional body; processing, by the at least one integrated circuit, the sensor output; and sing, by the at least one integrated circuit, the processed sensor output to control the display to output information as an overlay to a real-world view visible through the lens.
[00180] The method enhances the wearer’s experience by providing augmented reality (AR) functionality. Information that is derived at least from a sensor output is used for controlling a display, making it possible to blend information relating to the environment of the ski goggles with a real-world view visible through the lens. The arrangement of the battery body and functional body facilitate comfort to the wearer by providing a design that balances weight distribution.
[00181] The ski goggles may be the ski goggles according to any embodiment disclosed herein. [00182] The method may be performed automatically by the ski goggles or system according to any embodiment disclosed herein.
[00183] Another aspect of the invention is directed to the use of the ski goggles described herein together with a functional application provided on the wireless communication terminal described herein for tracking and/or displaying on the ski goggles display and/or on the terminal display weather information, device information, ski mode choice information, live map navigation information, reference point information, area recognition information, character recognition information, sports data information, SOS emergency call information and/or danger warning information (i.e. as specified further below).
Brief Description of the Drawings
[00184] The ski goggles, communication terminal, system, and method according to the invention are described in more detail hereinbelow by way of an exemplary embodiment and with reference to the attached drawings, in which:
Fig. 1 shows a perspective view of ski goggles according to the invention;
Fig. 2 shows another perspective view of the ski goggles of Fig. 1 ;
Fig. 3 shows yet another perspective view of the ski goggles of Fig. 1 ;
Fig. 4 shows a detailed view of a light waveguide display (AR display) of the inventive ski goggles;
Fig. 5 shows a block diagram of a functional body of the ski goggles of Fig. 1 ;
Fig. 6 shows a system comprising the ski goggles of Fig. 1 and a user terminal according to the invention;
Figs. 7-18 show as screenshots from the AR display the functionalities of an APP configured for the wireless communication terminal for use with the inventive ski goggles for activity tracking and information displaying on particularly the AR display;
Figs. 19-33 show respective screenshots from the APP on the terminal display;
Fig. 34 shows a system comprising ski goggles according to the invention;
Fig. 35 shows an artificial intelligence-based processing of data captured using the ski goggles according to the invention; Fig. 36 shows another artificial intelligence-based processing of data captured using the ski goggles according to the invention;
Fig. 37 shows another artificial intelligence-based processing of data captured using the ski goggles according to the invention;
Fig. 38 shows a flow chart of a method performed by the system of Fig. 34;
Fig. 39 shows a block diagram of a specific implementation of the functional body of the ski goggles of Fig. 1 ;
Fig. 40 shows a block diagram of another specific implementation of the functional body of the ski goggles of Fig. 1 ; and
Fig. 41 shows a block diagram of a yet another specific implementation of the functional body of the ski goggles of Fig. 1 .
Description of Embodiments
[00185] In the following description certain terms are used for reasons of convenience and are not intended to limit the invention. The terms “right”, “left”, “up”, “down”, “under" and “above" refer to directions in the figures. The terminology comprises the explicitly mentioned terms as well as their derivations and terms with a similar meaning. Also, spatially relative terms, such as "beneath", "below", "lower", "above", "upper", "proximal", "distal", and the like, may be used to describe one element's or feature's relationship to another element or feature as illustrated in the figures. These spatially relative terms are intended to encompass different positions and orientations of the devices in use or operation in addition to the position and orientation shown in the figures. For example, if a device in the figures is turned over, elements described as "below" or "beneath" other elements or features would then be "above" or "over" the other elements or features. Thus, the exemplary term "below" can encompass both positions and orientations of above and below. The devices may be otherwise oriented (rotated 90 degrees or at other orientations), and the spatially relative descriptors used herein interpreted accordingly. Likewise, descriptions of movement along and around various axes include various special device positions and orientations.
[00186] To avoid repetition in the figures and the descriptions of the various aspects and illustrative embodiments, it should be understood that many features are common to many aspects and embodiments. Omission of an aspect from a description or figure does not imply that the aspect is missing from embodiments that incorporate that aspect. Instead, the aspect may have been omitted for clarity and to avoid prolix description. In this context, the following applies to the rest of this description: If, in order to clarify the drawings, a figure contains reference signs which are not explained in the directly associated part of the description, then it is referred to previous or following description sections. Further, for reason of lucidity, if in a drawing not all features of a part are provided with reference signs it is referred to other drawings showing the same part. Like numbers in two or more figures represent the same or similar elements.
[00187] As used herein, the term “APP” is used to refer to machine-readable instruction code, without implying any limitations to a particular operating system and/or computing framework. The APP may be executed on a mobile device, in particular a mobile communication terminal, without being limited thereto.
[00188] Fig. 1 , Fig. 2, and Fig. 3 are perspective views of ski goggles 10 according to the invention. The ski goggles 10 comprise a lens 11 , a lens frame 20 that retains the lens 11 , a functional body 30, and a battery body 40.
[00189] The functional body 30 may be a functional unit. The battery body 40 may be a battery unit. Any reference to a functional/battery “body” may be considered to encompass or be a reference to a functional/battery “unit.” The functional/battery unit may respectively be a component that is attached to the lens 11 on a minor side thereof as an integral unit during manufacture. I.e. , the functional/batter unit may be formed and only then attached to the lens 11 .
[00190] The battery body 40 includes one or several batteries. The battery or batteries may include one or several rechargeable batteries, in particular lithium ion batteries. A battery housing 42 of the battery body 40 may be configured to be coupled to a connector portion 43 of the battery body 40 in a reversibly releasable manner. The battery housing 42 may be the housing of the one or several batteries. Conductive pathways are provided in the lens frame 20 to supply components of the functional body 40 with energy.
[00191] The functional body 30 comprises one or several sensors. The one or several sensors may include sensors configured to sense an orientation, position, and/or acceleration of the ski glasses 10. [00192] In exemplary embodiments, which will be described in more detail below, the one or more sensors comprise a sensor (e.g., a six-axis gyroscope) configured to sense a three-dimensional orientation of the ski goggles 10 and a three-dimensional acceleration of the ski goggles 10. The three-dimensional orientation may be defined by three angles (e.g., three Euler angles) defining the orientation relative to a reference frame. The three-dimensional acceleration may be defined by three acceleration values defining the acceleration along, e.g., three coordinate axes.
[00193] The functional body 30 comprises one or several integrated circuit(s) (IC(s)).The one or several IC(s) may comprise one or several application specific integrated circuit(s) (ASIC(s)), controller(s), processor(s), field programmable gate array(s) (FPGA(s)), or combinations thereof.
[00194] The IC(s) are configured to control a display 51 of the functional body 30. The display 51 extends through a lateral passage of the frame 20 into a space defined by the frame 20 and the lens 11. The display 51 is arranged such that it remains spaced from a contact area defined by an abutment rim 24 on the frame 20.
[00195] The display 51 is a transparent or semitransparent display. By virtue of this configuration, information output via the display 51 are visible as an overlay to a real- world scene that can be seen through the lens 11 , as illustrated in Fig. 4.
[00196] During field operation of the ski goggles 10, the IC(s) control the display 51 to output information. The information depends at least on sensor output of the one or several sensors, processed by the one or several IC(s). The information may include information relevant for skiing, such as a velocity, snow condition information, slope difficulty rating, etc. This information may include data derived by processing the sensor output and/or data retrieved via a communication interface circuit of the functional body 40. For illustration, velocity information may be derived by the IC(s) by integrating accelerometer measurements. Altitude may be derived by the IC(s) by processing a global navigation satellite system (GNSS) receiver output. Snow condition information may be derived from a temperature sensor output or may be obtained by querying a remote server, using a location derived from a GNSS receiver output. Slope difficulty rating may be obtained by querying a remote server, using a location derived from a GNSS receiver output. All these determination steps may be performed automatically by the ski goggles 10 and, more particularly, using the processing capability of the one or several IC(s).
[00197] The functional body 30 may comprise one or several wireless interface circuit(s). The functional body 30 may comprise a Bluetooth (BT) and/or wireless local area network (WLAN) interface, e.g., an IEEE 802.11 compliant interface. The wireless interface circuit(s) may be configured to transmit, under the control of the IC(s), results of processing a sensor output to a wireless communication terminal. The wireless interface circuit(s) may be configured to retrieve, using the results of the sensor output processing (e.g., a location derived thereby), information for displaying via the display 51.
[00198] The ski goggles 10 comprise transducers that allow the ski goggles 10 to play audio signals and to receive and process voice signals. For illustration, microphones and speakers may be integrated into the ski goggles 10. The wireless communication interface(s) of the ski goggles 10 allow audio and/or video signals to be relayed between the ski goggles 10 and a wireless communication terminal. This improves convenience for accepting incoming calls and/or establishing outgoing calls while skiing.
[00199] A camera having a camera objective 15 may be provided on the lens frame 20. The lens 11 has an aperture or recess 14 that is positioned such that, when the lens 11 is retained by the lens frame 20, the camera objective 15 is positioned at the aperture or recess 14, leaving the camera objective 15 unobstructed by the lens. The lens frame 20 has image data paths that allow the IC(s) of the functional body to receive image frames captured by the camera. At least some of the image frames may be transmitted to a wireless communication terminal via the wireless interface circuit(s).
[00200] As shown in Fig. 1 , Fig. 2, and Fig. 3, the battery body 40 and functional body 30 are provided on opposing lateral sides of the lens frame 20. The functional body 30 is attached to a first side 21 of the lens frame 20. The battery body 40 is attached to a second side 22 of the lens frame 20, which is opposite the first side 21. The first side and the second side 22 may be the minor (i.e. , shorter) sides of the lens frame 20 that extend to the left and right when the goggles are worn on a head with horizontal facing direction. [00201] The attachment of the functional body 30 to the lens frame 20 may allow movement (e.g., slight tilting or slight displacement) of the functional body 30 relative to the lens frame 20. The attachment of the battery body 40 to the lens frame 20 may allow movement (e.g., slight tilting or slight displacement) of the battery body 40 relative to the lens frame 20. Different head sizes and/or head shapes and/or helmet sizes and/or helmet shapes can be more easily accommodated thereby, without adverse effects on wearer comfort.
[00202] To accommodate the movement of the functional body 30 and/or battery body 40 relative to the lens frame 20, there may be provided soft linking materials that attach the functional body 30 to the lens frame 20 and the battery body 40 to the lens frame 20. The soft linking material may be a plastic material. The soft linking material may be a polymeric material. The linking material may implement a living hinge that accommodates the relative movement of the functional body 30 and/or battery body 40 relative to the lens frame 20.
[00203] Attachment of the functional body 30 and of the battery body 40 to the lens frame 20 may be attained by means of injection molding, ultrasonic welding, or other techniques.
[00204] The lens frame 20 may have first and second attachment features on the first and second sides 21 , 22, respectively, to facilitate attachment of the functional body 30 and battery body 40. For illustration, and as best seen in Fig. 4, the lens frame 20 may have a stepped recess on which the functional body 30 can be attached to the lens frame 20. Suitable mating engagement means ensure the desired degree of movability to accommodate different head sizes and/or shapes. The coupling of the lens frame 20 to the battery body 40 ensures a conductive pathway from an output terminal of the battery body 40 to the conductors of the frame that supply power to the functional body 30 and camera. The functional body 30 is coupled to the lens frame 20 so as to ensure a conductive pathway between the battery body 40 and the electronic components of the functional body 30. The functional body 30 is coupled to the lens frame 20 so as to ensure a data communication pathway between the camera and the IC(s) 52 of the functional body 30.
[00205] The functional body 30 may comprise a guide portion 39 for guiding a strap 13 of the ski goggles on and along the functional body 30. The battery body 40 may comprise another guide portion 49 for guiding the strap 13 of the ski goggles 10 on and along the battery body 40. The guide portions 39, 49 may respectively define a passage that allows the strap 13 to pass therethrough.
[00206] The functional body 30 may comprise one or several hard keys 31 disposed on an outer surface thereof, e.g., at a forward end of the outer surface adjacent the lens 11. Additionally or alternatively, the battery body 40 may comprise one or several hard keys 41 disposed on an outer surface thereof, e.g., at a forward end of the outer surface adjacent the lens 11. The hard keys 31 , 41 may be assigned various frequently used functions, such as accepting an incoming call, dialing, etc.
[00207] The lens 11 can be retained by the lens frame 20 in a reversibly removable manner. The lens 11 can be temporarily removed for, e.g., cleaning, without affecting any of the electrical connections, and can be re-attached subsequently. The lens 11 can be exchanged, e.g. when needed for repair purposes.
[00208] The lens 11 may be configured to improve vision in snow, thereby making it easier to perform obstacle detection. The lens 11 may enhance colors brightness and contrast. The lens 11 may also be configured to automatically change color with sunlight intensity. Contrary to standard blue blocker filters, the lens 11 presents a specific spectral curve that acts as a blue attenuator but still allows some blue light to reach the eye, e.g., in a spectral range between 380-500 nm. This makes it easier for the wearer to read snow irregularities like bumps and hollows. Skiing and snowboarding become safer.
[00209] Fig. 5 is a functional block diagram of a functional body 30. It is understood that additional and/or alternative components may be used in other embodiments.
[00210] The functional body comprises the display 51 protruding from a wall 32. The wall 32 may define a housing having a cavity 33 in which various components are housing. These components include:
- processor(s) or other IC(s) 52 for performing the processing and control functions described herein; one or several wireless interface circuit(s) 54, 55, such as a Bluetooth chipset 54 and/or a chipset 55 configured for communication in accordance with an IEEE 802.11 standard;
- one or several sensors, such as a gyroscope 58 (which may be a three-axis gyroscope or a so-called six-axis gyroscope operative to sense both 3D orientation and 3D acceleration) and/or a GNSS receiver 57;
- additional interface circuits, such as a display driver circuit 53 for driving the display 51 , a camera interface 56 for receiving image frames from the camera, and/or an audio circuit 59 for controlling speakers and processing microphone output signals.
[00211] In any of the embodiments, the display 51 may be or may comprise an optical waveguide display. A main body of the optical waveguide display may be embedded in or arranged on a circuit board of the functional body 30. The display 51 extends over part of the lens 11 , so as to selectively display information as an overlay over part of a field of view.
[00212] When the display 51 is or comprises an optical waveguide display, the display driver circuit 51 may be or may comprise a light engine. The light engine may be operative to generate optical signals that are being guided along the optical waveguide display until they information is output as an overlay to the real-world view. Integration of the light engine into the functional body 30 facilitates balancing of the ski goggles (and, thus, safety and comfort) and prevents non-transparent components from blocking part of the user’s field of view otherwise visible through the lens 11 .
[00213] The display 51 is independent of and separate from the lens 11. In particular, when the lens 11 is exchanged or temporarily removed, it is not required to disconnect any electrical connection to the display. The lens 11 does not have to include any conductive traces interfaced with the battery body 40 and/or functional body 30.
[00214] In accordance with an embodiment, the invention provides augmented reality (AR) intelligent ski goggles 10. The AR ski goggle includes: the lens frame 20, the battery body 30 connected to the lens frame 20, and the functional body 30. The functional body 30 comprises a light waveguide display module 51 , a camera module, a communication module, a GPS module, a further sensor module, a voice module, and at least one chip. The light waveguide display 51 is positioned within the circumference of the lens frame 11 and connected to a circuit board of the functional body 30. The camera is arranged so that the camera objective 15 is located at an upper end of the lens 11 in a laterally central position. The wiring for the camera is hidden in the lens structure 20 and connected to the functional body 30. The communication module 54, 55, GPS module 57, sensor unit 58, and voice module 59 are communicatively connected to the at least one chip 52.
[00215] The AR ski goggles 10 integrate several different interactive devices and data acquisition devices to achieve many different intelligent functions, which improve functionality for the wearer while skiing.
[00216] Two buttons 31 , 41 can be arranged on each side of the lens frame 20 for operation and use. In an embodiment, these buttons provide the following functions:
- SOS button: When a user presses this SOS button for more than a threshold time (e.g., 5 seconds), an SOS signal will broadcast to other users within a certain range (e.g., 5 km) who wear the goggles. Additionally or alternatively, an emergency number is called. This function allows the user to be found by the nearest people and professional rescue team.
- Photo I Video button: when the user presses the photo and video button, it allows the user to take a photo or start video.
- Answering call I start music playing button: In call answering and music playing, when the user presses this button, the user will answer a call-in, or start the music.
- Chat group button: The user can create or join a chat group in app, when the user presses this chat group button, it allows the user to talk to a ski team, akin to dedicated portable communication terminals.
[00217] The various components of the ski goggles 10 may have the following functions: communication components 54, 55: A Bluetooth module 54 (e.g., Bluetooth 4.2 module) is configured for data communication with wireless communication terminals (calls, music functions). A WiFi (e.g., a WiFi 2.4Ghz) module 55 is configured to signal connection with wireless communication terminals, sharing a wireless communication terminals hotspot, for data transmission;
- GPS module 57: used for location determination and map use for positioning;
- sensor module 58: a six-axis gyroscope or another inertial measurement unit (IMU) (e.g., a Micro-Electro-Mechanical-Systems (MEMS)-IMU) can be used to determine the user's posture and glasses state judgment
- camera module with camera and camera interface 56: for camera image frame capture (e.g., high-definition images);
- display 51 : Array light waveguide configured to display usage data and for data interaction
- voice module 59: for voice interaction.
- noise-resistant speaker (integrated into, e.g., functional body 30 and/or battery body 40): for audio output.
[00218] Fig. 6 is a block diagram of a system 60. The system 60 comprises the ski goggles 10 according to the invention. The system 60 comprises a wireless communication terminal 70. The wireless communication terminal 70 may be a cellphone, without being limited thereto. The wireless communication terminal 70 may be configured for bi-directional communication with the ski goggles 10. The wireless communication terminal 70 may be configured to serve as a communication hub, facilitating retrieval of skiing related data (such as avalanche risk, snow condition information, slope difficulty rating, and/or weather conditions) from one or several servers 81 , 82 via a wide area network (WAN) 80. The communication terminal 70 may also enable audio, video, and/or data communication between the wearer of the ski goggles and other mobile communication terminals 78, 79 of a cellular network.
[00219] The communication terminal 70 comprises a wireless communication chipset 74 (which may include a Bluetooth and/or WiFi chip) configured to perform bi-directional data communication with the ski goggles 10. The wireless communication terminal 70 comprises a terminal display 72. The wireless communication terminal 70 comprises at least one processor 73 configured to process the data transmission from the ski goggles 10 and use the processed data transmission to perform activity tracking. Results of the activity tracking may be output via the terminal display 72.
[00220] The processor(s) 73 may execute machine-readable instruction code, which may be embodied in a non-transitory storage medium. The machine-readable instruction code may cause the processor(s) 73 to perform the activity tracking and/or other functions. For illustration, the following functions may be provided by the communication terminal 70:
- tracking ski performance: By tracking and analyzing the data collected from the ski goggles 10 (such as velocity data and/or altitude data), the user can review a ski or snowboarding performance via the machine-readable instruction code. The user’s skill score can be ranked in, e.g., social networking scenarios and can be used for online (e.g., cloud-based) ski competition scenarios.
- map information: navigation on ski slope and interesting points, like cable lift, restaurant and friend location can be displayed. The GPS module of the ski goggles provides location information. A navigation routine can be executed to determine a route to the destination by calculating the shortest distance according to the wearer’s level of skill. For illustration, the navigation routine will prevent the wearer of the ski goggles from being guided to a slope that has a difficulty rating (e.g., black slope) that exceeds the wearer’s skill level.
- social functions and networking: The machine-readable instruction code can allow users to create a team and team chat room, it allows the team to talk through the chat room directly without calling via phone. This is particularly useful for a ski instructor and his/her student group, or for parents and their children.
- making photos and videos and sharing them on social media or with friends: The machine-readable instruction code can allow other users to watch the video and photo online instantly.
- ski events and calendars: The machine-readable instruction code can allow users to publish his/her ski trip that can be viewed by other users on the event calendar, and the other users can select to join his ski plan. - SOS and security: The machine-readable instruction code collects the information of avalanche risk and gives notice when the user steps into the risky area. The machine-readable instruction code alerts the user in avalanche accidents. An SOS broadcast will send an emergency alert and the SOS location to other users within a certain range (e.g., 5 km), and will also call the emergency number to obtain professional help.
[00221] Various effects and advantages are attained by the ski goggles disclosed herein as compared to conventional ski goggles comprising a display, such as those of WO 2011/044680 A1 and WO 2013/000075 A1 . As compared to both WO 2011/044680 A1 and WO 2013/000075 A1 , the ski goggles disclosed herein provide a transparent or semitransparent display. Thus, the display does not block part of the field of view visible through the lens. This enhances safety and comfort. Moreover, as compared to the ski goggles of WO 2011/044680 A1 , the display of the present ski goggles is comprised by the functional body. Thus, the risk of inadvertent loss of display function that can be caused by wiring along part of the display frame and display mounting structure is reduced by the ski goggles disclosed herein, which in turn increases safety and comfort. Moreover, as compared to the ski goggles of WO 2013/000075 A1 , the ski goggles are operative such that the processed sensor output is not only used to control the provision of the information via the display, but also to control the wireless interface circuit. This increases safety and comfort, as the data transmission is based (at least) on the processed sensor output, allowing it to be initiated automatically and/or to include information critical in safety-relevant applications.
[00222] This description and the accompanying drawings that illustrate aspects and embodiments of the present invention should not be taken as limiting-the claims defining the protected invention. In other words, while the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. Various mechanical, compositional, structural, electrical, and operational changes may be made without departing from the spirit and scope of this description and the claims. In some instances, well-known circuits, structures and techniques have not been shown in detail in order not to obscure the invention. Thus, it will be understood that changes and modifications may be made by those of ordinary skill within the scope and spirit of the following claims. In particular, the present invention covers further embodiments with any combination of features from different embodiments described above and below.
For example, it is possible to operate the invention in an embodiment wherein:
- the ski goggles 10 have additional, fewer, or alternative wireless interface circuits than described in association with the drawings;
- the ski goggles 10 have additional or alternative sensors (e.g., in which the ski goggles 10 do not have an acceleration sensor but rely on position sensing for determining velocity).
[00223] Fig. 7 is an illustration of a screenshot from a light waveguide display (which is one implementation of an AR display) of the inventive AR ski goggles showing general information available in connection with the respective APP. In the upper section of the AR display there are displayed as general information e.g. weather information (e.g. sunshine or cloudiness or rain/snow and the temperature), device information (e.g. date and time, user information, WI-FI Bluetooth strength, battery level, altitude and compass data).
[00224] The ski goggles 10 may be operative such that the display 51 is controlled to output choices of different usage modes. The ski goggles 10 may be operative to enable a selection of the usage modes, e.g., by means of hard keys provided on the ski goggles and/or by means of voice input. The ski goggles 10 may be operative such that different monitoring functions are performed, using camera images and/or sensor output captured by the ski goggles 10, depending on which usage mode is selected.
[00225] Fig. 8 shows a screenshot that represents choices available for a skiing mode, i.e. piste (i.e. regular pistes), backcountry (i.e. touring off-piste) or park (i.e. snowpark with ski or snowboard jump facilities). The ski goggles 10 and/or the wireless communication terminal 70 may be operative to control a user interface (e.g., the display 51 and/or a user interface of the wireless communication terminal 70) to provide information on the available usage modes and/or enable a selection thereof. The skiing goggles 10 and/or the wireless communication terminal 70 may be operative to record parameters (such as location traces and/or velocity traces and/or other parameters) derived from the processed sensor output, with the parameters that are being recorded being dependent on the selected usage mode. At least some of the parameters may be shown on the display 51 as overlay on the real-world view. The skiing goggles 10 and/or the wireless communication terminal 70 may be operative to perform route guidance and/or navigation functions, with the display 51 being controlled to output information relating to route guidance or navigation. The route guidance or navigation may be performed in relation to objects that are geo-stationary (such as landmarks or other points of interest (POIs)) or in relation to objects that can move themselves. An example for the latter scenario comprises navigation or route guidance for directing a user of the ski goggles 10 to another person who has agreed to share his/her geocoordinates with the user of the ski goggles 10. The routing and navigation may be performed in a manner which takes into account the routes accessible to a skier, optionally also the selected skiing mode and/or skill levels. For example, the routing and navigation may be implemented in such a manner that a user of the ski goggles 10 is guided to a target (which may be stationary, such as a landmark or POI, or movable, such as a fellow skier) in a manner which ensures that the pathway does not require skiing on pathways not compatible with the user’s skill level and/or which respect the selected usage mode. For illustration, when a usage mode has been selected for skiing on designated ski slopes, the routing and navigation may be implemented in such a manner that the user of the ski goggles 10 is not guided along pathways that are not in accordance with the selected usage mode. Routing and navigation may be performed using techniques known from the field of vehicle navigation (but applied to a directed graph network that represents pathways accessible to skiers), such as Dijkstra or A* route search techniques. Conformity with the selected usage mode and/or user skill levels can be ensured by either hard avoid options (eliminating parts of the network representing the skiing paths) and/or by imposing high costs (in the sense of a cost function optimized in the route search) for graph links that are not in conformity with the selected usage mode and/or user skill level.
[00226] Figs. 9 to 14 are screenshots that relate to a map navigation functions. By means of GNSS (e.g., the GPS) and the other sensors (such as gyroscope and/or MEMS IMU) of the ski goggles 10, it is possible to localize the ski goggles 10 and measure the movement speed. Fig. 9 shows an exemplary display graphics for navigation in real time, immersive navigation in real time and speed in real time. The ski goggles may be operative to update the output information (i.e. , the content of what is being displayed) in an ongoing basis (i.e., continually, e.g., with a repetition interval that may be fixed or variable). Fig. 10 illustrates an exemplary display graphics for outputting a current position of ski comrades. The real time position of the ski comrades and their current speed can be shown on the map, e.g., on the a right or left AR side portion of the display 51. The real time positions may be displayed using their GNSS (e.g., GPS)- based locations, which may be communicatively shared between the various ski goggles, either directly or using communication terminals.
[00227] Fig. 11 is a screenshot that shows how the ski goggles 10 are operative to assist the user to navigate to the position of a friend (in the sense of a contact who has agreed to share GNSS coordinates with the user of the ski goggles 10). When the user has found the current position of the person who has agreed to share the GNSS coordinates (such a person will also be referred to as a ski comrade) (as shown in the upper screen shot), the ski goggles 10 enable the user to actuate a navigation symbol (on the AR display or the terminal display) in order to create a live navigation in order to follow a track of the ski comrade (middle and lower screen shots). The navigation may be coordinated with the skiing skills of the ski driver in order to create a track suited for him/her which e.g. does not guide a beginner to pistes for advanced ski drivers. The ski goggles 10 may also allow the user to choose the telephone or video call symbol on the display 51 of the ski goggles or of the wireless communication terminal 70 in order to make live telephone or video calls.
[00228] Fig. 12 is a screenshot that shows how points of interest (POIs), such as reference points or other landmarks, are displayed. The ski goggles 10 and/or the wireless communication terminal 70 may allow the user to choose on the AR or the APP map (i.e. on the wireless communication terminal - which may generally display the same information as the AR display) a point of interest, such as a restaurant, a cable car or a first aid station in order to show the location and the opening times of the point of reference as well as a real time navigation track to the respective destination.
[00229] Fig. 13 is a screenshot that shows the functionality of area recognition. If the user of the ski goggles 10 directs the camera of the ski goggles 10 to a landmark, information regarding the landmark may be shown on the display. For example, if the user of the ski goggles 10 directs the camera onto a mountain, the name of the mountain will be shown on the AR display 51 of the ski goggles and/or the display of the communication terminal 60. When the user of the ski goggles 10 directs the camera onto a landmark such as a restaurant or a cable car, the ski goggles 10 and/or the display of the communication terminal 60 may be operative to display the name of the latter on the AR display 51 and/or the display of the communication terminal. GNSS (e.g., GPS) position and/or an artificial intelligence (Al)-based recognition system may be used, as will be described in more detail below.
[00230] The ski goggles 10, optionally in combination with the wireless communication terminal 70 and/or a computing system remote from the ski goggles 10, may be operative to perform image processing tasks. Al-based techniques may be used. For illustration, person recognition and/or landmark recognition and/or slope difficulty determination may be performed, using GNSS sensor data (such as GNSS-based coordinates) and/or image data as input of the processing. The output of the processing may specify an identifier for a person and/or landmark and/or slope difficulty, which may be used to control the display 51. Alternatively or additionally, direct communication between ski goggles 10 according to the invention may be used to identify users in proximity to each other and display information relating to such users on the display 51 .
[00231] Fig. 14 is a screenshot that shows the functionality of AR related recognition of users of ski goggles 10. By means of GNSS (e.g., GPS) localization and the recognition from ski goggles to ski goggles the user names or other identifiers of other users may be identified and displayed.
[00232] The ski goggles 10 may be operative to collect and aggregate data relating to the activity performed. The parameters collected and aggregated may depend on the selected usage mode if the ski goggles 10 allow different usage modes to be selected. The ski goggles 10 may use the processed sensor output to determine the activity- related parameters. The parameters may be stored in the ski goggles 10 and/or the wireless communication terminal 70 and/or a computing system separate therefrom.
[00233] Figs. 15 and 16 are screenshots that are directed to such a further functionality of displaying sports data. The ski goggles 10 is provided with a sensor which enables to track and collect the sports data of the user of the ski goggles 10. Such data may comprise, for example, speed, maximum speed, altimeters, skiing kilometers and so forth. The ski goggles 10 may be operative to transmit any one or any combination of these parameter to the AR display after each skiing activity (e.g., downhill along a ski slope). The sports-related movement data may be divided with respect to the ski mode into piste, backcountry and park. I.e., the aggregation may be performed separately for each usage mode. Fig. 16 illustrates the skiing data in the park mode, i.e. displaying for example a jump length, a jump height, a flight time and a flip (in degrees). As explained above, the type of parameters that are recorded may be dependent on the selected usage mode.
[00234] The ski goggles 10 and/or APP executed on the wireless communication terminal 70 may be operative to perform safety-relevant functions. The ski goggles 10 and/or APP executed on the wireless communication terminal 70 may be operative to determine, responsive to a user input and/or responsive to the processed sensor output, whether a safety-relevant function (such as triggering an emergency call) is to be performed. The safety-related functions may comprise the provision of danger warnings.
[00235] Fig. 17 is a screenshot that is directed to a functionality of an SOS emergency call. If the user presses an SOS button on the ski goggles (or on the AR display or the terminal display) and holds down the latter, an emergency call is sent to persons in the vicinity of a predetermined distance (e.g. 3 kilometers) wearing the same ski goggles and who can then decide whether it is necessary to call for help. Alternatively or additionally, the emergency call functionality may be triggered automatically responsive to the processed sensor output indicating that a situation (such as accident and/or avalanche) has occurred that may have caused the user of the ski goggles 10 to be unconscious.
[00236] Fig. 18 is a screenshot that is directed to the functionality of displaying danger warnings on particularly the AR display of the ski goggles 10 or the display of the wireless communication terminal 70. In the upper screenshot a danger indicator is shown as e.g. color (e.g. red)-marked area if the user of the ski goggles 10 is near to cliffs, woods or other potentially dangerous areas. The lower screenshot represents a warning regarding speed reduction on the AR display when the ski driver is too fast or near to a crowded area. Crowded areas may be identified using Al-based image processing techniques (as will be explained in more detail below) or by processing GNSS traces of several ski goggles.
[00237] The ski goggles 10 and/or the wireless communication terminal 70 may enable different usage modes and/or different skill levels to be specified by the user of the ski goggles 10. The ski goggles 10 and/or the wireless communication terminal 70 may enable the user of the ski goggles 10 to make a selection from a pre-defined set of usage modes and a pre-defined set of skill levels. The ski goggles 10 and/or the wireless communication terminal 70 may use this information for, e.g., determining avoid and/or prefer options for routing and navigation, for determining which parameters are to be recorded and/or aggregated, or for other purposes.
[00238] Fig. 19 is a screenshot that shows a display graphics enabling the ski mode and ski level choice for the ski goggles 10 when recording skiing data (first functional module). Firstly, the user of the ski goggles 10 is enabled to choose between the three ski modes piste, backcountry and park, in order to record the performance of the ski driver in different ski parameters as for example speed, driving time, altitude, turns and/or other. The three modes have different functional/technical requirements for the ski driver. Secondly, the user of the ski goggles 10 is enabled to choose between the levels beginner, intermediate and expert. The three modes cause different data (e.g., different physical parameters derived from the processed sensor output) to be recorded.
[00239] In the mode piste (cf. Fig. 20 on the right), data may be recorded that may be provided as a total summary of a ski day. The data in this mode may comprise any one or any combination of: a total ski distance (e.g. in km, a duration (on the mountain), a total ski time (e.g. in h), an average ski speed (e.g. in km/h), a number of runs, a biggest slope and a highest altitude. Further, for each run, the ski goggles 10 and/or wireless communication terminal 70 may determine any one or any combination of: the ski time, the ski distance (e.g. in km), the fastest speed (e.g. in km/h), the altitude and the slope. Further, for the latest (i.e., most recent), the ski goggles 10 and/or wireless communication terminal 70may record the recorded the ski time, the ski distance (e.g. in km), the fastest speed (e.g. in km/h), the altitude and the slope.
[00240] In the mode park (cf. Fig. 21), the ski goggles 10 and/or wireless communication terminal 70 may determine and record, for each run, any one or any combination of the following: the average ski speed (e.g. in km/h), a length of a jump (e.g. in m), a height of a jump (e.g. in m), a time in the air (e.g. in s) and a number of spins (e.g. 360°, 720°, 1080° etc.). Further, for the total summary of a ski day, the ski goggles 10 and/or wireless communication terminal 70 may determine and record a longest jump distance, a maximum spin numbers, a longest time in the air and a highest jump).
[00241] In the mode backcountry (cf. Fig. 20 on the left), the ski goggles 10 and/or wireless communication terminal 70 may determine and record any one or any combination of: the ski time, the ski distance (e.g. in km), the fastest speed (e.g. in km/h), the average speed (e.g. in km/h), the altitude and a vertical drop (altitude difference from top to bottom).
[00242] The machine-readable instruction code (also referred to APP herein) executed by the wireless communication terminal 70 and/or the ski goggles 10 may also enable sharing of information among different users of ski goggles having the configuration disclosed herein, and/or among different users of wireless communication terminals 60 executing the machine-readable instruction code. The machine-readable instruction code may be operative to allow groups of users to be defined via the ski goggles 10 and/or the wireless communication terminal 70. The machine-readable instruction code may be operative to enable, when executed by the wireless communication terminal 70 and/or the ski goggles, transmission of data derived from the processed sensor output (such as GNSS-based position information and/or skiing activity-related data) to share such data among selected users of the ski goggles, e.g., users assigned to a same group. The machine-readable instruction code may additionally or alternatively be operative to enable, when executed by the wireless communication terminal 70 and/or the ski goggles, communication between the users assigned to the same group. The communication enabled by the machine-readable instruction code may comprise one, several, or all of: text message communication, voice communication, image data transmission, video data transmission, transmission of activity-related data. The machine-readable instruction code may be operative to enable, when executed by the wireless communication terminal 70 and/or the ski goggles, the generation of information based on the processed sensor data and camera images captured by the ski goggles 10 and provision of the information for retrieval by users of the same group and/or users that do not necessarily have to belong the same group.
[00243] Thus, the APP can offer the possibility of performing data processing related to skiing in groups (and may have a functional module for the group-related activities) in a further mode sessions (cf. Fig. 21 ). In the mode sessions friends may be gathered who want to go skiing in a same skiing resort on a same day. The machine-readable instruction code may enable a user or several users to choose a date in order to start a group specifying one, several, or all of: a time, a location, a maximum number of skiers/members, a (required) ski level and whether the group shall be open to the public (cf. Figs. 22 and 23). The machine-readable instruction code may enable other users to view the group and allow them to join if they decide to do so. The machine-readable instruction code may enable an initiator to also invite friends into the group (cf. Fig. 23 on the right). The machine-readable instruction code may enable the invited friends to choose whether they want to join the group or not, when they see the group information. The machine-readable instruction code may be operative to transfer the formed group to a next communication function, namely a chat group.
[00244] Alternatively or additionally, the machine-readable instruction code may enable the user to decide to join the group of another user. The users may see the groups of other users with which they have made friends and may decide whether they want to join these groups (cf. Fig. 24).
[00245] Alternatively or additionally, the machine-readable instruction code may enable the group administrator who has the authorization to invite users and exclude users to determine which user is allowed to join which user group. Alternatively or additionally, the machine-readable instruction code may enable the group administrator to set a maximum size of the group (cf. Fig. 25).
[00246] The machine-readable instruction code, when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative such that a group formation is automatically performed in a communication module and in the community module of the APP (cf. Fig. 26). The community module may comprise two parts, namely friends and chat group. When a group has successfully been formed in the module sessions or if a user joins another group, that user will see the chat group in the community. The group chat function is implemented in the chat group. There are two possibilities to implement the chat, namely via the APP on the wireless communication terminal 70 and/or via the ski goggles 10.
[00247] The machine-readable instruction code, when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative such that when chatting via the APP a user is enabled to send a voice message directly from the chat group. This may be performed responsive to a command, e.g., by tactile command (e.g., by pressing down and holding down the microphone logo and clicking onto other users in order to play a user’s voice) or via another input modality, such as a voice command. In order to chat via the ski goggles 10, the user wears the ski goggles 10 and is enabled to hear the voice directly via the speakers of the ski goggles 10. On one side of the ski goggles frame there can be arranged a button for the voice communication in the chatroom (group chat button). The user presses down and holds down the group chat button in order to speak and the other members of the chat room may receive and hear the message. Other ways of activating this functions are possible, e.g., by way of voice command.
[00248] As a further option, the machine-readable instruction code, when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative such that users in the chat groups may send photographs (cf. Fig. 27). This may be done in such a manner that sharing of photographs is enabled. Other data sharing may be supported as well, such as sharing activity-related data.
[00249] The machine-readable instruction code, when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to perform map- related functions. The map-related functions may comprise any one or any combination of: identifying paths accessible to a user of the ski goggles (based on, e.g., map data, crowdsourced data from other ski goggles users, the processed sensor output and/or images as processed by an Al model); performing navigation tasks, such as route search and/or route guidance; performing POI identification; performing landmark identification, without being limited thereto. Al-based processing techniques may be used in this context.
[00250] Thus, the machine-readable instruction code, when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to provide a further functionality that is implemented as a map navigation module (i.e., a further functional module). The machine-readable instruction code, when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative such that a ski track and mountain map and navigation are generated. By using the current ski resort's data of snowcast and mountain map data, optionally in combination with crowdsourced terrain data processed with an Al program, such information may be generated. Satellite and/or aerial imagery may be used by the machine-readable instruction code for performing or assisting in the localization. Thus, the machine- readable instruction code, when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to build up a unique ski tracks map and provide corresponding navigation. Secondly, by means of GNSS (e.g., GPS) and/or the processed sensor output of the other sensors of the ski goggles 10, the speed and the displacement may be determined. The navigation takes the level of the skier into consideration and creates a suitable track for the skier without guiding, for example a beginner onto the intermediate piste. Thirdly, the current position of the user’s ski friends and their current driving speed may be displayed on the map generated by the machine-readable instruction code (cf. Fig. 28).
[00251] As previously explained, navigation tasks may comprise navigation to a movable target, such as a location of another skier who has agreed to share his/her location with the user of the ski goggles.
[00252] In order to navigate to the location of such a skier, the machine-readable instruction code, when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to enable the user of the ski goggles 10 to search the current position of this friend and click onto the navigation logo when the position is found in order to create a live navigation and to follow the track to the position of this friend (cf. Fig. 29).
[00253] The map-related functions may also comprise displaying POIs, such as points of orientation or other landmarks. The machine-readable instruction code, when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to enable the user to choose, via the ski goggles 10 or an APP-generated map on the wireless communication terminal 10, such a POI, as for example a restaurant, a cable car or a first aid station in order to display the location and the opening hours of the point of orientation as well as real time navigation track to the respective destination (cf. Fig. 30). Fig. 31 shows an exemplary navigation track to a restaurant chosen by the user.
[00254] The machine-readable instruction code, when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to enable the type of information provided by the display 51 of the ski goggles 10 to be configured. In such a configuration mode, the machine-readable instruction code may allow the user to set which information is to be displayed via the display 51. For illustration, in the configuration mode, the user may be enabled to freely choose the AR display settings in the machine-readable instruction code executed on the wireless communication terminal 70 in order to get the most suited and comfortable amount of information in order not to impede the joy of skiing (cf. Fig. 32). Such a configuration also is useful for enhanced safety. [00255] Alternatively or additionally, the machine-readable instruction code, when executed by the ski goggles 10 and/or the wireless communication terminal 70, may be operative to enable a user to aggregate and share information. For illustration, an information board or information wall may be generated and provided for access. . This function allows users of the ski goggles 10 to upload ski photos into the personal information, to leave messages, to record voice protocols and get into contact with other ski comrades/friends (cf. Fig. 33).
[00256] As already discussed, artificial intelligence (Al)-based processing may be applied to at least part of the data captured using the ski goggles 10. The Al-based processing may be implemented in the ski goggles 10, the wireless communication terminal 70, or a computing resource 81 , 82 separate therefrom. The Al-based processing may be performed for various purposes, such as for performing a navigation function, object identification, communication functions, and/or for safety-relevant functions. For illustration, the Al-based processing may comprise a classifier to classify terrain, possible dangers, and difficulties (e.g., slope difficulty levels).
[00257] Fig. 34 shows a system comprising the ski goggles 10 and at least one processing circuit 90. While the at least one processing circuit 90 is shown separately from the ski goggles 10 for clarity, the at least one processing circuit 90 may comprise one or several processing circuits that are incorporated into the functional body of the ski goggles 10. Alternatively or additionally, the at least one processing circuit 90 may comprise one or several processing circuits that are incorporated into the wireless communication terminal 70 or a computing resource 81 , 82 separate from the ski goggles 10 and the wireless communication terminal 70.
[00258] The at least one processing circuit 90 is operative to execute one or several Al models 91 . The one or several Al models 91 may comprise an Al model operative to receive data captured using the ski goggles 92. The input data 92 to the one or several Al models 91 may comprise one or several images, sensor output of one or several sensors of the ski goggles 10, or processed sensor output generated by the ski goggles 10 or separately therefrom in a preprocessing step. The one or several Al models 91 may have an input layer or other input operative to receive the data captured by the ski goggles 10 or data obtained therefrom by preprocessing. The Al model input 102 may be based on data captured by the camera and other sensors of the ski goggles 10. The one or several Al models 90 may comprise classifiers or other Al models. [00259] An Al model output 93 of the one or several Al models 91 may be provided to an output generation 94. The output generation 94 may generate output for provision via a communication interface and/or display (such as the display 51 of the ski goggles or the terminal display of the wireless communication terminal 70) and/or audio device.
[00260] The output generation 94 may provide the output 95 to a GUI controller 96. Alternatively or additionally, the output generation 94 may provide the output 95 to a communication interface control.
[00261] The GUI controller 96 may cause the output to be displayed via the display 51 of the ski goggles or the terminal display of the wireless communication terminal 70.
[00262] The at least one processing circuit 90 may comprise one or several integrated circuits, such as integrated semiconductor circuits (e.g., processor(s), controller(s), FPGAs, ASICs) and/or circuits comprising quantum gates. In particular, any reference to artificial intelligence or machine learning as used herein also encompasses quantum machine learning techniques.
[00263] The at least one Al model may comprise a recurrent neural network (RNN) or a convolutional neural network (CNN). The RNN or CNN may have an input layer operative to receive image data captured by the ski goggles 10 and/or other sensor data (such as the processed sensor output or the sensor output prior to processing). The RNN or CNN may be operative to perform a classification or identification task. For illustration, the RNN or CNN may be operative to identify potentially hazardous areas within an image. The output may be used to provide, as overlay to the real-world view or an image, a marking designating a hazardous area or an area otherwise unsuitable for the user of the ski goggles. Alternatively or additionally, the RNN or CNN may be operative to identify an avalanche risk based on the image and other sensor data. Alternatively or additionally, the RNN or CNN may be operative to identify people shown in images, which is useful for communication functions.
[00264] Fig. 35 shows an Al model 100 that may be executed by the at least one processing circuit 90. The Al model 100 has an input layer 101 , an output layer 102, and several hidden layers 103. The Al model 100 is operative to process inputs 104, 105 received by the input layer 101 to generate the output 106 provided by the output layer.
[00265] In Fig. 35, the input of the Al model 100 comprises GNSS-based data 104 and image data 105. The output 106 may comprise a classification relevant for safety- related functions, such as an avalanche risk, a terrain difficulty (such as a skiing slope difficulty level), or other data that is useful in the various functions disclosed herein.
[00266] Fig. 36 shows the Al model 100 that may be executed by the at least one processing circuit 90 when applied to image data 105 showing a person. The Al model output layer 106 may in this case provide an output 106 including an identifier for that person. Thus, the Al model 100 may operate as an object recognition module.
[00267] The Al model 100 may be operative to process time-series data. This may be particularly useful for situation analysis, e.g., for identifying hazardous situations. LSTM cells and/or GRUs may be stacked in the Al model 100 to process the time series data.
[00268] Fig. 37 shows an implementation of a LSTM cell 110 that may be used in an Al model 91 operative to process time-series data. Such processing may be useful for, e.g., identifying safety-critical situations. A stack of LSTM cells 110 may be provided in the Al model 91 . In Fig. 37, C designates a cell state. The parameter h designates a hidden state. The parameter x designates an input. The subscript respectively designates the time. The subscript t designates the time of the input processed by the cell illustrated in Fig. 37. The subscript t - 1 designates the preceding time.
[00269] During training, parameters of the LSTM cell or of several LSTM cells arranged in a LSTM cell stack may be trained. The training may comprise training of a forget gate (designated by ft), an input gate (designated by it) and an output gate (designated by ot).
[00270] Training the various gates may comprise training the weight parameters and bias parameters of each of the gates of the LSTM cell.
[00271] The training of the at least one Al model 91 may be performed using training data. The training data is training data obtained from one or several cameras and/or additional sensors, as provided in the ski goggles 10. [00272] The training data may be split into a training set (which may include, e.g., 70% of the training data), into a validation set (which may include, e.g., 15% of the training data), and into a test set (which may include, e.g., 15% of the training data). Data augmentation may be performed, to account for the fact that critical slope situations (such as avalanche risk situations or cliffs) are fortunately rare compared to safe situations.
[00273] The output of the at least one Al model 91 may be used in various ways, e.g. for controlling a head-up display or an audio device to provide output that depends on the output 93 of the at least one Al model 91 .
[00274] Fig. 38 is a flow chart of a method 120, comprising the training 121 of the Al model(s), the storing 122 of the Al model(s) in a storage or memory device for use by at least one processing circuit, using 123 the Al model to process data captured using the ski goggles 10, and controlling 124 an interface (e.g., the display 51) to provide an output that depends on the Al-based processing. The output may comprise visual and/or audio output and/or data output via a data interface.
[00275] The output that is provided may comprise any or any combination of the following:
- motion tracking of various skiing scenarios as jumping time/air time, jumping distance, jumping height and ski dynamic motion;
- local geographical POIs such as people, ski-lifts, restaurants, etc., optionally including distance to the object;
- performance information and performance evaluation capturing using local onboard sensors and cameras of the ski goggles 10;
- emergency information delivered directly to rescue services using the sensors of the ski goggles and the wireless communication capabilities of the ski goggles 10 and/or the wireless communication terminal 70;
- generation of a local map for unmapped areas; generating point of interest information for landmarks such as ski lifts, ski slopes, etc.; collision prevention for skiers using sensors, cameras and Al-model based processing;
- providing user notification about geographical landmarks hazards (e.g. cliffs, exposed rocks, obstacles) by using information provided by the global navigation satellite system information (as position, velocity heading and elevation data separate or
- combined) combined with a local map. Al-based image processing may be applied for this purpose.
- providing alternative slope planning depending on identified hazards or other criteria;
- displaying notifications regarding distance to a common friend (POI);
- providing an identifier for a person on the display 51 ; the identifier may be determined by utilized by GNSS location sharing between the different ski goggles (the current location and direction of view is known) and/or object recognition, which may be implemented in an Al-based manner;
- generating a user awareness indicator regarding ski slope level of difficulty and ski slope ratings using local maps, GNSS-based location, and, optionally, Al- based image processing;
- generating and executing a personal digital ski coach.
[00276] Al-based techniques may be used, for example, for one or several of the following:
- data fusion (e.g., fusing image data with other sensor data or fusing different types of sensor data);
- data fusion of GNSS receivers located in proximity to each other for enhanced accuracy;
- performing object recognition, e.g., to identify various snow conditions or exposed rocks or skiing slope difficulty levels or hazardous terrain features (such as cliffs). [00277] Additional processing techniques that do not involve Al-based processing may be used in addition to or instead of Al-based processing. Such processing may comprise data fusion techniques, data sharing, use of aerial or satellite images that are retrievable from servers, use of image data for generating a world model of a surrounding of the skier, and similar techniques.
[00278] The ski goggles 10 may also provide a digital ski coach functionality. Feedback can be provided based on sensor measurements included in the ski goggles 10 and/or in other wearables (such as ski boots or other ski outfit) to provide feedback, optionally including suggestions for performance improvement.
[00279] Communication techniques such as Bluetooth Low Energy (BLE) beacons or other wireless communication techniques may be used to establish relative spatial proximity of users of different ski goggles 10. Such data may be fused which image processing results, which may be based on one or several trained neural networks, for example.
[00280] The various processing techniques may be used to generate output for displaying as an overlay to a real-world view via the display 51 .
[00281] Fig. 39 shows a particularly preferred embodiment of the functional body 30 of the ski goggles 10 in which the display 51 comprises or is an optical waveguide display. The display driver 53 comprises an electrooptical transducer operative to generate electromagnetic signals (e.g., in the visible range of the electromagnetic spectrum) and provide them to the optical waveguide display. The display driver 53 may comprise a light engine 130 operative to generate light at one or, preferably, several different wavelengths and to supply the signals to the optical waveguide display. The optical waveguide display may be operative to guide the generated optical signals for generating the optical output that is visible on the optical waveguide display as an overlay to the real-world view.
[00282] Fig. 40 and Fig. 41 show a particularly preferred embodiment of the functional body 30 of the ski goggles 10 in which the at least one sensor comprises one or several microelectromechanical systems (MEMS) sensors 128 (Fig. 40) or a sensor set 129 comprising a gyroscope, inertial measurement unit (IMU), barometer, and magnetometer (Fig. 41 ). The MEMS sensors 128 in Fig. 40 may be operative to implement an IMU, barometer, and magnetometer.
[00283] While not explicitly shown, the display driver 53 in the functional body configurations of Fig. 40 and Fig. 41 may comprise an electrooptical transducer operative to generate electromagnetic signals (e.g., in the visible range of the electromagnetic spectrum) and provide them to the optical waveguide display. The display driver 53 may comprise a light engine 130 operative to generate light at one or, preferably, several different wavelengths and to supply the signals to an optical waveguide display 51 .
[00284] The sensor combinations disclosed in association with Fig. 40 and Fig. 41 are robust in field use in skiing applications and provide sensor data of particular interest to skiers, such as: acceleration, orientation of the ski goggles in space, altitude. This sensor data may be processed to generate information for provision via the display 51 , a communication terminal 70 coupled to the ski goggles 10 and/or for otherwise monitoring the activity of the wearer of the ski goggles.
[00285] The disclosure also covers all further features shown in the Figs, individually although they may not have been described in the afore or following description. Also, single alternatives of the embodiments described in the figures and the description and single alternatives of features thereof can be disclaimed from the subject matter of the invention or from disclosed subject matter. The disclosure comprises subject matter consisting of the features defined in the claims or the exemplary embodiments as well as subject matter comprising said features.
[00286] Furthermore, in the claims the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single unit or step may fulfil the functions of several features recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. The terms “essentially”, “about”, “approximately” and the like in connection with an attribute or a value particularly also define exactly the attribute or exactly the value, respectively. The term “about” in the context of a given numerate value or range refers to a value or range that is, e.g., within 20%, within 10%, within 5%, or within 2% of the given value or range. Components described as coupled or connected may be electrically or mechanically directly coupled, or they may be indirectly coupled via one or more intermediate components. Any reference signs in the claims should not be construed as limiting the scope.
[00287] A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. In particular, e.g., a computer program can be a computer program product stored on a computer readable medium which computer program product can have computer executable program code adapted to be executed to implement a specific method such as the method according to the invention. Furthermore, a computer program can also be a data structure product or a signal for embodying a specific method such as the method according to the invention.

Claims

1. Ski goggles (10), comprising: a lens (11 ); a lens frame (20) configured to retain the lens (11 ); a functional body (30) comprising a cavity (33), wherein the functional body (30) further comprises at least one wireless interface circuit (54, 55) and at least one integrated circuit (52) arranged within the cavity (33); a battery body (40) comprising one or several batteries (43) for supplying power to the functional body (30); wherein the battery body (40) and the functional body (30) are attached to the lens frame (20) on opposite sides (21 , 22) of the lens frame (20); characterized in that the functional body (30) comprises a display (51 ) extending along part of the lens in a space defined by the lens frame (20) and the lens, the display (51 ) being transparent or semitransparent to output information as an overlay to a real-world view visible through the lens (11 ); the functional body (30) comprises at least one sensor (57, 58) arranged within the cavity (33); and the at least one integrated circuit (52) is configured to process a sensor output of the at least one sensor (57, 58) and use the processed sensor output to control the display (51 ) to output the information and cause a data transmission by the at least one wireless interface circuit (54, 55).
2. Ski goggles (10) according to claim 1 , wherein the display (51 ) comprises an optical waveguide display.
3. Ski goggles (10) according to claim 2, wherein the functional body comprises a display driver (53), the display driver (130) comprising an electrooptical transducer (130) operative to supply optical signals to the waveguide display.
4. Ski goggles (10) according to any one of the preceding claims, wherein the at least one integrated circuit (52) is configured to use data received via the wireless interface circuit (54, 55) to control the display (51 ).
5. Ski goggles (10) according to any one of the preceding claims, wherein the at least one sensor (57, 58) is configured to determine a three-dimensional, 3D, orientation and/or 3D acceleration of the functional body (30), wherein the at least one integrated circuit (52) is configured to use the sensor output to determine a velocity of the ski goggles (10) as the information, optionally wherein the data transmission comprises the transmission of the determined velocity.
6. Ski goggles (10) according to any one of the preceding claims, wherein the at least one sensor comprises a global navigation satellite system, GNSS, receiver (57) and the at least one integrated circuit is configured to process a GNSS receiver output and use the processed GNSS receiver output to at least one of: control the display (51 ) to output the processed GNSS receiver output; control the at least one wireless interface circuit (54, 55) such that the data transmission includes the processed GNSS receiver output; control the at least one wireless interface circuit (54, 55) to perform a data retrieval; optionally wherein the processed GNSS receiver output comprises altitude information.
7. Ski goggles (10) according to any one of the preceding claims, wherein the at least one sensor comprises at least one microelectromechanical system (128) operative as one, several, or all of: an inertial measurement unit, a magnetometer, a barometer.
8. Ski goggles (10) according to any one of the preceding claims, wherein the at least one integrated circuit (52) is configured to use the processed sensor output to perform safety-related functions.
9. Ski goggles (10) according to claim 8, wherein the safety-related functions comprise: controlling the display (51 ) to output a hazard warning or risk assessment, optionally wherein the hazard warning or risk assessment comprises an avalanche risk and/or a slope difficulty rating retrieved via the at least one wireless interface circuit (54, 55) using the processed sensor output; and/or controlling the at least one wireless interface circuit (54, 55) to transmit an emergency signal.
10. Ski goggles (10) according to any one of the preceding claims, further comprising a camera arranged on the lens frame (20), wherein the camera has a camera objective (15) and the lens (11 ) has a recess or aperture (14) into which the camera objective (15) projects, the at least one integrated circuit (52) being configured to receive image frames captured by the camera.
11. Ski goggles (10) according to claim 10, wherein the integrated circuit (52) is configured to control the wireless interface circuit (54, 55) such that the data transmission includes at least some of the image frames.
12. Ski goggles (10) according to claim 10 or claim 11 , wherein the integrated circuit (52) is configured to process the image frames and use the processed image frames to control the display (51 ) to output the information.
13. Ski goggles (10) according to any one of the preceding claims, further comprising an audio interface (59), optionally wherein the audio interface comprises several transducers.
14. A wireless communication terminal (70), comprising: a wireless communication chipset (74) configured to perform bi-directional data communication with the ski goggles (10) according to any one of the preceding claims; a terminal display (72); and at least one processor (73) configured to process the data transmission from the ski goggles (10) and use the processed data transmission to perform activity tracking and control the terminal display (72) to output results of the activity tracking.
15. The wireless communication terminal (70) of claim 14, wherein the activity tracking comprises using the processed data transmission to track an altitude and/or a velocity of the ski goggles (10) as a function of time.
16. The wireless communication terminal (70) of claim 14 or claim 15, wherein the wireless communication terminal (70) is configured to use the processed data transmission to retrieve, via a cellular network or wireless local area network (80), an avalanche risk and/or a slope difficulty rating for a geolocation defined by the data transmission from the ski goggles (10) and transmit the avalanche risk and/or the slope difficulty rating to the ski goggles (10); and/or the wireless communication terminal (70) is configured to relay audio and/or video data to and from the ski goggles (10).
17. A system (60), comprising the wireless communication terminal (70) of any one of claims 14 to 16 and the ski goggles (10) of any one of claims 1 to 13.
18. A method of controlling a display (51 ) of ski goggles (10), wherein the display (51 ) is comprised by a functional body (30) attached to a lens frame (20) of the ski goggles (10) that retains a lens, the display (51 ) being transparent or semitransparent and extending in a space defined by the lens frame (20) and the lens, the method comprising the following steps: receiving, by at least one integrated circuit (52) arranged in a cavity of the functional body (30), a sensor output of at least one sensor arranged in the cavity of the functional body (30); processing, by the at least one integrated circuit (52), the sensor output; and using, by the at least one integrated circuit (52), the processed sensor output to control the display (51 ) to output information as an overlay to a real-world view visible through the lens.
19. Use of the ski goggles (10) according to any one of claims 1 to 13 together with a functional application provided on a wireless communication terminal (70) for tracking and/or displaying on the ski goggles display (51 ) and/or on the terminal display (72) weather information, device information, ski mode choice information, live map navigation information, reference point information, area recognition information, character recognition information, sports data information, SOS emergency call information and/or danger warning information.
20. Machine-readable instruction code comprising instructions which, when executed by at least one processing circuit (52, 73), cause the at least one processing circuit (52, 73) to process data captured by the ski goggles (10) according to any one of claims 1 to 12 and generate output based on a result of the processing.
21. The machine-readable instruction code of claim 20, wherein the machine- readable instruction code comprises instructions which, when executed by the at least one processing circuit (52, 73), cause the at least one processing circuit (52, 73) to process the data using at least one artificial intelligence, Al, model.
22. The machine-readable instruction code of claim 21 , wherein the at least one Al model comprises a recurrent neural network or a convolutional neural network (100) having an input layer (101 ) and an output layer (102), wherein the input layer (101 ) is operative to receive at least some of the processed sensor data and image data captured by the ski goggles (10).
23. The machine-readable instruction code of claim 22, wherein the output layer (102) is operative to provide a classification result indicative of a terrain difficulty and/or an object recognition result.
24. The machine-readable instruction code of any one of claims 21 to 23, wherein the at least one Al model comprises an Al model operative to process time-series data and having an input operative to receive time-series data.
25. The machine-readable instruction code of claim 24, wherein the Al model operative to process time-series comprises an output operative to provide a situation analysis result.
26. The machine-readable instruction code of claim 25, wherein the situation analysis result comprises identification of a skiing accident and/or an avalanche accident.
27. The machine-readable instruction code of any one of claims 24 to 26, wherein the Al model operative to process time-series data comprises at least one gated recurrent unit, GRU, or long short term memory, LSTM, cell (110).
PCT/EP2023/073082 2022-09-26 2023-08-23 Ski goggles, wireless communication terminal, and method of controlling a display of ski goggles WO2024068142A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CH11092022 2022-09-26
CHCH001109/2022 2022-09-26
CH3522023 2023-03-30
CHCH000352/2023 2023-03-30

Publications (1)

Publication Number Publication Date
WO2024068142A1 true WO2024068142A1 (en) 2024-04-04

Family

ID=87845685

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2023/073082 WO2024068142A1 (en) 2022-09-26 2023-08-23 Ski goggles, wireless communication terminal, and method of controlling a display of ski goggles

Country Status (1)

Country Link
WO (1) WO2024068142A1 (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100243756A1 (en) * 2009-02-27 2010-09-30 Paul Stuart Auerbach Avalanche rescue device
WO2011044680A1 (en) 2009-10-13 2011-04-21 Recon Instruments Inc. Control systems and methods for head-mounted information systems
US20110227820A1 (en) 2010-02-28 2011-09-22 Osterhout Group, Inc. Lock virtual keyboard position in an augmented reality eyepiece
WO2013000075A1 (en) 2011-06-29 2013-01-03 Recon Instruments Inc. Modular heads-up display systems
CN209167686U (en) 2018-07-10 2019-07-26 深圳市科乐科技有限公司 A kind of augmented reality skiing goggle
US20200015536A1 (en) * 2018-07-12 2020-01-16 Sarah Nicole Ciccaglione Smart safety helmet with heads-up display
WO2021050147A1 (en) * 2019-09-13 2021-03-18 Microsoft Technology Licensing, Llc Wearable electronic devices having multiple layers of electromagnetic spectrum specific paint for enhanced thermal performance
CN112782870A (en) 2021-02-06 2021-05-11 广州钜龙运动用品有限公司 Intelligence skiing mirror
CN214252800U (en) 2021-02-06 2021-09-21 广州钜龙运动用品有限公司 Intelligence skiing mirror
CN215117044U (en) 2021-03-23 2021-12-10 上海太阳能工程技术研究中心有限公司 Full-spectrum intelligent self-adaptive ski goggles

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100243756A1 (en) * 2009-02-27 2010-09-30 Paul Stuart Auerbach Avalanche rescue device
WO2011044680A1 (en) 2009-10-13 2011-04-21 Recon Instruments Inc. Control systems and methods for head-mounted information systems
US20110227820A1 (en) 2010-02-28 2011-09-22 Osterhout Group, Inc. Lock virtual keyboard position in an augmented reality eyepiece
WO2013000075A1 (en) 2011-06-29 2013-01-03 Recon Instruments Inc. Modular heads-up display systems
CN209167686U (en) 2018-07-10 2019-07-26 深圳市科乐科技有限公司 A kind of augmented reality skiing goggle
US20200015536A1 (en) * 2018-07-12 2020-01-16 Sarah Nicole Ciccaglione Smart safety helmet with heads-up display
WO2021050147A1 (en) * 2019-09-13 2021-03-18 Microsoft Technology Licensing, Llc Wearable electronic devices having multiple layers of electromagnetic spectrum specific paint for enhanced thermal performance
CN112782870A (en) 2021-02-06 2021-05-11 广州钜龙运动用品有限公司 Intelligence skiing mirror
CN214252800U (en) 2021-02-06 2021-09-21 广州钜龙运动用品有限公司 Intelligence skiing mirror
CN215117044U (en) 2021-03-23 2021-12-10 上海太阳能工程技术研究中心有限公司 Full-spectrum intelligent self-adaptive ski goggles

Similar Documents

Publication Publication Date Title
US10834986B2 (en) Smart safety helmet with heads-up display
CN113885208B (en) Display device, display system, and control method of display device
CN110892455A (en) Image processing apparatus, image processing method for image processing apparatus, and program
US20160378112A1 (en) Autonomous vehicle safety systems and methods
US20200106818A1 (en) Drone real-time interactive communications system
CN104781873A (en) Image display device and image display method, mobile body device, image display system, and computer program
CN108431667A (en) Information processing unit, information processing method and program
TW201603791A (en) Blind-guide mobile device positioning system and operation method thereof
US11710422B2 (en) Driving analysis and instruction device
US11110933B2 (en) Driving support device, wearable device, driving support system, driving support method, and computer-readable recording medium
Schreiner Where we at? Mobile phones bring GPS to the masses
CN105286166A (en) Back vision front-playing intelligent safety helmet
CN105882528A (en) Road condition sharing method and device of balance car and balance car
CN107231342A (en) terminal, server, system, management method and recording medium
CN109479093A (en) Image processing apparatus and image processing method
WO2024068142A1 (en) Ski goggles, wireless communication terminal, and method of controlling a display of ski goggles
US20220031006A1 (en) A Visual Communication System for a Helmet
US20210174573A1 (en) Image processing apparatus, display system, computer readable recoring medium, and image processing method
CN114152253A (en) All-weather hiking auxiliary system and method based on deep learning and big data
CN107107834A (en) Projection display device, electronic equipment, driver's visuognosis image sharing method and driver's visuognosis image sharing program
KR100718838B1 (en) A handheld sub terminal and operation method for mountain climbing information
KR20190094673A (en) Snow goggle system with function of augmented reality
CN108366899A (en) A kind of image processing method, system and intelligent blind-guiding device
JP6953744B2 (en) Information sharing system, information sharing method, terminal device and information processing program
US20210208584A1 (en) Moving body control device, moving body control method, and computer readable recording medium