WO2020089608A1

WO2020089608A1 - Computer-implemented method for suppressing motion sickness and display device for carrying out the method

Info

Publication number: WO2020089608A1
Application number: PCT/GB2019/053057
Authority: WO
Inventors: James John Shore RUSSELL
Original assignee: Oxford University Innovation Ltd
Current assignee: Oxford University Innovation Ltd
Priority date: 2018-11-02
Filing date: 2019-10-29
Publication date: 2020-05-07
Anticipated expiration: 2021-05-02
Also published as: GB201817962D0

Abstract

A computer-implemented method for suppressing motion sickness. The method comprises receiving movement data representing a movement of a display device in the display plane of the display device; generating compensation data by filtering the movement data to preferentially select at least one frequency lower than 0.5 Hz; and controlling the display of image data on a display of the display device such that the position of an image formed by the image data is shifted in antiphase to the movement of the display device represented by the compensation data.

Description

COMPUTER-IMPLEMENTED METHOD FOR SUPPRESSING MOTION SICKNESS AND DISPLAY DEVICE FOR CARRYING OUT THE METHOD

The present invention relates to a method for suppressing motion sickness, a display device for carrying out the method and a computer program product for carrying out the method.

Motion sickness is a condition that leads to symptoms ranging from discomfort and dizziness to nausea and vomiting in affected people. Motion sickness arises from a disconnect between the vestibular system’s sense of movement and visually perceived movement, for example when reading a book or watching a movie in a moving vehicle. It is estimated that one in three people across the world suffer from the symptoms of motion sickness to varying degrees of severity. There thus exists a strong need for technologies that suppress the symptoms of motion sickness.

Current remedies against motion sickness include medication, such as anti-histamines and Hyoscine, and accessories such as travel wristbands that are to be worn during long commutes, including Acupressure bands or wristbands applying mild electric shocks.

However, these remedies are only effective to a limited degree for many people suffering from motion sickness. More recently, technologies have been proposed that generate visual stimuli to counteract the mismatch between visually perceived movement and the vestibular system’s sense of movement. However, these proposed technologies are not readily accessible and remain difficult to implement, and are often not very effective at suppressing motion sickness.

There is thus a need for improved technologies that more effectively suppress motion sickness and are readily accessible for people suffering from the condition.

According to an aspect of the invention, there is provided a computer-implemented method for suppressing motion sickness. The method comprises receiving movement data representing a movement of a display device in the display plane of the display device. The method further comprises generating compensation data by filtering the movement data to preferentially select at least one frequency lower than 0.5 Hz. The method further comprises controlling the display of image data on a display of the display device such that the position of an image formed by the image data is shifted in antiphase to the movement of the display device represented by the compensation data.

The method may be used to effectively reduce the disconnect between visually perceived movement and movement experienced by the vestibular system of a user, by shifting the position of an image visually perceived by the user in accordance with physical movement. Preferentially selecting frequencies lower than 0.5 Hz for this compensation ensures that the user can still recognize details of the image formed by the image data, while motion sickness is suppressed.

In an embodiment, the compensation data is generated by filtering the movement data to preferentially select movement data within a compensation frequency range from 0.1 Hz to 0.5 Hz. Preferably, the compensation frequency range is from 0.15 Hz to 0.25 Hz. The effects of motion sickness are mostly felt at frequencies above 0.1 Hz. Lower frequency components, for example a zero frequency component due to gravity, only have a marginal effect on motion sickness. Further, oscillations felt by a user during travel in a vehicle are most intense in the frequency range from 0.15 Hz to 0.25 Hz. Selecting compensation data within the above frequency ranges thus allows effective suppression of motion sickness, while maintaining a user’s ability to read and recognize details of the image formed by the image data.

In an embodiment, the contribution of gravity is removed from the movement data or compensation data. This may be achieved by receiving orientation data of the display device, calculating the contribution of gravity to the received movement data or generated

compensation data based on the received orientation data, and subtracting the contribution of gravity from the received movement data or generated compensation data. The contribution of gravity to the movement data has no effect on motion sickness, and thus is not relevant to motion sickness suppression. Removing the contribution of gravity ensures that constants in the movement data do not result in movement of the image.

In an embodiment, the display of the image data is controlled such that the position of an image formed by the image data is shifted by a relocation distance that is directly proportional to the magnitude of the movement of the display device represented by the compensation data. The ratio of the relocation distance to the magnitude of the movement may be equal to a proportionality variable. The proportionality variable may be adjusted.

For example, the proportionality variable may be freely adjusted by the user. This allows a user to tune motion sickness suppression to fit the user’s individual requirements, and to achieve a compromise between motion sickness suppression and the ability to recognize details of the image.

In an embodiment, the proportionality variable may be adjusted as a function of time. For example, the proportionality variable may be decreased at an adaption rate. The adaption rate may be such that the relocation distance is reduced to zero within 2 to 3 days. The adaption of a user to continued movement may thus be taken into account, such that motion sickness suppression may be reduced and the user’s ability to recognize details may be improved as the user adapts to the continuous movement. Motion sickness suppression may thus automatically be reduced when the user does not require such suppression anymore.

In an embodiment, the adaption rate is calculated by predictive modelling. The predictive modelling may comprise providing a predictive model for modelling motion sickness of a user and receiving an initial model output indicative of a modelled measure of motion sickness of the user. The initial model output may be calculated by the predictive model based on an initial adaption rate. The predictive modelling may further comprise receiving a user input indicative of a real measure of motion sickness from the user, and adjusting the initial adaption rate such that the modelled measure of motion sickness more closely corresponds to the real measure of motion sickness to calculate the adaption rate. Feedback from the user may thus be used to tailor the predictive model and generate an accurate model of the user’s response to continued motion. This accurate model may then be used to automatically adjust the amount of motion sickness suppression required for the user after a set time.

In an embodiment, the method further comprises receiving face image data indicative of a position on the display of the display device at which a user of the display device is looking. The proportionality variable may be adjusted based on the received face image data. Alternatively or additionally, the size of the image formed by the image data may be adjusted based on the received face image data. This allows the method to take into account the portion of the image on the display device at which the user is looking, and ensure that this portion is not moved from the visible area of the display of the display device. When there is no risk of the portion moving from the visible area of the display of the display device, motion sickness suppression may be enhanced.

In an embodiment, the size of the image formed by the image data is adjusted. The size of the image formed by the image data may be adjusted in correlation with the adjusting of the proportionality variable. For example, as the proportionality variable is reduced (such that the image formed by the image data is moved less in response to the same external movement experienced by the display device), the size of the image may be increased. This ensures use of a larger percentage of the visible area of the display, without increasing the risk of the image moving off the visible area of the screen. The size of the image may be adjusted as a function of time, for example when the proportionality variable is also adjusted as a function of time. This allows adaption of the user to continued movement to be taken into account.

In an embodiment, the maximum distance by which an image formed by the image data is shifted from a reference position of the image on the display is limited. This allows limiting the extent of the edge of the image that may be moved off the visible area of the display, ensuring that at least the centre portion of the image remains visible on the display at all times. The image may also be entirely prevented from moving off the visible area of the display by appropriately setting the maximum distance.

The reference position may be adjusted based on the received face image data.

Additionally or alternatively, the maximum distance may be adjusted based on the received face image data. This may reduce the risk of the image extending beyond the edges of the display when the user looks at an edge region of the display.

In an embodiment, controlling the display of the image data on the display is carried out with a time delay after receiving the movement data. The time delay may be, for each frequency component of the compensation data, an integer multiple of the period of each respective frequency component of the compensation data. Movement of the image may thus appear to coincide (i.e. the time delay may appear to be 0) with movement of the display device to a user, for example if the movement data exhibits overall periodicity over time.

This effectively avoids any time delay between experiencing movement and compensating for movement, thus more effectively suppressing motion sickness.

According to another aspect of the invention, there is provided a display device comprising means for carrying out the steps of the method. The display device may, for example, be one of a mobile device, a mobile phone, a tablet, an e-reader and a laptop. Such a display device may be used for effective motion sickness suppression, while maintaining a user’s ability to discern details of an image displayed on the display device.

The invention will be more clearly understood from the following description, given by way of example only, with reference to the accompanying drawings, in which:

Figure 1 schematically depicts a display device for carrying out a method according to an embodiment.

Figure 2 depicts a flow chart of a method for suppressing motion sickness according to an embodiment.

Figure 3 depicts an embodiment of the predictive model of the internal processes that lead to motion sickness. The same references are used for similar features throughout the drawings. The features shown in the figures are not necessarily to scale and the size or arrangements depicted are not limiting. It will be understood that the figures may include optional features which are not essential to any embodiments. Furthermore, not all of the features are depicted in each figure and the figures may only show a few of the components relevant for describing a particular feature.

Figure 1 schematically depicts a display device 100 for carrying out the method for suppressing motion sickness. The display device 100 may be a mobile device, for example a mobile phone, such as a smart phone, as shown in Figure 1. Alternatively, the display device 100 may be any other mobile device, such as an e-book reader, a tablet, or a laptop. Further alternatively, the display device 100 may not be a mobile device, and may for example be incorporated in a vehicle. In some embodiments, the display device 100 is embedded in the back of a vehicle seat, such as a car seat, a bus seat, a train seat, a plane seat or a boat seat.

The display device 100 comprises a display 130. The display 130 extends along an x- axis and a y-axis of the display device, which form a display plane or image plane. An image formed by image data 132, 134 is displayed on the display 130. The image data 132, 134 may include content image data 134. The content image data 134 may comprise any image data of interest to the user, such as a page of an e-book, a webpage, a picture, a movie, image data representing a third party application, or any combination of the above. Preferably, the content image data 134 represents an operating system running on the display device 100, such that the display device 100 may be used for suppressing motion sickness on the operation system level.

Optionally, the image data 132, 134 may further include background image data 132. The background image data 132 may be any image data without content of interest to a user of the display device 100, for example a background of a single colour (e.g. a dark colour such as blue or black) or a background with some texture but no information of interest to the user. The background image data 132 may be provided, for example, by an operating system running on the processor of the display device 100. Alternatively, the background image data 132 is provided by a computer- implemented method according to the present invention.

The background image formed by the background image data 132 may surround the content image formed by the content image data 134, as shown in Figure 1. The size of the content image may be fixed, such that the content image covers a fixed proportion of the display 130, for example about 50% to 100% of the visible area of the display 130.

Alternatively, the size of the content image may be variable, such that the proportion of the display 130 covered by the content image is variable, for example in a range from 50% to 100% of the visible area of the display 130. In some embodiments, a user of the display device 100 can freely set the proportion of the visible area of the display 130 covered by the content image. In some embodiment, no background image is provided, and the content image covers the entire area of the display 130 when the content image is at a central position on the display 130. Put another way, the size of the content image may be equal to the size of the visible area of the display 130. The background image may only become visible as the content image is moved on the display 130, as described below.

The display device 100 comprises one or more acceleration sensors 110. The acceleration sensor 110 may measure the acceleration of the display device 100 in an x- direction along the x-axis, a y-direction along the y-axis and a z-direction along a z-axis. The acceleration sensor 110 outputs acceleration data along the x-axis, along the y-axis and along the z-axis. The acceleration data along the x-axis and along the y-axis is an example of movement data. The acceleration data along each of the x, y and z-axes is also an example of orientation data. As shown in Figure 1, the x-direction, the y-direction and the z-direction are mutually perpendicular to each other. The z-direction is perpendicular to the image plane.

The x, y and z directions may each be at an angle to a horizontal plane H, and may each be at a respective orientation angle q_c, 0_y, q_z to a vertical direction V that is perpendicular to the horizontal plane H. The vertical direction V extends in the direction of gravity.

The display device 100 may comprise one or more angular velocity sensors 120. The angular velocity sensor 120 may, for example, comprise one or more gyroscopes. The angular velocity sensor 120 may measure the angular velocity of the display device 100 about the x-axis, about the y-axis and about the z-axis. The angular velocity sensor 120 outputs angular velocity data. Angular velocity data is another example of orientation data.

The display device 100 may comprise a camera 140, such as a front-facing camera 140. The camera 140 may capture and output face image data of the face of a user of the display device 100. The face image data may be indicative of an orientation of the head of a user. The face image data may be indicative of a position on the display 130 of the display device 100 that a user is looking at.

Figure 2 depicts a flow chart showing a method 200 for suppressing motion sickness. The method 200 may be a computer-implemented method 200 and may be carried out by the display device 100. The method 200 may be carried out by instructions of a computer program product, such as a mobile application or app, that is stored in memory of the display device 100 and is executed by a processor of the display device 100. In step Sl of the method 200, movement data representing a movement of the display device 100 in the display plane of the display device, so in the x-direction and in the y- direction, is received. The movement data may comprise acceleration data in the x-direction and in the y-direction generated by the acceleration sensor 110. Additionally or alternatively, the movement data may comprise position data indicative of a position of the display device 100 in the x- and y-directions or velocity data indicative of a velocity of the display device 100 in the x- and y-directions. The position data and/or velocity data may be generated by the display device 100 from the acceleration data and passed on to the method 200.

Alternatively, the position data and/or velocity data may be generated in the method 200 based on the received acceleration data.

In step S2, the movement data is filtered in order to generate compensation data. The filtering comprises preferentially selecting at least one frequency lower than 0.5 Hz, i.e. preferentially selecting at least one frequency within a compensation frequency range from 0 Hz to 0.5 Hz, to generate the compensation data from the movement data. The average ratio of compensation data to movement data is higher within the compensation frequency range than outside the compensation frequency range. In an embodiment, the compensation data comprises, or consists of, data with frequencies below 0.5 Hz. This means that in the frequency domain, the compensation data comprises, or consists of, frequency components with a frequency below 0.5 Hz. Research (Bames G R, Benson A J:“Vision during angular oscillation: the dynamic interaction of visual and vestibular mechanisms”, Aviation Space and Environmental Medicine, 49-1:340-345, 1978) has shown that the ability of humans to distinguish and process visual details begins to deteriorate at frequencies above 0.5 Hz. The inventor has found that using an upper cut-off frequency of 0.5 Hz for motion sickness compensation achieves an efficient trade-off between a user’s ability to read and recognize details of an image that is to be moved and the effectiveness of motion sickness suppression.

In an embodiment, the movement data is filtered so as to preferentially select compensation data having a frequency in the range from 0.1 Hz to 0.5 Hz, i.e. the

compensation frequency range may range from 0.1 Hz to 0.5 Hz. Alternatively, the compensation frequency range may be any sub range within the range from 0.1 Hz to 0.5 Hz, for example a range from 0.15 Hz to 0.25 Hz. Research (Wylie et al.,“Motion sickness incidence: Exploratory studies of habituation, pitch and roll and the refinement of a mathematical model” Santa Barbara, Calif: Human Factors Research Inc, Technical Report 1733-2, 1976) has shown that the effects of motion sickness are mostly felt at frequencies exceeding 0.1 Hz. Movement at frequencies below 0.1 Hz is slow enough to only have a marginal effect on motion sickness. Targeting movement data within a frequency range from 0.1 Hz to 0.5 Hz thus ensures that motion sickness compensation is achieved for movement that is most likely to lead to motion sickness, while maintaining a user’s ability to read and recognize details of the content image formed by the content image data 134.

In an embodiment, the compensation frequency range ranges from 0.15 Hz to 0.25 Hz, or is any sub-range within the range from 0.15 Hz to 0.25 Hz. The inventor has found that oscillations acting on the display device 100 during travel in a vehicle, for example in a car, are most intense in the frequency range from 0.15 Hz to 0.25 Hz. Selecting

compensation data within this frequency range thus allows compensation for the strongest oscillations felt by a user during travel in a vehicle, allowing effective suppression of motion sickness.

The movement data may be filtered by a frequency filter so as to preferentially select the compensation data. The frequency filter may affect movement data with a frequency within the compensation frequency range to a lesser degree than movement data with a frequency outside the compensation frequency range. The frequency filter may be a low-pass frequency filter with upper -3dB or cut-off frequency within a range from 0.25Hz to lHz, in particular 0.25Hz to 0.5Hz. Alternatively, the frequency filter may be a bandpass frequency filter with additional lower -3dB or cut-off frequency within a range from 0 Hz to 0.15 Hz, for example O.OlHz to 0.15 Hz. The frequency filter may have vertical frequency cut-offs, or may roll off above and below the frequency cut-offs.

Step S2 may, for example, be implemented using a maximum gradient approach. The maximum gradient approach may be used to set the upper limit of the compensation frequency range. Such an approach is faster than using a fast Fourier transform to select the compensation data from the movement data detected by the acceleration sensor 110. In the maximum gradient approach, the position of the display device 100 in the x-direction and in the y-direction is measured at a time t and at a time t + At, where At is a fixed sampling time interval. The fixed sampling time interval At may correspond to the time required to carry out steps Sl to S3 of the method 200 once. A maximum allowable change in position within each sampling time interval At is predetermined. The maximum allowable change in position may, for example, be calculated based on the upper limit of the compensation frequency range f_max and the size of the gap d_ref between the content image (for example in the reference position) and the edge of the visible area of the screen. For example, the maximum allowable change in position may be calculated as the product 2nf_maxd_refAt. The position data of the content image data 134 is modified (in step S3) so as to counteract (and compensate for) any change in position of the display device 100 up to the maximum allowable change. If the change in position of the display device 100 exceeds the maximum allowable change, the position data of the content image data 134 is modified so as to counteract the maximum allowable change. The maximum allowable change within the fixed sampling time interval At thus sets the maximum speed, and therefore the maximum frequency component of the movement data, at which the image formed by the content image data 134 is repositioned on the display 130.

Additionally, in step S2, low frequency components of the movement data may be discarded by subtracting a time average of a plurality of recently received movement data, e.g. movement data received in earlier sampling time intervals, from the movement data so as to preferentially select compensation data. This may set the lower cut-off of the

compensation frequency range.

Alternatively, step S2 may be implemented by applying a frequency filter on the results of a fast Fourier transform (FFT) of the movement data so as to preferentially select the compensation data. The frequency filter may, for example, be a low pass filter with a cut off frequency or -3dB point at 0.5 Hz. Alternatively, the frequency filter may be a bandpass filter with cut-off frequencies or -3dB points at 0.1 Hz and 0.5 Hz, in particular at 0.15 Hz and 0.25 Hz. The cut-offs of the frequency filter may correspond to the upper and lower limits of the compensation frequency range.

In step S3, the display of image data, for example the content image data 134, on the display 130 of the display device 100 is controlled, such that the position of an image formed by the image data is shifted in antiphase to the movement of the display device 100 represented by the selected compensation data.

The image formed by the content image data 134 may be moved relative to a stationary image formed by the background image data 132. For example, with reference to Figure 1, if the display device 100 is moved in an up-direction, then the image formed by the content image data 134 on the display 130 is moved in a down-direction, and vice versa.

This reduces the disconnect between the movement a user visually perceives and the movement a user’s vestibular system senses. Motion sickness is thus effectively suppressed.

Alternatively, the image formed by the background image data 132 may move in unison with the image formed by the content image data 134, such that an image formed by the image data 132, 134 covers the entire visible area of the display 130, for example for the entire duration of method 200. This is especially useful if the background image data 132 represents a texture or pattern, the movement of which is discernible by the user of the display device 132. Movement of the background image may thus contribute to reducing the disconnect between visually perceived movement and movement sensed by the user’s vestibular system, more effectively suppressing motion sickness.

The maximum distance by which an image formed by the image data is shifted from a reference position on the display 130 may be limited. The reference position may, for example, be a central position on the visible area of the display 130. Limiting the maximum distance by which the image is shifted can ensure that at least parts of the image, for example parts that contain information of interest to the user, remain on the visible area of the display 130. The maximum distance may, for example, be the distance between the edge of the content image and the visible area of the display 130. The content image may thus fully remain on the visible area of the display 130. Alternatively, the maximum distance may be chosen such that parts of the content image, for example the edge regions of the content image, may not remain visible on the visible area of the display 130. This is especially useful when the content image covers the entire visible area of the display 130. Optionally, the image formed by the image data may be forced back to the reference position on the display 130 over time, for example using a forcing function.

The position of the image formed by the image data may be moved by the same amount as the movement of the display device 100 represented by the compensation data. Alternatively, the amount of movement of the image relative to the amount of movement of the display device 100 represented by the compensation data may be proportional by a proportionality constant or a proportionality variable. The proportionality constant or variable may be in the range from 0.1 to 2, for example, such that a lcm movement of the display device in an up-direction leads to a 1 mm to 2cm movement of the image in a down- direction. The proportionality variable may be freely set or adjusted by a user of the display device 100, such that the user can tune movement of the image on the display 130 of the display device 100. The size of the image formed by the image data 132, 134 may be adjusted in correlation with the adjusting of the proportionality variable, for example such that the size of the image increases as the proportionality variable decreases. The

proportionality variable (and optionally the size of the image, for example in correlation with the adjustment of the proportionality variable) may also be adjusted automatically over time, for example to account for adaption of a user to continued movement.

There may be a time delay between the step S 1 of receiving the movement data and the step S3 of controlling the display of the image on the display 130. This time delay is due to the time it takes the display device 100 to execute steps S2 and S3. The time delay is preferably significantly less than half the period of the highest frequency of the compensation frequency range. For example, the time delay may be significantly less than half the period of the frequency of 0.5 Hz, so significantly less than 1 second, to ensure that the visual- vestibular disconnect is alleviated and not enhanced. The inventor has found, using a predictive model that will be described further below, that the effects of motion sickness increase with an increase in the time delay, and that a time delay of less than 40 ms is desirable to suppress motion sickness for up to two hours in an average car journey.

Preferably, the time required to carry out the method 200 is thus less than 40ms.

In some embodiments, the content image may be moved on the display 130, for each frequency component of the compensation data, with a time delay of an integer multiple of the period of each respective frequency component of the compensation data. For example, if the compensation data comprises a frequency component of 0.5 Hz, then movement of the content image based on this frequency component may be deliberately delayed by 2 seconds (one whole period of this frequency component) after measuring the frequency component. Movement of the content image based on a frequency component of 0.1 Hz may be deliberately delayed by 10 seconds after measuring the 0.1 Hz frequency component.

Movement of the content image thus may appear to coincide (i.e. the time delay may appear to be 0) with movement of the display device 100 to a user. This is possible as the movement data may exhibit overall periodicity over time, for example when being caused by a vehicle driving on a road. It is thus possible to predict future movement data based on past movement data and calculate the compensation data for supressing motion sickness using the display device 100 in advance. This effectively avoids any time delay, thus more effectively suppressing motion sickness.

The steps Sl to S3 of the method 200 may be executed iteratively at a fixed frame rate. The frame rate may be limited by the time required to carry out the method 200. The frame rate may be at least 25 Hz, allowing for a time delay of less than 40 ms. Preferably the frame rate is at least 60 Hz, which is the frame rate at which the display 130 of many display devices 100, for example smartphones, are refreshed.

The movement data received from the acceleration sensor 110 may comprise gravity data due to the gravity experienced by the acceleration sensor 110. The contribution of gravity to the movement data depends on the orientation of the display device 100 in relation to the vertical direction V. It is desirable to remove the contribution of gravity from the movement data and/or from the compensation data before step S3 of the method 200, so as to ensure that only movement data and/or compensation data arising from temporary movement experienced by the display device 100, such as movement due to acceleration in a car or other vehicle, is compensated for.

The contribution of gravity to the movement data may be removed by implementing a lower cut-off frequency in the frequency filter. The frequency filter may filter the gravity component, which is a 0Hz frequency component, from the movement data.

Alternatively, the contribution of gravity may be removed from the movement data by receiving orientation data, such as the orientation angles q_c and 0_y of the display device 100.

In some embodiments, it may be assumed that the x-axis extends in the horizontal plane H, and so gravity does not contribute to the movement data along the x-axis. The contribution of gravity to the movement data, in particular in the x-direction and the y-direction of the display device 100, may be calculated based on the received orientation data. The calculated contribution of gravity may then be subtracted from the movement data received in step S 1. This results in movement data not having a gravity contribution.

The orientation data may be determined, for example, by prompting a user to hold the display device 100 at a desired orientation, for example such that the y-axis of the display device 100 is parallel to the vertical direction V, or at a pre-determined orientation angle 0_y (for example 45°) with respect to the vertical direction V. The user may confirm when the display device 100 is at the desired orientation, and the contribution of gravity may be calculated and subtracted from the movement data based on the desired orientation. It may be assumed that the x-direction is perpendicular to the vertical direction V under normal operation, and that gravity does not contribute to movement data in the x-direction received by the display device 100.

Alternatively, a user may freely enter the orientation data, such as the orientation angles Ox, 0_y at which the display device 100 is held, into the display device 100. The contribution of gravity may then be calculated and subtracted from the movement data based on the entered orientation data.

Further alternatively, the orientation angle 0_y of the display device 100 with respect to the vertical direction V (which is the direction along which gravity acts on the acceleration sensor) may be automatically determined. The orientation angle 0_y of the display device 100 may be calculated based on orientation data, such as the acceleration data received from the acceleration sensor 110, the angular velocity data received from the angular velocity sensor 120, or a combination of the acceleration data and the angular velocity data. A Kalman filter may be used, for example, to combine both of the acceleration data and the angular velocity data to arrive at a more accurate estimate of the orientation angle 9_y. A Kalman filter allows use of the angular velocity data to correct for inaccuracies in the acceleration data, which is vulnerable to drift due to integration processes. The estimate of the orientation angle 9_y may thus be more accurate. Alternatively, a weighted average of the orientation angle calculated based on the acceleration data and the orientation angle calculated based on the angular velocity data may be used to arrive at a more reliable estimate for the orientation angle 9_y.

Research has shown that the vestibular system adapts to the continuous exposure to movement, and that the effects of motion sickness diminish over time, usually within 2 to 3 days. It is thus desirable to reduce the amount of compensation for the visual-vestibular disconnect over time. For example, the relocation distance by which an image is moved on the display to compensate for movement of the device 199 may be adjusted, in particular reduced, over time. Put another way, the amplitude of shifting the content image data 134 on the display may be adjusted, in particular reduced, over time. Additionally or alternatively, the size of the image formed by the content image data 134 may be adjusted, in particular increased, over time. The proportion of the visible area of the display 139 covered by the content image may thus be increased over time. The proportion of the visible area of the display covered by the background image may be decreased accordingly, such that the image data 132, 134 (comprising the content image data 134 and the background image data 134) may continue to cover the entire visible area of the display 139.

The relocation distance may be proportional to the movement of the display device 199 by a proportionality variable. This proportionality variable may be adjusted to account for adaption of a user to continued movement. The proportionality variable may be steadily decreased over time at an adaption rate. The adaption rate may be a constant, for example chosen such that the proportionality variable is reduced to zero within 2 to 3 days. Such adaption might be useful, for example, for passengers of cruise ships on a several day journey.

The size of the content image formed by the content image data 134 may be variable, and may be adjusted, in particular increased, as a function of time. For example, the size of the content image may be variable within a range from 59% to 199% of the visible display area of the display 139 of the display device 199. In some embodiments, the size of the content image may be adjusted in correlation with the adjusting of the proportionality variable. For example, as the proportionality variable decreases such that the relocation distance decreases, the size of the content image may be increased proportionally. The size of the content image may be increased by the same amount by which the relocation distance is decreased. This allows a larger proportion of the visible display area to be used over time, while reducing the risk of the content image seemingly extending beyond the borders of the display 130.

The adaption rate may be freely set by a user. Alternatively, the adaption rate may be determined by predictive modelling. The predictive modelling may comprise providing a predictive model 300 for modelling motion sickness of a user. An example of such a predictive model 300 is schematically depicted in Figure 3. The predictive modelling may further comprise receiving an initial model output indicative of a modelled measure of motion sickness of the user. The initial model output may be calculated by the predictive model based on an initial adaption rate. The initial adaption rate may, for example, be an adaption rate for the average user, for example an adaption rate that is chosen such that the proportionality variable is reduced to zero within 2 days. The predictive modelling may further comprise receiving a user input indicative of a real measure of motion sickness from the user. For example, the user may be asked to provide a measure on a scale from 0 to 100% of the degree of motion sickness the user experiences after being exposed to continued movement for some time. The real measure may be compared to the modelled measure. The initial adaption rate may then be adjusted such that the modelled measure of motion sickness more closely corresponds to the real measure of motion sickness. For example, if the modelled measure indicates that the degree of motion sickness of an average user after four hours of continued exposure to movement is higher than the real measure provided by the user, it may be determined that the user’s adaption rate is higher than the average user’s adaption rate. The adaption rate for adjusting the proportionality variable may then be adjusted accordingly, providing a more accurate adaption rate tailored to the user of the display device 100.

An embodiment of the predictive model 300 that may be used for predictive modelling is schematically depicted in Figure 3. The predictive model 300 comprises a sensory input stage 310, a frequency selection stage 320, an adaption stage 330, an output stage 340 for a measure of motion sickness, and an emesis output stage 350 for a measure of the onset of emesis.

The sensory input stage 310 is used to model the disconnect between the movement experienced by vestibular system and the visually perceived movement. The visually perceived movement may be the movement of the image data on the display 130 of the display device 100. The movement experienced by the vestibular system may correspond to the movement data, preferably after subtraction of the gravity component. Movement perceived by a user’s otolithic membrane may be disregarded for the purposes of the predictive model 300. The sensory input stage 310 may provide the difference between the compensation data and the movement data to the frequency selection stage 320.

The frequency selection stage 320 may filter the signal received from the sensory input stage 310 using a bandpass filter, for example using a bandpass filter with -3dB points at 2.5 Hz and 0.015 Hz and a center frequency of 0.2 Hz. This ensures that only frequencies that give rise to motion sickness are considered by the predictive model 300. A key frequency signal, for example a dominant frequency signal, may be selected from the filtered signal. The key frequency signal may then be provided to the adaption stage 330.

The adaption stage 330 models adaption of a user to continued movement. The adaption stage 330 applies a transfer function to the received key frequency signal, which transfer function includes an adaption rate. The adaption rate may be chosen to be an adaption rate of an average user, for example such that motion sickness does no longer occur after 2 days. The adaption rate may be a parameter that can be updated for an individual user based on feedback of a user on the accuracy of the predictive model output. The adaption stage 330 provides an adapted key frequency signal to the output stage 340.

The output stage 340 may apply a gain factor to the adapted key frequency signal, and multiply the result with a measure of the time for which the display device 100 has experienced continuous movement. This may result in a modelled measure of motion sickness of the user. The modelled measure may be indicative of a probability that emesis occurs. This modelled measure may be compared to one or more real measures provided by the user at one or more different points in time. The parameters used in the predictive model 300, for example the adaption rate and the gain factor, may be adjusted for a specific user so as to generate a more accurate predictive model 300 for the specific user.

The modelled measure of motion sickness may be provided to an emesis output stage 350. The emesis output stage may compare the modelled measure to a threshold value above which emesis is expected to occur. The threshold value may be an average threshold for the average user, and may be updated for a specific user based on user feedback. The emesis output stage may provide a measure indicative of when emesis will occur, i.e. the time of continued exposure to movement that will result in a specific user vomiting. This measure could be used in the display device 100 to provide a warning, such as a message, on the display 130. This warning may prompt a user of the display device 100 to take a break from a car journey, for example. The method 200 may further comprise receiving face image data indicative of a position on the display of the display device at which a user of the display device is looking. The face image data may be received, for example, from the camera 140 of the display device 100. The face image data may be used to adjust movement parameters of the image formed by the image data on the display 130. For example, movement of the image may be different in a situation when the user looks at the edge of the display 130 compared to a situation in which the user looks at the center of the display 130. This is because it may be critical that the content image remains fully on the visible area of the display 130 when the user looks at an edge of the display 130, whereas the edges of the content image may be allowed to extend beyond the visible area of the display 130 when the user looks at the center of the display 130.

The proportionality variable may be adjusted based on the received face image data. For example, the proportionality variable may be decreased when the received face image data indicates that the user is looking at the edge of the display 130 compared to when the received face image data indicates that the user is looking at the center of the display 130. Movement of the image formed by the image data is thus reduced when the user looks at the edge of the display, reducing the risk that the image extends beyond the edges of the visible area of the display 130.

Additionally or alternatively, the size of the image formed by the image data may be adjusted based on the received face image data. For example, the size may be decreased when the received face image data indicates that the user is looking at the edge of the display 130 compared to when the received face image data indicates that the user is looking at the center of the display 130. This reduces the risk of the image extending beyond the edges of the display 130 when the user looks at an edge region of the display 130, while improving the user’s ability to discern details of the image when the user looks at the center of the display 130.

Additionally or alternatively, the reference position on the display of the image formed by the image data may be adjusted based on the received face image data. For example, when the user looks at the center of the display 130, the reference position of the image may be a central position on the visible area of the display 130. The distance by which the image may be moved from this reference position may be limited by the maximum distance. When the user looks at the upper edge of the display 130, the reference position may be shifted downwards compared to the central position. When the user looks at the lower edge of the display 130 the reference position may be shifted upwards compared to the central position. As such, the reference position may be shifted from an initial reference position in a direction opposite to the location at which a user is looking as indicated by the face image data. The image may be forced to move towards the adjusted reference position by a forcing function.

Additionally or alternatively, the maximum distance by which the image may be moved from the reference position may be adjusted based on the received face image data.

For example, the maximum distance may be decreased when the received face image data indicates that the user is looking at the edge of the display 130 compared to when the received face image data indicates that the user is looking at the center of the display 130.

This may ensure that the image does not extend beyond the edges of the display 130 when the user looks at an edge region of the display 130, while allowing more movement and so improving the effectiveness of motion sickness suppression when a user looks at the center of the display 130.

When introducing elements or features of the present disclosure and the exemplary embodiments, the articles "a", "an", "the" and "said" are intended to mean that there are one or more of such elements or features. The terms "comprising", "including" and "having" are intended to be inclusive and mean that there may be additional elements or features other than those specifically noted. It is further to be understood that the method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order discussed or illustrated, unless specifically identified as an order of performance. It is also to be understood that additional or alternative steps may be employed.

It is specifically intended that the present invention not be limited to the embodiments and illustrations contained herein and the claims should be understood to include modified forms of those embodiments including portions of the embodiments and combinations of elements of different embodiments as come within the scope of the following claims. It is explicitly stated that all value ranges or indications of groups of entities disclose every possible intermediate value or intermediate entity for the purpose of original disclosure as well as for the purpose of restricting the claimed invention, in particular as limits of value ranges.

Claims

1. A computer-implemented method for suppressing motion sickness, the method comprising: a) receiving movement data representing a movement of a display device in the display plane of the display device;

b) generating compensation data by filtering the movement data to preferentially select at least one frequency lower than 0.5 Hz; and

c) controlling the display of image data on a display of the display device such that the position of an image formed by the image data is shifted in antiphase to the movement of the display device represented by the compensation data.

2. The method of claim 1, wherein the compensation data is generated by filtering the movement data to preferentially select movement data within a compensation frequency range from 0.1 Hz to 0.5 Hz, preferably from 0.15 Hz to 0.25 Hz.

3. The method of any preceding claim, further comprising a step of removing the

contribution of gravity from the movement data or compensation data, the step comprising receiving orientation data of the display device,

calculating the contribution of gravity to the received movement data or generated compensation data based on the received orientation data, and

subtracting the contribution of gravity from the received movement data or generated compensation data.

4. The method of any preceding claim, wherein the display of the image data is controlled such that the position of an image formed by the image data is shifted by a relocation distance that is directly proportional to the magnitude of the movement of the display device represented by the compensation data,

wherein the ratio of the relocation distance to the magnitude of the movement is equal to a proportionality variable.

5. The method of claim 4, further comprising adjusting the proportionality variable, preferably as a function of time.

6. The method of claim 5, wherein the adjusting of the proportionality variable comprises decreasing the proportionality variable at an adaption rate, optionally wherein the adaption rate is such that the relocation distance is reduced to zero within 2 to 3 days.

7. The method of claim 6, wherein the adaption rate is calculated by predictive modelling.

8. The method of claim 7, wherein the predictive modelling comprises

providing a predictive model for modelling motion sickness of a user,

receiving an initial model output indicative of a modelled measure of motion sickness of the user, wherein the initial model output is calculated by the predictive model based on an initial adaption rate,

receiving a user input indicative of a real measure of motion sickness from the user, adjusting the initial adaption rate such that the modelled measure of motion sickness more closely corresponds to the real measure of motion sickness to calculate the adaption rate.

9. The method any of claims 4 to 8, further comprising

receiving face image data indicative of a position on the display of the display device at which a user of the display device is looking, and

adjusting the proportionality variable based on the received face image data, and/or adjusting the size of the image formed by the image data based on the received face image data.

10. The method of any preceding claim, further comprising adjusting the size of the image formed by the image data as a function of time.

11. The method of any of claims 5 to 8, further comprising adjusting the size of the image formed by the image data,

wherein the size of the image formed by the image data is adjusted in correlation with the adjusting of the proportionality variable.

12. The method of any preceding claim, wherein the maximum distance by which an image formed by the image data is shifted from a reference position of the image on the display is limited.

13. The method claim 12, further comprising

adjusting the reference position based on the received face image data, and/or adjusting the maximum distance based on the received face image data.

14. The method of any preceding claim, wherein the image formed by the image data covers the entire visible area of the display.

15. The method of any preceding claim, wherein step c) is carried out with a time delay after step a), wherein the time delay is, for each frequency component of the compensation data, an integer multiple of the period of each respective frequency component of the compensation data.

16. The method of any preceding claim, wherein all steps of the method are executed iteratively at a fixed frame rate, optionally wherein the fixed frame rate is at least 25 Hz, preferably at least 60 Hz.

17. A display device comprising means for carrying out the steps of the method of any preceding claim.

18. The display device of claim 17, wherein the display device is one of a mobile device, a mobile phone, a tablet, an e-reader and a laptop.

19. A computer program product comprising instructions which, when the program is executed by a display device, cause the display device to carry out the steps of the method of any one of claims 1-16.