EP3545387A1 - Verfahren und vorrichtung zur bereitstellung eines bildes - Google Patents

Verfahren und vorrichtung zur bereitstellung eines bildes

Info

Publication number
EP3545387A1
EP3545387A1 EP17873373.9A EP17873373A EP3545387A1 EP 3545387 A1 EP3545387 A1 EP 3545387A1 EP 17873373 A EP17873373 A EP 17873373A EP 3545387 A1 EP3545387 A1 EP 3545387A1
Authority
EP
European Patent Office
Prior art keywords
user
zooming
image
posture
axis
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP17873373.9A
Other languages
English (en)
French (fr)
Other versions
EP3545387A4 (de
Inventor
Peng XIAO
Zengwan CHEN
Haoliang Zhu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Priority claimed from PCT/KR2017/013483 external-priority patent/WO2018097632A1/en
Publication of EP3545387A1 publication Critical patent/EP3545387A1/de
Publication of EP3545387A4 publication Critical patent/EP3545387A4/de
Withdrawn legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/003—Navigation within 3D models or images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04806—Zoom, i.e. interaction techniques or interactors for controlling the zooming operation

Definitions

  • the present disclosure relates to virtual reality (VR) technology, and in particular, to a method and a device for zooming in or out on an image on a VR device.
  • VR virtual reality
  • zooming in or out on an image displayed on a two-dimensional display device may be controlled by pressing buttons of the device or touching the screen of the device.
  • VR technology there is a need to provide a scheme implementing zooming in or out on an image displayed on a VR device.
  • One or more example embodiments provide a method and a device for zooming in or out on an image on a VR device.
  • the present disclosure provides a virtual reality (VR) device and method for zooming in or out on an image on a VR device.
  • VR virtual reality
  • FIG. 1 is a flowchart illustrating a method of operating a VR device according to an example embodiment
  • FIG. 2A is a diagram illustrating an example of an image displayed on a virtual reality device according to an example embodiment
  • FIG. 2B is a view showing the focus of two eyes of the user on the image of FIG. 2A;
  • FIG. 2C is a diagram illustrating an operation for identifying a portion that the user observes on the image of FIG. 2B;
  • FIG. 2D is an illustration of the operation of zooming down the identified portion on the image of FIG. 2C;
  • FIG. 3A and FIG. 3B are flowcharts of a method of operating a virtual reality device according to an example embodiment
  • FIG. 4 is a diagram illustrating a method of determining a three-dimensional rectangular coordinate system in a virtual reality device according to an example embodiment
  • Fig. 5 is a schematic diagram illustrating the components of a VR device according to an example embodiment.
  • FIG. 6 is a diagram illustrating a coordinate system in which a user's rotational motion is defined in a virtual reality device according to an example embodiment.
  • One or more example embodiments provide a method and a device for zooming in or out on an image on a VR device.
  • a method for zooming in or out on an image on a VR device may include: identifying that a posture of a user of the VR device is at least one of a zooming in posture and a zooming out posture, based on a displacement vector of a head of the user; zooming in the image, in response to a zooming in condition being satisfied, wherein the zooming in condition comprises the posture of the user being the zooming in posture; and zooming out the image, in response to a zooming out condition being satisfied, wherein the zooming out condition comprises the posture of the user being the zooming out posture.
  • the identifying may include identifying that the posture of the user is the zooming in posture, in response to a first component of the displacement vector in a first positive direction of a first axis of a three-dimensional rectangular coordinate system being greater than a first threshold and a second component of the displacement vector in a second axis and a third component of the displacement vector in a third axis of the three-dimensional rectangular coordinate system being less than a second threshold; and identifying that the posture of the user is the zooming out posture, in response to at least one of: the first component of the displacement vector in a negative direction of the first axis being greater than the first threshold and the second and third components of the displacement vector in the second axis and the third axis being less than the second threshold, and the second component of the displacement vector in a second positive direction of the second axis being greater than the first threshold and the first and third components of the displacement vector in the first axis and the third axis being less than the second threshold.
  • the method may further include: determining whether angular velocities of the head of the user in each directions for roll, pitch, and yaw, which are rotations about the first axis, the second axis, and the third axis, respectively, are smaller less than an angular velocity threshold. At least one of the zooming in the image and the zooming out the image may be performed in response to the angular velocities being less than the angular velocity threshold.
  • the method may further include: identifying a focus of eyes of the user on the image.
  • the zooming in the image and the zooming out the image may be performed by using the focus as a zoom center.
  • the method may further include: identifying a focus of eyes of the user on the image; and identifying a part of the image, the part being observed by the user according to the focus.
  • the zooming in the image may include zooming in the part of the image.
  • the zooming out the image may include zooming out the part of the image.
  • the zooming in condition may further include at least one of: a zoom level of the image is lower than a first zoom level threshold, and a focus of eyes of the user is located on the image.
  • the zooming out condition further comprises at least one of: the zoom level of the image is higher than a second zoom level threshold, and the focus of the eyes of the user is located on the image.
  • Each of the zooming in condition and the zooming out condition may further include: an absolute value of a difference between a first acceleration vector in a moving direction of a vehicle estimated by the vehicle on which the user is boarding and a second acceleration vector in the moving direction of the vehicle estimated by the VR device is greater than a threshold.
  • the zooming in the image may include: estimating a first velocity of the head of the user in a positive direction of a first axis of a three-dimensional rectangular coordinate system; continuing the zooming in the image in response to the first velocity being greater than or equal to 0; and stopping the zooming in the image in response to the first velocity being less than 0.
  • the zooming out the image may include: estimating a second velocity of the head of the user in one of a negative direction of the first axis and a positive direction of a second axis of the three-dimensional rectangular coordinate system; continuing the zooming out the image in response to the second velocity being greater than or equal to 0; and stopping the zooming out the image in response to the second velocity being less than 0.
  • the zooming in posture may be tilting the head to left of the user, and the zooming out posture may be tilting the head to right of the user.
  • the zooming in posture may be tilting the head to the right of the user, and the zooming out posture may be tilting the head to the left of the user.
  • the zooming in posture may be tilting the head forward of the user, and the zooming out posture may be tilting the head backward of the user.
  • the zooming in posture may be tilting the head backward of the user, and the zooming out posture may be tilting the head forward of the user.
  • a virtual reality (VR) device for providing an image may include: a sensor configured to estimate a movement of a head of a user of the VR device; and a controller configured to: identify that a posture of the user is at least one of a zooming in posture and a zooming out posture, based on a displacement vector of the movement, zoom in the image, in response to a zooming in condition being satisfied, wherein the zooming in condition includes the posture of the user being zooming in posture, and zoom out the image, in response to a zooming out condition being satisfied, wherein the zooming out condition includes the posture of the user being the zooming out posture.
  • a sensor configured to estimate a movement of a head of a user of the VR device
  • a controller configured to: identify that a posture of the user is at least one of a zooming in posture and a zooming out posture, based on a displacement vector of the movement, zoom in the image, in response to a zooming in condition being satisfied, wherein the zooming in condition includes the posture of the
  • the controller may be further configured to: identify that the posture of the user is the zooming in posture, in response to a first component of the displacement vector in a first positive direction of a first axis of a three-dimensional rectangular coordinate system being greater than a first threshold and a second component of the displacement vector in a second axis and a third component of the displacement vector in a third axis of the three-dimensional rectangular coordinate system being less than a second threshold; and identify that the posture of the user is the zooming out posture, in response to at least one of the first component of the displacement vector in a negative direction of the first axis being greater than the first threshold and the second and third components of the displacement vector in the second axis and the third axis being less than the second threshold, the second component of the displacement vector in a second positive direction of the second axis being greater than the first threshold and the first and third components of the displacement vector in the first axis and the third axis being less than the second threshold.
  • the controller may be further configured to: determine whether angular velocities of the head of the user in each directions for roll, pitch, and yaw, which are rotations about the first axis, the second axis, and the third axis, respectively, are less than an angular velocity threshold; zoom in the image in response to the angular velocities being less than the angular velocity threshold; and zoom out the image in response to the angular velocities being less than the angular velocity threshold.
  • the controller may be further configured to: identify a focus of eyes of the user on the image; zoom in the image by using the focus as a zoom center; and zoom out the image by using the focus as the zoom center.
  • the controller may be further configured to: identify a focus of eyes of the user on the image; identify a part of the image, the part being observed by the user according to the focus; zoom in the part of the image; and zoom out the part of the image.
  • the zooming in condition may further include at least one of: a zoom level of the image is lower than a first zoom level threshold, and a focus of eyes of the user is located on the image.
  • the zooming out condition may further include at least one of: the zoom level of the image is higher than a second zoom level threshold, and the focus of the eyes of the user is located on the image.
  • Each of the zooming in condition and the zooming out condition may further include: an absolute value of a difference between a first acceleration vector in a moving direction of a vehicle estimated by the vehicle on which the user is boarding and a second acceleration vector in the moving direction of the vehicle estimated by the VR device is greater than a threshold.
  • the controller may be further configured to: estimate a first velocity of the head of the user in a positive direction of a first axis of a three-dimensional rectangular coordinate system; continue the zooming in the image in response to the first velocity being greater than or equal to 0; stop the zooming in the image in response to the first velocity being less than 0; estimate a second velocity of the head of the user in one of a negative direction of the first axis and a positive direction of a second axis of the three-dimensional rectangular coordinate system; continue the zooming out the image in response to the second velocity being greater than or equal to 0; and stop the zooming out the image in response to the second velocity being less than 0.
  • the zooming in posture may be tilting the head to left of the user, and the zooming out posture may be tilting the head to right of the user.
  • the zooming in posture may be tilting the head to right of the user, and the zooming out posture may be tilting the head to left of the user.
  • the zooming in posture may be tilting the head forward of the user, and the zooming out posture may be tilting the head backward of the user.
  • the zooming in posture may be tilting the head backward of the user, and the zooming out posture may be tilting the head forward of the user.
  • FIG. 1 is a flowchart illustrating a method of operating a VR device according to an example embodiment.
  • FIG. 6 is a diagram illustrating a coordinate system in which a user's rotational motion is defined in a virtual reality device according to an example embodiment.
  • the VR device may sense a movement of a head of a user of the virtual reality device.
  • the VR device may identify whether the posture of the head of the user is a zooming in posture or a zooming out posture. In other words, the VR device may determine whether the posture of the head corresponds to a zoom-in command or a zoom-out command.
  • the posture of the head of the user may mean a specific action, movement, orientation and/or velocity of the user's head.
  • the posture of the user's head may be expressed as a displacement vector of the user's head.
  • the displacement vector of the user's head may be a vector that is defined by a movement of the user's head from a first position to a second position.
  • the zooming in posture i.e., a posture that corresponds to a zoom-in command
  • zooming out posture i.e., a posture that corresponds to a zoom-out command
  • the posture for zooming in may be tilting the head to the left; and the posture for zooming out may be tilting the head to the right.
  • the posture for zooming in may be tilting the head to the right; and the posture for zooming out may be tilting the head to the left.
  • the posture for zooming in may be tilting the head forward; and the posture for zooming out may be tilting the head backward.
  • the posture for zooming in may be tilting the head backward; and the posture for zooming out may be tilting the head forward.
  • the zooming in posture and the zooming out posture may be stored in the memory unit of the virtual reality device or may be set by a user of the virtual reality device using a setup program stored in the memory unit of the virtual reality device.
  • the posture of the user's head of the virtual reality device is a zooming in posture or a zooming out posture may be identified through a process that includes the following steps.
  • the direction indicated by a displacement vector of the posture for zooming in may be defined as the positive direction of the X-axis
  • the direction being perpendicular to the X-axis and pointing to the head of the user may be defined as the positive direction of the Y-axis
  • the direction being perpendicular to the X-axis and Y-axis may be defined as the positive direction of the Z-axis, thereby a first coordinate system is established.
  • components of a displacement vector of the head of the user in the directions of the X-axis, the Y-axis and the Z-axis may be estimated from an acceleration sensor of the VR device.
  • the posture of the head of the user may be identified as the posture for zooming in.
  • the first threshold value and the second threshold value may be stored in the memory unit of the virtual reality device, or may be set by a user of the virtual reality device using a setup program stored in the memory unit of the virtual reality device.
  • the posture of the head of the user may be identified as the posture for zooming out.
  • the absolute value of the component of the displacement vector of the head of the user in the positive direction of the X-axis is greater than a first threshold
  • the absolute values of the components of the displacement vector of the head of the user in the directions of the Y-axis and the Z-axis are both less than a second threshold and the angular velocity of the head of the user in each directions for roll, pitch, and yaw, which are rotations about the X-axis, the Y-axis, and the Z-axis, respectively, are all less than an angular velocity threshold
  • the posture of the head of the user may be identified as the posture for zooming in.
  • the yaw may indicate a rotation angle about the Z-axis.
  • the yaw may be defined as a counterclockwise rotation when looking the center of the coordinate axis on the Z-axis.
  • the pitch may indicate a rotation angle about the Y-axis.
  • the pitch may be defined as a counterclockwise rotation when looking the center of the coordinate axis on the Y-axis.
  • the roll may indicate a rotation angle about the X-axis.
  • the roll may be defined as a counterclockwise rotation when looking the center of the coordinate axis on the X-axis.
  • the absolute value of the component of the displacement vector of the head of the user in the negative direction of the X-axis is greater than the first threshold
  • the absolute values of the components of the displacement vector of the head of the user in each directions for roll, pitch, and yaw, which are rotations about the X-axis, the Y-axis, and the Z-axis, respectively are both less than the second threshold and the angular velocity of the head of the user in the direction of the X-axis
  • the Y-axis and the Z-axis are all less than the angular velocity threshold
  • the posture of the head of the user may be identified as the posture for zooming out.
  • the process of detecting the posture of the head of the user of the VR device may include the following steps.
  • the direction of the initial sight (e.g., forward eye gaze) of the user of the VR device obtained is defined as the positive direction of the X-axis
  • the direction which is perpendicular to the X-axis and points to the top of the head of the user is defined as the positive direction of the Y-axis
  • the direction which is perpendicular to the X-axis and the Y-axis and points to the left of the user's head is defined as the positive direction of the Z-axis.
  • components of a displacement vector of the head of the user in the directions of the X-axis, the Y-axis and the Z-axis of the first three-dimensional rectangular coordinate system may be obtained from an acceleration sensor of the VR device.
  • the user may be identified as having a posture of tilting the head forward.
  • the user may be identified as having a pose of tilting the head backward.
  • the absolute value of the component of the displacement vector of the head of the user in the positive direction of the X-axis is greater than a first threshold
  • the absolute values of the components of the displacement vector of the head of the user in each direction for roll, pitch, and yaw, which are rotations about the X-axis, the Y-axis, and the Z-axis, respectively are both less than a second threshold
  • the angular velocity of the head of the user in the directions of the X-axis, the Y-axis and the Z-axis are all less than a predetermined threshold
  • the user may be identified as having a posture of tilting the head forward.
  • the absolute value of the component of the displacement vector of the head of the user in the negative direction of the X-axis is greater than a first threshold
  • the absolute values of the components of the displacement vector of the head of the user in the directions of the Y-axis and the Z-axis are both less than a second threshold
  • angular velocity of the head of the user in each direction for roll, pitch, and yaw, which are rotations about the X-axis, the Y-axis, and the Z-axis, respectively, are all less than an angular velocity threshold
  • the user may be identified as having a posture of tiling the head forward.
  • the image displayed on the VR device may be zoomed in on; and if the posture of the head of the user of the VR device is the posture for zooming out, the image displayed on the VR device may be zoomed out on.
  • the focus of the user's eyes e.g., eye gaze
  • the image on the VR device may be zoomed in on by taking the focus of user's eyes on the image as the center of zooming.
  • the focus of the user's eyes on the image may be detected.
  • the image on the VR device may be zoomed out on by taking the focus of the user's eyes on the image as the center of zooming.
  • the method may further include: determining whether the zoom level (e.g., magnification power, resolution, etc.) of the current image is lower than a first zoom level threshold, and/or determining whether the focus of the user's eyes is located on the image, and if the focus of the user's eyes is located on the image, then zooming in on the image on the VR device; or otherwise, performing no zooming in operation on the image on the VR device.
  • the zoom level e.g., magnification power, resolution, etc.
  • the method may further include: Determining whether the zoom level of the current image is higher than a second zoom level threshold, and/or determining whether the focus of the user's eyes is located on the image, and if the focus of the user's eyes is located on the image, then zooming out on the image on the VR device; or otherwise, performing no zooming out operation on the image on the VR device.
  • the method may further include the following solutions.
  • a first acceleration sensor may be included in the vehicle, and a second acceleration sensor may be included in the VR device. If the posture for zooming in is tilting the head forward, and the posture for zooming out is tilting the head backward; or, the posture for zooming in is tilting the head backward, and the posture for zooming out is tilting the head forward, a second three-dimensional rectangular coordinate system may be established by defining the moving direction of the vehicle as the positive direction of the X-axis, and defining the X-axis, the Y-axis and the Z-axis being perpendicular to one another. In other words, the measurements obtained by the second acceleration sensor of the VR device may be compensated or offset by the measurements obtained by the first acceleration sensor of the vehicle.
  • the method may further include the following procedure before zooming in on the image on the VR device.
  • the difference between the component of a first acceleration vector calculated by the first acceleration sensor and the component of a second acceleration vector calculated by the second acceleration sensor on the VR device in the positive direction of the X-axis of the second three-dimensional rectangular coordinate system may be estimated. And if the absolute value of the difference is greater than a predetermined value, the image on the VR device may be zoomed in on, or otherwise, no zooming in operation on the image on the device may be performed.
  • the method may further include the following procedure before zooming out on the image on the VR device.
  • the difference between the component of the first acceleration vector calculated by the first acceleration sensor on the vehicle and the component of the second acceleration vector calculated by the second acceleration sensor on the VR device in the positive direction of the X-axis of the second three-dimensional rectangular coordinate system may be estimated. And if the absolute value of the difference is greater than a value, the image on the VR device may be zoomed out, or otherwise, no zooming out operation on the image on the device may be performed.
  • the executing entities of the above operations 100-102 may be the VR device, and the VR device may communicates with the first acceleration sensor included on the vehicle or a transceiver of the vehicle communicating values of the first acceleration sensor in real time and the communication may follow a same protocol specification, such as Internetworking Technology (ITO) protocols.
  • ITO Internetworking Technology
  • FIG. 1 A method of operating a virtual reality device according to an example embodiment of the present disclosure is described with reference to FIGS. 1 and 2.
  • FIG. 1 A method of operating a virtual reality device according to an example embodiment of the present disclosure is described with reference to FIGS. 1 and 2.
  • FIG. 2A is a diagram illustrating an example of an image displayed on a virtual reality device according to an example embodiment of the present disclosure.
  • FIG. 2B is a view showing the focus of two eyes of the user on the image of FIG. 2A.
  • FIG. 2C is a diagram illustrating an operation for identifying a portion that the user observes on the image of FIG. 2B.
  • FIG. 2D is an illustration of the operation of zooming down the identified portion on the image of FIG. 2C.
  • the method may further include the following procedure before zooming in on the image on the VR device, if the posture of the head of the user of the VR device is the posture for zooming in.
  • the VR device may detect the focus of two eyes of the user on the image displayed on the virtual reality device including the two coins as shown in FIG. 2A. Also, as shown in FIG. 2C, the VR device may identify the portion of the image that the user is observing according to the focus of the user's eyes. At this time, the boundaries of the object on the image where the focus of the user's two eyes are located can be detected by the boundary detection techniques. That is, the boundary of the right coin can be detected as shown in FIG. 2C. Also, as shown in FIG. 2D, an operation of zooming in may be performed only on the portion identified in the image. That is, only the right coin may be zoomed in as shown in FIG. 2D. And in operation 102, the method may further include the following procedure before zooming out on the image on the VR device, if the posture of the head of the user of the VR device is the posture for zooming out.
  • the focus of the user's two eyes may be detected as shown in FIG. 2B on the image displayed on the virtual reality device including the two coins as shown in FIG. 2A. Also, as shown in FIG. 2C, it is possible to identify the portion of the image that the user is observing according to the focus of the user's eyes. At this time, the boundaries of the object on the image where the focus of the user's two eyes are located may be detected by the boundary detection techniques. That is, the boundary of the right coin can be detected as shown in FIG. 2C. Also, an operation of zooming out may be performed only on the portion identified in the image. That is, only the right coin may be zoomed out in FIG. 2C.
  • FIG. 3A and FIG. 3B are flowcharts of a method of operating a virtual reality device according to an example embodiment of the present disclosure.
  • FIG. 4 is a diagram illustrating a method of determining a three-dimensional rectangular coordinate system in a virtual reality device according to an example embodiment of the present disclosure.
  • the posture for zooming in may be tilting the head forward
  • the posture for zooming out may be tilting the head backward.
  • the method may include the following steps.
  • the VR device may identify the direction of the initial sight of the user of the VR device, and may determine coordinate system.
  • the VR device may define the direction of the initial sight as the positive direction of the X-axis (e.g., direction of a forward gaze). Then, the VR device may further define the direction perpendicular to the X-axis and pointing to the top of the head of the user as the positive direction of the Y-axis, and define the direction perpendicular to the X-axis and Y-axis and pointing to the left of the user's head as the positive direction of Z-axis. Thereby, a first three-dimensional rectangular coordinate system may be established.
  • the VR device may obtain the direction of the initial sight of the user of the VR device via eye tracking technology.
  • the VR device may use the eye tracking technique to obtain the focus of the user's eyes on the image on the VR device, and then may determine the direction from the center of the user's eyes to the focus as the positive direction of X-axis.
  • the VR device may estimate components of a displacement vector of the head of the user in the directions of the X-axis, the Y-axis and the Z-axis from an acceleration sensor included in the VR device.
  • the virtual reality device may be a head-mounted display (HMD) that can be worn on the user's head.
  • the head-mounted display may be in the form of a helmet, eye glasses and so on.
  • the virtual reality apparatus may include at least one of various sensors such as an accelerometer for measuring acceleration, a gyroscope for measuring angular velocity, and a magnetometer as a geomagnetic sensor.
  • the VR device may determine whether the first condition is satisfied or not.
  • the first condition may include that the absolute value of the component of the displacement vector of the head of the user in the positive direction of the X-axis is greater than a first threshold, and the absolute values of the components of the displacement vector of the head of the user in directions of the Y-axis and the Z-axis are both less than a second threshold. If the first condition is satisfied, operation 304 may be executed; or otherwise, operation 307 may be executed.
  • the VR device may estimate the angular velocities of the head of the user in each directions for roll, pitch, and yaw, which are rotations about the X-axis, the Y-axis, and the Z-axis, respectively by using a sensor.
  • the yaw may indicate a rotation angle about the Z-axis.
  • the yaw may be defined as a counterclockwise rotation when looking the center of the coordinate axis on the Z-axis.
  • the pitch may indicate a rotation angle about the Y-axis.
  • the pitch may be defined as a counterclockwise rotation when looking the center of the coordinate axis on the Y-axis.
  • the roll may indicate a rotation angle about the X-axis.
  • the roll may be defined as a counterclockwise rotation when looking the center of the coordinate axis on the X-axis.
  • the VR device may determine whether the angular velocities of the head of the user in each directions for roll, pitch, and yaw, which are rotations about the X-axis, the Y-axis, and the Z-axis, respectively, are all less than a first angular velocity threshold, if the angular velocities are all less than the first angular velocity threshold, the VR device may executes operation 306; or otherwise, may execute operation 307.
  • the VR device may identify the posture of the head of the user as tilting the head forward, and thus, the VR device may identify the posture of the head of the user as the zooming in posture. And the VR device may zoom in on the image displayed, and then may return to operation 302.
  • the VR device may further determine whether a zooming in condition is satisfied. If the zooming in condition is satisfied, the VR device may zoom in on the image; or otherwise, the VR device may return to operation 302 without zooming in on the image.
  • the zooming in condition may include, for example, any one or combination of the following: the zoom level (e.g., magnification power, resolution, etc.) of the image is lower than a first zoom level threshold (e.g., maximum zoom level), and the focus of the user's eyes is on the image.
  • the zoom level e.g., magnification power, resolution, etc.
  • a first zoom level threshold e.g., maximum zoom level
  • the VR device may further detect the focus of the user's eyes on the image. And the VR device may zoom in on the image by taking the focus of the user's eyes (e.g., eye gaze of the user) on the image as the center of zooming. In other words, the image may be zoomed in around the location of the user's eye gaze on the image.
  • the VR device may further detect the focus of the user's eyes on the image.
  • the VR device may zoom in on the image by taking the focus of the user's eyes (e.g., eye gaze of the user) on the image as the center of zooming. In other words, the image may be zoomed in around the location of the user's eye gaze on the image.
  • the VR device may further detect the component of the moving speed of the head of the user in the positive direction of the X-axis of the first three-dimensional rectangular coordinate system. If the moving speed is greater than or equal to 0, the VR device may continue to zoom in on the image displayed by itself until a zooming in termination condition of the image is satisfied (for example, the zoom level of the image is greater than or equal to a maximum zoom level). And if the moving speed is less than 0, the VR device may stop zooming in on the image displayed by itself.
  • the VR device determines whether a second condition is satisfied.
  • the condition may include that the absolute value of the component of the displacement vector of the head of the user in the negative direction of the X-axis is greater than a first threshold, and the absolute values of the components of the displacement vector in the directions of the Y-axis and the Z-axis direction are both less than a second threshold. If the second condition is satisfied, operation 308 may be executed; or otherwise, operation 302 may be executed.
  • the VR device may determine whether the angular velocities of the head of the user in each directions for roll, pitch, and yaw, which are rotations about the X-axis, the Y-axis, and the Z-axis, respectively, are less than a second angular velocity threshold. If at least one of the angular velocities is less than the second angular velocity threshold, the VR device may execute operation 309; or otherwise, the VR device may return to operation 302.
  • the VR device may determine the posture of the head of the user of the VR device as tilting the head backward, and thus, the VR device may determine the posture of the head of the user of the VR device as the zooming out posture. And the VR device may zoom out on the displayed image, and then may return to operation 302.
  • the VR device may further determine whether a zooming out condition is satisfied or not. If the zooming out condition is satisfied, the VR device may zoom out on the image; or otherwise, the VR device may return to operation 302 without zooming out on the image.
  • the zooming out condition may include, for example, any one or combination of: the zoom level of the image is higher than a second zoom level threshold (e.g., minimum zoom level), and the focus of the user's eyes is on the image.
  • a second zoom level threshold e.g., minimum zoom level
  • the VR device may further detect the focus of the user's eyes on the image. And the VR device may zoom out on the image by taking the focus of the user' eyes (e.g., eye gaze of the user) on the image as the center of zooming.
  • the VR device may further detect the focus of the user's eyes on the image.
  • the VR device may zoom out on the image by taking the focus of the user' eyes (e.g., eye gaze of the user) on the image as the center of zooming.
  • the VR device may further detect the component of the moving speed of the head of the user in the negative direction of the X-axis of the first three-dimensional rectangular coordinate system. If the moving speed is greater than or equal to 0, the VR device may continue to zoom out on the image displayed by itself until a zooming out termination condition of the image is satisfied (for example, the zoom level of the image is less than or equal to a minimum zoom level). And if the moving speed is less than 0, the VR device may stop zooming out on the image displayed by itself.
  • FIGS. 3A and 3B may be changed.
  • the order of determining steps of the first condition and the second condition may be swapped.
  • some of the operations of FIG. 3A and 3B may be omitted or additional operations may be performed therebetween.
  • the thresholds of the present disclosure may be predetermined and stored in the memory unit of the virtual reality device or may be set by a user of the virtual reality device using a setup program stored in the memory unit of the virtual reality device. Further, in the present disclosure, the first threshold values and the second threshold values may be the same or different from each other.
  • Fig. 5 is a schematic diagram illustrating the components of a VR device according to an example embodiment.
  • the VR device 50 may include at least one of a sensor 51, a controller 52, a transceiver 53, memory 54, and a display 55.
  • the sensor 51 may be configured to detect the posture of the head of the user of the VR device 50 in real time, and transmit information regarding the detected posture of the head to the controller 52.
  • the controller 52 may be a processor, such as a central processing unit (CPU), an application-specific integrated circuit (ASIC), a system-on-chip (SoC), etc., configured to control other components of the VR device 50 to perform the various operations of the VR device 50.
  • the controller 52 may be configured to zoom in on the image on the VR device 50 if the posture of the head detected by the sensor 51 is a posture for zooming in; and zoom out on the image on the VR device 50 if the posture of the head detected by the sensor 51 is a posture for zooming out.
  • the posture for zooming in detected by the sensor 51 may be tilting the head to the left; and the posture for zooming out may be tilting the head to the right.
  • the posture for zooming in detected by the sensor 51 may be tilting the head to the right; and the posture for zooming out may be tilting the head to the left.
  • the posture for zooming in detected by the sensor 51 may be tilting the head forward; and the posture for zooming out may be tilting the head backward.
  • the posture for zooming in detected by the sensor 51 may be tilting the head backward; and the posture for zooming out may be tilting the head forward.
  • the VR device may further include a sight detector, which is configured to detect the focus (e.g., gaze) of the user's eyes on the image in real time and transmit it to the controller 52.
  • a sight detector configured to detect the focus (e.g., gaze) of the user's eyes on the image in real time and transmit it to the controller 52.
  • the controller 52 may zoom in on the image on the VR device by taking the focus of the user's eyes on the image detected by the sight capture module as the center of zooming.
  • the controller 52 may zoom out on the image on the VR device by taking the focus of the user's eyes on the image detected by the sight capture module as the center of zooming
  • the controller 52 may first detect the focus of the user's eyes on the VR image, identify a part of the image that the user is observing according to the focus of the user's eyes, and then zoom in on the part of the image that the user is observing.
  • the controller 52 may first detect the focus of the user's eyes on the VR image, identify a part of the image that the user is observing according to the focus of the eyes, and then zoom out on the part of the image that the user is observing.
  • the controller 52 may be further configured to determine whether the zoom level of the image is lower than a first zoom level threshold (e.g., maximum zoom level), and/or, whether the focus of the user's eyes is on the image. If one or both of these conditions are met, the controller 52 may zoom in on the image; or otherwise, the controller 52 may not zoom in on the image.
  • a first zoom level threshold e.g., maximum zoom level
  • the controller 52 may be further configured to determine whether the zoom level of the image is higher than a second zoom level threshold (e.g., minimum zoom level), and/or, whether the focus of the user's eyes is on the image. If one of both of these conditions are met, the controller 52 may zoom out on the image; or otherwise, the controller 52 may not zoom out on the image.
  • a second zoom level threshold e.g., minimum zoom level
  • the sensor 51 when detecting the posture of the head of the user of the VR device, the sensor 51 may first define the direction indicated by the posture for zooming in as the positive direction of the X-axis, define the direction being perpendicular to the X-axis and pointing to the top of the head of the user as the positive direction of the Y-axis, and define the direction being perpendicular to the X-axis and the Y-axis as the positive direction of the Z-axis.
  • a first coordinate system may be established.
  • the sensor 51 may further obtain components of a displacement vector of the head of the user in the directions of the X-axis, the Y-axis, and the Z-axis from an acceleration sensor of the VR device.
  • the sensor 51 may confirm the posture of the head of the user as the posture for zooming in.
  • the sensor 51 may confirm the posture of the head of the user as the posture for zooming out.
  • the senor 51 may further obtain angular velocities of the head of the user in each directions for roll, pitch, and yaw, which are rotations about the X-axis, the Y-axis, and the Z-axis, respectively, from a gyroscope of the VR device.
  • the sensor 51 may determine that the posture of the head is the posture for zooming in when the component of the displacement vector of the head of the user in the positive direction of the X-axis is greater than the first threshold, the absolute values of the components of the displacement vector of the head of the user in the directions of the Y-axis and the Z-axis are both less than the second threshold, and the angular velocities of the head of the user in each directions for roll, pitch, and yaw, which are rotations about the X-axis, the Y-axis, and the Z-axis, respectively, are all smaller than an angular velocity threshold.
  • the sensor 51 may determine the posture of the head is the posture for zooming out when the component of the displacement vector of the head of the user in the negative direction of the X-axis is greater than the first threshold, the absolute values of the components of the displacement vector of the head of the user in the directions of the Y-axis and the Z-axis are both less than the second threshold, and the angular velocities of the head of the user in each directions for roll, pitch, and yaw, which are rotations about the X-axis, the Y-axis, and the Z-axis, respectively, are all smaller than an angular velocity threshold.
  • the sensor 51 when the posture for zooming in is tilting the head forward and the posture for zooming out is tilting the head backward, the sensor 51 may further implement the following functions.
  • the sensor 51 may obtain the direction of the initial sight of the user of the VR device from the sight detector, define the direction of the initial sight as the positive direction of the X-axis, define the direction being perpendicular to the X-axis and pointing to the top of the head of the user as the positive direction of the Y-axis, and define the direction being perpendicular to the X-axis and the Y-axis and pointing to the left of the head of the user as the positive direction of the Z-axis.
  • a first coordinate system may be established.
  • the sensor 51 may further obtain components of the displacement vector of the head of the user in the directions of the X-axis, the Y-axis, and the Z-axis from the acceleration sensor of the VR device.
  • the sensor 51 may confirm the posture of the head of the user as tilting the head forward.
  • the sensor 51 may confirm the posture of the head of the user as tilting the head backward.
  • the senor 51 may further obtain angular velocities of the head of the user in each directions for roll, pitch, and yaw, which are rotations about the X-axis, the Y-axis, and the Z-axis, respectively, from a gyroscope of the VR device.
  • the sensor 51 may determine that the posture of the head is tilting the head forward when the component of the displacement vector of the head of the user in the positive direction of the X-axis is greater than the first threshold, the absolute values of the components of the displacement vector of the head of the user in the directions of the Y-axis and the Z-axis are both less than the second threshold, and the angular velocities of the head of the user in each directions for roll, pitch, and yaw, which are rotations about the X-axis, the Y-axis, and the Z-axis, respectively, are all smaller than an angular velocity threshold.
  • the sensor 51 may determine that the posture of the head is tilting the head backward when the component of the displacement vector of the head of the user in the negative direction of the X-axis is greater than the first threshold, the absolute values of the components of the displacement vector of the head of the user in the directions of the Y-axis and the Z-axis are both less than the second threshold, and the angular velocities of the head of the user in each directions for roll, pitch, and yaw, which are rotations about the X-axis, the Y-axis, and the Z-axis, respectively, are all smaller than an angular velocity threshold.
  • the controller 52 may implement the following functions.
  • the controller 52 may calculate a difference between the component of a first acceleration vector calculated by the first acceleration sensor on the vehicle in the moving direction of the vehicle and the component of a second acceleration vector calculated by the second acceleration sensor on the VR device in the moving direction of the vehicle. And if the absolute value of the difference is greater than a predetermined value, the controller 52 may zoom in on the image on the VR device; or otherwise, the controller 52 may not zoom in on the image.
  • the controller 52 may calculate a difference between the component of a first acceleration vector calculated by the first acceleration sensor on the vehicle in the moving direction of the vehicle and the component of a second acceleration vector calculated by the second acceleration sensor on the VR device in the moving direction of the vehicle. And if the absolute value of the difference is greater than a value, the controller 52 may zoom out on the image on the VR device; or otherwise, the controller 52 may not zoom out on the image.
  • the controller 52 may detect the velocity component of the head of the user in the positive direction of the X-axis. If the velocity component is greater than or equal to 0, the controller 52 may continue zooming in on the image on the VR device; or if the velocity component is less than 0, the controller 52 may stop zooming in on the image on the VR device.
  • the controller 52 may detect the velocity component of the head of the user in the negative direction of the X-axis. If the velocity component is greater than or equal to 0, the controller 52 may continue zooming out on the image on the VR device; or if the velocity component is less than 0, the controller 52 may stop zooming out on the image on the VR device.
EP17873373.9A 2016-11-25 2017-11-24 Verfahren und vorrichtung zur bereitstellung eines bildes Withdrawn EP3545387A4 (de)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
CN201611049218.1A CN106557170A (zh) 2016-11-25 2016-11-25 对虚拟现实设备上的图像进行缩放的方法及装置
KR1020170157514A KR20180059366A (ko) 2016-11-25 2017-11-23 가상 현실 기기 상에서의 이미지 줌인 및 줌아웃을 위한 방법 및 장치
PCT/KR2017/013483 WO2018097632A1 (en) 2016-11-25 2017-11-24 Method and device for providing an image

Publications (2)

Publication Number Publication Date
EP3545387A1 true EP3545387A1 (de) 2019-10-02
EP3545387A4 EP3545387A4 (de) 2019-12-11

Family

ID=58443798

Family Applications (1)

Application Number Title Priority Date Filing Date
EP17873373.9A Withdrawn EP3545387A4 (de) 2016-11-25 2017-11-24 Verfahren und vorrichtung zur bereitstellung eines bildes

Country Status (3)

Country Link
EP (1) EP3545387A4 (de)
KR (1) KR20180059366A (de)
CN (1) CN106557170A (de)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107464291B (zh) * 2017-08-22 2020-12-29 广州魔发科技有限公司 一种脸部图像的处理方法及装置
CN111629195A (zh) * 2019-02-28 2020-09-04 邓云天 一种3dvr照片的显示方法
CN111338476A (zh) * 2020-02-25 2020-06-26 上海唯二网络科技有限公司 通过头戴式vr显示设备实现人机交互的方法及装置
CN115442581B (zh) * 2021-12-29 2024-01-16 北京罗克维尔斯科技有限公司 车载vr眼镜控制方法、装置及计算机设备

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103118202A (zh) * 2013-03-12 2013-05-22 上海斐讯数据通信技术有限公司 一种通过摇动控制手机的方法
US9213403B1 (en) * 2013-03-27 2015-12-15 Google Inc. Methods to pan, zoom, crop, and proportionally move on a head mountable display
KR102129464B1 (ko) * 2013-07-01 2020-07-03 엘지전자 주식회사 디스플레이 장치 및 그것의 제어 방법
CN103558915B (zh) * 2013-11-01 2017-11-07 王洪亮 人体耦合智能信息输入系统及方法
CN104796594B (zh) * 2014-01-16 2020-01-14 中兴通讯股份有限公司 一种预览界面特殊效果即时呈现方法及终端设备
US20150346813A1 (en) * 2014-06-03 2015-12-03 Aaron Michael Vargas Hands free image viewing on head mounted display
US10073516B2 (en) * 2014-12-29 2018-09-11 Sony Interactive Entertainment Inc. Methods and systems for user interaction within virtual reality scene using head mounted display
CN105898346A (zh) * 2016-04-21 2016-08-24 联想(北京)有限公司 控制方法、电子设备及控制系统
CN106131483A (zh) * 2016-06-24 2016-11-16 宇龙计算机通信科技(深圳)有限公司 一种基于虚拟现实的巡检方法及相关设备、系统

Also Published As

Publication number Publication date
CN106557170A (zh) 2017-04-05
KR20180059366A (ko) 2018-06-04
EP3545387A4 (de) 2019-12-11

Similar Documents

Publication Publication Date Title
WO2018097632A1 (en) Method and device for providing an image
EP3545387A1 (de) Verfahren und vorrichtung zur bereitstellung eines bildes
WO2017008224A1 (zh) 一种移动物体的距离检测方法、装置及飞行器
WO2014109498A1 (en) Head mounted display providing eye gaze calibration and control method thereof
WO2020130648A1 (en) Electronic device for adaptively altering information display area and operation method thereof
WO2015170796A1 (en) Vehicle and control method thereof
WO2011052827A1 (ko) 이동 로봇의 슬립 감지 장치 및 방법
EP3134764A1 (de) Kopfmontierte anzeige und verfahren zur steuerung davon
WO2011139070A2 (en) Method and apparatus for recognizing location of user
WO2015190650A1 (en) Wearable device and method of controlling therefor
WO2015122566A1 (en) Head mounted display device for displaying augmented reality image capture guide and control method for the same
WO2013069877A1 (en) Parking assisting system
WO2015163536A1 (en) Display device and method for controlling the same
WO2012144761A2 (ko) 3차원 이동과 회전 입력이 가능한 유니버셜 모션 컨트롤러
WO2017082539A1 (ko) 사용자 스타일링을 위한 증강현실 제공 장치 및 방법
EP3167610A1 (de) Anzeigevorrichtung mit akkreditierungsumfang zusammen mit einer tiefe eines virtuellen objekts und steuerungsverfahren dafür
WO2014148673A1 (en) 3d display device and method for controlling the same
WO2020153810A1 (en) Method of controlling device and electronic device
WO2019112114A1 (ko) 글래스 타입 단말기 및 그것의 동작방법
WO2016076463A1 (ko) 플라잉 봇의 제어 장치 및 제어 방법
WO2022039404A1 (ko) 광시야각의 스테레오 카메라 장치 및 이를 이용한 깊이 영상 처리 방법
JP2020137000A (ja) 映像記録装置およびヘッドマウントディスプレイ
WO2010058927A2 (ko) 안면 영상 촬영장치
WO2018164316A1 (ko) 전방향 영상 촬상 방법 및 이러한 방법을 수행하는 장치
WO2019004565A1 (ko) 증강현실 객체의 정합을 위한 캘리브레이션 방법 및 이를 수행하기 위한 헤드 마운트 디스플레이

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20190624

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

A4 Supplementary search report drawn up and despatched

Effective date: 20191108

RIC1 Information provided on ipc code assigned before grant

Ipc: G06F 3/0484 20130101ALI20191104BHEP

Ipc: G06F 3/01 20060101AFI20191104BHEP

Ipc: G06F 3/0487 20130101ALI20191104BHEP

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20210323

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20231201