WO2025001996A1 - 佩戴偏差检测方法、装置、电子设备及存储介质 - Google Patents

佩戴偏差检测方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2025001996A1
WO2025001996A1 PCT/CN2024/100639 CN2024100639W WO2025001996A1 WO 2025001996 A1 WO2025001996 A1 WO 2025001996A1 CN 2024100639 W CN2024100639 W CN 2024100639W WO 2025001996 A1 WO2025001996 A1 WO 2025001996A1
Authority
WO
WIPO (PCT)
Prior art keywords
eye
deviation
center point
coordinates
feature points
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/100639
Other languages
English (en)
French (fr)
Inventor
池立盈
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2025001996A1 publication Critical patent/WO2025001996A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the embodiments of the present disclosure relate to the field of virtual reality technology, and in particular to a wearing deviation detection method, device, electronic device, and storage medium.
  • the terminal devices worn by users will perform eye tracking on the user's eyes to achieve gaze point detection, thereby obtaining better image display effects and human-computer interaction purposes.
  • eye tracking technology is usually achieved by processing eye images taken by a terminal device worn by the user.
  • the wearing posture of the terminal device is abnormal, the accuracy of gaze point detection will be reduced, thereby affecting the image display effect and human-computer interaction efficiency.
  • the embodiments of the present disclosure provide a wearing deviation detection method, device, electronic device and storage medium to overcome the problem that the accuracy of gaze point detection is reduced when the wearing posture of the terminal device is abnormal.
  • an embodiment of the present disclosure provides a wearing deviation detection method, comprising: acquiring an eye image; obtaining at least two eye feature points based on the eye image, wherein the eye feature points are distributed on the contour of the eye area in the eye image; obtaining deviation information based on position characteristics of at least two of the eye feature points, wherein the deviation information characterizes the deviation type and/or deviation amount of the current wearing posture of the terminal device relative to the standard wearing posture.
  • the present disclosure provides a wearing deviation detection device, comprising: a collection module for collecting eye images; a processing module for obtaining at least two eye images according to the eye images; Feature points, the eye feature points are distributed on the contour of the eye area in the eye image; a detection module is used to obtain deviation information based on the position characteristics of at least two of the eye feature points, and the deviation information represents the deviation direction and/or deviation distance of the current wearing posture of the terminal device relative to the standard wearing posture.
  • an embodiment of the present disclosure provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the wearing deviation detection method described in the first aspect and various possible designs of the first aspect.
  • an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored.
  • a processor executes the computer execution instructions, the wearing deviation detection method described in the first aspect and various possible designs of the first aspect is implemented.
  • an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the wearing deviation detection method as described in the first aspect and various possible designs of the first aspect.
  • FIG1 is a diagram showing an application scenario of a wearing deviation detection method provided by an embodiment of the present disclosure
  • FIG2 is a flow chart of a wearing deviation detection method according to an embodiment of the present disclosure
  • FIG3 is a flow chart of a specific implementation method of step S102 in the embodiment shown in FIG2 ;
  • FIG4 is a schematic diagram of eye feature points provided by an embodiment of the present disclosure.
  • FIG5 is a flow chart of a specific implementation method of step S1023 in the embodiment shown in FIG3 ;
  • FIG6 is a schematic diagram of a process for determining a position extreme point provided by an embodiment of the present disclosure
  • FIG7 is a flow chart of a specific implementation of step S103 in the embodiment shown in FIG2 ;
  • FIG8 is a flowchart of a specific implementation method of step S1031 in the embodiment shown in FIG5 ;
  • FIG9 is a schematic diagram of a longitudinal center point coordinate provided by an embodiment of the present disclosure.
  • FIG10 is a second flow chart of a wearing deviation detection method provided in an embodiment of the present disclosure.
  • FIG11 is a flowchart of a possible specific implementation of step S205 in the embodiment shown in FIG10 ;
  • FIG12 is a flowchart of another possible specific implementation of step S205 in the embodiment shown in FIG10 ;
  • FIG13 is a schematic diagram of second deviation information provided by an embodiment of the present disclosure.
  • FIG14 is a structural block diagram of a wearing deviation detection device provided in an embodiment of the present disclosure.
  • FIG15 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure.
  • FIG. 16 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure.
  • user information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • user information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • FIG1 is an application scenario diagram of the wearing deviation detection method provided in an embodiment of the present disclosure.
  • the wearing deviation detection method provided in an embodiment of the present disclosure can be applied to products based on virtual reality (VR) and augmented reality (AR). More specifically, it can be applied to the scenario of pre-use detection of wearable terminal devices with eye tracking functions.
  • the method provided in an embodiment of the present disclosure can be applied to wearable terminal devices, such as the virtual reality glasses 11 shown in the figure.
  • the virtual reality glasses 11 detects the current wearing posture (position, angle) of the virtual reality glasses 11 by executing the wearing deviation detection method provided in this embodiment, and feeds back the detection results to the user in the form of images and sounds.
  • the virtual reality glasses 11 play a voice "The wearing position is 10 mm to the left" Users can adjust the wearing posture of virtual reality glasses according to the above detection results, so as to achieve more accurate eye tracking effect.
  • the eye tracking function implemented by the wearable terminal device is usually achieved by taking the user's eye image by a camera facing the user's eye in the terminal device, and then processing the eye image to identify the position of the light spot formed by corneal reflection in the eye image, thereby realizing the gaze point detection based on the light spot position and realizing the eye tracking function.
  • the wearing posture of the terminal device is abnormal, the quality of the light spot in the captured eye image will be reduced, which will further reduce the accuracy of the gaze point detection and affect the effect of eye tracking.
  • the user will only be notified when there is a serious or large posture abnormality.
  • FIG. 2 is a flow chart of a wearing deviation detection method provided in an embodiment of the present disclosure.
  • the method of this embodiment can be applied in a terminal device, and the wearing deviation detection method includes:
  • Step S101 collecting eye images.
  • Step S102 obtaining at least two eye feature points according to the eye image, wherein the eye feature points are distributed on the contour of the eye region in the eye image.
  • the execution subject of the method provided in this embodiment can be a wearable terminal device, such as smart glasses, virtual reality helmets, etc.
  • An image acquisition unit (such as a camera) facing the user's eyes is set in the terminal device. After the terminal device receives a user instruction or detects that the terminal device is worn by the user, the image acquisition unit is used to capture the user's eyes, thereby obtaining an eye image. More specifically, the image contains an eye area.
  • the eye image in this embodiment can be an image containing the eye area corresponding to the user's binocular eyes in the same frame image, or it can be an image containing only the eye area corresponding to the user's monocular eyes.
  • the eye image can be a binocular image containing the left eye area and the right eye area, or it can be two frames of monocular images (i.e., a left eye image and a left eye image).
  • multiple image acquisition units can be set in the terminal device, so as to capture the user's eyes from different angles at the same time, so as to obtain eye images synchronously captured at the same time.
  • the specific implementation method can be set as needed, which will not be repeated here.
  • step S101 includes:
  • Step S1021 extracting features of the eye image to obtain a first feature map, where the first feature map represents pixel features of the eye image;
  • Step S1022 Based on the pre-trained prediction model, the first feature map is processed to determine the contour line of the eye area in the eye image;
  • Step S1023 obtaining at least two eye feature points based on the contour line of the eye area.
  • FIG4 is a schematic diagram of an eye feature point provided by an embodiment of the present disclosure.
  • the eye image is subjected to feature extraction to obtain a first feature map, and the first feature map characterizes the arrangement characteristics of the pixels in the eye image, that is, the pixel features.
  • the first feature map is used as input and processed using a pre-trained prediction model to obtain the pixel points corresponding to the contour of the eye area in the eye image, that is, the contour line of the eye area, which is formed by connecting a plurality of contour nodes.
  • At least two eye feature points can be determined, such as the eye feature point P1 and the eye feature point P2 shown in the image (shown as P1 and P2 in the figure).
  • the above-mentioned two eye feature points are respectively located at the left end point and the right end point of the eye area, corresponding to the left corner of the eye and the right corner of the eye of the human eye, respectively.
  • the method for determining the above-mentioned eye feature point P1 and eye feature point P2 can be achieved by obtaining the coordinates of each contour node of the contour line corresponding to the eye area, and determining the leftmost contour node as the eye feature point P1 and the rightmost contour node as the eye feature point P2 according to the coordinates.
  • step S1023 in the process of determining the eye feature points, at least two extreme position points in the horizontal direction and the vertical direction (or other two vertical directions) can be determined based on the circumscribed rectangle of the contour line, and at least four extreme position points are used as eye feature points.
  • the coordinates of the at least four extreme position points can be determined according to the shape of the contour line.
  • the specific implementation of step S1023 includes:
  • Step S1023A Obtain at least four position extreme value points of the contour line of the eye area according to the tangent direction of each contour node in the contour line.
  • Step S1023B Obtain at least four eye feature points based on at least four position extreme value points.
  • At least four contour lines of the eye area can be determined based on the tangent direction of each contour node in the contour line and according to the change of the tangent direction. Position extreme value points, and then obtain the corresponding at least four eye feature points.
  • a contour line composed of contour line nodes is obtained, and the implementation method of the contour line can be: an array composed of the coordinates of multiple contour line nodes.
  • the direction vectors of each contour line node and the adjacent contour line nodes are calculated to obtain the tangent direction of each contour node (that is, the vector node of the direction vector), and the specific implementation method will not be repeated.
  • the contour node with a specific value of the tangent direction (for example, 0) is determined as the position extreme value point.
  • FIG6 is a schematic diagram of a process for determining position extreme points provided by an embodiment of the present disclosure.
  • the leftmost position extreme point P1 and the rightmost position extreme point P2 are first determined according to the coordinates of the contour nodes, and the specific implementation method is not repeated here.
  • the target direction vectors of the position extreme point P1 and the position extreme point P2, and the direction Seta_1 of the target direction vector are calculated.
  • the tangent direction of each contour node is calculated, and the contour nodes P3 and P4 corresponding to the tangent direction that is the same as the direction Seta_1 of the target direction vector vec_1 are determined as position extreme points.
  • the contour node whose tangent direction is parallel to the eye area is obtained as the position extreme point, so that the distribution of the position extreme point on the eye contour is more uniform and reasonable, thereby improving the accuracy of the subsequent calculation of the deviation information based on the eye feature points.
  • the contour nodes on the contour line of the eye area can be further selected to increase the number of eye feature points.
  • a number of eye feature points are uniformly added between adjacent basic points to improve the accuracy of calculating subsequent deviation information.
  • there is a first limiting distance between any two adjacent eye feature points and the first limiting distance is determined based on the resolution of the eye image. That is, when the resolution of the eye image is higher, the number of eye feature points inserted between the basic points is relatively more.
  • the resolution of the eye image is lower, the number of eye feature points inserted between the basic points is less.
  • the content information in the eye image is fully utilized, the position error of the eye contour is reduced, the accuracy of the eye area is improved, and the accuracy of the subsequent deviation information calculated based on the contour line of the eye area is improved.
  • Step S103 obtaining deviation information according to the positional features of at least two eye feature points, wherein the deviation information represents the deviation type and/or deviation amount of the current wearing posture of the terminal device relative to the standard wearing posture.
  • the deviation information corresponding to the terminal device is determined based on the position features formed by the positional relationship between the eye feature points, that is, the deviation type and/or deviation amount of the current wearing posture of the terminal device relative to the standard wearing posture.
  • the deviation types include: wearing to the left, wearing to the right, wearing to the upper left, wearing tilted (high on the left and low on the right, high on the right and low on the left), etc.
  • the deviation amounts include: wearing 10 mm to the left, wearing 5 mm to the upper right, wearing tilted 10 degrees, etc.
  • the position features include the position coordinates of the eye feature points in the eye image, as shown in FIG7 , and the specific implementation of step S103 includes:
  • Step S1031 Obtain eye center point coordinates based on the position coordinates of at least two eye feature points.
  • Step S1032 Obtain a deviation vector according to the eye center coordinates and the reference coordinates, where the reference coordinates are the standard eye center coordinates corresponding to the standard wearing posture.
  • Step S1033 Obtain deviation information according to the deviation vector.
  • the position coordinates of the eye feature points can be obtained, such as the coordinates of the leftmost position extreme point and the rightmost position extreme point, that is, the coordinates of the reference point referred to in the above embodiment. Afterwards, based on the shape characteristics of the eye area, the coordinates of the leftmost position extreme point and the rightmost position extreme point are averaged to obtain the approximate coordinates of the center point of the eye area, that is, the eye center point coordinates.
  • the eye feature points used in the above steps may also include the topmost position extreme point, the bottommost position extreme point, and other eye feature points located between the reference points, and the coordinates of all the above uniformly distributed eye feature points are averaged to obtain the eye center point coordinates, thereby improving the accuracy of the eye center point coordinates.
  • the specific implementation method can be set as needed and will not be repeated here.
  • the preset reference coordinates i.e., the standard eye center coordinates corresponding to the standard wearing posture
  • the eye center coordinates and the reference coordinates are calculated to obtain the deviation vector between the two, and then the corresponding direction and distance, i.e., the deviation information, are obtained based on the deviation vector.
  • the deviation information can be a value of [direction, distance] in the form of a key-value pair, or it can be data in other forms, which is not specifically limited here.
  • the eye center coordinates are calibrated by pre-calibrated reference coordinates to obtain deviation information that characterizes the type and/or amount of deviation of the wearing posture, thereby realizing the inspection and calibration of the wearing position without using the light spot reflected by the iris, thereby improving the accuracy and stability of subsequent gaze point detection.
  • step S1031 includes:
  • Step S1031A Acquire shooting angle information, where the shooting angle information represents the shooting angle when the image acquisition unit captures the eye image.
  • Step S1031B Determine the longitudinal weighting coefficient according to the shooting angle information.
  • Step S1031C Perform weighted averaging on the longitudinal coordinates according to the longitudinal weighting coefficient to obtain the longitudinal center point coordinates.
  • Step S1031D Obtain the eye center coordinates based on the longitudinal center coordinates.
  • the shooting angle information is the shooting angle of an image acquisition unit, such as a camera built into a terminal device, when shooting an eye image.
  • the position of the corresponding built-in image acquisition unit may be different, so the captured eye image will produce a corresponding pitch angle.
  • the shooting angle information corresponds to the model of the terminal device one by one.
  • the shooting angle information is preset in the terminal device, or a fixed value obtained by the terminal device through a server.
  • the shooting angle information can be represented by an angle value or a vector value, without specific limitation.
  • FIG9 is a schematic diagram of a longitudinal center point coordinate provided by an embodiment of the present disclosure.
  • the corresponding shooting angle information is Info_1
  • the corresponding shooting angle is Theta_1.
  • the image acquisition unit D1 calculates the eye center point coordinate
  • the coordinates of each eye feature point are averaged without considering the shooting angle information, and the eye center point coordinate obtained is P1.
  • the corresponding weighting coefficient is obtained, and after the ordinate of each eye feature point is weighted, the coordinate of the eye center point is averaged based on the coordinate of the eye feature point, and the eye center point coordinate obtained is P2.
  • the eye center point coordinate P2 after weighted processing by the shooting angle information Info_1 is closer to the actual eye center point position.
  • the longitudinal weighting coefficient is obtained by shooting angle information, and the longitudinal coordinates are weighted averaged to make the generated longitudinal center point coordinates more accurate, that is, the eye center point coordinates are more accurate, thereby improving the accuracy of the subsequent deviation information obtained based on the eye center point coordinates.
  • an eye image is collected; at least two eye feature points are obtained according to the eye image, and the eye feature points are distributed on the contour of the eye area in the eye image; deviation information is obtained according to the position characteristics of the at least two eye feature points, and the deviation information represents the current wearing position of the terminal device.
  • FIG10 is a second flow chart of the wearing deviation detection method provided by the embodiment of the present disclosure. Based on the embodiment shown in FIG2 , this embodiment further refines step S103 , and the wearing deviation detection method includes:
  • Step S201 collecting eye images, the eye images include left eye images and left eye images.
  • Step S202 obtaining at least two eye feature points according to the eye image, where the eye feature points include a first feature point corresponding to the left eye image and a second feature point corresponding to the right eye image.
  • the image acquisition unit acquires the left eye image corresponding to the left eye and the right eye image corresponding to the right eye respectively, and then processes the left eye image and the right eye image respectively, so as to obtain the first feature point corresponding to the left eye image and the second feature point corresponding to the right eye image.
  • the first feature point is located on the contour of the eye area in the left eye image
  • the second feature point is located on the contour of the eye area in the right eye image;
  • the specific method of obtaining the first feature point corresponding to the left eye image and the second feature point corresponding to the right eye image can refer to the introduction of the specific implementation method of obtaining eye feature points for a monocular image in the embodiment shown in FIG2, which will not be repeated here.
  • the left eye image and the right eye image may be acquired by different image acquisition units.
  • the left eye image and the right eye image may be aligned first, that is, the two may be converted to the same image coordinate system, and the alignment may be performed by referring to the positional relationship between the image acquisition units. The details are not repeated here.
  • the aligned left eye image and the right eye image are processed to obtain the corresponding first feature points and second feature points.
  • the obtained first feature points and second feature points are located in the same image coordinate system, which is convenient for subsequent distance measurement.
  • the left eye image and the right eye image may be taken by the same image acquisition unit, and the left eye image and the right eye image may be obtained by (logically) dividing the acquired eye image, and the left eye image and the right eye image may be respectively shown as an image area or pixel set of the eye image.
  • the left eye image and the right eye image are always in the same image coordinate system, and the eye image can be directly processed in the subsequent processing process.
  • Step S203 Obtain the left eye region corresponding to the left eye image according to the position feature of the first feature point The coordinates of the first center point.
  • Step S204 obtaining the coordinates of the second center point corresponding to the right eye area in the left eye image according to the positional features of the second feature point.
  • the first center point coordinates of the left eye region in the left eye image and the second center point coordinates corresponding to the right eye region in the right eye image can be obtained.
  • the first center point coordinates can be obtained by calculating the weighted average of the coordinates of each first feature point in the left eye image.
  • the specific implementation method can refer to the calculation method of the eye center point coordinates in the embodiment shown in FIG2, which will not be repeated here.
  • Step S205 Obtain deviation information according to the positional relationship between the first center point coordinates and the second center point coordinates.
  • deviation information is obtained, that is, the deviation type and/or deviation amount of the current wearing posture of the terminal device relative to the standard wearing posture.
  • the horizontality of the line formed by the first center point coordinates and the second center point coordinates is used to obtain the deviation information characterizing whether the terminal device is worn horizontally.
  • step S205 includes:
  • Step S2051 Calculate a weighted sum based on the first center point coordinates and the second center point coordinates to obtain weighted center point coordinates;
  • Step S2052 Obtain first deviation information based on the positional relationship between the weighted center point coordinates and the reference coordinates.
  • the first deviation information represents the deviation distance of the current wearing posture of the terminal device relative to the standard wearing posture in the horizontal direction or the vertical direction.
  • the center point coordinates of the first center point coordinate and the second center point coordinate are calculated, that is, the center point coordinates are obtained by calculating the average value of the first center point coordinate and the second center point coordinate.
  • the horizontal weighting coefficient and the vertical weighting coefficient corresponding to the first center point coordinate and the second center point coordinate are obtained, and then based on the horizontal weighting coefficient and the vertical weighting coefficient, the first center point coordinate and the second center point coordinate are weighted respectively, and the weighted average value is calculated to obtain the weighted center point coordinate.
  • the specific method for obtaining the horizontal weighting coefficient and the vertical weighting coefficient can refer to the relevant introduction of obtaining the vertical weighting coefficient in the embodiment shown in Figure 2, which will not be repeated here.
  • the first deviation information is obtained based on the positional relationship between the weighted center point coordinates and the reference coordinates.
  • the first deviation information characterizes the deviation distance of the current wearing posture of the terminal device relative to the standard wearing posture in the horizontal direction and/or vertical direction, that is, the first deviation information characterizes whether the current wearing position of the terminal device is offset in the horizontal and vertical directions, and the specific offset distance.
  • the first deviation information Info_1 [10, -20], which indicates that the current wearing posture of the terminal device is 10 mm to the left in the horizontal direction and 20 mm below in the vertical direction relative to the standard posture.
  • the corresponding weighted center point coordinates are obtained through the first center point coordinates and the second center point, and based on the weighted center point coordinates, the first deviation information characterizing the wearing posture deviation of the terminal device in the horizontal direction or the vertical direction is obtained.
  • the accuracy and stability are better, and the accuracy of gaze point detection is further improved.
  • step S205 includes:
  • Step S2053 Generate an inclination vector according to the first center point coordinates and the second center point coordinates.
  • Step S2054 obtaining second deviation information according to the direction of the inclination vector, where the second deviation information represents the inclination angle of the current wearing posture of the terminal device relative to the standard wearing posture.
  • FIG13 is a schematic diagram of a second deviation information provided by an embodiment of the present disclosure.
  • the first center point coordinates P1 and the second center point coordinates P2 are connected to obtain the inclination vector Vp (shown as Vp in the figure). Afterwards, the inclination angle of the inclination vector Vp in the image coordinate system is obtained, that is, the angle Theta relative to the horizontal direction (0 degrees), and the second deviation information is generated.
  • Theta when Theta is greater than zero, it represents that the wearing posture of the terminal device is left-leaning relative to the standard wearing posture (the terminal device is worn horizontally), that is, the left is low and the right is high; and when Theta is less than zero, it represents that the wearing posture of the terminal device is right-leaning relative to the standard wearing posture, that is, the left is high and the right is low.
  • the inclination vector is generated through the coordinates of the first center point and the second center point, and the second deviation information representing the wearing inclination angle of the terminal device is generated based on the inclination vector, so as to realize Now let's detect the tilt posture of the terminal device.
  • the step of obtaining the deviation information (second deviation information) provided in the steps of this embodiment can be combined with the step of obtaining the deviation information (first deviation information) provided in the embodiment shown in Figure 11, that is, the terminal device synchronously or asynchronously obtains the first deviation information and the second deviation information (without limiting the order), and then uses the set of the first deviation information and the second deviation information as the deviation information for subsequent steps, thereby realizing posture detection in multiple dimensions of horizontal, vertical, and tilt, and further improving the accuracy of gaze point detection.
  • step S205 the method further includes:
  • Step S206 If the deviation information is greater than or equal to the deviation threshold, first indication information generated based on the deviation information is played in the terminal device, and the process returns to step S201, wherein the first indication information is used to indicate the action of adjusting the terminal device;
  • Step S207 If the deviation information is less than the deviation threshold, the second indication information is played in the terminal device, and the second indication information indicates that the current wearing posture of the terminal device is the standard wearing posture.
  • a first prompt message is generated based on the deviation information, and the first indication message is played to the user through the human-computer interaction unit of the terminal device to prompt the user.
  • the deviation information is Info_1
  • the content represented is: the wearing posture is biased to the left (deviation type), then the corresponding first prompt message generated is an icon of "wearing position is biased to the left", or voice or text "please move the terminal device to the right”.
  • the deviation information is Info_2
  • the content represented is: the wearing posture is tilted downward by 10 degrees on the left side (deviation type and deviation amount), then the corresponding first prompt message generated is an icon of "wearing position is tilted downward by 10 degrees on the left side", or voice or text "please rotate the left side of the terminal device upward by 10 degrees”.
  • the process returns to step S201, real-time eye images are collected again, and the above steps are repeated to achieve continuous detection and prompting of the wearing posture, until the deviation information is less than the deviation threshold, and the second indication information is displayed to inform the user that the current wearing posture of the terminal device is the standard posture, and the subsequent gaze point detection function can be executed and started.
  • the first indication information and the second indication information include at least one of the following: prompt text, logo, voice message, which can be set as needed and are not specifically limited here.
  • the user is automatically guided to manually adjust the wearing posture of the terminal device, so that the user can adjust the terminal device to the standard wearing posture faster and more accurately, thereby improving the efficiency and accuracy of the wearing posture adjustment. Further improve the accuracy of subsequent gaze point detection.
  • FIG14 is a structural block diagram of a wearing deviation detection device provided by an embodiment of the present disclosure.
  • the wearing deviation detection device 3 includes:
  • a processing module 32 configured to obtain at least two eye feature points according to the eye image, wherein the eye feature points are distributed on the contour of the eye region in the eye image;
  • the detection module 33 is used to obtain deviation information based on the position characteristics of at least two eye feature points, where the deviation information represents the deviation direction and/or deviation distance of the current wearing posture of the terminal device relative to the standard wearing posture.
  • the processing module 32 is specifically used to: extract features from the eye image to obtain a first feature map, the first feature map representing pixel features of the eye image. Based on the pre-trained prediction model, the first feature map is processed to determine the contour line of the eye area in the eye image. Based on the contour line of the eye area, at least two eye feature points are obtained.
  • the processing module 32 when the processing module 32 obtains at least two eye feature points based on the contour line of the eye area, it is specifically used to: obtain at least four position extreme value points of the contour line of the eye area according to the tangent direction of each contour node in the contour line. According to the at least four position extreme value points, at least four eye feature points are obtained.
  • the first limiting distance is determined based on the resolution of the eye image.
  • the position feature includes the position coordinates of the eye feature points in the eye image.
  • the detection module 33 is specifically used to: obtain the eye center point coordinates according to the position coordinates of at least two eye feature points. Obtain a deviation vector according to the eye center point coordinates and reference coordinates, where the reference coordinates are the standard eye center point coordinates corresponding to the standard wearing posture. Obtain deviation information according to the deviation vector.
  • the position coordinates include longitudinal coordinates; when the detection module 33 obtains the eye center coordinates according to the position coordinates of at least two eye feature points, it is specifically used to: obtain shooting angle information, the shooting angle information represents the shooting angle when the image acquisition unit shoots the eye image. According to the shooting angle information, determine the longitudinal weighting coefficient. According to the longitudinal weighting coefficient, perform weighted averaging on the longitudinal coordinates to obtain the longitudinal center coordinates. According to the longitudinal center coordinates, obtain the eye center coordinates. Mark.
  • the eye image includes a left eye image and a right eye image
  • the eye feature points include a first feature point corresponding to the left eye image and a second feature point corresponding to the left eye image
  • the detection module 33 is specifically used to: obtain the first center point coordinates corresponding to the left eye area in the left eye image according to the positional features of the first feature point.
  • the detection module 33 when the detection module 33 obtains the deviation information according to the positional relationship between the first center point coordinates and the second center point coordinates, it is specifically used to: calculate the weighted sum based on the first center point coordinates and the second center point coordinates to obtain the weighted center point coordinates. According to the positional relationship between the weighted center point coordinates and the reference coordinates, the first deviation information is obtained, and the first deviation information represents the deviation distance of the current wearing posture of the terminal device relative to the standard wearing posture in the horizontal direction or the vertical direction.
  • the detection module 33 when the detection module 33 obtains the deviation information according to the positional relationship between the first center point coordinates and the second center point coordinates, it is specifically used to: generate an inclination vector according to the first center point coordinates and the second center point coordinates. According to the direction of the inclination vector, the second deviation information is obtained, and the second deviation information represents the inclination angle of the current wearing posture of the terminal device relative to the standard wearing posture.
  • the detection module 33 is further used to: play indication information generated based on the deviation information in the terminal device until the deviation information is less than the deviation threshold, the indication information is used to characterize the action of adjusting the terminal device, wherein the indication information includes at least one of the following: prompt text, logo, voice message.
  • the collection module 31, the processing module 32 and the detection module 33 are connected in sequence.
  • the wearing deviation detection device 3 provided in this embodiment can implement the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated in this embodiment.
  • FIG. 15 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. As shown in FIG. 15 , the electronic device 4 includes:
  • the memory 42 stores computer executable instructions
  • the processor 41 executes the computer-executable instructions stored in the memory 42 to implement the wearing deviation detection method in the embodiments shown in Figures 2 to 13.
  • processor 41 and the memory 42 are connected via a bus 43 .
  • An embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored.
  • the computer-executable instructions are executed by a processor, they are used to implement the wearing deviation detection method provided in any of the embodiments corresponding to Figures 2 to 13 of the present disclosure.
  • the present disclosure provides a computer program product, including a computer program.
  • the computer program is executed by a processor, the wearing deviation detection method in the embodiments shown in FIGS. 2 to 13 is implemented.
  • FIG. 16 it shows a schematic diagram of the structure of an electronic device 900 suitable for implementing the embodiment of the present disclosure
  • the electronic device 900 may be a terminal device or a server.
  • the terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
  • PDAs personal digital assistants
  • PADs Portable Android Devices
  • PMPs portable multimedia players
  • vehicle terminals such as vehicle navigation terminals
  • fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG. 16 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.
  • the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903.
  • a processing device e.g., a central processing unit, a graphics processing unit, etc.
  • RAM random access memory
  • Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 903.
  • the processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904.
  • An input/output (I/O) interface 905 is also connected to the bus 904.
  • the following devices may be connected to the I/O interface 905: input devices 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909.
  • the communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data.
  • FIG. 16 shows an electronic device 900 having various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively. Set.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902.
  • the processing device 901 the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
  • the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.
  • Computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried.
  • This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above.
  • the computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device.
  • the program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
  • the computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
  • the computer-readable medium carries one or more programs.
  • the electronic device executes the method shown in the above embodiment.
  • Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages.
  • the program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer may be connected to the user's computer via any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
  • LAN Local Area Network
  • WAN Wide Area Network
  • each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function.
  • the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
  • each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or hardware.
  • the name of a unit does not limit the unit itself in some cases.
  • the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".
  • exemplary types of hardware logic components include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOCs systems on chips
  • CPLDs complex programmable logic devices
  • a machine-readable medium may be a tangible medium that can contain or store information for use by or in conjunction with an instruction execution system, apparatus, or device.
  • the machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • the machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination of the foregoing.
  • machine-readable storage media may include electrical connections based on one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
  • RAM random access memories
  • ROM read-only memories
  • EPROM or flash memory erasable programmable read-only memories
  • CD-ROMs portable compact disk read-only memories
  • magnetic storage devices or any suitable combination of the foregoing.
  • a wearing deviation detection method comprising:
  • Capture an eye image obtain at least two eye feature points based on the eye image, wherein the eye feature points are distributed on the contour of the eye area in the eye image; obtain deviation information based on positional characteristics of the at least two eye feature points, wherein the deviation information characterizes the deviation type and/or deviation amount of the current wearing posture of the terminal device relative to the standard wearing posture.
  • obtaining at least two eye feature points based on the eye image includes: performing feature extraction on the eye image to obtain a first feature map, wherein the first feature map represents pixel features of the eye image; processing the first feature map based on a pre-trained prediction model to determine a contour line of an eye region in the eye image; and obtaining at least two eye feature points based on the contour line of the eye region.
  • At least two eye feature points are obtained based on the contour line of the eye area, including: obtaining at least four position extreme points of the contour line of the eye area according to the tangent direction of each contour node in the contour line; and obtaining at least four eye feature points according to the at least four position extreme points.
  • the first limiting distance is determined based on the resolution of the eye image.
  • the position features include the position coordinates of the eye feature points in the eye image; the deviation information is obtained based on the position features of at least two of the eye feature points, including: obtaining the eye center point coordinates based on the position coordinates of at least two of the eye feature points; obtaining a deviation vector based on the eye center point coordinates and reference coordinates, the reference coordinates being the standard eye center point coordinates corresponding to the standard wearing posture; and obtaining the deviation information based on the deviation vector.
  • the eye image includes a left-eye image and a right-eye image
  • the eye feature points include a first feature point corresponding to the left-eye image and a second feature point corresponding to the right-eye image
  • the deviation information is obtained based on the position characteristics of at least two of the eye feature points, including: obtaining the first center point coordinates corresponding to the left-eye area in the left-eye image based on the position characteristics of the first feature point; obtaining the second center point coordinates corresponding to the right-eye area in the left-eye image based on the position characteristics of the second feature point; and obtaining the deviation information based on the positional relationship between the first center point coordinates and the second center point coordinates.
  • the deviation information is obtained according to the positional relationship between the first center point coordinates and the second center point coordinates, including: calculating a weighted sum based on the first center point coordinates and the second center point coordinates to obtain weighted center point coordinates; and obtaining first deviation information according to the positional relationship between the weighted center point coordinates and the reference coordinates, the first deviation information representing the deviation distance of the current wearing posture of the terminal device relative to the standard wearing posture in the horizontal direction or the vertical direction.
  • the deviation information is obtained according to the positional relationship between the first center point coordinates and the second center point coordinates, including: generating an inclination vector according to the first center point coordinates and the second center point coordinates; and obtaining second deviation information according to the direction of the inclination vector, wherein the second deviation information represents an inclination angle of the current wearing posture of the terminal device relative to the standard wearing posture.
  • the indication information after obtaining the deviation information based on the position characteristics of at least two of the eye feature points, it also includes: playing indication information generated based on the deviation information in the terminal device until the deviation information is less than a deviation threshold, the indication information is used to characterize the action of adjusting the terminal device, wherein the indication information includes at least one of the following: prompt text, logo, voice message.
  • a wearing deviation detection method comprising:
  • An acquisition module used for acquiring eye images
  • a processing module configured to obtain at least two eye feature points according to the eye image, wherein the eye feature points are distributed on the contour of the eye region in the eye image;
  • the detection module is used to obtain deviation information based on the position characteristics of at least two of the eye feature points, wherein the deviation information represents the deviation direction and/or deviation distance of the current wearing posture of the terminal device relative to the standard wearing posture.
  • the processing module is specifically used to: perform feature extraction on the eye image to obtain a first feature map, wherein the first feature map represents pixel features of the eye image; based on a pre-trained prediction model, process the first feature map to determine a contour line of an eye region in the eye image; and based on the contour line of the eye region, obtain at least two of the eye feature points.
  • the processing module when the processing module obtains at least two of the eye feature points based on the contour line of the eye area, it is specifically used to: obtain at least four position extreme points of the contour line of the eye area according to the tangent direction of each contour node in the contour line; and obtain at least four of the eye feature points according to the at least four position extreme points.
  • the first limiting distance is determined based on the resolution of the eye image.
  • the position features include the position coordinates of the eye feature points in the eye image; the detection module is specifically used to: obtain the eye center point coordinates based on the position coordinates of at least two of the eye feature points; obtain a deviation vector based on the eye center point coordinates and reference coordinates, the reference coordinates being the standard eye center point coordinates corresponding to the standard wearing posture; and obtain the deviation information based on the deviation vector.
  • the position coordinates include longitudinal coordinates; when the detection module 33 obtains the eye center point coordinates based on the position coordinates of the at least two eye feature points, it is specifically used to: obtain shooting angle information, the shooting angle information represents the shooting angle when the image acquisition unit captures the eye image; determine the longitudinal weighting coefficient based on the shooting angle information; perform weighted averaging on the longitudinal coordinates based on the longitudinal weighting coefficient to obtain the longitudinal center point coordinates; obtain the eye center point coordinates based on the longitudinal center point coordinates.
  • the eye image includes a left eye image and a left eye image.
  • the eye feature points include a first feature point corresponding to the left-eye image and a second feature point corresponding to the left-eye image;
  • the detection module is specifically used to: obtain the first center point coordinates corresponding to the left-eye area in the left-eye image according to the positional features of the first feature points; obtain the second center point coordinates corresponding to the right-eye area in the left-eye image according to the positional features of the second feature points; and obtain the deviation information according to the positional relationship between the first center point coordinates and the second center point coordinates.
  • the detection module when the detection module obtains the deviation information based on the positional relationship between the first center point coordinates and the second center point coordinates, it is specifically used to: calculate a weighted sum based on the first center point coordinates and the second center point coordinates to obtain weighted center point coordinates; obtain first deviation information based on the positional relationship between the weighted center point coordinates and the reference coordinates, the first deviation information characterizing the deviation distance of the current wearing posture of the terminal device relative to the standard wearing posture in the horizontal direction or vertical direction.
  • the detection module when the detection module obtains the deviation information based on the positional relationship between the first center point coordinates and the second center point coordinates, it is specifically used to: generate an inclination vector based on the first center point coordinates and the second center point coordinates; obtain second deviation information based on the direction of the inclination vector, the second deviation information characterizing the inclination angle of the current wearing posture of the terminal device relative to the standard wearing posture.
  • the detection module is further used to: play indication information generated based on the deviation information in the terminal device until the deviation information is less than a deviation threshold, the indication information being used to characterize the action of adjusting the terminal device, wherein the indication information includes at least one of the following: prompt text, logo, voice message.
  • an electronic device comprising: a processor, and a memory communicatively connected to the processor;
  • the memory stores computer-executable instructions
  • the processor executes the computer-executable instructions stored in the memory to implement the wearing deviation detection method as described in the first aspect and various possible designs of the first aspect.
  • a computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the first aspect and various possible designs of the first aspect are implemented.
  • an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the wearing deviation detection method as described in the first aspect and various possible designs of the first aspect.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Ophthalmology & Optometry (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

本公开实施例提供一种佩戴偏差检测方法、装置、电子设备及存储介质，通过采集眼部图像；根据眼部图像，得到至少两个眼部特征点，眼部特征点分布于眼部图像中眼部区域的轮廓上；根据至少两个眼部特征点的位置特征，得到偏差信息，偏差信息表征终端设备的当前佩戴位姿相对标准佩戴位姿的偏差类型和/或偏差量。通过采集眼部图像，并基于眼部图像中眼部区域的轮廓上的眼部特征点，得到终端设备的当前佩戴位姿相对标准佩戴位姿的偏差类型和/或偏差量，实现佩戴位姿异常时的提前检测和提示，避免由于终端设备的佩戴位姿异常而导致的注视点检测的准确率降低的问题，提高图像显示效果和人机交互效率。

Description

佩戴偏差检测方法、装置、电子设备及存储介质
本申请要求2023年6月26日递交的、标题为“佩戴偏差检测方法、装置、电子设备及存储介质”、申请号为2023107649197的中国发明专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开实施例涉及虚拟现实技术领域，尤其涉及一种佩戴偏差检测方法、装置、电子设备及存储介质。
背景技术
当前，在基于虚拟现实(Virtual Reality，VR)的应用场景中，用户佩戴的终端设备会针对用户眼部进行眼动追踪，以实现注视点检测，从而获得更好的图像显示效果和人机交互目的。
现有技术中，眼动追踪技术通常是通过用户佩戴的终端设备所拍摄的眼部图像进行处理而实现的，而当终端设备的佩戴位姿异常时，会导致注视点检测的准确率降低的问题，进而影响图像显示效果和人机交互效率。
发明内容
本公开实施例提供一种佩戴偏差检测方法、装置、电子设备及存储介质，以克服终端设备的佩戴位姿异常时，导致注视点检测的准确率降低的问题。
第一方面，本公开实施例提供一种佩戴偏差检测方法，包括：采集眼部图像；根据所述眼部图像，得到至少两个眼部特征点，所述眼部特征点分布于所述眼部图像中眼部区域的轮廓上；根据至少两个所述眼部特征点的位置特征，得到偏差信息，所述偏差信息表征所述终端设备的当前佩戴位姿相对标准佩戴位姿的偏差类型和/或偏差量。
第二方面，本公开实施例提供一种佩戴偏差检测装置，包括：采集模块，用于采集眼部图像；处理模块，用于根据所述眼部图像，得到至少两个眼部 特征点，所述眼部特征点分布于所述眼部图像中眼部区域的轮廓上；检测模块，用于根据至少两个所述眼部特征点的位置特征，得到偏差信息，所述偏差信息表征所述终端设备的当前佩戴位姿相对标准佩戴位姿的偏差方向和/或偏差距离。
第三方面，本公开实施例提供一种电子设备，包括：处理器，以及与所述处理器通信连接的存储器；所述存储器存储计算机执行指令；所述处理器执行所述存储器存储的计算机执行指令，以实现如上第一方面以及第一方面各种可能的设计所述的佩戴偏差检测方法。
第四方面，本公开实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如上第一方面以及第一方面各种可能的设计所述的佩戴偏差检测方法。
第五方面，本公开实施例提供一种计算机程序产品，包括计算机程序，该计算机程序被处理器执行时实现如上第一方面以及第一方面各种可能的设计所述的佩戴偏差检测方法。
附图说明
为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本公开实施例提供的佩戴偏差检测方法的一种应用场景图；
图2为本公开实施例提供的佩戴偏差检测方法的流程示意图一；
图3为图2所示实施例中步骤S102的具体实现方式的流程图；
图4为本公开实施例提供的一种眼部特征点的示意图；
图5为图3所示实施例中步骤S1023的具体实现方式的流程图；
图6为本公开实施例提供的一种确定位置极值点的过程示意图；
图7为图2所示实施例中步骤S103的具体实现方式的流程图；
图8为图5所示实施例中步骤S1031的具体实现方式的流程图；
图9为本公开实施例提供的一种纵向中心点坐标的示意图；
图10为本公开实施例提供的佩戴偏差检测方法的流程示意图二；
图11为图10所示实施例中步骤S205的一种可能的具体实现方式的流程图；
图12为图10所示实施例中步骤S205的另一种可能的具体实现方式的流程图；
图13为本公开实施例提供的一种第二偏差信息的示意图；
图14为本公开实施例提供的佩戴偏差检测装置的结构框图；
图15为本公开实施例提供的一种电子设备的结构示意图；
图16为本公开实施例提供的电子设备的硬件结构示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
需要说明的是，本公开所涉及的用户信息(包括但不限于用户设备信息、用户个人信息等)和数据(包括但不限于用于分析的数据、存储的数据、展示的数据等)，均为经用户授权或者经过各方充分授权的信息和数据，并且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准，并提供有相应的操作入口，供用户选择授权或者拒绝。
下面对本公开实施例的应用场景进行解释：
图1为本公开实施例提供的佩戴偏差检测方法的一种应用场景图，本公开实施例提供的佩戴偏差检测方法，可以应用于基于虚拟现实(Virtual Reality，VR)、增强现实(Augmented Reality，AR)的产品中。更具体地，可以应用于具有眼动追踪功能的可穿戴式终端设备的使用前检测的场景下。如图1所示，本公开实施例提供的方法，可以应用于穿戴式的终端设备，例如图中所示的虚拟现实眼镜11，用户在佩戴上述虚拟现实眼镜11后，虚拟现实眼镜11通过执行本实施例提供的佩戴偏差检测方法，对虚拟现实眼镜11当前的佩戴位姿(位置、角度)进行检测，并将检测结果以图像、声音的方式反馈给用户。例如图中所示，虚拟现实眼镜11播放语音“佩戴位置左偏10毫米” 使用户能够根据上述检测结果对虚拟现实眼镜的佩戴位姿进行调整，从而实现更加准确的眼动追踪效果。
现有技术中，穿戴式终端设备所实现的眼动追踪功能，通常是由终端设备内设的朝向用户眼部的摄像头拍摄用户的眼部图像，再通过对眼部图像进行处理，识别眼部图像中由角膜反射而形成的光斑的位置，从而基于光斑位置实现注视点检测，实现眼动追踪功能。然而，当终端设备的佩戴位姿异常时，会导致所拍摄的眼部图像中的光斑的质量降低，进而导致注视点检测的准确率降低，影响眼动追踪的效果。现有技术中对于上述佩戴位姿异常的问题，只有在出现严重的、较大的位姿异常时，才会通知用户，而在位姿异常不严重的情况下，通常是采用软件修正的方式进行处理，这导致在后续使用过程中，由于终端设备的佩戴位姿存本身存在偏差，会有更大概率出现注视点跳变、晃动的问题，影响图像显示效果和人机交互效率。本公开实施例提供一种佩戴偏差检测方法以解决上述问题。
参考图2，图2为本公开实施例提供的佩戴偏差检测方法的流程示意图一。本实施例的方法可以应用在终端设备中，该佩戴偏差检测方法包括：
步骤S101：采集眼部图像。
步骤S102：根据眼部图像，得到至少两个眼部特征点，眼部特征点分布于眼部图像中眼部区域的轮廓上。
示例性地，参考图1所示应用场景示意图，本实施例所提供的方法的执行主体可以为穿戴式的终端设备，例如智能眼镜、虚拟现实头盔等。终端设备内设置有面向用户眼部的图像采集单元(例如摄像头)，在终端设备接收到用户指令，或者检测终端设备被用户佩戴后，到通过图像采集单元对用户眼部进行图像采集，从而获得眼部图像。更具体地，即图像中包含有眼部区域的图像。本实施例中的眼部图像，可以是在同一帧图像中包含有用户双目对应的眼部区域的图像，也可以是仅包含有用户单目对应的眼部区域的图像。即眼部图像可以是包含有左目区域和右目区域的双目图像，也可以是两帧单目图像(即左目图像和左目图像)。更进一步地，示例性地，终端设备内可以设置多个图像采集单元，从而在同一时刻从不同角度对用户眼部进行采集，从而获得同一时刻下同步采集的眼部图像，具体实现方式可以根据需要设置，此处不再赘述。
示例性地，在采集到眼部图像后，对眼部图像进行特征识别，即可得到至少两个位于眼部图像中眼部区域的轮廓上的眼部特征点。在一种可能的实现方式中，如图3所示，步骤S101的具体实现方式包括：
步骤S1021：对眼部图像进行特征提取，得到第一特征图，第一特征图表征眼部图像的像素特征；
步骤S1022：基于预训练的预测模型，处理第一特征图，确定眼部图像中眼部区域的轮廓线；
步骤S1023：基于眼部区域的轮廓线，得到至少两个眼部特征点。
图4为本公开实施例提供的一种眼部特征点的示意图。如图4所示，以眼部图像为单目图像为例，在得到眼部图像后，对眼部图像进行特征提取，得到第一特征图，第一特征图表征眼部图像中像素点的排列特征，也即像素特征。之后，以第一特征图为输入，利用预训练的预测模型进行处理，可得到眼部图像中眼部区域的轮廓对应的像素点，也即眼部区域的轮廓线，该轮廓线由多个轮廓节点连接而成。进一步地，根据上述眼部区域的轮廓线的形状，可以确定至少两个眼部特征点，例如图像所示的眼部特征点P1和眼部特征点P2(图中示为P1和P2)。上述两个眼部特征点分别位于眼部区域的左端点和右端点，分别对应人眼的左眼角和右眼角。其中，示例性地，上述眼部特征点P1和眼部特征点P2的确定方法，可以通过获取眼部区域对应的轮廓线的各轮廓节点的坐标，并根据坐标将其中最左侧的轮廓节点确定为眼部特征点P1、将将其中最右侧的轮廓节点确定为眼部特征点P2。
进一步地，在一种可能的实现方式中，在确定眼部特征点的过程中，可以基于轮廓线的外接矩形，确定横向和纵向(或其他两个垂直方向)的各至少两个极限位置点，将至少四个极限位置点作为眼部特征点。其中，上述至少四个极限位置点的坐标，可以根据轮廓线的形状来确定。在一种可能的实现方式中，如图5所示，步骤S1023的具体实现方式包括：
步骤S1023A：根据轮廓线中各轮廓节点的切线方向，获得眼部区域的轮廓线的至少四个位置极值点。
步骤S1023B：根据至少四个位置极值点，得到至少四个眼部特征点。
示例性地，针对之前步骤中得到的轮廓线，可以基于轮廓线中各轮廓节点的切线方向，根据切线方向的变化，来确定眼部区域的轮廓线的至少四个 位置极值点，进而得到对应的至少四个眼部特征点。具体地，在通过预测模型的处理后，得到由轮廓线节点构成的轮廓线，该轮廓线的实现方式可以为：由多个轮廓线节点的坐标构成的数组。之后，基于轮廓线节点的坐标，计算各轮廓线节点与相邻的轮廓线节点的方向向量，从而得到各轮廓节点的切线方向(即方向向量的向量节点)，具体实现方式不再赘述。之后，基于各轮廓节点的切线方向的，将切线方向为特定值的轮廓节点(例如为0)，确定为位置极值点。
示例性地，图6为本公开实施例提供的一种确定位置极值点的过程示意图。如图6所示，在获得轮廓线的数据后，首先根据轮廓节点的坐标，确定最左端的位置极值点P1和最右端的位置极值点P2，具体实现方式不再赘述。之后，计算位置极值点P1和位置极值点P2的目标方向向量，以及该目标方向向量的方向Seta_1。之后，基于上述步骤，计算各轮廓节点的切线方向，并将与目标方向向量vec_1的方向Seta_1相同的切线方向对应的轮廓节点P3和轮廓节点P4，确定为位置极值点。本实施例中，通过计算轮廓节点的切线，得到所在切线方向与眼部区域平行的轮廓节点作为位置极值点，使位置极值点在眼部轮廓上的分布更加均匀、合理，从而提高后续基于眼部特征点计算偏差信息的精确度。
进一步地，在上述基于位置极值点得到眼部特征点的基础上，还可以进一步的选取眼部区域的轮廓线上的轮廓节点，从而增加眼部特征点的数量。例如，在上述基于位置极值点生成的眼部特征点(以下称为基础点)的基础上，在相邻的基础点之间，均匀增加若干个眼部特征点，从而提高计算后续偏差信息的精确度。其中，任意两个相邻的眼部特征点之间具有第一限制距离，第一限制距离基于眼部图像的分辨率确定。即当眼部图像的分辨率越高，则相对的，在基础点之间插入的眼部特征点的数量越多。反之，当眼部图像的分辨率越低，则在基础点之间插入的眼部特征点的数量越少。从而充分利用眼部图像中的内容信息，减少眼部轮廓的位置误差，提高眼部区域的精确度，进而提高后续基于眼部区域的轮廓线计算偏差信息的精度。
步骤S103：根据至少两个眼部特征点的位置特征，得到偏差信息，偏差信息表征终端设备的当前佩戴位姿相对标准佩戴位姿的偏差类型和/或偏差量。
示例性地，在基于之前的步骤，得到眼部特征点的位置后，基于眼部特征点之间的位置关系构成的位置特征，来确定终端设备对应的偏差信息，即终端设备的当前佩戴位姿相对标准佩戴位姿的偏差类型和/或偏差量。具体地，偏差类型包括：佩戴偏左、佩戴偏右、佩戴偏左上、佩戴倾斜(左高右低、右高左低)等。偏差量包括：佩戴偏左10毫米、佩戴偏右上5毫米，佩戴倾斜10度等。在一种可能的实现方式中，位置特征包括眼部特征点在眼部图像中的位置坐标，如图7所示，步骤S103的具体实现方式包括：
步骤S1031：根据至少两个眼部特征点的位置坐标，得到眼部中心点坐标。
步骤S1032：根据眼部中心点坐标和参考坐标，得到偏差向量，参考坐标为标准佩戴位姿对应的标准眼部中心点坐标。
步骤S1033：根据偏差向量，得到偏差信息。
示例性地，首先，基于之前的步骤，可以得到眼部特征点的位置坐标，例如最左端的位置极值点的坐标、最右端的位置极值点的坐标，即上述实施例中所指的基准点的坐标。之后，基于眼部区域的形状特征，对最左端的位置极值点的坐标和最右端的位置极值点的坐标求平均，即可得到眼部区域的中心点的近似坐标，即眼部中心点坐标。进一步地，在一种可能的实现方式中，上述步骤中所使用的眼部特征点，除了上述基准点外，还可以包括最上端的位置极值点、最下段的位置极值点，以及位于基准点之间的其他眼部特征点，并将上述所有均匀分布的眼部特征点的坐标求平均，得到眼部中心点坐标，从而提高眼部中心点坐标的准确性，具体实现方式可根据需要设置，此处不再赘述。
之后，获取预设的参考坐标，即标准佩戴位姿对应的标准眼部中心点坐标，并对眼部中心点坐标和参考坐标进行计算，得到二者之间的偏差向量，再基于偏差向量获得对应的方向和距离，即偏差信息。其中，该偏差信息可以是键值对形式的[方向，距离]的数值，也可以是其他形式的数据，此处不进行具体限制。本实施例中，通过预先标定的参考坐标对眼部中心坐标进行校准，从而得到表征佩戴位姿偏差类型和/或偏差量的偏差信息，实现了在不使用虹膜反射的光斑的情况下，对佩戴位置的检查和校准，提高了后续进行注视点检测的准确性和稳定性。
进一步地，在一种可能的实现方式中，位置坐标包括纵向坐标，如图8所示，步骤S1031的具体实现方式包括：
步骤S1031A：获取拍摄角度信息，拍摄角度信息表征图像采集单元拍摄眼部图像时的拍摄角度。
步骤S1031B：根据拍摄角度信息，确定纵向加权系数。
步骤S1031C：根据纵向加权系数对纵向坐标进行加权平均，得到纵向中心点坐标。
步骤S1031D：根据纵向中心点坐标，得到眼部中心点坐标。
示例性地，拍摄角度信息是图像采集单元，例如终端设备内置的摄像头，对眼部图像进行拍摄时的拍摄角度，对于不同的终端设备，其对应的内置的图像采集单元的位置可能不同，因此所拍摄的眼部图像会产生对应的俯仰角度。示例性地，该拍摄角度信息与终端设备的型号一一对应，拍摄角度信息是预设在终端设备内，或者终端设备通过服务器获得的固定值，拍摄角度信息可以通过角度值或向量值表示，具体不做限制。
进一步地，在得到拍摄角度信息后，通过拍摄角度信息，对视觉特征点的纵坐标进行加权，从而获得更加准确、更加贴近真实情况的纵向中心点坐标。图9为本公开实施例提供的一种纵向中心点坐标的示意图，如图9所示，根据图像采集单元D1(图中示为D1)在终端设备其内部的位置，其所对应的拍摄角度信息为Info_1，对应拍摄角度为Theta_1。在图像采集单元D1在计算眼部中心点坐标时，在不考虑拍摄角度信息的情况下，对各眼部特征点的坐标进行平均，得到的眼部中心点坐标为P1。而在基于拍摄角度信息Info_1，得到对应的加权系数，并对各眼部特征点的纵坐标进行加权处理后，基于眼部特征点的坐标进行平均，得到的眼部中心点坐标为P2。通过拍摄角度信息Info_1加权处理后的眼部中心点坐标P2，更加接近真实的眼部中心点位置。本实施例中，通过拍摄角度信息，得到纵向加权系数，并对纵向坐标进行加权平均，使生成的纵向中心点坐标更加准确，即眼部中心点坐标更加准确，从而提高后续基于眼部中心点坐标得到的偏差信息的准确性。
在本实施例中，通过采集眼部图像；根据眼部图像，得到至少两个眼部特征点，眼部特征点分布于眼部图像中眼部区域的轮廓上；根据至少两个眼部特征点的位置特征，得到偏差信息，偏差信息表征终端设备的当前佩戴位 姿相对标准佩戴位姿的偏差类型和/或偏差量。通过采集眼部图像，并基于眼部图像中眼部区域的轮廓上的眼部特征点，得到终端设备的当前佩戴位姿相对标准佩戴位姿的偏差类型和/或偏差量，实现佩戴位姿异常时的提前检测和提示，避免由于终端设备的佩戴位姿异常而导致的注视点检测的准确率降低的问题，提高图像显示效果和人机交互效率。
参考图10，图10为本公开实施例提供的佩戴偏差检测方法的流程示意图二。本实施例在图2所示实施例的基础上，进一步对步骤S103进行细化，该佩戴偏差检测方法包括：
步骤S201：采集眼部图像，眼部图像包括左目图像和左目图像。
步骤S202：根据眼部图像，得到至少两个眼部特征点，眼部特征点包括左目图像对应的第一特征点和左目图像对应的第二特征点。
示例性地，本实施例中，通过图像采集单元分别采集左目对应的左目图像，以及右目对应的右目图像，之后，分别对左目图像和右目图像进行处理，可以得到左目图像对应的第一特征点和右目图像对应的第二特征点。其中，第一特征点位于左目图像中的眼部区域的轮廓上、第二特征点位于右目图像中的眼部区域的轮廓上；左目图像对应的第一特征点和右目图像对应的第二特征点的具体获得方式，可参考图2所示实施例中针对单目图像获取眼部特征点的具体实现方式的介绍，此处不再赘述。
示例性地，左目图像和右目图像可以是通过不同的图像采集单元进行采集的，在获得左目图像和右目图像后，可以先对左目图像和右目图像进行对齐，即将二者转换至同一图像坐标系下，可以通过参考图像采集单元之间的位置关系进行对齐，具体不再赘述。之后，对对齐后的左目图像和右目图像进行处理，获得对应的第一特征点和第二特征点。则获得的上述第一特征点和第二特征点位于同一图像坐标系下，便于后续的距离测算。在另一种可能的实现方式中，左目图像和左目图像可以是由同一个图像采集单元拍摄的，并通过对采集的眼部图像进行(逻辑)划分，而得到左目图像和右目图像，可以将左目图像和右目图像分别示为眼部图像的一个图像区域或像素集合。此种情况下，左目图像和右目图像始终处于同一个图像坐标系中，后续处理过程中，可以直接对眼部图像进行处理即可。
步骤S203：根据第一特征点的位置特征，得到左目图像中左目区域对应 的第一中心点坐标。
步骤S204：根据第二特征点的位置特征，得到左目图像中右目区域对应的第二中心点坐标。
示例性地，在获得第一特征点和第二特征点后，基于二者各自的位置特征，可以得到左目图像中左目区域的第一中心点坐标，以及右目图像中右目区域对应的第二中心点坐标。具体地，以左目图像为例，可以通过计算左目图像中各第一特征点的坐标的加权平均值，得到第一中心点坐标，具体实现方式可参考图2所示实施例中关于眼部中心点坐标的计算方法，此处不再赘述。
步骤S205：根据第一中心点坐标和第二中心点坐标的位置关系，得到偏差信息。
示例性地，在得到第一中心点坐标和第二中心点坐标之后，基于二者之间的位置关系，得到偏差信息，即终端设备的当前佩戴位姿相对标准佩戴位姿的偏差类型和/或偏差量。具体地，例如，第一中心点坐标和第二中心点坐标构成的连线的水平度，来得到表征终端设备的佩戴是否水平的偏差信息。
在一种可能的实现方式中，如图11所示，步骤S205的具体实现方式包括：
步骤S2051：基于第一中心点坐标和第二中心点坐标计算加权和，得到加权中心点坐标；
步骤S2052：根据加权中心点坐标与参考坐标的位置关系，得到第一偏差信息，第一偏差信息表征终端设备的当前佩戴位姿相对标准佩戴位姿在水平方向或竖直方向上的偏差距离。
示例性地，首先，通过第一中心点坐标和第二中心点坐标，计算二者的中心点坐标，即通过计算第一中心点坐标和第二中心点坐标的平均值，得到中心点坐标。在此基础上，基于图像采集单元的拍摄角度信息，获得第一中心点坐标和第二中心点坐标各自对应的横向加权系数和纵向加权系数，再基于横向加权系数和纵向加权系数，分别对第一中心点坐标和第二中心点坐标进行加权，计算加权平均值，得到加权中心点坐标。其中，横向加权系数和纵向加权系数的具体获取方法，可参考图2所示实施例中获取纵向加权系数的相关介绍，此处不再赘述。
进一步地，在得到加权中心点坐标后，基于加权中心点坐标与参考坐标的位置关系，得到第一偏差信息。第一偏差信息表征终端设备的当前佩戴位姿相对标准佩戴位姿在水平方向和/或竖直方向上的偏差距离，即第一偏差信息表征终端设备当前的佩戴位置是否水平、竖直方向的偏移，以及具体的偏移距离。具体地，例如第一偏差信息Info_1＝[10,-20]，其表示终端设备当前的佩戴位姿相对标准位姿，在水平方向上，偏左10毫米；在竖直方向上，偏下20毫米。
本实施例中，通过第一中心点坐标和第二中心点，得到对应的加权中心点坐标，并基于加权中心点坐标得到表征终端设备在水平方向或竖直方向的佩戴位姿偏差的第一偏差信息，相比使用单目图像进行判断的方案，准确性、稳定性更好，进一步提高注视点检测的准确性。
在另一种可能的实现方式中，如图12所示，步骤S205的具体实现方式包括：
步骤S2053：根据第一中心点坐标和第二中心点坐标，生成倾斜度向量。
步骤S2054：根据倾斜度向量的方向，得到第二偏差信息，第二偏差信息表征终端设备的当前佩戴位姿相对标准佩戴位姿的倾斜角度。
示例性地，在另一种可能的实现方式中，在获得第一中心点坐标和第二中心点坐标之后，基于第一中心点坐标和第二中心点坐标，生成倾斜度向量，之后，基于倾斜度向量的方向，得到第二偏差信息，其中，第二偏差信息表征终端设备的当前佩戴位姿相对标准佩戴位姿的倾斜角度。图13为本公开实施例提供的一种第二偏差信息的示意图。如图13所示，示例性地，通过上述步骤，得到第一中心点坐标P1和第二中心点坐标P2后，连接第一中心点坐标P1和第二中心点坐标P2，得到倾斜度向量Vp(图中示为Vp)。之后，获取倾斜度向量Vp在图像坐标系中的倾斜角度，即相对水平方向(0度)的夹角Theta，生成第二偏差信息。其中，示例性地，当Theta大于零时，表征终端设备的佩戴位姿相对标准佩戴位姿(终端设备水平佩戴)左倾，即左低右高；而当Theta小于零时，表征终端设备的佩戴位姿相对标准佩戴位姿右倾，即左高右低。
本实施例中，通过第一中心点坐标和第二中心点，生成倾斜度向量，并基于倾斜度向量生成表征终端设备的佩戴倾斜角度的第二偏差信息，从而实 现对终端设备的倾斜位姿的检测。本实施例步骤所提供的偏差信息(第二偏差信息)的获取步骤，可以与图11所示实施例所提供的偏差信息(第一偏差信息)的获取步骤相结合，即终端设备同步或异步获得第一偏差信息和第二偏差信息(不限制顺序)，之后将第一偏差信息和第二偏差信息的集合，作为偏差信息进行后续的步骤，从而实现水平、竖直、倾斜多个维度的位姿检测，进一步提高注视点检测的准确性。
可选地，在步骤S205之后，还包括：
步骤S206：若偏差信息大于或等于偏差阈值，则在终端设备内播放基于偏差信息生成的第一指示信息，并返回步骤S201，其中，第一指示信息用于表征调整终端设备的动作；
步骤S207：若偏差信息小于偏差阈值，则在终端设备内播放第二指示信息，第二指示信息表征终端设备当前的佩戴位姿为标准佩戴位姿。
示例性地，在得到偏差信息后，若根据偏差信息所表征的终端设备当前的佩戴位姿与标准佩戴位置的偏差量大于偏差阈值，则基于偏差信息生成第一提示信息，并通过终端设备的人机交互单元，向用户播放该第一指示信息，以对用户进行提示。例如，偏差信息为Info_1，所表征的内容为：佩戴位姿偏左(偏差类型)，则生成对应的第一提示信息为“佩戴位置偏左”的图标，或者，语音或文字“请将终端设备向右移动”。再例如，偏差信息为Info_2，所表征的内容为：佩戴位姿左侧下倾10度(偏差类型和偏差量)，则生成对应的第一提示信息为“佩戴位置左侧下倾10度”的图标，或者，语音或文字“请将终端设备左侧向上旋转10度”。
进一步地，在显示完第一指示信息后，返回步骤S201，重新采集实时的眼部图像，并重复上述步骤，实现对佩戴位姿的持续检测和提示，直至偏差信息小于偏差阈值后，显示第二指示信息，以告知用户终端设备当前的佩戴位姿为标准位姿，可以执行、启动后续的注视点检测的功能。其中，第一指示信息、第二指示信息包括以下至少一种：提示文字、标识、语音消息，可根据需要设置，此处不做具体限定。
本实施例中，通过持续性的显示由偏差信息生成的指示信息，来实现用户手动调节终端设备的佩戴位姿的自动引导，从而使用户可以更快、更准确的将终端设备调整到标准佩戴位姿，提高佩戴位姿调整的效率和准确性，并 进一步提高后续注视点检测的准确性。
对应于上文实施例的佩戴偏差检测方法，图14为本公开实施例提供的佩戴偏差检测装置的结构框图。为了便于说明，仅示出了与本公开实施例相关的部分。参照图14，佩戴偏差检测装置3包括：
采集模块31，用于采集眼部图像；
处理模块32，用于根据眼部图像，得到至少两个眼部特征点，眼部特征点分布于眼部图像中眼部区域的轮廓上；
检测模块33，用于根据至少两个眼部特征点的位置特征，得到偏差信息，偏差信息表征终端设备的当前佩戴位姿相对标准佩戴位姿的偏差方向和/或偏差距离。
在本公开的一个实施例中，处理模块32，具体用于：对眼部图像进行特征提取，得到第一特征图，第一特征图表征眼部图像的像素特征。基于预训练的预测模型，处理第一特征图，确定眼部图像中眼部区域的轮廓线。基于眼部区域的轮廓线，得到至少两个眼部特征点。
在本公开的一个实施例中，处理模块32在基于眼部区域的轮廓线，得到至少两个眼部特征点时，具体用于：根据轮廓线中各轮廓节点的切线方向，获得眼部区域的轮廓线的至少四个位置极值点。根据至少四个位置极值点，得到至少四个眼部特征点。
在本公开的一个实施例中，任意两个相邻的眼部特征点之间具有第一限制距离，第一限制距离基于眼部图像的分辨率确定。
在本公开的一个实施例中，位置特征包括眼部特征点在眼部图像中的位置坐标。检测模块33，具体用于：根据至少两个眼部特征点的位置坐标，得到眼部中心点坐标。根据眼部中心点坐标和参考坐标，得到偏差向量，参考坐标为标准佩戴位姿对应的标准眼部中心点坐标。根据偏差向量，得到偏差信息。
在本公开的一个实施例中，位置坐标包括纵向坐标；检测模块33在根据至少两个眼部特征点的位置坐标，得到眼部中心点坐标时。具体用于：获取拍摄角度信息，拍摄角度信息表征图像采集单元拍摄眼部图像时的拍摄角度。根据拍摄角度信息，确定纵向加权系数。根据纵向加权系数对纵向坐标进行加权平均，得到纵向中心点坐标。根据纵向中心点坐标，得到眼部中心点坐 标。
在本公开的一个实施例中，眼部图像包括左目图像和左目图像，眼部特征点包括左目图像对应的第一特征点和左目图像对应的第二特征点；检测模块33。具体用于：根据第一特征点的位置特征，得到左目图像中左目区域对应的第一中心点坐标。根据第二特征点的位置特征，得到左目图像中右目区域对应的第二中心点坐标。根据第一中心点坐标和第二中心点坐标的位置关系，得到偏差信息。
在本公开的一个实施例中，检测模块33在根据第一中心点坐标和第二中心点坐标的位置关系，得到偏差信息时。具体用于：基于第一中心点坐标和第二中心点坐标计算加权和，得到加权中心点坐标。根据加权中心点坐标与参考坐标的位置关系，得到第一偏差信息，第一偏差信息表征终端设备的当前佩戴位姿相对标准佩戴位姿在水平方向或竖直方向上的偏差距离。
在本公开的一个实施例中，检测模块33在根据第一中心点坐标和第二中心点坐标的位置关系，得到偏差信息时。具体用于：根据第一中心点坐标和第二中心点坐标，生成倾斜度向量。根据倾斜度向量的方向，得到第二偏差信息，第二偏差信息表征终端设备的当前佩戴位姿相对标准佩戴位姿的倾斜角度。
在本公开的一个实施例中，在根据至少两个眼部特征点的位置特征，得到偏差信息之后，检测模块33，还用于：在终端设备内播放基于偏差信息生成的指示信息，直至偏差信息小于偏差阈值，指示信息用于表征调整终端设备的动作，其中，指示信息包括以下至少一项：提示文字、标识、语音消息。
其中，采集模块31、处理模块32和检测模块33依次连接。本实施例提供的佩戴偏差检测装置3可以执行上述方法实施例的技术方案，其实现原理和技术效果类似，本实施例此处不再赘述。
图15为本公开实施例提供的一种电子设备的结构示意图，如图15所示，该电子设备4包括：
处理器41，以及与处理器41通信连接的存储器42；
存储器42存储计算机执行指令；
处理器41执行存储器42存储的计算机执行指令，以实现如图2-图13所示实施例中的佩戴偏差检测方法。
其中，可选地，处理器41和存储器42通过总线43连接。
相关说明可以对应参见图2-图13所对应的实施例中的步骤所对应的相关描述和效果进行理解，此处不做过多赘述。
本公开实施例提供一种计算机可读存储介质，计算机可读存储介质中存储有计算机执行指令，计算机执行指令被处理器执行时用于实现本公开图2-图13所对应的实施例中任一实施例提供的佩戴偏差检测方法。
本公开实施例提供一种计算机程序产品，包括计算机程序，该计算机程序被处理器执行时实现如图2-图13所示实施例中的佩戴偏差检测方法，
参考图16，其示出了适于用来实现本公开实施例的电子设备900的结构示意图，该电子设备900可以为终端设备或服务器。其中，终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、个人数字助理(Personal Digital Assistant，简称PDA)、平板电脑(Portable Android Device，简称PAD)、便携式多媒体播放器(Portable Media Player，简称PMP)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图16示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图16所示，电子设备900可以包括处理装置(例如中央处理器、图形处理器等)901，其可以根据存储在只读存储器(Read Only Memory，简称ROM)902中的程序或者从存储装置908加载到随机访问存储器(Random Access Memory，简称RAM)903中的程序而执行各种适当的动作和处理。在RAM 903中，还存储有电子设备900操作所需的各种程序和数据。处理装置901、ROM 902以及RAM 903通过总线904彼此相连。输入/输出(I/O)接口905也连接至总线904。
通常，以下装置可以连接至I/O接口905：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置906；包括例如液晶显示器(Liquid Crystal Display，简称LCD)、扬声器、振动器等的输出装置907；包括例如磁带、硬盘等的存储装置908；以及通信装置909。通信装置909可以允许电子设备900与其他设备进行无线或有线通信以交换数据。虽然图16示出了具有各种装置的电子设备900，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装 置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置909从网络上被下载和安装，或者从存储装置908被安装，或者从ROM 902被安装。在该计算机程序被处理装置901执行时，执行本公开实施例的方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备执行上述实施例所示的方法。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言-诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言一诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(Local Area Network，简称LAN)或广域网(Wide Area Network，简称WAN)-连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对该单元本身的限定，例如，第一获取单元还可以被描述为“获取至少两个网际协议地址的单元”。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结 合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
第一方面，根据本公开的一个或多个实施例，提供了一种佩戴偏差检测方法，包括：
采集眼部图像；根据所述眼部图像，得到至少两个眼部特征点，所述眼部特征点分布于所述眼部图像中眼部区域的轮廓上；根据至少两个所述眼部特征点的位置特征，得到偏差信息，所述偏差信息表征终端设备的当前佩戴位姿相对标准佩戴位姿的偏差类型和/或偏差量。
根据本公开的一个或多个实施例，所述根据所述眼部图像，得到至少两个眼部特征点，包括：对所述眼部图像进行特征提取，得到第一特征图，所述第一特征图表征所述眼部图像的像素特征；基于预训练的预测模型，处理所述第一特征图，确定所述眼部图像中眼部区域的轮廓线；基于所述眼部区域的轮廓线，得到至少两个所述眼部特征点。
根据本公开的一个或多个实施例，基于所述眼部区域的轮廓线，得到至少两个所述眼部特征点，包括：根据所述轮廓线中各轮廓节点的切线方向，获得所述眼部区域的轮廓线的至少四个位置极值点；根据所述至少四个位置极值点，得到至少四个所述眼部特征点。
根据本公开的一个或多个实施例，任意两个相邻的眼部特征点之间具有第一限制距离，所述第一限制距离基于所述眼部图像的分辨率确定。
根据本公开的一个或多个实施例，所述位置特征包括所述眼部特征点在眼部图像中的位置坐标；所述根据至少两个所述眼部特征点的位置特征，得到偏差信息，包括：根据至少两个所述眼部特征点的位置坐标，得到眼部中心点坐标；根据所述眼部中心点坐标和参考坐标，得到偏差向量，所述参考坐标为所述标准佩戴位姿对应的标准眼部中心点坐标；根据所述偏差向量，得到所述偏差信息。
根据本公开的一个或多个实施例，所述位置坐标包括纵向坐标；根据所述至少两个眼部特征点的位置坐标，得到眼部中心点坐标，包括：获取拍摄角度信息，所述拍摄角度信息表征图像采集单元拍摄所述眼部图像时的拍摄角度；根据所述拍摄角度信息，确定纵向加权系数；根据所述纵向加权系数对所述纵向坐标进行加权平均，得到纵向中心点坐标；根据所述纵向中心点坐标，得到所述眼部中心点坐标。
根据本公开的一个或多个实施例，所述眼部图像包括左目图像和左目图像，所述眼部特征点包括所述左目图像对应的第一特征点和所述左目图像对应的第二特征点；所述根据至少两个所述眼部特征点的位置特征，得到偏差信息，包括：根据所述第一特征点的位置特征，得到所述左目图像中左目区域对应的第一中心点坐标；根据所述第二特征点的位置特征，得到所述左目图像中右目区域对应的第二中心点坐标；根据所述第一中心点坐标和所述第二中心点坐标的位置关系，得到所述偏差信息。
根据本公开的一个或多个实施例，所述根据所述第一中心点坐标和所述第二中心点坐标的位置关系，得到所述偏差信息，包括：基于所述第一中心点坐标和所述第二中心点坐标计算加权和，得到加权中心点坐标；根据所述加权中心点坐标与参考坐标的位置关系，得到第一偏差信息，所述第一偏差信息表征所述终端设备的当前佩戴位姿相对标准佩戴位姿在水平方向或竖直方向上的偏差距离。
根据本公开的一个或多个实施例，所述根据所述第一中心点坐标和所述第二中心点坐标的位置关系，得到所述偏差信息，包括：根据所述第一中心点坐标和所述第二中心点坐标，生成倾斜度向量；根据所述倾斜度向量的方向，得到第二偏差信息，所述第二偏差信息表征所述终端设备的当前佩戴位姿相对标准佩戴位姿的倾斜角度。
根据本公开的一个或多个实施例，在根据至少两个所述眼部特征点的位置特征，得到偏差信息之后，还包括：在所述终端设备内播放基于所述偏差信息生成的指示信息，直至所述偏差信息小于偏差阈值，所述指示信息用于表征调整所述终端设备的动作，其中，所述指示信息包括以下至少一项：提示文字、标识、语音消息。
第二方面，根据本公开的一个或多个实施例，提供了一种佩戴偏差检测 装置，包括：
采集模块，用于采集眼部图像；
处理模块，用于根据所述眼部图像，得到至少两个眼部特征点，所述眼部特征点分布于所述眼部图像中眼部区域的轮廓上；
检测模块，用于根据至少两个所述眼部特征点的位置特征，得到偏差信息，所述偏差信息表征终端设备的当前佩戴位姿相对标准佩戴位姿的偏差方向和/或偏差距离。
根据本公开的一个或多个实施例，所述处理模块，具体用于：对所述眼部图像进行特征提取，得到第一特征图，所述第一特征图表征所述眼部图像的像素特征；基于预训练的预测模型，处理所述第一特征图，确定所述眼部图像中眼部区域的轮廓线；基于所述眼部区域的轮廓线，得到至少两个所述眼部特征点。
根据本公开的一个或多个实施例，所述处理模块在基于所述眼部区域的轮廓线，得到至少两个所述眼部特征点时，具体用于：根据所述轮廓线中各轮廓节点的切线方向，获得所述眼部区域的轮廓线的至少四个位置极值点；根据所述至少四个位置极值点，得到至少四个所述眼部特征点。
根据本公开的一个或多个实施例，任意两个相邻的眼部特征点之间具有第一限制距离，所述第一限制距离基于所述眼部图像的分辨率确定。
根据本公开的一个或多个实施例，所述位置特征包括所述眼部特征点在眼部图像中的位置坐标；所述检测模块，具体用于：根据至少两个所述眼部特征点的位置坐标，得到眼部中心点坐标；根据所述眼部中心点坐标和参考坐标，得到偏差向量，所述参考坐标为所述标准佩戴位姿对应的标准眼部中心点坐标；根据所述偏差向量，得到所述偏差信息。
根据本公开的一个或多个实施例，所述位置坐标包括纵向坐标；所述检测模块33在根据所述至少两个眼部特征点的位置坐标，得到眼部中心点坐标时，具体用于：获取拍摄角度信息，所述拍摄角度信息表征图像采集单元拍摄所述眼部图像时的拍摄角度；根据所述拍摄角度信息，确定纵向加权系数；根据所述纵向加权系数对所述纵向坐标进行加权平均，得到纵向中心点坐标；根据所述纵向中心点坐标，得到所述眼部中心点坐标。
根据本公开的一个或多个实施例，所述眼部图像包括左目图像和左目图 像，所述眼部特征点包括所述左目图像对应的第一特征点和所述左目图像对应的第二特征点；所述检测模块，具体用于：根据所述第一特征点的位置特征，得到所述左目图像中左目区域对应的第一中心点坐标；根据所述第二特征点的位置特征，得到所述左目图像中右目区域对应的第二中心点坐标；根据所述第一中心点坐标和所述第二中心点坐标的位置关系，得到所述偏差信息。
根据本公开的一个或多个实施例，所述检测模块在根据所述第一中心点坐标和所述第二中心点坐标的位置关系，得到所述偏差信息时，具体用于：基于所述第一中心点坐标和所述第二中心点坐标计算加权和，得到加权中心点坐标；根据所述加权中心点坐标与参考坐标的位置关系，得到第一偏差信息，所述第一偏差信息表征所述终端设备的当前佩戴位姿相对标准佩戴位姿在水平方向或竖直方向上的偏差距离。
根据本公开的一个或多个实施例，所述检测模块在根据所述第一中心点坐标和所述第二中心点坐标的位置关系，得到所述偏差信息时，具体用于：根据所述第一中心点坐标和所述第二中心点坐标，生成倾斜度向量；根据所述倾斜度向量的方向，得到第二偏差信息，所述第二偏差信息表征所述终端设备的当前佩戴位姿相对标准佩戴位姿的倾斜角度。
根据本公开的一个或多个实施例，在根据至少两个所述眼部特征点的位置特征，得到偏差信息之后，所述检测模块，还用于：在所述终端设备内播放基于所述偏差信息生成的指示信息，直至所述偏差信息小于偏差阈值，所述指示信息用于表征调整所述终端设备的动作，其中，所述指示信息包括以下至少一项：提示文字、标识、语音消息。
第三方面，根据本公开的一个或多个实施例，提供了一种电子设备，包括：处理器，以及与所述处理器通信连接的存储器；
所述存储器存储计算机执行指令；
所述处理器执行所述存储器存储的计算机执行指令，以实现如上第一方面以及第一方面各种可能的设计所述的佩戴偏差检测方法。
第四方面，根据本公开的一个或多个实施例，提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如上第一方面以及第一方面各种可能的设计所 述的佩戴偏差检测方法。
第五方面，本公开实施例提供一种计算机程序产品，包括计算机程序，该计算机程序被处理器执行时实现如上第一方面以及第一方面各种可能的设计所述的佩戴偏差检测方法。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。

Claims (14)

  1. 一种佩戴偏差检测方法，包括：
    采集眼部图像；
    根据所述眼部图像，得到至少两个眼部特征点，所述眼部特征点分布于所述眼部图像中眼部区域的轮廓上；
    根据至少两个所述眼部特征点的位置特征，得到偏差信息，所述偏差信息表征终端设备的当前佩戴位姿相对标准佩戴位姿的偏差类型和/或偏差量。
  2. 根据权利要求1所述的方法，其中所述根据所述眼部图像，得到至少两个眼部特征点，包括：
    对所述眼部图像进行特征提取，得到第一特征图，所述第一特征图表征所述眼部图像的像素特征；
    基于预训练的预测模型，处理所述第一特征图，确定所述眼部图像中眼部区域的轮廓线；
    基于所述眼部区域的轮廓线，得到至少两个所述眼部特征点。
  3. 根据权利要求2所述的方法，其中基于所述眼部区域的轮廓线，得到至少两个所述眼部特征点，包括：
    根据所述轮廓线中各轮廓节点的切线方向，获得所述眼部区域的轮廓线的至少四个位置极值点；
    根据所述至少四个位置极值点，得到至少四个所述眼部特征点。
  4. 根据权利要求1所述的方法，其中任意两个相邻的眼部特征点之间具有第一限制距离，所述第一限制距离基于所述眼部图像的分辨率确定。
  5. 根据权利要求1所述的方法，其中所述位置特征包括所述眼部特征点在眼部图像中的位置坐标；所述根据至少两个所述眼部特征点的位置特征，得到偏差信息，包括：
    根据至少两个所述眼部特征点的位置坐标，得到眼部中心点坐标；
    根据所述眼部中心点坐标和参考坐标，得到偏差向量，所述参考坐标为所述标准佩戴位姿对应的标准眼部中心点坐标；
    根据所述偏差向量，得到所述偏差信息。
  6. 根据权利要求5所述的方法，其中所述位置坐标包括纵向坐标；根据所述至少两个眼部特征点的位置坐标，得到眼部中心点坐标，包括：
    获取拍摄角度信息，所述拍摄角度信息表征图像采集单元拍摄所述眼部图像时的拍摄角度；
    根据所述拍摄角度信息，确定纵向加权系数；
    根据所述纵向加权系数对所述纵向坐标进行加权平均，得到纵向中心点坐标；
    根据所述纵向中心点坐标，得到所述眼部中心点坐标。
  7. 根据权利要求1所述的方法，其中所述眼部图像包括左目图像和左目图像，所述眼部特征点包括所述左目图像对应的第一特征点和所述左目图像对应的第二特征点；
    所述根据至少两个所述眼部特征点的位置特征，得到偏差信息，包括：
    根据所述第一特征点的位置特征，得到所述左目图像中左目区域对应的第一中心点坐标；
    根据所述第二特征点的位置特征，得到所述左目图像中右目区域对应的第二中心点坐标；
    根据所述第一中心点坐标和所述第二中心点坐标的位置关系，得到所述偏差信息。
  8. 根据权利要求7所述的方法，其中所述根据所述第一中心点坐标和所述第二中心点坐标的位置关系，得到所述偏差信息，包括：
    基于所述第一中心点坐标和所述第二中心点坐标计算加权和，得到加权中心点坐标；
    根据所述加权中心点坐标与参考坐标的位置关系，得到第一偏差信息，所述第一偏差信息表征所述终端设备的当前佩戴位姿相对标准佩戴位姿在水平方向或竖直方向上的偏差距离。
  9. 根据权利要求7所述的方法，其中所述根据所述第一中心点坐标和所述第二中心点坐标的位置关系，得到所述偏差信息，包括：
    根据所述第一中心点坐标和所述第二中心点坐标，生成倾斜度向量；
    根据所述倾斜度向量的方向，得到第二偏差信息，所述第二偏差信息表征所述终端设备的当前佩戴位姿相对标准佩戴位姿的倾斜角度。
  10. 根据权利要求1所述的方法，其中在根据至少两个所述眼部特征点的位置特征，得到偏差信息之后，还包括：
    在所述终端设备内播放基于所述偏差信息生成的指示信息，直至所述偏差信息小于偏差阈值，所述指示信息用于表征调整所述终端设备的动作，其中，所述指示信息包括以下至少一项：
    提示文字、标识、语音消息。
  11. 一种佩戴偏差检测装置，包括：
    采集模块，用于采集眼部图像；
    处理模块，用于根据所述眼部图像，得到至少两个眼部特征点，所述眼部特征点分布于所述眼部图像中眼部区域的轮廓上；
    检测模块，用于根据至少两个所述眼部特征点的位置特征，得到偏差信息，所述偏差信息表征终端设备的当前佩戴位姿相对标准佩戴位姿的偏差方向和/或偏差距离。
  12. 一种电子设备，包括：处理器，以及与所述处理器通信连接的存储器；
    所述存储器存储计算机执行指令；
    所述处理器执行所述存储器存储的计算机执行指令，以实现如权利要求1至10中任一项所述的佩戴偏差检测方法。
  13. 一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如权利要求1至10任一项所述的佩戴偏差检测方法。
  14. 一种计算机程序产品，包括计算机程序，该计算机程序被处理器执行时实现权利要求1至10中任一项所述的佩戴偏差检测方法。
PCT/CN2024/100639 2023-06-26 2024-06-21 佩戴偏差检测方法、装置、电子设备及存储介质 Ceased WO2025001996A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310764919.7A CN116758624A (zh) 2023-06-26 2023-06-26 佩戴偏差检测方法、装置、电子设备及存储介质
CN202310764919.7 2023-06-26

Publications (1)

Publication Number Publication Date
WO2025001996A1 true WO2025001996A1 (zh) 2025-01-02

Family

ID=87953011

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/100639 Ceased WO2025001996A1 (zh) 2023-06-26 2024-06-21 佩戴偏差检测方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN116758624A (zh)
WO (1) WO2025001996A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119557814A (zh) * 2025-01-23 2025-03-04 无锡真源科技有限公司 一种室内定位蓝牙信标异常点的检测方法和装置
CN120831223A (zh) * 2025-09-17 2025-10-24 歌尔股份有限公司 光学参数测量方法、装置、电子设备及系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116758624A (zh) * 2023-06-26 2023-09-15 北京字跳网络技术有限公司 佩戴偏差检测方法、装置、电子设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103380625A (zh) * 2011-06-16 2013-10-30 松下电器产业株式会社 头戴式显示器及其位置偏差调整方法
US20170102767A1 (en) * 2015-10-12 2017-04-13 Samsung Electronics Co., Ltd. Head mounted electronic device
CN110174936A (zh) * 2019-04-04 2019-08-27 阿里巴巴集团控股有限公司 头戴式可视设备的控制方法、装置及设备
CN116758624A (zh) * 2023-06-26 2023-09-15 北京字跳网络技术有限公司 佩戴偏差检测方法、装置、电子设备及存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107665040A (zh) * 2016-07-27 2018-02-06 Fove股份有限公司 视线检测系统、偏移检测方法、偏移检测程序
JP6732617B2 (ja) * 2016-09-21 2020-07-29 株式会社ソニー・インタラクティブエンタテインメント 情報処理装置および画像生成方法
CN113496140A (zh) * 2020-03-18 2021-10-12 北京沃东天骏信息技术有限公司 虹膜定位方法以及美瞳虚拟试戴方法和装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103380625A (zh) * 2011-06-16 2013-10-30 松下电器产业株式会社 头戴式显示器及其位置偏差调整方法
US20170102767A1 (en) * 2015-10-12 2017-04-13 Samsung Electronics Co., Ltd. Head mounted electronic device
CN110174936A (zh) * 2019-04-04 2019-08-27 阿里巴巴集团控股有限公司 头戴式可视设备的控制方法、装置及设备
CN116758624A (zh) * 2023-06-26 2023-09-15 北京字跳网络技术有限公司 佩戴偏差检测方法、装置、电子设备及存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119557814A (zh) * 2025-01-23 2025-03-04 无锡真源科技有限公司 一种室内定位蓝牙信标异常点的检测方法和装置
CN120831223A (zh) * 2025-09-17 2025-10-24 歌尔股份有限公司 光学参数测量方法、装置、电子设备及系统

Also Published As

Publication number Publication date
CN116758624A (zh) 2023-09-15

Similar Documents

Publication Publication Date Title
WO2025001996A1 (zh) 佩戴偏差检测方法、装置、电子设备及存储介质
CN108450058B (zh) 实时自动车载相机校准
WO2024125329A1 (zh) 直播特效渲染方法、装置、设备、可读存储介质及产品
WO2020248900A1 (zh) 全景视频的处理方法、装置及存储介质
CN107786808B (zh) 生成代表与光场数据相关联的散景的数据的方法和装置
US20250292444A1 (en) Video synthesis method, apparatus, device, medium and product
WO2020140976A1 (zh) 图像获取方法、装置、点读设备、电子设备及存储介质
WO2024198855A1 (zh) 场景渲染方法、装置、设备、计算机可读存储介质及产品
CN115002442B (zh) 一种图像展示方法、装置、电子设备及存储介质
CN110740315A (zh) 摄像头矫正方法、装置、电子设备及存储介质
WO2024174862A1 (zh) 一种虚拟对象挂载方法、装置、设备及介质
WO2025247205A1 (zh) 三维人脸模型的摆正处理方法、装置、设备及介质
WO2025092906A1 (zh) 一种纹理映射方法、装置、电子设备及存储介质
WO2025161766A1 (zh) 图像处理方法、装置、设备、计算机可读存储介质及产品
CN116434315A (zh) 视线方向检测方法、装置、电子设备及存储介质
WO2022027191A1 (zh) 平面矫正方法及装置、计算机可读介质和电子设备
WO2025039634A1 (zh) 视图优化处理方法、对象的展示方法和装置
WO2024179384A1 (zh) 手势识别方法、装置、设备、介质及产品
CN117827552A (zh) 增强现实设备的测试方法、系统及电子设备
CN115410242B (zh) 视线估计方法和装置
CN118262068A (zh) 渲染控制方法、设备、存储介质及程序产品
KR102534449B1 (ko) 이미지 처리 방법, 장치, 전자 장치 및 컴퓨터 판독 가능 저장 매체
CN114668365A (zh) 视力检测方法
CN115205131B (zh) 畸变图像校正方法及装置、计算机可读介质和电子设备
US12548261B2 (en) Information processing method and device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24830641

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE