CN114356088A - Viewer tracking method and device, electronic equipment and storage medium - Google Patents
Viewer tracking method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114356088A CN114356088A CN202111659288.XA CN202111659288A CN114356088A CN 114356088 A CN114356088 A CN 114356088A CN 202111659288 A CN202111659288 A CN 202111659288A CN 114356088 A CN114356088 A CN 114356088A
- Authority
- CN
- China
- Prior art keywords
- target user
- user
- naked eye
- face image
- display
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000001815 facial effect Effects 0.000 claims abstract description 22
- 238000004590 computer program Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 11
- 238000012544 monitoring process Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 10
- 230000003287 optical effect Effects 0.000 description 4
- 210000003128 head Anatomy 0.000 description 3
- 238000001356 surgical procedure Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Landscapes
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The embodiment of the invention discloses a viewer tracking method, a viewer tracking device, electronic equipment and a storage medium. The method is applied to naked eye 3D display equipment and comprises the following steps: acquiring facial images of candidate users in a viewing range of naked eye 3D display equipment through an image collector; determining the similarity between the face image of each candidate user and a first preset face image, and determining a first target user according to the comparison result of the similarity and a first preset threshold; and presenting a naked eye 3D display picture to the first target user according to the target relative position of the first target user relative to the screen. By executing the scheme, an ideal naked eye 3D display effect can be presented to the main user in a scene where multiple users participate in watching, and the 3D experience of the user can be improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of naked eye 3D human eye tracking, in particular to a viewer tracking method, a viewer tracking device, electronic equipment and a storage medium.
Background
The naked eye 3D display equipment utilizes the principle that the angles of two eyes of a person for observing an object are slightly different, so that the distance of the object can be distinguished, stereoscopic vision is generated, images seen by the left eye and the right eye are separated, and a user does not need to experience stereoscopic feeling by means of stereoscopic glasses (namely naked eyes). The 3D picture can be displayed through naked eye 3D display equipment.
The viewer tracking method in the related art tracks only one user and makes it view desired 3D display contents. When a plurality of people are in front of the screen, the display is switched to multi-view display, but the presented 3D effect is not ideal; or the first tracked person is kept tracked, so that other persons appearing behind in front of the screen cannot see the ideal 3D display content, and the eye tracking is not switched to another viewer until the first tracked user leaves the tracking picture or the face thereof is temporarily twisted to an angle at which no face is detected. It is not suitable for scenes where multiple people are involved in viewing and contain key characters.
Disclosure of Invention
The embodiment of the invention provides a viewer tracking method, a viewer tracking device, electronic equipment and a storage medium, which can realize the presentation of an ideal naked eye 3D display effect to a main user in a scene in which multiple users participate in viewing, and can improve the 3D experience of the user.
In a first aspect, an embodiment of the present invention provides a viewer tracking method, which is applied to a naked-eye 3D display device, and the method includes:
acquiring facial images of candidate users in a viewing range of the naked eye 3D display equipment through an image collector;
determining the similarity between the face image of each candidate user and a first preset face image, and determining a first target user according to the comparison result of the similarity and a first preset threshold;
and presenting a naked eye 3D display picture to the first target user according to the target relative position of the first target user relative to the screen.
In a second aspect, an embodiment of the present invention further provides an observer tracking apparatus configured on a naked-eye 3D display device, where the apparatus includes:
the face image acquisition module is used for acquiring face images of candidate users in the viewing range of the naked eye 3D display equipment through an image collector;
the first target user determination module is used for determining the similarity between the face image of each candidate user and a first preset face image and determining a first target user according to the comparison result of the similarity and a first preset threshold;
and presenting a naked eye 3D display picture to the first target user according to the target relative position of the first target user relative to the screen.
In a third aspect, an embodiment of the present invention further provides an electronic device, where the electronic device includes:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement a viewer tracking method as in any one of the embodiments of the invention.
In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the viewer tracking method according to any one of the embodiments of the present invention.
According to the technical scheme provided by the embodiment of the invention, the face images of all candidate users in the viewing range of the naked eye 3D display equipment are acquired through the image collector; determining the similarity between the face image of each candidate user and a first preset face image, and determining a first target user according to the comparison result of the similarity and a first preset threshold; and presenting a naked eye 3D display picture to the first target user according to the target relative position of the first target user relative to the screen. By executing the technical scheme provided by the embodiment of the invention, an ideal naked eye 3D display effect can be presented to a main user in a scene where multiple users participate in watching, and the 3D experience of the user can be improved.
Drawings
FIG. 1 is a flow chart of a viewer tracking method provided by an embodiment of the present invention;
FIG. 2 is a diagram illustrating a scenario in which a specific user is a primary doctor according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an eye tracking monitoring picture including three candidate users in a viewing range of a naked eye 3D display device according to an embodiment of the present invention;
FIG. 4 is a flow chart of another viewer tracking method provided by an embodiment of the present invention;
FIG. 5 is a diagram illustrating a scenario in which a particular user is a primary doctor and other doctors according to an embodiment of the present invention;
FIG. 6 is a schematic view of a scenario in which a first target user is not detected and only one candidate user is detected within a viewing range according to an embodiment of the present invention;
FIG. 7 is a flow chart of yet another viewer tracking method provided by an embodiment of the present invention;
fig. 8 is a schematic diagram of a human eye tracking monitoring picture in which a first target user cannot be detected in a viewing range of a naked eye 3D display device according to an embodiment of the present invention;
fig. 9 is a schematic diagram of a human eye tracking monitoring picture for re-detecting a first target user in a viewing range of a naked eye 3D display device according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of a viewer tracking device according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Fig. 1 is a flow chart of a viewer tracking method provided by an embodiment of the present invention. The method may be performed by a viewer tracking apparatus, which may be implemented in software and/or hardware, which may be configured in an electronic device for viewer tracking, such as a naked eye 3D display device. The method is applied to scenes for presenting naked eye 3D display pictures to specific users. As shown in fig. 1, the technical solution provided by the embodiment of the present invention specifically includes:
and S110, acquiring the face image of each candidate user in the viewing range of the naked eye 3D display equipment through an image collector.
The naked eye 3D display device can be directly watched by naked eyes without wearing 3D glasses to realize a display device with a 3D effect, and the naked eye 3D display device can be a naked eye 3D display. The image collector can be a built-in or external user position tracking sensor of the naked eye 3D display device, and can obtain human eye tracking monitoring pictures or head tracking monitoring pictures in the viewing range of the naked eye 3D display device. The user position tracking sensor can be any type of sensor such as a double camera which can acquire the relative screen position of the candidate user and the face image. Based on the imaging principle of the naked eye 3D display device, generally, a left eye image and a right eye image acquired by a user can be just displayed with a 3D display effect at a set position in front of a screen, a set of the positions is a viewing range, and the viewing range is a range in which the user can view the naked eye 3D display effect near the naked eye 3D display device. For example, the viewing range of the naked eye 3D display device may be a plus-minus 45-degree angle range with a normal perpendicular to the center of the screen as a reference line, the viewing range of the naked eye 3D display device may also be a plus-minus 60-degree angle range with a normal perpendicular to the center of the screen as a reference line, the viewing range may also be within 50-100 centimeters of a viewer from the naked eye 3D display device, and the viewing range of the naked eye 3D display device may be set according to actual needs. The range shown between the dotted lines in fig. 2, 5, and 6 is a viewing range of the naked eye 3D display device. The candidate user may be a user viewing a naked eye 3D display through a naked eye 3D display device. According to the scheme, the image collector can be used for obtaining the face images of the candidate users in the watching range of the naked eye 3D display equipment.
And S120, determining the similarity between the face image of each candidate user and a first preset face image, and determining a first target user according to the comparison result of the similarity and a first preset threshold value.
The first preset face image may be a face image of a specific user previously entered in a face image feature library. In the scene shown in fig. 2, the first preset facial image is a facial image of a doctor knife registered in advance. The first preset threshold may be 98%, the first preset threshold may be 99%, and the first preset threshold may be set according to actual needs. According to the scheme, the face images of the candidate users can be acquired through naked eye 3D display equipment, the similarity between the face images of the candidate users and the first preset face image is determined, and the first target user is determined according to the comparison result of the similarity and the first preset threshold value.
For example, as shown in fig. 2 and 3, the number of candidate users is 3, and the scheme acquires the face images of the main doctor, other doctors and an assistant, and respectively matches each face image with the pre-stored face image of the main doctor to determine the similarity. As shown in fig. 3, 1 indicates a face image of a doctor of the doctor, 2 indicates a face image of another doctor, and 3 indicates a face image of an assistant. If the main surgeon is in the candidate users in the viewing range of the naked eye 3D display device, and the main surgeon faces the naked eye 3D display device, but does not form a certain inclination angle with the naked eye 3D display device, the similarity between the facial image of the main surgeon and the pre-stored facial image of the specific user is 99.99%, and if the first preset threshold is set to be 99%, the main surgeon is determined to be the first target user according to the comparison result between the similarity and the first preset threshold. And the similarity between the facial images of other doctors or assistants and the facial image of the main doctor is far lower than the first preset threshold value, so the other doctors and the assistant are not necessarily the first target user.
The determining process of the similarity between the face image of the candidate user and the first preset face image may be to compare the detected corresponding features of the face image of the candidate user with the first preset face image, and when there is a matching record, determine that the face corresponding to the face image of the candidate user and the face of the matched specific user are the face of the same user. The determination process of the similarity between the face image of the candidate user and the first preset face image can also be a process of pre-training a judgment model by means of a machine learning algorithm such as a neural network, namely, a sample containing the face image and whether the face is a specific face is used as a training sample of the judgment model, and the training is carried out on the sample to obtain the trained judgment model. Correspondingly, the similarity determination may be to actually predict the face image of the candidate user as an input of the trained judgment model, and the predicted conclusion is whether the face is the face of the specific person.
And S130, presenting a naked eye 3D display picture to the first target user according to the target relative position of the first target user relative to the screen.
After the fact that the main surgeon is the first target user is determined, the fact that the main surgeon is located in the monitoring picture can be recognized through facial feature recognition, and ideal 3D display content is provided for the main surgeon according to the target relative position of the main surgeon relative to the screen. For example, by moving the relative position between the 3D light splitting device and the display screen in the naked-eye 3D display device, or moving the content displayed on the display screen, the first target user is provided with the desired 3D display content.
It should be noted that, in the process of presenting the 3D display image to the first target user by taking the first target user as the eye tracking object, if it is detected that the user with the higher priority than the first target user falls into the visual field range of the naked eye 3D display device, the eye tracking target is switched from the first target user to the user with the high priority, and the correct 3D display content is presented to the user with the higher priority based on the target relative position of the user with the higher priority relative to the screen. In other words, the scheme preferentially displays the 3D picture by taking the user with higher priority as the human eye tracking object.
Most of the prior art human eye tracking technologies track only one person and enable the person to view ideal 3D display content. When the front of the screen becomes a plurality of people, the display is switched to multi-view display; or the first tracked person to keep track, the other persons appearing behind in front of the screen cannot see the ideal 3D display content unless the first tracked person leaves the tracking picture or its face is temporarily twisted to an angle at which no face is detected, and the eye tracking will switch to another viewer.
The viewer tracking method in the prior art is not suitable for a scene in which a plurality of persons participate in surgical treatment or a scene in which a plurality of persons visit. The reason is that: for a scene that multiple persons participate in surgical treatment, when the multiple persons appear in a human eye tracking picture, the system tracks the human eyes by the first appearing user, even if the user is not a main doctor who needs to pay attention to naked eye 3D display content; even if the system carries out eye tracking on the main surgeon at present, as long as the main surgeon does not watch the naked eye 3D display screen for a moment by twisting the head, the face of the main surgeon cannot be detected temporarily on a human eye tracking picture, the system is switched to other personnel for carrying out eye tracking, and even if the main surgeon watches the naked eye 3D display equipment again later, the system cannot be automatically switched back to the main surgeon for carrying out eye tracking. This makes the primary surgeon need to keep staring at the naked eye 3D display device all the time during the surgery, which lacks flexibility in operation and activity.
In a scene visited by multiple persons, for example, when a visiting group containing a key person experiences naked eye 3D display content, in order to avoid losing tracking of the key person by human tracking, the key person needs to appear on a monitoring picture first and keep staring at a naked eye 3D display screen, otherwise, once the key person is lost and switched to other people, even if the key person is captured by the monitoring picture again, the key person is not automatically switched to the key person for tracking.
According to the technical scheme provided by the embodiment of the invention, the face images of all candidate users in the viewing range of the naked eye 3D display equipment are acquired through the image collector; determining the similarity between the face image of each candidate user and a first preset face image, and determining a first target user according to the comparison result of the similarity and a first preset threshold; and presenting a naked eye 3D display picture to the first target user according to the target relative position of the first target user relative to the screen. By executing the technical scheme provided by the embodiment of the invention, an ideal naked eye 3D display effect can be presented to a main user in a scene where multiple users participate in watching, and the 3D experience of the user can be improved.
Fig. 4 is a flowchart of a viewer tracking method according to an embodiment of the present invention, which is optimized based on the foregoing embodiment. The same points can be referred to the above embodiments, and the embodiment is omitted here. As shown in fig. 4, the viewer tracking method in the embodiment of the present invention may include:
s210: and acquiring the face image of each candidate user in the viewing range of the naked eye 3D display equipment through an image collector.
S220: and determining the similarity between the face image of each candidate user and a first preset face image, and determining a first target user according to the comparison result of the similarity and a first preset threshold value.
In this embodiment, optionally, determining the first target user according to the comparison result between the similarity and the first preset threshold includes: and if the similarity between the face image of the candidate user and a first preset face image is greater than or equal to a first preset threshold value, determining that the candidate user is a first target user.
Illustratively, as shown in fig. 2, if the number of the first preset face images is 1, and the first preset face image is a face image of a main surgeon, the first preset threshold value is 98%, and if the candidate user is the main surgeon, the similarity between the face image of the candidate user and the first preset face image is 98.1% and is greater than the first preset threshold value 98%, the candidate user is determined to be the first target user.
Therefore, the candidate user is determined to be the first target user if the similarity between the face image of the candidate user and the first preset face image is greater than or equal to the first preset threshold value. The specific user can be determined through facial feature recognition, a reliable tracking object is provided for human eye tracking of the specific user, and 3D use experience of the user can be improved.
In a possible implementation manner, optionally, determining the first target user according to the comparison result between the similarity and the first preset threshold includes: and if the number of the candidate users with the similarity greater than or equal to the first preset face image is at least two, determining the candidate user with the highest priority as the first target user according to the priority sequence.
Illustratively, as shown in fig. 5, if the number of the first preset face images is at least one, and the number of the candidate users is also plural, for example, the candidate users are the doctor of the main surgeon and the other doctors, the first preset face images are the face images of the doctor of the main surgeon and the face images of the other doctors. The number of the candidate users, the similarity of which with the first preset face image is greater than or equal to the first preset threshold value, is at least two according to the similarity determined by the face images of the candidate users and the first preset face image. Thus, the first target user needs to be determined according to the order of priority among the candidate users corresponding to the first preset face image. For example, if the candidate users having a similarity greater than or equal to the first preset face image are the main surgeon and the other doctors, and the priority of the main surgeon is higher than that of the other doctors, the main surgeon is determined to be the first target user.
Thus, by determining the candidate user with the highest priority as the first target user according to the priority order if the number of candidate users having a similarity greater than or equal to the first preset face image is at least two. The first target user can be sequentially determined through facial feature recognition and the priority of the specific user, a reliable tracking object is provided for human eye tracking of the specific user, and 3D use experience of the user can be improved.
S230: and presenting a naked eye 3D display picture to the first target user according to the target relative position of the first target user relative to the screen.
S240, if the first target user is not detected in the viewing range of the naked eye 3D display device and only the face image of one candidate user is detected, determining that the candidate user is a second target user.
Specifically, as shown in fig. 6, if the first target user is not in the viewing range of the naked-eye 3D display device, for example, the main surgeon has not yet entered the monitoring picture, or the main surgeon has just moved out of the monitoring picture, or the main surgeon twists his head so that the naked-eye 3D display device cannot detect the facial image of the first target user when performing face detection in the viewing range of the first target user. According to the scheme, other candidate users in the viewing range of the naked eye 3D display device are identified, and if only one candidate user is determined, such as other doctors, the candidate user is used as a tracking object of the naked eye 3D display, namely a second target user.
And S250, presenting a naked eye 3D display picture to the second target user according to the target relative position of the second target user relative to the screen.
After the other doctor is determined to be the second target user, the other doctor can be identified to be located in the monitoring picture through facial feature identification, and ideal 3D display content is provided for the other doctor according to the target relative position of the other doctor relative to the screen. For example, by moving the relative position between the 3D light splitting device and the display screen in the naked-eye 3D display device, the first target user is provided with ideal 3D display content. Or move the content displayed on the display screen so that the first target user is provided with the desired 3D display content.
According to the technical scheme provided by the embodiment of the invention, the face images of all candidate users in the viewing range of the naked eye 3D display equipment are acquired through the image collector; determining the similarity between the face image of each candidate user and a first preset face image, and determining a first target user according to the comparison result of the similarity and a first preset threshold; presenting a naked eye 3D display picture to a first target user according to the target relative position of the first target user relative to a screen; if the first target user is not detected in the viewing range of the naked eye 3D display device and only the face image of one candidate user is detected, determining that the candidate user is a second target user; and presenting a naked eye 3D display picture to the second target user according to the target relative position of the second target user relative to the screen. By executing the scheme, an ideal naked eye 3D display effect can be presented to the main user in a scene where multiple users participate in watching, and the 3D experience of the user can be improved.
Fig. 7 is a flowchart of a viewer tracking method according to an embodiment of the present invention, which is optimized based on the foregoing embodiment. The same points can be referred to the above embodiments, and the embodiment is omitted here. As shown in fig. 7, the viewer tracking method in the embodiment of the present invention may include:
and S310, acquiring the face image of each candidate user in the viewing range of the naked eye 3D display equipment through an image collector.
S320, determining the similarity between the face image of each candidate user and a first preset face image, and determining a first target user according to the comparison result of the similarity and a first preset threshold value.
S330, presenting a naked eye 3D display picture to the first target user according to the target relative position of the first target user relative to the screen.
In one possible embodiment, optionally, the method further comprises: and if the first target user is not detected in the viewing range of the naked eye 3D display equipment, presenting a 2D display picture to the candidate user or presenting a multi-view display picture to the candidate user.
As shown in fig. 8, 2 represents a face image of another doctor, and 3 represents a face image of an assistant. If the first target user is the main surgeon, but the main surgeon cannot be detected in the viewing range of the naked eye 3D display device by the scheme, only other candidate users are detected, and the scheme can present 2D display pictures to the other candidate users or present different 3D display pictures to different candidate users according to the positions of the other candidate users relative to the screen.
Thus, if the first target user is not detected in the viewing range of the naked eye 3D display device, the candidate user is presented with the 2D display picture or the candidate user is presented with the multi-view display picture. The method and the device can realize that the user with low requirement on the 3D picture can provide the corresponding display picture, improve the processing efficiency and save the computing resource.
In another possible embodiment, optionally, presenting a naked eye 3D display to the first target user according to a target relative position of the first target user with respect to a screen includes: according to the target relative position of the first target user relative to the screen, adjusting the relative position between a 3D light splitting device and a display screen in the naked eye 3D display equipment, and presenting a naked eye 3D display picture to the first target user; or adjusting display content on a display screen in the naked eye 3D display equipment according to the target relative position of the first target user relative to the screen, and presenting a naked eye 3D display picture to the first target user.
The 3D light splitting device can be a plurality of groups of columnar prisms and forms a certain inclined included angle with the 3D display screen, and each group of columnar prisms corresponds to one group of arrangement diagrams. According to the scheme, the relative position between the 3D light splitting device and the display screen in the naked eye 3D display equipment can be adjusted according to the target relative position of the first target user relative to the screen, so that a correct naked eye 3D display picture is presented to the first target user, and the eye tracking of the first target user is realized. Or according to the scheme, the display content on the display screen in the naked eye 3D display equipment can be adjusted according to each group of arrangement patterns according to the target relative position of the first target user relative to the screen, and a naked eye 3D display picture is presented to the first target user, so that the human eye tracking of the first target user is realized. For implementation of naked eye 3D display based on human eye tracking, reference may be made to the detailed description in the related art.
Therefore, a naked eye 3D display picture is presented to the first target user by adjusting the relative position between the 3D light splitting device and the display screen in the naked eye 3D display equipment or adjusting the display content on the display screen in the naked eye 3D display equipment according to the target relative position of the first target user relative to the screen. Naked eye 3D display with human eye tracking for a specific user can be realized.
S340, if the first target user is not detected in the viewing range of the naked eye 3D display device, determining the similarity between the face image of each candidate user and a second preset face image, and determining the second target user according to the comparison result of the similarity and a second preset threshold value.
Illustratively, the number of the specific users is two, including the doctor of the main doctor and the other doctors, and the second preset facial image is a pre-recorded facial image of the other doctors. The second preset threshold may be 98%, the second preset threshold may be 99%, and the second preset threshold may be set according to actual needs. According to the scheme, if the first target user is not detected in the viewing range of the naked eye 3D display device, the second target user is determined according to the comparison result of the similarity between the face image of each candidate user and the second preset face image and the second preset threshold value. Specifically, if the candidate user is another doctor or an assistant, the similarity between the face image of the candidate user as another doctor and the second preset face image is 100%, and if the similarity is greater than the second preset threshold, it is determined that the other doctor is the second target user. And if the similarity between the face image of the candidate user as the assistant and the second preset face image is far smaller than the second preset threshold, determining that the assistant is not the second target user.
In this embodiment, optionally, determining the second target user according to a comparison result between the similarity between the face image of each candidate user and the second preset face image and a second preset threshold, includes: if the similarity between the face image of the candidate user and a second preset face image is larger than or equal to a second preset threshold value, taking the candidate user as a second target user; or if the similarity between the face image of the candidate user and a second preset face image is maximum, taking the candidate user as a second target user; or if the number of the candidate users with the similarity greater than or equal to the second preset face image is at least two, determining the candidate user with the highest priority as the second target user according to the priority order.
Illustratively, if the number of the second preset face images is only one, and the second preset face images are face images of other doctors, the candidate users are other doctors and assistants. If the similarity between the facial images of other doctors and the second preset facial image is greater than or equal to a second preset threshold value, taking the other doctors as second target users; or, if the similarity between the face image of the other doctor and the second preset face image is greater than the similarity between the face image of the assistant and the second preset face image, the other doctor is taken as the second target user.
If the number of the second preset face images is two and the second preset face images are face images of other doctors and face images of assistants, the candidate users are other doctors and assistants. And if the candidate users with the similarity degree with the second preset facial image larger than or equal to the second preset threshold value are other doctors and assistants, determining other doctors with the highest priority as second target users according to the priority orders of the other doctors and the assistants.
Therefore, by determining the second target user under different conditions, the human eye tracking and the 3D display of a plurality of specific users can be realized, and the switching of tracking users can be flexibly carried out.
And S350, presenting a naked eye 3D display picture to the second target user according to the target relative position of the second target user relative to the screen.
After the other doctors are determined to be the second target users, the other doctors can be identified to be located in the monitoring picture through facial feature recognition, and ideal 3D display content is provided for the other doctors according to the target relative positions of the other doctors relative to the screen. For example, by moving the relative position between the 3D light splitting device and the display screen in the naked-eye 3D display device, or moving the content displayed on the display screen, the second target user is provided with the desired 3D display content.
As shown in fig. 9, 1 indicates a face image of a doctor of the assistant of the doctor of the assistant. According to the scheme, if the first target user is detected to fall into the visual field range of naked eye 3D display equipment again in the process of presenting the 3D display picture to the second target user by taking the second target user as the eye tracking object, the eye tracking target is switched from the second target user to the first target user, and correct 3D display content is presented to the first target user continuously according to the target relative position of the first target user relative to the screen. In other words, the scheme preferentially displays the 3D picture by taking the user with higher priority as the human eye tracking object.
According to the technical scheme provided by the embodiment of the invention, the face images of all candidate users in the viewing range of the naked eye 3D display equipment are acquired through the image collector; determining the similarity between the face image of each candidate user and a first preset face image, and determining a first target user according to the comparison result of the similarity and a first preset threshold; presenting a naked eye 3D display picture to a first target user according to the target relative position of the first target user relative to a screen; if the first target user is not detected in the viewing range of the naked eye 3D display device, determining the similarity between the face image of each candidate user and a second preset face image, and determining a second target user according to the comparison result of the similarity and a second preset threshold; and presenting a naked eye 3D display picture to the second target user according to the target relative position of the second target user relative to the screen. By executing the scheme, an ideal naked eye 3D display effect can be presented to the main user in a scene where multiple users participate in watching, and the 3D experience of the user can be improved.
Fig. 10 is a schematic structural diagram of a viewer tracking apparatus configured in a naked-eye 3D display device according to an embodiment of the present invention, where the apparatus may be configured in an electronic device for viewer tracking. As shown in fig. 10, the apparatus includes:
a face image obtaining module 410, configured to obtain, by an image collector, a face image of each candidate user in a viewing range of the naked eye 3D display device;
a first target user determining module 420, configured to determine similarity between the face image of each candidate user and a first preset face image, and determine a first target user according to a comparison result between the similarity and a first preset threshold;
and a display image presenting module 430, configured to present a naked eye 3D display image to the first target user according to a target relative position of the first target user with respect to a screen.
Optionally, the first target user determining module 420 is specifically configured to determine that the candidate user is the first target user if the similarity between the face image of the candidate user and the first preset face image is greater than or equal to a first preset threshold.
Optionally, the first target user determining module 420 is specifically configured to determine, if the number of candidate users whose similarity to the first preset face image is greater than or equal to the first preset threshold is at least two, the candidate user with the highest priority as the first target user according to the priority order.
Optionally, the apparatus further includes a second target user determining unit, configured to determine, if the first target user is not detected within the viewing range of the naked-eye 3D display device, a similarity between a face image of each candidate user and a second preset face image, and determine a second target user according to a comparison result between the similarity and a second preset threshold; and the display picture presenting unit is used for presenting a naked eye 3D display picture to the second target user according to the target relative position of the second target user relative to the screen.
Optionally, the second target user determining unit is specifically configured to, if the similarity between the face image of the candidate user and a second preset face image is greater than or equal to a second preset threshold, use the candidate user as a second target user; or if the similarity between the face image of the candidate user and a second preset face image is maximum, taking the candidate user as a second target user; or if the number of the candidate users with the similarity greater than or equal to the second preset face image is at least two, determining the candidate user with the highest priority as the second target user according to the priority order.
Optionally, the apparatus further includes a second target user determining module, configured to determine that the candidate user is a second target user if the first target user is not detected in the viewing range of the naked-eye 3D display device and only a face image of one candidate user is detected; and presenting a naked eye 3D display picture to the second target user according to the target relative position of the second target user relative to the screen.
Optionally, the apparatus further includes another display screen presenting module, configured to present a 2D display screen to the candidate user or present a multi-view display screen to the candidate user if the first target user is not detected within the viewing range of the naked-eye 3D display device.
Optionally, the display image presenting module 430 is specifically configured to adjust a relative position between a 3D optical splitter and a display screen in the naked-eye 3D display device according to a target relative position of the first target user with respect to the screen, and present a naked-eye 3D display image to the first target user; or adjusting display content on a display screen in the naked eye 3D display equipment according to the target relative position of the first target user relative to the screen, and presenting a naked eye 3D display picture to the first target user.
The device provided by the above embodiment can execute the viewer tracking method executed by the naked-eye 3D display device provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
Fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 11, the electronic device includes:
one or more processors 510, one processor 510 being illustrated in FIG. 11;
a memory 520;
the apparatus may further include: an input device 530 and an output device 540.
The processor 510, the memory 520, the input device 530 and the output device 540 of the apparatus may be connected by a bus or other means, and fig. 11 illustrates the connection by a bus as an example.
The memory 520, which is a non-transitory computer-readable storage medium, may be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to a viewer tracking method in embodiments of the present invention. The processor 510 executes various functional applications and data processing of the computer device by executing software programs, instructions and modules stored in the memory 520, namely, a viewer tracking method implementing the above-described method embodiments, namely:
acquiring facial images of candidate users in a viewing range of the naked eye 3D display equipment through an image collector;
determining the similarity between the face image of each candidate user and a first preset face image, and determining a first target user according to the comparison result of the similarity and a first preset threshold;
and presenting a naked eye 3D display picture to the first target user according to the target relative position of the first target user relative to the screen.
The memory 520 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the computer device, and the like. Further, the memory 520 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 520 may optionally include memory located remotely from processor 510, which may be connected to a terminal device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 530 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the computer apparatus. The output device 540 may include a display device such as a display screen.
Embodiments of the present invention provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a viewer tracking method as provided by embodiments of the present invention, that is:
acquiring facial images of candidate users in a viewing range of the naked eye 3D display equipment through an image collector;
determining the similarity between the face image of each candidate user and a first preset face image, and determining a first target user according to the comparison result of the similarity and a first preset threshold;
and presenting a naked eye 3D display picture to the first target user according to the target relative position of the first target user relative to the screen.
Any combination of one or more computer-readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (11)
1. A viewer tracking method is applied to naked eye 3D display equipment and is characterized by comprising the following steps:
acquiring facial images of candidate users in a viewing range of the naked eye 3D display equipment through an image collector;
determining the similarity between the face image of each candidate user and a first preset face image, and determining a first target user according to the comparison result of the similarity and a first preset threshold;
and presenting a naked eye 3D display picture to the first target user according to the target relative position of the first target user relative to the screen.
2. The method according to claim 1, wherein determining the first target user according to the comparison result of the similarity with the first preset threshold comprises:
and if the similarity between the face image of the candidate user and a first preset face image is greater than or equal to a first preset threshold value, determining that the candidate user is a first target user.
3. The method according to claim 1, wherein determining the first target user according to the comparison result of the similarity with the first preset threshold comprises:
and if the number of the candidate users with the similarity greater than or equal to the first preset face image is at least two, determining the candidate user with the highest priority as the first target user according to the priority sequence.
4. The method of claim 1, further comprising:
if the first target user is not detected in the viewing range of the naked eye 3D display device, determining the similarity between the face image of each candidate user and a second preset face image, and determining a second target user according to the comparison result of the similarity and a second preset threshold;
and presenting a naked eye 3D display picture to the second target user according to the target relative position of the second target user relative to the screen.
5. The method according to claim 4, wherein determining the second target user according to the comparison result of the similarity with the second preset threshold comprises:
if the similarity between the face image of the candidate user and a second preset face image is larger than or equal to a second preset threshold value, taking the candidate user as a second target user; alternatively, the first and second electrodes may be,
if the similarity between the face image of the candidate user and a second preset face image is maximum, taking the candidate user as a second target user; alternatively, the first and second electrodes may be,
and if the number of the candidate users with the similarity between the candidate users and the second preset face image larger than or equal to the second preset threshold value is at least two, determining the candidate user with the highest priority as the second target user according to the priority sequence.
6. The method of claim 1, further comprising:
if the first target user is not detected in the viewing range of the naked eye 3D display device and only the face image of one candidate user is detected, determining that the candidate user is a second target user;
and presenting a naked eye 3D display picture to the second target user according to the target relative position of the second target user relative to the screen.
7. The method of claim 1, further comprising:
and if the first target user is not detected in the viewing range of the naked eye 3D display equipment, presenting a 2D display picture to the candidate user or presenting a multi-view display picture to the candidate user.
8. The method of claim 1, wherein presenting a naked eye 3D display to the first target user according to a target relative position of the first target user with respect to a screen comprises:
according to the target relative position of the first target user relative to the screen, adjusting the relative position between a 3D light splitting device and a display screen in the naked eye 3D display equipment, and presenting a naked eye 3D display picture to the first target user;
or adjusting display content on a display screen in the naked eye 3D display equipment according to the target relative position of the first target user relative to the screen, and presenting a naked eye 3D display picture to the first target user.
9. A viewer tracking apparatus configured for a naked eye 3D display device, comprising:
the face image acquisition module is used for acquiring face images of candidate users in the viewing range of the naked eye 3D display equipment through an image collector;
the first target user determination module is used for determining the similarity between the face image of each candidate user and a first preset face image and determining a first target user according to the comparison result of the similarity and a first preset threshold;
and the display picture presenting module is used for presenting a naked eye 3D display picture to the first target user according to the target relative position of the first target user relative to the screen.
10. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the viewer tracking method of any of claims 1-8.
11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a viewer tracking method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111659288.XA CN114356088B (en) | 2021-12-30 | 2021-12-30 | Viewer tracking method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111659288.XA CN114356088B (en) | 2021-12-30 | 2021-12-30 | Viewer tracking method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114356088A true CN114356088A (en) | 2022-04-15 |
| CN114356088B CN114356088B (en) | 2024-03-01 |
Family
ID=81104564
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111659288.XA Active CN114356088B (en) | 2021-12-30 | 2021-12-30 | Viewer tracking method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114356088B (en) |
Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101401426A (en) * | 2006-03-15 | 2009-04-01 | 欧姆龙株式会社 | Tracking device, tracking method, tracking device control program, and computer-readable recording medium |
| CN102300110A (en) * | 2010-06-24 | 2011-12-28 | 索尼公司 | Display device |
| CN102572483A (en) * | 2011-12-02 | 2012-07-11 | 深圳超多维光电子有限公司 | Tracking type autostereoscopic display control method, device and system, and display equipment |
| US20130050445A1 (en) * | 2011-08-31 | 2013-02-28 | Kabushiki Kaisha Toshiba | Video processing apparatus and video processing method |
| JP2013055681A (en) * | 2012-10-30 | 2013-03-21 | Toshiba Corp | Video processing apparatus and video processing method |
| US20130243388A1 (en) * | 2012-03-15 | 2013-09-19 | O2Micro, Inc. | System and Method for Controlling Video Player |
| JP2014013617A (en) * | 2013-10-08 | 2014-01-23 | Toshiba Corp | Video display device and video display method |
| CN104008732A (en) * | 2014-05-14 | 2014-08-27 | 陆俊 | Method and system for adjusting brightness of LED backlight of liquid crystal display |
| CN104732210A (en) * | 2015-03-17 | 2015-06-24 | 深圳超多维光电子有限公司 | Target human face tracking method and electronic equipment |
| CN107172417A (en) * | 2017-06-30 | 2017-09-15 | 深圳超多维科技有限公司 | A kind of method for displaying image, the apparatus and system of bore hole 3D screens |
| CN107833263A (en) * | 2017-11-01 | 2018-03-23 | 宁波视睿迪光电有限公司 | Feature tracking method and device |
| US20180239579A1 (en) * | 2015-10-28 | 2018-08-23 | Kyocera Corporation | Reproduction apparatus |
| CN111815676A (en) * | 2020-07-06 | 2020-10-23 | 浙江大华技术股份有限公司 | Face tracking method, device, equipment and storage device |
| CN112333542A (en) * | 2020-08-20 | 2021-02-05 | 深圳Tcl新技术有限公司 | Program recommendation page determining method, device and equipment and readable storage medium |
| CN112437229A (en) * | 2020-11-10 | 2021-03-02 | 苏州爱医斯坦智能科技有限公司 | Picture tracking method and device, electronic equipment and storage medium |
| CN112929635A (en) * | 2019-12-05 | 2021-06-08 | 北京芯海视界三维科技有限公司 | Display control method of naked eye 3D display screen and multi-view naked eye 3D display |
| CN113438464A (en) * | 2021-06-22 | 2021-09-24 | 纵深视觉科技(南京)有限责任公司 | Switching control method, medium and system for naked eye 3D display mode |
| CN113495976A (en) * | 2020-04-03 | 2021-10-12 | 百度在线网络技术（北京）有限公司 | Content display method, device, equipment and storage medium |
-
2021
- 2021-12-30 CN CN202111659288.XA patent/CN114356088B/en active Active
Patent Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090096871A1 (en) * | 2006-03-15 | 2009-04-16 | Omron Corporation | Tracking device, tracking method, tracking device control program, and computer-readable recording medium |
| CN101401426A (en) * | 2006-03-15 | 2009-04-01 | 欧姆龙株式会社 | Tracking device, tracking method, tracking device control program, and computer-readable recording medium |
| CN102300110A (en) * | 2010-06-24 | 2011-12-28 | 索尼公司 | Display device |
| US20130050445A1 (en) * | 2011-08-31 | 2013-02-28 | Kabushiki Kaisha Toshiba | Video processing apparatus and video processing method |
| CN102572483A (en) * | 2011-12-02 | 2012-07-11 | 深圳超多维光电子有限公司 | Tracking type autostereoscopic display control method, device and system, and display equipment |
| US20130243388A1 (en) * | 2012-03-15 | 2013-09-19 | O2Micro, Inc. | System and Method for Controlling Video Player |
| JP2013055681A (en) * | 2012-10-30 | 2013-03-21 | Toshiba Corp | Video processing apparatus and video processing method |
| JP2014013617A (en) * | 2013-10-08 | 2014-01-23 | Toshiba Corp | Video display device and video display method |
| CN104008732A (en) * | 2014-05-14 | 2014-08-27 | 陆俊 | Method and system for adjusting brightness of LED backlight of liquid crystal display |
| CN104732210A (en) * | 2015-03-17 | 2015-06-24 | 深圳超多维光电子有限公司 | Target human face tracking method and electronic equipment |
| US20180239579A1 (en) * | 2015-10-28 | 2018-08-23 | Kyocera Corporation | Reproduction apparatus |
| CN107172417A (en) * | 2017-06-30 | 2017-09-15 | 深圳超多维科技有限公司 | A kind of method for displaying image, the apparatus and system of bore hole 3D screens |
| CN107833263A (en) * | 2017-11-01 | 2018-03-23 | 宁波视睿迪光电有限公司 | Feature tracking method and device |
| WO2019085519A1 (en) * | 2017-11-01 | 2019-05-09 | 宁波视睿迪光电有限公司 | Method and device for facial tracking |
| CN112929635A (en) * | 2019-12-05 | 2021-06-08 | 北京芯海视界三维科技有限公司 | Display control method of naked eye 3D display screen and multi-view naked eye 3D display |
| CN113495976A (en) * | 2020-04-03 | 2021-10-12 | 百度在线网络技术（北京）有限公司 | Content display method, device, equipment and storage medium |
| CN111815676A (en) * | 2020-07-06 | 2020-10-23 | 浙江大华技术股份有限公司 | Face tracking method, device, equipment and storage device |
| CN112333542A (en) * | 2020-08-20 | 2021-02-05 | 深圳Tcl新技术有限公司 | Program recommendation page determining method, device and equipment and readable storage medium |
| CN112437229A (en) * | 2020-11-10 | 2021-03-02 | 苏州爱医斯坦智能科技有限公司 | Picture tracking method and device, electronic equipment and storage medium |
| CN113438464A (en) * | 2021-06-22 | 2021-09-24 | 纵深视觉科技(南京)有限责任公司 | Switching control method, medium and system for naked eye 3D display mode |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114356088B (en) | 2024-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108919958B (en) | Image transmission method and device, terminal equipment and storage medium | |
| JP2020170569A (en) | Authentication system, authentication method, and program | |
| US10416764B2 (en) | Method for operating an eye tracking device for multi-user eye tracking and eye tracking device | |
| US11295550B2 (en) | Image processing method and apparatus, and terminal device | |
| US9489574B2 (en) | Apparatus and method for enhancing user recognition | |
| CN109086726A (en) | A kind of topography's recognition methods and system based on AR intelligent glasses | |
| US9696798B2 (en) | Eye gaze direction indicator | |
| JP6454851B2 (en) | 3D gaze point location algorithm | |
| KR101455200B1 (en) | Learning monitering device and method for monitering of learning | |
| KR20190020779A (en) | Ingestion Value Processing System and Ingestion Value Processing Device | |
| CN111325107B (en) | Detection model training method, device, electronic equipment and readable storage medium | |
| CN112101123B (en) | Attention detection method and device | |
| CN113205057A (en) | Face living body detection method, device, equipment and storage medium | |
| CN113438464A (en) | Switching control method, medium and system for naked eye 3D display mode | |
| US11294474B1 (en) | Controlling video data content using computer vision | |
| KR101847446B1 (en) | Apparatus and method for eye-tracking base on cognition data network | |
| US11361590B2 (en) | Method and apparatus for monitoring working state | |
| KR20140003107A (en) | Apparatus and method for providing augmented reality | |
| US20150178589A1 (en) | Apparatus for processing digital image and method of controlling the same | |
| CN114356088B (en) | Viewer tracking method and device, electronic equipment and storage medium | |
| CN112651270A (en) | Gaze information determination method and apparatus, terminal device and display object | |
| CN111857336B (en) | Head-mounted device, rendering method thereof, and storage medium | |
| WO2018061299A1 (en) | Information processing device and information processing method | |
| KR102356165B1 (en) | Method and device for indexing faces included in video | |
| CN114299598A (en) | Method for determining fixation position and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |