WO2023113766A1

WO2023113766A1 - Applying privacy actions

Info

Publication number: WO2023113766A1
Application number: PCT/US2021/063025
Authority: WO
Inventors: Tirath Ramdas
Original assignee: Hewlett-Packard Development Company, L.P.
Priority date: 2021-12-13
Filing date: 2021-12-13
Publication date: 2023-06-22

Abstract

In some examples, a computing device can include a processing resource and a memory resource storing instructions to cause the processing resource to perform facial detection of a user in a frame of a video feed, perform facial recognition on a face of the user in the frame to determine an identity of the user, and apply a privacy action to the frame based on the facial detection and the facial recognition.

Description

APPLYING PRIVACY ACTIONS

Background

[0001] Computing devices can allow a user to utilize computing device operations for work, education, gaming, multimedia, and/or other uses. Computing devices can be portable to allow a user to carry or otherwise bring the computing device with while in a mobile setting. A computing device can allow a user to utilize computing device operations for work, education, gaming, multimedia, and/or other general use in a mobile setting.

Brief Description of the Drawings

[0002] Figure 1 illustrates an example of a system for applying privacy actions consistent with this disclosure.

[0003] Figure 2 illustrates an example of facial detection and recognition of a user in a frame consistent with this disclosure.

[0004] Figure 3 illustrates an example of application of a privacy action to a frame without a user consistent with this disclosure.

[0005] Figure 4 illustrates an example of application of a privacy action to a frame with a user consistent with this disclosure.

[0006] Figure 5 illustrates an example of application of a privacy action to a frame with a first user and a second user consistent with this disclosure.

[0007] Figure 6 illustrates an example of a computing device for applying privacy actions consistent with this disclosure.

[0008] Figure 7 illustrates a block diagram of an example system for applying privacy actions consistent with this disclosure.

[0009] Figure 8 illustrates an example of a method for applying privacy actions consistent with this disclosure.

Detailed Description [0010] A user may utilize a computing device for various purposes, such as for business and/or recreational use. As used herein, the term “computing device” refers to an electronic system having a processing resource, memory resource, and/or an application-specific integrated circuit (ASIC) that can process information. A computing device can be, for example, a laptop computer, a notebook, a desktop, a tablet, and/or a mobile device, among other types of computing devices.

[0011] A user may utilize their computing device during a virtual meeting (e.g., conducted via a user’s computing device). As used herein, the term “virtual meeting” refers to a conference extended by computing device software in which the reception and transmission of audio signals, video signals, or combinations thereof occurs between users in different locations. For example, multiple users may conduct a virtual meeting in which the first user can communicate, through the first user’s computing device, with the second user through the second user’s computing device and vice versa via transmission of audio signals, video signals, or combinations thereof therebetween. During the virtual meeting, a camera can capture the first user and generate an electrical signal for transmission to another computing device so that the first user is viewable by a second user via the another computing device. As used herein, the term “camera” refers to a device that records visual images.

[0012] A camera can capture a particular field of view. As used herein, the term “field of view” refers to a physical area that is imaged by a camera. For example, a user may be captured by a cameras field of view. However, the field of view of the camera may additionally capture a space surrounding the user, such as walls, windows, and/or other objects included in the space.

[0013] In some instances, the camera may capture within its field of view sensitive information not intended to be broadcast during a virtual meeting. For example, printouts, whiteboards, other screens, etc. may include sensitive information (e.g., information that may be confidential, privileged, or otherwise not intended for broadcasting).

[0014] Additionally, the camera may capture other persons who are not intended to be broadcast during the virtual meeting. For example, a person who is not intended to be broadcast during the virtual meeting may stray within the field of view of the camera (intentionally or unintentionally). Such an occurrence may happen in an uncontrolled environment, such as in a household, when the user is attending the virtual meeting in a public space (e.g., a coffee shop, airport, etc.), among other examples.

[0015] In previous approaches, virtual meeting applications may utilize “virtual backgrounds”. As used herein, the term “virtual background” refers to a simulated background that appears behind a participant in a video conference. For example, a virtual background may be applied that surrounds/obscures a field of view of the camera but allows the user to be visible during the virtual meeting.

[0016] However, utilizing a virtual background can include various drawbacks. For example, if the user steps out of the field of view of the camera without deactivating the camera, persons stepping into the field of view of the camera who are not intended to be broadcast during the virtual meeting may still be captured by the camera. Additionally, persons stepping into the field of view of the camera while the user is also in the field of view of the camera are captured by the camera. Further, the camera may still inadvertently capture sensitive information located in a field of view of the camera, particularly where there is movement within the field of view of the camera as the virtual background adjusts locations.

[0017] Additionally, some cameras may include a physical privacy cover. However, the user has to remember to activate the privacy cover in order to obtain privacy during a virtual meeting.

[0018] Applying privacy actions according to the disclosure can allow for application of a virtual background to obscure the field of view of a camera based on detected faces in a frame as well as facial recognition techniques. Such an approach can obscure the field of view of the camera, as well as occlude persons not intended to be broadcast. Such an approach can provide additional and more thorough privacy to users and/or other unintended persons during a virtual meeting as compared with previous approaches.

[0019] Figure 1 illustrates an example of a system 100 for applying privacy actions consistent with this disclosure. The system 100 can include computing device 102, users 110-1, 110-2, and external computing device 112. The computing device 102 can include a camera 104 and a microphone 108.

[0020] As mentioned above, a user 110-1 may be utilizing camera 104 for a virtual meeting. For example, the camera 104 can capture the user 110-1 within the field of view 106 of the camera 104, and the image of the user 110-1 can be transmitted to other computing devices such that other users may view the user 110- 1 during the virtual meeting.

[0021] However, as described above, within the field of view 106 of the camera 104 may be other sensitive information that the user 110-1 may desire to keep secret or confidential. Additionally, while the user 110-1 may intend to be within the field of view 106, another user 110-2 (e.g., illustrated in Figure 1 via dashed lines to indicate user 110-2 may or may not be within the field of view 106) may enter the field of view 106 (e.g., intentionally or unintentionally) and be unintentionally broadcasted to other attendees of the virtual meeting. As such, the computing device 102 can utilize applying privacy actions consistent with the disclosure to mitigate such events, as is further described herein.

[0022] In order to apply privacy actions, the computing device 102 can retrieve a frame from a video feed of the camera 104. As used herein, the term “frame” refers to a still image. As used herein, the term “video feed” refers to a series of frames that comprise a video. For example, as the camera 104 captures information, the resulting information is transformed into a video feed comprising a series of successive frames that, when played together in chronological order, make up a video. Accordingly, the computing device 102 can retrieve a frame from the video feed, where the frame can include objects located within the field of view 106 of the camera 104. As illustrated in Figure 1 , the user 110-1 may be within the field of view 106 and as such, can be included as a portion of the frame.

[0023] Utilizing the frame, the computing device 102 performs facial detection of the user 110-1 in the frame. The computing device 102 can utilize face detection techniques in order to determine coordinates of a face of the user 110-1 included in the frame. For example, the computing device 102 may utilize machine learning, artificial neural networks, and/or other face detection techniques in order to perform facial detection of the user 110-1 to determine coordinates on the frame corresponding to the face of the user 110-1 in the frame.

[0024] In an example in which no faces are determined to be in the frame (e.g., facial detection does not detect a face), the computing device 102 can determine a privacy action should be taken. As used herein, the term “privacy action” refers to an action in which information is obscured. For example, in an instance in which user 110-1 (or 110-2) is not detected in the frame (e.g., are not within the field of view 106 of the camera 104), the computing device can apply a virtual background over the entire frame, cut off the video feed so that it is not transmitted to other attendees of the virtual meeting, among other techniques as is further described in connection with Figure 3.

[0025] In response to a face of the user 110-1 being detected in the frame, the computing device can perform facial recognition on the frame. Facial recognition can be performed at the coordinates of the frame determined during the facial detection to determine an identity of the user 110-1 by comparing a scan of the face of the user 110-1 in the frame with a facial encoding. As used herein, the term “facial encoding” refers to a set of indicators that, when compared with a facial scan, express a likeness against the facial scan. The facial encoding can be, for example, an output of a calibration scan (or set of calibration scans) of a first person, where the calibration scan(s) include the set of indicators which can correspond to portions of a face of the first person. The set of indicators can be compared against subsequent scans of faces to determine a likeness of the subsequently scanned faces with that of the set of indicators (e.g., the first person). For example, a scan of the face of the user 110-1 can be compared with a facial encoding to determine whether the scan and the facial encoding match. In such a way, the computing device 102 can verify the identity of the user 110-1.

[0026] The computing device 102 can compare the scan of the face of the user 110-1 to the facial encoding. In some examples, the facial encoding can be scanned and recorded by the camera 104 prior to the virtual meeting beginning. For example, as the user 110-1 joins the virtual meeting, the camera 104 may perform a calibration scan to scan the face of the user 110-1 and save the scan as the facial encoding. Accordingly, as frames are retrieved by the computing device 102, the face of the user 110-1 can be repeatedly checked against the facial encoding to ensure the user within the field of view 106 is the user 110-1.

[0027] However, examples of the disclosure are not so limited to performing a calibration scan prior to the virtual meeting beginning. For example, the user 110-1 may have a facial scan performed which can be saved as the facial encoding and utilized across different virtual meetings. Such a facial encoding may be saved locally at the computing device 102. However, in some examples, the facial encoding of the user 110-1 may be stored externally to the computing device 102 (e.g., at external computing device 112) and can be associated with a user profile of the user 110-1. The external computing device 112 may be connected to the computing device 102 via a wired or wireless network relationship. Accordingly, if the user 110-1 utilizes a different computing device (e.g., from computing device 102), the different computing device can utilize the facial encoding associated with the user 110-1 upon connection with the external computing device 112.

[0028] As mentioned above, the computing device 102 can be connected to the external computing device 112 via a wired or wireless network relationship. Examples of such a network relationship can include a local area network (LAN), wide area network (WAN), personal area network (PAN), a distributed computing environment (e.g., a cloud computing environment), storage area network (SAN), Metropolitan area network (MAN), a cellular communications network, Long Term Evolution (LTE), visible light communication (VLC), Bluetooth, Worldwide Interoperability for Microwave Access (WiMAX), Near Field Communication (NFC), infrared (IR) communication, Public Switched Telephone Network (PSTN), radio waves, and/or the Internet, among other types of network relationships.

[0029] Although the computing device 102 is described above as performing the facial recognition, examples of the disclosure are not so limited. For example, the external computing device 112 may perform the facial recognition. For instance, the computing device 102 can transmit the scanned face of the user 110-1 to the external computing device 112, the external computing device 112 can compare the scanned face to the facial encoding, determine an identity of the user 110-1 , and transmit the determined identity of the user 110-1 to the computing device 102. Utilizing the external computing device 112 to perform facial recognition can reduce computational loads on the computing device 102.

[0030] In an example in which the computing device 102 performs face detection, determines there is a face of a user 110 in the frame (e.g., within the field of view 106 of the camera 104) but it is in fact the user 110-2 that is within the frame, a scan of the face of the user 110-2 would not match a facial encoding. In such an example, if the user 110-2 is the user that is within the frame, the computing device 102 can perform a privacy action by applying a virtual background over the entire frame, cut off the video feed so that it is not transmitted to other attendees of the virtual meeting, among other techniques so as to not broadcast the image of user 110-2 as is further described in connection with Figure 4. if the user 110-2 is within the frame at the same time that user 110-1 is within the frame, the computing device 102 can perform a privacy action by occluding the user 110-2 from the frame while still keeping user 110-1 visible in the frame so that an image of user 110-2 is not transmitted to other attendees of the virtual meeting, as is further described in connection with Figure 5.

[0031] As mentioned above, the computing device 102 can apply a privacy action to the frame based on the facial detection and the facial recognition. For example, if no face is detected, the computing device 102 can apply a privacy action by applying a virtual background to the frame (e.g., as is further described in connection with Figure 3). If a face is detected and it is determined that the user of the face is not someone who is to be viewed during the virtual meeting, the computing device 102 can apply a privacy action by applying a virtual background to the frame (e.g., as is further described in connection with Figure 4). If multiple users 110-1, 110-2 are within the field of view 106, one user 110-1 is to be viewed during the virtual meeting, but the other user 110-2 is not to be viewed during the virtual meeting, the computing device 102 can apply a privacy action by occluding the user 110-2 from the frame (e.g., as is further described in connection with Figure 5). [0032] In some examples, the computing device 102 can apply a privacy action by generating a lock signal. The lock signal can cause the computing device 102 and/or other peripheral devices to perform additional/other privacy actions. The computing device 102 can transmit the lock signal to cause the additional/other privacy actions to occur. For example, the lock signal lock a user interface 105 of the computing device 102, mute an audio output device 107 of the computing device 102, mute a microphone 108 of the computing device 102, among other examples, as are further described herein.

[0033] As illustrated in Figure 1, the computing device 102 includes a user interface 105. The user interface 105 can be, for example, a display with which a user 110-1 can view the virtual meeting (e.g., view other attendees, view presented content, etc.). However, the virtual meeting may not be intended for viewing for some users (e.g., user 110-2) and as such, if the user 110-1 is not in the field of view 106 or if the user 110-2 is in the field of view 106, the computing device 102 can lock the user interface 105 so that the virtual meeting is frozen on the user interface 105. In some examples, the computing device 102 can lock the user interface 105 so that the user interface 105 is shut off.

[0034] Additionally, as illustrated in Figure 1 , the computing device 102 includes an audio output device 107. The audio output device 107 can be, for example, a speaker with which a user 110-1 can listen to audio output from the virtual meeting (e.g., listen to other attendees, listen to presented content, etc.). However, such audio may not be intended for listening for some users (e.g., user 110-1) and as such, if the user 110-1 is not in the field of view 106 or if the user 110- 2 is in the field of view 106, the computing device 102 can mute the audio output device 107 so that no audio is output from the audio output device 107.

[0035] Additionally, as illustrated in Figure 1 , the computing device 102 includes a microphone 108. As used herein, the term “microphone” refers to a device that converts sound into an electrical signal. While conducting a virtual meeting, the user 110-1 may audibly speak and the microphone 108 can convert such audible speech into an electrical signal for transmission to other attendees of the virtual meeting so that the other attendees can hear what the user 110-1 is saying. However, as some users (e.g., user 110-2) may not be intended to be viewed during the virtual meeting, they may also not be intended to be heard. Additionally, if the user 110-1 is not in the field of view 106 (e.g., and not in the frame), the user 110-1 may not intend for other noises to be picked up by the microphone 108. Accordingly, in some examples, the computing device 102 can perform an additional privacy action by muting the microphone 108.

[0036] As a privacy action is performed to the frame (or not performed), the computing device 102 can merge the frame into the video feed for viewing during the virtual meeting. For example, other attendees of the virtual meeting view the modified frame instead of the unmodified video feed, allowing for the privacy actions to prevent other attendees of the virtual meeting from viewing sensitive information within the field of view 106 of the camera 104 and/or users who are not intended to be viewed in the virtual meeting, as is further described in connection with Figures 2- 6.

[0037] The computing device 102 is described above as retrieving a frame from the video feed of the camera, detecting the face of a user 110, performing facial recognition on a detected face (if detected), and applying a privacy action to the frame. As the video feed comprises a plurality of frames, in some examples, the computing device 102 can perform the above steps on every frame in the video feed of the camera 104. However, in some examples, the computing device 102 can perform the above to every Nth frame, where N is a natural number. For example, the computing device 102 can perform the above steps every fifth frame, at a frequency greater than every fifth frame, or at a frequency less than every fifth frame, among other examples. Such a sampling frequency approach may be employed by the computing device 102 to save on computational resources of the computing device 102, in some examples.

[0038] Figure 2 illustrates an example of facial detection and recognition of a user in a frame 220 consistent with this disclosure. As illustrated in Figure 2, the frame 220 can include an area 222, a user 210, and the user’s face 214.

[0039] As previously described in connection with Figure 1 , the camera (e.g., camera 104, previously described in connection with Figure 1) can capture within a field of view (e.g., field of view 106, previously described in connection with Figure 1) a user 210 having a face 214, as well as a physical area 222. The physical area 222 can include the space in which the user 210 is utilizing the camera to, for example, attend a virtual meeting via a computing device (e.g., computing device 102, previously described in connection with Figure 1).

[0040] Once the computing device retrieves the frame 220 from the video feed of the camera, the computing device can perform, at step 230-1 , facial detection of a face 214 of the user 210 in the frame 220. For example, utilizing facial detection techniques, the computing device can detect that a face 214 of the user 210 is in the frame 220. The computing device can determine coordinates of the face 214 of the user 210 in the frame 220.

[0041] In response to the face 214 being detected, the computing device can perform, at step 230-2, facial recognition on the user 210 on the determined coordinates of the face 214 to determine an identity of the user 210. The computing device can compare a scan of the face 214 of the user 210 with a facial encoding, as previously described in connection with Figure 1.

[0042] During facial recognition, a bounding area 226 can be generated around the face 214 of the user 210. As used herein, the term “bounding area” refers to a defined space that serves as a point of reference for objection detection. For example, the bounding area 226 can be an area that defines the coordinates on the frame 220 in which the face 214 of the user 210 is located. The coordinates can be, for example, coordinates determined during facial detection of the face 214 of the user 210. The bounding area 226 can define the area on the frame 220 that is compared against the facial encoding. That is, the bounding area 226 can correspond to the area on the frame 220 that is the face 214 of the user 210 and can be compared against the facial encoding to determine an identity of the user 210.

[0043] If the scan of the face 214 of the user 210 matches the facial encoding, the computing device can determine the user identity of the user 210 corresponds to a user that is to be viewed in the virtual meeting. The computing device can apply a privacy action to the frame 220 by applying the virtual background 224 to the frame 220. While the user 210 is shown in the frame 220, the virtual background 224 can obscure sensitive information that may be located in the area 222 located around the user 210 (e.g., printouts, whiteboards, other screens, etc.). The frame 220 having the privacy action can be merged back into the video feed from the camera so that the user 210 is seen by other attendees of the virtual meeting, but the area 222 is obscured by the virtual background 224. The computing device can repeat this process for subsequent frames of the video feed.

[0044] Figure 3 illustrates an example of application of a privacy action to a frame 320 without a user consistent with this disclosure. As illustrated in Figure 3, the frame 320 can include an area 322.

[0045] A camera (e.g., camera 104, previously described in connection with Figure 1) can capture within a field of view (e.g., field of view 106, previously described in connection with Figure 1) a physical area 322. The physical area 322 can include the space in which a user is utilizing the camera to, for example, attend a virtual meeting via a computing device (e.g., computing device 102, previously described in connection with Figure 1).

[0046] In some examples, the user may step away from the field of view. In such an instance, the user may no longer be within the field of view of the camera. The computing device can perform a privacy action, as is further described herein.

[0047] Once the computing device retrieves the frame 320 from the video feed of the camera, the computing device can perform, at step 330-1 , facial detection to detect whether a face of a user is present in the frame 320. As illustrated in Figure 3, there is no user include in the frame 320. Accordingly, the computing device can determine there is no face detected in the frame 320 at step 330-1.

[0048] In some examples, the computing device can perform a debounce action in response to the computing device not detecting a face of a user in the frame 320. As used herein, the term “debounce action” refers to a process in which a device waits a certain amount of time before performing an action. The debounce action can, accordingly, include waiting a predetermined amount of time before taking a privacy action, determining a similarity to a previous frame, determining a similarity of a bounding area to a previous frame (e.g., as is further described in connection with Figures 4 and 5), etc. Performing a debounce action can prevent a privacy action from being taken when a user is briefly out of the field of view of the camera, has their face obscured by an object for a short amount of time (e.g., by a coffee mug while taking a sip of coffee, while wearing a mask, etc.), etc.

[0049] For example, the computing device can perform a debounce action by waiting a predetermined amount of time before applying the privacy action to the frame 320. The predetermined amount of time can, in some instances, correspond to an amount of frames within the amount of time. For example, the computing device may wait two frames (which may be less than one second), and then again perform facial detection to detect whether a face of a user is present in the frame 320.

[0050] In response to the computing device still not detecting the face of the user in the frame 320 (e.g., following a debounce action), the computing device can apply the privacy action to the frame 320 in response to the face of the user not being detected in the frame 320. For example, at step 330-2, the computing device can apply the privacy action by applying the virtual background 324 to the frame 320. The frame 320 having the virtual background 324 covering the entire frame 320 can prevent other attendees of the virtual meeting from viewing the area 322 within the field of view of the camera. The computing device can repeat this process for subsequent frames (e.g., until a user’s face is again detected within the field of view of the camera).

[0051] In some examples, the computing device can apply a privacy action by cutting off the video feed. In such an approach, no video feed is transmitted to the other attendees of the virtual meeting.

[0052] In some examples, the computing device can apply a privacy action by generating a lock signal to lock a user interface of the computing device, mute an audio output device, mute a microphone of the computing device, among other examples in addition to and/or alternatively to applying the virtual background 324. Locking the user interface can prevent unintentional viewing of the virtual meeting, muting an audio output device can prevent unintentional broadcast of the virtual meeting, and muting the microphone can prevent unintended noises from being transmitted to other attendees of the virtual meeting while no user is detected within the field of view of the camera.

[0053] Figure 4 illustrates an example of application of a privacy action to a frame with a user consistent with this disclosure. As illustrated in Figure 4, the frame 420 can include an area 422, a user 410, and the user’s face 414.

[0054] A camera (e.g., camera 104, previously described in connection with Figure 1) can capture within a field of view (e.g., field of view 106, previously described in connection with Figure 1) a user 410 having a face 414, as well as a physical area 422. The physical area 422 can include the space in which the user 410 is utilizing the camera to, for example, attend a virtual meeting via a computing device (e.g., computing device 102, previously described in connection with Figure 1). However, in some instances, the user 410 may not be a user that is intended to be viewed during the virtual meeting. For example, while the user who is intended to be viewed during the virtual meeting steps away from the field of view of the camera (e.g., to take a break, use the restroom, get a snack, etc.), the viewer 410 may step into the field of view of the camera (e.g., intentionally or unintentionally). The computing device can take a privacy action accordingly, as is further described herein.

[0055] Once the computing device retrieves the frame 420 from the video feed of the camera, the computing device can perform, at step 430-1 , facial detection of a face 414 of the user 410 in the frame 420. For example, utilizing facial detection techniques, the computing device can detect that a face 414 of the user 410 is in the frame 420. The computing device can determine coordinates of the face 414 of the user 410 in the frame 420.

[0056] In response to the face 414 being detected, the computing device can perform, at step 430-2, facial recognition on the determined coordinates of the face 414 of the user 410 to determine an identity of the user 410. The computing device can compare a scan of the face 414 of the user 410 with a facial encoding, as previously described in connection with Figure 1.

[0057] During facial recognition, a bounding area 426 can be generated around the face 414 of the user 410. For example, the bounding area 426 can be an area that defines the coordinates on the frame 420 in which the face 414 of the user 410 is located. The coordinates can be, for example, coordinates determined during the facial detection of the face 414 of the user 410. The bounding area 426 can define the area on the frame 420 that is compared against the facial encoding. That is, the bounding area 426 can correspond to the area on the frame 420 that is the face 414 of the user 410 and can be compared against the facial encoding to determine an identity of the user 410.

[0058] If the scan of the face 414 of the user 410 matches the facial encoding, the computing device can determine the user identity of the user 410 corresponds to a user that is to be viewed in the virtual meeting (e.g., as previously described in connection with Figure 2). However, as mentioned above, the user 410 in Figure 4 may not be a user that is intended to be viewed during the virtual meeting.

[0059] Accordingly, in response to the scan of the face 414 of the user 410 not matching the facial encoding, the computing device can perform a debounce action. In an example in which the user 410 is a user intended to be viewed during the virtual meeting, but the user’s face is briefly obscured by an object for a short amount of time, a privacy action does not have to be applied. The debounce action can prevent such a privacy action from being applied.

[0060] In some examples, the computing device can perform a debounce action by waiting a predetermined amount of time before applying the privacy action to the frame 420. The predetermined amount of time can, in some instances, correspond to an amount of frames within the amount of time. For example, the computing device may wait two frames (which may be less than one second), and then again determine whether the face 414 of the user 410 matches the facial encoding.

[0061] In some examples, the computing device can perform a debounce action by comparing the bounding area 426 to a bounding area in a previous frame. For example, the computing device can compare the bounding area 426 to a bounding area in a previous frame to determine a similarity between the frames. In response to the similarity of the bounding area 426 exceeding a threshold similarity, the computing device can wait to perform a privacy action. In response to the similarity of the bounding area 426 not exceeding the threshold similarity, the computing device can perform the privacy action. For example, the computing device can compare the bounding area 426 to a bounding area of a previous frame and determine the bounding area 426 is 90% similar to the previous frame, which exceeds a threshold similarity of 85%. The computing device can accordingly wait to perform the privacy action. However, in an example in which the computing device determines the bounding area 426 is 60% similar to the previous bounding area (which does not exceed the threshold similarity of 85%), the computing device can perform the privacy action.

[0062] Following the debounce action, the computing device can again perform facial recognition on the frame 420. For example, the face 414 of the user 410 can be compared against the facial encoding to determine an identity of the user 410.

[0063] In response to the scan of the face 414 of the user 410 again not matching the facial encoding, the computing device can expand, at step 430-3, the bounding area 426 from the face 414 of the user 410 to include other portions of a body of the user 410. As illustrated in Figure 4, the expanded bounding area 428 can comprise the face 414 of the user 410, as well as other portions of the user 410 that may be included in the field of view of the camera (e.g., shoulders, parts of the arms, part of the torso, etc.).

[0064] Since the identity of the user 410 still does not match the facial encoding corresponding to the user, the computing device can apply a privacy action at step 430-4. For example, at step 430-4, the computing device can apply the privacy action by applying the virtual background 424 to the frame 420. The frame 420 having the privacy action can be merged back into the video feed from the camera so that the user 410 is not seen by other attendees of the virtual meeting. The computing device can repeat this process for subsequent frames of the video feed. The frame 420 having the virtual background 424 covering the entire frame 420 can prevent other attendees of the virtual meeting from viewing the user 410 who is not intended to be viewed in the virtual meeting, even though the user 410 is within the field of view of the camera. The computing device can repeat this process for subsequent frames (e.g., until a user’s face that is of a user who is to be viewed in the virtual meeting is again detected within the field of view of the camera).

[0065] In some examples, the computing device can apply a privacy action by cutting off the video feed. In such an approach, no video feed is transmitted to the other attendees of the virtual meeting.

[0066] In some examples, the computing device can apply a privacy action by generating a lock signal to lock a user interface of the computing device, mute an audio output device, mute a microphone of the computing device, among other examples in addition to and/or alternatively to applying the virtual background 424. Locking the user interface can prevent unintentional viewing of the virtual meeting, muting an audio output device can prevent unintentional broadcast of the virtual meeting, and muting the microphone can prevent unintended noises from being transmitted to other attendees of the virtual meeting while no user is detected within the field of view of the camera.

[0067] Figure 5 illustrates an example of application of a privacy action to a frame with a first user and a second user consistent with this disclosure. As illustrated in Figure 5, the frame 520 can include an area 522, a first user 510-1 , the first user’s face 514-1, a second user 510-2, and the second user’s face 514-2.

[0068] A camera (e.g., camera 104, previously described in connection with Figure 1) can capture within a field of view (e.g., field of view 106, previously described in connection with Figure 1) a first user 510-1 having a face 514-1 and a second user 510-2 having a face 514-2, as well as a physical area 522. The physical area 522 can include the space in which the user 510-1 is utilizing the camera to, for example, attend a virtual meeting via a computing device (e.g., computing device 102, previously described in connection with Figure 1). However, while the first user 510-1 is attending the virtual meeting, the second user 510-2 (e.g., who is not a user that is intended to be viewed during the virtual meeting) may enter the field of view of the camera. The computing device can take a privacy action accordingly, as is further described herein.

[0069] Once the computing device retrieves the frame 520 from the video feed of the camera, the computing device can perform, at step 530-1 , facial detection of faces 514-1 and 514-2 of the users 510-1 and 510-2, respectively, in the frame 520. For example, utilizing facial detection techniques, the computing device can detect that faces 514-1 and 514-2 of the users 510-1 , 510-2 are in the frame 520. The computing device can determine coordinates of the face 514-1 and the face 514-2 of the users 510-1, 510-2, respectively, in the frame 520.

[0070] In response to the faces 514-1 and 514-2 being detected, the computing device can perform, at step 530-2, facial recognition on the users 510-1 and 510-2 on the determined coordinates of the faces 514-1 and 514-2 to determine identities of the users 510-1 and 510-2. The computing device can compare a scan of the face 514-1 of the user 510-1 and a scan of the face 514-2 of the user 510-2 with a facial encoding, as previously described in connection with Figure 1. [0071] During facial recognition, a bounding area 526-1 can be generated around the face 514-1 of the user 510-1 and a bounding area 526-2 can be generated around the face 514-2 of the user 510-2. For example, the bounding areas 526-1 and 526-2 can be areas that define the coordinates on the frame 520 in which the faces 514-1 , 514-2 of the users 510-1 , 510-2 are located. The coordinates can be, for example, coordinates determined during facial detection of the faces 514- 1 and 514-2 of the users 510-1 and 510-2. The bounding areas 526 can define the areas on the frame 520 that are compared against the facial encoding. That is, the bounding areas 526-1 , 526-2 can correspond to the areas on the frame 520 that are the faces 514-1 , 514-2 of the users 510-1 , 510-2 and can be compared against the facial encoding to determine identities of the users 510.

[0072] The scan of the face 514-1 of the user 510-1 can be determined to match the facial encoding, as the computing device can determine the user identity of the user 510-1 corresponds to a user that is to be viewed in the virtual meeting. However, as mentioned above, the user 510-2 is not a user that is intended to be viewed during the virtual meeting. Accordingly, the scan of the face 514-2 of the user 510-2 can be determined to not match the facial encoding.

[0073] In response to the scan of the face 514-2 of the user 510-2 not matching the facial encoding, the computing device can perform a debounce action. In an example in which the user 510-2 is a user intended to be viewed during the virtual meeting, but the user’s face is briefly obscured by an object for a short amount of time, a privacy action does not have to be applied. The debounce action can prevent such a privacy action from being applied.

[0074] In some examples, the computing device can perform a debounce action by waiting a predetermined amount of time before applying the privacy action to the frame 520. The predetermined amount of time can, in some instances, correspond to an amount of frames within the amount of time. For example, the computing device may wait two frames (which may be less than one second), and then again determine whether the face 514-2 of the user 510-2 matches the facial encoding.

[0075] In some examples, the computing device can perform a debounce action by comparing the frame 520 to a previous frame. For example, the computing device can compare the frame 520 to a previous frame to determine a similarity between the frames. For example, the computing device can compare the frame 520 to a previous frame and determine the frame 520 is 90% similar to the previous frame, which exceeds a threshold similarity of 85%. The computing device can wait to perform the privacy action. However, in an example in which the computing device determines the frame 520 is 60% similar to the previous frame (which does not exceed the threshold similarity of 85%), the computing device can perform the privacy action.

[0076] In some examples, the computing device can perform a debounce action by comparing the bounding area 526-2 to a bounding area in a previous frame. For example, the computing device can compare the bounding area 526-2 to a bounding area of a previous frame and determine the bounding area 526-2 is 90% similar to the previous frame, which exceeds a threshold similarity of 85%. The computing device can accordingly wait to perform the privacy action. However, in an example in which the computing device determines the bounding area 526-2 is 60% similar to the previous bounding area (which does not exceed the threshold similarity of 85%), the computing device can perform the privacy action.

[0077] Following the debounce action, the computing device can again perform facial recognition on the frame 520. For example, the face 514-2 of the user 510-2 can be compared against the facial encoding to determine an identity of the user 510-2.

[0078] In response to the scan of the face 514-2 of the user 510-2 again not matching the facial encoding, the computing device can expand, at step 530-3, the bounding area 526-1 , 526-2 from the faces 514-1 , 514-2 of the users 510-1 , 510-2 to include other portions of a body of the users 510-1 , 510-2. As illustrated in Figure 5, the expanded bounding areas 528-1 , 528-2 can comprise the faces 514-1 , 514-2 of the users 510-1 , 510-2, as well as other portions of the users 510-1 , 510-2 that may be included in the field of view of the camera (e.g., shoulders, parts of the arms, part of the torso, etc.)

[0079] Since the identity of the user 510-2 does not match the facial encoding, the computing device can apply a privacy action at step 530-4. For example, the computing device can apply the privacy action by occluding the expanded bounding area 528-2 from the frame 520. The privacy action can be a virtual background 524. That is, the computing device can apply the virtual background 524 to the frame 520. While the user 510-1 is shown in the frame 520, the user 510-2 is occluded from the frame 520 (e.g., is not shown). Further, the virtual background 524 can obscure sensitive information that may be located in the area 522 located around the user 510-1 (e.g., printouts, whiteboards, other screens, etc.). The frame 520 having the occluded user 510-2 can be merged back into the video feed from the camera so that the user 510-1 is seen by other attendees of the virtual meeting, but the user 510-2 is not seen by other attendees of the virtual meeting and the area 522 is obscured by the virtual background 524. The computing device can repeat this process for subsequent frames of the video feed.

[0080] Applying privacy actions according to the disclosure can allow for application of a virtual background to obscure the field of view of a camera based on detected faces in a frame as well as facial recognition techniques. Such an approach can obscure the field of view of the camera, as well as occlude persons not intended to be broadcast, and can provide additional and more thorough privacy to users and/or other unintended persons during a virtual meeting as compared with previous approaches.

[0081] Figure 6 illustrates an example of a computing device 602 for applying privacy actions consistent with this disclosure. As described herein, the computing device 602 may perform functions related to applying privacy actions. Although not illustrated in Figure 6, the computing device 602 may include a processor and a machine-readable storage medium. Although the following descriptions refer to a single processor and a single machine-readable storage medium, the descriptions may also apply to a system with multiple processors and multiple machine-readable storage mediums. In such examples, the computing device 602 may be distributed across multiple machine-readable storage mediums and across multiple processors. Put another way, the instructions executed by the computing device 602 may be stored across multiple machine-readable storage media and executed across multiple processors, such as in a distributed or virtual computing environment.

[0082] Processing resource 640 may be a central processing unit (CPU), a semiconductor-based microprocessor, and/or other hardware devices suitable for retrieval and execution of machine-readable instructions 644, 646, 648 stored in a memory resource 642. Processing resource 640 may fetch, decode, and execute instructions 644, 646, 648. As an alternative or in addition to retrieving and executing instructions 644, 646, 648, processing resource 640 may include a plurality of electronic circuits that include electronic components for performing the functionality of instructions 644, 646, 648. [0083] Memory resource 642 may be any electronic, magnetic, optical, or other physical storage device that stores executable instructions 644, 646, 648, and/or data. Thus, memory resource 328 may be, for example, Random Access Memory (RAM), an Electrically-Erasable Programmable Read-Only Memory (EEPROM), a storage drive, an optical disc, and the like. Memory resource 642 may be disposed within computing device 602, as shown in Figure 6.

[0084] The computing device 602 may include instructions 644 stored in the memory resource 642 and executable by the processing resource 640 to perform facial detection of a user in a frame of a video feed. The computing device 602 may utilize various approaches to facial detection to perform facial detection of the user in the frame.

[0085] The computing device 602 may include instructions 646 stored in the memory resource 642 and executable by the processing resource 640 to perform facial recognition on the face in the frame to determine an identity of the user. The computing device 602 may compare a scan of the users face to a facial encoding to determine the identity of the user.

[0086] The computing device 602 may include instructions 648 stored in the memory resource 642 and executable by the processing resource 640 to apply a privacy action to the frame based on the detected face and the facial recognition. The privacy action can include applying a virtual background to the frame, occluding a user from the frame, muting a microphone of the computing device 602, among other privacy actions.

[0087] Figure 7 illustrates a block diagram of an example system 750 for applying privacy actions consistent with this disclosure. In the example of Figure 7, system 750 includes a computing device 702 including a processing resource 752 and a non-transitory machine-readable storage medium 754. Although the following descriptions refer to a single processing resource and a single machine-readable storage medium, the descriptions may also apply to a system with multiple processors and multiple machine-readable storage mediums. In such examples, the instructions may be distributed across multiple machine-readable storage mediums and the instructions may be distributed across multiple processors. Put another way, the instructions may be stored across multiple machine-readable storage mediums and executed across multiple processors, such as in a distributed computing environment. [0088] Processing resource 752 may be a central processing unit (CPU), microprocessor, collection of multiple CPU’s, and/or other hardware device suitable for retrieval and execution of instructions stored in the non-transitory machine- readable storage medium 754. In the particular example shown in Figure 7, processing resource 752 may receive, determine, and send instructions 756, 758, 760, 762, 764. As an alternative or in addition to retrieving and executing instructions, processing resource 752 may include an electronic circuit comprising a number of electronic components for performing the operations of the instructions in the non-transitory machine-readable storage medium 754. With respect to the executable instruction representations or boxes described and shown herein, it should be understood that part or all of the executable instructions and/or electronic circuits included within one box may be included in a different box shown in the figures or in a different box not shown.

[0089] The non-transitory machine-readable storage medium 754 may be any electronic, magnetic, optical, or other physical storage device that stores executable instructions. Thus, the non-transitory machine-readable storage medium 754 may be, for example, Random Access Memory (RAM), an Electrically-Erasable Programmable Read-Only Memory (EEPROM), a storage drive, an optical disc, and the like. The executable instructions may be “installed” on the system 750 illustrated in Figure 7. The non-transitory machine-readable storage medium 754 may be a portable, external or remote storage medium, for example, that allows the system 750 to download the instructions from the portable/external/remote storage medium. In this situation, the executable instructions may be part of an “installation package”. [0090] Retrieve a frame 756, when executed by a processor such as processing resource 752, may cause system 750 to retrieve a frame from a video feed of a camera.

[0091] Determine whether a face is in the frame 758, when executed by a processor such as processing resource 752, may cause system 750 to perform facial detection of a user in the frame.

[0092] Perform facial recognition 760, when executed by a processor such as processing resource 752, may cause system 750 to perform, in response to the face being detected in the frame, facial recognition on the face in the frame to determine an identity of the user. [0093] Apply a privacy action 762, when executed by a processor such as processing resource 752, may cause system 750 to apply a privacy action to the frame based on the detected face and the facial recognition.

[0094] Merge the frame 764, when executed by a processor such as processing resource 752, may cause system 750 to merge the frame into the video feed for viewing during a virtual meeting.

[0095] Figure 8 illustrates an example of a method 866 for applying privacy actions consistent with this disclosure. The method 866 can be performed by various computing devices (e.g., computing devices 102, 702, previously described in connection with Figures 1 and 7).

[0096] At 868, the method 866 includes retrieving, by a computing device, a frame from a video feed of a camera.

[0097] At 870, the method 866 includes performing, by the computing device, facial detection of a user in the frame.

[0098] At 872, the method 866 includes performing, by the computing device in response to the face being detected in the frame, facial recognition on the frame to determine an identity of the user by comparing a scan of the face in the frame with a facial encoding.

[0099] At 874, the method 866 includes applying, by the computing device, a privacy action to the frame based on the detected face and the facial recognition.

[00100] At 876, the method 866 includes merging, by the computing device, the frame having the privacy action into the video feed for viewing during a virtual meeting.

[00101] In the foregoing detailed description of the disclosure, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration how examples of the disclosure may be practiced. These examples are described in sufficient detail to enable those of ordinary skill in the art to practice the examples of this disclosure, and it is to be understood that other examples may be utilized and that process, electrical, and/or structural changes may be made without departing from the scope of the disclosure. Further, as used herein, “a” can refer to one such thing or more than one such thing.

[00102] The figures herein follow a numbering convention in which the first digit corresponds to the drawing figure number and the remaining digits identify an element or component in the drawing. For example, reference numeral 100 may refer to element 102 in Figure 1 and an analogous element may be identified by reference numeral 302 in Figure 3. Elements shown in the various figures herein can be added, exchanged, and/or eliminated to provide additional examples of the disclosure. In addition, the proportion and the relative scale of the elements provided in the figures are intended to illustrate the examples of the disclosure, and should not be taken in a limiting sense.

[00103] It can be understood that when an element is referred to as being "on," "connected to", “coupled to”, or "coupled with" another element, it can be directly on, connected, or coupled with the other element or intervening elements may be present. In contrast, when an object is “directly coupled to” or “directly coupled with” another element it is understood that are no intervening elements (adhesives, screws, other elements) etc.

[00104] The above specification, examples and data provide a description of the method and applications, and use of the system and method of the disclosure. Since many examples can be made without departing from the spirit and scope of the system and method of the disclosure, this specification merely sets forth some of the many possible example configurations and implementations.

Claims

23 What is claimed is:

1. A computing device, comprising: a processing resource; and a memory resource storing non-transitory machine-readable instructions to cause the processing resource to: perform facial detection of a user in a frame of a video feed; perform facial recognition on a face of the user in the frame to determine an identity of the user; and apply a privacy action to the frame based on the facial detection and the facial recognition.

2. The computing device of claim 1 , including instructions to cause the processing resource to apply the privacy action in response to the identity of the user not matching a facial encoding corresponding to the user.

3. The computing device of claim 1 , including instructions to cause the processing resource to apply the privacy action in response to the face of the user not being detected in the frame.

4. The computing device of claim 1 , including instructions to cause the processing resource to retrieve the frame from the video feed of a camera.

5. The computing device of claim 1 , including instructions to cause the processing resource to apply the privacy action by applying a virtual background to the frame and merging the frame into the video feed.

6. The computing device of claim 1 , wherein: the user is a second user; and the memory resource includes instructions to cause the processing resource to: apply the privacy action by occluding the second user while a first user is still visible in the frame; and merge the frame into the video feed so that the first user is viewable in the frame in the video feed and the second user is not viewable in the frame in the video feed.

7. The computing device of claim 1 , including instructions to cause the processing resource to apply the privacy action by muting a microphone of the computing device.

8. A non-transitory machine-readable storage medium including instructions that when executed cause a processor of a computing device to: retrieve a frame from a video feed of a camera; perform facial detection of a user in the frame; perform, in response to the facial detection, facial recognition on a face of the user in the frame to determine an identity of the user; apply a privacy action to the frame based on the facial detection and the facial recognition; and merge the frame into the video feed for viewing during a virtual meeting.

9. The medium of claim 8, including instructions to cause the processor to: generate a bounding area around the face in the frame; and expand the bounding area from the face of the user to further include other portions of a body of the user.

10. The medium of claim 9, wherein in response to the identity of the user not matching a facial encoding corresponding to the user, the processor is to: occlude the expanded bounding area of the frame; and merge the frame including the occluded expanded bounding area into the video feed such that the expanded bounding area of the frame is not visible for the duration of the frame in the video feed during the virtual meeting.

11. A method, comprising: retrieving, by a computing device, a frame from a video feed of a camera; performing, by the computing device, facial detection of a user in the frame; performing, by the computing device in response to the facial detection, facial recognition on the frame to determine an identity of the user by comparing a scan of the face in the frame with a facial encoding; applying, by the computing device, a privacy action to the frame based on the facial detection and the facial recognition; and merging, by the computing device, the frame having the privacy action into the video feed for viewing during a virtual meeting.

12. The method of claim 11 , wherein the method includes: performing, by the computing device in response to the scan of the face in the frame not matching the facial encoding, a debounce action; and performing, by the computing device, the facial recognition on the frame following the debounce action.

13. The method of claim 11 , wherein the method includes: in response to the scan of the face in the frame not matching the facial encoding, expand the bounding area from the face of the user to further include other portions of a body of the user; and occluding the expanded bounding area from the frame.

14. The method of claim 13, wherein the method includes comparing the scan of the face in the frame with the facial encoding saved externally on an external computing device.

15. The method of claim 12, wherein the method includes generating the facial encoding during a calibration recording prior to the virtual meeting.