WO2017147892A1

WO2017147892A1 - OPERATING INTERNET OF THINGS DEVICES USING LiDAR METHOD AND APPARATUS

Info

Publication number: WO2017147892A1
Application number: PCT/CN2016/075588
Authority: WO
Inventors: Zhen Zhou; Ke HAN; Valluri R. Rao; Jonathan Eng
Original assignee: Intel Corp
Current assignee: Intel Corp
Priority date: 2016-03-04
Filing date: 2016-03-04
Publication date: 2017-09-08
Anticipated expiration: 2018-09-04
Also published as: DE112016006547T5

Abstract

Techniques and configurations for an apparatus(100) for controlling IoT devices(110, 112, 114) are provided. In one instance, an apparatus (100) may include a LiDAR scanner (102) having a laser (104) to scan (106) an environment(108) to obtain scan data. Scanning may include illuminating a point of the environment (108) and measuring a distance to that point based on a difference between a time of transmission of a beam to illuminate the point and a time of receipt of a beam reflected by the point. The apparatus (100) may further include a controller (130) to collect and analyze the scan data, detect a presence of a user (120, 122, 124) in the environment (108), detect and identify a gesture provided by the user (120, 122, 124 ), and provide a command to one or more IoT devices (110, 112, 114) that are responsive to commands provided by user gestures, based in part on the identified gesture.

Description

OPERATING INTERNET OF THINGS DEVICES USING LiDAR METHOD AND APPARATUS

Field

Embodiments of the present disclosure generally relate to the field of device control, and more particularly, to controlling Internet of Things devices with gestures.

Background

The nascent concept of Internet of Things (IoT) allows objects to be sensed and controlled remotely across existing network infrastructure, thus creating opportunities for direct integration between the physical world and computer-based systems, and resulting in improved efficiency, accuracy and economic benefit. The existing smart home IoT control solutions may include, but are not limited to, voice control, remote control, or gesture-based control.

However, the above options may have substantial shortcomings. For example, voice control may be impacted seriously by noisy environment, e.g., involving any kind of entertainment with sound, such as TV, a game, or music. Remote control may involve a use of a dedicated remote control device, such as a controller or a smartphone. Gesture control may involve a use of a camera to capture user gestures. Using a camera may be negatively impacted by the quality of ambient light and may cause privacy concerns.

Brief Description of the Drawings

Embodiments will be readily understood by the following detailed description in conjunction with the accompanying drawings. To facilitate this description, like reference numerals designate like structural elements. Embodiments are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings.

FIG. 1 is a block diagram illustrating an example apparatus 100 for controlling IoT devices, incorporated with the teachings of the present disclosure, in accordance with some embodiments.

FIG. 2 is a diagram illustrating some aspects of operation of an example apparatus for controlling IoT devices, in accordance with some embodiments.

FIG. 3 is an example process flow diagram for detection of a user in an environment, in accordance with some embodiments.

FIG. 4 is an example graph illustrating user detection, in accordance with some embodiments.

FIG. 5 is an example process flow diagram for user selection among a group of users in an environment, in accordance with some embodiments.

FIG. 6 is an example process flow diagram for IoT device selection among a group of IoT devices, in accordance with some embodiments.

FIG. 7 is a diagram illustrating some examples of gesture detection and identification using an example apparatus for controlling IoT devices, in accordance with some embodiments.

FIG. 8 is a diagram illustrating a radar image output of an example apparatus for controlling IoT devices in response to a scan of an environment in which the user may perform different gestures, in accordance with some embodiments.

FIG. 9 is an example process flow diagram for gesture detection and identification, in accordance with some embodiments.

FIG. 10 is an example process flow diagram for adding a location of a new IoT device to a description of IoT devices operated by the apparatus for controlling IoT devices, in accordance with some embodiments.

Detailed Description

Embodiments of the present disclosure include techniques and configurations for a method and apparatus for controlling IoT devices. In some embodiments, an apparatus may include at least one Light Detection and Ranging Device (LiDAR) scanner having a laser to scan an environment to obtain scan data. Scanning with LiDAR may include illuminating a point of the environment and measuring a distance to the illuminated point based on a difference between a time of transmission of a first light beam to illuminate the point and a time of receipt of a second light beam reflected by the illuminated point. The apparatus may further include a controller coupled with the at least one LiDAR scanner, to collect and analyze the scan data, detect a presence of a user in the environment, detect and identify a gesture provided by the user, and provide a command to one or more IoT devices that are responsive to commands provided by user gestures, based at least in part on the identified gesture.

In the following detailed description, reference is made to the accompanying drawings that form a part hereof, wherein like numerals designate like parts throughout, and in which are shown by way of illustration embodiments in which the subject matter of the present disclosure may be practiced. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present disclosure. Therefore, the following detailed description is not to be taken in a limiting sense, and the scope of embodiments is defined by the appended claims and their equivalents.

For the purposes of the present disclosure, the phrase “A and/or B” means (A) , (B) , or (A and B) . For the purposes of the present disclosure, the phrase “A, B, and/or C” means (A) , (B) , (C) , (A and B) , (A and C) , (B and C) , or (A, B, and C) .

The description may use perspective-based descriptions such as top/bottom, in/out, over/under, and the like. Such descriptions are merely used to facilitate the discussion and are not intended to restrict the application of embodiments described herein to any particular orientation.

The description may use the phrases “in an embodiment” or “in embodiments, ” which may each refer to one or more of the same or different embodiments. Furthermore, the terms “comprising, ” “including, ” “having, ” and the like, as used with respect to embodiments of the present disclosure, are synonymous.

The term “coupled with, ” along with its derivatives, may be used herein. “Coupled” may mean one or more of the following. “Coupled” may mean that two or more elements are in direct physical, electrical, or optical contact. However, “coupled” may also mean that two or more elements indirectly contact each other, but yet still cooperate or interact with each other, and may mean that one or more other elements are coupled or connected between the elements that are said to be coupled with each other. The term “directly coupled” may mean that two or more elements are in direct or indirect contact.

FIG. 1 is a block diagram illustrating an example apparatus 100 for controlling IoT devices, incorporated with the teachings of the present disclosure, in accordance with some embodiments. The apparatus 100 may be disposed, for example, in an environment 108, indicated by a dashed line. The environment 108 may comprise a part of a smart home environment, and include at least partially enclosed three-dimensional (3D) space, such as, for example, a room in a house, a hallway, a corridor, a kitchen, or any other enclosed facility. As shown, the environment 108 may include some, but not necessarily all, of the plurality of

IoT devices

110, 112, 114. The example environment 108 is shown as including

IoT devices

110 and 112, and not including IoT device 114, which may be placed outside of the environment 108, e.g., in adjacent room or outside the house. In some embodiments, all

IoT devices

110, 112, 114 may be located inside or outside the environment 108. The

IoT devices

110, 112, 114 may be configured to be responsive to commands provided by gestures of one or more users present in the environment 108, such as

users

120, 122, 124. A number of IoT devices in and outside the environment 108 and a number of users in the environment 108 may vary； the

devices

110, 112, 114 and

users

120, 122, 124 are shown in FIG. 1 for ease of understanding.

In embodiments, the apparatus 100 may include at least one Light Detection and Ranging Device (LiDAR) scanner 102 having a laser 104 to scan 106 the environment 108 to obtain scan data. For example, the LiDAR scanner 102 may measure a distance to an object (e.g., a point in an environment) by illuminating the object with the laser 104 and analyzing the reflected light. More specifically, the LiDAR scanner 102 may illuminate one point of the environment 108 at a time and measure a distance to the illuminated point based on a difference between a time of transmission of a light beam to illuminate the point and a time of receipt of a light beam reflected by the illuminated point. In some embodiments, the apparatus 100 may include two or more LiDAR scanners, as illustrated by LiDAR scanner 152, which is shown in FIG. 1 in a dashed line. In some embodiments, the LiDAR scanner 102 (and/or 152) may scan the environment 108 continuously and provide the scan data to the apparatus 100 in near-real time or real time.

The apparatus 100 may further include a controller 130 coupled with the LiDAR scanner 102, and configured to collect and analyze the scan data provided by the LiDAR scanner 102 (and, in some embodiments, by the LiDAR scanner 152) .

The controller 130 may include a human detection block 132 configured to detect a presence of a user (or users) 120, 122, 124 in the environment 108, based on the analysis of the scan data.

The controller 130 may further include a gesture detection and identification block 134 coupled with the human detection block 132 and configured to detect and identify a user among

users

120, 122, 124 present in the environment, who may select an IoT device and provide a command to the selected IoT device. The gesture detection and identification block 134 may be further configured to detect and identify a gesture provided by the identified user (or all users present in the environment 108) , based on the analysis of the scan data. The gesture detection and identification block 134 may be further configured to select the one or more IoT devices (e.g., IoT device 110) from a plurality of

IoT devices

110, 112, 114, to which the command may be addressed, based on further analysis of the scan data, for example, based on the identified user gesture. The gesture detection and identification block 134 may be further configured to determine a command to be provided to the selected IoT device that may correspond to the identified gesture, based at least in part on the identified gesture.

In some embodiments, the controller 130 may be configured to collect and analyze the scan data provided by the LiDAR scanner 102 and by the LiDAR scanner 152. In such embodiments, the controller 130 may include fusion logic block 148, shown in dashed lines in FIG. 1. The fusion logic block 148 may be configured to compare the scan data provided by each of the two (or more) LiDAR scanners and analyzed in

blocks

132 and 134, and detect the presence of the user in the environment and detect and identify the gesture provided by the user, based on a result of the comparison. For example, the fusion logic block may determine that both LiDAR scanners reported the same user position and the same gesture (within a predefined margin of error) , and accept the position and gesture as high confidence input. If the LiDAR scanners report different results regarding the user position and/or gesture, the fusion logic block 148 may select a result that has higher accuracy characteristics. To identify which results should be selected in this case, the fusion logic block 148 may choose the one with the best body direction (the line from shoulder to shoulder may be perpendicular to the line from LiDAR to body) .

In some embodiments, the controller 130 may include a processor 142 configured to operate

blocks

132, 134. The controller 130 may further include memory 144 having instructions that, when executed on the processor 142, may cause the controller 130 to perform operations described above in reference to

blocks

132, 134. The processor 142 may be implemented as having multi-cores, e.g., a multi-core microprocessor. Memory 144 may be temporal and/or persistent storage of any type, including, but not limited to, volatile and non-volatile memory, optical, magnetic, and/or solid state mass storage, and so forth. In some embodiments, memory 144 may store a library 151 of gestures and corresponding commands to control IoT devices, as described below in greater detail. In some embodiments, the library 151 may be accessible by the controller 130 and may be disposed at a different location, e.g., in a cloud. In some embodiments, memory 144 may store a list 154 of IoT devices configured to be controlled by the apparatus 100, as described below in greater detail. In some embodiments, the list 154 may be accessible by the controller 130 and may be disposed at a different location, e.g., in a cloud.

The controller 130 may also include a wireless communication block 136 configured to communicate with IoT devices in or outside the environment 108. For example, the wireless communication block 136 may provide a command corresponding to the identified user gesture to the selected IoT device (e.g., 110) . The wireless communication block 136 may be further configured to poll, continuously or periodically, the IoT devices in order to identify new IoT devices that may be added to the list 154. The wireless communication block 136 may communicate with the IoT devices using wireless communication protocols, such as, for example, WiFi or

according to wireless communication standards, such as Open Internet Consortium (OIC) or

for example. In other embodiments, the wireless communication block 136 may communicate with the IoT devices using cellular communication protocol like Long Term Evolution (LTE) or networking protocol like Ethernet.

The apparatus 100 may include other components 146 necessary for the functioning of the apparatus 100, which are not described herein for ease of understanding. For example, other components 146 may include a transceiver to communicate the scan data and/or results of the analysis of the scan data over one or more wired or wireless network (s) with any other suitable device, such as external computing device (not shown) . The other components 146 may further include a display device configured to display an image of a radar output provided by the LiDAR scanner 102.

The embodiments of an example apparatus 100 and related techniques for IoT device control described herein provide for a number of advantages compared to the known solutions. For example, a user may not need to use electronic wearable devices to control IoT devices, and may be free to use her hands for IoT device control as well as for conventional purposes.

The described embodiments provide for indifference to ambient conditions, such as ambient light intensity, ambient noise level, and the like. For example, a LiDAR scanner may measure distance to a target point in the environment by illuminating a target with a laser and analyzing the reflected light. Accordingly, a LiDAR scanner is equipped with its own light source, such as a laser, and may operate in any ambient conditions, even in the darkness.

The described embodiments may alleviate any power availability concerns that may typically arise with regard to mobile or wearable devices. For example, an apparatus for IoT device control based on a LiDAR scanner described herein may be implemented as a stationary device that may be powered from an outlet available in the environment (e.g., a room) .

Further, the described embodiments may alleviate any privacy concerns typically associated with conventional solutions that use cameras or other image capture devices.

FIG. 2 is a diagram illustrating some aspects of operation of an example apparatus for controlling IoT devices, in accordance with some embodiments. For ease of understanding, the like components of FIGS. 1 and 2 are indicated by like numerals. An image 200 of FIG. 2 illustrates an example implementation of the apparatus 100 in operation, namely, in detection of a presence of a user 202 in an environment, such as a room 204. As described, the LiDAR scanner 102 may continually scan the room 204 and push the scanned data to the controller 130 of the apparatus 100 (shown here as a laptop computer) . Any kind of a LiDAR scanner configured to scan a horizontal plane and detect objects may be used with the apparatus 100. In the example shown in image 200, the LiDAR scanner comprises RPLIDAR-A1M1, a 360 degree 2D laser scanner solution from

having the following parameters: simultaneous localization and mapping (SLAM) ready； 5.5 Hz (2000 sample/sec) ； 6 meters’ measurement range； 1 degree angular resolution； 0.2 cm distance resolution； and 360 degree scan. A diagram 220 illustrates an example image of a radar output of the LiDAR scanner. Based on the scan result by the LiDAR scanner, every object in the room may be described as dots in a radar image. For example, a body 206 of the user 202 may be indicated by a series of dots in the radar images, forming a pattern, as indicated by the word “body” in the diagram 220.

A moving object (e.g., a user) may be described as a series (pattern) of dots whose position may change in the radar image in different time instances. Accordingly, to detect a body of the user, the apparatus 100 may rely on human body movement. The apparatus 100 (e.g., controller 130 of FIG. 1) may analyze the disposition of the dots in the image to identify different patterns, which may include a position of the user, direction the user faces, and a type of gesture the user may indicate.

For example, the human detection block 132 of the controller 130 may analyze scan data provided by the LiDAR scanner 102 and determine whether the user 202 or multiple users are present in the room 204. For example, the human detection block 132 may detect the human body by comparing two consecutive (or spread apart in time) frames in LiDAR data. For example, data from a current frame may be compared with data from one or more earlier frames. For example, if there are more than four dots forming a continuous pattern, and the dots may be within 0.2 m to 1.0 m from respective dots in an earlier frame, it may be resolved that a human body has been detected. Using the LiDAR scanner described in the example above, human position was tracked on about 482 frames, with recall (the probability that the body is correctly detected when the human is moving) reaching about 94.3％and precision (the probability that the body is correctly detected in all detected bodies) reaching about 88.2％.

FIG. 3 is an example process flow diagram for detection of a user in an environment, in accordance with some embodiments. The process 300 may be performed by the apparatus 100, more specifically, by the controller 130 (e.g., human detection block 132 of the controller 130) .

The process 300 may begin at block 302 and include receiving scan data including two or more consecutive (e.g., taken at different time instances) frames with images of an environment from a LiDAR scanner (e.g., LiDAR scanner 102) . As discussed above, the images may include a plurality of dots forming a continuous pattern. In some embodiments, the images may include a plurality of subsets of dots, each subset forming a continuous pattern. Each continuous pattern may correspond to an object moving in the environment.

At block 304, the process 300 may include comparing the images included in at least two of the two or more consecutive frames. More specifically, one frame may have been taken at a current time instance and another frame may have been taken at one of preceding time instances.

At block 306, the process 300 may include determining differences between the images. For example, the differences in distances between the apparatus and respective dots in respective patterns in two frames spread apart in time may be determined. For example, there may be two patterns of dots in each image (frame) . Let us assume there are two frames, spread apart in time. The distances from each dot in a respective pattern to the apparatus may be determined for a first frame and for a second frame. Then, for each of the two patterns, the differences in respective distances between respective dots and the apparatus may be determined.

At block 308, the process 300 may include identifying one or more moving objects based on the determined differences. Continuing with the previous example, for each of two patterns, the differences in the distances from respective dots to the apparatus may be determined to be above a threshold or to fall within a predefined range. Then, an inference may be made that there are two moving objects in the environment, such as, for example, two users in the room. Time instance values of consecutive frames may be, for example about 180ms. The threshold or frequency of the LiDAR scanner may depend on implementation； in one instance it may be, for example, 5.5 Hz.

At block 310, the process 300 may include checking a size of the identified objects, and removing objects whose size does not fit the expected body size (e.g., determined to be too big or too small for an average body size) from further consideration.

FIG. 4 is an example graph illustrating user detection, in accordance with some embodiments. As shown, the square dots comprise a body pattern 402 in a current frame, and the round dots comprise a body pattern 404 in one of preceding frames (e.g., five frames before the current frame) . The dark dots (overlapping the square dots in the current frame) indicate a pattern 406 corresponding to a detected body of the user, based on the process described in reference to FIG. 3.

In summary, the apparatus 100 may identify multiple moving objects in the environment, and each object may correspond to a user. The apparatus 100 may further identify and select a user among multiple users who may issue a command (e.g., with a gesture) to control an IoT device. To select a user to control an IoT device among multiple users, the apparatus 100 may utilize a library of predefined and stored gestures. For example, the apparatus 100 (e.g., gesture detection and identification block 134 of the controller 130) may be configured to recognize a predefined start control gesture so that a user who produces such gesture may be identified as one to control an IoT device. In another example, multiple users may produce gestures to control multiple IoT devices in the environment. In such example, more than one user may attempt to control the same IoT device. The apparatus 100 may be configured to employ conflict resolution logic for such cases.

For example, if two users issue gestures corresponding to conflicting commands to control an IoT device (e.g., turn up the heat and turn down the heat in the room) , the controller 130 may resolve the conflict using a set of predetermined rules. Following the heat example, the apparatus 100 may determine that the room temperature is such that the heat may need to be turned up, not down, and proceed with a corresponding command. In another example, if more than one user issue the same start control gesture, the apparatus 100 may select (e.g., randomly or based on predefined rules) one of the users to control the IoT device. For example, apparatus 100 may select users in a particular, e.g., predefined order. In other words, whoever is the last user to issue a command to control the same IoT device may be set as a user to control the device.

FIG. 5 is an example process flow diagram for user selection among a group of users in an environment, in accordance with some embodiments. The process 500 may be performed by the apparatus 100, more specifically, by the controller 130 (e.g., gesture detection and identification block 134 of the controller 130) . The process 500 illustrates just one example of user selection. Other selection scenarios may be implemented, for example, based on the above description.

The process 500 may begin at block 502 and include detecting two or more gestures performed by two users of the multiple users in the environment. It may be assumed that the multiple users may have been identified using the process described in reference to FIG. 3.

At block 504, the process 500 may include comparing the gestures with predefined “start control gestures. ” The predefined start control gestures may be stored in a memory (e.g., memory 144) and may be accessible by the gesture detection and identification block 134.

At decision block 506, the process may include determining whether both gestures may be start control gestures.

If both gestures are determined to be start control gestures, at block 508 a user to control an IoT device may be selected based on a conflict resolution logic (e.g., using a predefined set of rules) , or randomly.

If both gestures are not determined to be start control gestures, at decision block 510 it may be determined whether at least one of the two gestures may be a start control gesture.

If it is determined that one of the gestures is a start control gesture, at block 512 a user who performed the start control gesture may be selected to control an IoT device.

If none of the gestures is determined to be a start control gesture, at block 514 a user selection to control an IoT device may be put on hold until further determination may be made, based, e.g., on further gestures performed by the users.

As noted above, the described process is just one example of user selection among a group of users in an environment. Other scenarios may be contemplated. For example, referring to block 510, if there is no start control gesture determined by the apparatus 100, it may be concluded at block 514 that every user present in the environment may control the IoT devices. Accordingly, the apparatus 100 may just respond to all commands provided by users present, for example, in a “first come, first serve” fashion.

In some embodiments, the user may select an IoT device to be controlled by the user’s gestures, from a plurality of IoT devices present in the environment or otherwise available for control by gestures. A list of such IoT devices with corresponding characteristics may be stored and may be accessible by the apparatus for controlling IoT devices, such as apparatus 100.

The IoT device selection may be accomplished in different ways. For example, if an IoT device to be controlled is present in the environment, the user may face it, walk over to it, and/or make an identifying gesture, in order to identify an IoT device to be controlled.

In another example, an IoT device selection gesture may be associated with a particular IoT device, and the library of selection gestures and corresponding IoT devices may be stored and made accessible by the controller 130 of the apparatus 100. In such cases, the IoT device may be located outside of the environment. Let us assume that the apparatus 100 is made aware of all IoT devices in and outside the environment (e.g., the IoT devices may be registered with the apparatus 100) . A user may be present in the environment in which the apparatus 100 is located. The user may perform an IoT device selection gesture to select an IoT device outside of the room. For example, the user may want to turn off outside lights on the porch of her house while sitting in her living room, where the apparatus 100 may be located. The user may perform a first gesture selecting the IoT device comprising an outside light, and the apparatus 100 may recognize the IoT device selection gesture. The user may then perform another gesture indicating a command to turn off the outside light on the porch. The apparatus 100 may transmit the corresponding command to the outside light switch.

FIG. 6 is an example process flow diagram for IoT device selection among a group of IoT devices, in accordance with some embodiments. The process 600 may be performed by the apparatus 100, more specifically, by the controller 130 (e.g., gesture detection and recognition block 134 of the controller 130) . The process 600 illustrates just one example of IoT device selection. Other selection scenarios may be implemented, for example, based on the above description.

The process 600 may begin at block 602 and include determining a type of gesture provided by the user. It may be assumed that the user to control an IoT may have been selected using the process described in reference to FIG. 5. As described above, a gesture may be associated with a particular IoT device and may be used to select that IoT device, regardless of whether or not the IoT device is present in the environment. For example, the user may point at an IoT device, if present in the environment. In another example, the user may perform a particular gesture (e.g., show a certain number of fingers, for example, two fingers, indicating a number (e.g., number two) of the IoT device registered with the apparatus 100 (e.g., stored in the list 154) , to be selected.

In some embodiments, the gesture may comprise a position of the user in relation to the IoT device, when the IoT device to be selected is present in the environment. For example, if the user is determined to face the IoT device, and/or stand next to the IoT device, e.g., for a time period above a threshold, it may be determined that the IoT device may have been selected by the user for control.

In another example, the user may walk over toward the IoT device and stand in front of the IoT device to be selected. The user’s movement toward the IoT device (e.g., direction of movement) may be detected (e.g., as discussed in reference to FIG. 3) and an inference may be made that the user intends to select the IoT device. In this respect, the direction of the user’s movement may be considered a type of gesture as well.

In some embodiments, if the IoT device may already have been selected for control according to the description above, the gesture may indicate a command to control the selected IoT device, and may be identified, e.g., using a library of gestures as described above.

At block 604, the IoT device may be selected, among a plurality of IoT devices, for control by the user according to the actions described in reference to block 602. In some embodiments, when the IoT device may already have been selected for control (e.g., according to the actions described in reference to block 602) , the apparatus 100 may transmit a command corresponding to the identified gesture to the selected IoT device.

For example, if the selected IoT device is an air conditioner, the user may issue a command to the air conditioner to indicate turning on/off, temperature up, temperature down, or the like. The user may issue a command by performing arm gestures (for example, both arms in front, left arm in front, right arm in front, or the like) . The performed and identified gesture may be compared to a library of predefined gestures and corresponding commands stored and/or accessible by the apparatus 100, and a command corresponding to the identified gesture may be provided to the air conditioner.

As discussed above, the apparatus 100 may be configured to detect and identify different gestures (gesture types) in order to select a user to control an IoT device, select an IoT device to be controlled by the user, and provide a command indicated by the identified gesture to the selected IoT device. The embodiments described herein provide some example techniques for detection and identification of user gestures.

FIG. 7 is a diagram illustrating some examples of gesture detection and identification using an example apparatus for controlling IoT devices, in accordance with some embodiments. For ease of understanding, the like components of FIGS. 1, 2, and 7 are indicated with like numerals. Image 702 illustrates an example implementation of the apparatus 100 in detection and identification of user’s hands 710 of a user 202 in the environment, such as a room 204. Image 704 illustrates an example implementation of the apparatus 100 in detection and identification of an arm gesture 712 by a user 202 in the room 204. Image 706 illustrates an example implementation of the apparatus 100 in detection and identification of a hand gesture 714 (e.g., a finger combination) by a user 202 in the room 204.

FIG. 8 is a diagram illustrating a radar image output of an example apparatus for controlling IoT devices in response to a scan of an environment in which the user may perform different gestures, in accordance with some embodiments. As described in reference to FIG. 2, every object in the room may be described as a series of dots forming a pattern in a radar image, based on the result of a scan by a LiDAR scanner. The image output 800 of a LiDAR scanner shows the patterns corresponding to a left hand, a right hand, and a body of the user in response to scanning the environment as shown in

images

702 or 704, respectively. The image output 802 of a LiDAR scanner shows the patterns corresponding to the fingers of a hand and a body of the user, in response to scanning the environment as shown in image 706. Gesture detection and identification based on the scan results (e.g., image outputs 800, 802) in response to performance of different types of gestures by the user will be described below in detail.

When the user’s arm or hand position is detected, as shown in FIGS. 7-8, there may be at least two kinds of gestures defined by the user’s arm/hand: a hand gesture or an arm gesture. These kinds of gestures may be used to select or unselect an IoT device and control it as discussed above. As noted above, every kind of gesture (e.g., arm or hand gesture) may include different types of gestures, and each type may correspond to a particular command to control an IoT device or a command to select a user or an IoT device. For example, hand gestures may include gestures with one finger or multiple (e.g., two) fingers.

A distance from the user to a LiDAR scanner, within which a gesture may be detected and recognized with a desired accuracy, may depend on a resolution of the LiDAR scanner. For example, a

LiDAR device may be able to detect finger gestures within about one meter range. Accordingly, hand gestures (e.g., finger combinations) may be used, for example, when a user is positioned at a short range from the LiDAR scanner (e.g., at a distance below a threshold) , in order to attain desired detection and identification accuracy. For example, the user and the apparatus for IoT device control may be located in a small space, such as a kitchen of a house, for example. Accordingly, the user may perform hand gestures so that they may be detected and identified with a desired accuracy.

Arm gestures may be used when the user is positioned at a long distance from the LiDAR scanner (e.g., at a distance above a threshold) , in order to attain desired detection and identification accuracy. For example, the user and the apparatus for IoT device control may be located in a space of a house that may be greater than a kitchen, for example, in a living room. Accordingly, the user may perform arm gestures in order for them to be detected and identified with a desired accuracy.

It will be appreciated that a distinction between short-range and long-range gestures may not be necessary when a LiDAR scanner has a desired (high) resolution, which may provide for desired accuracy of any kind or type of gesture at any distance within an environment, such as a room of a house, for example.

FIG. 9 is an example process flow diagram for gesture detection and identification, in accordance with some embodiments. The process 900 may be performed by the apparatus 100, more specifically, by the controller 130 (e.g., gesture detection and identification block 134 of the controller 130) .

The process 900 may begin at block 901 and include receiving scan data from one or more LiDAR scanners as described in reference to FIG. 1. If the scan data is provided by multiple, e.g., two LiDAR scanners, fusion logic may be used for subsequent actions of gesture detection and identification as described in reference to FIG. 1.

At block 902, the process 900 may include detecting a gesture performed by the user. It may be assumed that the user body has been detected as described in reference to FIGS. 2-4. For example, a number of dots forming a body pattern may be detected within particular distance from the LiDAR scanner, as described in reference to FIGS. 2-4. The gesture may be detected by ascertaining, for example, that one or more objects may be within a threshold distance from the user’s body. For example, it may be assumed that the left and right hands of the user may be detected within a certain range (e.g., about one meter) from the body (see FIG. 8, image output 800) .

At block 904, the process 900 may include recognizing the detected gesture as one of an arm gesture or a hand gesture. As discussed above, the arm gestures may be detected within a distance above a distance threshold, and the hand gestures may be detected within a distance below a distance threshold. For example, it may be determined that the user is within the threshold distance to the LiDAR scanner. Accordingly, it may be inferred that the user may be performing a hand gesture. Conversely, it may be determined that the user is not within the threshold distance to the LiDAR scanner. Accordingly, it may be inferred that the user may be performing an arm gesture.

Additional determinations may be required to recognize a gesture as a hand gesture or an arm gesture. For example, an arm may be recognized as an object of a certain size, such as thickness, e.g., within a particular thickness range. The fingers of a hand may be also recognized as objects of a certain size (e.g., thickness) , within another thickness range. Understandably, an arm thickness range may be greater than a finger thickness range. In another example, an object, such as an arm or a hand, may be detected within a certain distance from the detected body. Accordingly, the gesture may be determined as a hand gesture or an arm gesture, based at least in part on one or more of: estimated distance from the body of the user to the LiDAR scanner, estimated distance between the body and the detected object (or objects) , and/or estimated size (e.g., thickness) of the object (or objects) .

The arm may be easier to detect than a finger due to its size (e.g., thickness, length, or the like) . Accordingly, the arm may be displayed on the LiDAR as a bigger object than a finger (compare, e.g.,

images

800 and 802 of FIG. 8) . Further, an expected distance between two arms may be greater than an expected distance between the fingers. Accordingly, arm gestures may be easier to detect than hand gestures, with currently existing LiDAR scanners.

At block 906, the process 900 may include identifying a type of gesture that was identified earlier as a hand gesture or an arm gesture. For example, a hand gesture may include two types of gestures: a single finger gesture or a double finger gesture. It is understood that there may be various other types of hand gestures that may be identified, such as a three-finger gesture, a four-finger gesture, or a five-finger gesture (e.g., 706 in FIG. 7) in various combinations, and the like. For example, multiple (e.g., two) objects may be identified as having a certain thickness (corresponding to expected finger thickness range) , and may be further identified as being within an expected distance to each other (corresponding to expected distance between fingers) , and being within an expected distance from the body (as discussed in reference to block 904) . Accordingly, a determination may be made that the gesture is a hand gesture with multiple (e.g., two) fingers.

For example, fingers may be identified as an object with the thickness (radius) that may be less than 5 cm. A distance between the fingers and the body may be within a range of 1 to 100 cm. The distance between the fingers of a hand may be expected to be less than 10 cm. The objects on the radar screen (e.g., FIG. 8) that are not identified as fingers may be removed from further consideration.

For example, a series of dots in the image may be determined as a radius (e.g., thickness or wideness) of a finger or arm. The dot pattern of a finger may include a number of dots, and the length of that dot pattern may be determined. For example, four fingers in the image may be identified as made up by four lines of dots with clear spacing between them.

Arm gestures may include different gesture types, for example, left arm in front of a body of a user, right arm in front of the body of the user, both arms in front of the body of the user, or the like. Accordingly, the arm gesture identification may include a determination whether an arm (or arms) is (are) within a threshold distance from a front of a body of the user, wherein the threshold distance may define a distance from a front of a body within which an arm is to be detected. Further, if just one arm has been detected, it may be determined whether the arm is a left arm or a right arm, based, for example, on an expected distance between an arm and a side of the body that is closer to the arm than the other side.

At block 908, the process 900 may include determining a command that corresponds to the identified type of the one of the arm gesture or hand gesture. Such determination may be based on accessing a library of gestures and corresponding commands that may be pre-stored and accessible by the controller. For example, a left arm in front of the user’s body (or, for example, a one-finger hand gesture) may correspond to raising a temperature threshold for an air conditioner, a right arm in front of the user’s body (or a two-finger hand gesture) may correspond to lowering a temperature threshold for an air conditioner, and both arms in front of the user’s body (or a five-finger hand gesture) may correspond to turning the air conditioner off. It should be noted that the described embodiments represent examples of gestures and corresponding commands and are not limiting to this disclosure. Different kinds of gestures and corresponding commands may be contemplated that may be implemented with the embodiments of the present disclosure.

As briefly discussed in reference to FIG. 6, an IoT device may be selected by a user gesture and/or position. For example, the user may be facing toward an IoT device, or may be moving towards the IoT device, and the inference may be made that the user intends to select the IoT device. As described above, the IoT device description may be pre-stored and may be made accessible to the apparatus for controlling IoT devices (e.g., apparatus 100 of FIG. 1) .

In some instances, a new IoT device may be added to a list of IoT devices (e.g., 154) operated by the apparatus 100. For example, a user may purchase a new appliance and may wish to add the new appliance to a list of IoT devices operated by the apparatus for IoT device control. The user may add the appliance to the list of IoT devices in a number of different ways. For example, the user may use an application (e.g., residing on her smartphone) that may have a capability to add the new IoT device description (characteristics, etc. ) to a list of IoT devices operated by the apparatus for controlling IoT devices. The apparatus for controlling IoT devices may need to be aware of a location of the new IoT device, in addition to having device characteristics available to it, in order to enable control of the device with user gestures.

FIG. 10 is an example process flow diagram for adding a location of a new IoT device to a description of IoT devices operated by the apparatus for controlling IoT devices, in accordance with some embodiments. It may be assumed that the new IoT device description has been already stored (e.g., with reference to FIG. 1, in the memory 144 or device list 154 accessible by the controller 130 of the apparatus 100) . It may be further assumed that the new IoT device is located in the environment in which the apparatus for controlling IoT devices is disposed.

The process 1000 may begin at block 1002 and include receiving scan data from scanning the environment by the apparatus for controlling IoT devices as described in reference to FIGS. 1-4.

At block 1004, the process 1000 may include detecting user position. For example, the user may be detected as facing a certain direction, in which a new appliance (IoT device) may be placed. The user may be further detected as standing beside the IoT device, whose location may need to be identified. A user position may be treated as an approximate location of the IoT device.

At block 1006, the process 1000 may include detecting and identifying a user gesture. As described above, a particular gesture may select an IoT device. For example, a particular gesture may identify a particular IoT device that may be on the list of devices to be operated by the apparatus for IoT device control. In some embodiments, the user may perform a combination of standing near the IoT device (to identify device location) and a gesture to identify the device. In some embodiments, a user may provide user input, e.g., via an application (e.g., residing on her smartphone) that may identify the IoT device next to which the user may be standing. For example, the user may select an IoT device from a list (library) of IoT devices accessible by the apparatus for controlling IoT devices.

At decision block 1008 it may be determined whether a position and/or gesture detected at

blocks

1004 and 1006 may have identified an IoT device. If the gesture and/or position of the user (and/or user input as described above) did not identify an IoT device, at block 1010 the apparatus for controlling IoT devices may work in a normal mode, e.g., identifying a type of gesture and the like, as described earlier.

If the gesture and/or position of the user (and/or user input) identified an IoT device, at block 1012 the IoT device location may be stored, e.g., added to the IoT device description stored e.g., in a device list and accessible by the apparatus for controlling IoT devices.

Referring now back to FIG. 1, the various instructions stored in non-transitory computer-readable memory 144, when executed on the processor 142, to cause the controller 130 to perform operations described above in reference to

blocks

132, 134, 136, list 154 of IoT devices, and/or gestures that may be recognized and their corresponding commands, may be pre-stored at manufacturing time, or downloaded/inputted into controller 130 in the field. The instructions, list of IoT devices and/or gestures may be distributed via non-transitory computer-readable media, such as compact disc (CD) or transitory computer-readable media, such as signals.

The following paragraphs describe examples of various embodiments. Example 1 may be an apparatus, comprising: at least one Light Detection and Ranging Device (LiDAR) scanner having a laser to scan an environment to obtain scan data, wherein to scan includes to illuminate a point of the environment and measure a distance to the illuminated point based on a difference between a time of transmission of a first light beam to illuminate the point and a time of receipt of a second light beam reflected by the illuminated point； and a controller coupled with the at least one LiDAR scanner, to collect the scan data, and, based at least in part on the scan data, detect a presence of a user in the environment, detect and identify a gesture provided by the user, and provide a command that corresponds to the identified gesture to one or more Internet of Things (IoT) devices that are responsive to commands provided by user gestures.

Example 2 may include the subject matter of Example 1, wherein the environment includes at least some of the one or more of IoT devices, wherein to illuminate a point of the environment includes to illuminate one point of the environment at a time, and wherein the environment includes at least partially enclosed three-dimensional (3D) space.

Example 3 may include the subject matter of Example 1, wherein the controller includes a wireless communication block, to provide the command that corresponds to the identified gesture to the one or more IoT devices.

Example 4 may include the subject matter of Example 1, wherein the controller is to select the one or more IoT devices from a plurality of IoT devices that are responsive to commands provided by user gestures, based at least in part on the identified gesture.

Example 5 may include the subject matter of Example 4, wherein the plurality of IoT devices includes at least one of: an air conditioner, a light, a fan, a thermostat, or an appliance.

Example 6 may include the subject matter of Example 1, wherein the controller to detect a user presence in the environment includes: a human detection block to detect one or more moving objects in the environment, based at least in part on the scan data, wherein one or more moving objects comprise one or more users present in the environment； and a gesture detection and identification block coupled with the human detection block, based at least in part on the scan data, to: detect a gesture associated with one of the one or more moving objects； and identify a gesture as one of an arm gesture or a hand gesture.

Example 7 may include the subject matter of Example 6, wherein the scan data includes two or more consecutive frames that include images of the environment provided by scan data associated with respective time instances corresponding to the consecutive frames, wherein the human detection block to detect one or more moving objects in the environment includes to: compare the images included in at least two of the two or more consecutive frames； determine differences between the images； and identify the one or more users based on the determined differences.

Example 8 may include the subject matter of Example 6, wherein the gesture detection and identification block is further to: identify a type of the one of the arm gesture or hand gesture, and determine a command that corresponds to the identified type of the one of the arm gesture or hand gesture, wherein identification and determination is based at least in part on comparison of the detected gesture with at least some of a plurality of predefined gestures stored in a gesture library and accessible by the controller.

Example 9 may include the subject matter of Example 8, wherein the gesture detection and identification block to identify a type of the one of the arm gesture or hand gesture includes to: detect one or more objects within a threshold distance from a body of the user； estimate sizes of each of the one or more objects； estimate distances between the one or more objects； determine that the estimated distances do not exceed a threshold distance that defines a maximum distance between fingers of a hand； and determine whether the one or more objects correspond to one or more fingers of a hand of the user, based at least in part on the estimated sizes and the estimated distances.

Example 10 may include the subject matter of Example 8, wherein the gesture detection and identification block is to identify a type of the one of the arm gesture or hand gesture as a start control gesture, wherein the start control gesture indicates a user to control the one or more IoT devices.

Example 11 may include the subject matter of Example 8, wherein the gesture detection and identification block to identify a type of the one of the arm gesture or hand gesture includes to: detect an object within a first threshold distance from a front of a body of the user, wherein the first threshold distance defines a distance from a front of a body within which an arm is to be detected； and determine a distance between the object and a side of the body； compare the distance with a second threshold distance, wherein the second threshold distance defines a distance from a side of a body to an arm of the body； and determine whether the object corresponds to an arm of the user, based on a result of the comparison.

Example 12 may include the subject matter of Example 1, wherein the at least one LiDAR scanner comprises two or more LiDAR scanners, wherein the controller includes a fusion logic to collect and analyze scan data provided by the two or more LiDAR scanners, wherein the fusion logic is to: compare the scan data provided by each of the two or more LiDAR scanners； and detect the presence of the user in the environment, and detect and identify the gesture provided by the user, based on a result of the comparison.

Example 13 may include the subject matter of any of Examples 1 to 12, wherein the LiDAR scanner is to scan the environment substantially continuously and to collect the scan data in near-real time or real time.

Example 14 may be a method for controlling Internet of Things (IoT) devices, comprising: collecting, by a controller of an apparatus for controlling IoT devices, first scan data of an environment from a first Light Detection and Ranging Device (LiDAR) scanner and second scan data of the environment from a second LiDAR scanner； detecting, by the controller, a presence of a user in the environment, based at least in part on comparison of the collected first and second scan data； and detecting and identifying, by the controller, a gesture provided by the user, based at least in part on comparison of the collected first and second scan data, to control one or more IoT devices that are responsive to commands provided by user gestures, with a command that corresponds to the identified gesture.

Example 15 may include the subject matter of Example 14, further comprising: receiving, by the controller, the first and second scan data of the environment in real time or near-real time.

Example 16 may include the subject matter of Example 15, wherein the first or second scan data includes two or more consecutive frames that include images of the environment associated with respective time instances corresponding to the consecutive frames, wherein detecting a presence of a user includes: comparing, by the controller, the images included in the two or more consecutive frames； and determining, by the controller, differences between the images to identify one or more moving objects in the environment, wherein at least one of the one or more moving objects corresponds to a user who is present in the environment.

Example 17 may include the subject matter of Example 16, further comprising: identifying, by the controller, an object within a threshold distance from a body of the user； identifying, by the controller, a type of the object； and detecting, by the controller, a gesture indicated by the object, wherein detecting and identifying a gesture provided by the user is based at least in part on detecting the gesture indicated by the object.

Example 18 may include the subject matter of Example 17, further comprising: determining, by the controller, a command that corresponds to the identified type of the gesture.

Example 19 may include the subject matter of Example 18, further comprising: sending, by the controller, the command to the one or more IoT devices.

Example 20 may include the subject matter of Example 17, further comprising: selecting, by the controller, the one or more IoT devices from a plurality of IoT devices that are responsive to commands provided by user gestures, based at least in part on the detected and identified gesture.

Example 21 may be one or more non-transitory computer-readable media having instructions stored thereon that, in response to execution on a controller of an apparatus for controlling IoT devices, cause the controller to: collect first scan data of an environment from a first Light Detection and Ranging Device (LiDAR) scanner and second scan data of the environment from a second LiDAR scanner； detect a presence of a user in the environment, based at least in part on comparison of the collected first and second scan data； and detect and identify a gesture provided by the user, based at least in part on comparison of the collected first and second scan data, to control one or more IoT devices that are responsive to commands provided by user gestures, with a command that corresponds to the identified gesture.

Example 22 may be the one or more non-transitory computer-readable media of Example 21, wherein the instructions further cause the controller to receive the first and second scan data of the environment in real time or near-real time.

Example 23 may be the one or more non-transitory computer-readable media of Example 21, wherein the instructions further cause the controller to select the one or more IoT devices from a plurality of IoT devices that are responsive to commands provided by user gestures, based at least in part on the detected and identified gesture.

Example 24 may be the one or more non-transitory computer-readable media of any of Examples 21 to 23, wherein the instructions further cause the controller to identify a command that corresponds to the identified type of the gesture, and send the command to the one or more IoT devices.

Example 25 may be an apparatus for controlling Internet of Things (IoT) devices, comprising: means for collecting first scan data of an environment from a first Light Detection and Ranging Device (LiDAR) scanner and second scan data of the environment from a second LiDAR scanner； means for detecting a presence of a user in the environment, based at least in part on comparison of the collected first and second scan data； and means for detecting and identifying a gesture provided by the user, based at least in part on comparison of the collected first and second scan data, to control one or more IoT devices that are responsive to commands provided by user gestures, with a command that corresponds to the identified gesture.

Example 26 may include the subject matter of Example 25, further comprising: means for receiving the first and second scan data of the environment in real time or near-real time.

Example 27 may include the subject matter of Example 26, wherein the first or second scan data includes two or more consecutive frames that include images of the environment associated with respective time instances corresponding to the consecutive frames, wherein means for detecting a presence of a user includes: means for comparing the images included in the two or more consecutive frames； and means for determining differences between the images to identify one or more moving objects in the environment, wherein at least one of the one or more moving objects corresponds to a user who is present in the environment.

Example 28 may include the subject matter of Example 27, further comprising: means for identifying an object within a threshold distance from a body of the user； means for identifying a type of the object； and means for detecting a gesture indicated by the object, wherein detecting and identifying a gesture provided by the user is based at least in part on detecting the gesture indicated by the object.

Example 29 may include the subject matter of Example 28, further comprising: means for determining a command that corresponds to the identified type of the gesture.

Example 30 may include the subject matter of Example 29, further comprising: means for sending the command to the one or more IoT devices.

Example 31 may include the subject matter of Example 28, further comprising: means for selecting the one or more IoT devices from a plurality of IoT devices that are responsive to commands provided by user gestures, based at least in part on the detected and identified gesture.

Various operations are described as multiple discrete operations in turn, in a manner that is most helpful in understanding the claimed subject matter. However, the order of description should not be construed as to imply that these operations are necessarily order dependent. Embodiments of the present disclosure may be implemented into a system using any suitable hardware and/or software to configure as desired.

Although certain embodiments have been illustrated and described herein for purposes of description, a wide variety of alternate and/or equivalent embodiments or implementations calculated to achieve the same purposes may be substituted for the embodiments shown and described without departing from the scope of the present disclosure. This application is intended to cover any adaptations or variations of the embodiments discussed herein. Therefore, it is manifestly intended that embodiments described herein be limited only by the claims and the equivalents thereof.

Claims

An apparatus, comprising:

at least one Light Detection and Ranging Device (LiDAR) scanner having a laser to scan an environment to obtain scan data, wherein to scan includes to illuminate a point of the environment and measure a distance to the illuminated point based on a difference between a time of transmission of a first light beam to illuminate the point and a time of receipt of a second light beam reflected by the illuminated point； and

a controller coupled with the at least one LiDAR scanner, to collect the scan data, and, based at least in part on the scan data, detect a presence of a user in the environment, detect and identify a gesture provided by the user, and provide a command that corresponds to the identified gesture to one or more Internet of Things (IoT) devices that are responsive to commands provided by user gestures.
The apparatus of claim 1, wherein the environment includes at least some of the one or more of IoT devices, wherein to illuminate a point of the environment includes to illuminate one point of the environment at a time, and wherein the environment includes at least partially enclosed three-dimensional (3D) space .
The apparatus of claim 1, wherein the controller includes a wireless communication block, to provide the command that corresponds to the identified gesture to the one or more IoT devices.
The apparatus of claim 1, wherein the controller is to select the one or more IoT devices from a plurality of IoT devices that are responsive to commands provided by user gestures, based at least in part on the identified gesture.
The apparatus of claim 4, wherein the plurality of IoT devices include at least one of: an air conditioner, a light, a fan, a thermostat, or an appliance.
The apparatus of claim 1, wherein the controller to detect a user presence in the environment includes:

a human detection block to detect one or more moving objects in the environment, based at least in part on the scan data, wherein one or more moving objects comprise one or more users present in the environment； and

a gesture detection and identification block coupled with the human detection block, based at least in part on the scan data, to:

detect a gesture associated with one of the one or more moving objects； and

identify a gesture as one of an arm gesture or a hand gesture.
The apparatus of claim 6, wherein the scan data includes two or more consecutive frames that include images of the environment provided by scan data associated with respective time instances corresponding to the consecutive frames, wherein the human detection block to detect one or more moving objects in the environment includes to:

compare the images included in at least two of the two or more consecutive frames；

determine differences between the images； and

identify the one or more users based on the determined differences.
The apparatus of claim 6, wherein the gesture detection and identification block is further to:

identify a type of the one of the arm gesture or hand gesture, and

determine a command that corresponds to the identified type of the one of the arm gesture or hand gesture, wherein identification and determination is based at least in part on comparison of the detected gesture with at least some of a plurality of predefined gestures stored in a gesture library and accessible by the controller.
The apparatus of claim 8, wherein the gesture detection and identification block to identify a type of the one of the arm gesture or hand gesture includes to:

detect one or more objects within a threshold distance from a body of the user；

estimate sizes of each of the one or more objects；

estimate distances between the one or more objects；

determine that the estimated distances do not exceed a threshold distance that defines a maximum distance between fingers of a hand； and

determine whether the one or more objects correspond to one or more fingers of a hand of the user, based at least in part on the estimated sizes and the estimated distances.
The apparatus of claim 8, wherein the gesture detection and identification block is to identify a type of the one of the arm gesture or hand gesture as a start control gesture, wherein the start control gesture indicates a user to control the one or more IoT devices.
The apparatus of claim 8, wherein the gesture detection and identification block to identify a type of the one of the arm gesture or hand gesture includes to:

detect an object within a first threshold distance from a front of a body of the user, wherein the first threshold distance defines a distance from a front of a body within which an arm is to be detected； and

determine a distance between the object and a side of the body；

compare the distance with a second threshold distance, wherein the second threshold distance defines a distance from a side of a body to an arm of the body； and

determine whether the object corresponds to an arm of the user, based on a result of the comparison.
The apparatus of claim 1, wherein the at least one LiDAR scanner comprises two or more LiDAR scanners, wherein the controller includes a fusion logic to collect and analyze scan data provided by the two or more LiDAR scanners, wherein the fusion logic is to:

compare the scan data provided by each of the two or more LiDAR scanners； and

detect the presence of the user in the environment, and detect and identify the gesture provided by the user, based on a result of the comparison.
The apparatus of any of claims 1 to 12, wherein the LiDAR scanner is to scan the environment substantially continuously and to collect the scan data in near-real time or real time.
A method for controlling Internet of Things (IoT) devices, comprising:

collecting, by a controller of an apparatus for controlling IoT devices, first scan data of an environment from a first Light Detection and Ranging Device (LiDAR) scanner and second scan data of the environment from a second LiDAR scanner；

detecting, by the controller, a presence of a user in the environment, based at least in part on comparison of the collected first and second scan data； and

detecting and identifying, by the controller, a gesture provided by the user, based at least in part on comparison of the collected first and second scan data, to control one or more IoT devices that are responsive to commands provided by user gestures, with a command that corresponds to the identified gesture.
The method of claim 14, further comprising: receiving, by the controller, the first and second scan data of the environment in real time or near-real time.
The method of claim 15, wherein the first or second scan data includes two or more consecutive frames that include images of the environment associated with respective time instances corresponding to the consecutive frames, wherein detecting a presence of a user includes:

comparing, by the controller, the images included in the two or more consecutive frames； and

determining, by the controller, differences between the images to identify one or more moving objects in the environment, wherein at least one of the one or more moving objects corresponds to a user who is present in the environment.
The method of claim 16, further comprising:

identifying, by the controller, an object within a threshold distance from a body of the user；

identifying, by the controller, a type of the object； and

detecting, by the controller, a gesture indicated by the object,

wherein detecting and identifying a gesture provided by the user is based at least in part on detecting the gesture indicated by the object.
The method of claim 17, further comprising: determining, by the controller, a command that corresponds to the identified type of the gesture.
The method of claim 18, further comprising: sending, by the controller, the command to the one or more IoT devices.
The method of claim 17, further comprising: selecting, by the controller, the one or more IoT devices from a plurality of IoT devices that are responsive to commands provided by user gestures, based at least in part on the detected and identified gesture.
One or more non-transitory computer-readable media having instructions stored thereon that, in response to execution on a controller of an apparatus for controlling IoT devices, cause the controller to:

collect first scan data of an environment from a first Light Detection and Ranging Device (LiDAR) scanner and second scan data of the environment from a second LiDAR scanner；

detect a presence of a user in the environment, based at least in part on comparison of the collected first and second scan data； and

detect and identify a gesture provided by the user, based at least in part on comparison of the collected first and second scan data, to control one or more IoT devices that are responsive to commands provided by user gestures, with a command that corresponds to the identified gesture.
The one or more non-transitory computer-readable media of claim 21, wherein the instructions further cause the controller to receive the first and second scan data of the environment in real time or near-real time.
The one or more non-transitory computer-readable media of claim 21, wherein the instructions further cause the controller to select the one or more IoT devices from a plurality of IoT devices that are responsive to commands provided by user gestures, based at least in part on the detected and identified gesture.
The one or more non-transitory computer-readable media of any of claims 21 to 23, wherein the instructions further cause the controller to identify a command that corresponds to the identified type of the gesture, and send the command to the one or more IoT devices.