KR20240049072A - Method for controlling electronic device using spatial information and electronic device using spatial information - Google Patents

Abstract

Based on spatial information about a space containing at least one object and a task that the electronic device is set to perform, an object that interferes with the task is selected from among objects located in the space corresponding to the task, and the attribute information of the selected object is selected. Provide object movement guide information to the user of the electronic device, determine a movement path used to perform a task based on the user's response corresponding to the object movement guide information, and drive the electronic device according to the determined movement path. A method of controlling an electronic device using spatial information and an electronic device using spatial information are disclosed.

Description

Method for controlling electronic devices using spatial information and electronic devices using spatial information {METHOD FOR CONTROLLING ELECTRONIC DEVICE USING SPATIAL INFORMATION AND ELECTRONIC DEVICE USING SPATIAL INFORMATION}

It relates to a method of controlling an electronic device using spatial information and an electronic device using spatial information.

The Internet is evolving from a human-centered network where humans create and consume information to an IoT (Internet of Things) network that exchanges and processes information between distributed components such as objects. IoE (Internet of Everything) technology, which combines IoT technology with big data processing technology through connection to cloud servers, etc., is also emerging. IoT can be applied to fields such as smart home appliances, smart homes, smart buildings, and smart cities through the convergence and combination of existing IT (information technology) technology and various industries.

In an IoT environment, each of the electronic devices connected to each other can collect, generate, analyze, or process data, share the data with each other, and utilize it for each device's tasks. Recently, with rapid progress in the field of computer vision, various types of electronic devices that utilize neural network models that perform vision tasks are being developed. Accordingly, interest in connections between various types of electronic devices in the IoT environment is increasing.

According to an embodiment of the present disclosure, a method of controlling an electronic device using spatial information is provided. A method of controlling an electronic device using spatial information is based on spatial information about a space containing at least one object and a task that the electronic device is set to perform, and determines which of the objects located in the space corresponding to the task are interfering with the task. It includes the step of selecting an object. Additionally, a method of controlling an electronic device using spatial information includes providing object movement guide information according to attribute information of a selected object to a user of the electronic device. Additionally, a method of controlling an electronic device using spatial information includes determining a movement path used to perform the task based on a user's response corresponding to object movement guide information. Additionally, a method of controlling an electronic device using spatial information includes driving the electronic device according to a determined movement path.

According to one embodiment of the present disclosure, a computer-readable recording medium on which a program for executing the above-described method is recorded can be provided.

According to an embodiment of the present disclosure, an electronic device using spatial information is provided. An electronic device that uses spatial information includes a memory that stores one or more instructions, a processor that executes one or more instructions stored in the memory, and a sensing unit. By executing one or more instructions, the processor selects a task among objects located in the space corresponding to the task, based on spatial information about a space containing at least one object obtained through the sensing unit and a task that the electronic device is set to perform. Select objects that are in the way. Additionally, by executing the one or more instructions, the processor provides object movement guide information according to attribute information of the selected object to the user of the electronic device. Additionally, the processor determines a movement path used to perform the task based on the user's response corresponding to the object movement guide information by executing the one or more instructions. Additionally, the processor drives the electronic device along the determined movement path by executing the one or more instructions.

Figure 1 is a diagram to explain an in-home IoT environment in which electronic devices and external devices are connected to each other.
2A and 2B are flowcharts for explaining a spatial map.
FIGS. 3A, 3B, 3C, and 3D are diagrams for explaining a method of utilizing the layers that make up a spatial map.
Figure 4 is a flowchart illustrating a method of obtaining a spatial map according to an embodiment of the present disclosure.
FIG. 5 is a flowchart illustrating a method of controlling an electronic device using spatial information according to an embodiment of the present disclosure.
FIG. 6 is a detailed flowchart illustrating the step of selecting an object that interferes with a task among objects located in a space corresponding to a task according to an embodiment of the present disclosure.
FIG. 7 is a detailed flowchart illustrating the step of providing object movement guide information according to attribute information of an object selected as an object interfering with a task according to an embodiment of the present disclosure.
FIG. 8 is a diagram for explaining a first move request process according to an embodiment of the present disclosure.
Figure 9 is a diagram for explaining an example of providing object movement guide information to a user according to an embodiment of the present disclosure.
FIG. 10 is a diagram illustrating another example of providing object movement guide information to a user according to an embodiment of the present disclosure.
FIG. 11 is a diagram for explaining a second move request process according to an embodiment of the present disclosure.
Figure 12 is a diagram for explaining a process of selecting candidate locations to move the selected object according to an embodiment of the present disclosure.
FIG. 13 is a diagram illustrating an example of providing object movement guide information to a user according to an image evaluation result according to an embodiment of the present disclosure.
FIG. 14 is a detailed flowchart illustrating the step of determining a movement path used to perform a task according to an embodiment of the present disclosure.
15 and 16 are block diagrams showing the configuration of an electronic device using spatial information according to an embodiment of the present disclosure.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail. In the present disclosure, the expression “at least one of a, b, or c” refers to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “a, b and c", or variations thereof.

The terms used in this disclosure are general terms that are currently widely used as much as possible while considering the functions in this disclosure, but this may vary depending on the intention or precedent of a person working in the art, the emergence of new technology, etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the relevant description. Therefore, the terms used in this disclosure should be defined based on the meaning of the term and the overall content of this disclosure, rather than simply the name of the term.

Singular expressions may include plural expressions, unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by a person of ordinary skill in the technical field described herein. Additionally, terms including ordinal numbers such as 'first' or 'second' used in this specification may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When it is said that a part "includes" a certain element throughout the specification, this means that, unless specifically stated to the contrary, it does not exclude other elements but may further include other elements. Additionally, terms such as “unit” and “module” used in the specification refer to a unit that processes at least one function or operation, which may be implemented as hardware or software, or as a combination of hardware and software.

Functions related to artificial intelligence according to the present disclosure are operated through a processor and memory. The processor may consist of one or multiple processors. At this time, one or more processors may be a general-purpose processor such as a CPU, AP, or DSP (Digital Signal Processor), a graphics-specific processor such as a GPU or VPU (Vision Processing Unit), or an artificial intelligence-specific processor such as an NPU. One or more processors control input data to be processed according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, when one or more processors are dedicated artificial intelligence processors, the artificial intelligence dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

Predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is learned using a large number of learning data by a learning algorithm, thereby creating a predefined operation rule or artificial intelligence model set to perform the desired characteristics (or purpose). It means burden. This learning may be performed on the device itself that performs the artificial intelligence according to the present disclosure, or may be performed through a separate server and/or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

An artificial intelligence model may be composed of multiple neural network layers. Each of the plurality of neural network layers has a plurality of weight values, and neural network calculation is performed through calculation between the calculation result of the previous layer and the plurality of weights. Multiple weights of multiple neural network layers can be optimized by the learning results of the artificial intelligence model. For example, a plurality of weights may be updated so that loss or cost values obtained from the artificial intelligence model are reduced or minimized during the learning process. Artificial neural networks may include deep neural networks (DNN), for example, Convolutional Neural Network (CNN), Deep Neural Network (DNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN), or Deep Q-Networks, etc., but are not limited to the examples described above.

Below, with reference to the attached drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily implement the present invention. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein.

Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

FIG. 1 is a diagram illustrating an in-house IoT environment in which the electronic device 100 and external devices are connected to each other.

In this disclosure, the electronic device 100 is described on the premise that it is a robot vacuum cleaner, but it can be used in various types of assistant robots or mobile devices, Augmented Reality (AR) devices, Virtual Reality (VR) devices, and surrounding environments operated for the convenience of users. It may be a device that detects and provides a certain service in a specific location or space. The electronic device 100 may be equipped with various types of sensors and neural network models for scanning space and detecting objects within the space. For example, the electronic device 100 may be equipped with at least one of an image sensor such as a camera, a Light Detection And Ranging (LiDAR) sensor such as a Laser Distance Sensor (LDS), and a Time of Flight (ToF) sensor. there is. The electronic device 100 may be equipped with at least one model such as a Deep Neural Network (DNN), a Convolution Neural Network (CNN), a Recurrent Neural Network (RNN), or a Bidirectional Recurrent Deep Neural Network (BRDNN), and can be used in combination. You can also use it.

External devices connected to the electronic device 100 may be a cloud server 200 and various types of IoT devices 300-1, 300-2, and 300-3. As shown in FIG. 1, IoT devices may be a butler robot (300-1), a pet robot (300-2), a smart home camera (300-3), etc., but are not limited thereto, and may include electronic devices ( It may be the same device as 100). The butler robot 300-1, the pet robot 300-2, and the smart home camera 300-3 can each scan the space and detect objects in the space using various types of sensors.

According to an embodiment of the present disclosure, the electronic device 100, the butler robot 300-1, the pet robot 300-2, and the smart home camera 300-3 collect spatial scan information or objects from each. Using the information, a spatial map can be created and stored as spatial information about a space including at least one object. The electronic device 100, the butler robot 300-1, the pet robot 300-2, and the smart home camera 300-3 transmit or receive space scan information, object information, or space maps to each other and store them. By doing this, we can share with each other.

Even if there are devices in the same space, the space is scanned from different viewpoints and with different angles of view depending on the location of each device, the performance or sensing range of each device, whether each device is fixed or mobile, and the operating behavior of each device. Because objects are detected, sensing information, including video or audio, acquired from one device can be usefully used to learn an artificial intelligence model mounted on another device.

According to an embodiment of the present disclosure, any one of the electronic device 100, the butler robot 300-1, the pet robot 300-2, and the smart home camera 300-3 is a master device or a server. device, and the remaining devices can be slave devices or client devices. A device corresponding to a master device or server device can receive, store, and manage space scan information, object information, or space maps from other IoT devices. A device corresponding to a master device or server device can classify and store and manage the received information by location. For example, a device corresponding to a master device or a server device can classify, collect, and manage space scan information, object information, or space maps depending on whether it is the same space, the same area, or the same area. A device corresponding to a master device or a server device can maintain the up-to-dateness and accuracy of information related to the location by updating the stored first information with second information corresponding to the same location.

According to an embodiment of the present disclosure, the electronic device 100, the butler robot 300-1, the pet robot 300-2, and the smart home camera 300-3 may use spatial scan information, object information, or spatial scan information. By transmitting the map to the cloud server 200, the spatial scan information, object information, or spatial map can be stored and managed through the cloud server 200. For example, when it is impossible to transmit spatial scan information, object information, or a spatial map to the electronic device 100 because the IoT devices are turned off or a specific function is running on the IoT device, the electronic device 100 is connected to the cloud server. Space scan information, object information, or space map can be requested and received from (200).

As shown in FIG. 1, the cloud server 200 performs spatial scans received from each of the electronic device 100, the butler robot 300-1, the pet robot 300-2, and the smart home camera 300-3. You can manage information, object information, or space maps, and monitor your home space. The cloud server 200 may store and manage space scan information, object information, or space maps collected from a plurality of IoT devices for each registered user's account or registered location. For example, the cloud server 200 may classify, collect, and manage space scan information, object information, or space maps depending on whether it is the same space or the same area. The cloud server 200 responds to requests from the electronic device 100, the butler robot 300-1, the pet robot 300-2, and the smart home camera 300-3 located within the home, and provides home information such as a space map. Information about space can be transmitted.

According to an embodiment of the present disclosure, instead of the cloud server 200, an AI hub (e.g., an AI speaker) located within the home receives, stores, and manages spatial scan information, object information, or spatial maps from IoT devices within the home. can do. The AI hub can store and manage spatial scan information, object information, or spatial maps collected from multiple IoT devices by space or zone within the home.

According to an embodiment of the present disclosure, the AI hub located within the home may store and manage spatial scan information, object information, or spatial maps together with the cloud server 200. For example, the AI hub may process spatial scan information or object information to create or manage a spatial map, or convert the data to protect personal information and transmit it to the cloud server 200. The cloud server 200 can process information received from the AI hub, store and manage spatial scan information, object information, or spatial maps, and transmit it to the AI hub.

According to one embodiment of the present disclosure, an electronic device 100, such as a robot vacuum cleaner, may use a spatial map to perform a task such as cleaning. To this end, the electronic device 100 can scan the space using various types of sensors and update the space map with the latest space scan information. The electronic device 100 includes not only directly sensed information, but also a cloud server 200, a butler robot (300-1), a pet robot (300-2), and a smart home camera (300-3) that are connected to each other in an in-house IoT environment. The spatial map stored in the electronic device 100 may be updated using part or all of the spatial map received from.

For example, when a robot vacuum cleaner is fully charged at a charging station to clean a space within the home, it can perform cleaning using a space map stored in the robot cleaner. The robot vacuum cleaner may use the most recently used space map to clean the same space. However, since the state of the space at the time of previous cleaning is different from the state of the current space, it is desirable to reflect the latest information about objects located in the space in the space map in order to perform efficient cleaning. To this end, the robot vacuum cleaner can start from the charging station, travel a major route in advance, and directly collect object information in the space. However, if pre-driving is performed, it takes more time for pre-driving, and the battery may be consumed more depending on the pre-driving. At this time, the robot cleaner may update the space map stored in the robot cleaner by receiving the latest space map from another robot cleaner or at least one external device located in the same space.

The robot vacuum cleaner may utilize part or all of the spatial map received from an external device. The robot cleaner can use the spatial map received from the same type of robot cleaner as is, or can use information about objects whose positions are expected to change frequently to update the spatial map. Even if the robot vacuum cleaner receives a spatial map from a heterogeneous device, it can utilize part or all of the spatial map for the same space to update the spatial map.

2A and 2B are flowcharts for explaining a spatial map.

Referring to FIG. 2A, it shows a hierarchical structure between a spatial map stored in the electronic device 100, which is a robot vacuum cleaner, and a plurality of layers constituting the spatial map. As shown in FIG. 2A, the spatial map may be composed of a base layer, a semantic map layer, and a real-time layer, but is not limited thereto, and layers may be added or subtracted depending on the characteristics of the task.

The base layer provides information about the basic structure of the entire space, including walls, columns, and passageways. By processing 3D point cloud data, matching the coordinate system, and storing the location, the base layer can provide 3D information in space, location information of objects, movement trajectory information, etc. The base layer acts as a base map and geometric map.

The semantic map layer is a layer that provides semantic information on top of the base layer. The user of the electronic device 100 can assign semantic information such as 'Room 1', 'Room 2', 'access restricted area', etc. to the basic structure of the entire space of the base layer and use it to perform tasks of the electronic device 100. You can. For example, if the electronic device 100 is a robot vacuum cleaner, the user can set semantic information in the semantic map layer so that only 'Room 2' is cleaned, or the robot cleaner does not clean the 'restricted access area'.

A real-time layer is a layer that provides information on at least one object in space. Objects can include both static and dynamic objects. In the present disclosure, a real-time layer may include a plurality of layers based on attribute information of an object and may have a hierarchical structure between the layers. As shown in Figure 2a, the real-time layer may include a first layer, a second layer, and a third layer, but is not limited thereto, and the number of layers may be added or subtracted depending on the classification criteria of the object's attribute information. there is. Looking at Figure 2a, it can be seen that the first layer includes a system closet and a built-in cabinet, the second layer includes a table and a sofa, and the third layer includes a chair.

Referring to FIG. 2B, various examples of real-time layers including a plurality of layers based on object attribute information are shown.

The attribute information of an object may be information that can be classified by combining objective criteria such as the type, shape, size, height, etc. of the object or a plurality of criteria. Additionally, since the property information of an object may vary depending on the user and environment, the property information can be entered by labeling each object.

According to an embodiment of the present disclosure, when the attribute information of an object is the object's mobility level (ML), the first layer includes an object corresponding to ML 1, and the second layer includes ML 2 and ML 3. Objects corresponding to are included, and the third layer may include objects corresponding to ML 4. The mobility level of an object can be determined by applying objective characteristics of the object to predetermined classification criteria for evaluating mobility. For example, ML 1 is for objects that cannot move, ML 2 is for objects that are mobile but mostly stay stationary, ML 3 is for objects that are mobile but move occasionally, and ML 4 are objects that are mobile but often move. Each corresponds to an object.

According to an embodiment of the present disclosure, when the attribute information of an object is the position movement cycle of the object, the first layer includes an object that has not moved its position within one month, and the second layer includes an object that has not moved its position within one month. Objects that have moved location are included, and the third layer may include objects that have moved location within one week. Unlike the mobility level, which is classified based on the objective characteristics of the object, the position movement cycle may be different even for the same object depending on the user using the object or the environment in which the object is located. For example, the object 'A' may be an object frequently used by the first user, while it may be an object rarely used by the second user. The object 'B' may be an object frequently used in the first place, while it may be an object rarely used in the second place.

According to an embodiment of the present disclosure, when the attribute information of an object is the height at which the object is located, the first layer includes objects corresponding to 1 m or less, and the second layer includes objects corresponding to 1 m or more and 2 m or less. Objects are included, and the third layer may include objects exceeding 2m.

According to an embodiment of the present disclosure, classification criteria for a plurality of layers included in a real-time layer may be defined by the user. For example, a user can create a space map that reflects the characteristics of a task by combining and setting attribute information of multiple types of objects for classification criteria. For example, in the case of a robot vacuum cleaner, it generally moves below a height of 50 cm, so there is no need to consider objects located higher than 1 m, such as lamps or picture frames on the wall. Therefore, the user directly sets the classification criteria to distinguish each layer, so that the first layer is ML 1 and includes objects located below 1 m, and the second layer is ML 2 or ML 3 and includes objects located below 1 m. Included, the third layer is ML 4, and objects located below 1m can be included.

FIGS. 3A, 3B, 3C, and 3D are diagrams for explaining a method of utilizing the layers that make up a spatial map.

Depending on the type of electronic device 100 and IoT devices or the characteristics of the task, the spatial maps used in each device may be different. The electronic device 100 may utilize the existing spatial map stored in the electronic device 100 as is, but if there is a change in the space in which the task is to be performed, the spatial map may be updated to reflect the change. The electronic device 100 may update the existing spatial map by receiving a spatial map that already reflects changes in space from at least one external device. The electronic device 100 may generate a new space map based on an existing space map.

Referring to FIG. 3A, the electronic device 100 can load an existing space map (hereinafter referred to as a first space map) that has been stored. The first spatial map consists of a base layer, a first layer, a second layer, and a third layer. Hereinafter, for convenience of explanation, the description will be made on the assumption that the first to third layers include objects according to the arbitrary classification criteria of FIG. 2B. If the first space map was created just a few minutes ago or there is no change in the space since the first space map was used, the electronic device 100 uses the first space map as is and creates a new space map (hereinafter referred to as the second space map). ), and the second spatial map can be used to perform a new task.

Referring to FIG. 3B, the electronic device 100 can load the stored first space map. When the electronic device 100 performs a task, when object information of ML 4 that moves frequently is not needed or only object information that has not moved for more than a week is used, a base layer among the layers constituting the first spatial map, The second spatial map can be obtained by selecting the first layer and the second layer or removing the third layer from the first spatial map.

Referring to FIG. 3C, the electronic device 100 can load the stored first space map. When the electronic device 100 performs a new task, only object information of ML 1 is needed or only object information that has not moved for more than one month is used, the base layer and the first layer among the layers constituting the first spatial map The second spatial map can be obtained by selecting or removing the second layer and third layer from the first spatial map.

Referring to FIG. 3D, the electronic device 100 can load the stored first space map. When the electronic device 100 performs a new task, when it is necessary to reflect the latest information on objects corresponding to ML 2, ML 3, and ML 4 that can be moved, the base among the layers constituting the first spatial map A second spatial map can be obtained by selecting the layer and the first layer or removing the second layer and the third layer from the first spatial map. Thereafter, the electronic device 100 may extract the second layer and the third layer from the spatial map received from the external device and reflect them in the second spatial map to obtain a third spatial map. Alternatively, using at least one sensor provided in the electronic device 100, objects corresponding to ML 2, ML 3, and ML 4 are detected and reflected in the second space map to obtain a third space map. You can.

Figure 4 is a flowchart illustrating a method of obtaining a spatial map according to an embodiment of the present disclosure.

The electronic device 100 may acquire a first space map (S410). The first space map may be composed of a plurality of layers based on attribute information of an object. The first spatial map may be generated in the electronic device 100 or may be received from an external device of the electronic device 100.

The electronic device 100 may determine whether an update to the first spatial map is necessary. (S420) For example, the electronic device 100 may determine whether an update of the first spatial map is necessary depending on the characteristics of the task. there is. A task refers to a task that the electronic device 100 is set to perform through a purpose unique to the electronic device 100 or a function that can be executed on the electronic device 100. Setting information related to task performance may be directly input into the electronic device 100 by the user or transmitted to the electronic device 100 through a terminal such as a mobile device or a dedicated remote control. For example, if the electronic device 100 is a robot cleaner, the task of the robot cleaner may be cleaning the home or an area set by the user, scheduled cleaning according to a reservation function, low-noise mode cleaning, etc. When information used to perform a task is insufficient, the electronic device 100 may determine that an update to the first space map is necessary. If the electronic device 100 needs the latest information about object information in a space where a task will be performed, it may determine that an update is necessary. Alternatively, the electronic device 100 may determine whether an update to the first space map is necessary according to the elapsed time from when the first space map was acquired or a set update cycle. When there is no need to update the first space map, the electronic device 100 may utilize the first space map as a second space map used to perform a task.

When an update to the first spatial map is required, the electronic device 100 may acquire object information. (S430) The electronic device 100 uses at least one sensor to directly obtain spatial scan information or object information. It can be collected. The electronic device 100 may receive part or all of a spatial map, spatial scan information, or object information from an external device.

The electronic device 100 may update the spatial scan information or object information in the first spatial map using the acquired spatial scan information or object information (S440). For example, the electronic device 100 may move. For frequent objects, the object information or spatial scan information of the location where the object was located can be newly acquired and updated in the first space map so that the latest location information is reflected.

The electronic device 100 may acquire a second spatial map. (S450) The electronic device 100 uses the first spatial map as is or creates a first spatial map in which some object information or some layers have been modified. The second spatial map can be obtained by utilizing or updating the first spatial map.

Meanwhile, the second space map used to perform the task may be transformed or created into an appropriate map depending on the function of the electronic device 100 or the characteristics of the task. For example, if the electronic device 100 is a robot vacuum cleaner, it can generate a navigation map based on a spatial map and perform cleaning along a movement path provided by the navigation map.

FIG. 5 is a flowchart illustrating a method of controlling the electronic device 100 using spatial information according to an embodiment of the present disclosure.

The electronic device 100 selects an object that interferes with the task among objects located in the space corresponding to the task, based on spatial information about a space containing at least one object and a task that the electronic device 100 is set to perform. (S510) Spatial information may be a spatial map of a space including at least one object. The task that the electronic device 100 is set to perform may be determined by the user entering or setting in the electronic device 100 the purpose of the electronic device 100 or the task to be processed through functions executable in the electronic device 100. You can. The user of the electronic device 100 may directly input task-related settings into the electronic device 100 or transmit a task-related control command to the electronic device 100 through a user terminal. For example, if the electronic device 100 is a robot cleaner, the task that the robot cleaner is set to perform may be cleaning according to a space designated by the user as the location where the task is to be performed, a time to perform the task, or an operation mode. For example, the user of the electronic device 100 can set the robot cleaner to perform cleaning in a specific area, scheduled cleaning according to a reservation function, or low-noise mode cleaning according to the operation mode. Hereinafter, it will be described with reference to FIG. 6.

FIG. 6 is a detailed flowchart illustrating the step of selecting an object that interferes with a task among objects located in a space corresponding to a task according to an embodiment of the present disclosure.

The electronic device 100 may acquire a spatial map of the space as spatial information about a space including at least one object. (S610) The electronic device 100 may use the spatial map stored in the electronic device 100 and A spatial map may be obtained based on at least one of spatial maps received from an external device capable of communicating with the device 100. For example, the electronic device 100 may acquire a spatial map according to the method for obtaining a spatial map of FIG. 4 described above.

The electronic device 100 may use the spatial map to analyze a prediction of processing of a task that the electronic device 100 is set to perform. (S620) The electronic device 100 may analyze a prediction of the processing of a task that the electronic device 100 is set to perform. You can check the task and obtain a spatial map corresponding to the location where the task will be performed. The electronic device 100 can predict and analyze various cases of processing tasks in the spatial map. The electronic device 100 predicts task processing for a plurality of distinct cases based on the location of at least one branch point and at least one object in the virtual movement path of the electronic device 100 for performing the task. The results can be compared and analyzed.

According to an embodiment of the present disclosure, the electronic device 100 may create a task processing model that considers whether the direction changes at each junction or the location of each object moves. For example, the task processing model uses the location of the electronic device 100 as input, each layer of the task processing model may correspond to a branch point or the location of an object on the virtual movement path, and each node included in each layer This may be the location of the electronic device 100 on a virtual movement path depending on whether the direction changes or the object moves at the corresponding location. The task processing model may be designed so that when moving from each node included in each layer constituting the task processing model to the next node, a higher weight is applied to the direction in which the positions on the virtual movement path do not overlap. The task processing model passes through at least one node included in each of all layers constituting the task processing model, and when it reaches the last layer, it can be determined that processing of the task is complete. Based on the location corresponding to each node of the task processing model, the virtual movement path used to process the task can be tracked. For each tracked virtual movement path, the time required to perform the task, the amount of battery required to perform the task, and the degree of completion of the task can be analyzed.

According to an embodiment of the present disclosure, the electronic device 100 may generate various scenarios for processing a task by considering a change in direction at each junction or whether the location of each object moves. The electronic device 100 may run and analyze a simulation of a task in the acquired spatial map for each scenario. The electronic device 100 may analyze the time required to perform a task, the amount of battery required to perform the task, the degree of completion of the task, etc., for each scenario.

For example, if the electronic device 100 is a robot vacuum cleaner, the robot vacuum cleaner determines the time required for cleaning and the time required to perform cleaning for each virtual movement path tracked using a task processing model or for each scenario in which the simulation is executed. You can compare battery capacity, cleaning completion, etc.

The electronic device 100 may determine at least one object that interferes with the task based on the analysis result of predicting the processing of the task. (S630) The electronic device 100 determines the number of objects required to process the task. The best case can be selected based on at least one of the following criteria: time, battery capacity, and task completion. For example, the case where the time required to process a task or the battery capacity is the lowest, the degree of task completion is highest, or the case where the weighted average given weights for each criterion is the highest may be selected as the best case. The electronic device 100 can backtrack the virtual movement path corresponding to the best case and check whether there has been a positional movement of the object on the virtual movement path. At this time, at least one object whose position has moved may be determined to be an object that interferes with the task.

For example, if the electronic device 100 is a robot vacuum cleaner, the robot cleaner performs cleaning according to at least one criterion such as the time required to complete cleaning, the amount of battery required, or the area where cleaning was completed among the total area. The best case can be selected. The robot vacuum cleaner backtracks the virtual movement path corresponding to the best case, checks whether there has been a positional movement of the object on the virtual movement path, and, as a result of the positional movement, at least one object that contributed to the selection of the best case interferes with the task. It can be determined as an object.

Referring again to FIG. 5, the electronic device 100 may provide object movement guide information according to the properties of the selected object to the user of the electronic device 100 (S520). Refer to FIGS. 7 to 13 below. This explains.

FIG. 7 is a detailed flowchart illustrating the step of providing object movement guide information according to attribute information of an object selected as an object interfering with a task according to an embodiment of the present disclosure.

The electronic device 100 may check attribute information of an object selected as an object requiring movement in the space where the task will be performed. (S710) For example, the electronic device 100 may check the type of the selected object or the selected object. The attribute information of the object can be confirmed based on at least one of information on the layer to which the object belongs and the label of the selected object. The electronic device 100 may check at least one attribute information, such as mobility level, position movement cycle, height, and size, of the selected object.

The electronic device 100 may execute a move request process corresponding to attribute information of the identified objects for all selected objects. The move request process may include procedures for providing the user with a move request algorithm execution result corresponding to the attribute information of the object and confirming a response from the user.

As shown in FIG. 7, if the attribute information of the confirmed object is the object's mobility level as an example, the electronic device 100 may check the mobility level of the object selected as an object interfering with the task. .(S720) For convenience of explanation, it is assumed that the object's attribute information is the object's mobility level, but it is not limited to this and may be other types of attribute information such as the object's position movement cycle, height, size, etc. Additionally, the classification of the confirmed attribute information is not limited to the three cases as shown in FIG. 7, and there may be an appropriate number of classifications depending on each attribute information.

If the mobility level of the selected object is 4, the first movement request process can be executed (S730). This will be described below with reference to FIG. 8.

FIG. 8 is a diagram for explaining a first move request process according to an embodiment of the present disclosure.

The electronic device 100 may provide the user with analysis results and object movement guide information obtained by analyzing predictions about task processing (S810). For example, the electronic device 100 provides prediction results about task processing. Among these, the analysis result of analyzing the difference between the case in which an object interfering with the task is moved and the case in which the object is not moved, as well as object movement guide information requesting movement of the object interfering with the task, can be transmitted to the user terminal. If the electronic device 100 is a robot cleaner, analysis results that analyze differences such as how much cleaning time is shortened or how much the area that can be cleaned is increased when reflecting the movement of the selected object can be provided to the user. The electronic device 100 may request the user to move the selected object and may provide the user with information about an appropriate location for movement.

The electronic device 100 may receive the user's response after providing analysis results of predictions for task processing and object movement guide information. (S820) The electronic device 100 confirms the response from the user. , the first move request process may be terminated. The electronic device 100 receives a response regarding the movement of the object from the user terminal, provides a virtual simulation analysis result to the user terminal, and provides a response regarding the movement of the object when a predetermined time has elapsed after requesting the movement of the object. It can be processed as received.

Figure 9 is a diagram for explaining an example of providing object movement guide information to a user according to an embodiment of the present disclosure.

The electronic device 100 may transmit analysis results of predictions for task processing and object movement guide information to the user's user terminal 400 . For example, the electronic device 100 analyzes the difference between cases in which an object that interferes with the task is moved and a case in which the object that interferes with the task is not moved among the prediction results for processing the task and the movement of the object that interferes with the task. Object movement guide information requesting may be transmitted to the user terminal 400.

Referring to FIG. 9 , the electronic device 100 is a robot vacuum cleaner, and as a result of analyzing predictions for task processing, a bag on the living room floor is selected as an object that interferes with the task.

As shown in Figure 9, the robot vacuum cleaner moves the bag on the living room floor along with the analysis results of the difference in how much cleaning time is reduced when reflecting the movement of the bag selected as an object that interferes with the task. A message requesting the user can be transmitted to the user terminal 400. The message transmitted to the user terminal 400 may further include information about an appropriate location to move the bag.

The robot cleaner may transmit analysis results of predictions for task processing and object movement guide information to the user terminal 400 and then receive a user's response. The robot vacuum cleaner receives a response from the user terminal 400 indicating that the request to move the bag has been confirmed, or when a predetermined time has elapsed after requesting to move the bag, it processes the response as having received it from the user and determines that the bag has been moved. can do.

FIG. 10 is a diagram illustrating another example of providing object movement guide information to a user according to an embodiment of the present disclosure.

Unlike the previous embodiment of FIG. 9, the electronic device 100 may output analysis results of predictions for task processing and object movement guide information in the form of voice. For example, the electronic device 100 analyzes the difference between cases in which an object that interferes with the task is moved and a case in which the object that interferes with the task is not moved among the prediction results for processing the task and the movement of the object that interferes with the task. The requested object movement guide information can be delivered to the user through voice.

In the situation previously described in FIG. 9, as shown in FIG. 10, the robot cleaner is shown in the living room along with the analysis results of the difference in how much the cleaning time is reduced when reflecting the movement of the bag selected as an object that interferes with the task. It is possible to convey to the user in the form of a voice how to move the bag on the floor. The robot vacuum cleaner can further convey information about the appropriate location to move the bag through voice.

The robot vacuum cleaner can output analysis results of predictions for task processing and object movement guide information in the form of voice and then receive a user's response. When the robot vacuum cleaner receives a voice from the user indicating that the request to move the bag has been confirmed, or when a predetermined time has elapsed after requesting to move the bag, the robot vacuum cleaner may process it as having received a response from the user and determine that the bag has been moved.

Referring again to FIG. 7, if the mobility level of the selected object is 2 or 3, the second movement request process can be executed (S740). This will be described below with reference to FIG. 11.

FIG. 11 is a diagram for explaining a second move request process according to an embodiment of the present disclosure.

When the second movement request process starts, the electronic device 100 may generate a 3D spatial map of the area where the selected object is located using the spatial map (S1110). The electronic device 100 may generate the 3D spatial map. While creating, it is possible to secure an area where the selected object can be moved and a space where the electronic device 100 can move by reflecting the size of the object selected as an object interfering with the task.

The electronic device 100 may select candidate locations to move an object selected as an object that interferes with the task in the generated 3D space map (S1120). For example, the candidate location is the current location of the selected object. The closer the location is to and does not overlap with the user's main movement line, the higher the priority may be determined.

Figure 12 is a diagram for explaining a process of selecting candidate locations to move the selected object according to an embodiment of the present disclosure.

Referring to FIG. 12, when the electronic device 100 is a robot vacuum cleaner, a three-dimensional space map corresponding to the location of the robot cleaner is generated, and candidate locations for moving a table selected as an object interfering with the task are selected. It shows the process. When the robot cleaner creates a 3D space map, it can check the area where the table can move based on the space where the robot cleaner will fit and the size of the table. In order to perform cleaning, the robot cleaner secures in advance the space that the robot cleaner must pass through and can check areas where the table can move. The robot vacuum cleaner can select a candidate location for the table in areas where the table can move, excluding places that are passageways for the robot cleaner to enter and exit. According to a predetermined formula, the candidate position of the table may be given a higher score as it is closer to the current position of the table. As shown in Figure 12, the robot cleaner selected three candidate positions for moving the table, and it can be seen that scores of '0.9', '0.4', and '0.3' were assigned, respectively. The number of candidate locations can be set in advance, and the criteria for the minimum score to become a candidate location can be adjusted.

Referring again to FIG. 11, the electronic device 100 can acquire an image when the selected object moves for each candidate location through compositing the 3D space map and the image of the selected object. (S1130) The electronic device ( 100) can use image synthesis technology to generate an image showing how an object selected as an object interfering with a task moves to a space corresponding to a candidate location in a 3D space map.

The electronic device 100 can obtain an image evaluation result through the image evaluation model by inputting an image when an object moves into the image evaluation model. (S1140) The image evaluation model can be used to determine the purpose of the location where the electronic device 100 is located or It may be a model that performs a predetermined evaluation according to the user's settings. According to one embodiment, the image evaluation model may be a model that inputs a composite image of the appearance of an object when moving and outputs a result of scoring the interior aesthetic value, but is not limited to this. According to one embodiment, the image evaluation model may be a model that inputs a composite image of the appearance of an object when moving and outputs a result of scoring the safety level of the space.

The electronic device 100 may provide object movement guide information according to the image evaluation results. (S1150) For example, the electronic device 100 may move an object selected as an object that interferes with the task to a candidate location. In this case, a recommended location can be provided to the user based on the video evaluation results, such as how much interior aesthetic value it has or how safe it is. Based on the image evaluation result, the electronic device 100 may determine a candidate location with a high evaluation score among candidate locations as a recommended location. Alternatively, the electronic device 100 may provide a predetermined number of candidate locations with high evaluation scores for moving the selected object as recommended locations to the user.

FIG. 13 is a diagram illustrating an example of providing object movement guide information to a user according to an image evaluation result according to an embodiment of the present disclosure.

Referring to FIG. 13, when the electronic device 100 is a robot cleaner, the robot cleaner moves the table selected as an object interfering with the task to the first candidate location, the second candidate location, and the third candidate location, respectively. You can obtain a video showing the appearance of the poem. The robot vacuum cleaner can generate an image where the table is located at each candidate location by combining an image representing a 3D space map and an image of a table selected as an object that interferes with the task.

The robot vacuum cleaner can obtain image evaluation results through the image evaluation model by inputting images for each candidate location obtained through image synthesis into the image evaluation model. The robot vacuum cleaner can provide object movement guide information according to the image evaluation results. As shown in FIG. 13, the robot cleaner may transmit a recommended location according to the image evaluation result to the user terminal 400. The robot cleaner may transmit an image corresponding to a recommended location among images for each candidate location to the user terminal 400, and indicate the direction and distance of movement from the current location to the recommended location in the image.

Referring again to FIG. 11, the electronic device 100 may receive the user's response after providing object movement guide information according to the image evaluation result. (S1160) The electronic device 100 confirms the response from the user. , the second move request process may be terminated. The electronic device 100 receives a response regarding the movement of the object from the user terminal 400, provides object movement guide information according to the image evaluation result to the user terminal 400, and waits a predetermined time after requesting movement of the object. Once this has elapsed, it can be processed as having received a response regarding the movement of the object.

Referring again to FIG. 7, when the object's mobility level is 1, the object cannot be moved, so the move request process corresponding to the object can be processed as being executed even without executing a predetermined move request process.

The electronic device 100 can check whether the movement request process has been executed for all selected objects. (S750) The electronic device 100 checks the attribute information of the objects for all selected objects, and determines whether the movement request process has been executed. The operation can be repeated until the request process is executed.

Meanwhile, the electronic device 100 creates a three-dimensional spatial map of the area where the object selected as an object interfering with the task is located, and the electronic device 100 can move the object selected as an object interfering with the task. The area in which the electronic device 100 is located and the space in which the electronic device 100 can move can be confirmed. The electronic device 100 may select at least one candidate location from the generated 3D space map to move an object that is selected as an object that interferes with the task. The electronic device 100 may move an object selected as an object interfering with a task to one of the candidate locations. For example, if the user has previously set the electronic device 100 not to receive object movement guide information, or if the object is sufficiently movable by the electronic device 100, the electronic device 100 may interfere with the task. The object selected as the object can be moved to the candidate location. When moving in this way, the electronic device 100 uses the location where the object was moved as the starting point of the movement path used to perform the task, obtains the movement path used to perform the task, or executes the task at the location where the object was moved. You can also perform the procedure again by reselecting the object that is interfering with the process.

Referring again to FIG. 5, the electronic device 100 may obtain a movement path used to perform a task based on the user's response corresponding to the object movement guide information (S530). Hereinafter, referring to FIG. 14. Explain.

FIG. 14 is a detailed flowchart illustrating the step of determining a movement path used to perform a task according to an embodiment of the present disclosure.

The electronic device 100 can confirm the moved object among the selected objects based on the user's response corresponding to the object movement guide information. (S1410) The electronic device 100 can check not only the actually moved object but also the object from the user. If the movement of is confirmed, the selected object can be processed as having been moved. When the electronic device 100 receives a response from the user refusing to move the selected object, or when a predetermined time elapses without a response from the user after requesting the user terminal to move the selected object, the selected object is not moved. It can be treated as not.

The electronic device 100 may determine a movement path that reflects the object moved in the space corresponding to the task (S1420). For example, the electronic device 100 may generate a navigation map. The navigation map may provide a movement route used by the electronic device 100 to perform a task.

Referring again to FIG. 5, the electronic device 100 may drive the electronic device 100 according to the movement path. (S540) The electronic device 100 moves along the movement path provided by the navigation map and performs the task. can be performed.

15 and 16 are block diagrams showing the configuration of an electronic device 100 that uses spatial information according to an embodiment of the present disclosure.

Referring to FIG. 15, the electronic device 100 according to an embodiment may include a memory 110, a processor 120, and a sensing unit 130, but is not limited thereto and has a more general-purpose configuration. can be added For example, as shown in FIG. 16, the electronic device 100 includes, in addition to a memory 110, a processor 120, and a sensing unit 130, an input/output unit 140, a communication unit 150, and a driver 160. It may further include. Hereinafter, with reference to FIGS. 15 and 16, each configuration will be described in detail.

The memory 110 according to one embodiment may store a program for processing and control of the processor 120, and may store data (e.g., spatial information, Object information, spatial maps, movement paths, etc.) can be stored. The memory 110 may store instructions, data structures, and program code that the processor 120 can read. In the disclosed embodiments, operations performed by the processor 120 may be implemented by executing instructions or codes of a program stored in the memory 110.

The memory 110 according to one embodiment may be a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (for example, SD or XD). memory, etc.), and may include at least one of ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, and optical disk. It may include non-volatile memory, such as RAM (Random Access Memory) or SRAM (Static Random Access Memory).

The memory 110 according to an embodiment may store one or more instructions and/or programs that control the electronic device 100 using spatial information to perform a task. For example, the memory 110 may store a spatial information management module, a task processing module, a driving module, etc.

The processor 120 according to one embodiment may control operations or functions of the electronic device 100 to perform tasks by executing instructions or programmed software modules stored in the memory 110. The processor 120 may be comprised of hardware components that perform arithmetic, logic, input/output operations, and signal processing. The processor 120 may execute one or more instructions stored in the memory 110 to control overall operations in which the electronic device 100 performs a task using spatial information. The processor 120 can control the sensing unit 130, the input/output unit 140, the communication unit 150, and the driving unit 160 by executing programs stored in the memory 110.

The processor 120 according to one embodiment includes, for example, a Central Processing Unit, a microprocessor, a Graphics Processing Unit, Application Specific Integrated Circuits (ASICs), and Digital Signals (DSPs). Processors), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), Application Processors, Neural Processing Units, or hardware specialized for processing artificial intelligence models. It may consist of at least one of the designed artificial intelligence processors, but is not limited to this. Each processor constituting the processor 120 may be a dedicated processor for performing a certain function.

An artificial intelligence (AI) processor according to one embodiment may use an artificial intelligence (AI) model to perform computation and control to process a task that the electronic device 100 is set to perform. The AI processor may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as part of a general-purpose processor (eg, CPU or application processor) or graphics-specific processor (eg, GPU) and used in the electronic device 100. It may be mounted on .

The sensing unit 130 according to one embodiment may include a plurality of sensors configured to detect information about the environment surrounding the electronic device 100. For example, the sensing unit 130 includes an image sensor 131, a LiDAR (Light Detection And Ranging) sensor 132, an infrared sensor 133, an ultrasonic sensor 134, and a Time of Flight (ToF) sensor. It may include a sensor 135, a gyro sensor 136, etc., but is not limited thereto.

The image sensor 131 according to one embodiment may include a stereo camera, a mono camera, a wide angle camera, an around view camera, or a 3D vision sensor.

The LiDAR sensor 132 can detect the distance to the object and various physical properties by shining a laser on the target. The LiDAR sensor 132 can be used to detect surrounding objects, geographical features, etc. and model them into 3D images.

The infrared sensor 133 can be either an active infrared sensor that radiates infrared rays and detects changes by blocking the light, or a passive infrared sensor that does not have a light emitter and only detects changes in infrared rays received from the outside world. For example, an infrared proximity sensor may be installed around the wheels of the electronic device 100 and used as a fall prevention sensor by emitting infrared rays to the floor and then receiving them.

The ultrasonic sensor 134 can measure the distance to an object using ultrasonic waves, and can emit and detect ultrasonic pulses that convey information about the proximity of the object. The ultrasonic sensor 134 can be used to detect nearby objects and transparent objects.

The ToF sensor 135 calculates the distance at which light emitted from an object bounces and returns in terms of time, and can obtain the three-dimensional effect, movement, and spatial information of the object. The ToF sensor 135 enables high-level object recognition in complex spaces, dark places, and even obstacles in front of the eyes, allowing the electronic device 100 to avoid obstacles.

Gyro sensor 136 can detect angular velocity. The gyro sensor 136 can be used to measure the position and set the direction of the electronic device 100.

According to one embodiment, the sensing unit 130 may be used to generate spatial information about a space including at least one object using at least one sensor. For example, the electronic device 100 may include the same type or By obtaining spatial scan information or object information using a plurality of heterogeneous sensors, spatial information about a space including at least one object can be obtained.

As shown in FIG. 16, the electronic device 140 may further include an input/output unit 140, a communication unit 150, and a driver 160. Although not shown in FIG. 16, the electronic device 140 may further include components such as a power unit. You can.

The input/output unit 140 may include an input unit 141 and an output unit 142. The input/output unit 140 may be in a form where the input unit 141 and the output unit 142 are separated, or may be in an integrated form, such as a touch screen. The input/output unit 140 may receive input information from a user and provide output information to the user.

The input unit 141 may refer to a means through which a user inputs data to control the electronic device 100. For example, the input unit 141 may include a key pad, a touch panel (contact capacitance type, pressure resistance type, infrared detection type, surface ultrasonic conduction type, integral tension measurement type, piezo effect type, etc. ), etc. In addition, the input unit 141 may include a jog wheel, a jog switch, etc., but is not limited thereto.

The output unit 142 may output an audio signal, a video signal, or a vibration signal, and the output unit 142 may include a display unit, a sound output unit, and a vibration motor.

The display unit may display information processed by the electronic device 100. For example, the display unit may display a user interface for receiving user input. When the display unit and the touch pad form a layered structure to form a touch screen, the display unit can be used as an input device in addition to an output device. The display unit includes liquid crystal display, thin film transistor-liquid crystal display, organic light-emitting diode, flexible display, and 3D display. It can contain at least one. Depending on the implementation form of the electronic device 100, the electronic device 100 may include two or more display units.

The audio output unit may output audio data stored in the memory 110. The sound output unit may output sound signals related to functions performed in the electronic device 100. The sound output unit may include a speaker, buzzer, etc.

The vibration motor can output a vibration signal. For example, the vibration motor may output a vibration signal corresponding to the output of audio data or video data. The vibration motor can output a vibration signal when a touch is input to the touch screen.

The communication unit 150 is one that enables the electronic device 100 to communicate with external devices such as the cloud server 200, IoT devices 300-1, ??, 300-3, and the user terminal 400. It may include the above components. For example, the communication unit 150 may include a short-range wireless communication unit 151, a mobile communication unit 152, etc., but is not limited thereto.

The short-range wireless communication unit 151 includes a Bluetooth communication unit, a Bluetooth Low Energy (BLE) communication unit, a Near Field Communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, and an Ant+ communication unit. , WFD (Wi-Fi Direct) communication unit, UWB (ultra wideband) communication unit, infrared (IrDA (infrared Data Association) communication unit), microwave (uWave) communication unit, etc., but is not limited thereto.

The mobile communication unit 152 transmits and receives wireless signals with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to voice call signals, video call signals, or text/multimedia message transmission and reception.

The driving unit 160 may include components used for driving (driving) the electronic device 100 and operating devices within the electronic device 100. When the electronic device 100 is a robot cleaner, the driving unit 160 may include a suction unit, a traveling unit, etc., but is not limited thereto.

The suction unit functions to collect dust from the floor while sucking in air, and may include, but is not limited to, a rotating brush or broom, a rotating brush motor, an air intake port, a filter, a dust collection chamber, and an air outlet. The suction unit may additionally be equipped with a rotating brush structure that can sweep away dust from corners.

The driving unit may include, but is not limited to, a motor that rotates each wheel installed in the electronic device 100 and a timing belt installed to transmit power generated from the wheels.

According to an embodiment of the present disclosure, the processor 120 executes one or more instructions stored in the memory 110, thereby generating spatial information and electronic information about a space including at least one object obtained through the sensing unit 130. Based on the task that the device 100 is set to perform, an object that interferes with the task may be selected from among objects located in a space corresponding to the task.

The processor 120 may obtain a spatial map as spatial information by executing one or more instructions stored in the memory 110. The processor 120 executes one or more instructions stored in the memory 110, thereby generating at least one of the first spatial map stored in the electronic device 100 and the second spatial map received from an external device capable of communicating with the electronic device 100. Based on one, a spatial map can be obtained. The spatial map may include a plurality of layers based on attribute information of objects.

The processor 120 may analyze a prediction result for processing a task using the spatial map obtained by executing one or more instructions stored in the memory 110. The processor 120 executes one or more instructions stored in the memory 110, thereby generating a plurality of commands divided based on the location of at least one branch and at least one object on the virtual movement path of the electronic device 100 for performing the task. The predicted results for task processing according to the movement paths can be compared and analyzed. The processor 120 may determine at least one object that interferes with the task based on the analysis result.

According to an embodiment of the present disclosure, the processor 120 executes one or more instructions stored in the memory 110 to provide object movement guide information according to the attribute information of the object selected as an object interfering with the task to the electronic device ( It can be provided to 100) users. The processor 120 may check attribute information of an object selected as an object interfering with a task, execute a movement request process corresponding to the attribute information of the confirmed object, and provide object movement guide information to the user.

According to one embodiment, the processor 120 executes one or more instructions stored in the memory 110, thereby providing analysis results of predictions for task processing and object movement guide information to the user through the communication unit 150. It can be transmitted to the terminal 400. According to one embodiment, the processor 120 executes one or more instructions stored in the memory 110, thereby selecting a candidate for moving the selected object in the three-dimensional space map of the area where the object that is selected as an obstacle to the task is located. Locations can be selected. The processor 120 may obtain an image of the movement of an object selected as an object interfering with the task for each candidate location, input the acquired image into an image evaluation model, and evaluate the image through the image evaluation model. The processor 120 may transmit object movement guide information according to the image evaluation result to the user's user terminal 400 through the communication unit 150.

According to one embodiment of the present disclosure, the processor 120 executes one or more instructions stored in the memory 110 to determine the movement path used to perform the task based on the user's response corresponding to the object movement guide information. You can decide. According to one embodiment, based on the user's response, the processor 120 identifies the moved object among the objects selected as objects interfering with the task, and creates a movement path reflecting the moved object in the space corresponding to the task. It can be obtained.

According to an embodiment of the present disclosure, the processor 120 may drive the electronic device 100 according to the determined movement path by executing one or more instructions stored in the memory 110. If an unexpected object or an object judged to have been moved is detected while the electronic device 100 is driving along the movement path, the processor 120 bypasses such an object and then moves the electronic device 100 again along the movement path. You can run it or notify the user of the existence of the object.

Meanwhile, embodiments of the present disclosure may also be implemented in the form of a recording medium containing instructions executable by a computer, such as program modules executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and non-volatile media, removable and non-removable media. Additionally, computer-readable media may include computer storage media and communication media. Computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Communication media typically may include computer-readable instructions, data structures, or other data, such as modulated data signals, or program modules.

Additionally, computer-readable storage media may be provided in the form of non-transitory storage media. Here, 'non-transitory storage medium' only means that it is a tangible device and does not contain signals (e.g. electromagnetic waves). This term refers to cases where data is semi-permanently stored in a storage medium and temporary storage media. It does not distinguish between cases where it is stored as . For example, a 'non-transitory storage medium' may include a buffer where data is temporarily stored.

According to one embodiment, methods according to various embodiments disclosed in this document may be provided and included in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. A computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store or between two user devices (e.g. smartphones). It may be distributed in person or online (e.g., downloaded or uploaded). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) is stored on a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server. It can be temporarily stored or created temporarily.

According to an embodiment of the present disclosure, a method of controlling an electronic device 100 using spatial information is provided. A method of controlling the electronic device 100 using spatial information is based on spatial information about a space containing at least one object and a task that the electronic device 100 is set to perform, and an object located in the space corresponding to the task. It includes a step (S510) of selecting an object that interferes with the task. Additionally, the method of controlling the electronic device 100 using spatial information includes providing object movement guide information according to attribute information of the selected object to the user of the electronic device 100 (S520). Additionally, the method of controlling the electronic device 100 using spatial information includes determining a movement path used to perform a task based on a user's response corresponding to object movement guide information (S530). Additionally, the method of controlling the electronic device 100 using spatial information includes driving the electronic device 100 according to the determined movement path (S540).

Additionally, according to an embodiment of the present disclosure, the step of selecting an object that interferes with the task (S510) includes the step of acquiring a spatial map as spatial information (S610). Additionally, the step of selecting an object that interferes with the task (S510) includes the step of analyzing a prediction about processing of the task using the obtained spatial map (S620). Additionally, the step of selecting an object that interferes with the task (S510) includes a step of determining at least one object that interferes with the task based on the analysis result (S630).

In addition, the step of acquiring a spatial map (S610) is based on at least one of the first spatial map stored in the electronic device 100 and the second spatial map received from an external device capable of communicating with the electronic device 100, Obtain the map.

In addition, the step of analyzing the prediction (S620) is a task according to a plurality of movement paths divided based on the location of at least one branch and at least one object on the virtual movement path of the electronic device 100 for performing the task. Compare and analyze the predicted results for processing.

Additionally, according to an embodiment of the present disclosure, the step of providing object movement guide information to the user of the electronic device 100 (S520) includes the step of checking attribute information of the selected object (S710). In addition, the step of providing object movement guide information to the user of the electronic device 100 (S520) includes executing a movement request process corresponding to the attribute information of the identified object and providing object movement guide information to the user (S720). , S730, S740, S750);

In addition, the step of providing to the user (S730) includes the step of transmitting the analysis result of analyzing the prediction of the processing of the task and the object movement guide information to the user's user terminal (S810).

In addition, the step of providing to use (S740) includes the step of selecting candidate positions to move the selected object in the 3D spatial map of the area where the selected object is located (S1110, S1120). In addition, the step of providing to the user (S740) includes the step of acquiring an image when the object selected for each candidate location moves (S1130). Additionally, the step of providing for use (S740) includes inputting the acquired image into an image evaluation model and evaluating the image through the image evaluation model (S1140). Additionally, the step of providing to the user (S740) includes the step of transmitting object movement guide information according to the image evaluation results to the user's user terminal (S1150).

Additionally, according to an embodiment of the present disclosure, the step of determining the movement path (S530) includes the step of confirming the moved object among the selected objects based on the user's response (S1410). Additionally, the step of determining the movement path (S530) includes the step of obtaining a movement path that reflects the object moved in the space corresponding to the task (S1420).

Additionally, according to an embodiment of the present disclosure, the electronic device 100 is a robot cleaner.

According to one embodiment of the present disclosure, a computer-readable recording medium on which a program for executing the above-described method is recorded can be provided.

According to an embodiment of the present disclosure, an electronic device 100 that uses spatial information is provided. The electronic device 100 that uses spatial information includes a memory 110, a processor 120 that executes one or more instructions stored in the memory 110, and a sensing unit 130. The processor 120 executes one or more instructions, based on spatial information about a space containing at least one object obtained through the sensing unit 130 and a task that the electronic device 100 is set to perform. Among the objects located in the corresponding space, select the object that interferes with the task. Additionally, the processor 120 provides object movement guide information according to attribute information of the selected object to the user of the electronic device 100 by executing one or more instructions. Additionally, the processor 120 determines a movement path used to perform a task based on the user's response corresponding to the object movement guide information by executing one or more instructions. Additionally, the processor 120 drives the electronic device 100 according to the determined movement path by executing one or more instructions.

Additionally, according to an embodiment of the present disclosure, the processor 120 obtains a spatial map as spatial information by executing one or more instructions. Additionally, the processor 120 analyzes a prediction for processing of a task using the spatial map obtained by executing one or more instructions. Additionally, the processor 120 determines at least one object that interferes with the task based on the analysis result by executing one or more instructions.

Additionally, the processor 120 executes one or more instructions based on at least one of a first spatial map stored in the electronic device 100 and a second spatial map received from an external device capable of communicating with the electronic device 100. , obtain the spatial map.

Additionally, the processor 120 executes one or more instructions, thereby creating a plurality of movement paths divided based on the location of at least one branch and at least one object on the virtual movement path of the electronic device 100 for performing the task. Compare and analyze the predicted results for processing tasks according to .

Additionally, the spatial map includes a plurality of layers based on attribute information of objects.

Additionally, according to an embodiment of the present disclosure, the processor 120 verifies attribute information of the selected object by executing one or more instructions. Additionally, the processor 120 executes a movement request process corresponding to attribute information of the identified object by executing one or more instructions, and provides object movement guide information to the user.

Additionally, the electronic device 100 that uses spatial information further includes a communication unit 150. Additionally, the processor 120 executes one or more instructions and transmits analysis results of predictions for task processing and object movement guide information to the user's user terminal 400 through the communication unit 150.

Additionally, the electronic device 100 that uses spatial information further includes a communication unit 150. Additionally, the processor 120 executes one or more instructions to select candidate locations to move the selected object in the 3D spatial map of the area where the selected object is located. Additionally, the processor 120 executes one or more instructions to obtain an image when an object selected for each candidate location moves. Additionally, the processor 120 inputs the acquired image into an image evaluation model by executing one or more instructions, and evaluates the image through the image evaluation model. Additionally, the processor 120 transmits object movement guide information according to the image evaluation results to the user's user terminal 400 through the communication unit 150 by executing one or more instructions.

Additionally, according to an embodiment of the present disclosure, the processor 120 executes one or more instructions to confirm the moved object among the selected objects based on the user's response. Additionally, the processor 120 executes one or more instructions to obtain a movement path reflecting the moved object in the space corresponding to the task.

Additionally, according to an embodiment of the present disclosure, the electronic device 100 is a robot cleaner.

The foregoing description of the present disclosure is for illustrative purposes, and a person skilled in the art to which the present disclosure pertains will understand that the present disclosure can be easily modified into another specific form without changing its technical idea or essential features. will be. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as single may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present disclosure. do.

Claims (20)

Based on spatial information about a space including at least one object and a task that the electronic device 100 is set to perform, selecting an object that interferes with the task among objects located in a space corresponding to the task ( S510);
Providing object movement guide information according to attribute information of the selected object to the user of the electronic device 100 (S520);
determining a movement path used to perform the task based on the user's response corresponding to the object movement guide information (S530); and
Driving the electronic device 100 according to the determined movement path (S540);
A method of controlling the electronic device 100 using spatial information, including. According to claim 1,
The step of selecting an object that interferes with the task (S510) is,
Obtaining a spatial map using the spatial information (S610);
Analyzing a prediction for processing of the task using the obtained spatial map (S620); and
Based on the analysis result, determining at least one object that interferes with the task (S630);
Method, including. According to claim 1 or 2,
The step of acquiring the spatial map (S610) is,
A method of acquiring the spatial map based on at least one of a first spatial map stored in the electronic device 100 and a second spatial map received from an external device capable of communicating with the electronic device 100. According to any one of claims 1 to 3,
The step of analyzing the prediction (S620) is,
Comparing prediction results for processing of the task according to a plurality of movement paths divided based on the location of at least one branch and at least one object on the virtual movement path of the electronic device 100 for performing the task How to analyze. According to any one of claims 1 to 4,
The providing step (S520) is,
Confirming attribute information of the selected object (S710); and
A method comprising: executing a movement request process corresponding to the attribute information of the identified object and providing the object movement guide information to the user (S720, S730, S740, S750). According to any one of claims 1 to 5,
The step of providing to the user (S730) is,
Method comprising transmitting the analysis result of the prediction of processing of the task and the object movement guide information to the user's user terminal (S810). The method according to any one of claims 1 to 6,
The step of providing to the user (S740) is,
Selecting candidate locations to move the selected object in a three-dimensional space map of the area where the selected object is located (S1110, S1120);
Obtaining an image when the selected object moves for each candidate location (S1130);
Inputting the acquired image into an image evaluation model and evaluating the image through the image evaluation model (S1140); and
Transmitting the object movement guide information according to the image evaluation results to the user's user terminal (S1150);
Method, including. The method according to any one of claims 1 to 7,
The step of determining the movement path (S530) is,
Based on the user's response, confirming the moved object among the selected objects (S1410); and
Obtaining the movement path reflecting the moved object in a space corresponding to the task (S1420);
Method, including. The method according to any one of claims 1 to 8,
The method of claim 1, wherein the electronic device 100 is a robot vacuum cleaner. A computer-readable recording medium on which a program for executing the method of any one of claims 1 to 9 is recorded. a memory 110 that stores one or more instructions;
a processor 120 that executes the one or more instructions stored in the memory 110; and
Includes a sensing unit 130,
The processor 120 executes the one or more instructions,
Based on spatial information about a space containing at least one object acquired through the sensing unit 130 and a task that the electronic device 100 is set to perform, select the task from among objects located in the space corresponding to the task. Selecting an object that is in the way, providing object movement guide information according to attribute information of the selected object to the user of the electronic device 100, and based on the user's response corresponding to the object movement guide information, An electronic device (100) using spatial information that determines a movement path used to perform the task and drives the electronic device (100) according to the determined movement path. According to claim 11,
The processor 120 executes the one or more instructions,
Obtaining a spatial map using the spatial information, analyzing a prediction for processing of the task using the obtained spatial map, and determining at least one object that interferes with the task based on the analysis result, Electronic device (100). The method of claim 11 or 12,
The processor 120 executes the one or more instructions,
An electronic device 100 that acquires the spatial map based on at least one of a first spatial map stored in the electronic device 100 and a second spatial map received from an external device capable of communicating with the electronic device 100. ). The method according to any one of claims 11 to 13,
The processor 120 executes the one or more instructions,
Comparing prediction results for processing of the task according to a plurality of movement paths divided based on the location of at least one branch and at least one object on the virtual movement path of the electronic device 100 for performing the task Analyzing, electronic device 100. The method according to any one of claims 11 to 14,
The spatial map includes a plurality of layers based on attribute information of objects. The method according to any one of claims 11 to 15,
The processor 120 executes the one or more instructions,
The electronic device 100 checks attribute information of the selected object, executes a movement request process corresponding to the attribute information of the confirmed object, and provides object movement guide information to the user. The method according to any one of claims 11 to 16,
Further comprising a communication unit 150,
The processor 120 executes the one or more instructions,
An electronic device (100) that transmits analysis results of predictions for processing of the task and the object movement guide information to the user's user terminal (400) through the communication unit (150). The method according to any one of claims 11 to 17,
Further comprising a communication unit 150,
The processor 120 executes the one or more instructions,
Select candidate locations to move the selected object from a 3D space map of the area where the selected object is located, obtain an image when the selected object moves for each candidate location, and input the obtained image into an image evaluation model. An electronic device 100 that inputs, evaluates an image through the image evaluation model, and transmits the object movement guide information according to the image evaluation result to the user's user terminal 400 through the communication unit 150. ). The method according to any one of claims 11 to 18,
The processor 120 executes the one or more instructions,
Based on the user's response, the electronic device 100 identifies a moved object among the selected objects and acquires the movement path reflecting the moved object in a space corresponding to the task. The method according to any one of claims 11 to 19,
The electronic device 100 is a robot vacuum cleaner.

Images