KR20220080464A - Electronic apparatus and method for controlling thereof - Google Patents

Abstract

An electronic device is disclosed. The electronic device according to the present disclosure includes a communication interface, a memory, and a processor, wherein the processor generates a virtual robot and a virtual environment for driving simulation of the virtual robot, and receives control information for controlling the virtual robot from the robot control device acquisition, and performing a driving simulation for the virtual robot based on the control information. During the driving simulation, sensing data corresponding to the distance between the virtual robot and an object in the virtual environment is obtained, and the sensing data and the virtual robot are obtained. Based on the error rate of the virtual sensor included in the , virtual sensing data output when the virtual sensor measures the distance between the virtual robot and the object is acquired.

Description

ELECTRONIC APPARATUS AND METHOD FOR CONTROLLING THEREOF

The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device for providing sensing data of a virtual robot obtained in a simulation environment, and a control method thereof.

With the development of electronic technology, various robots are being developed. Recently, in order to reduce the manufacturing cost of the robot and check safety in advance, a simulator for configuring and simulating a virtual robot in a virtual environment is actively used.

Meanwhile, since the simulation environment is a virtual environment, various errors occurring in the real environment may not be reflected. For example, since the real sensor inevitably has an error rate while the virtual sensor of the virtual environment does not have an error rate, the measured data of the real sensor may be different from the sensing data of the virtual sensor even if the virtual environment is configured to be the same as the real environment. Therefore, when a real robot is manufactured by applying the sensing data of the virtual sensor as it is, a problem such as the real robot operating differently from the virtual robot or malfunctioning may occur.

In order to prevent this problem, it is necessary to provide a technology that reflects the error rate of the sensor in the sensing data acquired in the virtual environment.

One technical problem to be solved by the present invention is to provide simulation data in which an error rate of a sensor is reflected.

Another technical problem to be solved by the present invention is to provide parameters related to the robot based on simulation data.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to an exemplary embodiment of the present disclosure for solving the above-described technical problem, an electronic device includes: a communication interface; a memory storing at least one instruction; and a processor, wherein the processor generates a virtual robot and a virtual environment for driving simulation of the virtual robot, obtains control information for controlling the virtual robot from a robot control device, and based on the control information performing a driving simulation on the virtual robot, and while performing the driving simulation, sensing data corresponding to a distance between the virtual robot and an object in the virtual environment is obtained, and the sensing data and the virtual robot are An electronic device that obtains virtual sensing data output when the virtual sensor measures the distance between the virtual robot and the object based on the error rate of the included virtual sensor, and outputs the virtual sensing data to the robot control device can be provided.

According to another exemplary embodiment of the present disclosure for solving the above-described technical problem, there is provided a method of controlling an electronic device, the method comprising: generating a virtual robot and a virtual environment for simulating driving of the virtual robot; obtaining control information for controlling the virtual robot from a robot control device; performing a driving simulation on the virtual robot based on the control information; acquiring sensing data corresponding to a distance between the virtual robot and an object in the virtual environment while performing the driving simulation; acquiring virtual sensing data output when the virtual sensor measures a distance between the virtual robot and the object based on the sensing data and an error rate of a virtual sensor included in the virtual robot; and outputting the virtual sensing data to the robot control device. A control method comprising a may be provided.

The solutions of the problems of the present disclosure are not limited to the above-described solutions, and solutions that are not mentioned will be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the present specification and the accompanying drawings. will be able

According to various embodiments of the present disclosure as described above, the electronic device may perform a simulation based on the sensing data in which the error rate of the sensor is reflected to provide a parameter more suitable for the real environment. Therefore, trial and error for manufacturing the robot is reduced, the user's satisfaction is improved, and the cost of manufacturing the robot can be reduced.

In addition, the effects obtainable or predicted by the embodiments of the present disclosure are to be disclosed directly or implicitly in the detailed description of the embodiments of the present disclosure. For example, various effects predicted according to embodiments of the present disclosure will be disclosed in the detailed description to be described later.

1 is a view for explaining a concept of a simulation system according to an embodiment of the present disclosure.
2 is a block diagram illustrating a configuration of a simulation system according to an embodiment of the present disclosure.
3A is a diagram for explaining a method of learning a neural network model for acquiring virtual sensing data according to an embodiment of the present disclosure.
3B is a diagram for explaining a method of acquiring virtual sensing data using a neural network model according to an embodiment of the present disclosure.
4 is a diagram for explaining an error rate of a virtual sensor according to an embodiment of the present disclosure.
5 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.
6 is a block diagram illustrating a configuration of an electronic device according to another embodiment of the present disclosure.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail.

Terms used in the embodiments of the present disclosure are selected as currently widely used general terms as possible while considering the functions in the present disclosure, which may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, etc. . In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding disclosure. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the contents of the present disclosure, rather than the simple name of the term.

Embodiments of the present disclosure may be subjected to various transformations and may have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of the specific embodiments, and it should be understood to include all transformations, equivalents and substitutions included in the spirit and scope of the disclosure. In describing the embodiments, if it is determined that a detailed description of a related known technology may obscure the subject matter, the detailed description thereof will be omitted.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as "comprises" or "consisting of" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, but one or more other It should be understood that this does not preclude the possibility of addition or presence of features or numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present disclosure pertains can easily implement them. However, the present disclosure may be implemented in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present disclosure in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

1 is a diagram for explaining a concept of a simulation system according to an embodiment of the present disclosure.

The electronic device 100 may generate the virtual robot 11 and a virtual environment for driving simulation of the virtual robot 11 . In addition, the electronic device 100 may perform a driving simulation of the virtual robot 11 in a virtual environment. The virtual robot 11 may acquire sensing data related to the virtual environment from the virtual sensor 12 . For example, when the virtual sensor 12 is a LiDAR (Light Detection And Ranging) sensor, the virtual robot 11 uses the virtual sensor 12 to obtain distance information about the object 13 existing in the virtual environment. can be obtained And, the virtual robot 11 may drive based on the acquired sensing data.

Meanwhile, the user may manufacture the real robot 14 corresponding to the virtual robot 11 based on the simulation result in the virtual environment. For example, a user may mount a real sensor 15 having a parameter corresponding to a parameter (eg, a measurement range) of the virtual sensor 12 on the real robot 14 .

However, since the virtual sensor 12 does not have a measurement error unlike the real sensor 15 , sensing data obtained through the virtual sensor 12 and measurement data obtained through the real sensor 15 may be different. For example, even if the distance between the real robot 14 and the real object 16 in the real environment is the same as the distance between the virtual robot 11 and the virtual object 13 in the virtual environment, the The first distance d1 and the second distance d2 obtained through the actual sensor 15 may be different. Accordingly, the real robot 14 may operate differently from the virtual robot 11 . For example, the real robot 14 may travel on a different path than the virtual robot 11 .

As such, since the sensed data of the virtual sensor 12 and the real sensor 15 may be different, if the real robot 14 is implemented using the sensing result of the virtual sensor 12 as it is, the real robot 14 is intended It may not work as it is. To prevent this problem, the electronic device 100 may reflect the sensing error rate in the sensing data of the virtual sensor 12 similarly to the real sensor 15 , and obtain the sensing data reflecting the error rate and provide it to the user. .

2 is a block diagram illustrating a configuration of a simulation system according to an embodiment of the present disclosure. The simulation system 1000 may include the electronic device 100 and the robot control device 200 . The electronic device 100 is a configuration for performing a driving simulation for a virtual robot in a virtual environment. For example, the electronic device 100 may be a personal computer (PC), a server computer, a laptop computer, or a mobile phone. The robot control apparatus 200 may generate control information for controlling the robot. For example, the robot control device 200 may be a microcontroller or microcontroller unit (MCU) mounted on an actual robot. Meanwhile, the robot control device 200 may be implemented separately from the electronic device 100 , or may be implemented by being integrated into the electronic device 100 .

The electronic device 100 may include a communication interface 110 , a memory 120 , and a processor 130 . Hereinafter, each configuration of the electronic device 100 will be described in detail.

The communication interface 110 includes at least one circuit and may communicate with various types of external devices according to various types of communication methods. For example, the communication interface 110 may receive control information for controlling a virtual robot corresponding to a real robot to be controlled by the robot control apparatus 200 from the robot control apparatus 200 . Here, the control information may be a control message including information for driving the virtual robot (eg, a moving speed of the virtual robot, power for driving an actuator of the virtual robot, etc.). Also, the communication interface 110 may transmit the sensing data acquired by the electronic device 100 to the robot control device 200 .

Meanwhile, the communication interface 110 may perform data communication by wire or wirelessly. When communicating with an external device through a wired communication method, the communication interface 110 may be implemented as a wired communication module (eg, LAN, etc.). When communicating with an external device in a wireless communication method, the communication interface 110 is a Wi-Fi communication module, a cellular communication module, a 3G (3rd generation) mobile communication module, a 4G (4th generation) mobile communication module, a 4th generation LTE (Long) It may include at least one of a Term Evolution) communication module and a 5G (5th generation) mobile communication module.

The memory 120 may store an operating system (OS) for controlling overall operations of the components of the electronic device 100 and commands or data related to the components of the electronic device 100 . For example, the memory 120 may store information about a virtual environment and a virtual robot. Meanwhile, the memory 120 may be implemented as a non-volatile memory (eg, a hard disk, a solid state drive (SSD), a flash memory), a volatile memory, or the like.

The processor 130 may control the overall operation of the electronic device 100 . The processor 130 includes a virtual robot generation module 131 , a simulation environment building module 132 , a control information acquisition module 133 , a driving simulation module 134 , a sensing data acquisition module 135 , and a virtual sensing data acquisition module ( 136) may be included.

The virtual robot generating module 131 may generate a virtual robot (or a virtual robot model). A virtual robot is a robot existing in a virtual environment, and may include virtual components corresponding to components constituting the real robot. For example, when the real robot includes a lidar sensor, the virtual robot may include a virtual sensor corresponding to the lidar sensor. In addition, the virtual robot may include various virtual components according to a user input.

The simulation environment building module 132 may generate a virtual environment for simulating the driving of the virtual robot. The simulation environment building module 132 may obtain information about the virtual environment based on a user input, and may generate a virtual environment based on the obtained information. The simulation environment building module 132 may perform rendering to create a virtual environment. Meanwhile, the information about the virtual environment may include information about the size of the virtual environment, the topography, and an object included in the virtual environment.

The control information acquisition module 133 may acquire control information for controlling the virtual robot from the robot control device 200 through the communication interface 110 . For example, the control information acquisition module 133 may acquire information about the traveling speed of the virtual robot. Alternatively, the control information acquisition module 133 may acquire information on power for driving an actuator included in the virtual robot.

The driving simulation module 134 may perform driving simulation for the virtual robot based on the control information acquired through the control information acquisition module 133 . While the driving simulation is being performed, the sensing data acquisition module 135 may collect various information about the virtual environment. For example, the sensing data acquisition module 135 may acquire the virtual robot sensing data corresponding to a distance between objects existing in the virtual environment. The sensing data acquisition module 135 may acquire an image obtained by capturing a virtual environment using a virtual sensor, or may acquire depth information on the virtual environment.

Meanwhile, the sensing data acquired through the sensing data acquisition module 135 may be ground truth data that does not include an error that is inevitably generated in a real environment. For example, a lidar sensor in a real environment acquires sensing data having an error rate according to various factors including a distance from an object and a characteristic (eg, material or transmittance) of the object. In contrast, since the virtual robot acquires sensing data in a virtual environment, it is possible to acquire ground truth data having no error rate. As such, there is a difference between the sensed data obtained in the virtual environment and the sensed data obtained in the real environment. Therefore, when the real robot is modeled by applying the sensing data obtained in the virtual environment as it is, the real robot may malfunction. For example, when the sensed data corresponds to the distance between the robot and the object, the robot may move to an inefficient driving path, and thus the user's satisfaction and convenience may decrease.

To solve this problem, the virtual sensing data acquisition module 136 may convert the sensing data into virtual sensing data in which an error of the real environment is reflected. For example, the virtual sensing data acquisition module 136 may calculate an error based on an error rate of the virtual sensor. In addition, the virtual sensing data acquisition module 136 may acquire the virtual sensing data by adding the calculated error to the sensing data.

Meanwhile, the error rate of the virtual sensor may be set based on the error rate of the real sensor. For example, the error rate of the virtual sensor may include a first error rate occurring according to a distance of an object and a second error rate occurring according to a material of the object. When the virtual sensor corresponds to the lidar sensor, the error rate of the virtual sensor may include a third error rate corresponding to the error in the rotation angle of the motor provided in the lidar sensor. In addition, each error rate of the virtual sensor may have a specific value or a value within a specific range. For example, the error rate of the virtual sensor may have a value in the form of a normal distribution.

The virtual sensing data Dp according to an embodiment may be modeled as in [Equation 1] below.

[Equation 1]

Dp = Dr + (Dr * Ed) + (Dr * Ema) + (Dr * Emo)

Here, Dr is sensing data acquired through the sensing data acquisition module 135 , Ed is a first error rate, Ema is a second error rate, and Emo is a third error rate.

Meanwhile, the virtual sensing data acquisition module 136 may acquire the virtual sensing data by inputting the sensing data to the learned neural network model. The neural network model may be trained to output virtual sensing data when sensing data is input. The neural network model may be trained such that the difference between the output virtual sensing data and the actual data (or label data) measured in the real environment is minimized. In this case, the weight of the neural network model that has been trained may correspond to the error rate of the virtual sensor. That is, the first error rate, the second error rate, and the third error rate may correspond to weights of the learned neural network model, respectively.

The virtual sensing data acquisition module 136 may transmit the virtual sensing data to the robot control device 200 through the communication interface 110 . The robot control apparatus 200 may include a recognition module 210 , a determination module 220 , and a driving module 230 . The recognition module 210 may obtain virtual sensing data from the electronic device 100 and analyze the virtual sensing data. For example, when the virtual sensing data is an image captured around the virtual robot, the recognition module 210 may recognize an object included in the image. In this case, the recognition module 210 may generate a point cloud at a location corresponding to each object.

The determination module 220 may determine how to operate the virtual robot based on the virtual sensing data and the analysis result of the recognition module 210 . That is, the determination module 220 may determine an action to be performed by the virtual robot. For example, if an object exists within a preset distance from the virtual robot, the determination module 220 may determine an action to bypass the object and travel.

The driving module 230 may be configured to acquire driving information (or control information) for driving the virtual robot based on the determination result of the determination module 220 . For example, the driving module 230 may acquire a movement speed of the virtual robot for avoiding an object or a rotation torque of an actuator of the virtual robot. Driving information acquired through the driving module 230 may be transmitted to the electronic device 100 . Also, the electronic device 100 may repeatedly perform a simulation of the virtual robot based on the driving information.

Meanwhile, in FIG. 2 , each of the modules 131 to 136 has been described as a configuration of the processor 130 , but this is only an exemplary embodiment, and each of the modules 131 to 136 may be stored in the memory 120 . In this case, the processor 130 may load the plurality of modules 131 to 136 stored in the memory 120 from the nonvolatile memory to the volatile memory to execute respective functions of the plurality of modules 131 to 136 . . In addition, each module of the processor 130 may be implemented as software or a combination of software and hardware.

3A is a diagram for explaining a method of learning a neural network model for acquiring virtual sensing data according to an embodiment of the present disclosure. 3B is a diagram for explaining a method of acquiring virtual sensing data using a neural network model according to an embodiment of the present disclosure.

Referring to FIG. 3A , the electronic device 100 may acquire the virtual sensing data 32 by inputting the sensing data 31 into the neural network model 30 . In addition, the electronic device 100 may obtain an error by comparing the virtual sensing data 32 and the actual measurement data 33 . Here, the measured data 33 is label data measured based on an actual sensor in a real environment. The electronic device 100 may update the weights W1 , W2 , and W3 of the neural network model 30 based on the obtained error. The electronic device 100 may repeatedly perform the above-described operation until the error becomes smaller than a preset value. When learning is completed, the electronic device 100 may store the finally updated weight (or parameter) in the memory 120 .

Meanwhile, the neural network model 30 according to the present disclosure is not necessarily learned by the electronic device 100 . That is, the neural network model 30 is trained by an external device (or an external server), and the electronic device 100 may receive the neural network model 30 that has been trained by the external device.

Referring to FIG. 3B , the electronic device 100 may acquire the virtual sensing data 32 by inputting the sensing data 31 into the neural network model 30 on which training has been completed. Meanwhile, the weights W1 , W2 , and W3 of the neural network model 30 may correspond to the error rates Ed, Ema, and Emo of the aforementioned virtual sensor. That is, the electronic device 100 may acquire the error rates Ed, Ema, and Emo of the virtual sensor through the process of learning the neural network model 30 .

Meanwhile, the error rate of the virtual sensor according to the present disclosure may be selected from among a plurality of values falling within a specific range. For example, the error rate of the virtual sensor may be selected as one of values corresponding to a normal distribution function as shown in FIG. 4 . The first function 41 may correspond to the first error rate, the second function 42 may correspond to the second error rate, and the third function 43 may correspond to the third error rate.

In this case, the virtual sensing data Dp may be modeled as in Equation 2 below.

[Equation 2]

) + (Dr *

+ (Dr *

Dp = Dr + (Dr *

) + (Dr *

+ (Dr*

)는 제1 에러율,

는 제2 에러율,

는 제3 에러율을 의미한다.

는 정규 분포의 평균,

는 표준 편차를 의미한다. 한편, 정규 분포의 평균 및 표준 편차의 값은 사용자의 설정에 따라 달라질 수 있다.here,

) is the first error rate,

is the second error rate,

is the third error rate.

is the mean of the normal distribution,

is the standard deviation. Meanwhile, the values of the mean and standard deviation of the normal distribution may vary according to user settings.

Meanwhile, the electronic device 100 may randomly obtain each error rate based on [Equation 2]. When each error rate is obtained, the electronic device 100 may repeatedly perform a driving simulation of the virtual robot based on each obtained error rate. In addition, the electronic device 100 may acquire simulation data corresponding to each driving simulation and store it in the memory 120 . Here, the simulation data may include at least one of a driving route, a driving distance, a driving time, and power consumed for driving of the virtual robot to reach a specific destination. In addition, the simulation data may include difficulty and cost of a process for manufacturing a real robot corresponding to the virtual robot in each driving simulation.

The electronic device 100 may identify at least one driving simulation based on the simulation data. For example, the electronic device 100 may identify a first driving simulation having the smallest driving distance among a plurality of driving simulations. Alternatively, the electronic device 100 may identify a second driving simulation having the smallest driving time among a plurality of driving simulations. In addition, the electronic device 100 may identify a third driving simulation in which power consumed for driving is the smallest among the plurality of driving simulations. Then, the electronic device 100 may obtain an error rate (or parameter) corresponding to the identified driving simulation. The electronic device 100 may output the obtained error rate and provide it to the user. For example, when the first driving simulation is identified, the electronic device 100 may output a parameter corresponding to the first driving simulation and provide it to the user. As such, the electronic device 100 may recommend a specific parameter to the user. Accordingly, the user may manufacture an actual sensor based on the parameters provided by the electronic device 100 , and thus the user's convenience and satisfaction may be improved.

Meanwhile, in the above description, the error rate of the virtual sensor as a parameter of the virtual sensor has been mainly described, but this is only an example, and the electronic device 100 obtains virtual sensing data based on parameters for various information, and Optimal parameters can be provided. For example, the parameters of the virtual sensor may include information about specifications such as a detection distance, a detection range, a scanning period, and a sampling period of the virtual sensor. Alternatively, the parameter of the virtual sensor may include arrangement information such as an installation position or posture of the virtual sensor. In addition, the parameter according to the present disclosure may include information about a virtual environment (eg, map information) or a simulation purpose (eg, driving the shortest distance) as well as information about the virtual sensor.

5 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

The electronic device may generate a virtual robot and a virtual environment for driving simulation of the virtual robot ( S510 ). When the virtual robot and the virtual environment are generated, the electronic device 100 may display the generated virtual robot and the virtual environment.

The electronic device may obtain control information for controlling the virtual robot from the robot control device ( S520 ), and perform a driving simulation on the virtual robot based on the control information ( S530 ).

The electronic device may acquire sensing data corresponding to the distance between the virtual robot and the object in the virtual environment while performing the driving simulation ( S540 ). In this case, the electronic device 100 may acquire sensing data based on location information (or coordinate information) of the virtual robot and the object. Location information about the virtual robot and the object may be stored in the memory 120 .

The electronic device may obtain virtual sensing data output when the virtual sensor measures the distance between the virtual robot and the object based on the sensing data and the error rate of the virtual sensor included in the virtual robot ( S550 ). The electronic device 100 may obtain an error value based on the sensing data and the error rate of the virtual sensor. In addition, the electronic device 100 may acquire virtual sensing data by summing the sensing data and the error value. Meanwhile, the error rate of the virtual sensor may include at least one of a first error rate related to the distance between the virtual sensor and the object, a second error rate related to the material of the object, and a third error rate related to a motor included in the virtual sensor.

Meanwhile, the electronic device 100 may obtain virtual sensing data by inputting the learned neural network model. Here, the learned neural network model may be learned based on sensing data and a real sensor corresponding to the virtual sensor. And, the weight of the learned neural network model may correspond to the error rate of the virtual sensor. Meanwhile, the error rate of the virtual sensor may have a value corresponding to a normal distribution function. In this case, the electronic device 100 may repeatedly perform the driving simulation while changing the error rate of the virtual sensor. In addition, the electronic device 100 may identify at least one driving simulation based on at least one of a driving path, a driving distance, and a driving time of the virtual robot according to the driving simulation. The electronic device 100 may output an error rate of the virtual sensor corresponding to the identified driving simulation.

The electronic device may output virtual sensing data to the robot control device (S560). The robot control apparatus 200 may generate control information for controlling the virtual robot based on the virtual sensing data. When the robot control device 200 transmits control information to the electronic device 100 , the electronic device 100 may again perform a driving simulation for the virtual robot based on the control information.

6 is a block diagram illustrating a configuration of an electronic device according to another embodiment of the present disclosure. Referring to FIG. 6 , the electronic device 100 may include a communication interface 110 , a memory 120 , a processor 130 , an input unit 140 , and a display 150 . Meanwhile, since the communication interface 110 , the memory 120 , and the processor 130 have been described in detail with reference to FIG. 2 , a redundant description will be omitted.

The processor 130 may generate a virtual robot and a virtual environment for driving simulation of the virtual robot. The processor 130 may generate a virtual robot and a virtual environment based on a user input obtained through the input unit 140 . Here, the user input may include information on the virtual robot (eg, the size or shape of the virtual robot) and information on the virtual environment (eg, information on an object included in the virtual environment). The processor 130 may store the obtained information on the virtual robot and the virtual environment in the memory 120 .

The processor 130 may obtain control information for controlling the virtual robot from the robot control device 200 . The processor 130 may perform a driving simulation for the virtual robot based on the control information. While performing the driving simulation, the processor 130 may acquire sensing data corresponding to the distance between the virtual robot and the object in the virtual environment. For example, the processor 130 may acquire sensing data based on location information about a virtual robot and an object pre-stored in the memory 120 .

The processor 130 may acquire virtual sensing data output when the virtual sensor measures the distance between the virtual robot and the object based on the sensing data and the error rate of the virtual sensor included in the virtual robot. For example, the processor 130 may obtain an error value based on the sensing data and an error rate of the virtual sensor, and may obtain the virtual sensing data by summing the sensing data and the error value. Alternatively, the processor 130 may obtain virtual sensing data by inputting the sensing data into the learned neural network model.

Meanwhile, the error rate of the virtual sensor may have a value corresponding to a normal distribution function. In this case, the processor 130 may repeatedly perform the driving simulation while changing the error rate of the virtual sensor. The processor 130 may identify at least one driving simulation based on at least one of a driving path, a driving distance, and a driving time of the virtual robot according to the driving simulation. For example, the processor 130 may identify a driving simulation having the smallest driving distance among a plurality of driving simulations. The processor 130 may output an error rate of the virtual sensor corresponding to the identified driving simulation. For example, the processor 130 may control the display 150 to display an error rate of the virtual sensor.

The processor 130 may output virtual sensing data to the robot control device. For example, the processor 130 may control the communication interface 110 to transmit virtual sensing data to the robot control device 200 .

Functions related to artificial intelligence according to the present disclosure are operated through the processor 130 and the memory 120 . The processor 130 may include one or a plurality of processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a digital signal processor (DSP), or the like, a graphics-only processor such as a GPU, a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. One or a plurality of processors control to process input data according to a predefined operation rule or artificial intelligence model stored in the memory 120 . Alternatively, when one or more processors are AI-only processors, the AI-only processor may be designed with a hardware structure specialized for processing a specific AI model.

A predefined action rule or artificial intelligence model is characterized in that it is created through learning. Here, being made through learning means that a basic artificial intelligence model is learned using a plurality of learning data by a learning algorithm, so that a predefined action rule or artificial intelligence model set to perform a desired characteristic (or purpose) is created means burden. Such learning may be performed in the device itself on which artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and/or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

AI models can be created through learning. Here, being made through learning means that a basic artificial intelligence model is learned using a plurality of learning data by a learning algorithm, so that a predefined action rule or artificial intelligence model set to perform a desired characteristic (or purpose) is created means burden. The artificial intelligence model may be composed of a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values, and a neural network operation is performed through an operation between the operation result of a previous layer and the plurality of weights. The plurality of weights of the plurality of neural network layers may be optimized by the learning result of the artificial intelligence model. For example, a plurality of weights may be updated so that a loss value or a cost value obtained from the artificial intelligence model during the learning process is reduced or minimized.

The electronic device 100 according to the present disclosure may use an artificial intelligence model to acquire virtual sensing data. The processor 130 may perform a preprocessing process on the sensing data of the virtual sensor to convert it into a form suitable for use as an input of an artificial intelligence model. AI models can be created through learning. Here, being made through learning means that a basic artificial intelligence model is learned using a plurality of learning data by a learning algorithm, so that a predefined action rule or artificial intelligence model set to perform a desired characteristic (or purpose) is created means burden. The artificial intelligence model may be composed of a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values, and a neural network operation is performed through an operation between the operation result of a previous layer and the plurality of weights.

Inference prediction is a technology for logically reasoning and predicting by judging information. It is a technology that uses knowledge based reasoning, optimization prediction, preference-based planning, and recommendation. include

The artificial neural network may include a deep neural network (DNN), for example, a Convolutional Neural Network (CNN), a Deep Neural Network (DNN), a Recurrent Neural Network (RNN), a Generative Adversarial Network (GAN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN), or Deep Q-Networks, but is not limited to the above example.

The input unit 140 means a means for a user to input data for controlling the electronic device 100 . For example, the processor 130 may obtain information about the virtual robot (eg, the shape or size of the virtual robot) and information about the virtual sensor (eg, parameters of the virtual sensor) through the input unit 140 . Meanwhile, the input unit 140 may include, but is not limited to, a keyboard, a mouse, a key pad, a touch pad, and the like.

The display 150 may display various screens. For example, the display 150 may display a driving simulation process of the virtual robot. Alternatively, the display 150 may display a parameter (or an error rate) corresponding to a specific driving simulation. Meanwhile, the display 150 may be implemented as a liquid crystal display panel (LCD), organic light emitting diodes (OLED), or the like.

Meanwhile, the various embodiments described above may be implemented in a recording medium readable by a computer or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented by the processor itself. According to the software implementation, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

Meanwhile, computer instructions for performing the processing operation according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. When the computer instructions stored in the non-transitory computer-readable medium are executed by the processor, the processing operation according to the above-described various embodiments may be performed by a specific device.

The non-transitory computer-readable medium refers to a medium that stores data semi-permanently, not a medium that stores data for a short moment, such as a register, cache, memory, etc., and can be read by a device. Specific examples of the non-transitory computer-readable medium may include a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

Meanwhile, the device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' is a tangible device and only means that it does not contain a signal (eg, electromagnetic wave). It does not distinguish the case where it is stored as For example, the 'non-transitory storage medium' may include a buffer in which data is temporarily stored.

According to an embodiment, the method according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly or online between smartphones (eg: smartphones). In the case of online distribution, at least a portion of a computer program product (eg, a downloadable app) is stored at least in a machine-readable storage medium, such as a memory of a manufacturer's server, a server of an application store, or a relay server. It may be temporarily stored or temporarily created.

In the above, preferred embodiments of the present disclosure have been illustrated and described, but the present disclosure is not limited to the specific embodiments described above, and it is common in the technical field pertaining to the present disclosure without departing from the gist of the present disclosure as claimed in the claims. Various modifications can be made by those having the knowledge of

1000 electronic device 100 electronic device
110: communication interface 120: memory
130: processor 140: input interface
150: display 200: robot control unit

Claims (15)

In an electronic device,
communication interface;
a memory storing at least one instruction; and
processor; including;
The processor is
Create a virtual robot and a virtual environment for driving simulation of the virtual robot,
Obtaining control information for controlling the virtual robot from the robot control device,
performing a driving simulation for the virtual robot based on the control information,
while performing the driving simulation, acquiring sensing data corresponding to a distance between the virtual robot and an object in the virtual environment;
Obtaining virtual sensing data output when the virtual sensor measures the distance between the virtual robot and the object based on the sensing data and the error rate of the virtual sensor included in the virtual robot,
outputting the virtual sensing data to the robot control device
electronic device. According to claim 1,
The processor is
Acquiring the sensing data based on the location information about the virtual robot and the object stored in the memory
electronic device. According to claim 1,
The processor is
obtaining an error value based on the sensing data and the error rate of the virtual sensor,
acquiring the virtual sensed data by summing the sensed data and the error value
electronic device. According to claim 1,
The error rate of the virtual sensor is
At least one of a first error rate related to the distance between the virtual sensor and the object, a second error rate related to the material of the object, and a third error rate related to a motor included in the virtual sensor
electronic device. According to claim 1,
The processor is
To obtain the virtual sensed data by inputting the sensing data into a neural network model learned based on the measured data obtained based on the sensing data and the real sensor corresponding to the virtual sensor
electronic device. 6. The method of claim 5,
The error rate of the virtual sensor is
corresponding to the weight of the learned neural network model.
electronic device. According to claim 1,
The error rate of the virtual sensor has a value corresponding to a normal distribution function,
The processor is
repeatedly performing the driving simulation while changing the error rate of the virtual sensor,
identifying at least one driving simulation based on at least one of a driving route, a driving distance, and a driving time of the virtual robot according to the driving simulation;
outputting an error rate of the virtual sensor corresponding to the identified driving simulation
electronic device. A method for controlling an electronic device, comprising:
generating a virtual robot and a virtual environment for driving simulation of the virtual robot;
obtaining control information for controlling the virtual robot from a robot control device;
performing a driving simulation on the virtual robot based on the control information;
acquiring sensing data corresponding to a distance between the virtual robot and an object in the virtual environment while performing the driving simulation;
acquiring virtual sensing data output when the virtual sensor measures a distance between the virtual robot and the object based on the sensing data and an error rate of a virtual sensor included in the virtual robot; and
outputting the virtual sensing data to the robot control device; including
control method. 9. The method of claim 8,
The step of acquiring the sensing data includes:
acquiring the sensing data based on location information about the virtual robot and the object pre-stored in the electronic device;
control method. 9. The method of claim 8,
The step of obtaining the virtual sensing data includes:
obtaining an error value based on the sensing data and the error rate of the virtual sensor; and
Acquiring the virtual sensed data by summing the sensed data and the error value
control method. 9. The method of claim 8,
The error rate of the virtual sensor is
At least one of a first error rate related to the distance between the virtual sensor and the object, a second error rate related to the material of the object, and a third error rate related to a motor included in the virtual sensor
control method. 9. The method of claim 8,
The step of obtaining the virtual sensing data includes:
To obtain the virtual sensed data by inputting the sensing data into a neural network model learned based on the measured data obtained based on the sensing data and the real sensor corresponding to the virtual sensor
control method. 13. The method of claim 12,
The error rate of the virtual sensor is
corresponding to the weight of the learned neural network model.
control method. 9. The method of claim 8,
The error rate of the virtual sensor has a value corresponding to a normal distribution function,
The control method is
repeatedly performing the driving simulation while changing the error rate of the virtual sensor;
identifying at least one driving simulation based on at least one of a driving route, a driving distance, and a driving time of the virtual robot according to the driving simulation; and
Outputting an error rate of the virtual sensor corresponding to the identified driving simulation; further comprising
control method. A computer-readable non-transitory recording medium in which a program for executing the method of any one of claims 8 to 14 in a computer is recorded.

Images