KR20210074155A - Electronic device and control method thereof - Google Patents

Abstract

Disclosed are an electronic device and a control method thereof. The electronic device according to the present disclosure comprises: a memory; and a processor comprising at least one core which is set so as to execute an instruction corresponding to at least one safety function among a plurality of cores. The processor, while the electronic device operates in a first state, identifies whether or not at least one instruction is an instruction corresponding to a safety function through a learned neural network model when at least one instruction is executed on at least one core, and allows an operating state of the electronic device to be determined as one of a first state and a second state based on an identification result. Therefore, the present invention is capable of efficiently identifying whether or not a safety function is defective.

Description

Electronic device and control method thereof

The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device and a control method thereof for monitoring an instruction executed in a core to identify whether a fault exists when performing a safety function.

In order for the safety function of various devices to satisfy international safety standards, it is necessary to monitor whether the corresponding safety function satisfies FFI (Freedom from Interference) compared to general functions. At this time, the fact that the safety function satisfies the FFI means that the safety function can perform its function independently from the general function from the hardware or software point of view.

In order to satisfy the FFI, the independence of the CPU or memory that executes or stores the software program constituting the safety function must be ensured. However, when a new function is added as a package update of the device is performed, it cannot be guaranteed that the added new function does not interfere with the CPU performing the safety function. Therefore, when an update or the like to add a function occurs, there is a limitation in that it is necessary to separately check whether the safety function of the device is performed without a defect. It can be costly and time consuming to check whether a device's safety functions are performing flawlessly.

The present disclosure has been devised to solve the above-described problems, and an object of the present disclosure is to monitor whether an instruction executed in a core set to perform a safety function executes an instruction corresponding to a function different from the safety function, and control thereof to provide a method.

According to an embodiment of the present disclosure, an electronic device includes a processor including a memory and at least one core configured to execute an instruction corresponding to at least one safety function among a plurality of cores, wherein the processor includes the electronic device When at least one instruction is executed in the at least one core while operating in the first state, it is identified whether the at least one instruction is an instruction corresponding to the safety function through a learned neural network model, and the identification Based on the result, the operating state of the electronic device may be determined as one of the first state or the second state.

Meanwhile, according to an embodiment of the present disclosure, in the method of controlling an electronic device including a processor including at least one core configured to execute an instruction corresponding to at least one safety function among a plurality of cores, the electronic device If at least one instruction is executed in the at least one core while operating in the first state, identifying whether the at least one instruction is an instruction corresponding to the safety function through a learned neural network model; and The method may include determining the operating state of the electronic device as one of the first state and the second state based on the identification result.

As described above, according to various embodiments of the present disclosure, the electronic device monitors whether an instruction executed in a core configured to perform a safety function executes an instruction corresponding to a function different from the safety function, thereby allowing a user to perform various functions of the electronic device. Even with this update, it is possible to efficiently identify whether the safety function is faulty.

1 is a view for explaining the configuration and operation of an electronic device according to an embodiment of the present disclosure;
2 is a flowchart for explaining an operation of an electronic device according to an embodiment of the present disclosure;
3 is a view for explaining a process in which an electronic device identifies an instruction corresponding to a function different from a safety function according to an embodiment of the present disclosure;
4 is a view for explaining an operation of an electronic device according to an embodiment of the present disclosure;
5 is a sequence diagram illustrating operations of an electronic device and a server according to an embodiment of the present disclosure;
6 is a block diagram illustrating in detail the configuration of an electronic device according to an embodiment of the present disclosure;
7 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

The electronic device 100 of FIG. 1 may selectively operate in one of a first state and a second state. Meanwhile, in describing the present disclosure, the first state may be described as a normal state and the second state may be expressed as a safe state, but the present disclosure is not limited thereto. The normal state means a state in which the electronic device 100 is performing a normal function and a safety function. The safety function may include a function of monitoring and controlling risk factors that may occur while the electronic device 100 performs a general function. That is, the safety function monitors whether a general function performed by the electronic device 100 operates within a normal range, and when it is identified that the general function does not operate within a normal range, the number, speed, It may include a function to limit the location and the like. For example, when the general function is a speed control function of the electronic device 100 , the safety function includes an operation of monitoring whether the speed of the electronic device 100 exceeds a preset maximum speed and the speed of the electronic device 100 . When it is identified that the preset maximum speed is exceeded, it may be a function of reducing the magnitude of the speed. Accordingly, the electronic device 100 may perform a general function and a safety function of monitoring or controlling the general function while operating in a safe state. A related embodiment will be described in detail with reference to FIG. 4 .

Meanwhile, the safety function may include a safety function suggested by a standard set by the International Organization for Standardization according to the type of the electronic device 100 . For example, when the electronic device 100 is implemented as a personal assistance robot, the safety functions include emergency stop, protective stop, safe speed control, hazardous collision avoidance, work space limitation, safety output control, stability according to ISO 13482. Control functions may be included. However, this is only an embodiment, and when a general function is added according to an update of the electronic device, the safety function may also be added as a function for monitoring and controlling the added general function, and may be newly added by the user. is of course

The safe state refers to a state in which the electronic device 100 performs a predefined operation without performing a general function and a safety function. For example, when operating in a safe state, the electronic device 100 stops the currently performed general function and safety function, cuts off the power of the electronic device 100, or prevents the user from operating the safety function normally. You can provide a message that you can. Embodiments related to the conditions for the electronic device 100 to change the operating state from the normal state to the safe state and the operation performed by the electronic device 100 in the safe state will be described in detail later.

The processor 120 of the electronic device 100 of FIG. 1 may include a plurality of cores, and at least one of the plurality of cores may be a dedicated core set to execute only an instruction corresponding to a safety function. In addition, the electronic device 100 may monitor whether an instruction having a function different from the safety function is executed in the dedicated core. When it is identified that the dedicated core is executing an instruction corresponding to a function different from the safety function, the electronic device 100 may change the operation state from the normal state to the safe state.

Meanwhile, the electronic device 100 of the present disclosure may be implemented as a mobile robot or a control system capable of controlling the overall operation of the mobile robot. However, this is only an embodiment, and the electronic device 100 may be implemented as a device mounted on a transport device to control the operation of the transport device. For example, when the electronic device 100 is a part of a transport device that controls the transport device, the electronic device 100 may be, for example, an advanced driving assistance system such as an Advanced Driver Assist System (ADAS), or a part of the system. may be Alternatively, the electronic device 100 may be, for example, an electronic control device such as an Electronic Control Unit (ECU) that electronically controls various functions related to the operation of the transportation device, or may be a part of the device.

In another embodiment, when the electronic device 100 is an external device mounted on a transportation device, the electronic device 100 is, for example, a self-diagnostic device (on) connected to a vehicle connector (eg, an OBD terminal or an OBD connector). board diagnostics, OBD), driving assistance devices such as navigation, etc., or may be part of them. On the other hand, in the present disclosure, the transport device may be a device on which a person is boarded or a cargo is loaded, and may include, for example, a vehicle, an airplane, a motorcycle, a ship or a train.

And, as another embodiment, the electronic device 100 is a mobile terminal device such as a smartphone, a tablet personal computer, a mobile phone, a video phone, and an e-book reader. ), desktop personal computer, laptop personal computer, netbook computer, personal digital assistant, mobile medical device, camera, internet of things, or It may be a wearable device, or it may be a part of them.

Hereinafter, the above-described operation based on each configuration of FIG. 1 will be described in more detail with reference to the drawings. 1 is a diagram for describing an operation and configuration of an electronic device 100 according to an embodiment of the present disclosure. 1 , the electronic device 100 may include a memory 110 and a processor 120 . However, the configuration shown in FIG. 1 is an exemplary diagram for implementing embodiments of the present disclosure, and appropriate hardware and software configurations at a level obvious to those skilled in the art may be additionally included in the electronic device 100 .

The memory 110 may store commands or data related to at least one other component of the electronic device 100 . In addition, the memory 110 is accessed by the processor 120 , and data readout/writing/deletion/update by the processor 120 may be performed.

In particular, the memory 110 may store a plurality of instruction sets to be executed by a plurality of cores included in the processor 120 and the learned neural network model 20 . The plurality of instruction sets and the learned neural network model 20 stored in the non-volatile memory may be loaded into the volatile memory under the control of the processor 120 . In addition, the core included in the processor 120 may execute a plurality of instructions loaded into the volatile memory. In addition, the learned neural network model 120 loaded into the volatile memory 120 may monitor instructions executed in the dedicated core. Meanwhile, loading refers to an operation of loading and storing data stored in the non-volatile memory into the volatile memory so that the processor 120 can access it.

Meanwhile, an instruction means one action statement that the processor 120 can directly execute in a program writing language, and is a minimum unit for program execution or operation. An instruction consists of an instruction code (or operator) (opcode) and an operand (or argument) (operand). The command code may indicate the type of operation corresponding to the instruction, and the operand is the target of the operation of the command code, and is information about the address of the memory 110 or register in which data or data is stored. can be configured.

The plurality of instruction sets stored in the memory 110 may include instructions corresponding to a plurality of safety functions and instructions corresponding to general functions different from the safety functions.

In the learned neural network model 20, an instruction executed in a dedicated core set to execute only an instruction corresponding to a safety function among a plurality of cores included in the processor 120 by the control of the processor 120 corresponds to the safety function An artificial intelligence model that has been trained to monitor and identify whether an instruction is For example, referring to FIG. 1 , when the first core 10-1 and the second core 10-2 are dedicated cores set to execute only instructions corresponding to the safety function, the trained neural network model ( 20) may monitor instructions being executed in the dedicated cores 10-1 and 10-2. Specifically, when an instruction to be executed in the first core 10-1 and the second core 10-2 is input, the learned neural network model 20 responds to the safety function in which the input instruction is set to be executed in each core. Whether it is a corresponding instruction can be identified. A process in which the learned neural network model monitors whether an instruction executed in the dedicated cores 10-1 and 10-2 is an instruction corresponding to a function different from the safety function will be described in detail with reference to FIG. 3 .

In addition, although one trained neural network model 20 is implemented in FIG. 1 , this is only an example, and the number of learned neural network models 20 may also be determined according to the number of dedicated cores. For example, if there are two dedicated cores, the trained neural network model may be configured with two to monitor the instructions executed on each dedicated core. That is, as shown in FIG. 1 , one trained neural network model 20 can simultaneously monitor instructions executed in a plurality of dedicated cores 10-1 and 10-2, but a plurality of learned neural network models By dedicating these multiple dedicated cores one by one, it is possible to monitor the instructions executed on each dedicated core.

The learned neural network model 20 may be stored in the non-volatile memory and then loaded into the volatile memory under the control of the processor 120 . As an embodiment, when the power of the electronic device 100 is turned on, the neural network model 20 may be loaded in the volatile memory until the power is turned off by the control of the processor 120 . That is, when the power of the electronic device 100 is turned on, the learned neural network model 20 may be loaded into the volatile memory.

As another embodiment, when the electronic device 100 performs a predefined operation, the processor 120 may load the neural network model 20 into the volatile memory. That is, the processor 120 loads the neural network model into the volatile memory only when the electronic device 100 performs a predefined operation, thereby reducing the usage of the processor 120 . An operation for loading the neural network model 20 into the volatile memory may be defined by a user command of the electronic device 100 . That is, when performing a specific operation, the processor 120 may receive a user command to load the neural network model 20 into the volatile memory through an input unit implemented as a keyboard, a touch sensor, a microphone, or the like. Accordingly, when the electronic device 100 performs an operation defined by a user command, the processor 120 may load the neural network model 20 into the volatile memory.

For example, the processor 120 may receive a command to load the neural network model 20 into the volatile memory through a keyboard or a touch sensor when the electronic device 100 is stopped and then moves from the user. can Thereafter, when the electronic device 100 performs an operation of moving after being stopped, the processor 120 may load the learned neural network model 20 stored in the non-volatile memory into the volatile memory. And, it goes without saying that operations defined by a user command received by the processor 120 through an input unit that may be implemented in various devices may be added/deleted/modified.

On the other hand, non-volatile memory is a memory capable of retaining stored information even when power supply is interrupted (for example, flash memory, programmable read-only memory (PROM), magnetoresistive random-access memory (MRAM), and RRAM ( Resistive RAM)). In addition, the volatile memory 110 refers to a memory (eg, dynamic random-access memory (DRAM) and static RAM (SRAM)) that requires continuous power supply in order to maintain stored information.

1 illustrates an embodiment in which the volatile memory is included in the processor 120 as one component of the processor 120 , the volatile memory may be implemented as a component separate from the processor 120 . That is, in the description of the present disclosure, it will be described on the assumption that the volatile memory is included in the processor 120 as one component of the processor 120 , but this describes the operation of the electronic device 100 according to the present disclosure. It is for clarity purposes only, and it goes without saying that the volatile memory according to the present disclosure may be implemented as a component separate from the processor 120 .

The processor 120 may be electrically connected to the memory 120 to control the overall operation of the electronic device 100 . In describing the present disclosure, the processor 120 may include one or a plurality of processors, and may be implemented as a general-purpose processor such as a central processing unit (CPU). However, this is only an embodiment, and the processor 120 may be implemented as a graphics-only processor such as a graphics-processing unit (GPU), a visual processing unit (VPU), or an artificial intelligence-only processor such as a neural processing unit (NPU). Of course you can.

In particular, the processor 120 may include a plurality of cores 10 - 1 , 10 - 2 , 10 - 3 and 10 - 4 . Specifically, a plurality of cores may be included on a die 10 in the processor 120 . The core may be a component including a core processing circuit that performs various operations such as execution of instructions in the processor 120 . Although it is illustrated in FIG. 1 that a plurality of cores are included in one die 10 , this is only an exemplary embodiment and a plurality of die may exist. And, of course, when there are a plurality of dies, a predetermined number of cores may be included in each die.

Meanwhile, as shown in FIG. 1 , among a plurality of cores included in the processor 120 , the first core 10-1 and the second core 10-2 may be set to perform only at least one safety function. have. That is, the first core 10 - 1 and the second core 10 - 2 , which are dedicated cores set to perform only the safety function among the cores included in the processor 120 , have a safety function among the instruction sets included in the memory 110 . Only instructions corresponding to . However, as shown in FIG. 1 , not only two cores among the plurality of cores may be set to execute an instruction corresponding to a safety function, but also one or three or more cores may be set to perform only a safety function. In addition, the process in which the core included in the processor 120 executes the instruction is a process in which the core fetches at least one instruction among the instruction set included in the memory 110 and decodes the fetched instruction. may include.

When at least one instruction is executed in the first core 10-1 and the second core 10-2 while the electronic device 100 operates in a normal state, the processor 120 performs the learned Through the neural network model 20 , it may be identified whether at least one instruction executed in the first core 10 - 1 and the second core 10 - 2 is an instruction corresponding to a safety function. Specifically, when the electronic device 100 operates in a normal state, the first core 10-1 and the second core 10-2 may execute an instruction corresponding to the safety function loaded from the memory 120, , and the other cores 10 - 3 and 10 - 4 except for the dedicated core, an instruction corresponding to a general function loaded from the memory 120 may be executed. At this time, as shown in FIG. 1 , the trained neural network model 20 is an instruction in which the instructions executed in the first core 10-1 and the second core 10-2 correspond to a function different from the safety function. You can monitor and identify whether you are running or not.

On the other hand, the processor 120 provides information on a plurality of instructions corresponding to at least one safety function to be executed in the first core 10-1 and the second core 10-2 (or a context for the plurality of instructions ( context)), the neural network model 20 can be trained. For example, the information about the plurality of instructions may include information about an instruction code (opcode) of each of the plurality of instructions, an operand of the instruction code (opcode), and information about a register corresponding to each of the plurality of instructions. have. The information on the register corresponding to each of the plurality of instructions includes a register including an address where a specific instruction is executed and an instruction that can be executed next is stored (program counter), and a value of a register in which a specific instruction is stored. Alternatively, it may include stack information and the like. The stack refers to a temporary storage device that temporarily stacks data and allocates a part of the memory 110 or registers so that it can be accessed and used by the processor 120 when necessary. Meanwhile, an embodiment in which the neural network model 20 is learned by the processor 120 has been described so far, but this is only an embodiment. That is, the neural network model 20 may be a model previously learned through an external device or an external system, and may also be learned based on the learning data received by the processor 120 from the external server through the communication unit.

On the other hand, the learned neural network model 20 monitors whether an instruction corresponding to the safety function is executed in the dedicated cores 10-1 and 10-2 set to perform the safety function under the control of the processor 120, and can be identified. Specifically, the processor 120 may input instructions executed in the first core 10 - 1 and the second core 10 - 2 to the neural network model 20 learned in real time. In addition, the learned neural network model 20 may output information on whether the input instruction is an instruction corresponding to a safety function to be executed through the dedicated cores 10-1 and 10-2. For example, in the neural network model 20, at least one instruction executed in the dedicated cores 10-1 and 10-2 is an instruction corresponding to a safety function to be performed in the dedicated cores 10-1 and 10-2. Probability values can be output.

In one embodiment, when the obtained probability value is less than or equal to the threshold range, the processor 120 may identify the instruction executed in the dedicated cores 10-1 and 10-2 as an instruction corresponding to a function different from the safety function. . In another embodiment, when the obtained probability exceeds the threshold range, the processor 120 identifies that the instruction executed in the dedicated cores 10-1 and 10-2 is an instruction corresponding to the previously learned safety function. can do.

On the other hand, the processor 120 may determine the critical range according to the safety integrity standard (Safety Integrity Level) used in ANSI/ISA-S84.01 and IEC 61508. For example, as the safety integrity criterion of the safety function corresponding to the instruction to be executed by the dedicated cores 10 - 1 and 10 - 2 is higher, the processor 120 may determine the critical range as a high value. For example, if the safety function with the highest safety integrity standard level among safety functions corresponding to the instruction to be executed in the dedicated cores 10-1 and 10-2 is SIL 3, the processor 120 is a safety function among the safety functions. The critical range can be determined as a higher value than when the safety function with the highest integrity criterion is rated SIL 2. Accordingly, by increasing the critical range as the safety integrity standard of the safety function to be performed by the dedicated cores 10-1 and 10-2 executing the instruction is higher, the processor 120 assigns the instruction corresponding to the general function to the safety function. It is possible to reduce the probability of erroneous judgment with the corresponding instruction. However, this is only an example, and the threshold range may be a preset value according to the type of the electronic device 100 . Also, it goes without saying that the threshold range can be changed by a user command.

In addition, the processor 120 may determine the operating state of the electronic device 100 as either a safe state or a normal state based on the identification result. Specifically, when it is identified that the at least one instruction is an instruction corresponding to a function different from the safety function, the processor 120 may change the operating state of the electronic device 100 from the normal state to the safe state. When the state of the electronic device 100 is switched to the safe state, the processor 120 may provide a message indicating that an instruction corresponding to a general function different from the safety function is identified to at least one core. For example, the processor 120 may control the communication unit to transmit a message indicating that an instruction corresponding to a general function is identified to the terminal device of the user previously registered in the memory 110 . As another example, the processor 120 may control the display to display a message indicating that an instruction corresponding to a general function is identified, or may output the message in a voice form.

As another example, when the operating state of the electronic device 100 is switched to the safe state, the processor 120 may stop a normal operation performed by the electronic device 100 or cut off the power of the electronic device 100 . For example, when the electronic device 100 performs a normal operation by executing an instruction corresponding to a general function different from the safety function in the dedicated cores 10-1 and 10-2, the processor 120 is the current electronic device. A normal operation performed by the device 100 may be stopped or power may be cut off.

Meanwhile, when the state of the electronic device 100 is switched to the safe state, an operation to be controlled by the processor 120 may be determined by the safety integrity criterion. For example, when the safety integrity standard level of the safety function to be executed in the dedicated cores 10-1 and 10-2 is less than or equal to a preset level, when the operating state is switched to the safe state, the processor 120 is configured to correspond to the general function. Only a message indicating that the instruction has been identified is provided, and an operation of stopping an operation of a general function or turning off the power of the electronic device 100 may be controlled not to be performed. In addition, when the safety integrity standard of the safety function to be executed in the dedicated cores 10-1 and 10-2 exceeds a preset grade, when the safety state is switched, the processor 120 identifies an instruction corresponding to the general function. may control to stop a general function being performed by the electronic device 100 or cut off power while providing a message. However, this is only an exemplary embodiment, and the operation when the safe state is switched may be implemented in various ways, and of course, may be changed/added/modified by a user command.

On the other hand, when various functions of the electronic device 100 are updated through package update or the like, the processor 120 determines that an instruction to be executed in the dedicated cores 10 - 1 and 10 - 2 stored in the memory 110 is different from the safety function. Whether it is an instruction corresponding to can be identified through the learned neural network 20 . When various functions of the electronic device 100 are updated through package update, a plurality of instructions stored in the memory 110 may be added/modified/deleted. In this case, some of the instructions set to be executed by the dedicated cores 10-1 and 10-2 may include instructions corresponding to functions different from the safety function. Accordingly, when various functions of the electronic device are updated, the processor 120 determines whether an instruction to be executed in the dedicated cores 10-1 and 10-2 among the instructions stored in the memory 110 is an instruction corresponding to the safety function. It can be identified through the neural network model 20 . Since the process of identifying whether the instruction to be executed in the dedicated cores 10-1 and 10-2 through the learned neural network model 20 is an instruction corresponding to a function different from the safety function has been described above, the redundant description will be omitted.

And, when it is identified that some of the instructions to be executed in the dedicated cores 10-1 and 10-2 stored in the memory 110 are instructions corresponding to a function different from the safety function, the processor 120 changes the operation state from the normal state to the safe state. state can be switched.

Functions related to artificial intelligence according to the present disclosure are operated through a processor and a memory. The processor may consist of one or a plurality of processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a digital signal processor (DSP), or the like, a graphics-only processor such as a GPU, a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. One or a plurality of processors control to process input data according to a predefined operation rule or artificial intelligence model stored in the memory. Alternatively, when one or more processors are AI-only processors, the AI-only processor may be designed with a hardware structure specialized for processing a specific AI model.

The predefined action rule or artificial intelligence model is characterized in that it is created through learning. Here, being made through learning means that a basic artificial intelligence model is learned using a plurality of learning data by a learning algorithm, so that a predefined action rule or artificial intelligence model set to perform a desired characteristic (or purpose) is created means burden. Such learning may be performed in the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and/or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

The artificial intelligence model may be composed of a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values, and a neural network operation is performed through an operation between an operation result of a previous layer and a plurality of weight values. The plurality of weights of the plurality of neural network layers may be optimized by the learning result of the artificial intelligence model. For example, a plurality of weights may be updated so that a loss value or a cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. The artificial neural network may include a deep neural network (DNN), for example, a Convolutional Neural Network (CNN), a Deep Neural Network (DNN), a Recurrent Neural Network (RNN), a Restricted Boltzmann Machine (RBM), There may be a Deep Belief Network (DBN), a Bidirectional Recurrent Deep Neural Network (BRDNN), or a Deep Q-Networks, but is not limited to the above-described example.

2 is a flowchart illustrating an operation of the electronic device 100 according to an embodiment of the present disclosure. First, while the electronic device 100 operates in a normal state, at least one instruction may be executed in at least one core included in the processor 120 ( S210 ). In this case, the at least one core may be a dedicated core configured to execute only instructions corresponding to the safety function. And, when the electronic device 100 operates in the normal state, while the instruction corresponding to the safety function is executed in at least one core, the other cores except for at least one core of the plurality of cores execute the instruction corresponding to the general function. can

Then, the electronic device 100 may obtain a probability that the instruction executed by the dedicated core is an instruction corresponding to the safety function (S220). Specifically, the electronic device 100 may input an instruction executed by a dedicated core to the learned neural network model. The learned neural network model may identify whether the input instruction is an instruction corresponding to a safety function set to be executed in the dedicated core through a probability value.

Meanwhile, the electronic device 100 may train the neural network model based on information about a plurality of instructions corresponding to at least one safety function to be executed in the dedicated core. In this case, the information on the plurality of instructions (or context information on the plurality of instructions) may include an instruction code of each of the plurality of instructions, an argument of the instruction code, and information about a register corresponding to each of the plurality of instructions. have. In addition, the neural network model learned by the electronic device 100 may output a probability that an instruction corresponding to a safety function exists among instructions executed in the dedicated core. For example, as the pattern of the instruction currently executed in the dedicated core is different from the pattern of the instruction corresponding to the safety function, the neural network model may output a low probability value.

Then, the electronic device 100 may identify whether the probability value output by the learned neural network model is less than or equal to a threshold range ( S230 ). In this case, the electronic device 100 may determine the critical range according to the safety integrity standards used in ANSI/ISA-S84.01 and IEC 61508. For example, as the safety integrity standard of the safety function corresponding to the instruction to be executed by the dedicated cores 10 - 1 and 10 - 2 is higher, the electronic device 100 may determine the critical range as a high value. Accordingly, by increasing the critical range as the safety integrity standard of the safety function to be performed by the dedicated cores 10-1 and 10-2 is higher, the electronic device 100 converts the instruction corresponding to the general function to the safety function. It is possible to reduce the probability of erroneous judgment with the instruction corresponding to .

In one embodiment, the probability value exceeding the threshold range may mean that the instruction executed in the dedicated core has a high probability of being an instruction corresponding to a safety function. Accordingly, when the probability value output by the neural network model exceeds the threshold range, the electronic device 100 operates in a normal state to perform a normal function and a safety function.

As another embodiment, if the probability value is less than or equal to the critical range, it may mean that there is a high probability that the instruction executed in the dedicated core is an instruction corresponding to a function different from the safety function. Accordingly, when the probability value output by the neural network model is less than or equal to the threshold range, the electronic device 100 may change the operation state from the normal state to the safe state ( S240 ). When the electronic device 100 is switched to the safe state, the electronic device 100 may control to stop the current normal operation or the safe operation or to cut off the power of the electronic device 100 . As another example, the electronic device 100 may provide a message indicating that an instruction corresponding to a function different from the safety function is executed in the dedicated core. In this case, the electronic device 100 may output the message as a voice message or display the message. In addition, the electronic device 100 may output a notification sound corresponding to the message.

3 is for explaining a process of identifying whether an instruction executed in a dedicated core through a learned neural network model of the electronic device 100 is an instruction corresponding to a function different from a stable function, according to an embodiment of the present disclosure; It is a drawing.

As an embodiment of the present disclosure, the electronic device 100 inputs an instruction set 300 executed in a first core among at least one core set to only perform a safety function to the learned neural network model 20 . can do. The learned neural network model 20 may compare an instruction pattern included in the input instruction set with an instruction pattern corresponding to a safety function set to be executed in the first core. In addition, the learned neural network model 20 may output a probability value based on whether a pattern of instructions included in the instruction set is different from a pattern of instructions corresponding to a safety function. In this case, the probability value may be a probability value that some of the instructions executed in the first core are instructions corresponding to the safety function.

For example, as shown in FIG. 3 , when it is identified that two instructions (IMM R3, 0X4 and STORE R0, R2) among the plurality of instruction sets 300 are different from the pattern of instructions corresponding to the safety function, The learned neural network model 20 may output a low probability value. And, when the probability value is less than or equal to the threshold range, the learned neural network model 20 may output a signal 320 indicating that an abnormal pattern has been detected. When a signal indicating that the abnormal pattern is detected is output, the electronic device 100 may change the operation state from the normal state to the safe state.

Meanwhile, as another embodiment, the electronic device 100 may transmit information on the detected abnormal pattern to a server managing the electronic device 100 or a device of a manager. In addition, when receiving a signal from the server or the manager's device indicating that there is no problem in the abnormal pattern detected by the electronic device 100 , the electronic device 100 may change the operation state from the safe state to the normal state. As another example, upon receiving information capable of resolving a problem related to a detected abnormal pattern from the device of the server or manager, the electronic device 100 determines that an instruction executed in a dedicated core corresponds to a safety function based on the received information. Some of the plurality of instructions stored in the memory 110 may be modified to become the instructions. For example, the electronic device 100 may delete or modify the two instructions (IMM R3, 0X4, and STORE R0, R2) for detecting an abnormal pattern.

4 is a diagram for explaining an operation of the electronic device 100 according to an embodiment of the present disclosure. Although FIG. 4 illustrates a case in which the electronic device 100 is implemented as a retail bot, this is only an example and may be implemented in various devices.

When the electronic device 100 implemented as a retail bot operates in a normal state, it may provide a message 400 for responding to a customer and simultaneously perform a predefined general function and a safety function. For example, a general function performed by the electronic device 100 while operating in a normal state may be a function of adjusting the speed of the electronic device 100 . In addition, the safety function capable of monitoring or controlling the speed control function of the electronic device 100 may be the highest speed violation monitoring function. Accordingly, the electronic device 100 may perform a message or an operation to respond to a customer while performing a general function and a safety function corresponding to the moving speed.

Meanwhile, when a command or signal for updating various functions is received from an external device (eg, an external server, a recording medium, etc.) or a user, the electronic device 100 may update a general function or a safety function. For example, as shown in FIG. 4 , upon receiving update information on the speed control function received from the external device, the electronic device 100 may add a speed control function using a new sensor ( 410 ). Meanwhile, even if various general functions are updated, instructions executed by the dedicated core set to execute only instructions corresponding to the safety function should not be affected. That is, when the update of the general function does not affect the safety function, the electronic device 100 may guarantee that the safety function can be normally performed.

Accordingly, the electronic device 100 may monitor an instruction to be executed in a dedicated core set to perform only a safety function among a plurality of cores. Specifically, the electronic device 100 may identify whether an instruction to be executed by the dedicated core is an instruction corresponding to a safety function. For example, the electronic device 100 may obtain a probability value by comparing a pattern of an instruction to be executed in the dedicated core with a pattern of an instruction corresponding to a safety function. In this case, the probability value may be a probability value that the instruction executed in the dedicated core is an instruction corresponding to the safety function. When the obtained probability value is less than or equal to the threshold range, the electronic device 100 may identify the instruction to be executed by the dedicated core as an instruction corresponding to a function different from the safety function.

When it is identified that the instruction to be executed in the dedicated core is an instruction corresponding to a function different from the safety function, the electronic device 100 may determine that a problem in the safety function is found ( 420 ). That is, when it is identified that the instruction to be executed by the dedicated core is an instruction corresponding to a function different from the safety function, the electronic device 100 may identify that there is a high probability that the safety function cannot be normally performed. In addition, the electronic device 100 may change the operating state from the normal state to the safe state. When the electronic device 100 operates in a normal state even though a problem in the safety function is found, the maximum speed violation monitoring function does not operate properly, and other problems may occur.

Accordingly, the electronic device 100 operates in the safe state to provide a message 430 indicating that an instruction corresponding to a function different from the safety function has been executed in the dedicated core (eg, 'transition to the safe state', etc.). can The message may be output in voice form as shown in FIG. 4 , but this is only an example and may be displayed on a display. As another example, the electronic device 100 may cut off power or stop an operation of a general function or a safety function currently being performed. For example, the electronic device 100 may stop an operation of providing a response message to a customer while moving and provide a message indicating that a problem related to a safety function has occurred at the current location.

As another example, when the operating state is switched to the safe state, the electronic device 100 transmits information related to an instruction corresponding to a function different from the safety function identified as being executed in the dedicated core to a server or a manager managing the electronic device 100 . can be transmitted to the device of In addition, the electronic device 100 may determine an operation state based on information about an instruction received from a device of a server or a manager. For example, when receiving information that an instruction executed on a dedicated core corresponds to a safety function from the device of the server or manager, the electronic device 100 may change the operation state from the stable state back to the normal state . And as another example, when receiving information to delete or modify an instruction from the device of the server or administrator, the electronic device 100 modifies some of the plurality of instructions stored in the memory 110 based on the received information to obtain a dedicated core can be controlled to execute only the instruction corresponding to the safety function. In addition, the electronic device 100 may change the operating state from the safe state to the normal state.

5 is a sequence diagram illustrating an operation between the electronic device 100 and the server 200 according to an embodiment of the present disclosure. 5 is a sequence diagram for explaining a case in which a learned neural network model capable of identifying whether an instruction executed in a dedicated core of the electronic device 100 is an instruction corresponding to a function different from a safety function is stored in the server 200 to be.

First, the electronic device 100 may transmit an instruction executed in at least one core to the server 200 including the learned neural network model (S510). In this case, the at least one core may be a dedicated core configured to execute only instructions corresponding to the safety function. The server 200 may acquire information related to the received instruction through the learned neural network model (S520). The information related to the instruction obtained by the server 200 includes a probability value that the instruction received from the electronic device 100 is an instruction corresponding to a function different from the safety function, information about an abnormal pattern detected on the received instruction, and detected abnormality. Correction information for resolving the pattern may be included. Then, the server 200 may transmit information on the acquired instruction to the electronic device 100 (S530).

Then, the electronic device 100 may determine an operation state based on information about the instruction received from the server 200 ( S540 ). In an embodiment, when information is received from the server 200 that the probability that the instruction executed by the dedicated core is an instruction corresponding to a function different from the safety function is less than or equal to a threshold range, the electronic device 100 sets the operation state to the normal state. can be switched to a safe state. And, while operating in the safe state, the electronic device 100 may modify the plurality of instructions stored in the memory 110 based on the correction information for resolving the abnormal pattern received from the server 200 .

As another example, when information is received from the server 200 that the probability that the instruction executed by the dedicated core is an instruction corresponding to a function different from the safety function exceeds the threshold range, the electronic device 100 operates in a normal state while It can perform general and safety functions.

6 is a block diagram illustrating in detail the configuration of the electronic device 100 according to an embodiment of the present disclosure. 6 , the electronic device 100 includes a memory 110 , a processor 120 , a communication unit 130 , a display 140 , an input unit 150 , a driving unit 160 , a speaker 170 and A sensor 180 may be included. Meanwhile, since the memory 110 and the processor 120 have been described in detail with reference to FIG. 1 , redundant descriptions will be omitted.

The communication unit 130 includes a circuit and may communicate with a server (not shown) or an external device (not shown). Specifically, the processor 120 may receive and transmit various data or information from a server (not shown) or an external device (not shown) connected through the communication unit 130 . In addition, while the electronic device 100 operates in a normal state, the communication unit 130 may transmit to a server including the learned neural network model. That is, when a neural network model that can identify whether an instruction executed on a dedicated core is an instruction corresponding to a function different from the safety function is stored in an external server, the communication unit 130 executes on the dedicated core under the control of the processor 120 instruction can be transmitted to an external server. In addition, the communication unit 130 may receive information related to whether the instruction executed in the dedicated core corresponds to a function different from the safety function from the server including the neural network model. Accordingly, the processor 120 may determine whether to switch the operating state to the safe state through the information received through the communication unit 130 .

In addition, the communication unit 130 may receive training data for learning the neural network model stored in the memory 120 from an external server. The training data may include information about a command code of each of the plurality of instructions, an argument of the command code, and information about a register corresponding to each of the plurality of instructions.

Meanwhile, the communication unit 130 may include various communication modules to communicate with an external device. For example, the communication unit 130 may include a wireless communication module, for example, LTE, LTE Advance (LTE-A), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications) system), a cellular communication module using at least one of Wireless Broadband (WiBro), 5th generation (5G), or Global System for Mobile Communications (GSM). As another example, the wireless communication module may include, for example, at least one of wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), and Zigbee.

The display 140 may be implemented as a Liquid Crystal Display Panel (LCD), Organic Light Emitting Diodes (OLED), or the like, and in some cases may be implemented as a flexible display, a transparent display, or the like. Also, the display 140 may be implemented as a touch screen together with a touch panel. However, it is not limited to the above-described implementation, and the display 140 may be implemented differently depending on the type of the electronic device 100 .

In addition, the display 140 may display various information under the control of the processor 120 . In particular, the display 140 may display a message or an indicator that an instruction corresponding to a function different from the safety function is identified in the dedicated core.

The input unit 150 includes a circuit, and the processor 120 may receive a user command for controlling the operation of the electronic device 100 through the input unit 150 . The input unit 150 may include a touch sensor, a (digital) pen sensor, a pressure sensor, a key, or a microphone. The touch sensor may use, for example, at least one of a capacitive type, a pressure sensitive type, an infrared type, and an ultrasonic type.

In particular, a command for updating a new function may be input through the input unit 150 . In addition, the input unit 150 may receive an operation related to an operation of loading the neural network model stored in the non-volatile memory into the volatile memory. In addition, the input unit 150 may receive a user command for changing the operating state of the electronic device.

The driving unit 160 is configured to move the electronic device 100 and may include a motor and a plurality of wheels. Specifically, the driving unit 160 may change the moving direction and the moving speed of the electronic device 100 under the control of the processor 120 . And, when the safety state is switched, the driving unit 160 may stop the operation of the electronic device 100 under the control of the processor 120 .

The speaker 170 is configured to output not only various audio data on which various processing operations such as decoding, amplification, and noise filtering have been performed by an audio processing unit (not shown) but also various notification sounds or voice messages. In particular, the speaker 170 may output a message indicating that the instruction executed in the dedicated core is identified as an instruction corresponding to a function different from the safety function in a voice form or may output a notification sound.

The sensor 180 may detect various state information of the electronic device 100 . For example, the sensor 180 may include a motion sensor (eg, a gyro sensor, an acceleration sensor, etc.) capable of detecting motion information of the electronic device 100, and a sensor (eg, GPS) capable of detecting location information. (Global Positioning System) sensor), a sensor capable of detecting environmental information around the electronic device 100 (eg, a temperature sensor, a humidity sensor, a barometric pressure sensor, etc.), a sensor capable of detecting user information of the electronic device 100 It may include a sensor (eg, a blood pressure sensor, a blood sugar sensor, a pulse rate sensor, etc.), a sensor capable of detecting the user's presence (eg, a camera, a UWB sensor, an IR sensor, a proximity sensor, an optical sensor, etc.). In addition, the sensor 180 may further include an image sensor for photographing the outside of the electronic device 100 .

7 is a flowchart illustrating a method of controlling the electronic device 100 according to an embodiment of the present disclosure.

First, when at least one instruction is executed in at least one core while operating in a normal state, the electronic device 100 may identify whether the at least one instruction is an instruction corresponding to a safety function through the learned neural network model. There is (S710). In this case, the at least one core may be a dedicated core that executes only instructions corresponding to the safety function. That is, the electronic device 100 may identify whether an instruction executed by the dedicated core corresponds to a safety function. Specifically, the electronic device 100 may obtain a probability value by comparing a pattern of instructions executed in the dedicated core with a pattern of instructions set to be executed in the dedicated core through the learned neural model. The probability value may be a probability value that the instruction executed in the dedicated core is an instruction corresponding to a safety function. In addition, the electronic device 100 may identify whether at least one instruction executed by the dedicated core is an instruction corresponding to a function different from the safety function based on the probability value obtained through the neural network model.

Based on the identification result, the electronic device 100 may determine the operation state as either a normal state or a safe state (S720). Specifically, when the probability value obtained through the learned neural network model is less than or equal to the threshold range, the electronic device 100 may identify that an instruction corresponding to a function different from the safety function is being executed in the dedicated core. At this time, the electronic device 100 may switch the operating state to the safe state. In addition, the electronic device 100 may switch the operating state to a safe state to cut off power or stop a general function currently being performed. As another example, the electronic device 100 may provide a message indicating that an instruction corresponding to a function different from the safety function is being executed in the dedicated core while transitioning to the safe state.

Meanwhile, when the probability value obtained through the learned neural network exceeds the threshold range, the electronic device 100 may identify that the instruction corresponding to the safety function is executed in the dedicated core. In this case, the electronic device 100 may maintain an operating state in a normal state.

On the other hand, the drawings attached to the present disclosure are not intended to limit the technology described in the present disclosure to specific embodiments, and various modifications, equivalents, and/or alternatives of the embodiments of the present disclosure are provided. should be understood as including In connection with the description of the drawings, like reference numerals may be used for like components.

In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.

In this disclosure, expressions such as “A or B,” “at least one of A and/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, "A or B," "at least one of A and B," or "at least one of A or B" means (1) includes at least one A, (2) includes at least one B; Or (3) it may refer to all cases including both at least one A and at least one B.

As used in the present disclosure, expressions such as "first," "second," "first," or "second," may modify various elements, regardless of order and/or importance, and refer to one element. It is used only to distinguish it from other components, and does not limit the components.

A component (eg, a first component) is "coupled with/to (operatively or communicatively)" to another component (eg, a second component) When referring to "connected to", it will be understood that the certain element may be directly connected to the other element or may be connected through another element (eg, a third element). On the other hand, when it is said that a component (eg, a first component) is "directly connected" or "directly connected" to another component (eg, a second component), the component and the It may be understood that other components (eg, a third component) do not exist between other components.

The expression "configured to (or configured to)" as used in this disclosure depends on the context, for example, "suitable for," "having the capacity to" ," "designed to," "adapted to," "made to," or "capable of." The term “configured (or configured to)” may not necessarily mean only “specifically designed to” in hardware. Instead, in some circumstances, the expression “a device configured to” may mean that the device is “capable of” with other devices or parts. For example, the phrase "a coprocessor configured (or configured to perform) A, B, and C" may include a processor dedicated to performing the operations (eg, an embedded processor), or executing one or more software programs stored in a memory device. By doing so, it may mean a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

Various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage media readable by a machine (eg, a computer). As a device that can be called and operated according to the called command, it may include an electronic device (eg, the electronic device 100) according to the disclosed embodiments. When the command is executed by the processor, the processor directly or A function corresponding to the instruction may be performed using other components under the control of the processor. The instruction may include code generated or executed by a compiler or an interpreter. A device-readable storage medium includes: It may be provided in the form of a non-transitory storage medium, where the 'non-transitory storage medium' does not include a signal and means that it is tangible and data is semi-permanent in the storage medium Alternatively, temporary storage is not distinguished, for example, a 'non-transitory storage medium' may include a buffer in which data is temporarily stored.

According to an embodiment, the method according to various embodiments disclosed in the present disclosure may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product may be distributed in the form of a machine-readable storage medium (eg, compact disc read only memory (CD-ROM)) or online through an application store (eg, Play Store™). In the case of online distribution, at least a portion of the computer program product (eg, a downloadable app) is at least temporarily stored in a storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server, or is temporarily stored can be created with

Each of the components (eg, a module or a program) according to various embodiments may be composed of a singular or a plurality of entities, and some of the above-described sub-components may be omitted, or other sub-components may be various. It may be further included in the embodiment. Alternatively or additionally, some components (eg, a module or a program) may be integrated into a single entity to perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by a module, program, or other component may be sequentially, parallel, repetitively or heuristically executed, or at least some operations may be executed in a different order, omitted, or other operations may be added. can

Claims (20)

In an electronic device,
Memory; and
a processor including at least one core configured to execute an instruction corresponding to at least one safety function among the plurality of cores; and
The processor is
When at least one instruction is executed by the at least one core while the electronic device operates in the first state, whether the at least one instruction is an instruction corresponding to the safety function through a learned neural network model to identify,
An electronic device that determines the operating state of the electronic device as one of the first state and the second state based on the identification result. According to claim 1,
The processor is
The at least one core is an electronic device, characterized in that it is a dedicated core that executes only instructions corresponding to the safety function. 3. The method of claim 2,
The neural network model is a model learned based on information about instructions corresponding to the safety function,
Information about the plurality of instructions,
and information on a command code (opcode) of each of the plurality of instructions, an operand of the command code, and a register corresponding to each of the plurality of instructions. According to claim 1,
The processor is
obtaining a probability that at least one instruction executed in the at least one core is an instruction corresponding to the safety function through the neural network model;
The electronic device identifies the at least one instruction as an instruction corresponding to a function different from the safety function when the obtained probability is less than or equal to a threshold range. 5. The method of claim 4,
The critical range is
The electronic device, characterized in that determined by the safety integrity level (Safety Integrity Level) of the safety function. 5. The method of claim 4,
The processor is
When the at least one instruction is identified as an instruction corresponding to the safety function, the operation state of the electronic device is maintained in the second state;
If the at least one instruction is identified as an instruction corresponding to a function different from the safety function, the electronic device switches the operating state of the electronic device to the second state. According to claim 1,
The processor is
When an instruction corresponding to a general function to be performed in the remaining cores except for the at least one core among the plurality of cores is updated, the safety function among the instructions to be executed in the at least one core stored in the memory using the neural network model; An electronic device for identifying whether instructions corresponding to different functions exist. According to claim 1,
The processor is
When the operating state of the electronic device is switched to the second state, the electronic device controls to stop a normal operation performed by the electronic device or to cut off the power of the electronic device. According to claim 1,
The processor is
When the operating state of the electronic device is switched to the second state, the electronic device provides a message indicating that an instruction corresponding to a function different from the safety function is identified to the at least one core. According to claim 1,
The processor is
Communication unit; further comprising,
While the electronic state operates in the first state, controlling the communication unit to transmit at least one instruction executed in the at least one core to a server including a learned neural network,
The electronic device that receives, through the communication unit, information on whether the at least one instruction is an instruction corresponding to a function different from the safety function from the server. A method of controlling an electronic device including a processor including at least one core configured to execute an instruction corresponding to at least one safety function among a plurality of cores, the method comprising:
When at least one instruction is executed in the at least one core while the electronic device operates in the first state, it is identified whether the at least one instruction is an instruction corresponding to the safety function through a learned neural network model step; and
and determining the operating state of the electronic device as one of the first state and the second state based on the identification result. 12. The method of claim 11,
The control method, characterized in that the at least one core is a dedicated core that executes only instructions corresponding to the safety function. 13. The method of claim 12,
The neural network model is a model learned based on information about instructions corresponding to the safety function,
Information about the plurality of instructions,
and information on an instruction code (opcode) of each of the plurality of instructions, an operand of the instruction code, and information on a register corresponding to each of the plurality of instructions. 13. The method of claim 12,
The identifying step is
obtaining a probability that at least one instruction executed in the at least one core is an instruction corresponding to the safety function through the neural network model; and
and when the obtained probability is less than or equal to a threshold range, identifying the at least one instruction as an instruction corresponding to a function different from the safety function. 145. The method of claim 143,
The critical range is
Control method, characterized in that determined by the safety integrity level (Safety Integrity Level) of the safety function. 15. The method of claim 14,
The determining step is
When the at least one instruction is identified as an instruction corresponding to the safety function, the operation state of the electronic device is maintained in the second state;
and switching the operation state of the electronic device to the second state when the at least one instruction is identified as an instruction corresponding to a function different from the safety function. 12. The method of claim 11,
The identifying step is
When an instruction corresponding to a general function to be performed in the remaining cores except for the at least one core among the plurality of cores is updated, among the instructions to be executed in the at least one core stored in the memory of the electronic device using the neural network model, the A control method comprising a; identifying whether an instruction corresponding to a function different from the safety function exists. 12. The method of claim 11,
The conversion step is
and controlling to stop a normal operation performed by the electronic device or cut off power to the electronic device when the operating state of the electronic device is switched to the second state. 12. The method of claim 11,
The conversion step is
and providing a message indicating that an instruction corresponding to a function different from the safety function is identified to the at least one core when the operating state of the electronic device is switched to the second state. 12. The method of claim 11,
while the electronic state operates in the first state, transmitting at least one instruction executed in the at least one core to a server including a learned neural network; and
and receiving, from the server, information on whether the at least one instruction is an instruction corresponding to a function different from the safety function.

Images