KR20230171786A - Electronic apparatus and method of controlling thereof - Google Patents

Abstract

An electronic device is disclosed. The electronic device according to the present disclosure includes a memory that stores at least one instruction, a communication interface, a display, and a processor that executes the at least one instruction, where the processor acquires a 3D object corresponding to a device located in the factory, , upon receiving a user input to select a device from the list of devices located in the factory, the operation information of the device is received, the display is controlled to render a 3D object, and the operation of the device is preset based on the received operation information of the device. If it is determined to be out of range, the display is controlled to display a notification message.

Description

Electronic device and method of controlling the same { ELECTRONIC APPARATUS AND METHOD OF CONTROLLING THEREOF }

This disclosure relates to an electronic device and a control method thereof, and more specifically, to an electronic device and a control method for monitoring a production site within a factory by implementing the environment within the factory as a 3D object.

CPPS (Cyber Physical Production Systems) is a cyber-physical production system for smart factory design and operation. It is an integrated system that integrates the real physical world and the virtual world, including production systems such as products, processes, and facilities.

Although the construction and use of CPPS is a key technological element for realizing a smart factory, there were difficulties in introducing CPPS at small and medium-sized manufacturing companies due to cost and design difficulties.

The present disclosure is intended to solve the above-mentioned difficulties, and the purpose of the present disclosure is to provide an electronic device and a control method for real-time monitoring of devices in a factory using 3D objects and motion information of devices located in the factory.

An electronic device according to an embodiment of the present disclosure for achieving the above-described object includes a memory storing at least one instruction, a communication interface, a display, and a processor executing the at least one instruction, The processor acquires a 3D object corresponding to a device located in the factory, receives a user input for selecting the device from a list of devices located in the factory, receives operation information of the device, and operates on the received operation information. The display is controlled to render the 3D object based on the received operation information of the device, and if the operation of the device is determined to be outside a preset range based on the received operation information of the device, the display is controlled to display a notification message.

And, when at least one attribute information of color, position, angle, size, and rotation information of the 3D object changes according to a user input, the processor configures the display to display the 3D object based on the changed attribute information. You can control it.

Additionally, when the motion information is changed according to a user input, the processor may transmit the changed motion information to the device or an external device and control the display to render the 3D object based on the changed motion information. there is.

Additionally, the processor may obtain device information connected to the device and control the display to display the connected device information when the 3D object is selected by user input.

In addition, the processor sets a viewpoint in a virtual space displayed on the display, and when at least one viewpoint information among the position, direction, and viewing angle of the viewpoint changes according to the user's input, based on the changed viewpoint information. Thus, the display can be controlled to render the 3D object.

Meanwhile, a 3D object display method according to another embodiment of the present disclosure includes obtaining a 3D object corresponding to a device located in a factory, upon receiving a user input for selecting the device from a list of devices located in the factory, Receiving operation information of the device, rendering the 3D object based on the received operation information, and notifying if the operation of the device is determined to be outside a preset range based on the received operation information of the device. It includes the step of displaying a message.

And, when at least one attribute information of the color, position, angle, size, and rotation information of the 3D object changes according to the user input, the method may further include displaying the 3D object based on the changed attribute information. .

Additionally, when the motion information is changed according to a user input, the method may further include transmitting the changed motion information to the external device and rendering the 3D object based on the changed motion information.

The method may further include obtaining device information connected to the device and displaying the connected device information when the 3D object is selected by user input.

In addition, when setting a viewpoint in a virtual space displayed on the display and at least one viewpoint information among the position, direction, and viewing angle of the viewpoint is changed according to the user's input, the 3D display is displayed based on the changed viewpoint information. A step of rendering the object may be further included.

Meanwhile, in a computer-readable recording medium including a program for executing a 3D object display method according to another embodiment of the present disclosure, the 3D object display method includes obtaining a 3D object corresponding to a device located in a factory. Step, when a user input for entering operation information and communication setting information of the virtual device is received, and when a user input for selecting the device from a list of devices located in the factory is received, operation information of the device is received, It includes rendering the 3D object based on received motion information, and displaying a notification message when it is determined that the operation of the device is outside a preset range based on the received motion information of the device.

1 is a diagram for explaining an electronic device according to an embodiment of the present disclosure;
2 is a block diagram for explaining the configuration of an electronic device according to an embodiment of the present disclosure;
3A to 3D are diagrams for explaining a UI for inputting setting information of a 3D object created according to an embodiment of the present disclosure;
4A and 4B are diagrams for explaining a 3D object rendered according to a movement of a viewpoint displayed on a display according to an embodiment of the present disclosure;
5 is a diagram for explaining an electronic device that displays device information corresponding to a 3D object according to an embodiment of the present disclosure;
6 is a diagram for explaining an electronic device that monitors a device corresponding to a 3D object according to an embodiment of the present disclosure, and
FIG. 7 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail.

The terms used in the embodiments of the present disclosure have selected general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but this may vary depending on the intention or precedent of a technician working in the art, the emergence of new technology, etc. . In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description part of the relevant disclosure. Therefore, the terms used in this disclosure should be defined based on the meaning of the term and the overall content of this disclosure, rather than simply the name of the term.

Hereinafter, various embodiments of this document are described with reference to the attached drawings. However, this is not intended to limit the technology described in this document to specific embodiments, and should be understood to include various modifications, equivalents, and/or alternatives to the embodiments of this document. . In connection with the description of the drawings, similar reference numbers may be used for similar components.

In this document, expressions such as “have,” “may have,” “includes,” or “may include” refer to the existence of the corresponding feature (e.g., a numerical value, function, operation, or component such as a part). , and does not rule out the existence of additional features.

In this document, expressions such as “A or B,” “at least one of A or/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B” includes (1) at least one A, (2) at least one B, or (3) it may refer to all cases including both at least one A and at least one B.

Expressions such as “first,” “second,” “first,” or “second,” used in this document can modify various components regardless of order and/or importance, and can refer to one component. It is only used to distinguish from other components and does not limit the components. For example, a first user device and a second user device may represent different user devices regardless of order or importance. For example, a first component may be renamed a second component without departing from the scope of rights described in this document, and similarly, the second component may also be renamed to the first component.

Terms such as “module,” “unit,” and “part” used in this document are terms to refer to components that perform at least one function or operation, and these components are implemented in hardware or software. Alternatively, it can be implemented through a combination of hardware and software. In addition, a plurality of "modules", "units", "parts", etc. are integrated into at least one module or chip, except in cases where each needs to be implemented with individual specific hardware, and is integrated into at least one processor. It can be implemented as:

A component (e.g., a first component) is “(operatively or communicatively) coupled with/to” another component (e.g., a second component). When referred to as being “connected to,” it should be understood that any component may be directly connected to the other component or may be connected through another component (e.g., a third component). On the other hand, when a component (e.g., a first component) is said to be “directly connected” or “directly connected” to another component (e.g., a second component), It may be understood that no other component (e.g., a third component) exists between other components.

As used in this document, the expression “configured to” depends on the situation, for example, “suitable for,” “having the capacity to.” ," can be used interchangeably with "designed to," "adapted to," "made to," or "capable of." The term “configured (or set to)” may not necessarily mean “specifically designed to” in hardware. Instead, in some contexts, the expression “a device configured to” may mean that the device is “capable of” working with other devices or components. For example, the phrase "processor configured (or set) to perform A, B, and C" refers to a processor dedicated to performing the operations (e.g., an embedded processor), or by executing one or more software programs stored on a memory device. , may refer to a general-purpose processor (e.g., CPU or application processor) capable of performing the corresponding operations.

Terms used in this document are merely used to describe specific embodiments and may not be intended to limit the scope of other embodiments. Singular expressions may include plural expressions, unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by a person of ordinary skill in the technical field described in this document. Among the terms used in this document, terms defined in general dictionaries may be interpreted to have the same or similar meaning as the meaning they have in the context of related technology, and unless clearly defined in this document, have an ideal or excessively formal meaning. It is not interpreted as In some cases, even terms defined in this document cannot be interpreted to exclude embodiments of this document.

Below, with reference to the attached drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily practice them. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present disclosure in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

In this disclosure, the term user may refer to a person using an electronic device or a device (eg, an artificial intelligence electronic device) using an electronic device.

Hereinafter, an embodiment of the present disclosure will be described in more detail with reference to the attached drawings.

1 is a diagram for explaining an electronic device according to an embodiment of the present disclosure.

Electronic devices according to various embodiments of the present disclosure include, for example, smartphones, tablet PCs, mobile phones, video phones, e-book readers, desktop PCs, laptop PCs, netbook computers, workstations, servers, PDAs, and PMPs. (portable multimedia player), may include at least one of an MP3 player, a camera, or a wearable device.

As shown in FIG. 1, the electronic device 100 can display a 3D object about the environment within the factory. At this time, the 3D object including the environment within the factory may be an object that implements the environment within the actual factory in three dimensions. Specifically, the electronic device 100 may display 3D objects for the environment within the factory by rendering 3D objects corresponding to devices (or equipment), materials, walls, or floors within the actual factory.

In this disclosure, the device within the factory may refer to all devices located within the factory as well as devices used to produce goods in the factory. For example, in-factory devices may include in-factory production devices, quality detection devices, packaging devices, as well as movement devices, ventilation devices, drainage devices, soundproofing devices, communication devices, or sensors.

The electronic device 100 can obtain a 3D object for the environment within the factory and display it. For example, the electronic device 100 may acquire 3D objects corresponding to each device, material, wall, or floor within the factory and render them to display the 3D object for the environment within the factory.

The electronic device 100 can change the name, color, location, size, etc. of a 3D object based on user input. For example, when a user inputs coordinate values where the 3D object will be located in order to change the location of the 3D object, the electronic device 100 may display the 3D object at the input coordinate value location.

The electronic device 100 may receive operation information of devices within the factory. Specifically, the electronic device 100 may receive operation information of devices within the factory from devices within the factory or a control device (eg, PLC, Programmable Logic Controller) that controls the devices within the factory. To this end, the electronic device 100 may communicate with a device within the factory or a control device that controls the device within the factory.

Additionally, the electronic device 100 may monitor the operation of the device based on the received operation information of the device. Specifically, the electronic device 100 may periodically receive device operation information, detect that an abnormal operation has occurred when the device operation information is outside a preset range, and provide an alarm regarding this to the user.

Accordingly, the electronic device 100 may provide the user with information about the environment within the factory, allowing the user to monitor the environment information within the factory.

Figure 2 is a block diagram for explaining the configuration of an electronic device according to an embodiment of the present disclosure.

As disclosed in FIG. 2 , the electronic device 100 includes a communication interface 110, a display 120, a memory 130, and a processor 140.

The communication interface 110 is a component that allows the electronic device 100 to communicate with an external device (not shown). The electronic device 100 may communicate with a device within the factory or a control device that controls the device within the factory through the communication interface 110.

According to another embodiment, the electronic device 100 may communicate with an external server through the communication interface 110. At this time, the external server is a device within the factory or a device connected to the device within the factory, and the electronic device 100 receives operation information of the device within the factory or a device connected to the device within the factory through the external server, or sends control commands to the device within the factory. Can be transmitted.

The communication interface 110 may include various communication modules, such as a wired communication module (not shown), a short-range wireless communication module (not shown), and a wireless communication module (not shown).

Here, the wired communication module is a module for communicating with an external device (not shown) according to a wired communication method, such as wired Ethernet. Additionally, the short-range wireless communication module is a module for communicating with an external device (not shown) located nearby according to a short-range wireless communication method such as Bluetooth (BT), BLE (Bluetooth Low Energy), or ZigBee. Additionally, a wireless communication module is a module that is connected to an external network and communicates with an external device (not shown) according to a wireless communication protocol such as WiFi, IEEE, etc. In addition, wireless communication modules comply with various mobile communication standards such as 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), and 5G Networks. It may further include a mobile communication module that connects to a mobile communication network and performs communication.

The display 120 is a component for displaying 3D objects including the environment within the factory. In the present disclosure, the display 120 may render 3D objects corresponding to objects included in the environment within the factory, such as devices, materials, walls, or floors within the factory. In addition, the display 120 may display a UI for changing the name, color, location, size, etc. of the 3D object. Alternatively, the display 120 may display a UI for changing the viewpoint of a 3D object including the environment within the factory.

The display 120 may be implemented as various types of displays, such as Liquid Crystal Display (LCD), Organic Light Emitting Diodes (OLED) display, Plasma Display Panel (PDP), Wall, Micro LED, etc. The display 120 may also include a driving circuit and a backlight unit that may be implemented in the form of a-si TFT, low temperature poly silicon (LTPS) TFT, or organic TFT (OTFT). Meanwhile, the display 120 may be implemented as a touch screen combined with a touch sensor, a flexible display, a 3D display, etc.

The memory 130 is a component for storing various programs and data necessary for the operation of the electronic device 100. At least one instruction for the processor 140 to operate the electronic device 100 may be stored in the memory 130 .

Additionally, 3D objects corresponding to devices located within the factory may be stored in the memory 130. At this time, the 3D object stored in the memory 130 may be an object created in the electronic device 100 or an object received from an external device.

The memory 130 may be implemented as non-volatile memory (eg, hard disk, solid state drive (SSD), flash memory), volatile memory, etc. Meanwhile, the memory 130 may be implemented as a memory that is physically separate from the processor 140. In this case, the memory 130 may be implemented as a memory embedded in the electronic device 100 or as a memory detachable from the electronic device 100 depending on the data storage purpose.

For example, the memory 130 may include volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM), etc.), non-volatile memory (e.g., OTPROM ( one time programmable ROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (such as NAND flash or NOR flash, etc.), hard drive , or a solid state drive (SSD), memory card (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD) , xD (extreme digital), MMC (multi-media card), etc.), external memory that can be connected to a USB port (for example, USB memory), etc.

Additionally, the memory 130 may be implemented as internal memory such as ROM (eg, electrically erasable programmable read-only memory (EEPROM)) or RAM included in the processor 140.

The processor 140 is electrically connected to the memory 130 and can control the overall operation and functions of the electronic device 100. For example, the processor 140 can control hardware or software components connected to the processor 140 by running an operating system or application program, and can perform various data processing and calculations. Additionally, the processor 140 may load and process commands or data received from at least one of the other components into volatile memory and store various data in non-volatile memory.

To this end, the processor 140 is a dedicated processor (e.g., embedded processor) or a general-purpose processor (e.g., CPU (Central)) capable of performing the corresponding operations by executing one or more software programs stored in a memory device. It can be implemented as a processing unit or application processor.

In the present disclosure, the processor may consist of one or multiple processors. At this time, one or more processors may be a general-purpose processor such as a CPU, AP, or DSP (Digital Signal Processor), a graphics-specific processor such as a GPU or VPU (Vision Processing Unit), or an artificial intelligence-specific processor such as an NPU. One or more processors control input data to be processed according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, when one or more processors are dedicated artificial intelligence processors, the artificial intelligence dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

The processor 140 may acquire a 3D object for the environment within the factory. Specifically, the processor 140 may acquire a 3D object including components within the factory, such as devices (or equipment), materials, walls, or floors within the factory. At this time, the 3D object may be a 3D object with the same shape as a device, material, wall, or floor in the factory, but is not necessarily limited to this. For example, a 3D object may be a three-dimensional shape of a general shape such as a cuboid, a triangular pyramid, or a sphere.

At this time, the 3D object may be an object created by the electronic device 100 or an object received from an external device (not shown) such as a server.

The processor 140 may control the display 120 to render a 3D object according to a user's input. Specifically, when processor 140 receives a user input selecting one of a list of components within a factory, such as devices, materials, walls, or floors, the processor 140 displays a display to render a 3D object corresponding to the selected component. (120) can be controlled.

When the processor 140 receives a user input for selecting one device from a list of devices located in the factory, the processor 140 may receive operation information of the selected device and control the display 120 to render a 3D object of the selected device.

Specifically, when the processor 140 receives a user input for selecting one device from a list of devices located in the factory, the processor 140 selects the selected device from the selected device, a control device that controls the selected device, or an external device that receives operation information of the selected device. You can receive device operation information.

At this time, the operation information of the device may include information related to the operation of the device, such as whether the device is operating, type of operation, direction of operation, speed of operation, or whether an operation abnormality occurs.

Processor 140 may control display 120 to render a 3D object for the device based on the received motion information. For example, if the received operation information of the device is operation information of a mobile device moving between the first and second floors in a factory and includes information that the mobile device is located on the first floor, the processor 140 may provide information about the mobile device located on the first floor. The display 120 can be controlled to render a 3D object including a corresponding object.

If it is determined that the operation of the device is outside a preset range based on the received operation information of the device, the processor 140 may control the display 120 to display a notification message. A detailed description of this will be provided later in FIG. 6.

Meanwhile, the processor 140 may receive attribute information about a 3D object from the user. To this end, the processor 140 may control the display 120 to display a UI (User Interface) for receiving attribute information about a 3D object from a user.

Here, the attribute information may include attributes such as the type of object included in the 3D object, the location to be rendered, the object name, the object's color, size, rotation, type of motion, and motion speed.

When at least one of the color, position, angle, size, and rotation information of the 3D object changes according to the user's input, the processor 140 controls the display 120 to display the 3D object based on the changed attribute information. can do.

When the motion information for the device corresponding to the 3D object changes according to the user's input, the processor 140 transmits the changed motion information to the device or an external device, and displays the display 120 to render the 3D object based on the changed motion information. ) can be controlled. Here, the external device may include a control device or server for controlling the device corresponding to the 3D object.

For example, when a user command is input to perform operation B on a device that is performing operation A, the processor 140 transmits information about operation B to the device or an external device and creates a 3D image based on the operation information B. Objects can be rendered. At this time, the processor 140 may control the display 120 to display a 3D object corresponding to the device performing the B operation or to display B operation information.

Meanwhile, the processor 140 can obtain device information on devices connected to devices in the factory. Specifically, the processor 140 may obtain device information of a device electrically connected to the device through user input. Alternatively, the processor 140 may receive information about a device electrically connected to a device within the factory from a device within the factory or a control device that controls the device.

When a 3D object is selected by a user input, the processor 140 may control the display 120 to display device information connected to a device corresponding to the selected object. Here, the connected device information may include information such as the name, type, and operating status of the connected device. For example, when device A and a thermometer are connected, when a 3D object corresponding to device A is selected by user input, the processor 140 can control the display 120 to display the temperature detected by the thermometer.

Meanwhile, the processor 140 may change the viewpoint of the 3D object displayed on the display 120.

Specifically, the processor 140 can set a viewpoint in a virtual space displayed on the display 120 and change the viewpoint according to user input.

If at least one viewpoint information among the position, direction, and viewing angle of the viewpoint changes according to the user's input, the processor 140 may control the display 120 to render a 3D object based on the changed viewpoint information. A detailed description of this will be provided later in FIGS. 4A and 4B.

3A to 3D are diagrams for explaining a UI for inputting attribute information of a 3D object created according to an embodiment of the present disclosure.

As described above, the processor 140 may control the display 120 to display a UI for receiving attribute information of a 3D object from the user.

As shown in FIG. 3A, the processor 140 inputs attribute information such as type, name, color (31), size (32), rotation (33), and position (34) of the 3D object (e.g., device). The display 120 can be controlled to display a UI for receiving.

Here, rotation may mean a value indicating the degree to which a 3D object is rotated and rendered. For example, the processor 140 controls the display 120 to render the front side of the object when the rotation value is 0°, and controls the display 120 to render the right side (or left side) of the object when the rotation value is 90°. ), and if the rotation value is 180°, the display 120 is controlled to render the back of the object, and if the rotation value is 270°, the display 120 is controlled to render the left side (or right side) of the object. can do.

Additionally, the location may mean the coordinate value at which the 3D object will be displayed. The processor 140 may set a reference point on the background object on which the 3D object will be rendered, and determine a position where the 3D object will be displayed according to coordinate values based on the reference point. Additionally, the processor 140 may control the display 120 to render a 3D object at a determined location.

The processor 140 may control the display 120 to render a 3D object based on the input attribute information. For example, as shown in FIG. 3A, when color (31), size (32), rotation (33), and position (34) information of a 3D object corresponding to the device loader is input, the processor 140 determines the size. The display 120 can be controlled to render the front of the device loader (corresponding to a rotation degree of 0°) with a size of 100 and a color of 31 at the coordinates (-1, -11).

Meanwhile, the 'create event' icon 35 shown in FIG. 3A is a UI for inputting operation information of a 3D object.

When the user selects the 'Create event' icon 35 in FIG. 3A, the processor 140 displays the UI 37 for inputting the operation type or operation speed of the 3D object as shown in FIG. 3B. (120) can be controlled. Here, the operation type and operation speed may mean the operation type and operation speed of the device in the factory corresponding to the 3D object.

When the operation information of the device changes according to the user input, the processor 140 may transmit the changed operation information to the device or an external device and control the display 120 to render a 3D object based on the changed operation information.

At this time, the electronic device 100 or a device (not shown) that has received changed operation information from an external device may change its operation based on the received operation information.

The processor 140 may establish a connection with the device or an external device to transmit operation information to the device or an external device.

The 'create connect' icon 36 shown in FIG. 3A is a UI for setting up a connection with a device or an external device.

When the user selects the 'create connect' icon 36 in FIG. 3A, the processor 140 can control the display 120 to display the UI of FIG. 3C or 3D.

Specifically, the processor 140 may control the display 120 to display different UIs depending on the communication mode.

For example, when the connection mode 39 is 'MES mode' as shown in FIG. 3C, the processor 140 may transmit operation information through an external device such as a server. In this case, given that the electronic device 100 is already connected to an external device, the processor 140 can control the display 120 to display a UI for inputting only device identification information (e.g., equipment code).

As shown in FIG. 3D, when the connection mode 39 is 'OPCUA', the processor 140 can transmit operation information using the OPCUA protocol. In this case, the processor 140 may control the display 120 to display a UI for inputting the URL, port, tag, etc. of the device to transmit operation information.

3C and 3D illustrate the case where the communication mode is 'MES mode' or 'OPCUA', but the communication mode is not necessarily limited to this. For example, various communication modes such as HTTP (Hyper Text Transfer Protocol) and TCP/IP (Transfer Control Protocol/Internet Protocol) can be used, and the input content of the UI may vary depending on the communication mode.

FIGS. 4A and 4B are diagrams for explaining a 3D object rendered according to a movement of a viewpoint displayed on a display according to an embodiment of the present disclosure.

The processor 140 may set a viewpoint in a virtual space displayed on the display 120. In the present disclosure, the viewpoint refers to the point where the center of vision touches a component in the factory corresponding to the 3D object. For example, the viewpoint of the 3D object shown in FIG. 4A is located to the upper right of the 3D objects 43, 44, and 45.

The processor 140 may change the viewpoint of the rendered 3D object and control the display 120 to display the 3D object according to the changed viewpoint.

The processor 140 may display UIs 41 and 42 for changing the viewpoint of a 3D object. At this time, the processor 140 may display the UI 41 including a shape similar to a 3D object at an arbitrary point on the display 120. At this time, the processor 140 may control the display 120 to display the X-axis, Y-axis, and Z-axis together with the UI 41 to indicate the viewing direction. Alternatively, the processor 140 may control the display 120 to display the cube-shaped UI 42 and display the viewpoint direction 42-1 on each side of the UI 42.

If at least one viewpoint information among the position, direction, and viewing angle of the viewpoint changes according to the user's input, the processor 140 may control the display 120 to render a 3D object based on the changed viewpoint information.

Specifically, as shown in FIG. 4A, with 3D objects 43, 44, and 45 corresponding to devices in the factory displayed, the position of the viewpoint is displayed in the viewpoint direction 42 through the UI 41 or UI 42. When a user's input to change to -1) is received, the processor 140 may control the display 120 to render a 3D object corresponding to the device viewed from the viewpoint direction 42-1 based on the user input. . In this case, as shown in FIG. 4B, the 3D object 45 located behind the 3D object 44 may not be rendered.

FIG. 5 is a diagram for explaining an electronic device that displays device information corresponding to a 3D object according to an embodiment of the present disclosure.

As shown in FIG. 5, the processor 140 can display a 3D object for the environment within the factory. Specifically, the processor 140 may display 3D objects for the environment within the factory by rendering 3D objects corresponding to components within the factory.

The processor 140 may map a 3D object along with setting information input through the UI in FIGS. 3A to 3D. Then, the processor 140 receives operation information of the device corresponding to the 3D object by the connection method set through the communication connection UI of FIG. 3C or 3D, or transmits operation information according to user input to the device corresponding to the 3D object. Can be transmitted.

When a 3D object is selected, the processor 140 may control the display 120 to display a highlight on the selected 3D object in order to distinguish the selected 3D object from other 3D objects.

When a 3D object is selected, the processor 140 may control the display 120 to display information 52 about the selected 3D object. In particular, when the 3D object selected by the user corresponds to a device in the factory, the processor 140 may control the display 120 to receive information on the device in the factory corresponding to the selected 3D object and display it.

Specifically, the processor 140 may receive operation information from a device corresponding to the 3D object based on the setting information of the 3D object and control the display 120 to display the received information.

The processor 140 may receive not only information about the device corresponding to the 3D object, but also information about the device electrically connected to the device corresponding to the 3D object. Additionally, the processor 140 may control the display 120 to display operation information about the device and information about devices connected to the device. At this time, device information about the connected device may include operation information such as whether the connected device is operating, type of operation, operation speed, and abnormality. However, it is not limited to this, and device information may vary depending on the type of connected device. For example, if the connected device is a sensor such as a thermometer, hygrometer, or noise detection device, device information may include presence or absence of detection, detection results, etc.

For example, as shown in FIG. 5, when the 3D object 51 corresponding to device A in the factory is selected, the processor 140 displays 'resin current weighing result' and 'operation status' as device information for device A. )', you can also receive the temperature detection results of the thermometer connected to device A.

FIG. 6 is a diagram for explaining an electronic device that monitors a device corresponding to a 3D object according to an embodiment of the present disclosure.

The processor 140 may receive device operation information from a device corresponding to a 3D object.

Specifically, when a 3D object corresponding to a device is rendered on the display 120, the processor 140 may request operation information from the device corresponding to the 3D object according to the communication connection included in the setting information of the 3D object, and the device You can periodically receive device operation information from.

The processor 140 may determine whether the operation of the device is outside a preset operation range based on the received operation information of the device. Here, the preset operating range refers to the normal operating range of the device. To this end, the processor 140 may secure data containing information on the normal operating range of each device in advance.

If it is determined that the operation of the device is outside a preset range based on the received operation information of the device, the processor 140 may control the display 120 to display a notification message 61. However, it is not necessarily limited to this, and for example, the processor 140 may output a notification message through a speaker (not shown).

The processor 140 may control the display 120 to display a notification message 61 including the name of the device and the operating status of the device.

Accordingly, the user can easily identify which device in the factory has a problem.

FIG. 7 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

First, 3D objects for the environment within the factory are acquired (S710). At this time, the 3D object may be an object created by the electronic device 100 or an object received from an external device (not shown) such as a server.

And, the 3D object obtained based on the user's input can be rendered. Specifically, upon receiving a user input selecting one of a list of components within the factory, such as devices, materials, walls, or floors, a 3D object corresponding to the selected component may be rendered.

When receiving a user input for selecting one device from the list of devices located in the factory, operation information of the selected device can be received and a 3D object of the selected device can be rendered (S720).

At this time, operation information of the selected device may be received from the device, a control device that controls the selected device, or an external device that receives operation information of the selected device.

If the operation of the device is determined to be outside the preset range based on the received operation information of the device, a notification message may be displayed (S730).

Meanwhile, attribute information about a 3D object can be input from the user. To this end, a UI (User Interface) can be displayed to receive attribute information about the 3D object from the user.

If at least one attribute information among the color, position, angle, size, and rotation information of the 3D object changes according to the user's input, the 3D object may be displayed based on the changed attribute information.

If motion information about a device corresponding to a 3D object changes according to a user's input, the changed motion information can be transmitted to the device or an external device, and the 3D object can be rendered based on the changed motion information.

You can obtain device information of devices connected to devices in the factory. Specifically, device information of a device electrically connected to the device can be obtained through user input. Alternatively, information about a device electrically connected to a device within the factory may be received from a device within the factory or a control device that controls the device.

When a 3D object is selected by user input, device information connected to the device corresponding to the selected 3D object can be displayed.

Meanwhile, the viewpoint of the 3D object can be changed. Specifically, the viewpoint of a 3D object can be changed based on user input.

If at least one viewpoint information among the position, direction, and viewing angle of the viewpoint changes according to the user's input, a 3D object may be rendered based on the changed viewpoint information.

Various operations described above as being performed through the electronic device 100 may be performed through one or more electronic devices in the form of a control method or operation method of a system including an electronic device.

Meanwhile, the various embodiments described above may be implemented in a recording medium that can be read by a computer or similar device using software, hardware, or a combination thereof.

According to hardware implementation, the embodiments described in this disclosure include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs). ), processor, controller, micro-controllers, microprocessor, and other electrical units for performing functions.

Meanwhile, computer instructions for performing processing operations on a user device or administrator device according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. You can. The computer instructions stored in such a non-transitory computer-readable medium, when executed by a processor of a specific device, cause the specific device to perform processing operations of the user device and/or the administrator device according to the various embodiments described above.

A non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as registers, caches, and memories. Specifically, the various applications or programs described above may be stored and provided on non-transitory readable media such as CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, etc.

In addition, although preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the invention pertains without departing from the gist of the present invention as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical idea or perspective of the present invention.

100: electronic device 110: communication interface
120: display 130: memory
140: processor

Claims (11)

In electronic devices,
a memory storing at least one instruction;
communication interface;
display; and
Including a processor executing the at least one instruction,
The processor,
Obtain 3D objects corresponding to devices located in the factory,
Upon receiving a user input for selecting the device from a list of devices located in the factory, receiving operation information of the device and controlling the display to render the 3D object;
An electronic device that controls the display to display a notification message when it is determined that the operation of the device is outside a preset range based on the received operation information of the device.
According to paragraph 1,
The processor,
When at least one attribute information of color, position, angle, size, and rotation information of the 3D object changes according to a user input, the electronic device controls the display to render the 3D object based on the changed attribute information.
According to paragraph 1,
The processor is
When operation information for the device changes according to a user input, the electronic device transmits the changed operation information to the device or an external device and controls the display to render the 3D object based on the changed operation information.
According to paragraph 1,
The processor,
Obtain device information connected to the device,
An electronic device that controls the display to display the connected device information when the 3D object is selected by user input.
According to paragraph 1,
The processor,
Set the viewpoint in the virtual space displayed on the display,
When at least one viewpoint information among the position, direction, and viewing angle of the viewpoint changes according to a user's input, the electronic device controls the display to render the 3D object based on the changed viewpoint information.
In a method of displaying a 3D object,
Obtaining a 3D object corresponding to a device located in the factory;
Upon receiving a user input for selecting the device from a list of devices located in the factory, receiving operation information of the device and rendering the 3D object based on the received operation information; and
A 3D object display method including; displaying a notification message when it is determined that the operation of the device is outside a preset range based on the received operation information of the device.
According to clause 6,
When at least one attribute information of color, position, angle, size, and rotation information of the 3D object changes according to a user input, rendering the 3D object based on the changed attribute information; 3D object further comprising How to display.
According to clause 6,
When the operation information changes according to user input,
transmitting the changed operation information to the external device; and
rendering the 3D object based on the changed motion information; A method for displaying a 3D object, further comprising:
According to clause 6,
Obtaining device information connected to the device; and
A method for displaying a 3D object, further comprising: displaying the connected device information when the 3D object is selected by a user input.
According to clause 6,
setting a viewpoint in a virtual space displayed on the display; and
When at least one viewpoint information among the position, direction, and viewing angle of the viewpoint changes according to a user's input, rendering the 3D object based on the changed viewpoint information.
A computer-readable recording medium containing a program for executing a 3D object display method,
The method of displaying the 3D object is,
Obtaining a 3D object corresponding to a device located in the factory;
When a user input for entering operation information and communication setting information of the virtual device is received,
Upon receiving a user input for selecting the device from a list of devices located in the factory, receiving operation information of the device and rendering the 3D object based on the received operation information; and
A computer-readable recording medium comprising; displaying a notification message when it is determined that the operation of the device is outside a preset range based on the received operation information of the device.

Images