WO2021100886A1 - Smart wall and method for controlling same - Google Patents

Abstract

The present invention relates to a smart wall and a method for controlling same. The gist of the present invention is to: capture at least one first image including a user and an object in the foreground by using a first camera; capture at least one second image including the user and the object by using a second camera; recognize the user's movement on the basis of the at least one first image and the at least one second image; recognize the object on the basis of the at least one first image and the at least one second image; determine the user's intention on the basis of the recognized user's movement and the recognized object; and control a display to display manual information corresponding to the determined user's intention.

Description

Smart wall and its control method

The present invention relates to a smart wall and a control method thereof, and more specifically, a smart wall in which a display mounted on a smart wall provides information related to a part by automatically detecting the condition of a part mounted on the smart wall or whether there is an abnormality in the part. And a control method thereof.

Recently, as an infrastructure that connects all media devices and home appliances in the home to the Internet beyond smartphones has been established, smart home is emerging as a keyword of a new smart ecosystem. In particular, the experiences and values of users already accumulated through smartphones, which were personal media devices, began to expand to other devices.

In this way, as various attempts such as building a home network system in order for media devices and home appliances to work in conjunction with each other, the number of electronic devices in the home increases, and there is a hassle of connecting them one by one. Therefore, in order to reduce the number of electronic devices, an integrated work is underway to combine TVs, air purifiers, and speakers into one.

For example, a TV, air purifier, console game console, etc. can be installed in the mounting space of a smart wall.

However, in the case of the prior art, if a problem occurs in the air purifier after the replacement period of the air purifier filter, the user has to find the manual for replacing the air purifier filter one by one and cannot find the manual, because he does not know how to replace the air purifier filter. There is a problem that the user feels uncomfortable.

An embodiment of the present invention recognizes a user's motion and object by analyzing an image captured by a camera, determines a user's intention based on the recognized user's motion and the recognized object, and responds to the determined user's intention. Its purpose is to provide a smart wall that provides manual information and a control method thereof.

Another embodiment of the present invention is to provide a smart wall that displays information indicating a problem about the component when a problem occurs in a component mounted on the smart wall, and a control method thereof.

Another embodiment of the present invention is to provide a smart wall that automatically changes and displays a screen size based on the amount of manual information when displaying manual information corresponding to a user's intention, and a control method thereof. .

The technical problem to be achieved in the present invention is not limited to the above-mentioned technical problem, and other technical problems that are not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description. .

According to an aspect of the present invention, a smart wall includes a display for displaying a graphic image; A screen that covers the front surface of the display and is rolled on the display to adjust a screen size of the display; A driving roller positioned above the screen and winding the screen; A motor providing rotational force to the drive roller; A screen sensing unit that detects the size of the first area of the screen; A first camera for capturing at least one first image including a user and an object in front of the screen; A second camera capturing at least one second image including the user and the object; Controls driving of the motor based on the size of the first area detected by the screen sensing unit, recognizes a user's motion based on the at least one first image and the at least one second image, and the The user who recognizes the object based on at least one first image and the at least one second image, determines the user's intention based on the recognized motion of the user and the recognized object, and determines the display And a control unit that controls to display manual information corresponding to the intention of the user.

According to another aspect of the present invention, a method of controlling a smart wall includes: capturing at least one first image including a user and an object in front with a first camera; Capturing at least one second image including the user and the object with a second camera; Recognizing a user's motion based on the at least one first image and the at least one second image; Recognizing the object based on the at least one first image and the at least one second image; Determining a user's intention based on the recognized user's motion and the recognized object; And controlling the display to display manual information corresponding to the determined intention of the user.

According to an embodiment of the present invention, the user's motion and object are recognized by analyzing the image captured by the camera, the user's intention is determined based on the recognized user's motion and the recognized object, and the determined user's intention is By providing the corresponding manual information, the user's convenience can be improved because the user can easily recognize how to replace parts when replacing parts.

According to another embodiment of the present invention, when a problem occurs in a part mounted on a smart wall, information indicating a problem related to the part is displayed. When a problem occurs in the part, the user does not need to search the manual individually to find a solution to the problem. User convenience can be improved because it can be easily recognized.

According to another embodiment of the present invention, when the display displays manual information corresponding to the determined user's intention, more detailed information is controlled to automatically change the size of the displayed screen from line view to half view based on the amount of manual information. As information can be provided to users, user convenience can be improved.

The effects that can be obtained in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those of ordinary skill in the art from the following description. will be.

1 is a diagram showing an installation example of a smart wall of the present invention.

2 is an exploded perspective view of a smart wall according to an embodiment of the present invention.

3 is a block diagram of a smart wall according to an embodiment of the present invention.

4 is a diagram showing the configuration of a frame of a smart wall according to an embodiment of the present invention.

5 shows examples of various frame structures according to a combination of frame modules of the present invention.

6 is a diagram illustrating a frame module of a smart wall according to an embodiment of the present invention.

7 is a view for explaining a method of installing a frame structure of a smart wall according to an embodiment of the present invention.

8 is a view for explaining a basket of a smart wall according to an embodiment of the present invention.

9 is a diagram illustrating a change of a screen according to a size of a display unit of a smart wall according to an exemplary embodiment of the present invention.

10 is a diagram illustrating an example of a screen sensing unit of a smart wall.

FIG. 11 is a diagram illustrating an operation of a screen detection unit according to a change in size of a first area of the screen detection unit of FIG. 10.

12 is a diagram illustrating a modified example of the screen sensing unit of FIG. 10.

13 is a diagram illustrating another example of a screen sensing unit of a smart wall.

14 is a diagram illustrating a detailed structure of the screen sensing unit of FIG. 13.

15 is a diagram illustrating a configuration of a smart wall according to an embodiment of the present invention.

16 is a flowchart illustrating a method of controlling a smart wall according to an embodiment of the present invention.

17 is a diagram illustrating a hardware configuration of a smart wall according to an embodiment of the present invention.

18 is a diagram illustrating a hardware configuration of a smart wall according to an embodiment of the present invention.

19 is a diagram illustrating a line view, a half view, and a full view of a smart wall according to an embodiment of the present invention.

20 is a diagram illustrating a notification function of a smart wall according to an embodiment of the present invention.

21 is a diagram illustrating recognition of a user's motion and object recognition with a camera according to an embodiment of the present invention.

22 is a diagram illustrating displaying manual information of a smart wall according to an embodiment of the present invention.

23 is a diagram illustrating a notification function of a smart wall when a component installation is completed according to an embodiment of the present invention.

24 is a diagram illustrating determining a user's intention using a reinforcement learning model according to an embodiment of the present invention.

25 is a diagram illustrating recognition of a user's motion based on a current image and a previous image according to an embodiment of the present invention.

26 is a diagram illustrating a method of photographing a body point with a camera and recognizing a user's motion based on the body point, according to an embodiment of the present invention.

Hereinafter, exemplary embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but identical or similar elements are denoted by the same reference numerals regardless of reference numerals, and redundant descriptions thereof will be omitted. The suffixes "module" and "unit" for constituent elements used in the following description are given or used interchangeably in consideration of only the ease of writing the specification, and do not themselves have a distinct meaning or role from each other.

In addition, in describing the embodiments disclosed in the present specification, when it is determined that a detailed description of related known technologies may obscure the subject matter of the embodiments disclosed in the present specification, the detailed description thereof will be omitted.

In addition, the accompanying drawings are for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed in the present specification is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention It should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first and second may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another component.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. It should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this application, terms such as "comprises" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof does not preclude in advance.

1 is a view showing a wall-shaped frame of the smart wall 100 of the present invention. The present invention relates to a smart wall 100 in which electronic devices used at home, such as media devices and home appliances, are integrated on a wall so that they are not exposed to the exterior. Various electronic devices such as display unit, interphone, speaker, air purifier, humidifier, lighting, digital clock, router, and set-top box are mounted on the wall to provide a total space solution that minimizes the exposure of electronic devices in the space.

As the thickness of the display device 161 has recently become thinner, the thickness of the smart wall 100 of the present invention can be formed to be as thin as 15 to 20 cm, so that the indoor space can be prevented from being narrowed during installation. Rather, storage furniture and home appliances 160 may be embedded in the smart wall 100 to provide a neat appearance. If necessary, a part of the wall may be selectively drawn out so that the necessary home appliance may protrude, and the space where the home appliance is not buried may be utilized as a mounting space 114 such as a drawer.

A TV can be made thinner in recent years, and thus a thin display unit can be implemented. In reality, the TV itself serves as a cover for the frame and can be combined with the frame to form a front appearance. An opening 153 may be formed in the art wall 151 to expose the display portion of the display device 161, and a cover for opening and closing the opening 153 may be further provided.

Since the display unit using OLED is bent, it is exposed to the outside only when in use, and when not in use, it can be rolled up and stored inside the wall, thus providing a neat screen in appearance.

As for the projector, a short throw projector has been recently developed, and a large screen can be realized by mounting the projector on a wall, drawing it out in the form of a drawer, and outputting it to the screen. In this case, when not in use, the display unit is not visible on the outside of the smart wall 100 so that a cleaner appearance can be provided.

Figure 2 is an exploded view of the smart wall 100 of the present invention, Figure 3 is a block diagram of the smart wall 100 of the present invention, the interior of the front art wall 150 is a plurality of shape steel 115 (shape steel ) Can be placed in a frame structure arranged in a grid. The frame structure 110 is a rigid member and can be supported so as not to collapse even when an impact is applied to the wall. A panel may be attached to the front of the frame structure 110 to hide the electronic product 160 mounted on the frame structure 110 and be configured to appear as a wall. The front wall covered with such a panel is referred to as an art wall 150.

The art wall 150 can be changed differently according to a consumer's taste. After installation, you can easily change it to change the atmosphere of your home.

The section steel 115 arranged in a grid form may partition the mounting space 114 and mount the electronic product 160 in each mounting space 114. Each of the mounting spaces 114 may have the same size or multiple sizes, so that the mounted electronic product 160 may be modularized and disposed at any position in the frame structure 110.

In addition to the above-described display device 161, a sound device 162 such as a speaker may be mounted in the mounting space 114, and the sound device 162 may be configured in plural according to the number of channels. The art wall 152 covering the sound device 162 may be configured in a mesh shape so that sound can be output, or an opening exposed to the art wall 152 may be minimized by selectively opening and closing an opening through which sound is output. Water may be prevented from flowing through the opening by using a waterproof mesh or the like.

Seasonal household appliances such as a dehumidifier 163, a humidifier 165, an air purifier 164, and an air conditioner, a heater such as a heater 167 may be provided. Such seasonal home appliances are generally exposed to the outside when they are not in use, and dust accumulates and looks bad, so it is common to put a cover on it or put it in a warehouse. However, if it is embedded in the smart wall 100 as in the present invention, the art wall 150 can cover it so that it is not exposed to the outside even when it is left as it is even when it is not used. That is, a separate storage space is provided for seasonal home appliances. There is no need to do it.

The dehumidifier 163 or the humidifier 165 may implement a drawer-type space for collecting/storing water to remove collected water or fill water. An opening 154 through which air passes in order to discharge water or collect moisture around it is required.

Even if these openings 154 are not necessarily formed in the art wall 150, the gap between the art wall 150 and the frame structure 110 may be used.

The air purifier 164 may be provided with a cover that can be opened and closed so that the filter can be periodically exchanged.

Each electronic product may be individually controlled, but may further include a controller 180 for integrally controlling the electronic product mounted on the smart wall. The control unit may itself include the memory 185 to store the collected data.

A dehumidifier, a humidifier, and an air purifier 164 can be installed at the bottom of the smart wall 100 so that the user can easily operate it. In addition, when a set-top box or a WIFI router 175 is mounted, a home network system can be implemented. I can. Through the WIFI router 175, not only the functional units mounted on the smart wall 100, but also other electronic devices such as a computer or portable terminal located outside the smart wall 100 can be connected and interworked with each other, and the smart wall 100 It may include a controller 180 that controls each electronic product 160 mounted on the device.

An LED or a small display panel may be disposed on the upper part of the smart wall 100 to provide simple information such as weather and time event information to the user, or may be used as lighting. By arranging lighting on the smart wall 100 itself, it is possible to add an interior effect.

The router 175 is an intermediate device through which the router organizes and guides the route when different communication methods or signals are used. In addition, the functional unit mounted on the smart wall 100 may not necessarily be connected by wire, but may be connected and controlled through the WIFI router 175.

In addition, since the router 175 plays a role of connecting the external network and the internal network at the same time, the user can access the home network as well as the electronic product 160 mounted on the smart wall 100 through the router 175 connected to the external network from the outside. You can control other home appliances and media devices connected to it.

In addition, the controller 180 may control electronic devices of the smart wall 100 in connection with a mobile device. That is, when a mobile device is mounted on the smart wall 100, media images and music from the mobile device are output, or when a video call is received through the mobile device, information of the mobile device is transmitted through Bluetooth or Wi-Fi. An image of the other party may be output to the display device 161 of ).

In addition, a wall pad connected to a camera or method system at the entrance may be provided in the mounting space 114 of the smart wall 100. Wall pads that were buried in the existing bearing wall were difficult to repair and replace in case of failure, and difficult to change their location. However, if the wall pad is configured in a unit form and provided in the smart wall 100, there is an advantage in that maintenance is easy.

The smart wall 100 can be installed not only in the living room but also in the room. By configuring the walls between the rooms as the smart wall 100, the smart wall 100 may be used in both directions in neighboring rooms. For example, if one air conditioner or air purifier is passed in both directions, it can be used in both spaces. Instead of the TV and speaker 162 used in the living room, a monitor and a PC may be mounted and used according to the purpose of the room. Alternatively, other functional units such as an electronic blackboard may be mounted and used according to the age of the child.

When installed in an office, a smart wall 100 equipped with a functional unit such as an electronic blackboard, an air purifier, a video conference system, and a PC can be used. Since the electronic product 160 mounted on the smart wall 100 is covered by the art wall 150, a separate exterior case is not required.

If the smart wall 100 is manufactured according to the user's installation location at the factory and then installed, the same module can be manufactured and supplied on a large scale when constructing a hotel/resort or apartment, reducing the defect rate and producing it at low cost.

However, the assembled smart wall 100 is difficult to transport, and when the building is completed, it is difficult to carry it into the building. Accordingly, when the smart wall 100 is installed when moving or interior in an existing building, the smart wall 100 may be implemented by installing a frame in the building and inserting a unit module. In this case, for convenience and standardization of installation, the frame structure 110 may be implemented using a plurality of module frames.

4 illustrates a method of combining frame modules 111, 112, and 113 of the smart wall 100 according to an embodiment of the present invention. In order to mount each component, frame modules 111, 112, and 113 in the form of a bookshelf including a mounting space 114 may be used. Frame modules 111, 112, and 113 may be manufactured by arranging a plurality of beam-shaped section steels 115 in a grid to partition the mounting space 114.

The frame modules 111, 112, 113 are a first frame module 111 in which the mounting spaces 114 are arranged side by side in a horizontal direction, and a second frame module for mounting a large product 160 such as a display device 161 It may include a third frame module 113 in which 112 and the mounting space 114 are arranged side by side in the vertical direction.

The second frame module 112 includes a large space so that all of the display devices 161 can be mounted regardless of the inch. If necessary, as shown in FIG. 4, the display device 161 may be fixed by providing a beam that partitions the mounting space 114 and the rigidity of the frame structure 110 may be improved.

The first frame module 111 may be stacked and disposed in a vertical direction with the second frame module 112. The length of the first frame module 111 in the horizontal direction may be the same as the length of the second frame module 112 in the horizontal direction, so that the grid may not be displaced when stacked in the vertical direction.

The first frame module 111 may be disposed on the upper side and the lower side of the second frame module 112, respectively, and according to the size of the first frame module 111 and the second frame module 112 or the size of the installation space. Accordingly, the number of stacked first frame modules 111 in the vertical direction may be increased or decreased.

The third frame module 113 in which the mounting spaces 114 are arranged in the vertical direction may have the same height as the first frame module 111 and the second frame module 112 stacked in the vertical direction. The third frame module 113 may be located in the lateral direction of the first frame module 111 and the second frame module 112 as shown in FIG. 4. A plurality of frame modules 111, 112, 113 may be combined with fasteners to form a rectangular parallelepiped frame structure 110.

FIG. 5 is a diagram showing a method of manufacturing the frame modules 111, 112, and 113 of the present invention, and a third frame module 113 in which the mounting space 114 is disposed in a vertical direction will be described as an example. Each of the frame modules 111, 112, and 113 may use a section steel 115 that is elongated in one direction. Shape steel is a structural member and collectively refers to a rod-shaped member having various cross-sectional shapes. It is possible to construct a column-shaped section steel having the same cross-section in the longitudinal direction by rolling or bending the plate. Section steel has an advantage of being able to obtain rigidity of a certain size or more with lighter and less material since there is an empty space than a beam-shaped member having a circular or square cross section.

The present invention may use a C-beam 115 having a C-shaped cross section as shown in FIG. 5. The C-beam 115 may be formed by vertically bending both ends of a metal plate in the width direction. If both ends are vertically bent only once, they have a C-shaped cross-section, and if they are vertically bent twice, they can have a C-shaped cross-section. In the internal space, a power or signal line connected to the electronic product 160 may be disposed, and an outlet module 140 to which power or data cables are fastened may be disposed.

The outlet module 140 may include a power terminal for supplying power, a data terminal for transmitting and receiving data such as an Internet line or cable input, and a connector for connecting a functional unit inserted in the frame with another functional unit. The connector has terminals on both sides of the frame, and data can be exchanged with each other by inserting a data connection line of each electronic device into the terminal. The outlet module 140 may include a rail structure that moves along the frame modules 111, 112, and 113.

A vent hole 116 may be formed in the section steel 115 as shown in FIG. 5. The vent hole 116 may reduce the weight of the section steel 115 and discharge heat generated from the electronic product 160. In order to dissipate heat more actively, a fan can be used to discharge hot air to the outside. When the heat of the electronic product 160 is discharged indoors, a heating effect may be obtained in winter. However, in summer, a structure that discharges heat to the outside can be added by circulating heat inside by using a fan, or discharging heat to the outside by adding to the wiring connected to the outdoor unit of the air conditioner.

Each frame module 111, 112, 113 may include an outer cavity forming a square around the outer circumference and an inner cavity for partitioning the mounting space 114 therebetween. The outer steel may include a vertical steel extending in a vertical direction and a horizontal steel extending in a horizontal direction. The first frame module 111 to the third frame module 113 may be assembled by arranging a horizontal steel between a pair of vertical steels. In order to fasten the vertical steel with the fixing member 120 fixed to the ceiling and the floor, it is necessary to arrange the upper and lower ends of the vertical steel to be exposed. Therefore, by placing the vertical steel outside the horizontal steel, it is possible to insert and fasten the vertical brackets 122 and 123 of the fixing member 120 to the end of the vertical steel.

The inner steel located inside the outer steel divides the space surrounded by the outer steel to form the mounting space 114, and the inner steel may be assembled so that it cannot be moved by welding to the outer steel, and some inner steels can be assembled using bolts or hooks. They can be combined so that they can be removed as needed.

In this way, by using a method of fastening between the frame modules 111, 112, and 113 in the form of the semi-finished product 160, the number of assembling in the field can be minimized. The dimensions of the product 160 can be kept constant and installation time can be reduced. In addition, because it can be modularized into a size that is easy to move, it can be easily transported to the site.

6 shows examples of various frame structures 110 according to a combination of frame modules 111, 112, and 113 of the present invention. 6A is a frame structure 110 in which a pair of first frame modules 111a, one second frame module 112a, and a pair of third frame modules 113 of FIG. 4 are assembled. ). Since the first frame module 111a has four mounting spaces 114, it may be difficult to install in a narrow indoor space due to its wider width.

Therefore, as shown in (b) of FIG. 6, a first frame module 111b having a short horizontal length including three mounting spaces 114 may be used. The second frame module 112b may have a length corresponding to the horizontal length of the first frame module 111b.

In addition, the third frame module 113 may be disposed on both sides of the first frame module 111 and the second frame module 112 vertically stacked. Since the floor height of a typical house is 220cm ~ 230cm, the height of the third frame module 113 may be configured to be a little less than 220cm. Although a five-stage vertical frame is shown in the drawing, a six-stage vertical frame may also be manufactured in preparation for a case of a double layer or a high floor height.

When installing in a narrower space, the third frame module 113 may be omitted and the frame structure 110 may be constructed using only the first frame module 111a and the second frame module 112a, or a large display device ( If 161 is not mounted, the frame structure 110 may be implemented by arranging only the third frame in a line. Each frame module 111, 112, 113 may be fastened with bolts or may be fastened by a welding method.

7 is a view for explaining a method of installing the frame structure 110 of the smart wall 100 according to an embodiment of the present invention. The frame structure 110 configured by fastening the frame modules 111, 112, and 113 may be fixed to the ceiling and the floor for installation in an indoor space.

The fixing bracket 125 installed on the ceiling or floor is a member that is bent in an L-shape, and one side 125b extends in a vertical direction from the ceiling or floor and the other side 125a from the ceiling or floor. It may include a first vertical bracket 122 inserted into the inner space of the vertical cavity of the first frame module 111 and the second frame module 112.

The first vertical bracket 122 may be formed to be longer than the vertical length of the first frame module 111 so as to fasten not only the first frame module 111 but also the second frame module 112. A second vertical bracket 123 disposed parallel to the first vertical bracket 122 and inserted into the third frame module 113 may be further included. Since the vertical steel of the third frame module 113 is one member, it does not need to be as long as the first vertical bracket 122.

The coupling bracket 121 coupled to the ends of the first vertical bracket 122 and the second vertical bracket 123 may form a T-shape together with the vertical brackets 122 and 123. The coupling bracket 121 is coupled to the vertical brackets 122 and 123, and is vertical from the seating portion 121b and the seating portion 121b that are seated on the upper surfaces of the first frame module 111 and the third frame module 113. It may include a fastening portion (121a) extending in the direction and fastened to the fixing bracket (125). The coupling bracket 121 and the fixing bracket 125 can be fastened using a screw, and the fastener to which the screw is fastened may be formed to be long in the vertical direction so that it can be adjusted according to the ceiling height.

When the smart wall 100 is installed over an existing wall, it may further include a fixing member 120 fastened to the existing wall.

Since the smart wall 100 has an audio device, such as the speaker 162, when it is used as a temporary wall, a space separated from the wall is generated. At this time, in order to block the sound of the two spaces, the smart wall 100 may be fixed by adding a soundproof structure so that sound does not leak to the ceiling, floor, and sidewalls that contact the smart wall 100.

8 is a view for explaining the basket 130 of the smart wall 100 according to an embodiment of the present invention. The frame modules 111, 112, and 113 may be divided into a plurality of mounting spaces 114 by using the inner section steel 115. As shown in FIG. 7, a space in which the electronic product 160 is mounted may be provided by inserting the basket 130 into the mounting space 114. The second frame module 112 has a relatively large mounting space 114 because the display device 161 is located, but the mounting space 114 of the first frame module 111 and the third frame module 113 is It can be formed in a standardized size. If the mounting space 114 of a standardized size is used, the basket 130 can also be standardized, and thus there is an advantage in that there is no need to manufacture parts having various shapes unnecessarily.

If the size of the mounting spaces are all the same, it is convenient for standardization, but a larger space may be required depending on the electronic product 160 to be mounted. For example, since the sizes of the air conditioner and the air purifier are different, the size of the mounting space 114 may be formed to be an integer multiple of the basic size in consideration of this difference. For example, a second mounting space 1142 having a horizontal size of an integer multiple of a based on the first mounting space 1141 having a horizontal size of a and a vertical size of b, or a third having a vertical size of an integer multiple of b It may include a mounting space (1143).

The size of the first mounting space 1141 in the horizontal direction corresponds to the size of the third frame module 113 in the horizontal direction, and the size in the vertical direction corresponds to the size of the first frame module 111 in the vertical direction. Here, the mounting space 114 means an internal space divided by the section steel 115, so the comparison between the size of the first frame module 111 and the size of the third frame module 113 is the difference due to the thickness of the section steel 115 Is ignored.

Since the first frame module 111 has a short length in the vertical direction, it may include only the first mounting space 1141 and the second mounting space 1142, and the third frame module 113 has a short length in the horizontal direction. Therefore, only the first mounting space 1141 and the third mounting space 1143 may be included.

9 is a diagram illustrating a change of a screen according to a size of the display device 161 of the smart wall 100 according to an exemplary embodiment of the present invention. As shown in (a) of FIG. 9, only half of the display device 161 is exposed to provide necessary information to the user, or as shown in (b) of FIG. 9, it is possible to obtain an interior effect by exposing it in a small size.

The smart wall 100 of the present invention may include a screen 200 covering the front surface of the display device 161. The screen 200 may use a material that can be wound on a roll, such as a fabric material, and covers the front surface of the display device 161 so that the display device 161 is not exposed to the outside when the display device 161 is not used. It acts as a mask.

The screen 200 may be wound around the driving roller 210 in order to adjust the area of the first area of the screen 200 positioned in front of the display device 161. When the number of times the screen 200 is wound around the driving roller 210 is large, the area of the first area is reduced and the display device 161 is exposed on the front side.

A part of the screen 200 (a first area) is located on the front of the display device 161 by winding the screen 200 half a turn around the driving roller 210, and a part of the screen 200 (a second area) is a display device. It can be configured to be located on the back of (161). When the area of the second area located on the rear surface of the display device 161 is increased, the area of the first area located on the front surface of the display device 161 is reduced. The size of the area exposed on the front surface of the display device 161 may be adjusted.

The controller 180 adjusts the area of the screen 200 located in front of the display device 161 by adjusting the rotation direction of the driving roller 210, and when the screen 200 reaches a preset position, the driving roller ( 210) can be stopped.

The control unit may control an image output to the display device 161 according to the position of the screen 200. For example, in a line view mode as shown in (a) of FIG. 9 or a half view mode as in (b) of FIG. 9, an image must be output only to a part of the display device 161. When an image is output to a portion where the screen 200 is located, there is a problem of unnecessary power consumption and providing a cropped image to the user.

When the driving roller is rotated, the screen 200 may be pushed from the surface of the driving roller, and in the case of an embodiment in which the driving roller is wound, the screen 200 that moves per rotation of the driving roller increases as the length of the screen 200 wound around the driving roller increases ( Since the length of 200) is different, it is difficult to accurately match the number of rotations of the motor that rotates the driving roller 210 and the moving distance of the screen 200.

In addition, in order to increase the synchronization rate with the location of the screen 200 and the image output to the display device 9161, a screen sensing unit that detects the size of the first area of the screen 200 is required. Since the screen 200 of the present invention moves only in the vertical direction, the size of the screen 200 can be known when the moving distance in the vertical direction or the position of the lower end of the first area of the screen 200 is sensed. It can be said to be a device that detects the length of the first area in the vertical direction.

10 is a diagram showing an example of a screen sensing unit of the smart wall 100. The screen sensing unit of this embodiment is a conductive pattern 241 formed on the surface of the screen 200, a pair of terminals 242, and a pair of And a determination unit that measures the resistance between the terminals 242 and determines that the size of the first region is larger as the resistance increases and the size of the first region is smaller as the resistance decreases.

When the pair of conductive patterns 241 is formed on the second surface 200b, which is the opposite surface of the first surface 200a exposed to the outside, the pair of conductive patterns 241 is not exposed to the user. The conductive pattern 241 formed on the second surface 200b includes a first vertical pattern 241a extending in a vertical direction, a second vertical pattern spaced apart from the first vertical pattern 241a and extending in a vertical direction, and the second vertical pattern 241a. A connection pattern 241c connecting the lower end of the first vertical pattern 241a and the lower end of the second vertical pattern 241b may be included.

The position of the first vertical pattern 241a and the second vertical pattern 241b is not limited on the second surface 200b of the screen 200, and the connection state with the terminal 242 in both the full view state and the line view state is maintained. It is sufficient if it has a length that can be maintained, and the connection pattern 241c has a straight line extending in the horizontal direction as shown in the drawing, but it does not have to be a straight line, and the first vertical pattern 241a and the second vertical pattern 241b A conductive member connecting the lower end of the suffix is sufficient.

The pair of terminals 242 contact the first vertical pattern 241a and the second vertical pattern 241b, respectively, and are fixed to the frame. Preventing the terminal 242 from affecting the vertical movement of the screen 200, and maintaining the contact state between the conductive pattern 241 and the terminal 242 even if the screen 200 is shaken in the front and rear direction For this reason, a member having elasticity may be used. A contact state between the conductive pattern 241 and the terminal 242 may be maintained by using a material having elasticity such as a sponge, or by using a member having elasticity such as a clip or pogo pin.

The pair of terminals 242 are connected to the conductive pattern 241, and when power is applied to the terminal 242, current flows through the conductive pattern 241. As shown in (c) of FIG. 10, the resistance between the terminals 242 may vary according to the length of the conductive pattern 241. The screen sensing unit may detect the size of the resistance to calculate an increase or decrease in the length of the conductive pattern 241 and calculate the size of the first area of the screen 200.

FIG. 11 is a view for explaining the operation of the screen detection unit according to a change in the size of the first area of the screen detection unit of FIG. 10, and FIG. 11(a) is a lower end of the display device 161 as shown in FIG. 9(b). A diagram showing the second surface 200b of the screen 200 in a line view mode in which only a portion of the screen is exposed. FIG. 11B is a diagram illustrating a second surface 200b of the screen 200 in a half view mode in which only half of the display unit is exposed as shown in FIG. 9A. FIG. 11C is a diagram illustrating a second surface 200b of the screen 200 in a full view mode exposing the entire area of the display device 161 as shown in FIG. 1.

When the screen 200 moves in the vertical direction, positions of the pair of terminals 242 in contact with the first vertical pattern 241a and the second vertical pattern 241b may be changed. The length of the conductive pattern 241 positioned between the pair of terminals 242 in FIG. 11(a) is the longest, and the conductive pattern 241 positioned between the pair of terminals 242 in FIG. 11(c) ) Is the shortest. Accordingly, in each mode, the length of the conductive pattern 241 positioned between the pair of terminals 242 is different, so the size of the resistance is also different. Since the resistance increases as the length of the conductive pattern 241 increases, R1 may be the largest and then R2 and R3 may decrease in size. The controller may store the resistance value at the designated stop position in the memory, and stop driving the motor when the resistance value stored in the memory and the resistance between the pair of terminals 242 are the same.

12 is a diagram illustrating a modified example of the screen sensing unit of FIG. 10. Since the screen 200 is made of a thin material such as fabric, it may be shaken or wrinkled when the screen 200 moves up and down. A load bar 209 may be disposed at the lower end of the first region to prevent such wrinkles and to maintain a flat and unfolded state. The load bar 209 provides a force to pull the first area of the screen 200 in a downward direction, and may prevent the screen 200 from bending in a horizontal direction. Since the first surface 200a is exposed to the user, it may be disposed on the second surface 200b as shown in FIG. 12.

The first vertical pattern 241a and the second vertical pattern 241b of the conductive pattern 241 may be disposed adjacent to the left and right ends of the screen 200 as shown in FIG. 12. The electromagnet 246 may be disposed in a frame adjacent to the first vertical pattern 241a and the second vertical pattern 241b, and a force for pulling the screen 200 in the horizontal direction may be provided by using the electromagnet 246.

When the screen 200 moves vertically, that is, when the motor is driven, the controller may detect the position of the screen 200 using the conductive pattern 241 without applying power to the electromagnet 246 . When the movement of the screen 200 is stopped, that is, when the motor is stopped, the control unit applies power to the electromagnet 246, and the electromagnet 246 provides magnetism that pulls the conductive pattern 241 so that the screen 200 is shown in FIG. As shown in FIG. 1, manpower is provided in the left and right directions, and the screen 200 may be unfolded flat. A pair of electromagnets 246 positioned on the left and right sides of the screen 200 may apply a force in the left and right directions so that the screen 200 can be flatly unfolded regardless of the stop position of the screen 200.

13 is a diagram illustrating another example of a screen sensing unit of the smart wall 100. The screen sensing unit of the present exemplary embodiment may include an ultrasonic sensor 243 that senses a position of the lower end of the first area. The ultrasonic sensor 243 emits ultrasonic waves and senses ultrasonic waves that are reflected and returned to the lower portion of the first area of the screen 200 to measure a distance between the lower end of the screen 200 and the ultrasonic sensor 243. The size of the first area of the screen 200 may be calculated on the basis, and the ultrasonic sensor 243 may be located under the mounting space 114. In addition to ultrasonic waves, infrared or other microwaves can be used to implement a distance sensor.

14 is a diagram illustrating a detailed structure of the screen sensing unit of FIG. 13. The ultrasonic sensor 243 includes a transmitter 243a that emits ultrasonic waves, and a receiver 243b that detects ultrasonic waves that are emitted from the transmitter 243a and reflected from the lower end of the first region.

As shown in (a) of FIG. 14, when the ultrasonic wave is not used, the ultrasonic sensor 243 is accommodated in a housing 244 in which an opening 244a is formed in order to prevent foreign substances from flowing into the ultrasonic sensor 243. The opening 244a may be selectively opened and closed using the sensor cover 245. The sensor cover 245 of this embodiment rotates and moves, and the control unit removes the sensor cover 245 only when the motor rotating the driving roller is driven (Fig. 14A), and the sensor cover 245 is usually It is possible to keep (244a) in the covered state (Fig. 14(b)).

When the screen sensing unit is used, the size of the first area of the screen 200 can be continuously monitored, and thus the driving of the screen 200 can be controlled to be performed naturally. As shown in (a) of FIG. 13, when the first movement starts, the speed of the motor gradually increases from 0 (acceleration section, v1), moving at a constant speed (constant speed section, v2), and then at the stop position (SP). When it is close, the speed can be reduced (v3, deceleration section), and it can be stopped at the stop position (SP) exactly as shown in (b) of FIG. 13.

The smart wall 100 of the present invention described above can provide a neat appearance by mounting multimedia devices such as TV and audio on the wall and being installed on the wall at the same height on the wall.

A cleaner appearance of the smart wall 100 may be provided by using the screen 200 that selectively exposes the display device 161 according to the usage mode of the display device 161.

In addition, since the location of the screen 200 can be accurately detected, the controller can provide an image synchronized with the location of the screen 200. The above detailed description should not be construed as limiting in all respects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

In addition, the smart wall and its control method will be described in detail below with reference to FIGS. 15 to 26. However, those skilled in the art may supplementary interpretations of FIGS. 15 to 26 or modify the embodiments with reference to FIGS. 1 to 14 described previously.

15 is a diagram illustrating a configuration of a smart wall according to an embodiment of the present invention.

Referring to FIG. 15, the smart wall 100 includes a microphone 122, an air purifier 130, a first outlet 132, a second outlet 134, a first camera 140, and a second camera 142. ), display 151, screen 152, driving roller 153, motor 154, screen sensing unit 155, first speaker 160, second speaker 162, memory 170, control unit Including 180.

The smart wall 100 includes a mounting cell in which at least one component is mounted. Here, the parts include an air cleaner and a console game machine.

The microphone 122 processes an external sound signal into electrical voice data. The processed voice data may be variously utilized according to a function (or an application program being executed) being executed in the smart wall 100. Meanwhile, the microphone 122 may be implemented with various noise removal algorithms for removing noise generated in the process of receiving an external sound signal.

The air purifying unit 130 purifies the air when the sensed concentration of the fine dust is greater than or equal to a threshold value.

The first discharge port 132 discharges the purified air into the first space.

The second outlet 134 discharges the purified air into the second space.

The first camera 140 captures at least one first image including a user and an object in front of the screen.

The second camera 142 captures at least one second image including the user and the object.

The first camera 140 and the second camera 142 include a TOF camera.

TOF camera refers to a camera with a TOF sensor attached to a general camera. Specifically, an image sensor that shoots and displays a scene derives a 2D-based result. If the TOF sensor intervenes here, depth measurement becomes possible and 3D results can be realized. TOF (Time of Flight) refers to the time when sound waves or light sources are transmitted to the subject and returned through the subject. The TOF sensor is a kind of component that can detect this role.

The depth map refers to an image containing information related to the distance from an observation point to an object surface in 3D computer graphics.

The TOF camera captures a first image including a depth map. When shooting with a TOF camera, an image is obtained as a result of applying the original image, the depth map, and the depth map to the original image.

The first speaker 160 outputs sound to the first space. The second speaker 162 has a property different from that of the first speaker and includes a second speaker 162 that outputs sound to a second space.

For example, the first speaker 160 may be a 5.1ch omnidirectional speaker, and the second speaker 162 may be a directional speaker. The topographic speaker is a speaker system with directivity that radiates sound only in a target direction, and means that two or more speakers of the same diameter are arranged on the same plane to operate as a single speaker as a whole. It is also called a column speaker because its height is larger than the width or depth, and has a column shape, and is widely used in gymnasiums, auditoriums, and outdoor stadiums. It does not occupy the floor area and has the advantage of being able to adjust the listening position to the height of the ear.

The memory 170 stores a preset motion image of the user.

The controller 180 determines whether to switch to the directional speaker based on the images of the first camera 140 and the second camera 142, and controls at least one of the first outlet 130 and the second outlet 132 do.

The display 151 displays a graphic image.

The screen 152 covers the front surface of the display and is rolled on the display to adjust the screen size of the display. As shown in FIG. 19, the screen size of the display may be a line view, a half view, or a full view.

The driving roller 153 is located above the screen and controls the screen to be wound.

The motor 154 provides rotational force to the driving roller 153.

The screen sensing unit 155 detects the size of the first area of the screen.

The memory 170 stores an image of an object.

The controller 180 controls the driving of the motor based on the size of the first area detected by the screen sensing unit 155, and the user Recognize the motion of, recognize the object based on at least one first image and the at least one second image, determine the user's intention based on the recognized motion of the user and the recognized object, The display is controlled to display manual information corresponding to the determined intention of the user.

16 is a flowchart illustrating a method of controlling a smart wall according to an embodiment of the present invention. The present invention is implemented by the controller 180.

At least one first image including a user and an object in the front is captured with a first camera (S1610).

At least one second image including the user and the object is captured with a second camera (S1620).

A user's motion is recognized based on at least one first image and the at least one second image (S1630).

The object is recognized based on at least one first image and the at least one second image (S1640).

The user's intention is determined based on the recognized user's motion and the recognized object (S1650).

The display is controlled to display manual information corresponding to the determined intention of the user (S1660).

17 is a diagram illustrating a hardware configuration of a smart wall according to an embodiment of the present invention.

Referring to FIG. 17, the smart wall 100 includes a screen 152 and a display 151. As the screen 152 moves up and down on the display 151, the display 151 may adjust the size of the area viewed by the user.

18 is a diagram illustrating a hardware configuration of a smart wall according to an embodiment of the present invention.

With reference to FIG. 18, the mounting space of the smart wall 100 will be described. The mounting space of the smart wall 100 refers to a space in which parts are mounted.

The mounting space of the smart wall 100 includes a driving roller 153, a first speaker 160, a second speaker 162, a display 151, a first camera 140, a second camera 142, and an air purifier. A first outlet 132 connected to and a second outlet 134 connected to the air purifier may be equipped with a device cabinet (not shown), a power distributor (not shown), an Internet distributor (not shown), and the like.

19 is a diagram illustrating a line view, a half view, and a full view of a smart wall according to an embodiment of the present invention. 19 includes FIGS. 19(a), 19(b), and 19(c).

Fig. 19(a) is a diagram showing a line view. Fig. 19(a) is a diagram showing a line view. 19(b) is a diagram showing a half view. 19(c) is a diagram showing a full view.

Referring to FIG. 19(a), the line view refers to a view in which the display 151 is exposed in the smallest size at the bottom of the smart wall, and provides simple notification information.

Referring to FIG. 19B, the half view refers to a view in which the display 151 is exposed in a medium size when the smart wall is stopped, and provides notification information with a larger amount of information.

Referring to FIG. 19(c), the full view refers to a view in which the display 151 is exposed in the largest size when the smart wall is stopped, and is used when watching a movie or watching TV.

20 is a diagram illustrating a notification function of a smart wall according to an embodiment of the present invention. FIG. 20 includes FIGS. 20(a) and 20(b).

20(a) is a diagram showing a line view of a smart wall. 20(b) is a diagram showing a smart notification function of a smart wall in line view.

When a problem related to at least one component occurs, the controller 180 controls the display 151 to display first information indicating a problem related to the at least one component.

For example, when the expiration date of the air purifier filter expires, as shown in FIG. 20(b), the controller 180 displays the information 2010 indicating the expiration of the air purifier filter use by the display 151 in a line view. Control to display.

For example, if the Internet connection is a bad signal, the controller 180 controls the display 151 to display information 2020 indicating a bad internet connection signal in a line view, as shown in FIG. 20(b). .

21 is a diagram illustrating recognition of a user's motion and object recognition with a camera according to an embodiment of the present invention. Fig. 21 includes Figs. 21(a) and 21(b).

FIG. 21(a) is a diagram illustrating recognition of a user's motion and object recognition by a camera mounted on a smart wall. 21B is a diagram showing an object recognized by the camera.

Referring to FIG. 21A, when the user 10 holds the object 20, the first camera 140 captures a first image, and the second camera 142 captures a second image.

The controller 180 recognizes the user's motion 2110 based on the first image and the second image, and the controller 180 recognizes the object 20 based on the first image and the second image.

The controller 180 determines the user's intention based on the recognized user's motion 2110 and the recognized object 20.

Referring to FIG. 21(b), the object 20 includes an air purifier filter 21, a game console 22, a set-top box 23, and a DVD player 24.

An embodiment of determining a user's intention will be described with reference to FIG. 21A. If the correlation between the object 20 and at least one part is more than a predetermined range, the controller 180 determines that the user's intention is to replace the at least one part. Here, the at least one component includes an air purifier filter 21, a game console 22, a set-top box 23, and a DVD player 24.

For example, if the object 20 is any one of the parts, the correlation is determined to be 1. If the object 20 is not related to the part, the correlation is determined to be zero.

For example, if the user's motion 2110 is holding the object 10 and the recognized object 20 is the air purifier filter 21, since the correlation with the part is 1, the controller 180 Decides that the air purifier filter 21 is to be replaced.

For example, if the user's motion 2110 is holding the object 10, and the recognized object 20 is the game console 22, the correlation with the part is 1, so the controller 180 has the intention of the user It is determined that the game console 22 is to be installed.

For example, if the user's motion 2110 is holding the object 10 and the recognized object 20 is the set-top box 23, the correlation with the part is 1, so the controller 180 has the intention of the user It is determined that the set-top box 23 is to be installed.

According to an embodiment of the present invention, in the case of object recognition, in order to increase the recognition rate, an object recognition target is designated in advance, and external characteristics of the object are learned in advance using an AI function.

In addition, it is possible to increase the accuracy of vision recognition by using two cameras at the same time. Here, the two cameras include a first camera 140 and a second camera.

22 is a diagram illustrating displaying manual information of a smart wall according to an embodiment of the present invention. Fig. 22 includes Figs. 22(a) and 22(b).

FIG. 22(a) is a diagram illustrating that the display displays manual information of a smart wall. 22(b) is a diagram illustrating an embodiment of manual information.

Referring to FIG. 22A, the controller 180 controls the display 151 to display manual information 30 corresponding to an object.

For example, the controller 180 determines the user's intention based on the user's motion and object, and controls the display 151 to display the manual information 30 corresponding to the user's intention. Specifically, since the user's intention is recognized as replacement of an object corresponding to a part, the display 151 is controlled to display the manual information 30 corresponding to the object.

An embodiment of changing the size of a screen and an embodiment of automatically switching from line view to half view will be described with reference to FIG. 22A.

When the display 151 displays manual information corresponding to the determined user's intention, the controller 180 controls to automatically change the size of the displayed screen based on the amount of manual information.

For example, if the amount of manual information is not large and is simple, the controller 180 determines the size of the screen as a line view. When the amount of manual information is large enough to be displayed in a line view, the controller 180 determines the size of the screen as a half view. When the amount of manual information is large enough to be displayed in half view, the controller 180 determines the size of the screen as full view.

Referring to FIG. 22B, the controller 180 controls the display 151 to display the air purifier filter replacement guide 2210. The air purifier filter guide 2210 includes at least one of an image and a video.

The controller 180 controls the display 151 to display the console game machine installation guide 2220. The console game machine installation guide 2220 includes at least one of an image and a video.

23 is a diagram illustrating a notification function of a smart wall when a component installation is completed according to an embodiment of the present invention. FIG. 23 includes FIGS. 23(a) and 23(b).

23(a) is a diagram illustrating that a display displays notification information of a smart wall. 23(b) is a diagram illustrating an embodiment of notification information.

Referring to FIG. 23A, when the installation of the at least one component is completed, the controller 180 controls the display 151 to display a notification message 30 indicating completion of the component installation. Specifically, the notification message 30 may be displayed in a line view.

Referring to FIG. 23B, the controller 180 controls the display 151 to display a notification message 2310 indicating completion of the air purifier filter replacement. The air purifier filter replacement complete notification message 2310 is displayed in a line view.

The controller 180 controls the display 151 to display the console game machine installation completion 2320. The console game machine installation completion 2320 is displayed in line view.

24 is a diagram illustrating determining a user's intention using a reinforcement learning model according to an embodiment of the present invention.

Referring to FIG. 24, the reinforcement learning model 300 (Reinforcement Learning, hereinafter, RL) is one of problems handled by machine learning, and the agent 301 defined in a certain environment 302 recognizes the current state 303 , It means a method of selecting an action that maximizes the reward 305 among the selectable actions 304.

The reinforcement learning model 300 includes an agent 301, an environment 302, a state 303, a reward 304, and an action 305.

Agent 301 means observing the state 303, choosing an action 305, and aiming at a goal. The agent 301 includes a learner and a decision maker.

The environment 302 refers to the rest except for the agent 301.

The state 303 refers to information indicating the current situation of a system user.

Action 304 means what the agent 301 does in the current situation.

Reward (305) refers to information indicating good or bad behavior (304).

Referring to FIG. 24, an embodiment of determining a user's intention using a reinforcement learning model reflecting feedback will be described.

The controller 180 receives the feedback from the user and determines the user's intention by using the reinforcement learning model 300 that reflects the received feedback. Feedback includes positive feedback and negative feedback. Here, the feedback corresponds to the compensation 305.

For example, as shown in FIG. 21, when the user 10 carries the object 20 and approaches the smart wall 100, the controller 180 indicates that the intention of the user 10 is the air purifier filter 21 Decide that you want to replace parts such as. When the user 10 actually replaces the air purifier filter 21, the controller 180 determines that the user's intention is correct. If the user 10 does not replace the air purifier filter 21 and does other actions, the controller 180 determines that the user's intention is wrong.

If the determined user's intention is correct, the controller 180 provides additional points to the reinforcement learning model 300, and if the determined user's intention is wrong, the control unit 180 provides the deducted points to the reinforcement learning model 300.

For example, if the determined user's intention is correct, the controller 180 provides an additional point + 10 to the reinforcement learning model 300, and if the determined user's intention is incorrect, the deduction point-10 is provided to the reinforcement learning model 300. to provide.

25 is a diagram illustrating recognition of a user's motion based on a current image and a previous image according to an embodiment of the present invention.

25 is a diagram illustrating prediction of a user behavior based on a 2D image according to an embodiment of the present invention. FIG. 25 includes FIGS. 25(a), 25(b) and 25(c).

FIG. 25(a) is a diagram illustrating a first action of a user expected one second before and a first action of an actual user. FIG. 25B is a diagram illustrating a second action of a user expected one second before and a second action of an actual user. FIG. 25(c) is a diagram showing a third action of a user expected one second before and a third action of an actual user.

Referring to FIG. 25(a), a screen image is analyzed based on a TV drama screen image, and an action expected by the controller 180 one second ago is a kiss, and the next screen is actually a kiss.

Referring to FIG. 25(b), the screen image is analyzed based on the TV drama screen image, and the action expected by the controller 180 one second ago is high five, and the next screen is actually high five.

Referring to FIG. 25(c), a screen image is analyzed based on a TV drama screen image, and the action expected by the controller 180 one second ago is Hug, and the next screen is actually Hug.

According to the present invention, it is possible to train artificial intelligence using a TV drama screen and predict which motion or object will appear on the next screen.

An embodiment of recognizing a user's motion based on a current image and a previous image will be described with reference to FIG. 25.

The controller 180 recognizes a user's motion based on a current image and a previous image maintaining a predetermined time interval among at least one first image.

For example, referring to FIG. 21, the image 3 seconds before is an image in which the user 10 carries the object 20 and approaches the smart wall 100. The current image is an image in which the user 10 holds the object 20 and approaches the smart wall 100 more closely.

Accordingly, the controller 180 recognizes the user's motion based on the current image and the previous image as the user 10 holding the object 20 and walking close to the smart wall in order to replace the parts of the smart wall 100. .

26 is a diagram illustrating a method of photographing a body point with a camera and recognizing a user's motion based on the body point, according to an embodiment of the present invention.

Referring to FIG. 26, when the user 10 carries the object 20 and moves, the camera 140 captures a user's body point and recognizes the user's motion based on this.

Specifically, when the user 10 carries the object 20 and moves, the controller 180 controls the first camera 140 and the second camera 144 to move the user's body points (11, 12, 13, 14, 15). ) To capture, and recognizes the user's motion based on the captured body points (11, 12, 13, 14, 15).

For example, body points include shoulder points 11, elbow points 12, wrist points 13, waist points 14, and knee points 15.

The controller 180 recognizes the user's motion based on the degree of movement and bending of the body point such as the shoulder point 11. Referring to FIG. 26, based on the body point, the controller 180 determines the user 10 to move the object 20 with both hands.

According to an embodiment of the present invention, the user's motion and object are recognized by analyzing the image captured by the camera, the user's intention is determined based on the recognized user's motion and the recognized object, and the determined user's intention is By providing the corresponding manual information, the user's convenience can be improved because the user can easily recognize how to replace parts when replacing parts.

According to another embodiment of the present invention, when a problem occurs in a part mounted on a smart wall, information indicating a problem related to the part is displayed. When a problem occurs in the part, the user does not need to search the manual individually to find a solution to the problem. User convenience can be improved because it can be easily recognized.

According to another embodiment of the present invention, when the display displays manual information corresponding to the determined user's intention, more detailed information is controlled to automatically change the size of the displayed screen from line view to half view based on the amount of manual information. As information can be provided to users, user convenience can be improved.

The smart wall and its control method according to the present invention are not limited to the configuration and method of the described embodiments as described above, but the embodiments are all or part of each embodiment so that various modifications can be made. It may be selectively combined and configured.

Meanwhile, the method for controlling a smart wall of the present invention can be implemented as a code that can be read by a processor on a recording medium that can be read by a processor provided in the smart wall. The processor-readable recording medium includes all types of recording devices that store data that can be read by the processor. Examples of recording media that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and also include those implemented in the form of carrier waves such as transmission through the Internet. . In addition, the recording medium readable by the processor may be distributed over a computer system connected through a network, so that code readable by the processor may be stored and executed in a distributed manner.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

Various embodiments have been described in the best mode for carrying out the present invention.

The present invention is used in a related field of a smart wall in which a display mounted on a smart wall automatically detects a condition of a component mounted on a smart wall or whether a component is abnormal or not.

It is apparent to those skilled in the art that various changes and modifications are possible in the present invention without departing from the spirit or scope of the present invention. Accordingly, the present invention is intended to cover modifications and variations of the present invention provided within the appended claims and their equivalents.

Claims (20)

1. In a smart wall including a mounting cell for mounting at least one component,

   A display that displays a graphic image;

   A screen that covers the front surface of the display and is rolled on the display to adjust a screen size of the display;

   A driving roller positioned above the screen and winding the screen;

   A motor providing rotational force to the drive roller;

   A screen sensing unit that detects the size of the first area of the screen;

   A first camera for capturing at least one first image including a user and an object in front of the screen;

   A second camera capturing at least one second image including the user and the object;

   Controlling the driving of the motor based on the size of the first area detected by the screen sensing unit,

   Recognizing a user's motion based on the at least one first image and the at least one second image,

   Recognizing the object based on the at least one first image and the at least one second image,

   Determine the user's intention based on the recognized user's motion and the recognized object,

   Including a control unit for controlling the display to display manual information corresponding to the determined intention of the user,

   Smart Wall.
2. The method of claim 1, wherein the control unit

   When a problem related to the at least one part occurs, controlling the display to display first information indicating a problem related to the at least one part,

   Smart Wall.
3. The method of claim 1, wherein the control unit

   If the correlation between the object and the at least one part is more than a preset predetermined range, the user's intention is to determine that the at least one part is to be replaced,

   Smart Wall.
4. The method of claim 3, wherein the control unit

   Controlling the display to display manual information corresponding to the object,

   Smart Wall.
5. The method of claim 1, wherein the control unit

   When the display displays manual information corresponding to the determined user's intention, controlling to automatically change the size of the displayed screen based on the amount of manual information

   Smart Wall.
6. The method of claim 1, wherein the control unit

   When the installation of the at least one component is completed, controlling the display to display a notification message indicating the completion of the component installation,

   Smart Wall.
7. The method of claim 1, wherein the control unit

   Receive feedback from users,

   Determining the user's intention using a reinforcement learning model that reflects the received feedback,

   Smart Wall.
8. The method of claim 7, wherein the control unit

   If the determined intention of the user is correct, additional points are provided to the reinforcement learning model,

   If the determined intention of the user is wrong, providing a deduction point to the reinforcement learning model,

   Smart Wall.
9. The method of claim 1, wherein the control unit

   Recognizing a user's motion based on a current image and a previous image maintaining a predetermined time interval among the at least one first image,

   Smart Wall.
10. The method of claim 2, wherein the control unit

    Controlling the first camera and the second camera to capture a user's body point,

    Recognizing the user's motion based on the captured body point,

    Smart Wall.
11. In the smart wall control method,

    Capturing at least one first image including a user and an object on the front side with a first camera;

    Capturing at least one second image including the user and the object with a second camera;

    Recognizing a user's motion based on the at least one first image and the at least one second image;

    Recognizing the object based on the at least one first image and the at least one second image;

    Determining a user's intention based on the recognized user's motion and the recognized object; And

    Controlling the display to display manual information corresponding to the determined intention of the user

    Smart wall control method comprising a.
12. The method of claim 11,

    If a problem related to the at least one part occurs, controlling the display to display first information indicative of a problem related to the at least one part,

    Smart wall control method.
13. The method of claim 11,

    If the correlation between the object and the at least one part is more than a preset predetermined range, the user's intention further comprises the step of determining that the at least one part is to be replaced,

    Smart wall control method.
14. The method of claim 11,

    Further comprising the step of controlling the display to display manual information corresponding to the object,

    Smart wall control method.
15. The method of claim 11,

    When the display displays manual information corresponding to the determined user's intention, controlling to automatically change the size of the displayed screen based on the amount of manual information,

    Smart wall control method.
16. The method of claim 11,

    When the installation of the at least one component is completed, controlling the display to display a notification message indicating the completion of the component installation,

    Smart wall control method.
17. The method of claim 11,

    Receiving feedback from a user; And

    Further comprising the step of determining the user's intention using the reinforcement learning model reflecting the received feedback,

    Smart wall control method.
18. The method of claim 17,

    If the determined intention of the user to be included is correct, providing additional points to the reinforcement learning model; And

    If the determined intention of the user to be included is wrong, further comprising providing a deduction point to the reinforcement learning model,

    Smart wall control method.
19. The method of claim 11,

    Recognizing a user's motion based on a current image and a previous image maintaining a predetermined time interval among the at least one first image,

    Smart wall control method.
20. The method of claim 12,

    Controlling the first camera and the second camera to capture a user's body point; And

    Further comprising the step of recognizing the user's motion based on the captured body point,

    Smart wall control method.

Images