KR102427761B1 - Did device, did system, and control method for a did device - Google Patents

Abstract

A DID device, a DID system, and a control method of the DID device are disclosed. The DID device according to one aspect of the present invention may include: a face recognition unit connected to an external server and configured to determine whether a user's face is detected in image information acquired through a face recognition camera and to determine a height of the user's face; a voice command recognition unit configured to acquire voice information through a voice recognition microphone; a transceiving unit configured to transmit the voice information to the external server and to receive text information corresponding to the voice information from the external server; and a command processing unit configured to generate screen output information and audio output information based on response information generated in response to the text information, transmit the screen output information to a display device, and deliver the audio output information to a speaker. The command processing unit may determine a screen division type of the display device according to the type of the response information. The present invention can effectively deliver information to users.

Description

DID device, DID system, and control method of DID device {DID DEVICE, DID SYSTEM, AND CONTROL METHOD FOR A DID DEVICE}

The present invention relates to a DID device, and more particularly, to a DID device capable of effectively delivering information to a user, a DID system, and a method for controlling the DID device.

A DID (Digital Information Display) device was developed to serve as a guide and publicity, and has recently been used in various places such as government offices, airports, subway stations, and hospitals. When the DID device is installed in a specific facility, the DID device guides the user about what is around the facility and what the environment is, so that the user can use the facility more conveniently.

Although the DID device has high potential for user convenience, the conventional DID device provides limited information and is not intuitive to use, so users do not use it frequently. As a result, many of the currently used DID devices only serve as billboards providing only an advertisement screen for most of the time.

Accordingly, the present invention has been derived to solve the above problems, and one aspect of the present invention is to provide a DID device, a DID system, and a method for controlling the DID device that can effectively deliver information to a user.

Other objects of the present invention will become clearer through the examples described below.

One aspect of the present invention provides a DID device. A DID device according to an aspect of the present invention is connected to an external server, and includes: a face recognition unit configured to determine whether a user's face is detected in image information obtained through a face recognition camera and to determine the height of the user's face; a voice command recognition unit configured to acquire voice information through a voice recognition microphone; a transceiver configured to transmit the voice information to the external server and receive text information corresponding to the voice information from the external server; and a command processing unit configured to generate screen output information and audio output information based on the response information generated in response to the text information, transmit the screen output information to a display device, and transmit the audio output information to a speaker and the command processing unit may determine a screen division type of the display device according to the type of the response information.

The DID device according to the present invention may have one or more of the following embodiments. For example, the command processing unit may transmit the text information to the external server through the transceiver and obtain the response information from the external server through the transceiver. In addition, the command processing unit compares a face recognition time corresponding to a time for maintaining the state in which the user's face is detected in the image information with a reference value, and when the face recognition time exceeds the reference value, preset guide information is provided to the speaker can be sent to

Another aspect of the present invention provides a DID system. A DID system according to an aspect of the present invention includes a group server connected to an external server; and a plurality of DID devices connected to the group server and installed in different areas of one facility, wherein each of the plurality of DID devices may include a sensor, and at least one of the plurality of DID devices comprises: a face recognition unit configured to determine whether a user's face is detected in the image information obtained through the face recognition camera and to determine the height of the user's face; a voice command recognition unit configured to acquire voice information through a voice recognition microphone; transmits the voice information to the group server, receives text information corresponding to the voice information from the group server, transmits a response information request corresponding to the text information to the group server, and responds to the response information request from the group server a transceiver configured to receive corresponding response information; and generating the response information request signal based on the text information, generating screen output information and audio output information based on the response information, transmitting the screen output information to a display device, and transmitting the audio output information to a speaker and a configured command processing unit, wherein the response information may include information detected by a sensor of at least one DID device among the plurality of DID devices, and the command processing unit may include the display device according to the type of response information. You can determine the screen division type of .

Another aspect of the present invention provides a method for controlling a DID device. A DID device control method according to an aspect of the present invention is a method for controlling a DID device connected to an external server, comprising: receiving image information from a face recognition camera; determining whether a user's face is detected in the image information; comparing a face recognition time corresponding to a time in which the user's face is maintained in the image information with a reference value and determining that the user is on standby when the face recognition time exceeds the reference value; receiving voice information when it is determined that the user is waiting; transmitting the voice information to the external server; receiving text information corresponding to the voice information; and generating screen output information and voice output information based on the response information generated in response to the text information, wherein the screen output information may designate a screen division form of the display device, and the screen division The form may be determined according to the type of the response information.

According to the problem solving means of the present invention as described above, various effects including the following items can be expected. However, the present invention is not established only when all of the following effects are exhibited.

According to an embodiment of the present invention, there is provided a DID device, a DID system, and a method for controlling a DID device that can provide various types of information quickly and can be used in an intuitive way to induce a user to use it.

1 is a block diagram illustrating a DID system according to an embodiment of the present invention.
2 is a conceptual diagram illustrating a first configuration example of a DID system according to an embodiment of the present invention.
3 is a conceptual diagram illustrating a second configuration example of a DID system according to an embodiment of the present invention.
4 is a diagram illustrating a screen that can be output by the DID interface according to an embodiment of the present invention.
5 is a flowchart illustrating the operation of a DID system according to an embodiment of the present invention.
6 is a flowchart illustrating the operation of a DID system according to an embodiment of the present invention.
7 is a flowchart illustrating the operation of a DID system according to an embodiment of the present invention.
8 is a flowchart illustrating a method of controlling a DID device according to an embodiment of the present invention.
9 is a flowchart illustrating a method of controlling a DID device according to an embodiment of the present invention.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings, and in the description with reference to the accompanying drawings, the same or corresponding components are given the same reference numbers regardless of the reference numerals, and duplicates thereof A description will be omitted.

1 is a block diagram illustrating a DID system 1000 according to an embodiment of the present invention. 2 is a conceptual diagram illustrating a first configuration example of the DID system 1000 according to an embodiment of the present invention, and FIG. 3 illustrates a second configuration example of the DID system 1000 according to an embodiment of the present invention. It is a conceptual diagram that 4 is a diagram illustrating a screen 152 that can be output by the DID interface 100 according to an embodiment of the present invention.

The DID system 1000 according to an embodiment of the present invention may largely include a DID interface 100 , a DID device 200 , a group server 300 , external servers 400 and 500 , and a DID management server 600 . can

Although the DID interface 100 and the DID device 200 are depicted as separate components in the drawing, the DID interface 100 and the DID device 200 may be integrated into one. Also, the DID system 1000 may include a plurality of the DID interface 100 and the DID device 200 . For example, when the DID system 1000 is applied to one public office, the DID interface 100 and the DID device 200 may be installed in main locations such as a lobby and an elevator hall of each floor, respectively.

The DID interface 100 may include a face recognition camera 110 , a voice recognition microphone 120 , a motion sensor 130 , a touch screen 140 , a display device 150 , and a speaker 160 . As described above, the DID interface 100 may be implemented in an integrated form with the DID device 200 . The DID interface 100 may be implemented, for example, in the form of a panel or a kiosk installed in a specific area of a facility to which the DID system 1000 is applied.

The face recognition camera 110 may be installed toward the front of the DID interface 100 and configured to acquire image information. Although the face recognition camera 110 itself may include a function for recognizing a face, it may be implemented in the form of a general camera.

The voice recognition microphone 120 may be installed toward the front of the DID interface 100 to obtain voice information. The voice recognition microphone 120 may be set to always acquire voice information, or it may be set to acquire voice information only when the DID interface 100 is switched to the voice input standby mode under the control of the DID device 200 . have.

The motion sensor 130 may be configured to detect a user's motion within a predetermined area in front of the DID interface 100 . The motion sensor 130 may, for example, emit a plurality of infrared rays from the upper or lower portion of the DID interface 100 at a predetermined angle to detect an input of the reflected rays.

The touch screen 140 may be configured to receive a user input by a user's contact. The touch screen 140 may be implemented in an integrated form with the display device 150 . The DID interface 100 according to some embodiments may include only one of the motion sensor 130 and the touch screen 140 .

The display device 150 outputs a screen shown to the user, and may be configured to visually provide information to the user. The display device 150 may output screen output information provided by the command processing unit 240 of the DID device 200 to be described later on the screen 152 (refer to FIG. 4 ).

The speaker 160 outputs a voice and/or sound signal to the user, and may be configured to provide information aurally to the user. The speaker 160 may output audio output information provided by the command processing unit 240 of the DID device 200 to be described later.

The DID device 200 includes a face recognition unit 210 , a voice command recognition unit 220 , a motion recognition unit 230 , a command processing unit 240 , a transceiver 250 , a sensor 260 and a database 270 . may include As described above, the DID device 200 may be implemented in an integrated form with the DID interface 100 , and if necessary, one DID device 200 may be connected to a plurality of DID interfaces 100 .

The face recognition unit 210 may determine whether the user's face is detected in the image information obtained through the face recognition camera 110 . The face recognition unit 210 may determine whether, for example, a shape that can be seen as corresponding to a human face in the image information exists and the shape satisfies a predetermined size condition. When the shape corresponding to the human face is detected to have a size within a predetermined range and the detected state is maintained in excess of the reference value, the face recognition unit 210 determines that the user is waiting in front of the DID interface 100 can do. The face recognition unit 210 may extract the height and/or size of the user's face in the acquired image information.

The voice command recognition unit 220 may be configured to acquire voice information through the voice recognition microphone 120 . The voice command recognition unit 220 may extract, for example, voice information during a period designated by the command processing unit 240 from among the voice information always received through the voice recognition microphone 120 , and is controlled by the command processing unit 240 . It may be set so that voice information is acquired through the voice recognition microphone 120 only when a signal is received.

The motion recognition unit 230 may be configured to recognize a user's input based on a detection signal of the motion sensor 130 and/or the touch screen 140 . For example, in the case of the DID interface 100 provided with the motion sensor 130 , the motion recognition unit 230 may recognize the user's hand motion based on the detection signal of the motion sensor 130 , and A corresponding input may be configured to be presented on the display device 150 . In addition, in the case of the DID interface 100 provided with the touch screen 140 , the motion recognition unit 230 may recognize a user input provided through the touch screen 140 .

The command processing unit 240 may correspond to a part that performs overall control of the DID device 200 . In particular, the command processing unit 240 may serve to identify the user's intention from the user input obtained from the voice command recognition unit 220 and the motion recognition unit 230 and provide information requested by the user. The command processing unit 240 may generate information requested by the user in the form of screen output information and audio output information, and may transmit them to the display device 150 and the speaker 160, respectively.

In the database 270 of the DID device 200 , setting information regarding a screen configuration of the display device 150 of the DID interface 100 associated with the DID device 200 may be stored. Here, the setting information on the screen configuration may include information on various screen division forms that the screen 152 of the display device 150 can take. That is, various arrangements in which the screen 152 can be divided into a plurality of division areas may be stored in the setting information regarding the screen configuration. The command processing unit 240 may designate a specific screen division type when generating screen output information, and may designate what content to output in each division area.

In some embodiments, the face recognition unit 210 or the command processing unit 240 may extract the height and/or size of the user's face from the user's face in the image information obtained through the face recognition camera 110 . The command processing unit 240 may further estimate the height and/or age of the user who is using the DID device 200 , and may reflect this to the screen output information.

The command processing unit 240 may generate screen output information by reflecting the user's key. For example, in providing the information requested by the user, the command processing unit 240 divides the screen 152 into three division zones 154 , 156 , and 158 in the vertical direction as shown in FIG. 4 . You can choose the shape. In this case, the command processing unit 240 outputs the information requested by the user to the middle partition 156 with the best visibility at the height of the user (at the height of the user's face), and related information and/or other information Screen output information may be generated to be output to the remaining divided areas 154 and 158 .

Also, the command processing unit 240 may estimate the age and/or gender of the user from the user's face recognized by the face recognition unit 210 . In this case, the command processing unit 240 may generate screen output information so that content that may be of interest to a user of the corresponding age group or gender is output to one or more of the divided areas 154, 156, and 158 of the screen 152. .

The transceiver 250 may transmit/receive information to and from the group server 300 and/or the external servers 400 and 500 under the control of the command processing unit 240 . That is, the DID device 200 may be connected to the group server 300 and/or the external servers 400 and 500 by the transceiver 250 . Here, when the DID device 200 is connected with the group server 300 and/or the external servers 400 and 500, it means that the DID device 200 is connected to enable information exchange through various wired and/or wireless networks.

The sensor 260 may be configured to measure specific environmental information at a location where the DID device 200 or the DID interface 100 is installed. The environmental information measured by the sensor 260 may include, for example, the temperature, humidity, fine dust concentration, etc. of an indoor space in which the DID device 200 or the DID interface 100 is installed or an outdoor space around the corresponding facility, and the DID The number of people entering and exiting the indoor space in which the device 200 or the DID interface 100 is installed may be included.

The database 270 may store information required by various components of the DID device 200 and information measured by the sensor 260 .

The group server 300 may include a sensor control center 310 , a controller 320 , and a database 330 . The group server 300 may serve to control a plurality of DID devices 200 . In some embodiments, one of the plurality of DID devices 200 may also serve as the group server 300 , and in some embodiments, the group server 300 may be omitted. In addition, in some embodiments, the group server 300 may be integrated with the DID management server 600 to be described later.

The sensor control center 310 may manage the sensors 260 of the plurality of DID devices 200 , and store measurement values collected by each sensor 260 in the database 330 . When the DID devices 200 are installed in different areas of one facility, the value of one sensor 260 among the plurality of DID devices 200 is excessively different from the sensor 260 of the other DID devices 200 . In the case of indicating the sensor control center 310 may notify this to the manager.

The controller 320 may control the overall operation of the group server 300 . When the DID device 200 requests information from the group server 300 , the controller 320 may provide the corresponding information to the DID device 200 from the sensor control center 310 and/or the database 330 .

The database 330 may store various data related to the operation of the sensor control center 310 and the control unit 320 .

In the DID system 1000 according to an embodiment of the present invention, the DID device 200 may be connected to one or more external servers 400 and 500 .

One type of external server 400 to which the DID device 200 can be connected may include a voice recognition module 410 , a chatbot module 420 , and a voice synthesis module 430 . The external server 400 may be, for example, a server that provides a service similar to an Intelligent Virtual Assistant (IVA) such as Naver's CLOVA service. The external server 400 may be implemented in a cloud platform.

The voice recognition module 410 may be configured to perform a voice recognition function. For example, when the command processing unit 240 transmits the voice information obtained from the voice command recognition unit 220 to the voice recognition module 410 through the transceiver 250 , the voice recognition module 410 is STT (Speech) to Text) conversion to create text information. The voice recognition module 410 may transmit the generated text information back to the DID device 200 , and the command processing unit 240 may use the text information to determine the user's intention.

The chatbot module 420 may be a module that performs a chatbot function. When the user pronounces, for example, a preset call phrase, the DID device 200 calls the chatbot module 420 so that the user's voice query is sent to the chatbot module ( 420) can be entered. The answer provided by the chatbot module 420 may be provided in the form of text information by the voice recognition module 410 .

The speech synthesis module 430 may be configured to perform a speech synthesis function. For example, when the command processing unit 240 transmits response information to be provided to the user to the voice synthesis module 430 through the transceiver 250 , the voice synthesis module 430 performs Text to Speech (TTS) conversion. Thus, it is possible to generate audio output information. The voice synthesis module 430 may transmit the generated voice output information to the DID device 200 again, and the command processing unit 240 may transmit the voice output information to the speaker 160 .

Meanwhile, another type of external server 500 to which the DID device 200 can be connected may be a server that provides various information. The external server 500 may provide, for example, weather forecast, fine dust status, latest news, etc., and the command processing unit 240 of the DID device 200 provides necessary information according to the identified user's intention. You can connect to the external server 500 by selecting it. The command processing unit 240 may receive necessary information from the connected external server 500 to create response information.

The DID management server 600 of the DID system 1000 according to an embodiment of the present invention may be connected to a plurality of DID devices 200 and/or one or more group servers 300 . The DID management server 600 may include a kiosk management unit 610 , a content management unit 620 , and a database 630 .

The kiosk management unit 610 may be used to manage the plurality of DID devices 200 . The kiosk manager 610 may store information about each DID device 200 in the database 630 .

The content manager 620 may transmit content that can be used to be output on the display device 150 of the DID interface 100 associated with the DID device 200 to the DID device 200 and/or the group server 300 . . If necessary, the content management unit 620 may store the screen size and possible screen division form of the display device 150 associated with each DID device 200 in the database 630, and the content in each screen size and screen division form You can save the settings for outputting .

As shown in FIG. 2 , the DID management server 600 has an interface server on the external network to connect between the internal network including the DID device 200 and/or the group server 300 and the Internet area using the https protocol, etc. can be implemented. As described above, in some embodiments, the group server 300 may be integrated with the DID management server 600 , and when the group server 300 is not used, one of the DID devices 200 is the DID management server 600 . ) can also be combined.

Meanwhile, as shown in FIG. 3 , the DID management server 600 may be implemented by placing an interface server in the cloud area. In this case, it is also possible to implement the DID management server 600 in the same cloud service as the external server 400 . In this case, too, the interface server for the DID management server 600 may be connected to the internal network including the DID device 200 and/or the group server 300 using https protocol or the like. If necessary, the administrator may connect the DID management server 600 to the mobile phone 650 .

Hereinafter, the operation of the DID system 1000 according to an embodiment of the present invention will be described in more detail with reference to FIGS. 5 to 7 . 5 to 7 are flowcharts illustrating the operation of the DID system according to an embodiment of the present invention.

Referring to FIG. 5 , the DID interface 100 may first provide image information to the DID device 200 ( S710 ). The DID interface 100 may obtain image information by constantly driving the face recognition camera 110 , which may be provided to the DID device 200 . Assuming that the DID interface 100 is installed in a specific area of a certain facility, the face recognition camera 110 of the DID interface 100 may continuously photograph pedestrians passing through the corresponding area. The DID interface 100 may normally output a guide and explanation for inducing the user to use the DID interface 100 through the display device 150 and/or the speaker 160 .

The DID device 200 may analyze the provided image information to determine whether the user is waiting (S720). For example, when a shape that can be seen as corresponding to a human face exists in the image information and the shape satisfies a predetermined size condition, the face recognition unit 210 of the DID device 200 detects the user's face. can be judged as When a shape corresponding to a person's face is detected with a size within a predetermined range and the detected state is maintained in excess of the reference value, the face recognition unit 210 of the DID device 200 displays the user in front of the DID interface 100 . can be considered to be waiting.

That is, at the location where the DID interface 100 is installed, many pedestrians may pass in front of the DID interface 100 , and some pedestrians may look at the DID interface 100 , but they all want to use the DID interface 100 . It won't be. Therefore, the user's face is detected with a size within a predetermined range and is located within a predetermined distance from the DID interface 100, and at the same time, the time at which the user's face is detected exceeds a preset reference value (for example, 3 seconds), and the DID interface 100 is activated. When it is determined that the pedestrian is not passing, the DID device 200 may determine that the user is waiting.

Of course, since the DID interface 100 may receive a user input in various ways, the step S720 of the DID device 200 determining whether the user is waiting is not limited to the above method. For example, when the user pronounces a call phrase at a volume higher than the reference value through the voice recognition microphone 120 or applies an input to the screen 152 through the motion sensor 130 or the touch screen 140, the DID device ( 200 ) may determine that the user is waiting in front of the DID interface 100 .

Meanwhile, the face recognition unit 210 of the DID device 200 may extract the height and/or size at which the user's face is located from the provided image information. The face recognition unit 210 may estimate the height and/or size of the face by image analysis, or may use a separate infrared sensor (not shown) provided in the DID interface 100 . The command processing unit 240 of the DID device 200 may estimate the user's age and/or gender from the extracted height and/or size of the user's face, and may be used to generate screen output information later.

When the face recognition unit 210 of the DID device 200 determines that the user is waiting in front of the DID interface 100 , the command processing unit 240 of the DID device 200 sends a user standby state to the DID interface 100 . A notification may be made (S730). This may be accomplished by transmitting a specific control signal to the DID interface 100 .

Upon receiving the notification of the user standby state, the DID interface 100 may be set to the voice input standby mode (S740). In the voice input standby mode, the DID interface 100 may receive the user's voice information through the voice recognition microphone 120 . Of course, when the voice input standby mode is set, the DID interface 100 may first output a guide message through the display device 150 and/or the speaker 160 . The guide message may include a greeting to welcome the user and a description of how to use the corresponding DID interface 100 .

The user may request information in various ways. As described above, the user may request necessary information using the interface output on the screen 152 of the display device 150 using the motion sensor 130 or the touch screen 140 . may be

When the user inputs a command by voice or requests information, the DID interface 100 may provide the received voice information to the DID device 200 (S750). The voice command recognition unit 220 of the DID device 200 may extract voice information from the received data and process such as noise removal, and the command processing unit 240 of the DID device 200 includes the transceiver 250 . Voice information may be transmitted to the external server 400 through (S760).

The voice recognition module 410 of the external server 400 may convert the received voice information into text (S770), and may provide the generated text information to the DID device 200 again (S780).

Upon receiving the text information corresponding to the voice information, the command processing unit 240 of the DID device 200 may determine the user's intention based on the text information (S790). The command processing unit 240 of the DID device 200 may extract a main keyword to determine the user's intention, and may determine where to refer to to obtain information requested by the user. For example, some types of information requested by the user may be stored in the database 270 of the DID device 200 (eg, an introduction to the area of the facility where the DID interface 100 is installed, and the screen 152 ). The next page of the displayed menu, etc.), may be requested to the group server 300 associated with the DID device 200 (eg, air quality in other areas of the facility where the DID interface 100 is installed, etc.), and an external server ( 400, 500) (eg, latest news, etc.).

6 is a flowchart illustrating a case in which the command processing unit 240 of the DID device 200 inquires the external server 400 to obtain information requested by the user.

As a result of determining the user's intention (S790), if it is determined that the information requested by the user should be obtained from the external server 400, the command processing unit 240 of the DID device 200 transmits the requested response information request to the external server ( 400) (S800). The external server 400 may generate response information (S810) and provide the corresponding response information to the DID device 200 again (S820).

Of course, if the user's intention is determined (S790) and it is determined that the information requested by the user can be obtained from the database 270 of the DID device 200, the step of requesting the corresponding information (S800) may be omitted. Therefore, the step of generating the response information in the external server 400 (S810) and the step of providing the response information (S820) can also be omitted.

On the other hand, in the step of determining the user's intention (S790), if it is determined that the user pronounces a phrase for calling the chatbot, the step of requesting response information (S800), the step of generating the response information (S810), and The step of providing response information ( S820 ) may correspond to a process in which the DID device 200 calls the chatbot module 420 of the external server 400 to mediate a conversation between the user and the chatbot module 420 .

The command processing unit 240 of the DID device 200 generates output information based on the response information received from the external server 400 or read from the database 270 and determines the screen output arrangement (S830). can do. Here, the output information may include screen output information and audio output information. The screen output information corresponds to the content to be output by the display device 150 of the DID interface 100 on the screen 152 , and the audio output information is the DID It corresponds to the content to be output by the speaker 160 of the interface 100 in the form of voice and/or sound.

One or more screen division types usable on the screen 152 of the display device 150 of the DID interface 100 and setting values associated with each screen division type may be stored in the database 270 of the DID device 200 . For example, FIG. 4 shows a screen division form in which the screen 152 is divided into three division areas 154 , 156 , and 158 along the vertical direction. Of course, the screen 152 may be divided not only in the vertical direction but also in the left and right directions, and may be divided into fewer or more than three division areas.

The command processing unit 240 of the DID device 200 may select a screen division form most suitable for the corresponding response information, and in particular, may select a screen division type based on the previously estimated height and/or age of the user. The command processing unit 240 may write the screen output information such that the main information requested by the user is displayed in a division corresponding to the height of the user's face and secondary related information or advertisements are displayed in the other division area.

Taking the screen 152 of FIG. 4 as an example, when the user requests information about air quality, the command processing unit 240 moves up and down as shown based on the response information corresponding to the user's request and the height of the user's face. A screen division form including three divisions can be selected, and the most important part of the response information is displayed in the middle division 156 with the best visibility from the estimated user's face height, and in the other divisions 154 An advertisement of a product related to air quality measurement may be displayed, and screen output information may be written so that news and announcements related to air quality or general news and announcements are displayed in another partition area 158 .

When the command processing unit 240 of the DID device 200 generates screen output information and audio output information, the DID device 200 may provide these output information to the DID interface 100 ( S840 ). The DID interface 100 may receive them and output them as a screen and a voice through the display device 150 and the speaker 160 (S850).

7 is a flowchart illustrating a case in which the command processing unit 240 of the DID device 200 inquires the group server 300 to obtain information requested by the user.

As a result of determining the user's intention (S790), if it is determined that the information requested by the user should be obtained from the group server 300, the command processing unit 240 of the DID device 200 transmits the requested response information request to the group server ( 300) (S860). The group server 300 may read the response information from the database 330 and provide the corresponding response information to the DID device 200 again (S820).

The DID device 200 generates output information based on the response information, determines the screen output arrangement (S830), and provides the screen output information and the audio output information to the DID interface 100 (S840) and the DID interface 100 ) of outputting the screen output information and the audio output information (S850) may be the same as described above, and the overlapping description will be omitted.

Hereinafter, a method of controlling the DID device 200 according to an embodiment of the present invention will be described with reference to FIGS. 8 and 9 . 8 and 9 are flowcharts illustrating a control method of the DID device 200 according to an embodiment of the present invention. For reference, although it is assumed in FIGS. 8 and 9 that the DID interface 100 and the DID device 200 are integrated, the DID interface 100 and the DID device 200 may be configured separately as described above. A plurality of DID interfaces 100 may be connected to one DID device 200 .

8 is a flowchart illustrating a case in which a user requests information by voice. First, the DID device 200 may receive image information (S910). For example, the DID interface 100 may drive the face recognition camera 110 to obtain image information and provide it to the DID device 200 . In this step, the DID device 200 may control the DID interface 100 to output a guide and explanation for inducing the user to use the DID interface 100 through the display device 150 and/or the speaker 160 . have.

The DID device 200 may determine whether the face recognition time exceeds a reference value with respect to the received image information (S920). Here, the face recognition time refers to a time during which a shape corresponding to a person's face is detected with a size within a predetermined range and the detected state is maintained. When a shape that can be seen as corresponding to a human face exists in the image information and the shape satisfies a predetermined size condition, it may be determined that the person is in front of the DID interface 100 . Also, when the detected state continues for a time exceeding the reference value, it may be estimated that the person intends to use the DID interface 100 .

If the face recognition time does not exceed the reference value, the DID device 200 may continue to receive the image information without performing a special operation (S910).

If the face recognition time exceeds the reference value, the DID apparatus 200 may estimate the user's characteristics (S925). For example, the DID device 200 may extract the height and/or size of the user's face, and may estimate the height of the user therefrom. In some embodiments, the DID device 200 may estimate the age and/or gender of the user.

Subsequently, the DID device 200 may transmit a guide message that can be output by the display device 150 and/or the speaker 160 of the DID interface 100 to the DID interface 100 . The guide message may include a greeting to welcome the user and a description of how to use the corresponding DID interface 100 , and may be output on a screen and/or voice. In some embodiments, the announcement message may be selected based on the user's estimated age and/or gender. One or more guide messages may be stored in the database 270 of the DID device 200, and one of them may be selected arbitrarily or by a specific algorithm.

At the same time, the DID device 200 may set the DID interface 100 to a voice input standby mode. In the voice input standby mode, the DID interface 100 may receive the user's voice information through the voice command recognition unit 220 .

The DID device 200 may analyze the voice information received from the DID interface 100 to determine the user's intention (S950). This includes the steps of extracting voice information from data input through the voice recognition microphone 120 of the DID interface 100 and transmitting the voice information to the external server 400 as described in relation to FIG. 5 (S760). ) and receiving text information corresponding to the voice information from the external server 400 ( S780 ) may be included.

The DID device 200 may determine a request processing method according to the identified user intention (S960). For example, if the user pronounces a calling phrase, voice information may be transmitted/received between the DID interface 100 and the external server 400 so that the user communicates with the chatbot module 420 of the external server 400 . . In addition, the response information may be read from the database 270 or requested from the group server 300 or the external servers 400 and 500 depending on where the user can obtain the requested information.

When the response information is obtained, the DID device 200 may generate output information based on the response information and determine the screen output arrangement ( S970 ). The output information may include screen output information and audio output information. The screen output information corresponds to the content to be output by the display device 150 of the DID interface 100 on the screen 152 , and the audio output information is the DID interface It corresponds to the content to be output by the speaker 160 of (100) in the form of voice and/or sound.

As described above, in the database 270 of the DID device 200, one or more screen division types usable on the screen 152 of the display device 150 of the DID interface 100 and setting values associated with each screen division type are stored. may be stored, the DID device 200 may select a screen division form most suitable for the corresponding response information, and may select a screen division form and content displayed in each division based on the user's height and/or age.

On the other hand, the voice output information is the most important part of the response information among the screen output information, and may include contents corresponding to the content displayed in the partition having the best visibility from the user's key. If necessary, the DID device 200 may write response information in text format and transmit it to the voice synthesis module 430 of the external server 400 , and the voice synthesis module 430 generates voice output information based on this. to transmit to the DID device 200 .

When the screen output information and the audio output information are written as described above, the DID device 200 may output a screen and output a voice through the display device 150 and the speaker 160 of the DID interface 100 (S980). . As a result, the user can receive the requested information visually and audibly through the display device 150 and the speaker 160 of the DID interface 100 .

9 is a flowchart illustrating a case in which a user requests information through a hand gesture or a touch screen input. Similar to the case illustrated in FIG. 8 , the DID device 200 may receive image information ( S910 ). This may be done, for example, through the face recognition camera 110 of the DID interface 100 .

The user may make a hand gesture or apply a contact to the touch screen 140 to use the DID interface 100 . When an input is sensed from the motion sensor 130 or the touch screen 140 of the DID interface 100, the DID device 200 may receive corresponding operation information (S915).

When the user's motion information is received and it is determined that the user is waiting to use the DID interface 100, the DID apparatus 200 may estimate the user's characteristics (S925). For example, the DID device 200 may extract the height and/or size of the user's face by analyzing the image information, and may estimate the user's height from this. In some embodiments, the DID device 200 may estimate the age and/or gender of the user.

The DID device 200 may determine a request processing method according to a user's input (S960). For example, response information may be read from the database 270 or may be requested from the group server 300 or the external servers 400 and 500 depending on where the user can obtain the requested information.

Subsequently, the DID device 200 generates output information based on the response information and determines the screen output arrangement (S970) and the screen output information and the screen output information through the display device 150 and the speaker 160 of the DID interface 100 The step of outputting the audio output information (S980) may be the same as described above, and a redundant description will be omitted.

The DID device 200 , the DID system 1000 and the DID device control method according to an embodiment of the present invention output a guide and a description guiding a user to use the DID interface 100 by outputting a guide and a description of the DID interface 100 . usability can be increased.

The user may request information using various types of inputs including voice and hand gestures, and the DID device 200 may visually and aurally output the information requested by the user through the DID interface 100 . This allows the user to use the DID interface 100 more intuitively, thereby increasing the usability of the DID interface 100 .

When the user uses the DID interface 100 , the DID device 200 detects the height and size of the user's face to estimate some features including the user's height, and when providing information requested by the user, the user The most effective screen division form and screen arrangement can be determined in consideration of the characteristics of , and information can be more effectively delivered to the user by allowing the display device 150 to provide the screen 152 based on this.

In particular, the DID device 200, the DID system 1000, and the DID device control method according to an embodiment of the present invention satisfy the system requirements of the DID device 200 itself by allowing some functions to be processed by the external server 400. While keeping it relatively low, it can quickly provide information in a very wide range of fields.

Although the above has been described with reference to one embodiment of the present invention, those of ordinary skill in the art can variously change the present invention within the scope without departing from the spirit and scope of the present invention described in the claims below. It will be appreciated that modifications and variations are possible.

1000: DID system
100: DID interface 200: DID device
300: group server 400, 500: external server

Claims (5)

A DID device connected with a group server, a DID management server and an external server,
In the image information obtained through the face recognition camera, a shape corresponding to a human face is detected to a size within a predetermined range, and by determining whether the detection time exceeds a preset reference value, it is determined whether a user's face is detected, and the a face recognition unit configured to determine the height of the user's face;
a voice command recognition unit configured to acquire voice information through a voice recognition microphone;
a transceiver configured to transmit the voice information to the external server and receive text information corresponding to the voice information from the external server; and
The height, age, and gender of the user are estimated from the user's face detected by the face recognition unit, and a guide message is selected based on the estimated age and gender of the user, and then the guide message is transmitted through a display device and a speaker. output, extracting main keywords from the text information to determine the user's intention, and generate response information from the database or sensor of the DID device according to the identified user's intention, or through the transceiver, the group server and the Receive response information from one or more of external servers, generate screen output information and audio output information based on the generated or received response information, transmit the screen output information to the display device, and transmit the audio output information to the display device a command processing unit configured to transmit to the speaker;
The database stores possible screen division types received from the DID management server,
The command processing unit determines the screen division type of the display device by selecting one of the possible screen division types stored in the database according to the type of the response information, and displays the display device according to the estimated height, age and gender of the user. Generates output to one or more of the divisions of the device's screen,
The response information includes information obtained from the database or the sensor, information obtained from at least one other DID device connected to the group server, or information obtained from the external server. DID device.
delete delete group servers connected to external servers;
a DID management server connected to the group server; and
A plurality of DID devices connected to the group server and the DID management server and installed in different areas of one facility,
Each of the plurality of DID devices includes a sensor,
At least one of the plurality of DID devices,
In the image information obtained through the face recognition camera, a shape corresponding to a human face is detected to a size within a predetermined range, and by determining whether the detection time exceeds a preset reference value, it is determined whether a user's face is detected, and the a face recognition unit configured to determine the height of the user's face;
a voice command recognition unit configured to acquire voice information through a voice recognition microphone;
a transceiver configured to transmit the voice information to the group server and receive text information corresponding to the voice information from the group server; and
The height, age, and gender of the user are estimated from the user's face detected by the face recognition unit, and a guide message is selected based on the estimated age and gender of the user, and then the guide message is transmitted through a display device and a speaker. output, extract a main keyword from the text information to determine the user's intention, generate response information from the sensor according to the identified user's intention, or at least one of the group server and the external server through the transceiver configured to receive response information from, generate screen output information and audio output information based on the generated or received response information, transmit the screen output information to the display device, and transmit the audio output information to the speaker a command processing unit;
When the response information is received from the group server, it includes information detected by a sensor of at least one DID device among the plurality of DID devices,
The DID management server stores the screen size and possible screen division form of the display device,
The command processing unit determines the screen division type of the display device by selecting one of the possible screen division types stored in the DID management server according to the type of the response information, and according to the estimated height, age and gender of the user DID system, characterized in that for generating the content to be output to at least one of the divided areas of the screen of the display device.
A method for controlling a DID device connected to a group server, a DID management server, and an external server, the method comprising:
Receiving image information from a face recognition camera;
determining whether a user's face is detected in the image information;
comparing a face recognition time corresponding to a time for which the user's face is maintained in the image information with a reference value and determining that the user is on standby when the face recognition time exceeds the reference value;
estimating the height, age and gender of the user from the face of the user when it is determined that the user is waiting;
selecting a guide message based on the user's age and gender and outputting the guide message through a display device and a speaker;
receiving voice information;
transmitting the voice information to the external server;
receiving text information corresponding to the voice information;
extracting main keywords from the text information to understand the intention of the user;
generating response information according to the identified intention of the user or receiving response information from at least one of the group server and the external server;
determining a screen division type of the display device by selecting one of the possible screen division types stored in the DID management server according to the type of the generated or received response information;
Content that is output to at least one of the divided areas of the screen of the display device based on the estimated height, age, and gender of the user, but generates screen output information and audio output information based on the generated or received response information and generating the screen output information and the audio output information so that
The response information includes information acquired from a database or sensor of the DID device, information acquired from at least one other DID device connected to the group server, or information acquired from the external server DID device control method.

Images