JP2019153160A - Digital signage device and program - Google Patents

Abstract

To provide a digital signage device and the like for displaying information to meet needs of a person in the vicinity.SOLUTION: A digital signage device comprises: a display part 35 covered by a half mirror; a microphone 36 for acquiring a voice from the vicinity of the display part 35; a text acquisition part for acquiring a text based on the voice acquired by the microphone 36; and an output part for outputting a content based on the text acquired by the text acquisition part to the display part 35. The output part outputs a search result using the text as a key.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a digital signage apparatus and a program.

  There has been proposed a digital signage device in which a display is provided on a temporary enclosure wall installed at a construction site (Patent Document 1). Information such as construction information, regional information and advertisements can be provided to pedestrians and neighbors around the construction site.

JP 2017-227113 A

  However, the digital signage device of Patent Document 1 cannot display information that meets the needs of people near the device.

  In one aspect, an object of the present invention is to provide a digital signage device that displays information according to the needs of people nearby.

  The digital signage device includes a display unit covered with a half mirror, a microphone that acquires voice from the vicinity of the display unit, a text acquisition unit that acquires text based on the voice acquired by the microphone, and the text acquisition unit And an output unit that outputs content based on the acquired text to the display unit.

  In one aspect, it is possible to provide a digital signage device that displays information according to the needs of people nearby.

It is explanatory drawing explaining the outline | summary of a digital signage system. It is explanatory drawing which shows the structure of a digital signage system. It is explanatory drawing explaining the record layout of inquiry DB. It is a flowchart explaining the flow of a process of a program. It is a flowchart explaining the flow of processing of a subroutine for content acquisition. It is explanatory drawing explaining the outline | summary of the digital signage system of Embodiment 2. FIG. It is explanatory drawing explaining the outline | summary of the digital signage system of Embodiment 2. FIG. 10 is a flowchart for explaining a processing flow of a program according to the second embodiment. It is a flowchart explaining the flow of processing of a subroutine for displaying minutes. 6 is a functional block diagram of a digital signage apparatus according to Embodiment 3. FIG. FIG. 10 is an explanatory diagram illustrating a configuration of a digital signage system according to a fourth embodiment.

[Embodiment 1]
FIG. 1 is an explanatory diagram for explaining the outline of the digital signage system 10. The digital signage system 10 includes a display unit 35, a microphone 36, and a speaker 37 attached to a wall. The display unit 35 is, for example, a liquid crystal display panel or an organic EL (electro-luminescence) display panel.

  The surface of the display unit 35 is covered with a translucent plate such as an acrylic plate or a glass plate. The translucent plate is a half mirror having a thin reflective film on one surface. The display surface of the display unit 35 and the translucent plate may be joined by so-called direct bonding. By directly joining the display surface and the translucent plate without interposing an air layer, it is possible to prevent deterioration in image quality due to light reflection between the translucent plate and the display surface.

  The microphone 36 is an array microphone including a plurality of microphones, and can adjust sensitivity and sound detection area as appropriate. The volume of the speaker 37 can also be adjusted as appropriate. The microphone 36 and the speaker 37 may be configured integrally with the display unit 35.

  A plate larger than the display unit 35 is used as the translucent plate that covers the display unit 35, and an exciter is fixed to a portion of the rear surface of the translucent plate that is not covered with the display unit 35 to configure the speaker 37. Also good. A speaker 37 that generates sound can be realized from the entire display unit 35. The microphone 36 and the speaker 37 may be attached to a ceiling or the like.

  The digital signage system 10 according to the present embodiment is installed and used in various places such as a concourse of a station, a bus stop, an inner wall of a public facility, or an outer wall of a building facing a road.

  FIG. 1A shows the digital signage system 10 when not in operation. Light is reflected by the translucent plate covering the display unit 35, and the display unit 35 functions as a mirror. Thereby, it is possible to produce a wide space.

  FIG. 1B shows the digital signage system 10 when operating. FIG. 1B shows a case where a user standing in front of the display unit 35 utters “weather”. The microphone 36 detects the voice of the user, acquires weather information, and displays it on the display unit 35. Further, a sound “Current weather is cloudy” is output from the speaker 37.

  For example, when the user utters “Sapporo weather”, the Sapporo weather is output from the display unit 35 and the speaker 37. When the user utters “Tomorrow's Osaka weather”, the next day's Osaka weather is output from the display unit 35 and the speaker 37. Similarly, various contents such as stock prices, transportation operation information, maps, news, and the like are output from the display unit 35 and the speaker according to the user's utterance.

  As described above, the digital signage system 10 according to the present embodiment provides information according to needs to individual users and becomes a mirror when not in use.

  FIG. 2 is an explanatory diagram showing the configuration of the digital signage system 10. The digital signage system 10 includes a digital signage device 11, a voice recognition server 30, and a field server 42 connected via a network such as the Internet, a public communication line, or a LAN (Local Area Network).

  The digital signage device 11 includes an information processing device 20 in addition to the display unit 35, the microphone 36, and the speaker 37 described above. The information processing apparatus 20 according to the present embodiment is a small computer that can be fixed to the back surface of the display unit 35. The information processing apparatus 20 may be a general-purpose personal computer, a tablet, a smartphone, or the like.

  The information processing device 20 includes a first CPU (Central Processing Unit) 21, a main storage device 22, an auxiliary storage device 23, a communication unit 24, a display I / F (Interface) 25, a microphone I / F 26, a speaker I / F 27, and a bus. Prepare.

  The first CPU 21 is an arithmetic control device that executes a program according to the present embodiment. As the first CPU 21, one or a plurality of CPUs, a multi-core CPU, or the like is used. The first CPU 21 is connected to each hardware unit constituting the information processing apparatus 20 via a bus.

  The main storage device 22 is a storage device such as an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), or a flash memory. The main storage device 22 temporarily stores information necessary during the processing performed by the first CPU 21 and a program being executed by the first CPU 21.

  The auxiliary storage device 23 is a storage device such as an SRAM, a flash memory, a hard disk, or a magnetic tape. The auxiliary storage device 23 stores a program to be executed by the first CPU 21, an inquiry DB 51, and various information necessary for executing the program.

  The inquiry DB 51 may be stored in another storage device connected to the information processing apparatus 20 via a network or the like. Details of the contact DB 51 will be described later. The communication unit 24 is an interface that performs communication with a network.

  The display I / F 25 is an interface that connects the display unit 35 and the bus. The microphone I / F 26 is an interface that connects the microphone 36 and the bus. The speaker I / F 27 is an I / F that connects the speaker 37 and the bus.

  Wireless communication such as a cable or Bluetooth (registered trademark) between the display unit 35 and the display I / F 25, between the microphone 36 and the microphone I / F 26, and between the speaker 37 and the speaker I / F 27. Connected by.

  The voice recognition server 30 is a server that acquires voice data via a network and converts it into text data by voice recognition. The voice recognition server 30 includes a second CPU 31, a main storage device 32, an auxiliary storage device 33, and a communication unit 34.

  The second CPU 31 is an arithmetic control device that executes a program according to the present embodiment. As the second CPU 31, one or a plurality of CPUs, a multi-core CPU, or the like is used. The second CPU 31 is connected to each hardware part constituting the voice recognition server 30 via a bus.

  The main storage device 32 is a storage device such as SRAM, DRAM, or flash memory. The main storage device 32 temporarily stores information necessary during the processing performed by the second CPU 31 and a program being executed by the second CPU 31.

  The auxiliary storage device 33 is a storage device such as an SRAM, a flash memory, a hard disk, or a magnetic tape. The auxiliary storage device 33 stores a program to be executed by the second CPU 31 and various information necessary for executing the program.

  The voice recognition server 30 of this embodiment is an information processing apparatus such as a general-purpose personal computer or server machine. Further, the voice recognition server 30 according to the present embodiment may be a virtual machine that operates on a large computer.

  The field server 42 includes various content providing servers such as a weather server 421, a stock price server 422, a transportation operation information server, a map server, a news server, etc. (not shown). Each field server 42 may be a server opened to general users via the Internet or a server managed by an operator of the digital signage system 10 according to the present embodiment.

  FIG. 3 is an explanatory diagram illustrating the record layout of the inquiry destination DB 51. The inquiry DB 51 is a database that records the field of content output by the digital signage system 10 in association with the acquisition source from which the content is acquired.

  The inquiry DB 51 has a field field, an acquisition source field, and a format field. The field of the content is recorded in the field of field. In the acquisition source field, the URL (Uniform Resource Locator) of the field server 42 from which the content is acquired is recorded. In the format field, a format such as a parameter used when acquiring necessary information from the field server is recorded.

  The inquiry destination DB 51 may record an API (Application Programming Interface) and parameters used when acquiring content.

  FIG. 4 is a flowchart for explaining the flow of processing of the program. Note that at the stage of starting the program, no content is output on the display unit 35, and the screen appears to be a mirror.

  The first CPU 21 acquires audio data through the microphone 36 (step S501). As described above, the microphone 36 is an array microphone including a plurality of microphones. The microphone 36 outputs audio data obtained by A / D (Analog / Digital) conversion of audio to the bus. The microphones 36 can amplify and acquire the voice uttered in a predetermined area in front of the display unit 35 by synthesizing the waveforms of the sounds received by the respective microphones with a predetermined time difference. The microphone 36 may output audio data obtained by A / D converting audio received by each microphone to the bus, and the first CPU may synthesize the audio data with a predetermined time difference.

  The first CPU 21 temporarily stores the acquired audio data in the main storage device 22 or the auxiliary storage device 23. The first CPU 21 analyzes the audio data and performs language determination for determining which language, such as Japanese, English, or Chinese, is used (step S502). Since the method for performing language determination has been conventionally known, detailed description thereof is omitted.

  The first CPU 21 determines whether or not the language determination is successful (step S503). For example, when the volume of the acquired voice is low, or when the voice is a language other than a language such as a car noise, the first CPU 21 determines that the language determination has failed because the language cannot be determined. To do. If it is determined that the language determination has failed (NO in step S503), the first CPU 21 returns to step S501.

  If it is determined that the language determination is successful (YES in step S503), the first CPU 21 recognizes the voice data temporarily stored in the main storage device 22 or the auxiliary storage device 23 and the language type determined in step S502. It transmits to the server 30 (step S504).

  The second CPU 31 receives the transmitted voice data and language (step S601). The second CPU 31 performs language recognition of the transmitted voice data and converts it into text data (step S602). Since language recognition has been widely performed in the past, detailed description is omitted. In addition, by performing language recognition based on the language type acquired together with the voice data, the second CPU 31 can accurately perform language recognition in a short processing time. The second CPU 31 transmits text data corresponding to the voice data received in step S601 to the information processing apparatus 20 (step S603).

  The first CPU 21 receives text data (step S510). The first CPU 21 analyzes the meaning of the text data (step S511). The semantic analysis is performed by a natural language processing technique such as morphological analysis. Since a method for analyzing the meaning of text has been conventionally known, a detailed description thereof will be omitted.

  The first CPU 21 determines whether or not the semantic analysis has succeeded (step S512). For example, if the text received in step S510 does not include the word in the language determined in step S502, the first CPU 21 determines that the semantic analysis has not been successful. If it is determined that the semantic analysis has not been successful (NO in step S512), the first CPU 21 returns to step S501.

  If it is determined that the semantic analysis is successful (YES in step S512), the first CPU 21 starts a content acquisition subroutine (step S513). The content acquisition subroutine is a subroutine for acquiring content output from the display unit 35 and the speaker 37 based on text data. The processing flow of the content acquisition subroutine will be described later.

  The first CPU 21 outputs the acquired content from the display unit 35 and the speaker 37 (step S514). The first CPU 21 determines whether or not to end the process (step S515). For example, when the operation end time set in advance is reached, or when an operation end instruction is received from the administrator via the network, the first CPU 21 determines to end the process.

  If it is determined that the process is to be ended (YES in step S515), the first CPU 21 ends the process. If it is determined not to end the process (NO in step S515), the first CPU 21 returns to step S501.

  FIG. 5 is a flowchart for explaining the flow of processing of the content acquisition subroutine. The content acquisition subroutine is a subroutine for acquiring content output from the display unit 35 and the speaker 37 based on text data.

  The first CPU 21 determines the field of the content to be displayed based on the result of the semantic analysis of the text (step S521). For example, when it is determined that the question is related to the weather, the first CPU 21 determines that the field is weather.

  The first CPU 21 searches the inquiry DB 51 using the determined field as a key, and extracts a record. The first CPU 21 generates a command for acquiring content based on the format field of the extracted record (step S522). Note that the first CPU 21 desirably generates a command for acquiring content using the language used by the user.

  The first CPU 21 transmits a command to the URL recorded in the acquisition source field of the extracted record. The first CPU 21 receives the command, the field server 42 performs search processing based on the command, and transmits the content that is the search result. The first CPU 21 acquires content (step S523). The first CPU 21 ends the process.

  The content acquired by the first CPU 21 uses content in an arbitrary format such as text information such as “current weather is cloudy”, a still image such as a weather map, and a moving image such as a transition of the weather map for the past 24 hours. it can. Specification of what type of content is to be acquired may be included in the command created in step S522.

  The speaker 37 may be a parametric speaker that has a sharp directivity and allows only a person at a specific position to hear a sound. For example, the digital signage system 10 that can be installed in a quiet place such as a library or an art museum can be provided.

  The digital signage device 11 may include a human sensor. For example, when the first CPU 21 detects a user who has stopped in front of the display unit for a certain period of time, the first CPU 21 may output characters and voices such as “Please speak if there is a question” to prompt the user to input. good.

  The display unit 35 may be a touch panel including a touch sensor on the surface. In addition to voice, the digital signage device 11 that allows the user to input by touch operation can be provided.

  According to this embodiment, it is possible to provide a digital signage device that displays information according to the needs of people nearby. According to the present embodiment, it is possible to provide the digital signage device 11 having an effect of making the space look wide when not in use.

  According to the present embodiment, the speech recognition server 30 performs language recognition after determining which language is the speech in step S502 described with reference to FIG. Accordingly, it is possible to prevent the resources of the speech recognition server 30 from being consumed for processing sounds other than languages and for processing in the case of various languages. As described above, it is possible to provide the digital signage system 10 that can be used comfortably by the user with a short time from the user's voice to the display of the content.

  According to the present embodiment, since content is acquired from various field servers 42 via a network, it is possible to display content according to the language used by the user. By using an array microphone as the microphone 36, it is possible to provide the digital signage system 10 that amplifies and acquires the voice of the user in front of the display unit 35 even in a noisy environment.

[Embodiment 2]
The present embodiment relates to a digital signage system 10 having a minutes mode in which a user's voice is converted into text and displayed on a display unit 35. Description of portions common to the first embodiment is omitted.

  6 and 7 are explanatory diagrams for explaining the outline of the digital signage system according to the second embodiment. In FIG. 6, two users are standing in front of the display unit 35 and talking in Japanese. The contents of the conversation are converted into text and displayed on the display unit 35.

  In FIG. 7, the user has a conversation in English. The display unit 35 displays English text. Note that the text displayed on the display unit 35 is transmitted to the user by e-mail or the like after the minutes mode ends.

  FIG. 8 is a flowchart for explaining the processing flow of the program according to the second embodiment. Steps up to step S512 are the same as the process flow of the program according to the first embodiment described with reference to FIG.

  If it is determined that the semantic analysis has not been successful (NO in step S512), the first CPU 21 returns to step S501. If it is determined that the semantic analysis is successful (YES in step S512), the first CPU 21 determines whether or not the text means activation of the minutes mode (step S531).

  If it is determined that it means the start of the minutes mode (YES in step S531), the first CPU 21 starts a minutes display subroutine (step S532). The minutes display subroutine is a subroutine for converting the user's voice into text and displaying it on the display unit 35. The process flow of the minutes display subroutine will be described later.

  If it is determined that it does not mean that the minutes mode is activated (NO in step S531), the first CPU 21 activates a content acquisition subroutine (step S513). The content acquisition subroutine is the same as the subroutine described with reference to FIG.

  The first CPU 21 outputs the acquired content from the display unit 35 and the speaker 37 (step S514). After step S514 or step S532, the first CPU 21 determines whether or not to end the process (step S515).

  If it is determined that the process is to be ended (YES in step S515), the first CPU 21 ends the process. If it is determined not to end the process (NO in step S515), the first CPU 21 returns to step S501.

  FIG. 9 is a flowchart for explaining the processing flow of the minutes display subroutine. The minutes display subroutine is a subroutine for converting the user's voice into text and displaying it on the display unit 35.

  The first CPU 21 acquires audio data through the microphone 36 (step S541). The first CPU 21 temporarily stores the acquired audio data in the main storage device 22 or the auxiliary storage device 23. The first CPU 21 analyzes the audio data and performs language determination for determining which language, such as Japanese, English, Chinese, or the like (step S542).

  The first CPU 21 determines whether or not the language determination is successful (step S543). If it is determined that the language determination has failed (NO in step S543), the first CPU 21 returns to step S541.

  If it is determined that the language determination has succeeded (YES in step S543), the first CPU 21 identifies a speaker of speech (step S544). Here, the identification means identifying which speaker has spoken out of about several speakers in front of the display unit 35.

  For example, the first CPU 21 can identify the speaker by voiceprint analysis or the like. Further, the first CPU 21 may identify the speaker by estimating the position of the speaker based on the output of each microphone constituting the microphone array of the microphone 36.

  The first CPU 21 transmits the voice data temporarily stored in the main storage device 22 or the auxiliary storage device 23 and the language type determined in step S502 to the voice recognition server 30 (step S545).

  The second CPU 31 receives the transmitted voice data and language (step S621). The second CPU 31 performs language recognition of the transmitted voice data and converts it into text data (step S622). The second CPU 31 transmits text data to the information processing apparatus 20 (step S623).

  The first CPU 21 receives text data (step S550). The first CPU 21 temporarily stores the received text data in the main storage device 22 or the auxiliary storage device 23. The first CPU 550 displays text data on the display unit 35 (step S551).

  In step S551, the first CPU distinguishes and displays the directions according to the direction of the balloon as illustrated in FIGS. The first CPU may distinguish and display the speaker in an arbitrary format such as a balloon color, a character color, or a font instead of the balloon direction. The first CPU 21 may display text data without distinguishing speakers.

  The first CPU 21 determines whether or not to end the process (step S555). For example, if no audio data is detected for a predetermined time, the first CPU 21 determines to end the process. When it determines with not complete | finishing a process (it is NO at step S555), 1st CPU21 returns to step S541.

  If it is determined that the process is to be ended (YES in step S555), the first CPU 21 ends the process. Note that the first CPU 21 may transmit the text stored in the main storage device 22 or the auxiliary storage device 23 to an e-mail address or the like designated by the user before finishing the process. The first CPU 21 may store the text stored in the main storage device 22 or the auxiliary storage device 23 in a predetermined server before displaying the process, and display the access URL on the display unit 35.

  According to the present embodiment, it is possible to provide a digital signage system 10 that automatically creates minutes. The user may use the minutes mode alone. A digital signage system 10 that performs so-called dictation can be provided.

  In the minutes display subroutine described with reference to FIG. 9, the text data received in step S550 may be machine translated and then displayed in step S551. For example, by displaying a speech by a Japanese speaker in English as shown in FIG. 7 and a speech by an English speaker in Japanese as shown in FIG. 6, communication between users who use different languages can be performed. The digital signage system 10 that supports the application can be provided.

[Embodiment 3]
FIG. 10 is a functional block diagram of the digital signage apparatus 11 according to the third embodiment. The digital signage apparatus 11 includes a text acquisition unit 81 and an output unit 82 in addition to the microphone 36 and the display unit 35 described above.

  The display unit 35 is covered with a half mirror. The microphone 36 acquires sound from the vicinity of the display unit 35. The text acquisition unit 81 acquires text based on the voice acquired by the microphone 36. The output unit 82 outputs content based on the text acquired by the text acquisition unit 81 to the display unit 35.

[Embodiment 4]
The present embodiment relates to a mode for realizing the digital signage system 10 of the present embodiment by operating a general-purpose computer and a program 97 in combination. FIG. 11 is an explanatory diagram illustrating a configuration of the digital signage system 10 according to the fourth embodiment. Note that description of portions common to the first embodiment is omitted.

  The digital signage system 10 of this embodiment includes a digital signage device 11, a voice recognition server 30, and a field server 42 connected via a network. The digital signage apparatus 11 includes a computer 90, a display unit 35, a microphone 36, and a speaker 37.

  The computer 90 includes a first CPU 21, a main storage device 22, an auxiliary storage device 23, a communication unit 24, a display I / F 25, a microphone I / F 26, a speaker I / F 27, a reading unit 95, and a bus.

  The program 97 is recorded on a portable recording medium 96. The first CPU 21 reads the program 97 via the reading unit 95 and stores it in the auxiliary storage device 23. The first CPU 21 may read the program 97 stored in the semiconductor memory 98 such as a flash memory mounted on the computer 90. Furthermore, the first CPU 21 may download the program 97 from another server computer (not shown) connected via the communication unit 24 and a network (not shown) and store the program 97 in the auxiliary storage device 23.

  The program 97 is installed as a control program for the computer 90, loaded into the main storage device 22, and executed. Thereby, the computer 90 functions as the information processing apparatus 20 described above.

The technical features (components) described in each embodiment can be combined with each other, and new technical features can be formed by combining them.
The embodiments disclosed herein are illustrative in all respects and should not be considered as restrictive. The scope of the present invention is defined not by the above-described meaning but by the scope of the claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of the claims.

DESCRIPTION OF SYMBOLS 10 Digital signage system 11 Digital signage apparatus 20 Information processing apparatus 21 1st CPU
22 Main storage device 23 Auxiliary storage device 24 Communication unit 25 Display I / F
26 Microphone I / F
27 Speaker I / F
30 Voice recognition server 31 2nd CPU
32 Main storage device 33 Auxiliary storage device 34 Communication unit 35 Display unit 36 Microphone 37 Speaker 42 Field server 421 Weather server 422 Stock price server 51 Inquiry DB
81 Text Acquisition Unit 82 Output Unit 90 Computer 95 Reading Unit 96 Portable Recording Medium 97 Program 98 Semiconductor Memory

Claims (4)

A display unit covered with a half mirror;
A microphone for acquiring sound from the vicinity of the display unit;
A text acquisition unit for acquiring text based on the voice acquired by the microphone;
A digital signage apparatus comprising: an output unit that outputs content based on the text acquired by the text acquisition unit to the display unit. The digital signage apparatus according to claim 1, wherein the output unit outputs a search result using the text as a key. A language determination unit for determining a language based on the voice acquired by the microphone;
The digital signage apparatus according to claim 1, wherein the text acquisition unit acquires the text corresponding to the language specified by the language determination unit. Get audio through the microphone,
Get text based on the acquired voice,
A program that causes a computer to execute a process of outputting content based on the acquired text to a display unit that is disposed in the vicinity of the microphone and covered with a half mirror.

Images