JP2016086215A - Information output device, information output method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a user to recognize the existence or position of invisible operation means.SOLUTION: In a digital signage device 1, a control section 23 detects the position of a user based on an image captured by an imaging section 34, and determines whether or not operation buttons 32a-32d, invisible for the user because any one of them hides a screen 27 having a three-dimensional part 271b, exists based on the position of the user thus detected. When a determination is made that an operation button invisible for the user exists, a guidance of the operation button invisible for the user is outputted on the screen 27.SELECTED DRAWING: Figure 7

Description

  The present invention relates to an information output device, an information output method, and a program.

  2. Description of the Related Art Conventionally, an image output device connected to an image supply device, a reflection member, and a screen, output light that projects content from the image output device is reflected by the reflection member, and output light reflected by the reflection member A video output device-equipped device that can enhance the impression to the viewer (user) by projecting the image onto a screen formed in the shape of the outline of the content is known (for example, see Patent Document 1). .

JP 2011-150221 A

  By the way, in order to give viewers a more realistic impression, a technique for projecting content onto a three-dimensionally formed screen has been studied. However, for example, when a three-dimensional screen is used, there is a problem that operation means such as operation buttons are hidden behind the screen depending on the position of the user and cannot be viewed.

  2 is a view of the screen 27 viewed from the front, FIG. 3A is a view of the screen 27 viewed from above, and FIG. 3B is a view of the screen 27 viewed obliquely from above. For example, as shown in FIG. 3A, when the screen 27 is viewed from above, the user cannot visually recognize the operation button 32a among the four operation buttons 32a to 32d. As shown in FIG. 3B, when the screen is viewed obliquely from above, the user cannot visually recognize the operation buttons 32a to 32c among the four operation buttons 32a to 32d.

  An object of the present invention is to make it possible to recognize the presence and position of an operation means that cannot be visually recognized by a user.

In order to solve the above-described problem, an information output device according to a first aspect of the present invention provides:
Detecting means for detecting the position of the user;
A determination unit that determines whether or not there is an operation unit that the user cannot visually recognize based on the position of the user detected by the detection unit;
An output means for outputting guidance of the operating means that the user cannot visually recognize when it is determined by the determining means that there is an operating means that the user cannot visually recognize;
Is provided.

  According to the present invention, it is possible to recognize the presence and position of an operation unit that cannot be visually recognized by the user.

It is a block diagram which shows the functional structure of the digital signage apparatus in this embodiment. It is a front view which shows schematic structure of the digital signage apparatus in this embodiment. (A) is the figure which looked at the digital signage apparatus from upper direction, (b) is the figure which looked at the digital signage apparatus from diagonally upward. It is a figure which shows the example of data storage of the operation button view permission rejection determination table. It is a figure for demonstrating position X, Y of eyes, and the distance D between eyes. It is the figure which showed typically the cross section which looked at the screen from upper direction. It is a flowchart which shows the content output process performed by the control part of FIG. It is a figure which shows the example which displayed the display which guides the position of an operation button on the screen.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following embodiments, a case where the digital signage apparatus 1 is applied as an information output apparatus according to the present invention will be described as an example. The present invention is not limited to the illustrated example.

[Configuration of Digital Signage Device 1]
FIG. 1 is a block diagram showing a main control configuration of the digital signage apparatus 1. FIG. 2 is a front view showing a schematic configuration of the digital signage apparatus 1. The digital signage apparatus 1 is installed in a store or the like, for example, and outputs contents such as product descriptions to the user.

  As shown in FIGS. 1 and 2, the digital signage apparatus 1 includes a projection unit 21 built in the housing 20 and a user interface unit 22 provided on the front surface of the housing 20.

First, the projection unit 21 will be described.
The projection unit 21 includes a control unit 23, a projector 24, a storage unit 25, and a communication unit 26. The projector 24, the storage unit 25, and the communication unit 26 are connected to the control unit 23 as shown in FIG.

  The control unit 23 includes a CPU (Central Processing Unit) that executes various programs stored in the program storage unit 251 of the storage unit 25 to perform predetermined calculations and control of each unit, and a memory that is a work area when the program is executed (Both not shown). The control unit 23 functions as a detection unit and a determination unit by executing a content output process to be described later in cooperation with a program stored in the program storage unit 251 of the storage unit 25.

  The projector 24 is a projection device that converts the image data output from the control unit 23 into projection light and irradiates the screen 27 with the projection light. For example, the projector 24 performs a display operation by individually turning on / off each tilt angle of a plurality of micromirrors arranged in an array (in the case of XGA, horizontal 1024 pixels × vertical 768 pixels) at high speed. A DLP (Digital Light Processing) (registered trademark) projector using a DMD (digital micromirror device), which is a display element that forms a light image with the reflected light, is applicable.

  The storage unit 25 is configured by an HDD (Hard Disk Drive), a nonvolatile semiconductor memory, or the like. The storage unit 25 is provided with a program storage unit 251 as shown in FIG. The program storage unit 251 stores a system program executed by the control unit 23, a processing program for executing various processes including a content output process described later, data necessary for executing these programs, and the like. Yes.

  The storage unit 25 includes a content storage unit 252, a screen type storage unit 253, and a table storage unit 254.

  The content storage unit 252 stores content data for outputting content such as product descriptions to the screen 27 or the like (see FIG. 2). The content data is composed of moving image data composed of a plurality of frame images and audio data corresponding to each frame image. The content data is stored in association with an ID (1 to 4 in this case) for identifying the content.

  The screen type storage unit 253 stores information indicating the type of the screen 27 attached to the housing 20. For example, the type of the screen 27 is set by the operator using the operation unit 32 or the like when the screen 27 is attached.

The table storage unit 254 stores an operation button view authorization rejection determination table 254a. The operation button view authorization rejection determination table 254a indicates whether or not each of the operation buttons 32a to 32d is visible from the type of the screen 27 currently attached to the housing 20 and the position of the user in the content output process described later. It is the table provided in order to determine.
Here, as illustrated in FIGS. 3A and 3B, the operation buttons 32 a to 32 d of the operation unit 32 may or may not be visible depending on the position of the user (the position of the user's viewpoint). This is because it is hidden by the three-dimensional screen 27, and whether or not the user can visually recognize it depends on the shape of the screen 27 even if the position of the user is the same.

  FIG. 4 shows a data storage example of the operation button view authorization rejection determination table 254a. The operation button view approval / disapproval determination table 254a includes information indicating the type of the screen 27 and the position of the user (here, the user's eye position X, eye position Y, and eye interval in the captured image captured by the imaging unit 34). In correspondence with the combination with the distance D), a flag indicating whether or not the user viewing the screen from that position can visually recognize each of the operation buttons 32a to 32d of the operation unit 32 (0 = visible), 1 = not visible ). In the operation button view approval / disapproval determination table 254a, the operation buttons 32a to 32d are indicated as buttons a to d, respectively. The information stored in the operation button view approval / disapproval determination table 254a is obtained by storing results obtained in advance by experiments.

  Here, with reference to FIG. 5, the positions X and Y of the eyes and the distance D between the eyes will be described. In FIG. 5, as an example, a case where the captured image obtained by the imaging unit 34 is 640 × 480 pixels is illustrated. As shown in FIG. 3, the upper left of the coordinates of the captured image acquired by the imaging unit 34 is defined as x = 0, y = 0, the lower right is defined as x = 639, and y = 479. The eye position X is an average of the x-coordinates of the two eyes (for example, the center of the eyes) of the user recognized from the captured image, and the range of values is 0-639. The eye position Y is the average of the y coordinates of the two eyes of the user recognized from the captured image, and the range of values is 0-479. The distance between eyes D is the distance between the two eyes of the user recognized from the captured image, and the range of values is 0-320.

  The approximate direction of the user position (the position of the user's viewpoint) can be specified by the eye positions X and Y. Moreover, the approximate distance from the digital signage apparatus 1 of a user's position can be specified by the distance D between eyes. The distance between eyes D becomes smaller as the user is away from the digital signage apparatus 1, and the distance between eyes D becomes larger as the user is closer to the digital signage apparatus 1.

  The communication unit 26 includes a modem, a router, a network card, and the like, and performs communication with an external device connected to the communication network.

Next, the user interface unit 22 will be described.
As shown in FIG. 2, the user interface unit 22 includes a screen 27, an operation unit 32, an audio output unit 33, an imaging unit 34, and an LCD (Liquid Crystal Display) 35. 2 and 3 show an example in which a human image is projected on the screen 27. FIG.

  The screen 27 is attached to the housing 20 in a replaceable manner, and can be replaced with a screen 27 having an appropriate shape from a plurality of types of screens having different shapes according to the content.

FIG. 6 is a diagram schematically showing a cross section of the screen 27 as viewed from above.
The screen 27 includes a diffuse transmission part 271 formed of, for example, an acrylic plate, and a Fresnel lens 272 disposed on the back side of the diffusion transmission part 271.

  The diffuse transmission portion 271 includes a flat plate portion 271a formed in a flat plate shape and a three-dimensional portion 271b formed in a three-dimensional manner so as to protrude forward. In the present embodiment, the content projected on the screen 27 is for a person to explain a product and the like, and the flat plate portion 271a is, for example, a posting unit on which various information related to the product is displayed. In order to ensure, it is formed in a flat plate shape. The three-dimensional part 271b has a hollow shape with an open back side. The three-dimensional part 271b is a part on which projection light forming a human image is projected. If pursuing reality, the three-dimensional part 271b is preferably formed in a three-dimensional shape that is closer to a person. Furthermore, in the three-dimensional part 271b, it is preferable that the surface of at least some of the parts is matted. By the mat processing, it is possible to prevent the external light from being reflected on the surface of the diffuse transmission part 271 and the image from becoming difficult to see.

  The Fresnel lens 272 is arranged on the back side of the diffusing and transmitting unit 271 and refracts the projection light projected from the projector 24 at a predetermined angle and faces the screen 27 (a virtual facing position of the user). ) And is guided toward the diffuse transmission part 271. Thereby, an image with less distortion can be displayed on the screen 27 having the three-dimensional part 271b without correcting the original image data.

  The operation unit 32 includes, for example, operation buttons 32 a to 32 d, detects a press signal of the operation button, and outputs it to the control unit 23.

  The audio output unit 33 converts the audio data instructed by the control unit 23 into an audio signal and outputs the audio signal from the speaker.

The imaging unit 34 captures an image of a space in front of the screen 27 and generates a captured image. Although not shown, the imaging unit 34 includes a camera including an optical system and an imaging element, and an imaging control unit that controls the camera.
The optical system of the camera is directed in a direction in which a person (user) in front of the screen 27 can be photographed. The imaging element is, for example, an image sensor such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS). The imaging element converts the optical image that has passed through the optical system into a two-dimensional image signal.

  The LCD 35 displays the functions and the like of the operation buttons 32a to 32d in association with the operation buttons 32a to 32d. In the present embodiment, the operation buttons 32a to 32d have a content switching function, and the contents of the switching destination content and the like are displayed in the area of the LCD 35 corresponding to each operation button 32a to 32d.

  The operation unit 32, the audio output unit 33, the imaging unit 34, and the LCD 35 are connected to the control unit 23 as shown in FIG.

[Operation of Digital Signage Device 1]
Next, the operation of the digital signage apparatus 1 will be described.

  FIG. 7 shows a flowchart of content output processing executed in the digital signage apparatus 1. The content output process is executed in cooperation with the program stored in the control unit 23 and the program storage unit 251 when the power of the digital signage apparatus 1 is turned on.

  In the content output process, first, the control unit 23 acquires information on the type of the screen 27 attached to the housing 20 from the screen type storage unit 253 (step S1).

Next, the control unit 23 causes the screen 27 and the audio output unit 33 to output content (step S2).
Specifically, the control unit 23 reads the content data of the content output from the content storage unit 252, sequentially outputs the frame images to the projector 24, and causes the projector 24 to project it onto the screen 27. Also, the audio data of the read content data is output to the audio output unit 33, and the audio of the content is output. In addition, the functions of the operation buttons 32a to 32d are displayed on the LCD 35. The content is, for example, a person image projected on the three-dimensional part 271b of the screen 27 that describes the product.

  The content is output in the order of the IDs associated with the content data described above, and when the last content is output, it returns to the first content, but any one of the operation buttons A to D is pressed. Then, the content corresponding to the pressed button is switched. Thereafter, the contents are output in the order of IDs from the switched contents.

  Next, the control unit 23 determines whether or not a user has been detected (step S3). Specifically, the control unit 23 causes the imaging unit 34 to capture an image to acquire a captured image, performs face recognition processing on the acquired captured image, and when a face is recognized from the captured image, the user performs Judge that it was detected. The face recognition process can be performed using a known image processing technique based on features such as eyes, nose, and mouth.

  When it is determined that the user has not been detected (step S3; NO), the control unit 23 returns to step S2, causes the image capturing unit 34 to perform image capturing at predetermined time intervals during output of the content, and obtains a captured image obtained. The face recognition process is performed.

  When it is determined that the user has been detected (step S3; YES), the control unit 23 detects the position of the user by acquiring the user's eye positions X and Y and the inter-eye distance D (step S4). Specifically, the eyes are extracted from the face area in the captured image acquired by the imaging unit 34, and the average of the positions of the two eyes is calculated and acquired as the eye positions X and Y. Further, the distance between the centers of the two eyes is measured and acquired as an inter-eye distance D.

  Next, the control unit 23 refers to the operation button view authorization rejection determination table 254a stored in the table storage unit 254, and determines the type of the screen 27 acquired in step S1 and the eye position X, acquired in step S4. Based on Y and the distance D between eyes, it is determined whether or not there is an operation button that the user cannot visually recognize (step S5).

When it is determined that there is no operation button that cannot be visually recognized by the user (step S5; NO), the control unit 23 returns to step S2.
When it is determined that there is an operation button that cannot be visually recognized by the user (step S5; YES), the control unit 23 overwrites the content data with a display that guides the position of the operation button that cannot be visually recognized by the user (step S6). Return.

  The control unit 23 repeatedly executes the processing from step S2 to step S6 until the power of the digital signage device 1 is turned off.

FIG. 8 shows an example of the display on the screen 27 when returning to step S2 after overwriting the content data with a display for guiding the position of the operation button that the user cannot visually recognize in step S6.
As shown in FIG. 8, an arrow or the like for guiding the position of the operation button that the user cannot visually recognize is displayed on the screen 27 so as to overlap the content. Therefore, the user can recognize the presence and position of an operation button that cannot be visually recognized.

  As described above, according to the digital signage apparatus 1, the control unit 23 detects the position of the user, and any one of the operation buttons 32a to 32d has the three-dimensional unit 271b based on the detected position of the user. It is determined whether or not there are operation buttons 32 a to 32 b that cannot be visually recognized by the user by hiding on the screen 27. When it is determined that there is an operation button that cannot be visually recognized by the user, the operation button guidance that the user cannot visually recognize is output to the screen 27.

  Accordingly, it is possible to recognize the presence and position of the operation means that cannot be visually recognized by the user.

  Further, for example, by determining whether or not there are operation buttons 32a to 32d that cannot be visually recognized by the user based on the position of the user and the type of the installed screen, the screen 27 has a different shape. It becomes possible to determine the operation buttons 32a to 32d that the user cannot visually recognize even after replacement.

  Also, for example, face recognition processing is performed on the captured image acquired by the imaging unit 34, eyes are extracted from the recognized face image, and the position of the user's eyes and the distance between the eyes are detected, thereby detecting the user's position. By doing so, it becomes possible to easily detect the position of the user.

  Further, by displaying and displaying on the screen 27 the guidance indicating the position of the operation button that cannot be visually recognized by the user, the presence and position of the operation button that cannot be visually recognized by the user can be easily recognized.

  In addition, the description content in the said embodiment is a suitable example of the digital signage apparatus which concerns on this invention, and is not limited to this.

  For example, in the above embodiment, the case where the position of the operation button that cannot be visually recognized by the user is guided by an arrow or the like has been described as an example. It may be displayed.

  Further, in the above-described embodiment, the guidance of the position of the operation button is performed by display, but it may be performed by voice output. For example, a message such as “Two operation buttons are hidden under the three-dimensional screen” may be output by voice.

  In the above embodiment, the case where the eye positions X and Y and the inter-eye distance D are used as parameters for specifying the position of the user has been described as an example. However, the present invention is not limited to this. For example, the user's position may be specified by adding the line-of-sight direction or the like.

  Further, in the above-described embodiment, the case where there is an operation unit that cannot be visually recognized by the user due to the operation buttons 32a to 32d being hidden by the solid portion 271b of the screen 27 has been described as an example. The present invention is not limited to 271b and can be applied to a case where there is an operation unit that cannot be visually recognized by the user due to some convex portion in the casing 20 of the digital signage apparatus 1.

  Moreover, in the said embodiment, although demonstrated taking the case where the solid part 271b of the screen 27 was a humanoid, the shape of the solid part 271b is not limited to this.

Further, when the faces of a plurality of users are recognized by the face recognition process, information on operation means that cannot be visually recognized by each user may be displayed.
Further, the position of the user may be detected only by face recognition processing.
Further, when the recognized user is far away (for example, when the distance D between eyes is equal to or smaller than a predetermined threshold), even if the invisible operation means exists, the information on the operation means may not be displayed. Good.

  In addition, the detailed configuration and detailed operation of the digital signage apparatus can be changed as appropriate without departing from the spirit of the invention.

Although several embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalents thereof.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
Detecting means for detecting the position of the user;
A determination unit that determines whether or not there is an operation unit that the user cannot visually recognize based on the position of the user detected by the detection unit;
An output means for outputting guidance of the operating means that the user cannot visually recognize when it is determined by the determining means that there is an operating means that the user cannot visually recognize;
An information output device comprising:
<Claim 2>
A screen having a three-dimensional part;
The information output device according to claim 1, wherein the determination unit determines whether or not there is an operation unit that cannot be visually recognized by the user by being hidden by the screen based on the position of the user detected by the detection unit. .
<Claim 3>
Multiple types of screens with different shapes can be exchanged,
The information according to claim 2, wherein the determination unit determines whether there is an operation unit that the user cannot visually recognize based on the position of the user detected by the detection unit and the type of screen installed. Output device.
<Claim 4>
The information output device according to any one of claims 1 to 3, wherein the detection unit detects a position of an eye of the user and a distance between eyes as the position of the user.
<Claim 5>
The detection unit includes an imaging unit that acquires a captured image of the user, performs face recognition processing on the captured image acquired by the imaging unit, extracts eyes from the recognized face image, and positions the eyes of the user The information output device according to claim 4, wherein the distance between eyes is detected.
<Claim 6>
The information output device according to any one of claims 2 to 5, wherein the output unit displays and outputs a guide indicating the position of the operation unit that the user cannot visually recognize on the screen.
<Claim 7>
Detecting the position of the user;
Determining whether there is an operation means that the user cannot visually recognize based on the detected position of the user;
When it is determined that there is an operation means that the user cannot visually recognize, and a guide for the operation means that the user cannot visually recognize;
Information output method.
<Claim 8>
Computer
Detecting means for detecting the position of the user;
Determination means for determining whether or not there is an operation means that the user cannot visually recognize based on the position of the user detected by the detection means;
An output means for outputting guidance of the operating means that cannot be visually recognized by the user when the determining means determines that there is an operating means that the user cannot visually recognize;
Program to function as.

DESCRIPTION OF SYMBOLS 1 Digital signage apparatus 20 Case 21 Projection part 22 User interface part 23 Control part 24 Projector 25 Storage part 251 Program storage part 252 Content storage part 253 Screen seed | species storage part 254 Table storage part 26 Communication part 27 Screen 271 Diffuse transmission part 271a Flat plate Unit 271b three-dimensional unit 272 Fresnel lens 32 operation unit 33 audio output unit 34 imaging unit 35 LCD

Claims (8)

Detecting means for detecting the position of the user;
A determination unit that determines whether or not there is an operation unit that the user cannot visually recognize based on the position of the user detected by the detection unit;
An output means for outputting guidance of the operating means that the user cannot visually recognize when it is determined by the determining means that there is an operating means that the user cannot visually recognize;
An information output device comprising: A screen having a three-dimensional part;
The information output device according to claim 1, wherein the determination unit determines whether or not there is an operation unit that cannot be visually recognized by the user by being hidden by the screen based on the position of the user detected by the detection unit. . Multiple types of screens with different shapes can be exchanged,
The information according to claim 2, wherein the determination unit determines whether there is an operation unit that the user cannot visually recognize based on the position of the user detected by the detection unit and the type of screen installed. Output device.   The information output device according to any one of claims 1 to 3, wherein the detection unit detects a position of an eye of the user and a distance between eyes as the position of the user.   The detection unit includes an imaging unit that acquires a captured image of the user, performs face recognition processing on the captured image acquired by the imaging unit, extracts eyes from the recognized face image, and positions the eyes of the user The information output device according to claim 4, wherein the distance between eyes is detected.   The information output device according to any one of claims 2 to 5, wherein the output unit displays and outputs a guide indicating the position of the operation unit that the user cannot visually recognize on the screen. Detecting the position of the user;
Determining whether there is an operation means that the user cannot visually recognize based on the detected position of the user;
When it is determined that there is an operation means that the user cannot visually recognize, and a guide for the operation means that the user cannot visually recognize;
Information output method. Computer
Detecting means for detecting the position of the user;
Determination means for determining whether or not there is an operation means that the user cannot visually recognize based on the position of the user detected by the detection means;
An output means for outputting guidance of the operating means that cannot be visually recognized by the user when the determining means determines that there is an operating means that the user cannot visually recognize;
Program to function as.

Images