WO2021210155A1

WO2021210155A1 - Display device, display method, and program

Info

Publication number: WO2021210155A1
Application number: PCT/JP2020/016841
Authority: WO
Inventors: 佐藤　隆; 誉宗巻口; 正典横山; 高田　英明
Original assignee: 日本電信電話株式会社
Priority date: 2020-04-17
Filing date: 2020-04-17
Publication date: 2021-10-21

Abstract

A display device 1 according to the present embodiment has: a plurality of projectors 20-1 to 20-4 that provide different videos for a plurality of individual users on a screen 50 shared by the plurality of users; a plurality of sensors 30-1 to 30-4 that are associated with the projectors 20-1 to 20-4, that obtain operations of the plurality of individual users, and that operate corresponding cursors; and a control unit 103 that, with respect to the plurality of individual projectors 20-1 to 20-4, cause cursors operated by the sensors 30-1 to 30-4 corresponding to the projectors 20-1 to 20-4 to be displayed in the videos provided by the projectors 20-1 to 20-4 in modes different from those of cursors operated by the other sensors 30-1 to 30-4.

Description

Display device, display method, and program

The present invention relates to a display device, a display method, and a program.

Non-Patent Document 1 proposes a system in which a plurality of users instruct coordinates with a laser pointer on a shared screen and perform operations such as drawing a figure. In Non-Patent Document 1, the laser pointer is changed by the user and the coordinates are acquired by the camera.

However, when sharing the same screen with multiple users, there is a problem that it is difficult to grasp one's cursor and the screen display becomes complicated as the number of users increases.

The present invention has been made in view of the above, and an object of the present invention is to provide a display device that can intuitively visually recognize one's own input on a screen shared by a plurality of users.

The display device of one aspect of the present invention is associated with a plurality of projection units that provide different images for each of the plurality of users on a display surface shared by the plurality of users, and the plurality of projection units. A plurality of input units that acquire each user's operation and operate the corresponding cursor, and each of the plurality of projection units are operated by the input unit corresponding to the projection unit in the image provided by the projection unit. It has a control unit that displays the first cursor in a manner different from that of the second cursor group operated by the other input units.

According to the present invention, it is possible to provide a display device that can intuitively visually recognize one's own input on a screen shared by a plurality of users.

FIG. 1 is a diagram for explaining an outline of the display device of the present embodiment. FIG. 2 is a top view showing an example of the configuration of the display device. FIG. 3 is a cross-sectional view showing an example of the configuration of the display device. FIG. 4 is a top view showing an example of the configuration of another display device. FIG. 5 is a diagram showing an example of the configuration of the information processing device. FIG. 6 is a diagram showing an example of the correspondence between the sensor held by the information processing device and the projector. FIG. 7 is a flowchart showing a processing flow of the information processing apparatus. FIG. 8 is a diagram showing an example of a cursor image. FIG. 9 is a diagram showing an example of the hardware configuration of the information processing device.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

As shown in FIG. 1, the display device 1 of the present embodiment is a tubular display device having an opening on the upper surface, and is provided with a circular reflective screen 50 on the inner bottom of the main body. Users 200A to 200C look into the bottom screen 50 through the opening. A plurality of cursors operated by each of the users 200A to 200C are displayed on the screen 50. Users 200A to 200C can operate the cursor displayed on the screen 50 by the pointing hand gesture. In the images viewed by each user 200A to 200C, the cursor operated by the user is highlighted in a manner different from that of the cursor group operated by other users.

As shown in FIGS. 2 and 3, the display device 1 includes a plurality of projectors 20-1 to 20-4 on the upper portion of the housing 40. The projectors 20-1 to 20-4 are fixed to the inside of the housing 40 and project an image on the screen 50. The screen 50 is a reflective optical screen that couples the iris surfaces (planes corresponding to the aperture of the lens) of the projectors 20-1 to 20-4 at positions corresponding to the projection distance and the focal length. Projectors 20-1 to 20-4 can provide different images for each user 200A to 200C to the shared screen 50. Specifically, each user 200A to 200C can see the image projected by the projectors 20-1 to 20-4 arranged opposite to each other. For example, the user 200A can see the image projected by the projector 20-3, and the user 200C facing the user 200A sees the image projected by the projector 20-1 arranged on the opposite side of the projector 20-3. Can be done.

Sensors 30-1 to 30-4 are mounted on each of the projectors 20-1 to 20-4. The sensors 30-1 to 30-4 sense the movements of the users 200A to 200C, and transmit the sensing data to the information processing device 10 described later. The information processing device 10 recognizes the pointing hand gestures of the users 200A to 200C based on the sensing data of the sensors 30-1 to 30-4, and operates the cursors of the users 200A to 200C respectively.

The information processing device 10 associates the projectors 20-1 to 20-4 with the facing sensors 30-1 to 30-4. For example, the projector 20-3 that projects the image viewed by the user 200A and the sensor 30-1 that detects the pointing hand gesture of the user 200A are associated with each other. The information processing device 10 calculates the coordinates of each cursor based on the sensing data of the sensors 30-1 to 30-4. The information processing device 10 emphasizes the cursor operated based on the sensors 30-1 to 30-4 corresponding to the projectors 20-1 to 20-4 in the images supplied to the projectors 20-1 to 20-4, respectively. To display. For example, in the image projected by the projector 20-3, the cursor operated based on the sensing data of the sensor 30-1 is emphasized more than the cursor group operated based on the sensing data of the sensors 30-2 to 30-4. Is displayed. That is, the user 200A sees the image in which the cursor operated by the user 200A is emphasized and displayed.

Other users

200B and 200C also see the image displayed with the cursor operated by each user highlighted.

The mounting positions of the projectors 20-1 to 20-4 and the sensors 30-1 to 30-4 are examples, and are not limited to these. The arrangement position of the screen 50 is not limited to the bottom inside the main body of the display device 1. The screen 50 may be arranged on the wall, and a plurality of projectors may be arranged so as to face the screen 50 and project images at different projection angles. Also in this case, different images can be visually recognized depending on the viewpoint position. It is desirable that the number of projectors and sensors is equal to or greater than the number of users who use them at the same time. When multiple sensors respond to one user, select one representative sensor, associate it with the cursor, and display it in the image that can be seen from that position. Examples of the method of selecting the representative sensor include a method of selecting the sensor having the largest response and a method of selecting the sensor located in the center of the responding sensor group. When one sensor responds to a plurality of users, the cursors of the plurality of users must be displayed at the same time on an image that is commonly seen by the plurality of users.

FIG. 4 shows an example of another display device. In the display device 1 of FIG. 4, 60 projectors 20 are arranged side by side in a circle. Each projector 20 outputs images of the entire circumference of the subject taken at different angles. By superimposing the iris surfaces produced by the adjacent projectors 20 in the left-right direction, linear blending can be optically realized, and a three-dimensional image that smoothly complements the intermediate viewpoint can be presented. The user can see the entire circumference of the three-dimensional image projected on the screen 50 by changing the viewpoint position along the outer circumference of the display device 1. In the display device 1 of FIG. 4, since one user sees an image in which images of a plurality of adjacent projectors 20 are superimposed, the sensor and the projector 20 do not have a one-to-one correspondence with each other. A plurality of projectors may be associated with one sensor.

Further, the display unit including the projectors 20-1 to 20-4 is not limited to the above-mentioned spatial imaging iris surface method, but is an integral type display, a parallax barrier type, or a 3D display using 3D glasses (optical shutter, Anything such as a polarizing plate) that can show different images depending on the viewpoint position is sufficient. For example, the integral method is a method in which a lens array is arranged in front of a display surface and an image to be displayed is switched according to a viewing angle. The parallax barrier method is a method in which barrier layers are arranged in front of a display surface and the image to be displayed is switched according to the viewing angle.

Further, the input unit including the sensors 30-1 to 30-4 is not limited to the above-mentioned hand gesture recognition, and each of the users 200A to 200C can operate the cursor individually, such as mouse, joystick, or laser pointer recognition. Anything is fine.

Next, the information processing device 10 of the present embodiment will be described with reference to FIG. The information processing device 10 is a device that inputs cursor operation information from sensors 30-1 to 30-4 in the display device 1 and supplies images to projectors 20-1 to 20-4.

The information processing device 10 shown in FIG. 5 includes coordinate input units 101-1 to 101-4, image output units 102-1 to 102-4, a control unit 103, and a storage unit 104.

The coordinate input units 101-1 to 101-4 receive the sensing data from the sensors 30-1 to 30-4 and calculate the coordinates of the corresponding cursors. That is, the coordinate input units 101-1 to 101-4 receive the operation information of the cursor from each of the users 200A to 200C, and calculate the coordinates of the cursor based on the operation information.

The image output units 102-1 to 102-4 supply images to be shown to each of the users 200A to 200C to each of the projectors 20-1 to 20-4.

The control unit 103 identifies the correspondence between the projectors 20-1 to 20-4 and the sensors 30-1 to 30-4 with reference to the information stored in the storage unit 104, and the image output units 102-1 to 102- The highlighting cursor is switched every 4. For example, the control unit 103 superimposes the highlighted cursor image on the coordinates of the cursor operated by the corresponding sensors 30-1 to 30-4 on each of the original images supplied to the projectors 20-1 to 20-4. However, the normally displayed cursor image is superimposed on the coordinates of the other cursors. The original video supplied to the projectors 20-1 to 20-4 may be input from the outside, or may be read out stored in the storage unit 104.

As shown in FIG. 6, the storage unit 104 stores the correspondence between the sensors 30-1 to 30-4 and the projectors 20-1 to 20-4. In the example of FIG. 6, in addition to the sensor ID and the projector ID, two types of cursor images A and B are associated with each other. The cursor image A is a highlighting cursor image, and the cursor image B is a normal display cursor image. For example, in the example of FIG. 6, the sensor 30-3 (the sensor ID is 3) corresponds to the projector 20-1 (the projector ID is 1). The image of the highlight 3 is used for the cursor operated based on the sensing data of the sensor 30-3, and the image using the image of the simple 3 is supplied to the projector 20-1 for the other cursors.

Next, the operation of the information processing device 10 will be described with reference to FIG. 7.

In step S11, the information processing device 10 receives sensing data from each of the sensors 30-1 to 30-4, and obtains cursor coordinates corresponding to each of the sensors 30-1 to 30-4. For sensors 30-1 to 30-4 in which users 200A to 200C do not exist, the cursor coordinates may not be displayed (the cursor may not be displayed).

The information processing device 10 executes the following steps S12 and S13 for each of the projectors 20-1 to 20-4.

In step S12, the information processing device 10 displays the emphasized cursor at the cursor coordinates operated by the sensors 30-1 to 30-4 corresponding to the projectors 20-1 to 20-4 to be processed.

In step S13, the information processing apparatus 10 normally displays a cursor group other than the cursor displayed in step S12, and supplies the image on which the cursor is superimposed to the projectors 20-1 to 20-4 to be processed.

For example, when the information processing device 10 holds the table shown in FIG. 6, the information processing device 10 displays the image supplied to the projector 20-1 with the highlight 3 at the cursor coordinates operated by the sensor 30-3. The image is superimposed, and the simple 3 image is superimposed on the other cursor coordinates.

FIG. 8 shows an example of the cursor image 501 for normal display and the cursor images 511 to 515 for highlighting. The cursor image 511 is a enlarged version of the cursor. The cursor image 512 has a different contour or shadow. The cursor image 513 has different colors. You may change the shape of the cursor. The cursor image 514 is obtained by changing the direction of the cursor. The cursor image 515 is annotated near the cursor.

When the cursor image has top and bottom or characters are displayed near the cursor, the cursor image is rotated according to the viewing angle of the users 200A to 200C, that is, the direction of the displayed image, and the cursor is visually recognized by the users 200A to 200C. It may be easier to do.

The cursor that is normally displayed, that is, the cursor that is operated by another user, may be switched between display and non-display. For example, when the user 200A sets his / her own cursor to be hidden, the cursor operated by the user 200A is not displayed in the images of the

other users

200B and 200C.

As described above, the display device 1 of the present embodiment includes a plurality of projectors 20-1 to 20-4 and a projector 20 that provide different images for each of the plurality of users on the screen 50 shared by the plurality of users. It is associated with -1 to 20-4, and has a plurality of sensors 30-1 to 30-4 that acquire the operations of each of the plurality of users and operate the corresponding cursors. The information processing device 10 sets the cursor operated by the sensors 30-1 to 30-4 corresponding to the projectors 20-1 to 20-4 in the image projected by each of the projectors 20-1 to 20-4. It is displayed in a mode different from that of the cursor operated by the sensors 30-1 to 30-4. As a result, in the image projected on the shared screen 50, the cursor operated by each user is displayed in a different manner from the cursor group operated by other users, so that each user can easily move his / her cursor. It becomes possible to grasp.

The information processing device 10 described above includes, for example, a central processing unit (CPU) 901, a memory 902, a storage 903, a communication device 904, an input device 905, and an output device 906, as shown in FIG. A general-purpose computer system including the above can be used. In this computer system, the information processing device 10 is realized by the CPU 901 executing a predetermined program loaded on the memory 902. This program can be recorded on a computer-readable recording medium such as a magnetic disk, an optical disk, or a semiconductor memory, or can be distributed via a network.

1 ... Display device 20-1 to 20-4 ... Projector 30-1 to 30-4 ... Sensor 40 ... Housing 50 ... Screen 10 ... Information processing device 101-1 to 101-4 ... Coordinate input unit 102-1 to 102 -4 ... Image output unit 103 ... Control unit 104 ... Storage unit

Claims

A plurality of projection units that provide different images for each of the plurality of users on a display surface shared by a plurality of users.
A plurality of input units associated with the plurality of projection units, acquiring operations of the plurality of users and operating the corresponding cursors, and
For each of the plurality of projection units, in the image provided by the projection unit, the first cursor operated by the input unit corresponding to the projection unit is different from the second cursor group operated by the other input units. A display device having a control unit for displaying with.
The display device according to claim 1.
The first cursor is a display device that is highlighted by at least one feature with respect to the second cursor group in size, contour, color, shape, cursor orientation, and annotation display.
The display device according to claim 1.
A display device that displays only the first cursor and does not display the second cursor group.
The display device according to any one of claims 1 to 3.
The display surface is a display device that reflects images from the plurality of projection units to form a spatially imaged iris surface.
The display device according to claim 4.
A display device that provides a three-dimensional image to the user by superimposing the spatial imaging iris surface formed by the image from the adjacent projection unit.
A step of providing different images for each of the plurality of users from each of the plurality of projection units on a display surface shared by the plurality of users.
A step in which each of the plurality of input units associated with the plurality of projection units acquires an operation of each of the plurality of users and operates a corresponding cursor.
For each of the plurality of projection units, in the image provided by the projection unit, the first cursor operated by the input unit corresponding to the projection unit is different from the second cursor group operated by the other input units. A display method that has steps to display in.
A program that operates a computer as each part of the display device according to any one of claims 1 to 5.