WO2014208169A1

WO2014208169A1 - Information processing device, control method, program, and recording medium

Info

Publication number: WO2014208169A1
Application number: PCT/JP2014/059531
Authority: WO
Inventors: 智也成田; 卓井上; 丈博萩原
Original assignee: ソニー株式会社
Priority date: 2013-06-26
Filing date: 2014-03-31
Publication date: 2014-12-31

Abstract

[Problem] To provide an information processing device, control method, program, and recording medium whereby information associated with an operator operating a laser pointer is clearly shown in correspondence with an indicating point displayed on a remote side, allowing a smooth teleconference connecting a plurality of distant locations. [Solution] An information processing device provided with a recognition unit that recognizes the position of an illuminated point due to a laser pointer on a projected image, an acquisition unit that acquires information associated with the operator of said laser pointer, and a first transmission unit that transmits the position of the illuminated point and the information associated with the operator to a remote side in order to display the following, in addition to the aforementioned projected image, on a display device on the remote side: a remote illuminated point at a coordinate position corresponding to the abovementioned illuminated point; and the information associated with the abovementioned operator.

Description

Information processing apparatus, control method, program, and storage medium

The present disclosure relates to an information processing apparatus, a control method, a program, and a storage medium.

Conventionally, video conferencing (video conferencing) systems have been used that connect two or more remote locations to perform interactive video and audio conferences. In other words, there was a situation where all members' faces could not be shown when trying to show the faces of participants in the conference.

Also, in the video conference system, there is a method of using a method in which a specific operator indicates with a laser pointer on a screen projected by a projector and transmits it to a remote conference room. .

In addition to the video conference as described above, an electronic conference system that is performed on a network by an information processing device and a plurality of terminal devices that can perform data communication with the information processing has recently become widespread. With regard to such an electronic conference system, for example, Patent Document 1 below discloses an optimum layout attempt according to an operation state.

Moreover, in the following patent document 2, in a plurality of video conference terminal devices displaying the same explanatory document data, coordinate information of the instructor's designated position is transmitted to the viewer side, and the viewer side video conference terminal device A communication conference system in which a certain mark is superimposed and displayed at the same position is disclosed.

JP 2011-134124 A JP 07-162826 A

However, in a video conferencing system, if the screen and the operator who is far from the screen are projected, the screen will be small and the projected content will be difficult to see. On the other hand, if only the screen is projected, the face of the explainer will not be captured. I didn't know who was explaining how.

Moreover, in the said patent document 1, it is not mentioned at all about transmitting the state of an operator, and in the said patent document 2, although an operator's image | video is transmitted, from a long distance with a laser pointer etc. No mention is made of the transmission of information about the operator when instructed.

Therefore, in the present disclosure, it is possible to smoothly hold a communication conference connecting a plurality of remote locations by clearly indicating the information of the operator operating the laser pointer in association with the indication point displayed on the remote side. A new and improved information processing apparatus, control method, program, and storage medium are proposed.

According to the present disclosure, the recognition unit that recognizes the position of the irradiation point by the laser pointer with respect to the local projection image, the acquisition unit that acquires the information of the operator of the laser pointer, and the display device installed on the remote side In order to display the information of the remote irradiation point located at the coordinate position corresponding to the irradiation point and the operator together with the projection image, the position of the irradiation point and the information of the operator are transmitted to the remote side. An information processing apparatus including a first transmission unit is proposed.

According to the present disclosure, in the step of recognizing the position of the irradiation point by the laser pointer with respect to the projection image on the local side, the step of acquiring information on the operator of the laser pointer, and the display device installed on the remote side, A step of transmitting the position of the irradiation point and the information of the operator to the remote side in order to display the remote irradiation point located at the coordinate position corresponding to the irradiation point and the information of the operator together with the projection image. And a control method is proposed.

According to the present disclosure, the computer is installed on the remote side, a recognition unit that recognizes the position of the irradiation point by the laser pointer on the local projection image, an acquisition unit that acquires information on the operator of the laser pointer, In order to display the remote irradiation point located at the coordinate position corresponding to the irradiation point and the operator information together with the projection image, the position of the irradiation point and the operator information are displayed on the display device. A program for functioning as a first transmission unit that transmits to the side is proposed.

According to the present disclosure, the computer is installed on the remote side, a recognition unit that recognizes the position of the irradiation point by the laser pointer on the local projection image, an acquisition unit that acquires information on the operator of the laser pointer, In order to display the remote irradiation point located at the coordinate position corresponding to the irradiation point and the operator information together with the projection image, the position of the irradiation point and the operator information are displayed on the display device. A storage medium storing a program for causing a first transmission section to transmit to the first transmission section and a program to function as the first transmission section is proposed.

As described above, according to the present disclosure, the information of the operator who operates the laser pointer is clearly associated with the indication point displayed on the remote side, thereby facilitating a communication conference connecting a plurality of remote locations. Can be performed.

It is a figure for demonstrating the outline | summary of the remote cooperation system by one Embodiment of this indication. It is a figure for demonstrating the case where the remote place cooperation system by one Embodiment of this indication is applied to the communication conference which connects three remote places. It is a block diagram which shows the internal structural example of each information processing apparatus which forms the remote cooperation system by 1st Embodiment. It is a figure for demonstrating the operation | movement process of the remote place cooperation system by 1st Embodiment. It is a figure for demonstrating the example of a display of the operation information displayed on the image projected on the screen of a remote side. It is a flowchart which shows the operation | movement process in the case of performing a silhouette display on the remote side in the remote cooperation system by 1st Embodiment. It is a figure for demonstrating the example of a superimposed display of a silhouette image. It is a block diagram which shows the internal structural example of each apparatus which forms the remote place cooperation system by 2nd Embodiment. It is a figure for demonstrating the other system configuration example of the remote place cooperation system by this indication. It is a block diagram which shows the internal structural example of each apparatus which forms the remote place cooperation system by 3rd Embodiment. It is a block diagram which shows the internal structural example of each apparatus which forms the remote place cooperation system by 4th Embodiment. It is a block diagram which shows the internal structural example of each apparatus which forms the remote place cooperation system by 5th Embodiment. It is a block diagram which shows the internal structural example of each apparatus which forms the remote place cooperation system by 6th Embodiment. It is a block diagram which shows the internal structural example of each apparatus which forms the remote place cooperation system by 7th Embodiment.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
1. 1. Overview of remote cooperation system according to an embodiment of the present disclosure Embodiments 2-1. First embodiment 2-2. Second embodiment 2-3. Third Embodiment 2-4. Fourth embodiment 2-5. Fifth embodiment 2-6. Sixth embodiment 2-7. 7. Seventh embodiment Summary

<< 1. Overview of remote cooperation system according to an embodiment of the present disclosure >>
First, an overview of a remote cooperation system according to an embodiment of the present disclosure will be described with reference to FIG. As shown in FIG. 1, a remote location cooperation system according to an embodiment of the present disclosure is formed by a plurality of

information processing devices

1A and 1B installed in a plurality of remote locations, and each information processing device (communication device) 1A, 1B is connected via a network.

The information processing apparatus 1A installed in the ROOM • A is connected to each device installed in the ROOM • A by wire / wireless. Specifically, the information processing apparatus 1A includes a projector 3A, a content display apparatus 4A, a camera 5A that captures an image projected on the screen S1, a camera 6A that captures an operator 7A of the laser pointer 2A, a microphone (hereinafter referred to as a microphone). It is connected to 8A and the speaker 9A. The system configuration shown in FIG. 1 is an example. For example, the microphone 8A and the camera 6A may be integrated, the content display device 4A and the information processing device 1A may be integrated, or the camera 5A may be built in the projector 3A.

The information processing apparatus 1A acquires content from the content display apparatus 4A and transmits projection image data (display signal) to the projector 3A. The contents are, for example, charts, sentences, other various graphic images, maps, websites, 3D objects, and the like, and are also referred to as image data for projection hereinafter. The content display device 4A is a storage device that stores content, and may be a desktop PC, a tablet terminal, a smart phone, or the like in addition to the notebook PC shown in FIG.

The projector 3A projects the projection image data transmitted from the information processing apparatus 1A onto the screen S1 according to the control of the information processing apparatus 1A.

The laser pointer 2A irradiates the screen S1 with the visible light laser beam and the invisible light marker at the same position or only with the visible light marker. The visible light laser light and the visible light marker are laser light and marker images that are irradiated with visible light visible to the human eye. The invisible light marker is a marker image irradiated with invisible light (invisible light) such as infrared rays or ultraviolet rays that cannot be seen by human eyes.

In addition, the non-visible light marker / visible light marker is a one-dimensional or two-dimensional barcode in which various figures such as a star shape and a cross, or specific information is embedded. The operator 7A makes a presentation while indicating an arbitrary portion of the screen S1 with a visible light laser beam or a visible light marker emitted from the laser pointer 2A.

The camera 5A captures the projected image projected on the screen S1, and outputs the captured image to the information processing apparatus 1A. The camera 5A has a non-visible light imaging function that performs non-visible light (invisible light) imaging such as an infrared camera or an ultraviolet camera, or a visible light imaging function that performs visible light imaging.

The information processing apparatus 1A uses an irradiation point P1 (visible light laser light, visible light marker, or invisible light marker) by the laser pointer 2A based on the visible / invisible light captured image of the projection image captured by the camera 5A. The coordinate position of the designated point on the projected image is recognized.

Further, the information processing apparatus 1A can calculate the three-dimensional position of the laser pointer 2A. Specifically, the information processing apparatus 1A determines the size of the shape of the visible / invisible light marker emitted from the laser pointer 2A based on the visible / invisible light captured image of the projection image captured by the camera 5A. From the distortion and the like, the three-dimensional position of the laser pointer 2A with respect to the projection image is calculated.

Further, the information processing apparatus 1A acquires information on the operator 7A of the laser pointer 2A. The information of the operator 7A is, for example, a name such as a name or a nickname, a title, an affiliation, a current location, a face image or an avatar image registered in advance. Specifically, the information processing apparatus 1A analyzes the visible / invisible light captured image of the projection image captured by the camera 5A, and the shape and color of the visible / invisible light marker emitted from the laser pointer 2A. The embedded information (user ID) is read, and the operator 7A of the laser pointer 2A is specified.

Further, the information processing apparatus 1A uses the camera 6A so that the operator 7A of the laser pointer 2A is included in the angle of view of the camera 6A different from the camera 5A that captures the projected image based on the calculated three-dimensional position. The movement (pan, tilt, zoom) may be controlled. Thereby, the information processing apparatus 1A can acquire a real-time image of the operator 7A of the laser pointer 2A. Further, the information processing apparatus 1A directs the microphone 8A having high directivity toward the operator 7A of the laser pointer 2A based on the calculated three-dimensional position, and utters (sounds) of the operator who has irradiated the laser pointer 2A. Data) can also be collected.

Then, the information processing apparatus 1A on the local side uses the image data for projection (display signal), the coordinate position of the irradiation point P1, the information of the operator 7A, the real-time video of the operator 7A captured by the camera 6A, and the microphone 8A. The collected voice data and the like are transmitted to the information processing apparatus 1B on the remote side.

On the other hand, the information processing apparatus 1B installed in the remote ROOM • B is connected to each device installed in the ROOM • B by wire / wireless. Specifically, the information processing device 1B is connected to a projector 3B ((an example of a display device)), a camera 6B that captures an image of a viewer 7B, a microphone 8B, and a speaker 9B.

The information processing apparatus 1B transmits the projection image data received from the information processing apparatus 1A to the projector 3B, and causes the projector 3B to project the image data onto the screen S2. At this time, the information processing apparatus 1B performs control to superimpose and display the remote irradiation point P1 ′ at the coordinate position corresponding to the coordinate position of the irradiation point P1 received from the information processing apparatus 1A on the projection image projected on the screen S2. To do. Thereby, even on the projection image projected on the screen S2 on the remote side, the same locus as the irradiation point P1 irradiated on the projection image projected on the screen S1 on the ROOM • A side can be displayed.

Further, as shown in FIG. 1, the information processing apparatus 1B superimposes and displays information (for example, name and face image) of the operator 7A of the laser pointer 2A in association with the remote irradiation point P1 '. As a result, the viewer 7B can intuitively understand who is operating and explaining the remote irradiation point P1 'superimposed on the screen S2 on the ROOM • A side. The information processing apparatus 1B may superimpose and display an image of the operator 7A captured in real time by the camera 6A as the information about the operator 7A.

In addition, when the information processing apparatus 1B receives audio data collected by the microphone 8A from the local information processing apparatus 1A, the information processing apparatus 1B controls the speaker 9B to reproduce from the speaker 9B. The information processing apparatus 1B can also transmit the utterance (voice data) of the viewer 7B collected by the microphone 8B to the information processing apparatus 1A of the ROOM • A.

As described above, in the remote cooperation system according to the present embodiment, the information of the operator 7A who is operating the laser pointer 2a is superimposed and displayed in association with the remote irradiation point P1 ′ displayed on the remote side. It is possible to intuitively understand who is operating and explaining, and to smoothly conduct a communication conference connecting a plurality of remote locations.

1 illustrates the case of a communication conference that connects two remote locations as an example, but the remote location cooperation system according to the present embodiment is not limited to this. For example, as shown in FIG. The same can be applied to the communication conference to be connected.

(Communication conference connecting three remote locations)
FIG. 2 is a diagram for describing a case where the remote location cooperation system according to an embodiment of the present disclosure is applied to a communication conference that connects three remote locations. In the example shown in FIG. 2, it is assumed that a communication conference connecting three remote locations, ROOM / A, ROOM / B, and ROOM / C is performed. As shown in FIG. 2, a remote location cooperation system according to an embodiment of the present disclosure is formed by a plurality of

information processing devices

1A, 1B, and 1C installed in a plurality of remote locations, and each information processing device (communication device). 1A, 1B, and 1C are connected via a network.

Since each device installed in ROOM • A and ROOM • B is the same as that in FIG. 1, description thereof is omitted here.

The information processing apparatus 1C installed in the ROOM · C is connected to each device installed in the ROOM · C by wire / wireless. Specifically, the information processing apparatus 1C includes a projector 3C, a content display apparatus 4C, a camera 5C that captures an image projected on the screen S3, a camera 6C that captures an operator 7C of the laser pointer 2C, a microphone 8C, and a speaker 9C. Connect with. The system configuration shown in FIG. 2 is an example, and for example, the microphone 8C and the camera 6C may be integrated, the content display device 4C and the information processing device 1C may be integrated, or the camera 5C may be built in the projector 3C.

In the system configuration described above, the information processing apparatus 1A transmits the coordinate position of the irradiation point P1, information on the operator 7A, and the like to the information processing apparatus 1B and the information processing apparatus 1C that are remote to the information processing apparatus 1A. In addition, the information processing apparatus 1C transmits the coordinate position of the irradiation point P3, information on the operator 7C, and the like to the information processing apparatus 1B and the information processing apparatus 1A that are remote to the information processing apparatus 1C.

For example, a remote irradiation point P1 'corresponding to the irradiation point P1 and a remote irradiation point P3' corresponding to the irradiation point P3 are superimposed and displayed on the projection image projected on the screen S2 of the ROOM • C. The remote irradiation point P1 ′ is displayed in the same locus as the irradiation point P1 irradiated on the projection image projected on the screen S1 on the ROOM • A side, and the remote irradiation point P3 ′ is displayed on the screen S3 on the ROOM • C side. The projected image is displayed on the projected image with the same locus as the irradiated point P3.

In addition, information on the operator 7A of the laser pointer 2A is displayed in a superimposed manner in association with the remote irradiation point P1 ′, and information on the operator 7C in the laser pointer 2C is displayed in a superimposed manner in association with the remote irradiation point P3 ′. . As a result, the viewer 7B can intuitively understand who is operating the remote irradiation points P1 'and P3' superimposed on the screen S2.

A remote irradiation point P3 ′ corresponding to the irradiation point P3 is superimposed and displayed on the projection image projected on the screen S1 of the ROOM • A, and on the projection image projected on the screen S3 of the ROOM • C, A remote irradiation point P1 ′ corresponding to the irradiation point P1 is displayed in a superimposed manner.

As described above, the remote location cooperation system according to this embodiment can be applied to a communication conference that connects three remote locations. Next, an internal configuration example of each configuration forming the remote location cooperation system according to the present embodiment will be described with reference to a plurality of embodiments.

<< 2. Each embodiment >>
<2-1. First Embodiment>
(2-1-1. Configuration)
FIG. 3 is a block diagram illustrating an internal configuration example of the information processing apparatus 1A-1 and the information processing apparatus 1B-1 that form the remote location cooperation system according to the first embodiment.

(Information processing apparatus 1A-1)
The information processing apparatus 1A-1 is an example of the information processing apparatus 1A installed in the ROOM • A described with reference to FIG. As shown in FIG. 3, the information processing apparatus 1A-1 includes a transmission / reception unit 11A, an irradiation position recognition unit 12, an operator information acquisition unit 13, a three-dimensional position calculation unit 14, a transmission unit 15A, an output control unit 16, and a reception. Part 17A.

The transmission / reception unit 11A has a function of transmitting / receiving data by connecting to each device installed in the ROOM • A by wire / wireless. Specifically, the transmission / reception unit 11A receives content from the content display device 4A, transmits image data for projection based on the content to the projector 3A, and receives a captured image of the projection image from the camera 5A. In addition, the transmission / reception unit 11A transmits an imaging control signal to the camera 6A that images the operator 7A, and receives a captured image from the camera 6A. In addition, the transmission / reception unit 11A transmits a sound collection control signal to the microphone 8A that collects the speech of the operator 7A, and receives sound collection data from the microphone 8A. Further, the transmission / reception unit 11A transmits the sound collection data transmitted from the remote information processing apparatus 1B-1 to the speaker 9A.

The method of wireless communication between the transmission / reception unit 11A and each device installed in the ROOM A is not particularly limited. For example, a wireless LAN, Wi-Fi (registered trademark), or Bluetooth (registered trademark) is connected. Also good.

The irradiation position recognition unit 12 irradiates the projected image with the visible light laser light or the visible light / invisible light marker by the laser pointer 2A based on the visible light / invisible light captured image of the projected image captured by the camera 5A. Recognize the coordinate position of P1. Specifically, for example, the irradiation position recognition unit 12 detects a difference between an image projected by the projector 3A (projection image data) and a visible / invisible light captured image obtained by capturing the projection image. The position coordinates of the irradiation position P1 are detected. The irradiation position recognizing unit 12 also analyzes the difference between the visible light / invisible light captured image of the previous frame of the currently projected image and the visible light / invisible light captured image of the currently projected image. The accuracy can be increased by adding to. The irradiation position recognition unit 12 outputs the recognized coordinate position of the irradiation point P1 to the transmission unit 15A.

The operator information acquisition unit 13 analyzes the visible / invisible light captured image of the projection image captured by the camera 5A, and embeds the shape and color of the visible / invisible light marker emitted from the laser pointer 2A. And the like, the operator 7A is specified, and the operator information is acquired. Specifically, for example, the operator information acquisition unit 13 has information on the operator associated with the shape and color of the visible / invisible light marker in the information processing apparatus 1A-1 or on the cloud. Extracted from an operator information DB (database) (not shown). Moreover, the operator information acquisition part 13 can also acquire operator information based on the information read from the visible light / invisible light marker. The operator information acquisition unit 13 outputs the acquired operator information to the transmission unit 15A.

The three-dimensional position calculation unit 14 calculates the three-dimensional position of the laser pointer 2A with respect to the projection image projected on the screen S. Specifically, the three-dimensional position calculation unit 14 forms the shape of the visible / invisible light marker emitted from the laser pointer 2A based on the visible / invisible light captured image of the projection image captured by the camera 5A. The three-dimensional position (positional relationship) of the laser pointer 2A with respect to the projection image is calculated from the size and distortion of The three-dimensional position calculation unit 14 outputs the calculated three-dimensional position to the transmission unit 15 </ b> A and the output control unit 16.

The transmission unit 15A is connected to the information processing apparatus 1B-1 on the remote side via a network and transmits data. For example, the transmission unit 15A includes the coordinate position of the irradiation point P1 recognized by the irradiation position recognition unit 12, the operator information acquired by the operator information acquisition unit 13, and the three-dimensional position of the laser pointer 2A (projection image and laser pointer). 2A), image data for projection, collected sound data transmitted from the microphone 8A, and the like are transmitted to the information processing apparatus 1B-1.

The receiving unit 17A is connected to the information processing apparatus 1B-1 on the remote side via a network and receives data. For example, the receiving unit 17A receives audio data collected on the remote side.

The output control unit 16 has a function of processing data output from each device installed in the ROOM A. Specifically, for example, the output control unit 16 performs content output processing. The content output processing is processing for generating projection image data (display signal) based on content data acquired from the content display device 4A, for example. Further, the output control unit 16 controls to reproduce the audio data of the viewer 7B received from the information processing apparatus 1B-1 via the receiving unit 17A from the speaker 9A.

The output control unit 16 also controls the control signal for controlling the movement of the camera 6A so that the camera 6A reflects the operator 7A based on the three-dimensional position of the laser pointer 2A, and the microphone 8A speaks the operator 7A. A control signal for controlling the direction of directivity so as to collect sound is transmitted from the transmission / reception unit 11A. What is calculated by the three-dimensional position calculation unit 14 is the positional relationship between the projection image (screen S1) and the laser pointer 2A. If the positional relationship between other devices is known, the output control unit 16 The relative positional relationship between the camera 6A and the operator 7A and the relative positional relationship between the microphone 8A and the operator 7A can be specified. For example, when the positional relationship between the projector 3A and the screen S1, the positional relationship between the camera 6A and the projector 3A, the positional relationship between the projector 3A and the camera 6A are known, and the positional relationship between the laser pointer 2A and the operator 7A is normal ( The output control unit 16 can specify the relative positional relationship between the camera 6A and the operator 7A.

(Information processing apparatus 1B-1)
The information processing apparatus 1B-1 is an example of the information processing apparatus 1B installed in the ROOM • B described with reference to FIG. As illustrated in FIG. 3, the information processing apparatus 1B-1 includes a transmission / reception unit 11B, a transmission unit 15B, a reception unit 17B, and a superimposed image signal generation unit 18.

The receiving unit 17B connects to the information processing apparatus 1A-1 on the remote side via a network and receives data. For example, the reception unit 17B receives the coordinate position of the irradiation point P1, operator information, the three-dimensional position of the laser pointer 2A, projection image data, sound collection data, and the like. The receiving unit 17B outputs the coordinate position of the irradiation point P1, operator information, the three-dimensional position of the laser pointer 2A, and the content to the superimposed image signal generation unit 18, and outputs the audio data to the transmission / reception unit 11B.

Based on the coordinate position of the irradiation point P1, the superimposed image signal generation unit 18 generates an image signal in which the remote irradiation point P1 ′ is superimposed on the coordinate position on the projection image data corresponding to the coordinate position of the irradiation point P1. Generate. As a result, the information processing apparatus 1B-1 can display the same locus on the projected image on the screen S2 on the ROOM • B side as the irradiation point P1 irradiated on the projected image on the screen S1 on the ROOM • A side.

The superimposed image signal generation unit 18 may also generate an image signal in which operator information (name, face image, avatar image, etc.) is superimposed at a position corresponding to the remote irradiation point P1 ′ on the projection image. Is possible.

Further, the superimposed image signal generation unit 18 superimposes a human (person) whole body image on the projection image data at a position corresponding to the three-dimensional position information (positional relationship) of the laser pointer 2A with respect to the projection image (screen S1). It is also possible to generate an image signal. Thereby, it seems that the operator 7A of the remote irradiation point P1 'is actually present through the screen S2. Specifically, the whole body image of a person is, for example, a silhouette image, an avatar, an operator's CG (computer graphic), an image obtained by cutting out the entire body of the operator from an image of the operator. In the example described later with reference to FIG. 7, a case where a silhouette image is used as a whole body image of a person will be described.

The superimposed image signal generation unit 18 transmits the generated image signal to the projector 3B via the transmission / reception unit 11B.

The transmission / reception unit 11B is connected to each device installed in the ROOM / B by wire / wireless and has a function of transmitting / receiving data. Specifically, the transmission / reception unit 11B transmits image data for projection based on the image signal generated by the superimposed image signal generation unit 18 to the projector 3B, and transmits audio data on the ROOM • A side to the speaker 9B. Moreover, the transmission / reception part 11B receives the captured image which imaged the viewer 7B from the camera 6B, and receives the audio | voice data which picked up the speech of the viewer 7B from the microphone 8B.

The method of wireless communication between the transmission / reception unit 11B and each device installed in the ROOM B is not particularly limited. For example, the wireless communication system is connected by a wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), or the like. Also good.

The transmission unit 15B transmits the utterance (voice data) of the viewer 7B collected by the microphone 8B and the captured image of the viewer 7B captured by the camera 6B to the information processing apparatus 1A-1 via the network.

The above has specifically described the internal configuration of each device forming the remote cooperation system according to the first embodiment. Note that the information processing apparatuses 1A-1 and 1B-1 may convert the voices collected from the

microphones

8A and 8B into text by performing voice recognition and output the text data to the other party. As a result, on the other side, the text data of the voice of the user talking at a remote location is displayed on the screen together with the vicinity of the remote irradiation point P1 ′ corresponding to the irradiation point P1 operated by the user at the remote location and the user's avatar image. Can be projected. Further, the information processing apparatuses 1A-1 and 1B-1 may translate the collected sound and output it to the other party. Subsequently, an operation process of the remote cooperation system according to the first embodiment will be described with reference to FIG.

(2-1-2. Operation processing)
FIG. 4 is a diagram for explaining the operation processing of the remote cooperation system according to the first embodiment. As shown in FIG. 4, first, in step S100, the information processing apparatus 1A-1 starts projecting content data. Specifically, the information processing apparatus 1A-1 transmits projection image data based on the content data to the projector 3A, and causes the projector 3A to project the image data onto the screen S1.

Next, in step S103, the information processing apparatus 1A-1 acquires a captured image of the projection image captured by the camera 5A.

Next, in step S106, the irradiation position recognition unit 12 of the information processing apparatus 1A-1 recognizes the coordinate position of the irradiation point P1 based on the captured image.

Next, in step S109, S103 and S106 are repeated until the coordinate position of the irradiation point P1 is recognized.

Next, when the coordinate position of the irradiation point P1 is recognized (S109 / Yes), in step S112, the operator information acquisition unit 13 of the information processing apparatus 1A-1 operates the irradiation point P1 with the laser pointer 2A. The information of the person 7A is acquired. Specifically, for example, the operator information acquisition unit 13 analyzes the captured image of the projection image captured by the camera 5A, and the shape and color of the visible / invisible light marker emitted from the laser pointer 2A, or An operator is specified based on information embedded in the visible / invisible light marker, and operator information is acquired.

Subsequently, in step S115, the three-dimensional position calculation unit 14 of the information processing apparatus 1A-1 calculates the three-dimensional position (relative positional relationship) of the laser pointer 2A with respect to the projection image (screen S1). Specifically, the three-dimensional position calculation unit 14 analyzes the captured image of the projection image captured by the camera 5A, and the shape size and distortion of the visible / invisible light marker irradiated from the laser pointer 2A. To calculate the three-dimensional position of the laser pointer 2A.

In step S118, the transmission unit 15A of the information processing apparatus 1A-1 acquires the content data acquired from the content display apparatus 4A (specifically, at least the currently projected image data) and the position of the irradiation point P1. The coordinates and the operator information are transmitted to the information processing apparatus 1B-1.

Next, in step S121, the output control unit 16 of the information processing apparatus 1A-1 determines the imaging direction of the camera 6A and the directivity of the microphone 8A based on the three-dimensional position calculated by the three-dimensional position calculation unit 14. Control is directed to the operator 7A. The captured image captured by the camera 6A and the audio data collected by the microphone 8A are also transmitted from the transmission unit 15A of the information processing apparatus 1A-1 to the information processing apparatus 1B-1.

On the other hand, in step S124, on the remote side, the information processing apparatus 1B-1 applies the remote irradiation point to the content data transmitted from the information processing apparatus 1A-1 (the same as the image currently projected on the ROOM • A side). An image signal on which P1 ′ and operator information are superimposed is generated. The remote irradiation point P1 'is superimposed on the same position as the coordinate position of the irradiation point P1. Image data for projection based on the image signal generated in this way is transmitted to the projector 3B and projected onto the screen S2.

As described above, the remote irradiation point P1 ′ on the projected image of the screen S2 of the remote ROOM • B is displayed with the same locus as the irradiation point P1 irradiated to the screen S1 of the ROOM • A side. . Further, the operator information is superimposed and displayed at the position corresponding to the remote irradiation point P1 ′, so that the viewer 7B can intuitively grasp who is operating the remote irradiation point P1 ′ on the ROOM • A side. obtain.

Here, the position corresponding to the remote irradiation point P1 ′ is, for example, in the vicinity of the remote irradiation point P1 ′, and the operator information is displayed near the remote irradiation point P1 ′. You can grasp intuitively whether you are operating. In addition, when the operator information is displayed near the remote irradiation point P1 ', there may be a situation where it is difficult to see the original content (projected image) indicated by the remote irradiation point P1'. Therefore, for example, as shown on the left side of FIG. 5, the operator's profile image, name, and location ("" linked to the cursor 30 of the remote irradiation point P1 'displayed on the image projected on the screen S2 by a solid line. The operator information 32 such as the last utterance sentence that has been voice-recognized may be superimposed and displayed at a position away from the cursor 30.

Alternatively, when the irradiation point P1 moves in a remote place (ROOM • A) or when a new operator appears, as shown in the right of FIG. 5, a solid line is drawn from the cursor 30 of the remote irradiation point P1 ′. The linked operator information 32 may be displayed for a few seconds in a dedicated area 35 provided around the reduced content portion.

Note that the cursor 30 (remote irradiation point P1 ') shown in FIG. 5 is included in the image projected from the projector 3B on the screen S2, but the method of clearly indicating the remote irradiation point P1' is not limited to this. For example, when a laser irradiation apparatus is installed in ROOM B, the information processing apparatus 1B-1 controls to actually irradiate the screen S2 with a visible laser beam according to the coordinate position of the irradiation point P1. You may specify remote irradiation point P1 '. This makes it possible to produce a highly realistic presentation on the remote side.

The operation processing according to this embodiment has been described above. The operation processing according to the present embodiment is not limited to the example shown in FIG. For example, in the remote location enhancement system according to the present embodiment, the three-dimensional position of the laser pointer 2A with respect to the projection image (screen S1) is transmitted to the remote side, and the silhouette image of the operator (person's An example of a whole body image) may be displayed. Hereinafter, a specific description will be given with reference to FIGS.

(Silhouette image display)
FIG. 6 is a flowchart showing an operation process when silhouette display is performed on the remote side in the remote cooperation system according to the first embodiment.

In steps S100 to S115 shown in FIG. 6, processing similar to that shown in FIG. 4 is performed.

Next, in step S119, the transmission unit 15A of the information processing apparatus 1A-1 uses the content data acquired from the content display apparatus 4A, the position coordinates of the irradiation point P1, the operator information, and the three-dimensional position as the information processing apparatus. To 1B-1.

Next, in step S121, the same processing as that shown in FIG. 4 is performed.

On the other hand, in step S125, on the remote side, the information processing apparatus 1B-1 adds a silhouette corresponding to the remote irradiation point P1 ′, the operator information, and the three-dimensional position to the content data transmitted from the information processing apparatus 1A-1. An image signal in which an image is superimposed is generated. Image data for projection based on the image signal generated in this way is transmitted to the projector 3B and projected onto the screen S2.

An example of superimposed display of silhouette images will be described with reference to FIG. As shown in FIG. 7, the remote irradiation point P1 ′ is superimposed on the projection image 36 projected on the screen S2 installed in the ROOM • B, and is further linked to the remote irradiation point P1 ′ and is a dummy silhouette image 38. Is superimposed. The display position of the silhouette image 38 is adjusted to correspond to the three-dimensional position of the laser pointer 2A (positional relationship with the screen S1). For example, when the operator 7A irradiates the screen S1 with laser light (irradiation point P1) from the left side on the ROOM A side, as shown in FIG. 7, a silhouette image is displayed on the left side of the remote irradiation point P1 ′. 38 is displayed. Thereby, the browsing person 7B who exists in ROOM * B can perform the face-to-face communication with the operator 7A in a remote place (ROOM * A) through a screen.

Further, as shown in FIG. 7, an operator information image 37 may be superimposed and displayed (projected) in association with the silhouette image 38.

Further, the member information 7A-1 and 7A-2 in ROOM • A may be superimposed and displayed (projected) on the projected image 36 projected on the screen S2. As a result, the viewer 7B can intuitively know who is in the remote area (ROOM · A) across the screen other than the operator.

<2-2. Second Embodiment>
In the first embodiment described above, the content (projected image data), operator information, the coordinate position of the irradiation point P1, and the like are individually transmitted to the remote side and displayed superimposed on the remote side ( Projected). However, the remote cooperation system according to the present disclosure is not limited to this. For example, content (projected image data) on the local side, operator information, the coordinate position of the irradiation point P1, and the like are combined to generate image data ( A second embodiment is also conceivable in which an image signal is generated and transmitted to the remote side. Hereinafter, a specific description will be given with reference to FIG.

FIG. 8 is a block diagram showing an internal configuration example of each device forming the remote location cooperation system according to the second embodiment. The information processing device 1A-2 and the information processing device 1B-2 according to the second embodiment shown in FIG. 8 are the same as the information processing device 1A-1 and the information processing device 1B-1 according to the first embodiment shown in FIG. Compared with the configuration, the information processing apparatus 1A-2 has a superimposed image signal generation unit 18 that the information processing apparatus 1B-1 has.

The superimposition image signal generation unit 18 included in the information processing apparatus 1A-2 is based on the projection image data corresponding to the coordinate position of the irradiation point P1 based on the coordinate position of the irradiation point P1 recognized by the irradiation position recognition unit 12. An image signal in which the remote irradiation point P1 ′ is superimposed on the coordinate position of is generated. The superimposed image signal generation unit 18 generates an image signal in which operator information is superimposed at a position corresponding to the remote irradiation point P1 'on the projection image.

The superimposed image signal generated in this way is transmitted from the transmission unit 15A to the information processing apparatus 1B-2. Then, the information processing apparatus 1B-2 transmits the superimposed image signal received by the reception unit 17B from the transmission / reception unit 11B to the projector 3B installed in the ROOM • B, and causes the projector 3B to project it onto the screen S2.

As described above, the remote site cooperation system according to the present embodiment generates an image in which the remote irradiation point P1 ′ and the operator information are superimposed on the projection image on the local side, and transmits the generated superimposed image signal to the remote side. It is also possible to do.

<2-3. Third Embodiment>
In the first and second embodiments described above, the information processing devices (communication devices) 1A and 1B (see FIG. 1) and the information processing devices 1A to 1C (see FIG. 2) are mutually connected via a network. The case of the system configuration connected by P2P (Peer to Peer) has been described. However, the system configuration of the remote cooperation system according to the present disclosure is not limited to the model connected by P2P, and may be a system configuration connected by a server-client as shown in FIG. 9, for example.

FIG. 9 is a diagram for explaining another system configuration example of the remote cooperation system according to the present disclosure. As shown in FIG. 9, the remote cooperation system according to the present embodiment may have a system configuration in which a plurality of information processing apparatuses 1A to 1C are connected to a server (communication server) 100, respectively. Variations of internal configuration examples of the devices forming the remote cooperation system when such a system configuration is applied will be described below with reference to third to seventh embodiments. In the third to seventh embodiments, a case of a server-client system in which a plurality of

information processing apparatuses

1A and 1B are connected to the server 100 will be described.

(Internal configuration example)
FIG. 10 is a block diagram illustrating an internal configuration example of each device forming the remote location cooperation system according to the third embodiment. The information processing device 1A-3 and the information processing device 1B-3 according to the third embodiment shown in FIG. 10 are the same as the information processing device 1A-1 and the information processing device 1B-1 according to the first embodiment shown in FIG. Compared with the configuration, the following points are different. That is, the transmission unit 15A of the information processing apparatus 1A-3 transmits data to the server 100-1, the reception unit 17A receives data from the server 100-1, and the reception unit 17B of the information processing apparatus 1B-3 -1 is received, and the transmission unit 15B transmits data to the server 100-1.

The server 100-1 has a transfer unit 110, transfers the data transmitted from the transmission unit 15A of the information processing device 1A-3 as it is to the information processing device 1B-3, and transmits the data to the transmission unit 15B of the information processing device 1B-3. The data transmitted from is directly transferred to the information processing apparatus 1A-3.

Since the contents of data transmitted and received between the information processing apparatus 1A-3 and the information processing apparatus 1B-3 are the same as those in the first embodiment, description thereof is omitted here.

As described above, according to the second embodiment, the information processing apparatuses 1A-3 and 1B-3 can realize remote cooperation through the server 100-1.

<2-4. Fourth Embodiment>
In the third embodiment, content (projected image data), operator information, and a superimposed image such as the coordinate position of the irradiation point P1 are generated on the remote side. The fourth embodiment Then, a superimposed image is generated on the local side. Hereinafter, a specific description will be given with reference to FIG.

FIG. 11 is a block diagram showing an example of the internal configuration of each device forming the remote location cooperation system according to the fourth embodiment. The information processing device 1A-4 and the information processing device 1B-4 according to the fourth embodiment shown in FIG. 11 are the same as the information processing device 1A-3 and the information processing device 1B-3 according to the third embodiment shown in FIG. Compared with the configuration, the information processing apparatus 1A-3 has a superimposed image signal generation unit 18 that the information processing apparatus 1B-3 has.

Similar to the second embodiment, the superimposed image signal generation unit 18 included in the information processing apparatus 1A-2 uses the coordinates of the irradiation point P1 based on the coordinate position of the irradiation point P1 recognized by the irradiation position recognition unit 12. An image signal is generated by superimposing the remote irradiation point P1 ′ on the coordinate position on the projection image data corresponding to the position. The superimposed image signal generation unit 18 generates an image signal in which operator information is superimposed at a position corresponding to the remote irradiation point P1 'on the projection image.

The superimposed image signal generated in this way is transmitted from the transmission unit 15A to the information processing apparatus 1B-2 via the server 100-1. Then, the information processing apparatus 1B-4 transmits the superimposed image signal received by the reception unit 17B from the transmission / reception unit 11B to the projector 3B installed in the ROOM • B, and causes the projector 3B to project it onto the screen S2.

As described above, the remote cooperation system according to the present embodiment generates an image in which the remote irradiation point P1 ′ and the operator information are superimposed on the projection image on the local side, even when the server-client connection is established. The superimposed image signal thus transmitted can be transmitted to the remote side.

<2-5. Fifth Embodiment>
In the third and fourth embodiments, the server 100-1 merely mediates transmission / reception of data between the information processing apparatuses. However, in the fifth embodiment, the server 100-2 executes the superimposed image signal. Is generated.

That is, in the third and fourth embodiments described above, content (projected image data), operator information, the coordinate position of the irradiation point P1, and the like are superimposed (synthesized) on the remote side or the local side. In the fifth embodiment, the superimposed image is generated on the server side. Hereinafter, a specific description will be given with reference to FIG.

FIG. 12 is a block diagram showing an example of the internal configuration of each device forming the remote location cooperation system according to the fifth embodiment. The information processing apparatuses 1A-5 and 1B-5 and the server 100-2 according to the fifth embodiment shown in FIG. 12 differ from the internal configurations according to the fourth embodiment shown in FIG. 11 in the following points. That is, the difference is that the server 100-2 includes the superimposed image signal generation unit 120 having the same function as the superimposed image signal generation unit 18 included in the information processing apparatus 1A-5, and the transmission / reception unit 111.

The superimposed image signal generation unit 120 included in the server 100-2 is based on the projection image data corresponding to the coordinate position of the irradiation point P1 based on the coordinate position of the irradiation point P1 transmitted from the information processing apparatus 1A-5. An image signal in which the remote irradiation point P1 ′ is superimposed on the coordinate position is generated. In addition, the superimposed image signal generation unit 120 generates an image signal in which operator information is superimposed at a position corresponding to the remote irradiation point P1 'on the projection image.

The superimposed image signal generated in this way is transmitted from the transmission / reception unit 111 to the information processing apparatus 1B-5. Then, the information processing apparatus 1B-5 transmits the superimposed image signal received by the reception unit 17B from the transmission / reception unit 11B to the projector 3B installed in the ROOM • B, and causes the projector 3B to project it onto the screen S2.

As described above, the remote site cooperation system according to the present embodiment generates an image in which the remote irradiation point P1 ′ and the operator information are superimposed on the projection image on the server side when the server-client connection is made, The superimposed image signal thus transmitted can be transmitted to the remote side.

<2-6. Sixth Embodiment>
In the fourth and fifth embodiments described above, the content output processing is performed by the local output control unit 16, but in the sixth embodiment, the server 100-3 also performs content output processing. As described above, the content output processing is processing for generating projection image data (display signal) based on content data acquired from the content display device 4A, for example. Hereinafter, the sixth embodiment will be specifically described with reference to FIG.

FIG. 13 is a block diagram showing an example of the internal configuration of each device forming the remote location cooperation system according to the sixth embodiment. The information processing apparatuses 1A-6, 1B-6 and the server 100-3 according to the sixth embodiment shown in FIG. 13 differ from the internal configurations according to the fifth embodiment shown in FIG. 12 in the following points. That is, the difference is that the server 100-3 includes the output control unit 130 having the same function as the output control unit 16 included in the information processing apparatus 1A-5, the transmission / reception unit 111, and the superimposed image signal generation unit 120.

The transmission / reception unit 111 of the server 100-3 receives content data from the content display device 4A installed in the ROOM A via the information processing device 1A-6 or from a content server on the network. In addition, the transmission / reception unit 111 of the server 100-3 uses the image data for projection (display signal) generated by the output control unit 130 based on the acquired content data as the information processing apparatuses 1A on the local side and the remote side. -6, send to 1B-6. Thereby, on the local side and the remote side, the same image data is projected from the

projectors

3A, 3B onto the screens S1, S2, respectively.

When the irradiation point P1 irradiated by the laser pointer 2A is operated (moved) on the local side, the transmission unit 15A of the information processing apparatus 1A-6 transmits the coordinate position of the irradiation point P1, operator information, and the laser pointer 2A. The three-dimensional position is transmitted to the server 100-3. The superimposed image signal generation unit 120 of the server 100-3 performs irradiation points on the projection image data output from the output control unit 130 based on the coordinate position of the irradiation point P1 received from the information processing apparatus 1A-6. An image signal is generated by superimposing the remote irradiation point P1 ′ on the coordinate position corresponding to the coordinate position of P1. In addition, the superimposed image signal generation unit 120 generates an image signal in which operator information is superimposed at a position corresponding to the remote irradiation point P1 'on the projection image.

The superimposed image signal generated in this way is transmitted from the transmission / reception unit 111 to the information processing apparatus 1B-6. Then, the information processing apparatus 1B-6 transmits the superimposed image signal received by the reception unit 17B from the transmission / reception unit 11B to the projector 3B installed in the ROOM • B, and causes the projector 3B to project it onto the screen S2.

Further, the output control unit 130 of the server 100-3 operates the imaging direction of the ROOM • A camera 6A and the directivity of the microphone 8A based on the three-dimensional position of the laser pointer 2A received from the information processing apparatus 1A-6. A control signal for controlling the movement of the camera 6A and the microphone 8A to be directed to the person 7A is transmitted to the information processing apparatus 1A-6.

As described above, in the sixth embodiment, the content output processing is performed on the server side, so that the processing burden on the local side can be reduced.

<2-7. Seventh Embodiment>
Next, a seventh embodiment will be described with reference to FIG. In the seventh embodiment, most processing is performed on the server side, and on the local side, only processing for transmitting a visible light / invisible light captured image for recognizing the coordinate position of the irradiation position P1 to the server side is performed.

FIG. 14 is a block diagram showing an example of the internal configuration of each device forming the remote cooperation system according to the seventh embodiment. The information processing apparatuses 1A-7 and 1B-7 and the server 100-4 according to the seventh embodiment shown in FIG. 14 differ from the internal configurations according to the sixth embodiment shown in FIG. 13 in the following points. That is, the irradiation position recognition unit 140, the operator information acquisition unit 13, and the irradiation position recognition unit 140 having the same functions as the three-dimensional position calculation unit 14 included in the information processing apparatus 1A-6, the operator information acquisition unit 150, and the server 100-4 has a three-dimensional position calculation unit 160.

Further, the other configurations of the server 100-4, the processing contents of the transmission / reception unit 111, the superimposed image signal generation unit 120, and the output control unit 130 are the same as those in the fifth embodiment.

With this configuration, in the sixth embodiment, the information processing apparatus 1A-7 continuously transmits the captured image of the projection image captured by the camera 5A to the server 100-4, and the server 100-4 performs irradiation. The position recognition unit 140 recognizes the coordinate position of the irradiation point P1 on the projection image. In addition, the operator information acquisition unit 150 of the server 100-4 also specifies the operator and acquires the operator information based on the captured image transmitted from the information processing apparatus 1A-7. The coordinate position of the irradiation point P1 and the operator information are output to the superimposed image signal generation unit 120 and are used to generate a superimposed image signal.

Further, the three-dimensional position calculation unit 160 of the server 100-4 calculates the three-dimensional position of the laser pointer 2A based on the captured image transmitted from the information processing apparatus 1A-7. The calculated three-dimensional position is output to the output control unit 130 to be used for controlling the camera 6A and the microphone 8A on the local side, or is output to the superimposed image signal generation unit 120 to output a silhouette image superimposed on the projection image. Or used to determine placement.

As described above, in the seventh embodiment, most processes such as the process of recognizing the coordinate position of the irradiation point P1 based on the captured image of the projection image projected on the local screen S1, and the process of acquiring operator information. Is performed on the server side, the processing load on the local side can be further reduced.

<< 3. Summary >>
As described above, in the remote cooperation system according to the present embodiment, information on the operator who operates the laser pointer on the local side is associated with the indication point (remote irradiation point) displayed (projected) on the remote side. By clearly displaying (projecting), it is possible to smoothly perform a communication conference connecting a plurality of remote locations.

The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present technology is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

For example, it is possible to create a computer program for causing the information processing apparatus 1 and the server 100 to perform the functions of the information processing apparatus 1 and the server 100 described above on hardware such as the CPU, ROM, and RAM incorporated in the information processing apparatus 1 and the server 100. A computer-readable storage medium storing the computer program is also provided.

Also, the flowcharts described above do not necessarily have to be processed in the order shown, and may be processed in parallel or in reverse order, for example. For example, in the flowchart shown in FIG. 4, steps S112 and S115 may be processed in parallel or in the reverse order. Moreover, step S121 and S118 and S124 may be processed in parallel.

In each of the above-described embodiments, on the remote side, a superimposed image (an image in which the remote irradiation point P1 and operator information are superimposed on the content) is projected on the screen S2 from the projector 3B (an example of a display device). However, the display output on the remote side is not necessarily limited to projection. For example, on the remote side, the superimposed image may be displayed and output on a display screen of a television device or a PC (personal computer).

In addition, this technique can also take the following structures.
(1)
A recognition unit for recognizing the position of the irradiation point by the laser pointer with respect to the projection image on the local side;
An acquisition unit for acquiring information of an operator of the laser pointer;
In the display device installed on the remote side, in order to display the projection image and the remote irradiation point located at the coordinate position corresponding to the irradiation point and the information of the operator, the position of the irradiation point and the operation A first transmitter for transmitting the information of the person to the remote side;
An information processing apparatus comprising:
(2)
The acquisition unit acquires information on the operator by analyzing a non-visible light / visible light marker irradiated by the laser pointer on the projection image based on a captured image obtained by capturing the projection image. The information processing apparatus according to (1).
(3)
The information processing apparatus according to (1) or (2), wherein the recognition unit recognizes a position coordinate of an invisible light / visible light marker irradiated from the laser pointer as a position of the irradiation point.
(4)
The information processing apparatus according to (2) or (3), wherein the invisible light / visible light marker is formed by a figure or a one-dimensional / 2-dimensional barcode.
(5)
The information processing apparatus includes:
Based on the captured image obtained by capturing the projection image, the three-dimensional position of the laser pointer with respect to the projection image is analyzed by analyzing the invisible light / visible light marker irradiated on the projection image by the laser pointer. The information processing apparatus according to any one of (1) to (4), further including a calculation unit that calculates.
(6)
The information according to (5), wherein the calculation unit calculates a three-dimensional position of the laser pointer based on at least one of a shape, a size, an inclination, and a distortion of the invisible light / visible light marker. Processing equipment.
(7)
The information processing apparatus includes:
A second transmission unit configured to transmit an imaging control signal to a local imaging device so as to capture an image of the operator of the laser pointer according to the three-dimensional position calculated by the calculation unit; ) Or the information processing apparatus according to (6).
(8)
The information processing apparatus includes:
A third transmission unit that transmits a sound collection control signal to the directional sound collection unit on the local side so as to face the operator of the laser pointer according to the three-dimensional position calculated by the calculation unit; The information processing apparatus according to any one of (5) to (7), comprising:
(9)
The first transmission unit displays the three-dimensional position calculated by the calculation unit on the remote side in order to display a whole body image of a person at a position corresponding to the three-dimensional position on a projection image. The information processing apparatus according to any one of (5) to (8), wherein the information processing apparatus transmits to the side.
(10)
Recognizing the position of the irradiation point by the laser pointer on the local projection image;
Obtaining information of an operator of the laser pointer;
In the display device installed on the remote side, in order to display the projection image and the remote irradiation point located at the coordinate position corresponding to the irradiation point and the information of the operator, the position of the irradiation point and the operation Sending the information of the person to the remote side;
Including a control method.
(11)
Computer
A recognition unit for recognizing the position of the irradiation point by the laser pointer with respect to the projection image on the local side;
An acquisition unit for acquiring information of an operator of the laser pointer;
In the display device installed on the remote side, in order to display the projection image and the remote irradiation point located at the coordinate position corresponding to the irradiation point and the information of the operator, the position of the irradiation point and the operation A first transmitter for transmitting the information of the person to the remote side;
Program to function as.
(12)
Computer
A recognition unit for recognizing the position of the irradiation point by the laser pointer with respect to the projection image on the local side;
An acquisition unit for acquiring information of an operator of the laser pointer;
In the display device installed on the remote side, in order to display the projection image and the remote irradiation point located at the coordinate position corresponding to the irradiation point and the information of the operator, the position of the irradiation point and the operation A first transmitter for transmitting the information of the person to the remote side;
A storage medium storing a program for functioning as a computer.

1, 1A (1A-1 to 1A-7), 1B (1B-1 to 1B-7), 1C

Information processing device

2, 2A, 2C Laser pointer 3A to 3C Projector 4A, 4B Content display device 5A, 5B Camera (For projection image shooting)
6A to 6C camera (for operator / viewer shooting)
7A, 7C Operator 7B Viewer 8A-8C Microphone 9A-

9C Speaker

11A, 11B Transmission /

reception unit

12, 140 Irradiation

position recognition unit

13, 150 Operator

information acquisition unit

14, 160 Three-dimensional

position calculation unit

15A,

15B Transmission unit

16 , 130

Output control unit

17A,

17B Reception unit

18, 120 Superimposed image signal generation unit 100 (100-1 to 100-4) Server 110 Transfer unit 111 Transmission / reception unit S1 to S3 Screen P1, P2 Irradiation point P1 ', P2' Remote Irradiation point

Claims

A recognition unit for recognizing the position of the irradiation point by the laser pointer with respect to the projection image on the local side;
An acquisition unit for acquiring information of an operator of the laser pointer;
In the display device installed on the remote side, in order to display the projection image and the remote irradiation point located at the coordinate position corresponding to the irradiation point and the information of the operator, the position of the irradiation point and the operation A first transmitter for transmitting the information of the person to the remote side;
An information processing apparatus comprising:
The acquisition unit acquires information on the operator by analyzing a non-visible light / visible light marker irradiated by the laser pointer on the projection image based on a captured image obtained by capturing the projection image. The information processing apparatus according to claim 1.
The information processing apparatus according to claim 1, wherein the recognition unit recognizes a position coordinate of an invisible light / visible light marker irradiated from the laser pointer as a position of the irradiation point.
The information processing apparatus according to claim 2, wherein the invisible light / visible light marker is formed by a figure or a one-dimensional / two-dimensional barcode.
The information processing apparatus includes:
Based on the captured image obtained by capturing the projection image, the three-dimensional position of the laser pointer with respect to the projection image is analyzed by analyzing the invisible light / visible light marker irradiated on the projection image by the laser pointer. The information processing apparatus according to claim 1, further comprising a calculation unit for calculating.
The information processing according to claim 5, wherein the calculation unit calculates a three-dimensional position of the laser pointer based on at least one of a shape, a size, a tilt, and a distortion of the invisible light / visible light marker. apparatus.
The information processing apparatus includes:
6. The apparatus according to claim 5, further comprising a second transmission unit configured to transmit an imaging control signal to a local imaging device so as to image the operator of the laser pointer in accordance with the three-dimensional position calculated by the calculation unit. The information processing apparatus described in 1.
The information processing apparatus includes:
A third transmission unit that transmits a sound collection control signal to the directional sound collection unit on the local side so as to face the operator of the laser pointer according to the three-dimensional position calculated by the calculation unit; The information processing apparatus according to claim 5, comprising:
The first transmission unit displays the three-dimensional position calculated by the calculation unit on the remote side in order to display a whole body image of a person at a position corresponding to the three-dimensional position on a projection image. The information processing apparatus according to claim 5, wherein the information processing apparatus transmits the information to the side.
Recognizing the position of the irradiation point by the laser pointer on the local projection image;
Obtaining information of an operator of the laser pointer;
In the display device installed on the remote side, in order to display the projection image and the remote irradiation point located at the coordinate position corresponding to the irradiation point and the information of the operator, the position of the irradiation point and the operation Sending the information of the person to the remote side;
Including a control method.
Computer
A recognition unit for recognizing the position of the irradiation point by the laser pointer with respect to the projection image on the local side;
An acquisition unit for acquiring information of an operator of the laser pointer;
In the display device installed on the remote side, in order to display the projection image and the remote irradiation point located at the coordinate position corresponding to the irradiation point and the information of the operator, the position of the irradiation point and the operation A first transmitter for transmitting the information of the person to the remote side;
Program to function as.
Computer
A recognition unit for recognizing the position of the irradiation point by the laser pointer with respect to the projection image on the local side;
An acquisition unit for acquiring information of an operator of the laser pointer;
In the display device installed on the remote side, in order to display the projection image and the remote irradiation point located at the coordinate position corresponding to the irradiation point and the information of the operator, the position of the irradiation point and the operation A first transmitter for transmitting the information of the person to the remote side;
A storage medium storing a program for functioning as a computer.