WO2022249900A1

WO2022249900A1 - Information processing device

Info

Publication number: WO2022249900A1
Application number: PCT/JP2022/020208
Authority: WO
Inventors: 徳道柴田; 優紀北本; 隆寿吉見; 敬太郎稲本; 好朗栃澤
Original assignee: Ｎｅｃソリューションイノベータ株式会社
Priority date: 2021-05-26
Filing date: 2022-05-13
Publication date: 2022-12-01
Also published as: JPWO2022249900A1

Abstract

Provided is an information processing device that is connected with a plurality of user devices which display at least a partial region of entire data, said information processing device comprising: a detection unit that detects a predetermined condition; and an instruction unit that instructs, according to the result of the detection by the detection unit, at least some of the connected user devices to display a region which a user device corresponding to the condition is displaying.

Description

Information processing equipment

The present invention relates to an information processing device, an information processing method, a recording medium, a display device, and a display method.

　Image data acquired by an imaging device such as a 360-degree camera may be distributed. For example, Patent Literature 1 describes a system for displaying image data on a head-mounted display, which has a view control section, an image generation section, and an output section. According to Patent Literature 1, a view control unit detects the range of the user's view based on the orientation of the head mounted display. The image generator also generates an image by extracting the range of the user's field of view from the image data. The output unit then outputs the generated image to the head-mounted display for display.

JP 2019-139673 A

In the situation described in Patent Document 1, if multiple users display image data in their respective postures, the image data viewed by each user will be different. As a result, for example, since each user has a different display range, it may be difficult for another user to accurately grasp the point pointed out by one user.

In this way, in systems that share information such as image data, there are cases where it is difficult to share information smoothly.

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide an information processing apparatus, an information processing method, a recording medium, and a display that solve the problem that it is sometimes difficult to smoothly share information in a system that shares information such as image data. An object of the present invention is to provide an apparatus and a display method.

In order to achieve such an object, an information processing device, which is one aspect of the present disclosure,
An information processing device connected to a plurality of user devices that display at least a partial area of all data,
a detection unit that detects a predetermined condition;
an instruction unit that instructs at least some of the connected user devices to display the area displayed by the user device corresponding to the condition according to the detection result of the detection unit;
It has a configuration of

Further, an information processing method that is another aspect of the present disclosure includes:
An information processing device connected to a plurality of user devices that display at least a partial area of all data,
detecting a predetermined condition;
According to the detection result, at least some of the connected user devices are instructed to display the area displayed by the user device corresponding to the condition.

In addition, a recording medium that is another aspect of the present disclosure includes:
An information processing device connected to a plurality of user devices that display at least a partial area of all data,
detecting a predetermined condition;
A program is recorded for executing the process of instructing at least some of the connected user devices to display the area displayed by the user device corresponding to the conditions according to the detection result. , is a computer-readable recording medium.

In addition, a display device that is another aspect of the present disclosure includes:
a receiving unit that receives display area information, which is information for specifying an area being displayed on another user device in the entire data;
a display unit that displays an area corresponding to the state of the user out of the entire data and displays an area that is being displayed on another user device based on the display area information;
has
The display unit displays a region currently displayed on the user device indicated by the display instruction out of the entire data, based on the display instruction transmitted from the external device as a detection result of the predetermined condition.

In addition, a display method, which is another aspect of the present disclosure,
the user device
receiving display area information, which is information for specifying an area being displayed on another user device in the entire data;
displaying an area corresponding to the state of the user out of the entire data, and displaying an area being displayed on another user device based on the display area information;
When a display instruction transmitted from an external device as a result of detection of a predetermined condition is received, based on the received display instruction, an area currently being displayed on the user device indicated by the display instruction is displayed among the entire data. .

According to each configuration as described above, an information processing device, information processing method, recording medium, display device, and display method that enable smooth sharing of information in a system for sharing information such as image data are provided. can do

1 is a diagram illustrating a configuration example of an image sharing system according to a first embodiment of the present disclosure; FIG. 2 is a block diagram showing a configuration example of the on-site device shown in FIG. 1; FIG. 2 is a block diagram showing a configuration example of a relay device shown in FIG. 1; FIG. 2 is a block diagram showing a configuration example of a user device shown in FIG. 1; FIG. FIG. 4 is a diagram showing a display example of a screen display unit; FIG. 4 is a diagram showing a display example of a screen display unit; FIG. 4 is a diagram showing a display example of a screen display unit; 4 is a flowchart showing an operation example of a relay device; 4 is a flow chart showing an operation example of a user device; It is a figure which shows the hardware structural example of the information processing apparatus concerning 2nd Embodiment of this indication. It is a block diagram which shows the structural example of an information processing apparatus. It is a block diagram which shows the structural example of a display apparatus.

[First Embodiment]
A first embodiment of the present disclosure will be described with reference to FIGS. 1 to 9. FIG. FIG. 1 is a diagram showing a configuration example of an image sharing system 100. As shown in FIG. FIG. 2 is a block diagram showing a configuration example of the field device 200. As shown in FIG. FIG. 3 is a block diagram showing a configuration example of the relay device 300. As shown in FIG. FIG. 4 is a block diagram showing a configuration example of the user device 400. As shown in FIG. 5 to 7 are diagrams showing display examples of the screen display unit. FIG. 8 is a flow chart showing an operation example of the relay device 300 . FIG. 9 is a flow chart showing an operation example of the user device 400 .

In the first embodiment of the present disclosure, an image sharing system 100 in which a plurality of user devices 400 share image data acquired by a 360-degree camera of the field device 200 will be described. In the case of the image sharing system 100 according to the present embodiment, at least one of the entire image data acquired by the 360-degree camera, such as an image of an area extracted according to the line-of-sight direction of the user, in each user device 400 is selected according to the state of the user. view the region of the Further, as will be described later, in the case of the present embodiment, the image sharing system 100 detects predetermined conditions such as conditions regarding sound data. Then, the image sharing system 100 causes each user device 400 included in the image sharing system 100 to display the area of the image data displayed by the user corresponding to the detected condition.

FIG. 1 shows a configuration example of the image sharing system 100. As shown in FIG. Referring to FIG. 1 , image sharing system 100 includes field device 200 , relay device 300 and user device 400 . As shown in FIG. 1, image sharing system 100 includes a plurality of user devices 400 (user device 400-1, user device 400-2, . . . ). device 400). The image sharing system 100 may have a plurality of field devices 200 and relay devices 300 .

As shown in FIG. 1, the field device 200 and the relay device 300 are connected so as to be able to communicate with each other by wireless communication or the like. Also, the relay device 300 and the user device 400 are connected so as to be able to communicate with each other by wireless communication or the like.

The on-site device 200 is an information processing device that acquires image data. For example, the field device 200 has an imaging device such as a 360-degree camera, and acquires image data using the imaging device. The on-site device 200 has a microphone, a speaker, and the like, and is configured to be capable of inputting and outputting sound data representing human voices and the like. For example, the on-site device 200 is placed around a factory, store, or other site to be imaged, and is operated by a person on site.

FIG. 2 shows a configuration example of the on-site device 200. FIG. Referring to FIG. 2, the field device 200 has an image data acquisition unit 210, a sound data input/output unit 220, and a transmission/reception unit 230, for example. For example, the field device 200 has an arithmetic device such as a CPU and a storage device. For example, the on-site device 200 implements each of the above-described processing units by executing a program stored in a storage device by an arithmetic device.

The image data acquisition unit 210 acquires image data by, for example, transmitting a predetermined instruction to the imaging device. For example, the image data acquisition unit 210 acquires omnidirectional (or hemispherical) image data from a 360-degree camera that is an imaging device. Also, the image data acquisition unit 210 can store the acquired image data in the storage device of the field device 200 .

The sound data input/output unit 220 acquires sound data and outputs sound data by using a microphone, a speaker, or the like. For example, the sound data input/output unit 220 can acquire sound data such as voices uttered by people on site. Further, the sound data input/output unit 220 can output sound data received from the relay device 300 by the transmission/reception unit 230 described later.

The transmission/reception unit 230 transmits/receives various data to/from the relay device 300 . For example, the transmission/reception unit 230 transmits image data acquired by the image data acquisition unit 210 and sound data acquired by the sound data input/output unit 220 to the relay device 300, which is an external device. Further, the transmission/reception unit 230 receives sound data and the like from the relay device 300 .

The above is a configuration example of the on-site device 200 . The on-site device 200 may be composed of one information processing device, or may be composed of a plurality of communicatively connected devices such as a 360-degree camera, a speaker, and an information processing device. good.

It should be noted that the on-site device 200 may be configured so that the conversation destination for transmitting and receiving sound data can be set by the operation of the on-site person. In other words, the on-site device 200 may be configured to be able to have a conversation only with the user device 400 set by the operation of the on-site person. Note that even in a case where a conversation is performed only with the set user device 400, as will be described later, the sound data transmitted by the field device 200 and the user device 400 set as a conversation partner are The transmitted sound data can be configured to be transmitted to the user device 400 that is not set as a conversation partner.

The relay device 300 is an information processing device such as a server device that transmits and receives data between the field device 200 and the user device 400 . Also, as will be described later, the relay device 300 detects a predetermined condition. Then, the relay device 300 instructs each user device 400 to display the area of the image data displayed by the user corresponding to the detected condition.

3 shows a configuration example of the relay device 300. FIG. Referring to FIG. 3, the relay device 300 has, for example, a communication I/F section 310, a storage section 320, and an arithmetic processing section 330 as main components.

Note that FIG. 3 illustrates a case where the relay device 300 is realized by one information processing device. However, the function as the relay device 300 may be realized by a plurality of information processing devices that are communicably connected to each other, such as being realized on a cloud, for example.

The communication I/F unit 310 consists of a data communication circuit. Communication I/F section 310 performs data communication with external devices such as field device 200 and user device 400 connected via a communication line.

The storage unit 320 is a storage device such as a hard disk or memory. The storage unit 320 stores processing information and programs 324 required for various processes in the arithmetic processing unit 330 . The program 324 realizes various processing units by being read and executed by the arithmetic processing unit 330 . The program 324 is read in advance from an external device or recording medium via a data input/output function such as the communication I/F unit 310 and stored in the storage unit 320 . Main information stored in the storage unit 320 includes, for example, image information 321, sound information 322, display information 323, and the like.

The image information 321 is information including image data received from the field device 200 . The image information 321 is updated, for example, when the data receiving unit 331 receives image data from the field device 200 .

The sound information 322 is information including sound data received from the field device 200 or the user device 400 . The sound information 322 stores sound data for each field device 200 and user device 400 . In other words, the sound information 322 associates identification information for identifying the field device 200 and the user device 400 with sound data. The sound information 322 is updated, for example, when the data receiving unit 331 receives sound data from the field device 200 or the user device 400 .

The display information 323 indicates display area information that is information for specifying the area being displayed on the user device 400 in the entire image data. For example, the display information 323 includes display area information for each user device 400 . In other words, the display information 323 associates identification information for identifying the user device 400 with display area information. For example, the display information 323 is updated when the data receiving unit 331 acquires display area information from the user device 400 .

Note that the display area information is not specifically limited as long as the area being displayed on the user device 400 can be specified. For example, the display area information may be information corresponding to the direction of the user's line of sight or orientation, information indicating the area being displayed on the user device 400 such as coordinates, or any other information for specifying the area.

The arithmetic processing unit 330 has an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 330 reads the program 324 from the storage unit 320 and executes it, thereby realizing various processing units by cooperating the hardware and the program 324 . Main processing units realized by the arithmetic processing unit 330 include, for example, a data reception unit 331, a data transmission unit 332, a condition detection unit 333, a display instruction unit 334, and the like.

The data receiving unit 331 receives various data from the field device 200 and the user device 400 .

For example, the data receiving unit 331 receives image data from the field device 200 . Then, the data receiving section 331 stores the received image data in the storage section 320 as the image information 321 . The data receiving unit 331 also receives sound data from the field device 200 . Then, the data receiving section 331 stores the received sound data in the storage section 320 as the sound information 322 . At this time, the data receiving unit 331 stores the sound data so that the sound data received from the field device 200 can be identified, such as by associating the sound data with identification information for identifying the field device 200. can do

Also, for example, the data receiving unit 331 receives sound data from the user device 400 . Then, the data receiving section 331 stores the received sound data in the storage section 320 as the sound information 322 . At this time, the data receiving unit 331 can store sound data for each user device 400 , such as by associating the sound data with identification information for identifying the user device 400 . Also, the data receiving unit 331 receives display area information from the user device 400 . Then, data receiving section 331 stores the received display area information in storage section 320 as display information 323 . At this time, the data receiving unit 331 can store the display area information for each user device 400 , such as associating the display area information with identification information for identifying the user device 400 .

The data transmission unit 332 transmits various data to the field device 200 and the user device 400 . For example, the data transmission unit 332 transmits the received data to the user device 400 or the on-site device 200 each time it receives data from the on-site device 200 or the user device 400 .

For example, the data transmission unit 332 transmits sound data received from the user device 400 to the field device 200 . The data transmission unit 332 may transmit the sound data and the identification information in association with each other. It should be noted that the data transmission unit 332 may be configured to transmit only the sound data received from the preset user device 400 to the field device 200 when the conversation partner is set.

The data transmission unit 332 also transmits image data and sound data received from the field device 200 to the user device 400 . The data transmission unit 332 also transmits sound data received from another user device 400 to the user device 400 . The data transmission unit 332 may transmit the sound data and the identification information in association with each other. Also, the data transmission unit 332 can transmit display area information received from another user device 400 to the user device 400 . Note that the data transmission unit 332 may be configured to transmit sound data to all the user devices 400 regardless of whether or not a conversation partner is set.

The condition detection unit 333 detects a predetermined condition. For example, the condition detection unit 333 detects a condition according to the transmission/reception status of sound data. Also, the condition detection unit 333 identifies the user device 400 corresponding to the detected condition.

For example, the condition detection unit 333 detects a condition corresponding to the transmission/reception status of sound data according to the setting of the conversation partner. Specifically, for example, when the field device 200 sets a conversation between the field device 200 and a certain user device 400 , the condition detection unit 333 transmits and receives sound data between the field device 200 and the user device 400 . Detects that the condition that Then, the condition detection unit 333 identifies the user device 400 set as the conversation partner as the user device 400 corresponding to the detected condition.

Also, for example, the condition detection unit 333 detects a condition corresponding to the transmission/reception status of the sound data according to the status of the sound data included in the sound information 322 . Specifically, for example, if the condition detection unit 333 can determine that users or on-site users are having a conversation according to the state of the sound data, the on-site device 200 and the user device 400 or between the on-site device 200 and the user device 400 , that the condition that sound data is being transmitted and received between the user devices 400 is satisfied. Then, the condition detection unit 333 identifies the user device 400 determined to be having a conversation as the user device 400 corresponding to the detected condition. It should be noted that the condition detection unit 333 may determine whether or not it is possible to determine that the user is having a conversation using arbitrary means such as the amount or ratio of sound data per predetermined time. For example, the condition detection unit 333 may identify the user device 400 that is conversing with the field device 200 or the user device 400 that is conversing between users based on the ratio of the amount of sound data. In addition, when it is determined that no one is talking based on the amount of sound data, the condition detection unit 333 detects that the condition is that no one is in a conversation state, and determines that the pre-determined moderator or the like is satisfied. User device 400 or the like may be identified. In other words, the condition detection unit 333 may perform detection under the condition that no sound is detected (or the sound is below a predetermined level).

For example, as described above, the condition detection unit 333 detects conditions according to the transmission/reception status of sound data.

Note that the condition detection unit 333 may detect conditions other than those exemplified above. For example, the condition detection unit 333 may detect a condition that sound data including a specific keyword is being transmitted and received when a specific keyword is detected after characterizing the sound data. The condition detection unit 333 may detect conditions for sound data other than those exemplified above. Further, the condition detection unit 333 may detect a condition according to information other than sound data, such as PC button operation, mouse movement, line of sight, etc., to identify the user device.

The display instruction unit 334 instructs each user device 400 including the specified user device 400 to display the area being displayed on the user device 400 specified by the condition detection unit 333 in response to detection of the condition by the condition detection unit 333 . instruct against. The display instruction unit 334 may issue the above instruction to each user device 400 included in the image sharing system 100 other than the specified user device 400 .

For example, the display instruction unit 334 transmits a display instruction including identification information for identifying the user device 400 specified by the condition detection unit 333 to each user device 400 in response to detection of the condition by the condition detection unit 333 . do. For example, by transmitting such a display instruction, the display instruction unit 334 instructs each user device 400 to display the area being displayed on the specified user device 400 . Note that the display instruction may include display area information corresponding to the specified user device 400 in addition to the identification information. Further, when the condition detection unit 333 detects a condition that sound data is being transmitted/received between the user devices 400, the condition detection unit 333 identifies the plurality of user devices 400 that transmit/receive the sound data. Become. Therefore, the display instruction transmitted by the display instruction unit 334 may include a plurality of pieces of identification information.

The above is a configuration example of the relay device 300 . Among the functions of the relay device 300, the function as the data reception unit 331 and the function as the data transmission unit 332 may be realized using known technology. Further, the display instruction unit 334 may be configured to transmit the display instruction only to some of the user devices 400 included in the image sharing system 100 that satisfy a predetermined condition. In other words, the display instruction unit 334 may be configured to transmit a display instruction to user devices 400 that satisfy a predetermined condition among user devices 400 other than the specified user device 400 . In other words, the display instruction unit 334 does not necessarily have to transmit display instructions to all user devices 400 included in the image sharing system 100 . For example, the display instruction unit 334 may be configured to transmit a display instruction to user devices 400 belonging to a preset group among user devices 400 other than the specified user device 400 . Similarly, the data transmission unit 332 may be configured to transmit the display area information only to some of the user devices 400 included in the image sharing system 100 that satisfy a predetermined condition.

The user device 400 is an information processing device that displays at least a partial area of image data. For example, the user device 400 is an information processing device having a display device such as a head-mounted display, smart phone, or tablet. For example, the user device 400 has a sensor or the like, and can be configured to be able to detect the user's state such as the user's line-of-sight direction. Also, the user device 400 has a microphone, a speaker, and the like, and is configured to be capable of inputting and outputting sound data indicating a person's voice. For example, the user device 400 is owned by each user at a location different from the site, and is operated by the user. Note that at least some of the user devices 400 may exist on site.

4 shows a configuration example of the user device 400. FIG. Referring to FIG. 4, the user device 400 has main components such as a communication I/F unit 410, a screen display unit 420, an operation input unit 430, a sound input/output unit 440, and a storage unit 450. , and an arithmetic processing unit 460 .

The communication I/F unit 410 consists of a data communication circuit. Communication I/F section 410 performs data communication with an external device such as relay device 300 connected via a communication line.

The screen display unit 420 consists of a screen display device such as an LCD (Liquid Crystal Display). The screen display unit 420 can display various information stored in the storage unit 450, such as at least a partial area of the image data, according to an instruction from the arithmetic processing unit 460. FIG. The screen display unit 420 may be a display with a touch panel that functions as the operation input unit 430, or the like.

The operation input unit 430 consists of operation input devices such as a keyboard, mouse, and various sensors. The operation input unit 430 detects the operation of the user who has the user device 400 and outputs it to the arithmetic processing unit 460 .

The sound input/output unit 440 consists of sound input/output devices such as a microphone and a speaker. The sound input/output unit 440 acquires sound data and outputs sound data according to instructions from the arithmetic processing unit 460 . For example, the sound input/output unit 440 can acquire sound data such as voice uttered by the user. Also, the sound input/output unit 440 can output sound data and the like included in the sound information 452 described later.

The storage unit 450 is a storage device such as a hard disk or memory. The storage unit 450 stores processing information and programs 455 necessary for various processes in the arithmetic processing unit 460 . The program 455 realizes various processing units by being read into the arithmetic processing unit 460 and executed. The program 455 is read in advance from an external device or recording medium via a data input/output function such as the communication I/F unit 410 and stored in the storage unit 450 . Main information stored in the storage unit 450 includes, for example, image information 451, sound information 452, display information 453, instruction information 454, and the like.

The image information 451 is information including image data received from the field device 200 via the relay device 300 . The image information 451 is updated, for example, when the data receiving unit 461 receives image data from the relay device 300 .

The sound information 452 is information including sound data received from the field device 200 or the user device 400 via the relay device 300 . The sound information 452 stores sound data for each field device 200 and user device 400 . In other words, the sound information 452 associates identification information for identifying the field device 200 and the user device 400 with sound data. The sound information 452 is updated, for example, when the data receiving unit 461 receives sound data from the relay device 300 .

The display information 453 indicates display area information that is information for specifying the area being displayed on each user device 400 included in the image sharing system 100 . For example, the display information 453 includes display area information for each user device 400 . In other words, display information 453 associates identification information for identifying user device 400 with display area information. For example, the display information 453 is updated when the data receiving unit 461 acquires display area information from the relay device 300 .

The instruction information 454 indicates the display instruction received from the relay device 300. For example, the instruction information 454 is updated when a display instruction is received from the relay device 300 or the like.

The arithmetic processing unit 460 has an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 460 reads the program 455 from the storage unit 450 and executes it, thereby realizing various processing units by cooperating the hardware and the program 455 . Main processing units realized by the arithmetic processing unit 460 include, for example, a data reception unit 461, a data transmission unit 462, an image display unit 463, a display area information acquisition unit 464, a display area information transmission unit 465, and the like.

The data receiving unit 461 receives various data from the relay device 300 .

For example, the data receiving unit 461 receives image data from the relay device 300 . Then, the data receiving section 461 stores the received image data in the storage section 450 as the image information 451 . Also, the data receiving unit 461 receives sound data from the relay device 300 . Then, the data receiving section 461 stores the received sound data in the storage section 450 as the sound information 452 . At this time, the data receiving section 461 may store the sound data and the identification information in the storage section 450 in association with each other.

The data receiving unit 461 also receives display instructions from the relay device 300 . Then, data receiving section 461 stores the received display instruction in storage section 450 as instruction information 454 .

The data transmission unit 462 transmits various data to the relay device 300 . For example, the data transmission unit 462 transmits the acquired sound data to the relay device 300 every time it acquires the sound data. The data transmission unit 462 may transmit the sound data and the identification information in association with each other.

The image display unit 463 causes the screen display unit 420 to display at least a partial area of the image data included in the image information 451 . For example, the image display unit 463 can display a partial area according to the state of the user operating the user device 400, and display the area being displayed on another user device 400 based on the display information 453. I can.

For example, the image display unit 463 refers to the image information 451 and extracts an area corresponding to the user's state, such as the user's gaze direction detected by the sensor, from the image data. At this time, the image display unit 463 may extract the region by any method. Then, the image display unit 463 causes the screen display unit 420 to display the extracted area. In this manner, the image display unit 463 can cause the screen display unit 420 to display an area corresponding to the state of the user.

Note that the image display unit 463 may extract a region according to the user's selection state such as an operation on the operation input unit 430 instead of detecting the line-of-sight direction, or in addition to detecting the line-of-sight direction. The image display unit 463 may specify the user's state using any means and extract a region according to the specified state.

Also, the image display unit 463 can refer to the image information 451 and the display information 453 and cause the screen display unit 420 to display the area being displayed on the other user device 400 . In other words, the image display unit 463 can cause the screen display unit 420 to display the other person's display area, which is the area being displayed on any user device 400 . For example, the image display unit 463 may be configured to switch the user to be displayed on the screen display unit 420 according to the user's operation on the operation input unit 430 . Further, the screen display unit 463 may be configured so as to be able to switch the user to be displayed on the screen display unit 420 according to another user's operation on the other user device 400 . In other words, the screen display unit 463 may be configured to display the other person's display area corresponding to the common user in each of the user devices 400 included in the screen sharing system 100, or display the other person's display area corresponding to the user selected by the own device. It may be configured to display a corresponding others display area. In addition, switching of the user to be displayed may be performed by one or more predetermined user devices 400, or one or more user devices determined according to arbitrary conditions. 400 may be configured to do so.

In addition, the image display unit 463 can cause the screen display unit 420 to display the area being displayed on the user device 400 indicated by the display instruction, according to the display instruction received by the data reception unit 461 . In other words, the image display unit 463 can cause the screen display unit 420 to display the designated area, which is the area being displayed on the designated user device 400 . For example, when the data receiving unit 461 receives the display instruction, the image display unit 463 refers to the display information 453 to specify the display area information corresponding to the identification information included in the display instruction. Then, image display section 463 causes screen display section 420 to display the area indicated by the identified display area information. Alternatively, when display area information is included in the display instruction, image display section 463 causes screen display section 420 to display the area indicated by the display area information included in the display instruction.

Note that the image display unit 463 can cause the screen display unit 420 to display the instruction area instead of the other person's display area in response to the display instruction. The image display section 463 may cause the screen display section 420 to display the designation area together with the other person's display area.

In addition, in the present embodiment, the timing at which the image display unit 463 cancels the display according to the display instruction is not particularly limited. The image display unit 463 may cancel the display according to the display instruction at any timing.

FIGS. 5 to 7 show display examples on the screen display section 420 displayed by the image display section 463. FIG. For example, FIG. 5 shows a display example when no display instruction has been received. Referring to FIG. 5, the image display unit 463 causes the display unit to display the region extracted according to the state of the user. In addition, the image display unit 463 causes the other person's display area to be displayed on the other person's display area. For example, as described above, the image display unit 463 can display the other person's display area together with the area extracted according to the state of the user when no display instruction is received. It should be noted that it may be configured such that the user can select whether or not to display the other person's display area. When the user indicates that the other person's display area should not be displayed, the image display unit 463 may display the extracted area on the entire screen display unit 420 according to the state of the user.

Also, FIG. 6 shows a display example in a state in which a display instruction is received. Referring to FIG. 6, the image display unit 463 causes the display unit to display the region extracted according to the state of the user. In addition, the image display unit 463 causes the other person's display unit to display the designation area. For example, as described above, upon receiving a display instruction, the image display unit 463 can display the instruction area instead of the other person's display area. Note that even if the user has selected not to display the other person's display area, upon receiving the display instruction, image display unit 463 displays the instruction area as shown in FIG.

Also, FIG. 7 shows a display example when a display instruction includes a plurality of pieces of identification information. Referring to FIG. 7, when the display instruction includes a plurality of pieces of identification information, the image display section 463 can divide the other person's display section into a plurality of sections and display an instruction area on each of the divided sections. When the divided display as shown in FIG. 7 is performed, at least one of the divided regions can be selected according to the user's selection action such as the user's operation on the operation input unit 430 or the user's line-of-sight direction detection result. It can be configured as Further, when the user selects, only the selected area of the instruction area may be displayed on the other person's display section as shown in FIG. 6, for example. In other words, the image display section 463 may cause the screen display section 420 to display only some of the plurality of instruction areas according to the user's selection action.

It should be noted that FIGS. 5 to 7 show display examples, and the display on the screen display unit 420 is not limited to the cases shown in FIGS. For example, the positions and sizes of the display section and the other person's display section may be changed arbitrarily.

The display area information acquisition unit 464 acquires display area information that is information for specifying the area displayed by the image display unit 463 according to the state of the user. Then, display area information acquisition section 464 stores the acquired display area information in storage section 450 as display information 453 .

For example, the display area information acquisition unit 464 obtains, as the display area information, information corresponding to the line-of-sight direction of the user and the orientation of the user, information indicating the area being displayed on the user device 400 such as coordinates, and other information for specifying the area. , at least one of The display area information acquisition unit 464 may acquire the display area information at arbitrary timing such as predetermined intervals.

The display area information transmission unit 465 transmits the display area information acquired by the display area information acquisition unit 464 to the relay device 300 . For example, the display area information transmitting unit 465 can transmit the display area information to the relay apparatus 300 in association with the identification information of its own apparatus. For example, the display area information transmission unit 465 can transmit the display area information to the relay device 300 every time the display area information acquisition unit 464 acquires the display area information.

The above is a configuration example of the user device 400 . Next, an operation example of the relay device 300 and the user device 400 will be described with reference to FIGS. 8 and 9. FIG.

FIG. 8 shows an operation example of the relay device 300. FIG. Referring to FIG. 8, the condition detection unit 333 confirms whether or not a preset condition is satisfied (step S101). If the condition is not satisfied (step S101, No), the condition detection unit 333 continues detection of the condition. On the other hand, if the condition is satisfied (step S101, Yes), the condition detection unit 333 identifies the user device 400 corresponding to the detected condition (step S102).

The display instruction unit 334 instructs the user device 400 to display the area being displayed on the user device 400 specified by the condition detection unit 333 (step S103). In other words, the display instruction unit 334 transmits a display instruction to the user device 400 . For example, the display instruction unit 334 transmits display instructions to all user devices 400 included in the image sharing system 100 . The display instruction unit 334 may be configured to transmit a display instruction to user devices 400 satisfying a predetermined condition among all user devices 400 included in the image sharing system 100 .

The above is an operation example of the relay device 300 . Next, an operation example of the user device 400 will be described with reference to FIG.

Referring to FIG. 9, the image display unit 463 displays the other person's display area, which is the area being displayed on any user device 400, when no display instruction is received (step S201, No) (step S203). ). The user device 400 to be displayed may be selectable according to a user's operation or the like.

On the other hand, when the display instruction is received (step S201, Yes), the image display unit 463 displays the designated area, which is the area being displayed on the user device 400 designated by the display instruction (step S202). For example, the image display unit 463 displays the designated area instead of the other person's display area. The image display unit 463 may display the designation area together with the other person's display area.

The above is an operation example of the user device 400 .

As described above, the relay device 300 has the condition detection section 333 and the display instruction section 334 . According to such a configuration, the display instruction unit 334 instructs each predetermined user device to display the area being displayed on the user device 400 specified by the condition detection unit 333 in response to detection of a condition by the condition detection unit 333 . 400 can be instructed. As a result, in the user device 400 that has received the display instruction, it is possible to display the area that is being displayed in the user device 400 instructed by the display instruction. As a result, in an image sharing system for sharing image data, it becomes possible for the user, who is the originator of the conversation, to check the area being checked by the other user device 400 as well, thereby facilitating smooth sharing of information between sharers. It is possible to do

In addition, the image display unit 463 of the user device 400 is configured to display the area being displayed on the user device 400 indicated by the display instruction, according to the display instruction. As a result, the user operating the user device 400 can quickly check the area being checked by the user who is the originator of the conversation, and information can be smoothly shared with the originator of the conversation. becomes possible.

As an example, the present invention can be utilized in the field of education, such as when educating about work details in the manufacturing process, and in the field of business, such as when multiple users remotely check and review display examples in a store. For example, according to the present invention, it is possible to cause another user device 400 to display an area being checked by a teacher such as an expert in response to detection of a condition. As a result, it becomes possible for the user operating the other user device 400 to easily confirm what the teacher is looking at and checking or speaking, and it is possible to more easily share know-how. Become. Further, according to the present invention, it is possible to cause another user device 400 to display the region being checked by the user in conversation in response to the detection of the condition. As a result, it becomes possible for a user operating another user device 400 to easily confirm from which point of view the conversation is based, and it is possible to smoothly share information. The present invention may be utilized in situations other than those exemplified above.

In addition, in this embodiment, the case of sharing omnidirectional image data acquired by a 360-degree camera is exemplified. However, the present invention may be applied to shared systems other than those exemplified in this embodiment. For example, the present invention may be applied to an image sharing system capable of displaying a portion of two-dimensional image data on each user device, such as by each zooming the image data. Also, the present invention may be applied to an information sharing system other than image data that displays at least part of information on a user device. For example, the present invention may be applied to a system for sharing information such as a website that transmits display information indicating the website being browsed to a relay device.

[Second embodiment]
Next, a second embodiment of the present invention will be described with reference to FIGS. 10 to 12. FIG. In the second embodiment of the present invention, an outline of the configuration of an information processing device 500 and a display device 600 connected to a plurality of user devices that display at least a partial area of all data will be described.

FIG. 10 shows a hardware configuration example of the information processing device 500 . Referring to FIG. 10, the information processing apparatus 500 has the following hardware configuration as an example.
- CPU (Central Processing Unit) 501 (arithmetic unit)
- ROM (Read Only Memory) 502 (storage device)
・RAM (Random Access Memory) 503 (storage device)
Program group 504 loaded into RAM 503
- Storage device 505 for storing program group 504
A drive device 506 that reads and writes a recording medium 510 outside the information processing device
- A communication interface 507 that connects to a communication network 511 outside the information processing apparatus
An input/output interface 508 for inputting/outputting data
- A bus 509 connecting each component

Also, the information processing apparatus 500 can realize the functions of the detection unit 521 and the instruction unit 522 shown in FIG. The program group 504 is stored in the storage device 505 or the ROM 502 in advance, for example, and is loaded into the RAM 503 or the like by the CPU 501 as necessary and executed. The program group 504 may be supplied to the CPU 501 via the communication network 511 or may be stored in the recording medium 510 in advance, and the drive device 506 may read the program and supply it to the CPU 501 .

Note that FIG. 10 shows a hardware configuration example of the information processing device 500 . The hardware configuration of the information processing device 500 is not limited to the case described above. For example, the information processing apparatus 500 may be composed of part of the above-described configuration, such as not having the drive device 506 .

The detection unit 521 detects a predetermined condition. For example, the detection unit 521 detects conditions according to the state of transmission and reception of sound data, such as whether or not it is determined that the user is having a conversation based on the sound data.

The instruction unit 522 instructs at least some of the connected user devices to display the area displayed by the user device corresponding to the condition according to the detection result of the detection unit 521 . For example, the instruction unit 522 can perform the above instruction by transmitting a display instruction to instruct the user device to be displayed.

As described above, the information processing device 500 has the detection unit 521 and the instruction unit 522 . According to such a configuration, the instruction unit 522 causes at least one of the connected user devices to display the area displayed by the user device corresponding to the condition according to the detection result of the detection unit 521. You can give instructions to the department. As a result, the user device that has received the instruction can display the area that is being displayed on the target user device. This enables smooth sharing of information in an image sharing system that shares image data.

The information processing device 500 described above can be realized by installing a predetermined program in the information processing device 500 . Specifically, a program that is another aspect of the present invention detects a predetermined condition in an information processing device 500 connected to a plurality of user devices that display at least a partial area of all data, A program for realizing a process of instructing at least some of the connected user devices to display an area displayed by a user device corresponding to a condition according to a detection result.

Further, in the information processing method executed by the information processing device 500 described above, the information processing device 500 connected to a plurality of user devices displaying at least a partial area of the entire data detects a predetermined condition. Then, according to the detected result, at least some of the connected user devices are instructed to display the area displayed by the user device corresponding to the condition.

Even the invention of the program (or recording medium) or information processing method having the configuration described above achieves the above-described object of the present invention because it has the same actions and effects as the information processing apparatus 500. can do

Also, the present invention can achieve the object of the present invention for the same reason even with a display device 600 having a configuration as shown in FIG. Note that the hardware configuration of the display device 600 may be the same as that of the information processing device 500 described with reference to FIG. 10 . Therefore, description of the hardware configuration of the display device 600 is omitted.

Referring to FIG. 12, the display device 600 can realize the functions of the reception unit 621 and the display unit 622 by having the CPU acquire a group of programs and have the CPU execute them.

The receiving unit 621 receives display area information, which is information for specifying an area being displayed on another user device, among the entire data.

The display unit 622 displays an area corresponding to the state of the user out of the entire data, and displays an area being displayed on another user device based on the display area information. Further, when the display unit 622 receives a display instruction transmitted from an external device as a detection result of a predetermined condition, the display unit 622 displays the area currently displayed on the user device indicated by the display instruction out of the entire data based on the display instruction. Let

Even if the display device 600 having such a configuration, the display method performed by the display device 600, or the program (or recording medium) for realizing the display method on the display device 600, the present invention can be used as described above. can achieve the purpose of

<Appendix>
Some or all of the above embodiments may also be described as the following appendices. An outline of an information processing apparatus and the like according to the present invention will be described below. However, the present invention is not limited to the following configurations.

(Appendix 1)
An information processing device connected to a plurality of user devices that display at least a partial area of all data,
a detection unit that detects a predetermined condition;
an instruction unit that instructs at least some of the connected user devices to display the area displayed by the user device corresponding to the condition according to the detection result of the detection unit;
An information processing device.
(Appendix 2)
The information processing apparatus according to appendix 1, wherein the detection unit detects a condition according to a transmission/reception state of the sound data.
(Appendix 3)
The information processing apparatus according to appendix 1 or appendix 2, wherein the detection unit detects that a predetermined condition is satisfied when it can be determined that a conversation is being held according to the transmission/reception status of the sound data.
(Appendix 4)
It is connected to the field device that acquires the data,
The detection unit detects that a predetermined condition is satisfied when it can be determined that the field device and the user device are having a conversation;
The instruction unit causes at least one of the connected user devices to display an area displayed by the user device that can be determined to be having a conversation with the field device according to the detection result of the detection unit. The information processing apparatus according to any one of appendices 1 to 3, wherein the instruction is given to the unit.
(Appendix 5)
a receiving unit that receives display area information for specifying the area being displayed on the user device;
a transmitting unit configured to transmit the display area information received by the receiving unit to at least some of the connected user devices;
The information processing apparatus according to any one of appendices 1 to 4.
(Appendix 6)
6. The information processing apparatus according to appendix 5, wherein the instruction unit issues an instruction including the display area information to at least some of the connected user devices.
(Appendix 7)
An information processing device connected to a plurality of user devices that display at least a partial area of all data,
detecting a predetermined condition;
An information processing method for instructing at least a part of connected user devices to display an area displayed by a user device corresponding to a condition according to a detection result.
(Appendix 8)
An information processing device connected to a plurality of user devices that display at least a partial area of all data,
detecting a predetermined condition;
A program for realizing a process of instructing at least some of the connected user devices to display the area displayed by the user device corresponding to the condition according to the detection result.
(Appendix 9)
a receiving unit that receives display area information, which is information for specifying an area being displayed on another user device in the entire data;
a display unit that displays an area corresponding to the state of the user out of the entire data and displays an area that is being displayed on another user device based on the display area information;
has
The display unit displays, based on a display instruction transmitted from an external device as a detection result of a predetermined condition, a region being displayed on the user device indicated by the display instruction out of the entire data.
(Appendix 10)
the user device
receiving display area information, which is information for specifying an area being displayed on another user device in the entire data;
displaying an area corresponding to the state of the user out of the entire data, and displaying an area being displayed on another user device based on the display area information;
A display method comprising: upon receiving a display instruction transmitted from an external device as a result of detection of a predetermined condition, displaying an area currently displayed on the user device indicated by the display instruction out of the entire data, based on the received display instruction.

The programs described in each of the above embodiments and supplementary notes are stored in a storage device or recorded in a computer-readable recording medium. For example, the recording medium is a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

Although the present invention has been described with reference to the above-described embodiments, the present invention is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

In addition, the present invention enjoys the benefit of the priority claim based on the patent application of Japanese Patent Application No. 2021-088051 filed on May 26, 2021 in Japan. The contents are hereby incorporated by reference in their entirety.

100 image sharing system 200 field device 210 image data acquisition unit 220 sound data input/output unit 230 transmission/reception unit 300 relay device 310 communication I/F unit 320 storage unit 321 image information 322 sound information 323 display information 324 program 330 arithmetic processing unit 331 data Reception unit 332 Data transmission unit 333 Condition detection unit 334 Display instruction unit 400 User device 410 Communication I/F unit 420 Screen display unit 430 Operation input unit 440 Sound input/output unit 450 Storage unit 451 Image information 452 Sound information 453 Display information 454 Instruction information 455 program 460 arithmetic processing unit 461 data reception unit 462 data transmission unit 463 image display unit 464 display area information acquisition unit 465 display area information transmission unit 500 information processing device 501 CPU
502 ROMs
503 RAM
504 program group 505 storage device 506 drive device 507 communication interface 508 input/output interface 509 bus 510 recording medium 511 communication network 521 detection unit 522 instruction unit 600 display device 621 reception unit 622 display unit

Claims

An information processing device connected to a plurality of user devices that display at least a partial area of all data,
a detection unit that detects a predetermined condition;
an instruction unit that instructs at least some of the connected user devices to display the area displayed by the user device corresponding to the condition according to the detection result of the detection unit;
An information processing device.
The information processing apparatus according to claim 1, wherein the detection unit detects a condition according to a transmission/reception state of the sound data.
3. The information processing apparatus according to claim 1, wherein the detection unit detects that a predetermined condition is satisfied when it can be determined that the person is having a conversation according to the transmission/reception status of the sound data.
It is connected to the field device that acquires the data,
The detection unit detects that a predetermined condition is satisfied when it can be determined that the field device and the user device are having a conversation;
The instruction unit causes at least one of the connected user devices to display an area displayed by the user device that can be determined to be having a conversation with the field device according to the detection result of the detection unit. The information processing apparatus according to any one of claims 1 to 3, wherein an instruction is given to a department.
a receiving unit that receives display area information for specifying the area being displayed on the user device;
a transmitting unit configured to transmit the display area information received by the receiving unit to at least some of the connected user devices;
The information processing apparatus according to any one of claims 1 to 4.
6. The information processing apparatus according to claim 5, wherein the instruction unit issues an instruction including the display area information to at least some of the connected user devices.
An information processing device connected to a plurality of user devices that display at least a partial area of all data,
detecting a predetermined condition;
An information processing method for instructing at least a part of connected user devices to display an area displayed by a user device corresponding to a condition according to a detection result.
An information processing device connected to a plurality of user devices that display at least a partial area of all data,
detecting a predetermined condition;
A program is recorded for executing the process of instructing at least some of the connected user devices to display the area displayed by the user device corresponding to the conditions according to the detection result. , a computer-readable recording medium.
a receiving unit that receives display area information, which is information for specifying an area being displayed on another user device in the entire data;
a display unit that displays an area corresponding to the state of the user out of the entire data and displays an area that is being displayed on another user device based on the display area information;
has
The display unit displays, based on a display instruction transmitted from an external device as a detection result of a predetermined condition, a region being displayed on the user device indicated by the display instruction out of the entire data.
the user device
receiving display area information, which is information for specifying an area being displayed on another user device in the entire data;
displaying an area corresponding to the state of the user out of the entire data, and displaying an area being displayed on another user device based on the display area information;
A display method comprising: upon receiving a display instruction transmitted from an external device as a detection result of a predetermined condition, displaying an area currently displayed on the user device indicated by the display instruction out of the entire data, based on the received display instruction.