JP2014204412A - Information processing device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to more easily contact a desired person by using intuitive operation.SOLUTION: There is provided an information processing device comprising: an acquisition unit for acquiring a position designated by a user in an imaged picture in real space on a display screen including the imaged picture; a selection unit for selecting one or more objects from a plurality of objects, each of which corresponds to any position in the imaged picture, on the basis of the acquired position; and a display control unit for making a display device for displaying the display screen display information on each of two or more objects when the two or more objects are selected from the plurality of objects. The selection unit selects one object designated by the user from the two or more objects. When the one object is selected, communication identification information corresponding to the one object is acquired.

Description

  The present invention relates to an information processing apparatus and a program.

  In recent years, softphones implemented with application software have become popular in place of telephones implemented with hardware as in the past. Since the softphone is implemented by software, it is possible to relatively easily realize addition of functions to the softphone, cooperation between the softphone and other application software, and the like. Therefore, various applied technologies related to softphones have been proposed.

  For example, in Patent Document 1, when a person's face shown in the captured image is touched on a display screen of a captured image obtained from a camera, the person's face is identified by face recognition, and the telephone number of the person is set. A technique for obtaining and making a call to the person is disclosed.

JP 2007-208863 A

  However, in the technique disclosed in Patent Document 1, a call to the person cannot be made unless the position of the person's face is touched on the display screen. Therefore, for example, when the display screen itself is small, or when the person shown on the display screen is small, it is difficult to touch the position of the person's face, and it is difficult to make a call to the person. Can be. In order to reduce the accuracy of touch, for example, when the face closest to the touch position among the faces shown in the captured image is identified by face recognition, for example, the desired person and another person are close to each other. The face of the other person may be specified.

  Therefore, it is desirable to provide a mechanism that makes it possible to contact a desired person more easily through an intuitive operation.

  According to the present invention, an acquisition unit that acquires a position in the captured image specified by a user on a display screen that includes a captured image of real space, and any one of the captured images based on the acquired position A selection unit that selects one or more objects from a plurality of objects corresponding to positions, and a display that displays the display screen when two or more objects are selected from the plurality of objects. An information processing apparatus is provided that includes a display control unit that causes the apparatus to display information regarding each of the two or more objects. The selection unit selects one object specified by the user from the two or more objects. When the one object is selected, communication identification information corresponding to the one object is acquired.

  The two or more objects include a first object and a second object, and the information regarding each of the two or more objects includes a position in the captured image corresponding to the first object. An image of a first partial area in the captured image including the image and a second partial area in the captured image including a position in the captured image corresponding to the second object may be included.

  The display control unit may cause the display device to display the image of the first partial area and the image of the second partial area in parallel.

  The image of the first partial area is an enlarged image in which the first partial area is enlarged, and the image of the second partial area is enlarged in which the second partial area is enlarged. It may be an image.

  The image of the first partial area is an enlarged image obtained by enlarging the first partial area at a first enlargement ratio, and the image of the second partial area is the second part. An enlarged image in which the region is enlarged at the second enlargement ratio may be used.

  Further, the display control unit causes the display device to include a first boundary sign indicating a boundary between the image of the first partial region and a region other than the image, and the second partial region. A second boundary sign indicating a boundary between the image and a region other than the image may be displayed.

  The first boundary sign may include information related to the first object, and the second boundary sign may include information related to the second object.

  Further, the information regarding the first object includes first identification information for identifying the first object, and the information regarding the second object is for identifying the second object. Second identification information may be included.

  The information regarding each of the two or more objects may include identification information for identifying each of the two or more objects.

  The display control unit may cause the display device to display a sign for allowing the user to designate one object from the two or more objects.

  The one or more objects may be an object corresponding to a position in an area corresponding to the acquired position among the plurality of objects.

  The acquisition unit acquires one position in the captured image specified by the user on the display screen, and the one or more objects are the one acquired among the plurality of objects. It may be an object corresponding to a position in an area of a predetermined shape including the position.

  Further, the acquisition unit acquires two or more positions in the captured image specified by the user on the display screen, and the one or more objects are acquired from the plurality of objects. It may be an object corresponding to a position in an area formed based on two or more positions.

  The selection unit does not select any object when none of the positions corresponding to the plurality of objects is within the region corresponding to the acquired position, and the display control unit When no object is selected, a sign for causing the user to specify a position in the captured image may be displayed on the display screen on the display device.

  Further, the identification information for communication may be identification information for communication in a dedicated network.

  Further, the identification information for communication may be identification information for communication via a public network.

  The public network may include a mobile communication network.

  The display device may be a display surface of a touch panel, and the position in the captured image may be specified by a user touch on the touch panel.

  According to the present invention, the computer is configured to acquire the position based on the acquired position and the acquisition unit that acquires the position in the captured image specified by the user on the display screen including the captured image of the real space. The selection unit that selects one or more objects from a plurality of objects corresponding to any position of the image, and the display when two or more objects are selected from the plurality of objects A program for causing a display device that displays a screen to function as a display control unit that displays information on each of the two or more objects is provided. The selection unit selects one object specified by the user from the two or more objects. When the one object is selected, communication identification information corresponding to the one object is acquired.

  As described above, according to the present invention, it is possible to more easily contact a desired person through an intuitive operation.

It is explanatory drawing which shows an example of a schematic structure of the information processing system which concerns on one Embodiment. It is explanatory drawing for demonstrating the specific example of the terminal device which concerns on one Embodiment. It is a block diagram which shows an example of the hardware constitutions of the terminal device which concerns on one Embodiment. It is a block diagram which shows an example of a function structure of the terminal device which concerns on one Embodiment. It is explanatory drawing for demonstrating the example of acquisition of the position in the captured image contained in a display screen. It is explanatory drawing for demonstrating the example of the position which a some object (person) respond | corresponds respectively. It is explanatory drawing for demonstrating the example of the object selected when the position corresponding to one of objects and the position designated by the user correspond. It is explanatory drawing for demonstrating the example of the area | region for selecting an object. It is explanatory drawing for demonstrating the example of the object selected when the position designated by the user does not correspond to any position which any object respond | corresponds. It is explanatory drawing for demonstrating an example of the case where no object is selected even if the position in a captured image is designated by the user. It is explanatory drawing for demonstrating an example of the display screen displayed when one object is selected. It is explanatory drawing for demonstrating an example of the display screen displayed when two or more objects are selected. It is the 1st explanatory view for explaining an example of the generation process of the display screen containing the image of the 1st partial field, and the image of the 2nd partial field. It is the 2nd explanatory view for explaining an example of the generation process of the display screen containing the image of the 1st partial field, and the image of the 2nd partial field. It is the 3rd explanatory view for explaining an example of the generation process of the display screen containing the image of the 1st partial field, and the image of the 2nd partial field. It is the 4th explanatory view for explaining an example of the generation process of the display screen containing the image of the 1st partial field, and the image of the 2nd partial field. It is explanatory drawing for demonstrating an example of an object designation | designated promotion label | marker. It is explanatory drawing for demonstrating an example of a position designation | designated promotion label | marker. It is explanatory drawing for demonstrating an example of the display screen for telephone calls. It is a block diagram which shows an example of the software structure of the terminal device which concerns on one Embodiment. It is explanatory drawing for demonstrating the example of a menu screen. It is an example of the user interface of the softphone displayed on a screen. It is explanatory drawing for demonstrating the example of the display screen of a super-realistic client. It is explanatory drawing for demonstrating the example of the display screen of a telephone directory. It is explanatory drawing for demonstrating the example of a representative camera. It is explanatory drawing for demonstrating the example of the display screen of the telephone directory containing presence information. It is explanatory drawing for demonstrating an example of the information processing in the terminal device which concerns on one Embodiment. It is explanatory drawing for demonstrating an example of the information processing for provision and acquisition of the captured image according to presence information. It is explanatory drawing for demonstrating the example of acquisition of the position in the captured image contained in a display screen according to a 1st modification. It is explanatory drawing for demonstrating the 1st example of the display screen displayed according to a 3rd modification when two or more objects are selected. It is explanatory drawing for demonstrating the 2nd example of the display screen displayed according to a 3rd modification when two or more objects are selected. It is explanatory drawing for demonstrating the example of the information regarding the object contained in a boundary sign. FIG. 10 is an explanatory diagram for explaining an example of a display screen displayed according to a fourth modification when two or more objects are selected.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  Thereafter, << 1. Schematic configuration of information processing system >>, << 2. Configuration of terminal device >>, << 3. Flow of processing >>, << 4. Embodiments of the present invention will be described in the order of modification >>.

<< 1. Schematic configuration of information processing system >>
First, a schematic configuration of an information processing system according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is an explanatory diagram illustrating an example of a schematic configuration of an information processing system according to the present embodiment. Referring to FIG. 1, the information processing system exists over a plurality of locations, for example. In this example, the information processing system exists across the center office 10 and the satellite office 20 (or home office 20). The center office 10 is a relatively large office, and the satellite office 20 (or home office 20) is a relatively small office.

  In the center office 10, the information processing system includes a camera 11, a microphone 13, a sensor 15, a media distribution server 17, an information management server 18, and a LAN (Local Area Network) 19. The information processing system includes a terminal device 100, a display 21, and a LAN 23 in the satellite office 20 (or home office 20). The information processing system further includes a PBX (Private Branch eXchange) 40.

(Camera 11)
The camera 11 images a subject in the direction in which the camera 11 is facing (that is, the imaging direction). A plurality of cameras 11 are installed in the center office 10. Then, each installed camera 11 images a part or the whole of the center office from each installation position. Thus, in the information processing system, the center office is imaged from various positions. The captured image generated through the camera 11 may be a still image or a moving image (that is, a video).

  Moreover, the camera 11 can change direction automatically, for example. Furthermore, the camera 11 has a zoom function, for example. The zoom function may be an optical zoom function or a digital zoom function.

  Further, the camera 11 may be capable of changing the position. As an example, the camera 11 may be movable by a dolly. That is, the camera 11 may be movable along the rail. In this case, the camera 11 may move back and forth by controlling the motor for moving along the rail. As a result, even one camera 11 can generate a captured image in which the subject is captured from different positions.

  Further, when the camera 11 can change the position, the zoom function may be a zoom function by changing the position of the camera 11. As an example, the zoom function may be a dolly zoom function. For example, zooming in may be performed by moving the camera 11 toward the subject, and zooming out by moving the camera 11 in a direction away from the subject. Note that the zoom by dolly does not need to be a precise zoom like the optical zoom or the digital zoom. For example, in zooming in, it is only necessary to generate a captured image that shows a larger subject, and in zooming out, it is only necessary to generate a captured image that shows a smaller subject.

(Microphone 13)
The microphone 13 collects sound around the microphone 13. A plurality of microphones 13 are installed in the center office 10. And each installed microphone 13 collects the sound around each installation position in the center office. In this way, in the information processing system, sounds at various positions in the center office 10 are collected.

(Sensor 15)
The sensor 15 can include various types of sensors. For example, the sensor 15 includes a seat sensor that determines whether a person is in the seat. The seat sensor is installed in each seat and determines whether a person is sitting in each seat. The seat sensor is, for example, any sensor that can detect pressing.

(Media distribution server 17)
The media distribution server 17 distributes media (for example, audio, captured images, etc.) to the terminal device 100 in response to a request. For example, the media distribution server 17 acquires a captured image from the camera 11, acquires sound from the microphone 13, and distributes these media to the terminal device 100.

(Information management server 18)
The information management server 18 manages various information used in the information processing system. That is, the information management server 18 stores the various information and updates the various information in a timely manner.

  For example, the information management server 18 manages parameters regarding the camera 11, the microphone 13, and the sensor 15. Specifically, for example, the information management server 18 stores and updates information such as an installation position of the camera 11, an imaging direction (for example, a direction perpendicular to the camera lens), a zoom ratio, and the like as parameters of the camera 11. .

(LAN19)
The LAN 19 is a network that connects each device in the center office 10. The LAN 19 connects each device in the center office 10 and a device outside the center office 10 via the external network 30. The external network 30 includes the Internet, for example.

(Terminal device 100)
The terminal device 100 is used by a user. For example, the terminal device 100 provides a user with a function for performing communication such as telephone and e-mail. In addition, the terminal device 100 can be implemented as various types of devices. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 2 is an explanatory diagram for describing a specific example of the terminal device according to the present embodiment. Referring to FIG. 2, as an example, the terminal device 100A is a tablet terminal. As another example, the terminal device 100B is a smartphone. As yet another example, the terminal device 100C is a telephone with a display. For example, the telephone is provided with a soft phone. Note that specific examples of the terminal device 100 are not limited to these examples. As an example, the terminal device 100 may be a PC (Personal Computer).

(Display 21)
The display 21 displays one of the screens. For example, the display 21 displays a screen including a captured image generated through the camera 11. Thereby, many persons including the user of the terminal device 100 can see the state of the center office 10 via the display 21.

  Further, the display 21 outputs any sound, for example. For example, the display 21 outputs sound collected by the microphone 13. Thereby, many persons including the user of the terminal device 100 can hear the sound of the center office 10 via the display 21.

(LAN23)
The LAN 23 is a network that connects devices in the satellite office 20 (or the home office 20). The LAN 23 connects each device in the satellite office 20 to a device outside the satellite office 20 via the external network 30.

(PBX40)
The PBX 40 enables communication between devices via the external network 30. PBX40 is, for example, H.264. It operates according to H.323 or SIP (Session Initiation Protocol).

  Specifically, for example, the PBX 40 stores identification information for communication (for example, a telephone number) and an IP (Internet Protocol) address in association with each other. In response to the request, the PBX 40 converts the communication identification information into an IP address, and provides the IP address to the request source.

  The PBX 40 may be connected to the LAN 19 or the LAN 23.

<< 2. Terminal device configuration >>
Next, an example of the configuration of the terminal device 100 according to the present embodiment will be described with reference to FIGS.

<2.1. Hardware configuration>
First, an example of the hardware configuration of the terminal device 100 according to the present embodiment will be described with reference to FIG. FIG. 3 is a block diagram illustrating an example of a hardware configuration of the terminal device 100 according to the present embodiment. Referring to FIG. 3, a terminal device 100 includes a CPU (Central Processing Unit) 801, a ROM (Read Only Memory) 803, a RAM (Random Access Memory) 805, a bus 807, a storage device 809, a communication interface 811, a camera 813, a microphone. 815, a speaker 817, and a touch panel 820.

  The CPU 801 executes various processes in the terminal device 100. The ROM 803 stores a program and data for causing the CPU 801 to execute processing in the terminal device 100. The RAM 805 temporarily stores programs and data when the CPU 801 executes processing.

  A bus 807 connects the CPU 801, the ROM 803, and the RAM to each other. A storage device 809, a communication interface 811, a camera 813, a microphone 815, a speaker 817, and a touch panel 820 are further connected to the bus 807. The bus 807 includes, for example, a plurality of types of buses. As an example, the bus 807 includes a high-speed bus that connects the CPU 801, the ROM 803, and the RAM 805, and one or more other buses that are slower than the high-speed bus.

  The storage device 809 stores data to be temporarily or permanently stored in the terminal device 100. The storage device 809 may be, for example, a magnetic storage device such as a hard disk, or an EEPROM (Electrically Erasable and Programmable Read Only Memory), a flash memory, a MRAM (Magnetoresistive Random Access Memory). Further, non-volatile memories such as FeRAM (Ferroelectric Random Access Memory) and PRAM (Phase change Random Access Memory) may be used.

  The communication interface 811 is a communication unit included in the terminal device 100 and communicates with an external device via a network (or directly). The communication interface 811 may be an interface for wireless communication. In this case, for example, a communication antenna, an RF circuit, and other communication processing circuits may be included. The communication interface 811 may be an interface for wired communication. In this case, for example, a LAN terminal, a transmission circuit, and other communication processing circuits may be included.

  The camera 813 images a subject. The camera 813 includes, for example, an optical system, an image sensor, and an image processing circuit.

  The microphone 815 collects ambient sounds. The microphone 815 converts ambient sound into an electric signal, and converts the electric signal into digital data.

  The speaker 817 outputs sound. The speaker 817 converts digital data into an electric signal, and converts the electric signal into sound.

  The touch panel 820 includes a touch detection surface 821 and a display surface 823.

  Touch detection surface 821 detects a touch position on touch panel 820. More specifically, for example, when the user touches the touch panel 820, the touch detection surface 821 senses the touch, generates an electrical signal corresponding to the position of the touch, and converts the electrical signal to the touch position. Convert to information. The touch detection surface 821 can be formed according to an arbitrary touch detection method such as a capacitance method, a resistance film method, and an optical method, for example.

  The display surface 823 displays an output image (that is, a display screen) from the terminal device 100. The display surface 823 can be realized using, for example, a liquid crystal, an organic EL (Organic Light-Emitting Diode: OLED), a CRT (Cathode Ray Tube), or the like.

<2.2. Functional configuration>
Next, an example of a functional configuration of the terminal device 100 according to the present embodiment will be described with reference to FIGS. FIG. 4 is a block diagram illustrating an example of a functional configuration of the terminal device 100 according to the present embodiment. Referring to FIG. 4, the terminal device 100 includes a communication unit 110, an input unit 120, an imaging unit 130, a sound collection unit 140, a display unit 150, an audio output unit 160, a storage unit 170, and a control unit 180.

(Communication unit 110)
The communication unit 110 communicates with other devices. For example, the communication unit 110 is directly connected to the LAN 23 and communicates with each device in the satellite office 20. The communication unit 110 communicates with each device in the center office 10 via the external network 30 and the LAN 19. Specifically, for example, the communication unit 110 communicates with the camera 11, the microphone 13, the sensor 15, the media distribution server 17, and the information management server 200. Note that the communication unit 110 can be implemented by the communication interface 811, for example.

(Input unit 120)
The input unit 120 receives input from the user of the terminal device 100. The input unit 120 provides the input result to the control unit 180.

  For example, the input unit 120 detects a position designated by the user on the display screen. More specifically, for example, the input unit 120 is mounted by the touch detection surface 821 of the touch panel 820 and detects a touch position on the touch panel 820. Then, the input unit 120 provides the detected touch position to the control unit 180.

(Imaging unit 130)
The imaging unit 130 images a subject. For example, the imaging unit 130 captures an area in the front direction of the terminal device 100. In this case, the imaging unit 130 images the user of the terminal device 100. The imaging unit 130 provides the imaging result (that is, the captured image) to the control unit 180. Note that the imaging unit 130 may be implemented by a camera 813, for example.

(Sound collector 140)
The sound collection unit 140 collects sounds around the terminal device 100. For example, the sound collection unit 140 collects the voice of the user of the terminal device 100. The sound collection unit 140 provides a sound collection result (that is, sound data) to the control unit 180. Note that the sound collection unit 140 can be implemented by a microphone 815, for example.

(Display unit 150)
The display unit 150 displays an output image (that is, a display screen) from the terminal device 100. The display unit 150 displays a display screen according to control by the control unit 180. The display unit 150 can be mounted on the display surface 823 of the touch panel 820, for example.

(Audio output unit 160)
The voice output unit 160 outputs voice and the like from the terminal device 100. The sound output unit 160 outputs sound in accordance with control by the control unit 180. Note that the audio output unit 160 can be implemented by a speaker 817, for example.

(Storage unit 170)
The storage unit 170 stores a program and data for the operation of the terminal device 100. Note that the storage unit 170 may be implemented by the storage device 809, for example.

(Control unit 180)
The control unit 180 provides various functions of the terminal device 100. The control unit 180 includes a position acquisition unit 181, an object selection unit 183, a display control unit 185, a communication ID acquisition unit 187, and a telephone unit 189. The control unit 180 can be implemented by, for example, the CPU 801, the ROM 803, and the RAM 805.

(Position acquisition unit 181)
The position acquisition unit 181 acquires a position specified by the user on the display screen.

  Specifically, for example, the display screen is a screen displayed by a display device, and the display device is, for example, the display unit 150. When the input unit 120 detects a position designated by the user on the display screen, the position acquisition unit 181 acquires the position.

-Acquisition of the position in a captured image Especially in this embodiment, the position acquisition part 181 acquires the position in the said captured image designated by the user in the display screen containing the captured image of real space.

  For example, the display screen is a display screen displayed by the display unit 150. Further, for example, the captured image is a captured image generated through the camera 11 in the real space (center office 10). The captured image may be a captured image generated by the camera 11, or may be a captured image generated by processing the captured image generated by the camera 11. The display screen is a screen that includes the captured image in part or in whole.

  Further, for example, the position acquisition unit 181 acquires one position in the captured image specified by the user on the display screen. That is, one position is designated by the user.

  More specifically, for example, when the input unit 120 detects one position in the captured image specified by the user on the display screen of the captured image in the real space, the position acquisition unit 181 determines the one position. get. Hereinafter, with reference to FIG. 5, a specific example of acquiring the position in such a captured image will be described.

  FIG. 5 is an explanatory diagram for explaining an example of acquiring a position in a captured image included in the display screen. Referring to FIG. 5, a display screen 60 displayed by the display unit 150 is shown. The display screen 60 includes a captured image 61 generated through any camera 11 in the real space (center office 10). For example, the input unit 120 detects one position 71 in the captured image 61 specified by the user on the display screen 60. Then, the position acquisition unit 181 acquires the position 71. It should be noted that although the position 71 is clearly shown in FIG. 5 for the sake of explanation here, the position 71 is not actually displayed on the display screen 60.

  The display screen 60 further includes a button image 63A and a button image 63B for selecting another camera 11. For example, when the user designates the position of the button image 63, a captured image 61 generated through another camera 11 is displayed on the display screen 60.

  As described above, the position acquisition unit 181 acquires the position in the captured image specified by the user. Then, the position acquisition unit 181 provides the position in the captured image to the object selection unit 183.

  For example, the display device (for example, the display unit 150) is a display surface of a touch panel. And the position in a captured image is designated by the user's touch in a touch panel.

(Object selection unit 183)
The object selection unit 183 selects one or more objects from a plurality of objects respectively corresponding to any position of the captured image based on the acquired position in the captured image.

-Position in the captured image corresponding to the object As described above, the plurality of objects respectively correspond to any position of the captured image. For example, each of the plurality of objects is a person (or information about the person), and corresponds to a position in a captured image in which the person is captured. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 6 is an explanatory diagram for explaining an example of positions corresponding to a plurality of objects (persons). Referring to FIG. 6, a display screen 60, a captured image 61, and a button image 63 are shown as in FIG. Each of the four objects (persons) corresponds to a position 73 (specifically, positions 73A to 73D) in the captured image 61 where the person is captured.

  As described above, the object corresponds to the position in the captured image. Hereinafter, how the object (person) corresponds to the position in the captured image will be specifically described.

--First Example As a first example, the object corresponds to a predetermined position in the captured image.

  Specifically, for example, for each camera 11 (or for each setting of the camera 11), it is determined which range in the real space (center office 10) is imaged. That is, how the real space (center office 10) is captured in the captured image of each camera 11 is determined. Therefore, for example, a person (object) assumed to appear in the captured image corresponds to a predetermined position where the person is assumed to appear in the captured image. Specifically, for example, a person (object) whose own seat is captured in the captured image corresponds to a predetermined position near the person's own seat in the captured image.

  In the first example, the predetermined position in the captured image corresponding to the object is stored in the information management server 18, for example. Specifically, for example, for each camera 11 (or for each setting of the camera 11), a position in a captured image generated through the camera 11 and information of an object corresponding to the position are stored. Further, the terminal device 100 acquires the predetermined position from the information management server 18.

  In addition, the predetermined position in the captured image exists when a person (object) is actually present, and may not exist when a person is not actually present. That is, the predetermined position may or may not exist depending on whether or not a person (object) exists. As an example, when the sensor 15 determines that a person is in the seat, the predetermined position exists, and if not, the predetermined position may not exist.

--Second example As a second example, a three-dimensional virtual space corresponding to the real space is prepared, and a virtual object corresponding to the object (person or information about the person) is arranged in the three-dimensional virtual space. Is done. The object corresponds to a position in the captured image corresponding to the arrangement position (in the three-dimensional virtual space) of the virtual object corresponding to the object.

  Specifically, for example, the arrangement position of the camera 11 in the real space corresponds to one of the virtual positions in the three-dimensional virtual space. An image generated by rendering based on the virtual position corresponds to a captured image generated through the camera 11. Therefore, the arrangement position of the virtual object in the three-dimensional virtual space corresponds to any position in the captured image. For example, a person (object) corresponding to the virtual object corresponds to a position in the captured image corresponding to the arrangement position of the virtual object.

  In the second example, the position in the captured image corresponding to the object is calculated by the information management server 18, for example. Specifically, for example, in response to a request from the terminal device 100, the information management server 18 uses a three-dimensional virtual space, and from the arrangement position of the virtual object to which the object corresponds, in the captured image to which the object corresponds. Calculate the position. Then, the calculated position is provided to the terminal device 100.

  Further, the arrangement position of the virtual object may change according to the actual position of the person (object) to which the virtual object corresponds. In this case, for example, the actual position of the person (object) may be detected and provided by any sensor. And the arrangement position of the virtual object in the three-dimensional virtual space may change based on the actual position to be provided.

  The virtual object arranged in the three-dimensional virtual space exists when a person (object) is actually present, and may not exist when a person is not actually present. That is, the virtual object may or may not exist depending on whether or not a person (object) exists. As an example, when the sensor 15 determines that a person is in the seat, the virtual object exists, and if not, the virtual object may not exist.

--Third example In the third example of the above technique, an object (person) corresponds to a position where the object (person) is recognized in a captured image.

  Specifically, for example, a human face is recognized from the captured image. The person (object) corresponds to the recognized face position.

  In the third example, the position in the captured image corresponding to the object is calculated by the information management server 18, for example. Specifically, for example, the information management server 18 recognizes a human face from the captured image in response to a request from the terminal device 100. Furthermore, the information management server 18 acquires the position in the captured image where the face is recognized as the position in the captured image corresponding to the person (object). Then, the acquired position is provided to the terminal device 100.

—Object Selection Method——First Case First, for example, one position in a captured image is designated and acquired by a user, and the one position in the captured image is selected from the plurality of objects. Any of the objects matches the corresponding position. In this case, the object selection unit 183 selects an object corresponding to a position that matches the acquired position in the captured image. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 7 is an explanatory diagram for explaining an example of an object selected when a position corresponding to any object matches a position specified by the user. Referring to FIG. 7, a display screen 60 and a captured image 61 included in the display screen are shown. In FIG. 7, for the sake of easier understanding, a specific image of the captured image 61 is not shown, and a position 71 (that is, a position specified by the user) in the captured image and a position corresponding to the object. 73. In this example, for example, the acquired position 71 in the captured image 61 matches the position 73B to which the object corresponds. Therefore, the object selection unit 183 selects an object corresponding to the position 73B.

  For example, as described above, one object corresponding to a position that matches the position specified by the user is selected.

--Second case Secondly, for example, particularly in the present embodiment, the object selection unit 183 selects the position acquired in the captured image (ie, the position specified by the user) from among the plurality of objects. ) One or more objects corresponding to positions in the area corresponding to. That is, the one or more objects to be selected are objects corresponding to positions in the region corresponding to the acquired position among the plurality of objects.

  For example, one position in the captured image is designated and acquired by the user, and the one position in the captured image does not match any position corresponding to any one of the plurality of objects. In this case, the object selection unit 183 selects one or more objects corresponding to positions in the region corresponding to the acquired one position from the plurality of objects. In other words, the one or more objects to be selected are objects corresponding to positions in the region corresponding to the one position to be acquired among the plurality of objects.

  For example, the object selection unit 183 selects an object corresponding to a position in a region having a predetermined shape including the acquired position in the captured image from the plurality of objects. That is, the one or more objects to be selected are objects corresponding to positions within a predetermined shape area including the one position to be acquired among the plurality of objects. Hereinafter, a specific example of the area will be described with reference to FIG. 8, and a specific example of the selected object will be described with reference to FIG.

  FIG. 8 is an explanatory diagram for explaining an example of an area for selecting an object. Referring to FIG. 8, an acquired position 71 (that is, a position specified by the user) in the captured image and a region 75 having a predetermined shape including the position 71 are shown. For example, as a first example, the region 75 </ b> A is a circular region centered on the position 71. As a second example, the region 75B is a rectangular region (as an example, a square) centered on the position 71. When the position 71 is designated by the user, an object corresponding to a position in such a region 75 (for example, the region 75A or the region 75B) is selected.

  FIG. 9 is an explanatory diagram for explaining an example of an object selected when the position designated by the user does not match any position corresponding to any object. Referring to FIG. 9, a display screen 60 and a captured image 61 included in the display screen are shown. Also in FIG. 9, as in FIG. 7, for the sake of easier understanding, a specific image of the captured image 61 is not shown, and an acquired position 71 in the captured image 61 (that is, a position specified by the user). And a position 73 to which the object corresponds is shown. In this example, the acquired position 71 in the captured image does not match any position corresponding to any object. That is, the position 71 does not match any position 73. Therefore, the object selection unit 183 includes objects corresponding to the positions 73A and 73B in the area 75A having a predetermined shape including the acquired position 71 in the captured image 61 (a circular area 75A centered on the position 71). Select.

  For example, as described above, one or more objects corresponding to positions within a predetermined shape area including a position designated by the user are selected. As shown in FIG. 9, two or more objects may be selected, or one object may be selected.

--Third case Note that, for example, any position corresponding to the plurality of objects may not be in an area corresponding to the acquired position (that is, a position specified by the user) in the captured image. is there. In this case, the object selection unit 183 does not select any object. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 10 is an explanatory diagram for explaining an example of a case in which no object is selected even if a position in the captured image is designated by the user. Referring to FIG. 10, a display screen 60 and a captured image 61 included in the display screen are shown. Also in FIG. 10, as in FIGS. 7 and 9, for the sake of easier understanding, the specific image of the captured image 61 is not shown, and the acquired position 71 in the captured image 61 (that is, designated by the user). Position) and a position 73 corresponding to the object are shown. In this example, the acquired position 71 in the captured image does not match any position corresponding to any object. That is, the position 71 does not match any position 73. Also, none of the positions corresponding to the plurality of objects is within the region 75A centered on the position 71. In such a case, the object selection unit 183 does not select any object.

  As described above, the object selection unit 183 selects an object. Other operations regarding the object selection unit 183 will be described after the description of the display control unit 185 below.

(Display control unit 185)
The display control unit 185 controls display by the display device. The display device is, for example, the display unit 150.

-Display control of display screen including captured image For example, the display control unit 185 causes the display unit 150 to display a display screen including a captured image of real space.

  For example, as described above, the captured image is a captured image generated through the camera 11 in the real space (center office 10). The captured image may be a captured image generated by the camera 11, or may be a captured image generated by processing the captured image generated by the camera 11. The display screen is a screen that includes the captured image in part or in whole. For example, a display screen such as the display screen 60 shown in FIGS. 5 and 6 is displayed by the display unit 150.

  As a specific process, for example, the display control unit 185 acquires a captured image generated by the camera 11 via the communication unit 110. Then, the display control unit 185 generates a display screen including the captured image and causes the display unit 150 to display the display screen.

  In addition, for example, the captured image is a captured image generated through one selected camera 11 among a plurality of cameras 11 in the real space (center office 10). As described above, for example, when the user designates the position of the button image 63A or the image 63B shown in FIG. 5, another camera 11 is selected. Since the camera 11 can be selected in this way, the user can see a captured image from a desired position.

-Display screen after the user specifies the position in the captured image-First case: Case where one object is selected First, for example, when the user specifies a position in the captured image, One object is selected from the objects. In this case, information regarding the one object is displayed. That is, when one object is selected from a plurality of objects, the display control unit 185 causes the display unit 150 to display information regarding the one object.

  For example, the information related to the one object is an image of a partial region in the captured image including the position in the captured image corresponding to the one object. That is, the display control unit 185 is an image of a partial region in the captured image including the position in the captured image corresponding to the selected one object on the display unit 150. For example, such a partial area is an area in which a person (object) is captured. For example, the image of the partial area is an enlarged image in which the partial area is enlarged. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 11 is an explanatory diagram for explaining an example of a display screen displayed when one object is selected. Referring to FIG. 11, a display screen 80 is shown. As shown in FIGS. 5 and 7, the display screen 80 is displayed when a position 71 in the captured image 61 is designated by the user on the display screen 60 (that is, when an object corresponding to the position 73B is selected). It is a display screen displayed on the screen. Therefore, the display screen 80 includes a partial region image 81 in the captured image 61 including the position 73B. The partial area is an area where a person (object) is captured. The image 81 is an enlarged image in which the partial area is enlarged. As described above, the display control unit 185 causes the display unit 150 to display an enlarged image of the partial region including the position corresponding to the selected one object.

  Referring to FIG. 11, the display screen 80 further includes a button image 83. The button image 83 is an image for the user to specify switching to the original display screen. For example, when the user designates the position of the button image 83, the display screen 60 as shown in FIG.

--Second case: Case where two or more objects are selected Secondly, for example, when the user designates a position in a captured image, two or more objects are selected from the plurality of objects. . Particularly in this embodiment, in this case, the display control unit 185 causes the display unit 150 to display information on each of the two or more objects.

  For example, the two or more objects include a first object and a second object. The information regarding each of the two or more objects includes an image of a first partial area in the captured image including a position in the captured image corresponding to the first object, and the second An image of a second partial region in the captured image including a position in the captured image corresponding to the object is included. That is, the display control unit 185 causes the display unit 150 to display the first partial region image in the captured image including the position corresponding to the first object and the captured image including the position corresponding to the second object. And the image of the second partial area in the image is displayed. For example, the first partial region is a region where a first person (first object) is captured, and the second partial region is a region where a second person (second object) is captured.

  For example, the display control unit 185 causes the display unit 150 to display the image of the first partial area and the image of the second partial area in parallel.

  For example, the image of the first partial region is an enlarged image in which the first partial region is enlarged. Further, the image of the second partial area is an enlarged image in which the second partial area is enlarged. Hereinafter, a specific example of a display screen displayed when two or more objects are selected will be described with reference to FIG.

  FIG. 12 is an explanatory diagram for explaining an example of a display screen displayed when two or more objects are selected. Referring to FIG. 12, a display screen 80 is shown. This display screen 80 corresponds to the display screen 60 as shown in FIG. 5 when the position 71 in the captured image 61 is designated by the user as shown in FIG. 9 (that is, corresponding to the position 73A and the position 73B). This is a display screen displayed when an object is selected. Therefore, the display screen 80 includes a partial region image 85A in the captured image 61 including the position 73A and a partial region image 85B in the captured image 61 including the position 73B. The image 85 of each partial area is an area where a person (object) is captured. The image 85A is an image in which the partial area including the position 73A is enlarged, and the image 85B is an image in which the partial area including the position 73B is enlarged. Further, the image 85A and the image 85B are displayed in parallel. As described above, the display control unit 185 causes the display unit 150 to display an enlarged image of the partial region including the position corresponding to the selected object.

--- Generation Process of Display Screen Including Partial Area Image Here, an example of the generation process of the display screen 80 including the image 85A and the image 85B as shown in FIG. 12 will be described with reference to FIGS. explain.

  13, FIG. 14A, FIG. 14B, and FIG. 15 are explanatory diagrams for explaining an example of the generation process of the display screen 80 including the image 85A of the first partial region and the image 85B of the second partial region.

  First, referring to FIG. 13, a captured image 61 is shown. This captured image 61 is the same as that shown in FIG. For example, as shown in FIG. 9, 71 in the captured image 61 is designated by the user. Then, the first object corresponding to the position 73A and the second object corresponding to the position 73B are selected. Then, an enlargement process (zoom process) is performed on the partial area including the position 73A and the position 73B, and an enlarged image 86 of the partial area is generated.

  Next, referring to FIG. 14A, a partial region centered on the position 73A in the enlarged image 86 is cut out. As a result, the cut-out image 85A of the partial area is generated. Similarly, referring to FIG. 14B, a partial region centered on the position 73B in the enlarged image 86 is cut out. Thereby, an image 85B of the cut-out partial area is generated.

  Then, referring to FIG. 15, the image 85 </ b> A and the image 85 </ b> B are incorporated in the display screen 80 in parallel. That is, the display screen 80 in which the image 85A and the image 85B are arranged in parallel is generated.

  For example, as described above, the display screen 80 including the image 85A and the image 85B as shown in FIG. 12 is generated.

--- Display of operation promotion sign Further, the display control unit 185 causes the display unit 150 to display a sign (object designation promotion sign) for allowing the user to designate one of the two or more objects. May be. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 16 is an explanatory diagram for explaining an example of an object designation promotion sign. Referring to FIG. 16, a display screen 80, an image 85A, and an image 85B are shown as in FIG. Further, object designation promotion signs 87A and 87B for designating an object corresponding to the image 85A or an object corresponding to the image 85B are displayed.

--Third case: Case where no object is selected Third, for example, even if the user specifies a position in the captured image, no object is selected. In this case, the display control unit 185 causes the display unit 150 to display a sign (hereinafter referred to as “position designation promotion sign”) for allowing the user to designate a position in the captured image on the display screen. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 17 is an explanatory diagram for explaining an example of a position designation promotion mark. Referring to FIG. 17, a display screen 60 and a captured image 61 are shown as in FIG. Then, for example, as shown in FIG. 10, the position in the captured image is designated by the user, and as a result, no object is selected. In this case, the display control unit 185 causes the display unit 150 to display the position designation promotion mark 65 including the character information “Please touch again”.

  Such a re-designation promotion mark 65 allows the user to notice that it is necessary to designate the position in the captured image again. Therefore, convenience for the user is increased.

-Display screen during a call For example, the display control unit 185 causes the display unit 150 to display a display screen for a call during a call. This will be described later in connection with the telephone unit 189.

(Object selection unit 183 (continued))
Here, the operation of the object selection unit 183 will be described again.

--First case: Case where one object is selected As described above, when one object is selected, for example, as shown in FIG. 11, the one object includes a corresponding position. An image 81 of the partial area is displayed. In such a case, when the one object is specified by the user, the object selecting unit 183 selects the one object.

  As an example, the user designates the one object by touching any position in the image 81.

--Second case: a case where two or more objects are selected As described above, when two or more objects are selected, for example, as shown in FIG. An image 85 of a partial area including is displayed. In such a case, particularly in the present embodiment, the object selection unit 183 selects one object specified by the user from the two or more objects.

  As an example, the user specifies one object corresponding to the image 85A by touching any position in the image 85A. In addition, the user specifies one object corresponding to the image 85B by touching any position in the image 85B.

(Communication ID acquisition unit 187)
The communication ID acquisition unit 187 acquires communication identification information (hereinafter referred to as “communication ID”).

-Acquisition of Communication ID Corresponding to Object As described above, as an example, information on one selected object (for example, image 81 shown in FIG. 11) is displayed, and the one object is designated by the user. And selected. As another example, information about two or more selected objects (for example, the image 85A and the image 85B shown in FIG. 12) is displayed and specified by the user from the two or more objects. One object is selected. As in these examples, when one object is selected, a communication ID corresponding to the one object is acquired. That is, when one object is selected, the communication ID acquisition unit 187 acquires a communication ID corresponding to the one object.

  Specifically, for example, when the object selection unit 185 selects one object, the communication ID acquisition unit 187 acquires the object ID of the selected one object. Then, the communication ID acquisition unit 187 acquires a communication ID corresponding to the object ID.

  The object ID and the communication ID are provided to the terminal device 100 together with the position in the captured image corresponding to each object, for example, by the information management server, and stored in the storage unit 170. When one object is selected, the object ID of the one object is acquired from the storage unit 170 and provided to the communication ID acquisition unit 187. Then, the communication ID acquisition unit 187 acquires the communication ID corresponding to the provided object ID from the storage unit 170.

-Types of communication IDs-First example: Communication ID for dedicated network
For example, the communication ID is identification information for communication in a dedicated network. This makes it possible to contact a desired person on the dedicated network.

  As an example, the communication ID is a telephone number for a telephone in a private network.

  The dedicated network may include an extension telephone network or a wireless LAN communication network. The dedicated network may include two or more wireless LAN communication networks connected by a VPN (Virtual Private Network).

-Second example: Public network communication ID
The communication ID may be identification information for communication via a public network. This makes it possible to contact a desired person via the public network.

  As an example, the communication ID may be a telephone number for a telephone call in a public network.

  For example, the public network may include a mobile communication network. This makes it possible to contact a desired person via the mobile communication network. For example, it is possible to contact a mobile phone terminal. The public network may include the Internet or a public switched telephone network (PSTN).

  The communication ID acquisition unit 187 may acquire the communication ID while adding a special number (special use number). In this case, when acquiring the communication ID corresponding to the object ID from the storage unit 170, the communication ID acquisition unit 187 adds a special number to the communication ID and acquires the communication ID. Specifically, for example, when the communication ID acquisition unit 187 acquires the telephone number “yyyy” as the communication ID corresponding to the object ID from the storage unit 170, the communication ID acquisition unit 187 adds the special number “xx” to the telephone number “yyyy”. The telephone number “xx-yyyy” is acquired as the communication ID.

  Further, the communication ID acquisition unit 187 may include a communication ID adjustment unit that determines whether or not to add a special number and determines a special number to be added when acquiring the communication ID. When the communication ID acquisition unit 187 acquires the communication ID corresponding to the object ID from the storage unit 170, the communication ID adjustment unit uses an external communication control unit (the communication ID corresponding to the communication ID to be acquired). Based on the above, a communication control means (for example, PBX 40) is inquired to determine whether or not to add a special number and to determine the special number to be added. Here, the inquiry processing in the communication ID adjusting means is that the communication ID adjusting means is connected to the PBX 40 (DB of the PBX 40 (data database (station data) used for call control)). Note that the communication ID adjustment means is an H.264 compatible with the PBX 40. It corresponds to H.323 or SIP and is connected to the PBX 40. The communication ID adjustment means determines whether or not to add a special number in the communication ID adjustment means when the communication ID acquired by the communication ID acquisition unit 187 is PBX 40, which is an extension interconnection or a local line transmission. It is to confirm whether it corresponds to a connection or a dedicated line transmission connection, and to determine that a special number is added when it corresponds to a local line transmission connection or a dedicated line transmission connection. On the other hand, the communication ID adjusting means determines that the special number is not added if it can be confirmed that it corresponds to the extension interconnection. The process for determining the special number to be added in the communication ID adjusting means corresponds to the local transmission connection in the process of determining whether or not to add the special number, and if it is determined that the special number is added, the local line transmission special number is determined. A special number to be added with “0” is determined, and if it is determined that a special number is added in the processing for determining whether or not to add a special number, it is determined that a special number will be added. It is to be determined as a special number.

(Phone 189)
The telephone unit 189 provides a function for making a call. For example, the telephone unit 189 provides a softphone function.

-Calling For example, when the telephone unit 189 acquires the communication ID from the communication ID acquisition unit 187, the telephone unit 189 performs a telephone call using the communication ID. More specifically, for example, when the telephone unit 189 acquires the communication ID, the telephone unit 189 provides the communication ID to the PBX 40 via the communication unit 110 and acquires the IP address from the PBX 40. Then, the telephone unit 189 executes a series of sequences for establishing a session with a communication device having the IP address (that is, a communication device that is a destination).

  As described above, the telephone unit 189 performs a telephone call for a telephone call to a person (object) selected on the display screen. That is, the telephone unit 189 makes a telephone call to the communication device of the person.

-Communication of audio data and video data When the audio data from the communication device of the other party of the telephone is received by the communication unit 110, the telephone unit 189 outputs the audio of the audio data to the audio output unit 160 Let Further, the telephone unit 189 causes the communication unit 110 to transmit the voice data provided by the sound collection unit 140 to the communication device of the other party of the telephone. In addition, for example, the telephone unit 189 causes the communication unit 110 to transmit a captured image provided by the imaging unit 130 (for example, a captured image in which the user of the terminal device 100 is captured) to the communication apparatus of the other party of the telephone.

-Display Screen for Calls In addition, during such a call, the display control unit 185 displays a display screen for calls. Hereinafter, a specific example of the call display screen will be described with reference to FIG.

  FIG. 18 is an explanatory diagram for explaining an example of a call display screen. Referring to FIG. 18, a display screen 90 for a conference call is shown. The display screen 90 includes a partner-side captured image 91, a button image 93, and a self-side captured image 95.

  The partner side captured image 91 is, for example, a captured image acquired from a communication device of a telephone partner. For example, when the captured image from the communication device of the other party of the telephone is received by the communication unit 110, the telephone unit 189 uses the captured image as the other party captured image 91.

  The button image 93 is an image for ending the call. For example, when the user designates the position of the button image 93, the telephone unit 189 ends the telephone call. More specifically, for example, when the user touches the position of the button image 93 and a touch position corresponding to the button image 93 is detected, the telephone unit 189 executes a sequence for ending the call including disconnection of the session. To do. For example, the telephone unit 189 switches the display mode from the conversation mode to the overhead mode.

  The own side captured image 95 is a captured image provided by the imaging unit 130.

(Function and effect)
The example of the functional configuration of the terminal device 100 according to the present embodiment has been described above. As described above, based on the position in the captured image specified by the user on the display screen including the captured image of the space, one of the plurality of objects respectively corresponding to any position of the captured image is 1 One or more objects are selected. When two or more objects are selected from a plurality of objects, information regarding each of the two or more objects is displayed. Furthermore, one object designated by the user is selected from the two or more objects. When the one object is selected, a communication ID corresponding to the one object is acquired.

  Thereby, it becomes possible to contact a desired person more easily by intuitive operation.

  More specifically, for example, even if the user cannot pinpoint one person (object) in the captured image on the display screen, two or more persons (two or more objects) are selected. Information on the two or more persons is displayed. Then, the user can pinpoint one person (one object) from the information about the person displayed anew. Therefore, for example, even when the display screen itself is small or when the person shown on the display screen is small, one person can be finally selected. In addition, when it is unclear which of the two or more persons is specified, one person is not automatically selected, but two or more persons are selected. It is avoided that only one person is selected. That is, a desired person can be selected more easily.

  Further, a communication ID (for example, a telephone number) corresponding to the selected person is acquired. Therefore, it becomes possible to contact the selected person.

  That is, an intuitive operation of designating a person on the display screen makes it possible to contact a desired person more easily even when it is difficult to designate a person pinpoint on the display screen. .

  Specifically, as described above in relation to the display control unit 185, for example, two or more selected objects include a first object and a second object. Then, the image of the first partial area in the captured image including the position corresponding to the first object, and the second partial area in the captured image including the position corresponding to the second object. An image is displayed.

  Thus, by displaying the image of the partial area including the position corresponding to the person (object), it becomes possible to more intuitively specify a desired person from two or more persons. That is, a desired person can be designated by intuitive confirmation such as confirmation of appearance such as face, hairstyle, back view, and posture. Therefore, convenience for the user is increased. For example, it may be possible to specify a person more quickly and / or less burdensomely.

  Further, as described above with reference to the display control unit 185, for example, the image of the first partial area and the image of the second partial area are displayed in parallel.

  This makes it possible to compare two or more persons more easily. Therefore, convenience for the user is increased. For example, it may be possible to specify a person more quickly and / or less burdensomely.

  Further, as described above in relation to the display control unit 185, for example, the image of the first partial area is an enlarged image in which the first partial area is enlarged, and the second partial area is The image is an enlarged image in which the second partial region is enlarged.

  Thereby, it becomes possible to confirm each of two or more persons more easily. Therefore, convenience for the user is increased. For example, it may be possible to specify a person more quickly and / or less burdensomely.

  In addition, as described above with reference to the object selection unit 183, for example, an object corresponding to a position within the region corresponding to the position specified by the user among the plurality of objects is selected.

  Thus, the person can be selected without specifying the position corresponding to the person (object) by pinpointing. Therefore, it becomes possible to make designation by the user easier.

  Further, as described above in relation to the object selection unit 183, for example, a person (object) corresponding to a position within a predetermined shape area including one position designated by the user on the display screen is selected. .

  Thereby, a person (object) corresponding to a position near the position designated by the user is selected. Therefore, there is a high possibility that the selected object includes a person to be specified by the user (that is, a desired person).

  In addition, as described above with reference to the object selection unit 183, for example, if any position corresponding to the plurality of objects is not within the area corresponding to the position specified by the user, any object is selected. Is not selected. Further, as described above with reference to the display control unit 185, when no object is selected, a sign (that is, a position designation promoting sign) for causing the user to designate the position in the captured image is displayed. .

  As a result, the user can be notified that no person (object) has been selected. Therefore, the user can specify the person again more quickly.

  Further, as described above with reference to the position acquisition unit 181, for example, the position in the captured image is specified by the user's touch on the touch panel.

  Accordingly, it is possible to easily select a person (object) by an intuitive operation of touching a position near the person (object) on the display screen.

<2.3. Software configuration>
Next, an example of the software configuration of the terminal device 100 according to the present embodiment will be described with reference to FIG. FIG. 19 is a block diagram illustrating an example of a software configuration of the terminal device 100 according to the present embodiment. Referring to FIG. 19, the terminal device 100 includes an OS (Operating System) 840 and application software. The terminal device 100 includes a soft phone 851, a super-realistic client 853, and a telephone call control function 855 as application software.

  The OS 840 is software that provides basic functions for operating the terminal device 100. The OS 840 executes each application software.

  The soft phone 851 is application software for making a phone call using the terminal device 100. The ultra-realistic client 853 is application software for providing real space information to the terminal device 100. The telephone call control function 855 is application software that acquires a communication ID corresponding to an object (person).

  For example, when the call origination control function 855 acquires the communication ID, it provides it to the softphone 851 via the OS. Then, the soft phone 851 makes a call using the communication ID.

  Further, for example, the super-realistic client 853 may acquire state information indicating the state of a person in the real space (for example, the center office 10) and provide the information to the soft phone 851 via the OS. Then, the soft phone 851 may control outgoing calls based on the state information.

  The position acquisition unit 181, the object selection unit 183, and the communication ID acquisition unit 187 can be implemented by the telephone call control function 855. Further, the telephone unit 189 can be implemented by a soft phone 851, for example. Further, the display control unit 185 can be implemented by the soft phone 851, the super-realistic client 853, and the telephone call control function 855.

  Further, the telephone call control function 855 may be incorporated in the ultra-realistic client 853.

<2.4. Screen transition>
Next, referring to FIGS. 20 to 25, a screen until a display screen (for example, display screen 60 shown in FIG. 5) for selecting an object and acquiring a communication ID as described above is displayed. An example of transition will be described.

  FIG. 20 is an explanatory diagram for explaining an example of a menu screen. Referring to FIG. 20, a menu screen 1000 is shown. The menu screen includes an icon for starting each software. For example, the menu screen 1000 includes a soft phone 851 icon 1010, a super-realistic client 853 icon 1020, and a phone book icon 1030. For example, when the user designates the icon 1010, the softphone 851 is activated. When the user designates the icon 1020, the super-realistic client 853 is activated. Further, when the user designates the icon 1030, the phone book is activated.

  FIG. 21 shows an example of a softphone user interface displayed on the screen. Referring to FIG. 21, a softphone user interface 1110 is shown. When the user designates the icon 1010 and the softphone 851 is activated, the user interface 1110 is displayed in this way. The user can make / receive calls by operating the user interface 1110.

  FIG. 22 is an explanatory diagram for explaining an example of a display screen of a super-realistic client. Referring to FIG. 22, the display screen 60 of the ultra-realistic client is shown as in FIG. 5. When the user designates the icon 1020 and the ultra-realistic client 853 is activated, the display screen 60 is displayed in this way. At the stage where the ultra-realistic client 853 is activated, the camera 11 is selected based on the initial setting, and the display screen 60 including the captured image 61 generated through the camera 11 is displayed. Thereafter, another camera 11 can be selected by designating the button image 63 or the like.

  FIG. 23 is an explanatory diagram for explaining an example of a phonebook display screen. Referring to FIG. 23, a phonebook display screen 1130 is shown. When the user designates the icon 1030 and the phone book is activated, the display screen 1130 is displayed in this way. Display screen 1130 includes contacts 1131A to 1131D. Button images 1133A to 1133C are also included.

  For example, when the user designates a position other than the button image 1133A in the contact 1131A corresponding to “Arakawa”, the OS 840 activates the soft phone 851, and the telephone number corresponding to “Arakawa” is displayed on the soft phone 851. “2200” and an instruction to send are provided. Then, the soft phone 851 makes a call to the telephone number “2200” corresponding to “Arakawa”.

  When the user designates a position other than the button image 1133B in the contact 1131B corresponding to “Alex”, the OS 840 activates the softphone 851, and the telephone number corresponding to “Alex” is displayed on the softphone 851. “2010” and an instruction to send are provided. Then, the soft phone 851 makes a call to the telephone number “2010” corresponding to “Alex”.

  When the user designates the position of the button image 1133B, the OS 840 activates the super-realistic client 853 and provides the super-realistic client 853 with information related to “Alex” (for example, ID, presence information, etc.). To do. Then, the super-realistic client 853 provides information (for example, ID, presence information, etc.) regarding “Alex” to the information management server 18. Thereafter, the information management server 18 selects the camera 11 based on the presence information regarding “Alex”. If there are a plurality of cameras 11 corresponding to the presence information, one of the representative cameras (hereinafter referred to as “representative camera”) is selected. Then, the ultra-realistic client 853 acquires a captured image generated through the selected camera 11. The presence information is, for example, information indicating the position of a person (for example, a conference room, a work space, etc.). For example, the presence information is provided to the PBX 40 and acquired from the PBX 40.

  FIG. 24 is an explanatory diagram for describing an example of a representative camera. Referring to FIG. 24, the camera 11 disposed in each of the conference room 1210 and the workspace 1220 is shown. For example, the camera 11C is determined in advance as a representative camera when the presence information is “conference room”. For example, the camera 11D is determined in advance as a representative camera when the presence information is “work space”. In this way, for example, the camera 11 that can capture a wide range in a conference room or workspace is determined in advance as a representative camera.

  The presence information may be displayed in a telephone directory. FIG. 25 is an explanatory diagram for explaining an example of a phone book display screen including presence information. Referring to FIG. 25, a phonebook display screen 1130 is shown. In this example, each of contact 1131A and contact 1131D includes presence information. For example, the contact 1131A includes “conference room” as presence information of “Arakawa”. Contact 1131B includes “work space” as presence information of “Alex”.

<< 3. Process flow >>
Next, an example of information processing according to the present embodiment will be described with reference to FIGS. 26 and 27.

(Example of information processing in a terminal device)
FIG. 26 is an explanatory diagram for describing an example of information processing in the terminal device 100 according to the present embodiment.

  In step S301, the display control unit 185 causes the display unit 150 to display a display screen including a captured image of the real space (center office 10).

  In step S303, the position acquisition unit 181 determines whether a position in the captured image is specified by the user on the display screen. If the position is designated, the process proceeds to step S305. Otherwise, the process repeats step S301.

  In step S305, the position acquisition unit 181 acquires a position in the captured image specified by the user (that is, a position in the captured image specified by the user).

  In step S307, the object selection unit 183 determines a plurality of positions corresponding to any position of the captured image based on the acquired position in the captured image (that is, the position in the captured image designated by the user). One or more objects are selected from the objects.

  In step S309, the display control unit 185 determines whether one or more objects have been selected. If one or more objects are selected, the process proceeds to step S313. Otherwise, the process proceeds to step S311.

  In step S311, the display control unit 185 causes the display unit 150 to display a sign (that is, a position designation promotion sign) for allowing the user to designate a position in the captured image on the display screen.

  In step S313, the display control unit 185 determines whether two or more objects have been selected. If two or more objects are selected, the process proceeds to step S315. Otherwise, the process proceeds to step S323.

  In step S315, the display control unit 185 causes the display unit 150 to display an enlarged image of the partial region including the position corresponding to each of the two or more objects.

  In step S317, it is determined whether switching to the original display screen has been designated by the user. More specifically, for example, it is determined whether the position of the button image 83 shown in FIG. 12 is designated by the user. If the switching is designated, the process returns to step S301. Otherwise, the process proceeds to step 319.

  In step S319, it is determined whether any object has been designated by the user via the display screen including the enlarged image of the partial area. If any object is designated, the process proceeds to step S321. Otherwise, the process returns to step S315.

  In step S321, the object selection unit 183 selects one object designated by the user from the two or more objects. Then, the process proceeds to step S331.

  In step S323, the display control unit 185 causes the display unit 150 to display an enlarged image of a partial region including a position corresponding to one object.

  In step S325, it is determined whether switching to the original display screen has been designated by the user. More specifically, for example, it is determined whether the position of the button image 83 shown in FIG. 11 is designated by the user. If the switching is designated, the process returns to step S301. Otherwise, processing proceeds to step 327.

  In step S327, it is determined whether any object has been designated by the user via the display screen including the enlarged image of the partial area. If any object is designated, the process proceeds to step S329. Otherwise, the process returns to step S323.

  In step S329, the object selection unit 183 selects the one object. Then, the process proceeds to step S331.

  In step S331, the communication ID acquisition unit 187 acquires a communication ID corresponding to one selected object.

  In step S333, the telephone unit 189 makes a telephone call using the acquired communication ID. Then, the process ends.

(Example of information processing until acquisition of captured image)
FIG. 27 is an explanatory diagram for explaining an example of information processing for providing and acquiring a captured image according to presence information.

  In step S401, the terminal device 100 provides the PBX 40 with presence information regarding the user of the device itself.

  In step S403, the terminal device 100 provides the PBX 40 with a communication ID corresponding to a desired person (object). In step S404, the PBX 40 provides the terminal device 100 with presence information regarding the desired person (object).

  In step S407, the terminal device 100 provides the information management server 18 with presence information regarding the desired person (object). In step S409, the information management server 18 selects the camera 11 based on the presence information, and provides the terminal device 100 with information (camera information) regarding the selected camera 11.

  In step S411, the terminal device 100 provides the camera information to the media distribution server 17. Then, in step S413, the media distribution server 17 recognizes the camera 11 selected from the camera information, and provides the terminal device 100 with a captured image generated through the selected camera 11.

<< 4. Modification >>
Subsequently, a modification of the present embodiment will be described with reference to FIGS.

<4.1. First Modification>
First, a first modification of the present embodiment will be described with reference to FIG. In the example of the present embodiment described above, one position is designated by the user on the display screen including the captured image, and an object corresponding to a position in a region having a predetermined shape including the one position is selected. On the other hand, in the first modification example of the present embodiment, two or more positions are designated by the user on the display screen including the captured image, and the position corresponds to a position in an area formed based on the two or more positions. An object is selected.

(Position acquisition unit 181)
-Acquisition of the position in a captured image The position acquisition part 181 acquires the position in the said captured image designated by the user in the display screen containing the captured image of real space.

  In particular, in the first modification of the present embodiment, the position acquisition unit 181 acquires two or more positions in the captured image designated by the user on the display screen. That is, two or more positions are designated by the user.

  More specifically, for example, when the input unit 120 detects two or more positions in the captured image specified by the user on the display screen of the captured image in real space, the position acquisition unit 181 The above position is acquired. Hereinafter, with reference to FIG. 28, a specific example of obtaining the position in such a captured image will be described.

  FIG. 28 is an explanatory diagram for describing an example of acquiring a position in a captured image included in the display screen according to the first modification. Referring to FIG. 28, a display screen 60 displayed by the display unit 150 is shown. The display screen 60 includes a captured image 61 generated through any camera 11 in the real space (center office 10). For example, the input unit 120 detects a position set 77 including two or more positions in the captured image 61 specified by the user on the display screen 60. For example, the position set 77 is a set of positions that form a certain region. Then, the position acquisition unit 181 acquires a position set 77 including two or more positions. It should be noted that although the position set 77 is clearly shown in FIG. 28 for the sake of explanation here, the position set 77 is not actually displayed on the display screen 60.

(Object selection unit 183)
-Object Selection Method One or more objects to be selected are objects corresponding to positions in a region formed based on the two or more acquired positions among the plurality of objects. That is, the object selection unit 183 selects an object corresponding to a position in an area formed based on the two or more acquired positions from among the plurality of objects.

  More specifically, for example, referring to FIG. 28 again, the region formed based on the two or more positions is a region surrounded by a position set 77 as shown in FIG. For example, the position set 77 is two or more positions designated by the user on the display screen 60 (in the captured image 61), is detected by the input unit 120, and is acquired by the position acquisition unit 181. As an example, the position set 77 is a drag (swipe) by a user (an operation in which a finger is applied to the screen, moved on the screen, and released from the screen, which is a position separated from the position where the finger was placed and / or moved. This is a set of positions detected in response to an operation for designating two or more positions corresponding to the trajectory of the finger. Then, the object corresponding to the position in the area surrounded by the position set 77 is selected by the object selection unit 183.

  The area formed based on the two or more positions is not limited to the area surrounded by the position (position set 77) detected in response to the drag by the user. The area includes a tap by the user (an operation in which one or more fingers are put on the screen and separated from the screen and the position is specified by the position of the finger), and a pinch out (two or more fingers are put on the screen and the 2 An operation of widening the interval between two or more fingers and releasing the two or more fingers, and specifying two or more positions by the positions of two or more fingers when the fingers are released) and pinch-in ( An operation of placing two or more fingers on the screen, narrowing the interval between the two or more fingers, and releasing the two or more fingers, two or more at the positions of the two or more fingers when the fingers are released It may be an area corresponding to any (touch) operation technique such as an operation for designating the position of the user. As an example, the region formed based on the two or more positions includes a figure (for example, positions of two or more fingers when fingers are released) detected according to pinch-out or pinch-in (for example, positions of two or more fingers). It may be a region surrounded by a circle, a rectangle, etc.

  In addition, a position (position detected by the input unit 120) acquired by the position acquisition unit 181 may be used in a predetermined period (for example, 2 seconds). And the object corresponding to the position in the area | region formed with the position acquired in the said predetermined period may be selected. As an example, the region may be a region corresponding to a plurality of operations (for example, drag, pinch out and / or pinch in, etc.) in the predetermined period. As a specific example, on the display screen 60 (in the captured image 61), the user taps once with two fingers, taps once with one finger at a position different from the first tap, and further Pinch out with two fingers. In this case, an area corresponding to an operation (two taps and pinch out) within a predetermined detection period is formed in response to the first tap (start of designation of position by the user). In this way, the user can select an object with various combinations of operations.

  The control unit 180 may include an operation recognition unit that recognizes any (touch) operation method such as tap, drag, pinch out, and pinch in from a position specified by the user. In addition, the operation recognizing unit corresponds to a corresponding operation method (for example, tap, Any of a drug, a pinch out and a pinch in or a combination thereof may be determined.

  As a first example, when the captured image 61 in the terminal device 100 is a predetermined size (for example, a predetermined Cm, a predetermined inch, and a predetermined resolution) or more, the operation recognition unit taps, drags, pinches out, and pinches in the corresponding operation method. May be determined. On the other hand, when the captured image 61 in the terminal device 100 is a predetermined size or less, the operation recognition unit may determine the corresponding operation method as tap and drag.

  As a second example, the ratio of the display surface 823 and the touch detection surface 821 in the terminal device 100 (the size (size) of the touch detection surface 821 when the size (size) of the display surface 823 is 100%)). In the case of 70% or more, the operation recognition unit may determine the corresponding operation method as tap, drag, pinch out, and pinch in. On the other hand, when the ratio between the display surface 823 and the touch detection surface 821 in the terminal device 100 is 70% or less, the operation recognition unit may determine that the corresponding operation method is a tap.

  As a third example, when application software AAA (application software not shown in FIG. 19) is activated in terminal device 100, the operation recognition unit may determine the corresponding operation method as tap and drag. . On the other hand, when application software BBB (application software not shown in FIG. 19) is activated in terminal device 100, the operation recognition unit may determine the corresponding operation method as pinch out and pinch in.

  Further, a corresponding operation method may be determined, and information related to the corresponding operation method may be displayed superimposed on the captured image 61. Specifically, for example, when the operation recognizing unit determines that the corresponding operation method is dragging, information such as an icon indicating dragging is superimposed and displayed in the captured image 61 (for example, any one of the four corners). Good. Accordingly, the user can specify the position (position set 77) by an appropriate touch on the touch panel corresponding to various touches.

  The first modification example of the present embodiment has been described above. According to the first modification of the present embodiment, the user can select an object with a more intuitive operation. That is, it becomes possible to contact a desired person with a more intuitive operation.

<4.2. Second Modification>
Next, a second modification of the present embodiment will be described. In the example of the present embodiment described above, when the user designates a position in the captured image, two or more objects are selected from a plurality of objects. Then, an enlarged image (that is, an enlarged image for each object) in which the partial area including the position corresponding to the selected object is enlarged is displayed. In particular, in the second modification of the present embodiment, an enlarged image enlarged at an enlargement rate for each partial region is displayed.

(Display control unit 185)
-Display screen after the user specifies the position in the captured image-Second case: Two or more objects are selected For example, when the user specifies a position in the captured image, Two or more objects are selected from among them. Then, the display control unit 185 displays on the display unit 150 an image of the first partial area in the captured image including the position corresponding to the first object and a captured image including the position corresponding to the second object. And the image of the second partial area in the image is displayed.

  For example, the image of the first partial area is an enlarged image in which the first partial area is enlarged. Further, the image of the second partial area is an enlarged image in which the second partial area is enlarged.

  In particular, in the second modification of the present embodiment, the image of the first partial area is an enlarged image in which the first partial area is enlarged at a first enlargement ratio. The image of the second partial area is an enlarged image in which the second partial area is enlarged at a second enlargement ratio. That is, enlarged images with different enlargement ratios are displayed for each object.

  Heretofore, the second modification of the present embodiment has been described. According to the second modification of the present embodiment, the user can easily identify the contact person even if the person in the captured image has a different size in the captured image. Become.

<4.3. Third Modification>
Next, a third modification of the present embodiment will be described with reference to FIGS. In the example of the present embodiment described above, when the user designates a position in the captured image, two or more objects are selected from a plurality of objects. Then, an image of the partial area including the position corresponding to the selected object is displayed. In particular, in the third modification of the present embodiment, a boundary sign indicating the boundary between the image of the partial area and the area other than the image is displayed.

(Display control unit 185)
-Display screen after the user specifies the position in the captured image-Second case: Two or more objects are selected For example, when the user specifies a position in the captured image, Two or more objects are selected from among them. Then, the display control unit 185 displays on the display unit 150 an image of the first partial area in the captured image including the position corresponding to the first object and a captured image including the position corresponding to the second object. And the image of the second partial area in the image is displayed.

  In particular, in the third modification of the present embodiment, the display control unit 185 causes the display unit 150 to display a first boundary mark indicating a boundary between the image of the first partial region and a region other than the image. And a second boundary sign indicating a boundary between the image of the second partial region and a region other than the image. Hereinafter, a specific example of this point will be described with reference to FIGS. 29 and 30. FIG.

  FIG. 29 is an explanatory diagram for describing a first example of a display screen displayed according to the third modified example when two or more objects are selected. Referring to FIG. 29, as in the example of FIG. 12, the display screen 80 includes a partial region image 85 </ b> A and a partial region image 85 </ b> B. The display screen 80 includes a boundary sign 88A indicating a boundary between the image 85A and the other area, and a boundary sign 88B indicating a boundary between the image 85B and the other area. In this example, the boundary sign 88A is displayed inside the image 85A, and the boundary sign 88B is displayed inside the image 85B.

  FIG. 30 is an explanatory diagram for explaining a second example of a display screen displayed according to the third modification when two or more objects are selected. Referring to FIG. 30, as in the example of FIG. 12, the display screen 80 includes a partial region image 85 </ b> A and a partial region image 85 </ b> B. The display screen 80 includes a boundary sign 89A indicating a boundary between the image 85A and the other area, and a boundary sign 89B indicating a boundary between the image 85B and the other area. In this example, the boundary sign 89A is displayed outside the image 85A, and the boundary sign 89B is displayed outside the image 85B.

  The first boundary sign may include information related to the first object, and the second boundary sign may include information related to the second object. More specifically, for example, the information related to the first object includes first identification information for identifying the first object, and the information related to the second object includes the second information Second identification information for identifying the object may be included. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 31 is an explanatory diagram for describing an example of information regarding an object included in a boundary sign. Referring to FIG. 31, a display screen 80, a partial area image 85A, a partial area image 85B, a boundary indicator 88A, and a boundary indicator 88B are shown as in FIG. For example, as shown in FIG. 31, the boundary signs 88A and 88B include names (Takahashi and Arakawa) that are identification information of a person (object).

  Heretofore, the third modification example of the present embodiment has been described. According to the third modification of the present embodiment, a boundary sign is displayed. Thereby, it becomes possible to make the center of the image of the partial area stand out. That is, the user can easily recognize an object (for example, a person) corresponding to each image.

  In addition, since the boundary area includes information on an object (for example, a person), the user can more easily recognize an object (for example, a person) corresponding to each image. Furthermore, since the information is identification information of an object (for example, a person), the user can surely recognize an object (for example, a person) corresponding to each image.

<4.4. Fourth Modification>
Next, a fourth modification of the present embodiment will be described with reference to FIG. In the example of the present embodiment described above, when one or more objects are selected from the plurality of objects, information on each of the selected one or more objects is displayed. Then, as the information, an image of a partial area including the position corresponding to the selected object is displayed. On the other hand, in the fourth modification example of the present embodiment, identification information for identifying each of the one or more objects is displayed as information on each of the one or more selected objects.

  (Display control unit 185)

-Display screen after the user specifies the position in the captured image-Second case: Two or more objects are selected For example, when the user specifies a position in the captured image, Two or more objects are selected from among them. Particularly in this embodiment, in this case, the display control unit 185 causes the display unit 150 to display information on each of the two or more objects.

  In particular, in the fourth modification example of the present embodiment, the information regarding each of the two or more objects includes identification information for identifying each of the two or more objects. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 32 is an explanatory diagram for explaining an example of a display screen displayed according to the fourth modification when two or more objects are selected. Referring to FIG. 32, a display screen 60 is shown with reference to FIG. The display screen 60 includes a captured image 61 in real space. The display screen 60 also includes identification information 99A and identification information 99B for identifying each of the two selected persons (two objects). In this example, the identification information 99A and the identification information 99B are names (Takahashi and Arakawa) that are identification information of a person (object). As an example, the user designates the position of one of the identification information 99A and the identification information 99B, thereby designating a person (object) corresponding to the one identification information. Then, one person (one object) designated by the user is selected.

  The fourth modification example of this embodiment has been described above. According to the fourth modification of the present embodiment, it is possible to more easily contact a desired person by a simple display process.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  For example, as an example in which two or more objects are selected, the example in which two objects are selected has been described, but the present invention is not limited to this. Three or more objects may be selected, and information regarding the selected three or more objects may be displayed.

  Further, an example in which two or more objects are selected has been described. That is, the example in which the number of objects to be selected is not limited has been described, but the present invention is not limited to this. For example, an upper limit may be set for the number of objects to be selected. As an example, up to two objects may be selected. This makes it possible to specify an object more quickly and easily than when three or more objects are selected. As another example, a maximum of three objects may be selected. Considering that humans can more easily process and store less than three things, this makes it possible to specify objects more quickly and easily while displaying information about more objects Become.

  Moreover, although the example in which the communication ID is a telephone number has been described, the present invention is not limited to this. The communication ID may be an ID other than a telephone number. As an example, the communication ID may be a softphone ID other than a telephone number. As another example, the communication ID may be an ID for communication other than a telephone. For example, the communication ID may be a mail address or a short message ID. In this case, a mail or a short message may be transmitted using the communication ID.

  Moreover, although the example in which the position of the captured image is designated by a user touch on the display screen has been described, the present invention is not limited to this. For example, the position of the captured image may be specified by the user using input means other than the touch panel. For example, the position of the captured image may be specified by clicking with a mouse, or may be specified using another input means such as a button or a keyboard.

  Further, although an example in which the object corresponding to any position of the captured image is a person (or information related to a person) has been described, the present invention is not limited to this. For example, the object may be a communication device (or information regarding the communication device). As an example, the communication device may be a telephone.

  Moreover, although the captured image produced | generated by the camera, the audio | voice data produced | generated by the microphone, and the determination result by a sensor were demonstrated via the media delivery server, the example was described, This invention is not limited to this. As an example, these data may be provided via another device instead of the media distribution server. As another example, these data may be directly provided to the terminal device by a camera, a microphone, and a sensor.

  In addition, although the example in which the position acquisition unit, the object selection unit, the display control unit, the communication ID acquisition unit, and the telephone unit are provided by the terminal device has been described, the present invention is not limited to this. For example, these functions may be provided by a device other than the terminal device. As an example, these functions may be provided by any server.

  Moreover, although the position acquisition unit, the object selection unit, the display control unit, the communication ID acquisition unit, and the telephone unit have been described as being provided in the same device (terminal device), the present invention is not limited to this. The position acquisition unit, the object selection unit, the display control unit, the communication ID acquisition unit, and the telephone unit may be provided by separate devices. As an example, the position acquisition unit, the object selection unit, and the display control unit may be provided by a display installed in any server or satellite office, and the communication ID acquisition unit and the telephone unit may be provided by the terminal device. Good.

  Moreover, although the example in which the display screen of the captured image of the real space is displayed by the terminal device has been described, the present invention is not limited to this. For example, the display screen may be displayed by another device. As an example, the display screen may be displayed by a display installed in a satellite office. Then, the user may specify the position of the captured image included in the display image on the display.

  Further, the processing steps in the information processing of the present specification do not necessarily have to be executed in time series in the order described in the flowchart. For example, the processing steps in the information processing may be executed in an order different from the order described in the flowchart, or may be executed in parallel.

  In addition, it is possible to create a computer program for causing hardware such as a CPU, a ROM, and a RAM built in an information processing apparatus (for example, a terminal apparatus) to exhibit functions equivalent to the respective configurations of the information processing apparatus. A storage medium storing the computer program is also provided.

11 Camera 13 Microphone 15 Sensor 17 Media Distribution Server 18 Information Management Server 19, 23 LAN (Local Area Network)
21 Display 30 External network 40 PBX (Private Branch eXchange)
60, 80 Display screen 61 Captured image 71 Position specified by user 73 Position corresponding to object 85 Image of partial area (enlarged image)
99 Identification information 100 Terminal device 181 Position acquisition unit 183 Object selection unit 185 Display control unit

Claims (19)

An acquisition unit that acquires a position in the captured image specified by a user on a display screen including a captured image of real space;
A selection unit that selects one or more objects from a plurality of objects respectively corresponding to any position of the captured image based on the acquired position;
A display control unit that displays information on each of the two or more objects on a display device that displays the display screen when two or more objects are selected from the plurality of objects;
With
The selection unit selects one object specified by the user from the two or more objects,
When the one object is selected, identification information for communication corresponding to the one object is acquired.
Information processing device. The two or more objects include a first object and a second object;
The information regarding each of the two or more objects includes an image of a first partial area in the captured image including a position in the captured image to which the first object corresponds, and the second object Including an image of a second partial region in the captured image including a corresponding position in the captured image;
The information processing apparatus according to claim 1.   The information processing apparatus according to claim 2, wherein the display control unit causes the display device to display the image of the first partial area and the image of the second partial area in parallel. The image of the first partial region is an enlarged image in which the first partial region is enlarged,
The image of the second partial area is an enlarged image in which the second partial area is enlarged.
The information processing apparatus according to claim 2 or 3. The image of the first partial area is an enlarged image in which the first partial area is enlarged at a first enlargement ratio,
The image of the second partial area is an enlarged image in which the second partial area is enlarged at a second enlargement ratio.
The information processing apparatus according to claim 4.   The display control unit includes, on the display device, a first boundary sign indicating a boundary between the image of the first partial region and a region other than the image, and the image of the second partial region. The information processing apparatus according to claim 2, wherein a second boundary sign indicating a boundary with a region other than the image is displayed. The first boundary sign includes information about the first object;
The second boundary sign includes information about the second object;
The information processing apparatus according to claim 6. The information regarding the first object includes first identification information for identifying the first object;
The information related to the second object includes second identification information for identifying the second object.
The information processing apparatus according to claim 7.   The information processing apparatus according to claim 1, wherein the information regarding each of the two or more objects includes identification information for identifying each of the two or more objects.   The information processing according to any one of claims 1 to 9, wherein the display control unit causes the display device to display a sign for allowing a user to specify one object from the two or more objects. apparatus.   The information according to any one of claims 1 to 10, wherein the one or more objects are objects corresponding to positions in the region corresponding to the acquired position among the plurality of objects. Processing equipment. The acquisition unit acquires one position in the captured image specified by the user on the display screen,
The one or more objects are objects corresponding to positions in a region of a predetermined shape including the one position to be acquired among the plurality of objects.
The information processing apparatus according to claim 11. The acquisition unit acquires two or more positions in the captured image specified by the user on the display screen;
The one or more objects are objects corresponding to positions in an area formed based on the two or more acquired positions among the plurality of objects.
The information processing apparatus according to claim 11. The selection unit does not select any object when none of the positions corresponding to the plurality of objects is within the region corresponding to the acquired position,
The display control unit causes the display device to display on the display screen a sign for allowing a user to specify a position in the captured image when no object is selected.
The information processing apparatus according to any one of claims 11 to 13.   The information processing apparatus according to claim 1, wherein the identification information for communication is identification information for communication in a dedicated network.   The information processing apparatus according to claim 1, wherein the identification information for communication is identification information for communication via a public network.   The information processing apparatus according to claim 16, wherein the public network includes a mobile communication network. The display device is a display surface of a touch panel,
The position in the captured image is designated by a user touch on the touch panel.
The information processing apparatus according to claim 1. Computer
An acquisition unit that acquires a position in the captured image specified by a user on a display screen including a captured image of real space;
A selection unit that selects one or more objects from a plurality of objects respectively corresponding to any position of the captured image based on the acquired position;
A display control unit that displays information on each of the two or more objects on a display device that displays the display screen when two or more objects are selected from the plurality of objects;
Function as
The selection unit selects one object specified by the user from the two or more objects,
When the one object is selected, identification information for communication corresponding to the one object is acquired.
program.

Images