JP2011176599A - Space information visualization system and related information providing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a space information visualization system that extracts related information on an object such as a person and equipment and displays it on a space image in a superimposed manner, thereby achieving smooth communication. <P>SOLUTION: The space information visualization system includes: an imaging means for capturing an image in a space; a position detection means for positioning the devices in the space to determine positioning information showing a corresponding position on the image captured by the imaging means for each device; a related information collection means for collecting the related information on each object in which the devices are provided; and a related information holding means for holding the related information collected by the related information collection means in association with the positioning information of the devices determined by the position detection means. A display device displays, at positions corresponding to the positioning information of the devices from the position detection means, object images provided to the devices, and displays the related information of the objects. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a spatial information visualization system and related information providing device, for example, information on a person or device in an indoor space is displayed on a spatial image and visualized in real time, and the spatial information can be confirmed from a remote location. The present invention can be applied to a visualization system and a related information providing apparatus.

  For example, when working in a distributed environment, in order to achieve efficient communication between people and people and equipment, in the same way as when working in the same space, It is effective to grasp intuitively.

  In particular, if you can visualize various information in an indoor space where people and devices are operating in real time and confirm this from a remote location, you can intuitively understand the status of the person you want to contact and the status of the device you want to use. Can act quickly and smoothly according to the situation.

  Conventionally, there is a technique disclosed in Patent Document 1 as a technique for visualizing a person existing in an indoor space in real time. In Patent Document 1, in an indoor space where a person wearing a wireless tag is present, a plurality of tag receivers measure the reception intensity and estimate the position of the wireless tag based on the reception intensity. Furthermore, a method is described in which an image of a person wearing a wireless tag is cut out by detecting a person shape from a peripheral area of the position on a live image in an indoor space.

  By applying this method, the position of the wireless tag or a person image around it can be taken by a camera, so that a person existing in the indoor space can be visualized in real time.

JP 2009-124539 A

  By the way, in communication between people and between people and devices, it is necessary to proceed with the dialogue in a manner adapted to the situation while grasping the situation of the person and the device. Sometimes the situation and location of people and equipment estimated from the above are not enough.

  For example, if you are a person, your presence (for example, what state you are currently in), contact means (for example, phone calls, instant messages, etc.), history and schedule of actions (for example, where you have been so far) If you know related information, such as who you are meeting with), you can have a smooth conversation. On the other hand, if it is a device, you can understand related information such as operating status and environmental status (for example, power consumption, temperature, etc.). Equipment usage can be controlled smoothly.

  However, even if the technique described in Patent Document 1 is used, a person in an indoor space can be projected in real time, but there is a problem that related information such as a person and a device cannot be provided.

  Therefore, in order to smoothly and efficiently perform communication between people and between people and devices, for example, the position and image of a target such as a person or device are superimposed and displayed on a spatial image. There is a need for a spatial information visualization system that can extract and superimpose related information (for example, people and devices).

  Therefore, the present invention solves the above-mentioned problems, by identifying the target such as a person or a device, and further extracting the relevant information of the target such as a person or a device and superimposing it on a spatial image. An object of the present invention is to provide a spatial information visualization system and a related information providing apparatus capable of smoothly promoting communication.

  In order to solve such a problem, the spatial information visualization system according to the first aspect of the present invention detects the positions of devices provided in a plurality of objects existing in the space, and sets the corresponding positions in the display video of the display device. A spatial information visualization system that superimposes and displays a target video showing a target to visualize the space, and (1) an imaging unit that captures a video in the space; and (2) a position of each device in the space. (3) position detection means for obtaining positioning information indicating the corresponding position on the video imaged by the imaging means for each device; (3) related information collection means for collecting related information on each target provided with each device; 4) a related information holding means for holding the related information collected by the related information collecting means in association with the positioning information of each device obtained by the position detecting means; and (5) the display device includes the position detecting means. Each outfit from The position corresponding to the positioning information, and displays the object image provided in the apparatus, and displaying the related information of the target.

  The related information providing apparatus according to the second aspect of the present invention is an image pickup means for picking up an image in a space, and a position of each device provided in a plurality of objects existing in the space, and an image picked up by the image pickup means. A space for visualizing the space by superimposing and displaying the target video and the related information indicating each target at the corresponding position in the display video of the display device. In the related information providing apparatus constituting the information visualization system, (1) related information collecting means for collecting related information about each object provided with each apparatus, and (2) related information collected by the related information collecting means, And related information holding means for holding the information in association with the positioning information of each device obtained by the position detecting means.

  According to the present invention, for example, the position of a target such as a person or a device and the target image are superimposed and displayed on a spatial image, and further, related information of the target (for example, a person or a device) is extracted and superimposed. Thus, communication between people and between people and devices can be performed smoothly and efficiently.

It is a whole lineblock diagram showing the whole spatial information visualization system composition of an embodiment. It is an internal block diagram which shows the internal structure of the positioning system of embodiment. It is an internal block diagram which shows the internal structure of the application system of embodiment. It is an internal block diagram which shows the internal structure of the display terminal of embodiment. It is a sequence diagram which shows the process regarding the positioning of the ID terminal of embodiment. It is a sequence diagram which shows an example of the process which correct | amends the positioning data of the ID terminal of embodiment. It is a sequence diagram which shows an example of the process which correct | amends another positioning data of embodiment. It is a sequence diagram which shows the collection process of ID relevant information by the application system of embodiment. It is a figure which shows the example of the screen display displayed on the display part of the display terminal of embodiment. It is a sequence diagram which shows an example of the search of target ID relevant information from the display terminal of embodiment, and the execution process of an application.

(A) Main Embodiments Hereinafter, embodiments of the spatial information visualization system and related information providing apparatus of the present invention will be described with reference to the drawings.

(A-1) Configuration of Embodiment (A-1-1) Overall Configuration FIG. 1 is an overall configuration diagram illustrating the overall configuration of the spatial information visualization system of this embodiment. In FIG. 1, a spatial information visualization system 10 includes a plurality of ID terminals 1 (1-1 to 1-N: N is a positive integer), a positioning system 2, an information transmission device 3, a display terminal 4, an imaging device 5, an application. A system 6 is provided at least.

  ID terminals 1-1 to 1-N (hereinafter referred to as ID terminal 1) are terminals to which identification information (hereinafter referred to as ID data) is assigned, respectively. For example, the ID terminal 1 is a portable terminal (for example, a mobile phone, a PDA terminal, a PHS terminal, etc.) used by a person, a card terminal (for example, an employee ID card) possessed by a person, a personal computer, a printer, a FAX, for example. It is applied as a device mounted on a device such as an air conditioner.

  The ID terminal 1 has at least a wireless communication function and a reception strength measurement function, receives a wireless signal transmitted from a positioning system 2 described later, and measures the wireless strength (reception strength). The ID terminal 1 gives the measured wireless intensity data to the positioning system 2. Furthermore, the ID terminal 1 receives ID terminal control data relating to one or a plurality of applications from the application system 6. Here, the ID terminal control data is control information exchanged for performing an application. Thereby, information necessary for each application can be transmitted to the application system 6 or the application can be realized via the application system 6.

  The positioning system 2 measures the position of the ID terminal 1 existing in the space. The positioning system 2 periodically transmits a radio signal into the space and receives radio intensity data from the ID terminal 1 that has received the radio signal. The positioning system 2 obtains the position of the ID terminal 1 (also referred to as first positioning data) based on the wireless strength data received from the ID terminal 1.

  In addition, the positioning system 2 obtains image data and imaging parameters from the imaging device 5, feature data such as a person or a device associated with the ID data acquired from the application system 6, or shape features such as a person or a device. Using the shape feature amount data shown, the feature amount data or the shape feature amount data from the position corresponding to the first positioning data or its peripheral region within the image region based on the image pickup data from the image pickup device 5 and the image pickup parameters A matching feature is found and the position of this feature (also referred to as second positioning data) is obtained. Thereby, the first positioning data can be corrected, and the position of the target with high accuracy can be measured.

  Furthermore, the positioning system 2 provides the positioning data (here, the second positioning data) to the information transmission device 3. At this time, the positioning data is given to the information transmission device 3 and the application system 6 in association with at least the ID data of the ID terminal.

  In this embodiment, the second positioning data is given to the information transmission device 3 and the application system 6. However, the present invention is not limited to this, and the first positioning data may be given.

  The information transmission device 3 receives positioning data associated with the ID data from the positioning system 2, receives imaging data and imaging parameters from the imaging device 5, and ID-related data associated with the ID data from the application system 6. Receive. Then, the information transmission device 3 gives the received positioning data, imaging data and imaging parameters, and ID related data to the display terminal 4 as display data associated with each ID data.

  The display terminal 4 is a terminal having at least a display unit, an operation unit, a control unit, and the like. The display terminal 4 outputs an image to the display unit based on the display data received from the information transmission device 3. At this time, the display terminal 4 displays an image indicating a target such as a person or a device having the ID terminal 1 at a position corresponding to each positioning data in the area of the spatial video based on the imaging data and the imaging parameter display data. Or ID related information related to the object is displayed.

  As an example of processing using the display terminal 4, for example, a target image displayed on the display unit is selected via the operation unit, thereby requesting collection of ID-related information on the target, for example, By being selected together with an instruction of an application desired by the user, an application (for example, a telephone, an instant message, etc.) with the target can be started. In the case of this example, the display terminal 4 gives application input data to the application system 6 by selecting the target, whereby the application system 6 receives the ID related information (for example, presence, telephone number, instant mail address, etc.). Can be displayed on the display terminal 4. Then, the display terminal 4 can execute an application (for example, a telephone, an instant message, etc.) by the application system 6 by giving the application input data requesting the application to the application system 6.

  The imaging device 5 images a space where the ID terminal 1 exists. The imaging device 5 gives the captured imaging data and imaging parameters to the positioning system 2 and the information transmission device 3.

  When the application system 6 receives the application input data requesting the ID related information from the display terminal 4, the application system 6 searches and collects the ID related information associated with the requested position of the ID terminal 1, and obtains the ID related information. This is given to the display terminal via the information transmission device 3. At this time, the application input data includes at least the position of the ID terminal 1. That is, the positioning data of the ID terminal 1 or the ID data and positioning data of the ID terminal 1 is included.

  The application system 6 is a system that provides one or a plurality of applications. When the application system 6 receives application input data requesting execution of an application from the display terminal 4, the application system 6 executes the requested application.

  Furthermore, the application system 6 holds feature data indicating characteristics of people, devices, etc., in association with ID data. When receiving a request for feature data from the positioning system 2, the application system 6 responds to the requested ID data. The feature amount data to be given is given to the positioning system 2. Further, the application system 6 provides ID terminal control data to the ID terminal 1.

(A-1-2) Internal Configuration of Positioning System 2 FIG. 2 is an internal configuration diagram showing the internal configuration of the positioning system 2. In FIG. 2, the positioning system 2 includes at least a plurality (four in FIG. 2) of wireless transmission units 21-1 to 21-4 and a position estimation unit 22. The position estimation unit 22 includes at least a wireless transmission unit position information storage unit 221, a positioning data calculation unit 222, a positioning data correction unit 223, and a shape feature amount data storage unit 224.

  The wireless transmission units 21-1 to 21-4 are installed in advance in the space SP, and transmit wireless signals at predetermined intervals (for example, every second). Note that the output intensity for transmitting the radio signal is recognized by the position estimation unit 22 and is set in advance, for example. Further, the wireless signal includes identification information (for example, ID data) for identifying each of the wireless transmission units 21-1 to 21-4. Thereby, the ID terminal 1 that has received the radio signal can recognize which radio signal is transmitted from any of the radio transmission units 21-1 to 21-4, and the radio intensity data of each received radio signal can be obtained. This can be given to the position estimation unit 22.

  The position estimation unit 22 receives the wireless strength data of each wireless signal from the ID terminal 1, estimates the position of the ID terminal 1 using this wireless strength data, and gives the positioning data of each ID terminal 1 to the information transmission device 3. Is.

  The wireless transmission unit position information storage unit 221 stores position information of the wireless transmission units 21-1 to 21-4 disposed in the space SP.

  The positioning data calculation unit 222 illuminates radio propagation characteristics (propagation model) based on the radio strength data of each radio signal received from the ID terminal 1 and the position information of each radio transmission unit 21-1 to 21-4. However, the position of the ID terminal 1 (first positioning data) is obtained.

  The positioning data correction unit 223 searches for an image region or a peripheral region corresponding to the first positioning data, detects a feature that matches the feature amount data or the shape feature amount data from the region, and detects the position of the detected feature ( 2nd positioning data) is calculated | required. Further, the positioning data correction unit 223 gives the obtained positioning data to the information transmission device 3 and the application system 6.

  The shape feature amount data storage unit 224 stores feature amount data of the shape of the target person or the shape of the device in association with the ID data. The shape feature data storage unit 224 reads the stored shape feature data when the positioning data correction unit 223 corrects the positioning data using the shape feature data.

(A-1-3) Internal Configuration of Application System 6 FIG. 3 is a configuration diagram illustrating the configuration of the application system 6. 3, the application system 6 includes at least an application unit 61, a feature amount data management unit 62, a feature amount data holding unit 63, an ID related information management unit 64, and an ID related information holding unit 65.

  The application unit 61 is a processing unit or device that executes an application. In FIG. 3, for the sake of convenience of explanation, a case where a plurality of function units are provided is illustrated, but a configuration in which a plurality of devices (servers, etc.) that provide each function are dispersedly arranged may be used as long as they can cooperate with each function unit. .

  Examples of applications of the application unit 61 include a telephone function unit 611, a conference connection function unit 612, an instant message transfer function unit 613, a business management function unit 614, a presence function unit 615, a schedule function unit 616, and the like.

  The feature amount data management unit 62 registers feature amount data indicating a target feature in the feature amount data holding unit 63 in association with ID data. As the feature amount data, for example, data such as a human face image, upper body image, device image, and color can be applied. In addition, when the feature quantity data management unit 62 receives a request for feature quantity data from the positioning system 2, the feature quantity data management unit 62 retrieves feature quantity data corresponding to the requested ID data from the feature quantity data holding unit 63, and the retrieved feature quantity data. Is given to the positioning system 2.

  The feature data holding unit 63 stores target feature data in association with ID data.

  The ID related information management unit 64 registers or updates the target ID related information in the ID related information holding unit 65 in association with the position of the target ID terminal 1. Information necessary for each application of the application unit 61 can be applied to the ID related information, and can be, for example, presence information, schedule information, device usage status, telephone number, address information related to an instant message, and the like.

  The ID related information management unit 65 holds target ID related information in association with the position of the target ID terminal 1. The management items of the ID related information management unit 65 include, for example, “ID”, “positioning data”, “ID related data” and the like as shown in FIG. For example, in the example of FIG. 3, for the ID terminal 1 with “ID: 1”, “positioning data: (1, x, y)” and presence information “on the phone” are stored in association with each other as ID-related data. Is done.

(A-1-4) Internal Configuration of Display Terminal 4 FIG. 4 is a configuration diagram illustrating the internal configuration of the display terminal 4. In FIG. 4, the display terminal 4 includes at least a display unit 41, an operation unit 42, a control unit 43, and a display data holding unit 44.

  The display unit 41 is a display unit (for example, a display) that displays a display image. The operation unit 42 is operation means (for example, a mouse, a keyboard, etc.) operated by a user who operates the display terminal 4. The display unit 41 and the operation unit 42 may be, for example, a terminal in which the display unit and the operation unit are integrated, such as a touch panel.

  The control unit 43 controls the function of the display terminal 4. The main functions of the control unit 43 include a display data acquisition unit 431, a display data management unit 432, a display image processing unit 433, and an application processing unit 434.

  The display data acquisition unit 431 acquires display data associated with the ID data transmitted from the information transmission device 3.

  The display data management unit 432 associates each of positioning data, imaging data and imaging parameters, and ID-related data, which are display data acquired by the display data acquisition unit 431, with ID data, and ID position-related information. The information is registered / updated in the holding unit 44.

  The display image processing unit 433 forms a display image to be displayed on the display unit 41. The display image processing unit 433 determines the corresponding position on the display image based on the positioning data of the ID terminal 1 as the position of the ID terminal 1. Then, imaging data based on imaging data and imaging parameters is superimposed on the spatial image at the position of the ID terminal 1 to form a display image. Further, the display image processing unit 433 searches the position of the ID terminal 1 corresponding to the ID data associated with the ID related information with reference to the ID position-related information holding unit 44, and sets the ID to the searched position. A related image is superimposed on the spatial image to form a display image. Thereby, the position of the said ID terminal 1 can be pinpointed by selecting objects and ID related information, such as a person and an apparatus projected on a screen by a user.

  The application processing unit 434 requests related information of the ID terminal 1 from the position of the ID terminal 1 designated from the application system 6 or requests the application system 6 to execute the application.

  For example, when an object to be displayed on the display image is selected by a user operation, the application processing unit 434 refers to the display data holding unit 44 based on the selected area and specifies the positioning data of the ID terminal. Then, application input data including at least the positioning data of the ID terminal 1 is given to the application system 6. In addition, application input data for executing a predetermined application is given to the application system 6.

(A-2) Operation | movement of embodiment Next, operation | movement of the spatial information visualization process of this embodiment is demonstrated, referring drawings.

  In the following, there will be exemplified a case in which the situation of the office is confirmed from the remote display terminal 4 when a plurality of persons have ID terminals 1 incorporating different ID data and are in the same space of the office.

(A-2-1) Positioning of ID terminal 1 FIG. 5 is a sequence diagram showing processing related to positioning of the ID terminal 1.

  First, the four wireless transmission units 21-1 to 21-4 constituting the positioning system 2 output short-time wireless signals at regular intervals (for example, every second) (step S101).

  Here, as an example of the radio signal, for example, a signal conforming to a wireless LAN (IEEE802.11) or Bluetooth (registered trademark) (IEEE802.15) is used, and identification of the radio transmitters 21-1 to 21-4 is included therein. Information (for example, ID etc.) shall be included.

  When the ID terminal 1 receives the radio signals transmitted from the four radio transmission units 21-1 to 21-4, the ID terminal 1 measures the reception strength of each of the four radio signals. And ID terminal 1 gives the information which put together the receiving intensity data of each radio signal, and ID data of ID terminal 1 to position estimating part 22 of positioning system 2 as radio intensity data (Step S102).

  Here, when there is no disturbance such as an obstacle or another wireless transmission source between the ID terminal 1 and the wireless transmission units 21-1 to 21-4, the wireless signal is attenuated with the distance and the ID terminal. Therefore, the reception intensity data corresponding to the radio transmission units 21-1 to 21-4 close to the ID terminal 1 among the radio intensity data is transmitted to the radio transmission units 21-1 to 21-4 far from the ID terminal 1. The value is larger than the corresponding reception strength.

  Therefore, the position estimation unit 22 stores the position information of the four wireless transmission units 21-1 to 21-4 in advance in the wireless transmission unit position information storage unit 221, and the positioning data calculation unit 222 stores the wireless intensity data. Is used to estimate the position (x, y) of the ID terminal 1 in light of a wireless propagation model (step S103). Then, the position estimation unit 22 transmits positioning data (ID, x, y) obtained by combining the estimated position (x, y) of the ID terminal 1 and ID data of the ID terminal 1 to the information transmission device 3. .

  At this time, the position estimation unit 22 of the positioning system 2 requests the imaging device 3 for imaging parameters (step S104), and the imaging parameters received from the imaging device 3 (for example, the horizontal direction and the vertical direction). , Zoom amount, etc.)) (step S105), and the position of the ID terminal 1 on the imaging screen is stored and updated in association with the ID data (step S106).

  Here, as a method for estimating the position of the ID terminal, a conventionally known method may be used. However, for example, in this embodiment, a model in which the wireless propagation loss L [dB] can be approximated by Expression (1) is assumed. The position of the ID terminal 1 is calculated.

L = 20 log (d) + α (1)
Here, d is a propagation distance, and α is a constant determined by a radio wavelength.

  For example, according to the equation (1), if the difference of the reception intensity data is calculated for any two combinations of the four reception intensity data, the ratio of the distance from the two wireless transmission units to the ID terminal 1 is obtained. be able to. Therefore, the positioning data calculation unit 222 is based on the positions of the four wireless transmission units 21-1 to 21-4 and the ratio of the distances from the wireless transmission units 21-1 to 21-4 to the ID terminal 1. The position of the ID terminal 1 can be estimated.

  If there is a difference between the wireless propagation model and the actual wireless environment, an error may occur in the positioning data obtained by the above position estimation method. Therefore, the position estimation unit 22 may correct the positioning data as follows.

  FIG. 6 is a sequence diagram illustrating an example of processing for correcting the positioning data of the ID terminal 1.

  In FIG. 6, in the application system 6, the feature amount data management unit 62 displays feature amount data (for example, feature amount data of a human face image) indicating a feature to be mounted on each ID terminal 1 as a feature amount data holding unit. Registered in 63 and stored (step S201).

  After step S103, the position estimation unit 22 requests the feature amount data corresponding to the ID data from the application system 6 (step S202), and acquires the feature amount data from the application system 6 (step S203).

  Furthermore, the position estimation unit 22 requests imaging data from the imaging device 5 (step S204), and acquires imaging data and imaging parameters from the imaging device 5 (step S205).

  Then, in the position estimation unit 22, the positioning data correction unit 223 searches for an image area corresponding to the positioning data (step S206), and finds an image position matching the feature amount data from the image area or its vicinity area (step S206). S207). And you may make it transmit this image position to the information transmission apparatus 3 as positioning data (ID, x, y).

  That is, the application system 6 holds, for example, face image feature amount data for each ID data, and can output face image feature amount data corresponding to the ID data in response to an inquiry from the position estimation unit 22. The imaging device 5 can output imaging parameters including, for example, the horizontal direction, the vertical direction, and the zoom amount together with the imaging data. Therefore, the position estimation unit 22 acquires feature amount data of the face image based on the ID data, and further sets an area where the ID terminal 1 is likely to exist on the imaging data based on the imaging parameters and the positioning data. Then, a person's face is detected therefrom, and matching is performed with the feature amount data of the face image to find a person possessing the ID terminal. As a result, the positioning data (ID, x, y) can be corrected.

  FIG. 7 is a sequence diagram illustrating an example of a process for correcting another positioning data.

  In FIG. 7, in the positioning system 2, the shape feature quantity data storage unit 224 stores the feature quantity data of the person shape (step S301).

  The position estimation unit 22 requests imaging data from the imaging device 5 (step S302), and acquires imaging data and imaging parameters from the imaging device 5 (step S303).

  Then, in the position estimation unit 22, the positioning data correction unit 223 searches for an image area corresponding to the positioning data (step S304), and an image position matching the feature amount data of the person shape from the image area or its vicinity area. Is found (step S305). The image position may be transmitted to the information transmission device 3 and the application system 6 as positioning data (ID, x, y).

(A-2-2) ID-Related Information Collection Processing FIG. 8 is a sequence diagram showing ID-related information collection processing by the application system 6.

  ID-related information is information related to people, devices, and the like, and is various information such as, for example, people's presence information, people's schedule information, and device usage status (for example, usage time, power consumption value, etc.). . Therefore, various methods can be taken according to the type of ID related information.

  Therefore, FIG. 8 illustrates processing for registering presence, schedule, and device usage information as an example of ID-related information.

  Presence information and schedule information are transmitted from the ID terminal 1 or the display terminal 4 to the application system 6 when the user performs a registration operation on the ID terminal 1 or the display terminal 4 (step S401), for example (step S401). Steps S402 and S403). For example, human presence information can be registered in the application system 6 using SIP (Session Initiative Protocol) or the like. Further, for example, human schedule information can be registered in the application system 6 by a user inputting and transmitting a schedule using a user terminal (ID terminal 1 or display terminal 4).

  As for device usage status, for example, the device periodically detects usage status such as usage time and power consumption value (step S404), and transmits the detected device usage status information to the application system 6 (step S404). S405).

  As described above, the application system 6 acquires the ID related information. The application system 6 has received positioning data from the positioning system 2, and the ID related information management unit 64 registers the ID, the positioning data, and the related data in the ID related information holding unit 65 in association with each other. Update (step S406).

(A-2-3) Spatial Information Visualization Processing The imaging device 5 captures the space in which the ID terminal 1 exists, and the imaging data and imaging parameters (for example, horizontal direction, vertical direction, zoom amount, etc.) Is transmitted to the information transmission apparatus 3. The positioning system 2 transmits positioning data (ID, x, y) associated with the ID data to the information transmission device 3. Further, the application system 6 transmits ID related information associated with the ID data to the information transmission apparatus 3.

  Then, the information transmission device 3 uses the imaging data and imaging parameters from the imaging device 5, the positioning data associated with the ID data from the positioning system 2, and the ID related information associated with the ID data as display data. It transmits to the display terminal 4.

  The display terminal 4 displays the spatial information in which the ID terminal 1 exists on the display unit 41 based on the display data received from the information transmission device 3.

  FIG. 9 is a diagram illustrating an example of a screen display displayed on the display unit 41 of the display terminal 4. A display screen 70 illustrated in FIG. 9 is a search window 71 for searching for people, devices, and the like, and a video that displays image data superimposed on the screen with attribute information created based on ID-related information about people, devices, etc. A display window 72 and an object list 73 for displaying a list of objects displayed in the video display window 72 are configured.

  When displaying the attribute information 75 in the video display window 72, the display terminal 4 determines the position of the ID terminal 1 on the imaging data based on the positioning data and imaging parameters of each ID terminal 1, and displays the attribute information at that position. 75 is superimposed and displayed.

  Here, the attribute information display 75 displays one or a plurality of ID related information associated with the ID data. The attribute information display 75 is displayed at the position of the ID terminal 1. That is, the attribute information display 75 is displayed so as to be superimposed on the video of the ID terminal 1.

  Further, when the attribute information display 75 is superimposed on the video of the ID terminal 1, the content of the attribute information display 75 becomes difficult to see, and may be displayed near the video area of the ID terminal 1. For example, the attribute information display 75 may be displayed at a position shifted from the position of the ID terminal 1 by a predetermined amount in the horizontal direction and / or the vertical direction.

  The video display window 72 has an operation interface 74 for designating a display area, and the user operates the operation interface 74 to control the imaging parameters of the imaging device 5. As a result, it is possible to perform zoom-up and zoom-down by moving in the direction (horizontal direction and vertical direction) of the spatial image displayed in the image display window 72 and adjusting the zoom amount.

  FIG. 10 is a sequence diagram illustrating an example of processing for searching for target ID related information and executing an application from the display terminal 4.

  Upon receiving the user operation (step S501), the display terminal 4 transmits application input data including at least the positioning data of the ID terminal 1 to request the application system 6 to search for ID related information such as a person or a device. (Step S502).

  At this time, an object such as a person or a device to be searched by a user operation may be individually selected, or a plurality of people or devices may be selected, or a specific object may be selected. You may make it select for every group.

  Upon receiving the application input data, the application system 6 searches for ID related information corresponding to the positioning data of the ID terminal 1 based on the positioning data of the ID terminal 1 included in the application input data.

  For example, the application system 6 may retrieve the ID related information corresponding to the positioning data of the ID terminal 1 included in the application input data from the ID related information holding unit 65.

  Further, for example, in order to collect ID-related information (for example, human presence information, human schedule, device usage status, etc.) of the ID terminal 1 in real time, the ID terminal 1 included in the application input data Based on the positioning data, the ID related information holding unit 65 searches the ID data of the ID terminal 1. Then, using the ID data of the ID terminal 1, an ID related information acquisition request may be made to the ID terminal 1 to collect the ID related information of the ID terminal 1.

  The application system 6 gives the ID related information of the ID terminal 1 acquired in this way to the information transmission apparatus 3 (step S503).

  Upon receiving the application input data, the information transmission device 3 requests the positioning system 2 for positioning data specifying the ID data to be searched (step S504), and receives the positioning data from the positioning system 2 (step S505). ). At this time, the positioning system 2 may calculate or correct the positioning data described with reference to FIGS. Thereby, the position of the ID terminal 1 can be notified in real time.

  Further, the information transmission device 3 requests imaging data from the imaging device 5 (step S506), and receives imaging data and imaging parameters from the imaging device 5 (step S507).

  Then, the information transmission device 3 uses the imaging data and imaging parameters from the imaging device 5, the positioning data associated with the ID data from the positioning system 2, and the ID related information associated with the ID data as display data. It transmits to the display terminal 4 (step S508). Thereby, the ID related information of the ID terminal 1 to be searched can be displayed on the display unit 41 of the display terminal 4 (step S509).

  When the user selects a specific person or device on the video display window 72 or the object list 73 of the display terminal 4, application input data including the positioning data of the ID terminal 1 corresponding to the position of the selected object is the application. Output to the system.

  The application system 6 uses this as a trigger to display executable service options on the display terminal 4, and when the user selects the option (for example, telephone) (step S510), application input data that causes the application to be executed. (For example, a telephone request or the like) is given to the application system 6 (step S511), and the application system 6 outputs ID terminal control data (for example, a call request or the like) to the ID terminal 1 (step S512). Various applications are executed with the display terminal 4 (step S513).

  For example, by selecting a person on the video display window 72 of the display terminal 4, starting a phone call or instant message with the corresponding ID terminal 1, or selecting a device, the device operation mode ( For example, the power saving mode or the like is changed.

(A-3) Effect of Embodiment As described above, according to this embodiment, the position of the ID terminal 1 is measured, and this positioning data is used for information on camera images and for people and devices managed by the business system. By associating with ID-related information, information related to the user of the ID terminal can be presented intuitively on the camera video. Therefore, it is possible to visualize in real time various information in the indoor space where people and devices are operated, and confirm this from a remote location.

  Moreover, in this embodiment, after estimating the position of the ID terminal 1 based on the reception strength of the radio signal from the radio transmission units 21-1 to 21-4 received by the ID terminal 1, the camera video and the person Since the positioning system 2 is configured to correct the position estimation of the ID terminal 1 based on the feature amount data of the object and the object, the positions of the ID terminal 1 and the person can be grasped in detail and visualized on the camera video. .

  Furthermore, in this embodiment, when the user desires desired ID-related information of a person or an object by associating the positioning data of the ID terminal 1 with the ID-related information, the ID-related information is displayed at the position of the ID terminal 1. Can be displayed, or the application can be easily executed by selecting the application related to the ID related information. Therefore, it is possible to know the current person's presence status and future schedule, the usage status of the device, etc., and to smoothly communicate. Furthermore, it is possible to quickly achieve an application corresponding to these situations.

(B) Other Embodiments (B-1) In the above-described embodiment, the positioning system is configured such that the ID terminal receives wireless signals from four wireless transmission units, and the position estimation unit determines the reception strength of each wireless signal. Although the case where the ID terminal is configured to be positioned based on the above has been described, the same effects as those of the above-described embodiment can be obtained by using another conventionally known method using a radio signal as the configuration of the positioning system. For example, in actual operation, the number of radio transmission units may be increased or decreased in consideration of the balance between positioning costs and accuracy.

  In addition, the positioning of the ID terminal is performed by a positioning method that does not depend on the reception strength of the radio signal, for example, a method of measuring the transmission time of the radio signal and estimating the distance from the ID terminal to each radio transmission unit based on this. You may do it. Further, the ID terminal may be positioned using short-range wireless communication such as RFID between the ID terminal and the wireless transmission unit.

(B-2) In the above-described embodiment, a case has been described in which video and text attribute information are superimposed on all ID terminals included in the camera video and displayed on the display terminal. The information provision format to the display terminal may be designed in a form suitable for the application, and does not stick to the display format shown in the above-described embodiment. For example, for an ID terminal other than the ID terminal to be searched, it is possible to operate such that the attribute information is not displayed.

  If a virtual reality video prepared in advance is used instead of a camera video, it is possible to visualize a space where people and devices operate with a small amount of data. Further, the target such as a person or device displayed on the display terminal may be a person video or device video captured by the imaging device, or may be an alternative video (for example, an avatar) prepared in advance.

(B-3) In the above-described embodiment, a case has been described in which a plurality of people have ID terminals that incorporate different ID data, but the function of the ID terminal is incorporated in a personal computer, a printer, an air conditioner, a lighting, or the like. May be. In this case, if the ID data unique to the user of each device and the ID data unique to the device can be set in the ID terminal, in addition to the same effects as the above-described embodiment, the device-specific attribute information corresponds to the camera video. It can be presented. In this case, as an additional process, the position estimation unit acquires feature amount data related to the device shape from the application system based on the ID data, and further inputs the imaging data and imaging parameters from the imaging device, It is also possible to find the corresponding device from the vicinity of the image area corresponding to the positioning data, determine the positioning data (ID, x, y) from the position on the image, and output it to the information transmission apparatus.

(B-4) In the above-described embodiment, the display terminal exemplifies the case where the positioning data of the ID terminal is described in the application input data. This is intended to enable identification of corresponding ID-related information from the current position of the ID terminal based on the positioning data of the ID terminal. Therefore, if the ID related information of the corresponding ID terminal 1 can be identified from the current position of the ID terminal, the display terminal may include ID data of the ID terminal. In this case, the application system 6 searches the ID related information holding unit for positioning data corresponding to the ID data included in the received application input data, and searches for ID related information corresponding to the searched positioning data. May be.

1-1 to 1-N ... ID terminal, 2 ... positioning system, 3 ... information transmission device, 4 ... display terminal,
5 ... Imaging device, 6 ... Application system, 61 ... Application part,
62 ... feature quantity data management unit, 63 ... feature quantity data holding unit, 64 ... ID-related information management unit,
65 ... ID-related information holding unit, 10 ... spatial information visualization system.

Claims (3)

A spatial information visualization system that detects the positions of devices provided on a plurality of objects existing in a space and visualizes the space by superimposing and displaying the target images indicating the objects at corresponding positions in a display image of a display device. Because
Imaging means for capturing images in the space;
Position detecting means that measures the position of each device in the space and obtains positioning information indicating the corresponding position on the video imaged by the imaging means for each device;
Related information collecting means for collecting related information related to each of the objects provided with the devices;
And related information holding means for holding the related information collected by the related information collecting means in association with the positioning information of each device obtained by the position detecting means,
The display device displays the target video provided in the device at a position corresponding to the positioning information of each device from the position detection means, and displays the related information of the target. Spatial information visualization system. Application providing means for executing a predetermined application with each target provided with each device,
When the target video in the display video is selected, the display device gives an application request including positioning information of the target video to the application providing unit.
The application providing means refers to the related information holding means, and provides a predetermined application using the related information corresponding to the positioning information included in the application request. The spatial information visualization system described. Positioning information indicating the corresponding position on the video imaged by the imaging means for imaging the image in the space and the position of each device provided in the plurality of objects existing in the space and the imaging means A position detecting means for each device, and a spatial information visualization system for visualizing the space by superimposing and displaying the target video and the related information indicating each target at the corresponding position in the display video of the display device In the information providing device,
Related information collecting means for collecting related information related to each of the objects provided with the devices;
Related information holding means for holding the related information collected by the related information collecting means in association with the positioning information of each device obtained by the position detecting means. apparatus.

Images