JP2010198104A - Image display system, portable terminal system, portable terminal equipment, server, and image display method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display display information in an image being displayed, in a real time. <P>SOLUTION: A facility as a subject is imaged by a camera 25, and the image is displayed on a display 26. A feature point in an image being displayed is pointed by a pointer device of a key 22. Also, request information for the feature point information of the facility is input from the key 22, and transmitted to a server 40. Display information on the pointed point in the image is generated by a moving image processing function 241 and an object recognition function 242 which receive the information on the point and the feature point information of the facility that are transmitted from the server 40. A display 26 displays the display information in the image being displayed, by an information adding function 243. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video display system and method thereof, a mobile terminal system and method thereof, a mobile terminal device and method thereof, a server, and a control program and recording medium thereof, and more specifically, display information is superimposed on a captured video in real time. The present invention relates to a video display system and method for displaying, a mobile terminal system and method thereof, a mobile terminal device and method thereof, a server, a control program thereof, and a recording medium.

  Conventionally, research on mixed reality (MR), which is an environment in which real space and virtual space are fused, has been actively conducted. In this research, attention is focused on AR (Augmented Reality) which enhances the sense of reality by superimposing objects in the virtual space on the real space. As a device studied in the AR, a head mounted display (HMD) capable of superimposing and displaying a real image or a real scene image and a computer graphics (CG) image has been proposed.

Japanese Patent Application Laid-Open No. 2004-133867 discloses a related technique in which element information related to elements in the environment is superimposed and displayed on a real image captured by an imaging unit mounted on a portable terminal device moving in the environment and displayed on the display unit. Are listed.
This related technology installs the element information transmitting device at a required position in the environment where the user moves, while the element information acquisition means, the photographing means, the position measuring means, the posture measuring means, in the portable terminal device, A display means and a superimposing drawing processing means are provided, and imaging is performed based on position information measured by the position measuring means and posture information measured by the posture measuring means when the user approaches an element in the environment. This is an information providing system for displaying the element information acquired by the element information acquisition unit in a superimposed manner on the real image captured by the unit and displayed on the screen of the display unit.
Similarly to this related technology, related technology for superimposing or deriving required display data on the video being displayed in relation to the entire video captured by the camera or imaging means, It is described in Patent Documents 2 to 4.

JP 2004-102835 A JP 2006-329967 A JP 2007-292664 A JP 2008-123028 A

  By the way, the information providing system described in Patent Document 1 described above can display the element information acquired by the element information acquisition unit on the actual image displayed on the screen of the display unit, For superimposing display, an element information transmitting device is installed at a required position in the environment, while an element information acquisition means must be provided in the portable terminal device, and superposition control of element information acquired by the element information acquisition means is performed. For this purpose, the position measuring means and the posture measuring means are provided, and the structure can achieve the intended purpose only.

Therefore, the superimposed display of the element information on the live-action image can be performed only under the above-mentioned limited conditions, and the superimposed display can be performed without such conditions or according to other conditions. It can be said that it remains a problem whether or not it becomes possible.
In addition, the related techniques described in Patent Documents 2 to 3 cannot find any suggestion about display information display in the video in relation to the captured video and the imaging target.

  The present invention has been made in view of the above circumstances, and a video display system and method for displaying necessary display information determined in relation to a subject in real time at an arbitrary position in a video obtained by photographing the subject, and a mobile phone It is an object of the present invention to provide a terminal system and method thereof, a portable terminal device and method thereof, a server, a control program thereof, and a recording medium.

  In order to solve the above problems, a first configuration of the present invention relates to a video display system for displaying display information in a video obtained by photographing a subject, a display means for displaying a video obtained by photographing the subject, and the display means Indicating means for indicating a feature point in the image displayed on the display, and display control means for displaying display information corresponding to the instruction in the image displayed on the display means based on the instruction of the indicating means It is characterized by comprising.

  According to a second configuration of the present invention, there is provided a portable terminal system for displaying display information in a video obtained by photographing a subject with a terminal device. The display means of a terminal device for displaying a video obtained by photographing a subject facility; and the display means Instructing means of the terminal device instructing the feature point in the video displayed on the display, and display information corresponding to the instruction based on the instruction of the instructing means is displayed in the video displayed on the display means And a display control means.

A third configuration of the present invention relates to a video display method for displaying display information in a video obtained by shooting a subject. The subject is shot by a shooting means, and the shot video is displayed on a display means.
A feature point in the video displayed on the display means is indicated by the instruction means, and display information corresponding to the instruction is displayed in the video displayed on the display means based on the instruction from the instruction means. It is characterized by.

  A third configuration of the present invention relates to a video display method of a mobile terminal system that displays display information in a video obtained by shooting a subject with a terminal device. The subject is shot with a shooting means of the terminal device, and the shot video is recorded as described above. Displayed on the display means of the terminal device, the feature point in the video displayed on the display means is indicated by the instruction means of the terminal device, and the display information corresponding to the instruction based on the instruction of the instruction means is It is characterized by being displayed in the video displayed on the display means.

  According to the present invention, display information can be displayed in real time in accordance with an indication of a feature point in the video displayed on the display means.

It is a block diagram which shows the electric constitution of the portable terminal system which is Embodiment 1 of this invention. It is a block diagram which shows the electric constitution of the portable terminal device used with the portable terminal system. It is a figure which associates and shows the display in the portable terminal device and a part of the stadium projected on the display screen. It is a block diagram which shows the electric constitution of the portable terminal system which is Embodiment 2 of this invention. It is a block diagram which shows the electric constitution of the portable terminal device which is Embodiment 3 of this invention.

Embodiment 1

  1 is a block diagram showing an electrical configuration of a mobile terminal system according to Embodiment 1 of the present invention, FIG. 2 is a block diagram showing an electrical configuration of a mobile terminal device used in the mobile terminal system, and FIG. These are figures which show a display on the portable terminal device and a part of the stadium projected on the display screen in association with each other.

  The mobile terminal system 10 of this embodiment superimposes information that the user wants to know about the video at a required position in the video displayed on the display screen of a terminal device such as a mobile phone that the user has and uses. As shown in FIG. 1, the system that can be displayed generally includes a terminal device 20 such as a mobile phone, an information communication network 30, and a server 40.

As shown in FIG. 2, the terminal device 20 includes a wireless control unit 21, a key 22, a memory 23, a central processing device 24, a camera 25, and a display 26. The radio control unit 21 is detachably provided with an antenna 27 that transmits and receives radio waves to and from a base station described later.
The wireless control unit 21 is means for controlling transmission / reception between a communication signal transmitted and received by the antenna 27 and digital information in the memory 23.

The key 22 is configured to include a touch panel and a pointer device, and any facility (photographed facility) to be imaged by the camera 25 by operating the key 22, for example, information regarding the stadium (stadium name, seat position, watching sports, etc. Information) (request information for requesting the server 40 to return the feature point information of the subject facility), and the touch panel or pointer device is displayed on the screen of the display 26 described later. This is an input means for arbitrarily designating an arbitrary feature point in the video image, temporarily storing the instruction information in the memory 23, and using it for the arithmetic processing in the central processing unit 24. Here, the feature point is an arbitrary position in the video for deriving the display information to be added to the display information to be added to the video being displayed, for example, the position of the player, the position of the goal, or the like.
The memory 23 is connected to the radio control unit 21 described above, and is a storage unit that is connected to the key 22, the central processing unit 24, the camera 25, and the display 26 and stores digital information exchanged with these units. is there.

The central processing unit 24 includes a moving image processing function 241, an object recognition function 242, and an information addition function 243. The moving image processing function 241, the object recognition function 242 and the information addition function 243 are configured to perform the procedure of each function by reading a program stored in the memory 23 or the like and executing it by a processor (not shown). ing.
The moving image processing function 241 is a function for recognizing a player or the like displayed on the screen of the display 26 as a feature point based on instruction information stored in the memory 23.

The object recognition function 242 determines the position of the feature point based on the recognition by the moving image processing function 241 and outputs it as feature point information, or based on facility information (feature point information) received from the server 40 described later. This is a function for determining the location of a facility part (feature point) in a stadium or the like, for example, a goal, displayed on the screen of the display 26.
The information addition function 243 includes display information to be added to the image displayed on the screen of the display 26 based on one or a plurality of feature point information determined and output by the object recognition function 242 and its display position, for example, This is a function for obtaining the distance from the competitor or the like to the goal and the display position thereof and outputting them to the display 26.
The camera 25 is a photographing unit that selectively images an arbitrary facility part in a facility such as a stadium.
The display 26 is a display means for displaying a video (real video) captured by the camera 25 and necessary display information desired by the user in the real video.

As shown in FIG. 1, the information communication network 30 includes a plurality of base stations 31, a plurality of switching stations (web access) 32, and a network 33 such as a LAN (Local Area Network) or a WAN (Wide Area Network). This is a communication line network configured to form a communication line between the terminal device 20 and the server 40.
The server 40 is connected to the base station 31S of the information communication network 30, for example, via a wireless communication line, and includes a network connection unit 401 and an information processing function 402.

The network connection unit 401 of the server 40 is an interface unit that performs information transmission / reception processing via a wireless communication line. The information received in this embodiment is request information (stadium name, seat position, spectator sports identification, etc.) regarding the stadium and the like transmitted from the terminal device 20, and the transmitted information is in a live-action video such as a stadium. Is facility information (stadium or subject distance information, position information, etc.) (hereinafter also referred to as stadium or subject feature point information, which will be described later) necessary for generating display information to be added by the user.
The information processing function 402 is based on the request information about the stadium and the like delivered from the network connection unit 401 and the design information (design data) of the stadium and the like stored in advance in a memory or the like in the server 40 (not shown). Digital processing means for generating facility information necessary for generating display information to be added by the user in the video and passing it to the network connection unit 401. The function is stored in a storage means (not shown) in the server 40. By executing the program with a processor (not shown), the above generation process is performed.

Next, the operation of this embodiment will be described with reference to FIGS.
An explanation will be given by taking as an example the case of going to a stadium or the like and watching a competition, for example, soccer. For convenience of explanation, the base station in the service area where the mobile phone 20 is located is assumed to be a base station 31T among the base stations 31 in the information communication network 30. A base station that provides a communication service to the server 40 is a base station 31T among the base stations 31 in the information communication network 30.

  It is assumed that the player is in a soccer battle view using a mobile phone 20 possessed by a user who is a spectator who has entered the stadium. At that time, the user operates the key 22 of the mobile phone 20 and inputs information (request information) such as a stadium name, a seat position, and a watching sport, which are information regarding the stadium.

This request information is written from the key 22 to the memory 23 under the control of the central processing unit 24 and then sent to the radio control unit 21. The wireless control unit 21 converts the input digital request information into a wireless transmission signal and emits it as a radio wave via the antenna 27.
The radio wave is received by the base station 31T, and the digital request information demodulated by the base station 31T is transmitted to the base station 31S via the switching station 32 of the information communication network 30. The request information is transmitted to the server 40 via the wireless communication line.

When the request information is received by the network connection unit 401 of the server 40, the request information is passed to the information processing function 402.
The information processing function 402 that has received the request information calculates and generates facility information of the stadium based on the request information and the design information of the stadium. This information (stadium feature point information) is used to generate display information that the user wants to add to the video (actual video) captured by the mobile phone 20 possessed by the user and displayed on the screen of the display 26. Necessary information.
The feature point information output from the information processing function 402 is transmitted from the information processing function 402 to the base station 31S via the network connection unit 401 and the wireless communication line.

The feature point information received by the base station 31S is transmitted to the base station 31T via the information communication network 30, and is transmitted from the base station 31T to the mobile phone 20 via radio waves via radio waves.
In the cellular phone 20, radio waves are received by the antenna 27, and the received signal is converted into digital signal feature point information by the wireless control unit 21. The feature point information is stored in the memory 23 under the control of the central processing unit 24. Is remembered.

  In this way, the feature point information is given to the mobile phone 20 and the user points the camera 25 of the mobile phone 20 to the live scene that the user wants to watch in the stadium, and the video of the live scene is displayed on the screen of the display 26. In the state of being displayed, the feature points on the screen, and the feature points that the user is particularly interested in in the actual situation, that is, the feature points related to the information that the user wants to know are displayed on the touch panel or the key 22 Point through the pointer device.

Then, the feature point information (hereinafter referred to as a feature point on the live-action video) is stored in the memory 23 under the control of the central processing unit 24. Thereafter, the central processing unit 24 reads out the feature point information of the stadium and the feature point information on the live-action video from the memory 23 and uses them for processing by the moving image processing function 241 and the object recognition function 242.
The moving image processing function 241 and the object recognition function 242 measure the distance between the first place and the second place in the stadium based on the supplied stadium feature point information and the feature point information on the live-action video. This measurement is performed as follows.

  Feature point information (stadium feature point data) related to two feature points in the stadium (the above-mentioned first place and second place) corresponding to the two feature points indicated on the live-action video is as described above. As described above, the position is recognized and determined by the object recognition function 242 from the server 40 to the mobile phone 20, while the feature point information (feature point data) on the live-action video, that is, the feature point information of the stadium. The corresponding feature point information is instructed and input by the pointer device, recognized by the moving image processing function 241, and recognized by the object recognition function 242, and its position is determined.

  Each feature point information determined by the object recognition function 242 is passed to the information addition function 243, and the information addition function 243 obtains the distance between the first location and the second location as distance information. Then, the information adding function 243 sets the distance information to the specified feature point in the live-action video information being displayed on the display 26 or the video information in the vicinity thereof, and the real-time video information including the distance information is displayed on the display 26. Output. The display 26 visually displays the distance information at or near the designated feature point in the live-action image displayed in response to the real-image information including the distance information.

For example, as shown in FIG. 3, when the player p1 and the goal p2 shown on the screen of the display 26 of the mobile phone 20 are indicated by a pointer device, the feature point information is recognized by the moving image processing function 241. Each indicated position (feature point information) is determined by the object recognition function 242.
At this time, the feature point information of the player P1 and the goal P2 competing at the ground of the stadium is calculated and generated by the server 40 based on the request information regarding the stadium transmitted from the mobile phone 20 to the server 40. It is returned to the mobile phone 20 via the information communication network 30. Therefore, the object recognition function 242 recognizes the feature points of the player P1 and the goal P2 corresponding to the instruction, and determines the positions (feature point information) of the player P1 and the goal P2.

The information addition function 243 of the mobile phone 20 calculates the distance information from the player P1 to the goal P2 based on the determination result in the object recognition function 242, and responds to the above instruction in the live-action video information being displayed. Set in the video information area. The video information generated in this way is supplied to the display 26 and displayed on the screen.
In this way, information that the user desires to display, that is, the distance from the player P1 to the goal P2, can be displayed on the screen of the display 26.

  As described above, according to this embodiment, by indicating a feature point in a live-action image displayed on a display, display information that the user wants to know can be displayed in the vicinity of the indicated point in real time.

Embodiment 2

FIG. 4 is a block diagram showing an electrical configuration of the portable terminal system according to the second embodiment of the present invention.
The configuration of this embodiment is greatly different from that of the first embodiment in that display information that the user wants to display on the display screen of the display is generated on the server side and transmitted to the mobile terminal device.
Therefore, as shown in FIG. 4, the mobile terminal system 10A according to the present embodiment transmits a moving image, instruction information, and facility designation information (described later) taken from the mobile phone 20A to the security server 40A, as well as the first embodiment. A moving image processing function 241, an object recognition function 242 and an information addition function 243 provided in the mobile phone 20 are configured to be provided as a moving image processing function 403, an object recognition function 404 and an information addition function 405 in the security server 40A.

The moving image processing function 403, the object recognition function 404, and the information addition function 405 are configured in the same manner as in the first embodiment, and the functions thereof are also the same as those in the first embodiment, and thus the description thereof is omitted.
In addition, since the other configuration of this embodiment is the same as the configuration of the first embodiment, a detailed description of the configuration is omitted.

Next, the operation of this embodiment will be described with reference to FIG.
Since the basic operation is the same as that of the first embodiment except for the points described below, only the differences will be described. The difference is that a moving image captured by the camera 25 of the mobile phone 20A is transmitted from the memory 23, the wireless control unit 21, and the antenna 27 to the security server 40A via the information communication network 30. In addition, the facility designation information (designated facility designation information) corresponding to the request information in the first embodiment and the instruction information designated by the point device are also transmitted to the security server 40A in the same manner.

  On the other hand, the security server 40A receives the transmitted moving image, instruction information, and facility designation information via the network connection unit 401. The received moving image and instruction information are processed in the same manner as in the first embodiment by the moving image processing function 403, and the result is passed to the object recognition function 404. The object recognition function 404 calculates the feature point information of the subject facility based on the facility designation information received and the design data of the subject facility previously stored in the security server 40A, and the computation result and the moving image processing Based on the result passed from the function 403, the position of the feature point corresponding to the instruction information (feature point information (also referred to as feature point information on a live-action image)) is calculated. The information addition function 405 generates display information corresponding to the instruction information and outputs video information in which the display information is set.

  The video information generated by the security server 40A is transmitted from the network connection unit 401 to the mobile phone 20A via the information communication network 30, and displayed on the screen of the display 26 of the mobile phone 20A in the same manner as in the first embodiment. The display information is displayed in real time at a position corresponding to the instruction in the video (moving image) being displayed.

  As described above, also in this embodiment, display information corresponding to the instruction generated by the security server is displayed in real time at a position corresponding to the instruction in the video (moving image) displayed on the screen of the mobile phone. can do.

Embodiment 3

FIG. 5 is a block diagram showing an electrical configuration of the portable terminal device according to the third embodiment of the present invention.
The configuration of this embodiment is significantly different from that of the first embodiment. The video that is displayed on the display screen after the user performs all the generation processing of the display information that the user wants to display on the display screen of the display. The display information is displayed inside.
Therefore, as shown in FIG. 5, the mobile terminal device 10B of this embodiment includes, in addition to the moving image processing function 241, the object recognition function 242 and the information addition function 243 provided in the mobile phone 20B of the first embodiment. In addition, an information processing function 244 corresponding to the information processing function 402 provided in the server 40 of the first embodiment is provided, and a memory card 50 for storing design data of a shooting facility stored in advance in the server 40 is provided detachably. Configured. The design data of the film facility stored in the memory card 50 is for one or the number of film facilities desired by the user.

The information processing function 244 is configured in the same manner as in the first embodiment, and the function thereof is also the same as that in the first embodiment, and thus the explanation thereof is omitted.
In addition, since the other configuration of this embodiment is the same as the configuration of the first embodiment, a detailed description of the configuration is omitted.

Next, the operation of this embodiment will be described with reference to FIG.
Since the basic operation is the same as that of the first embodiment except for the points described below, only the differences will be described. The difference is that, in the first embodiment, the mobile phone 20B calculates the feature point information regarding the user-desired object facility sent from the server 40 to the mobile phone 20.
That is, in the information processing function 244, the generation of the feature point information for the object facility being photographed by the mobile phone 20B is basically the same as in the first embodiment, but in this embodiment, the display information is generated. Prior to this, the memory card 50 is read out, based on the design data of the subject facility. The generated feature point information of the subject facility is transferred from the object recognition function 242 to the information addition function 243 together with the feature point information on the live-action video, as in the first embodiment.

  The display information generated as described above is stored in the information area corresponding to the above instruction in the video information used for display on the screen of the display 26 in the information addition function 243, as in the first embodiment. The video information after the setting is supplied to the display 26 and is displayed there visually. Therefore, the user can see the desired video display.

  As described above, according to this embodiment, the display information desired to be displayed on the display is all generated by the mobile phone, and the display information is displayed in real time at the position corresponding to the instruction in the image being displayed on the display screen. Can be displayed.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration of the present invention is not limited to these embodiments, and the design does not depart from the gist of the present invention. These changes are included in the present invention.
For example, in addition to the configuration of each of the embodiments described above, display information is stored in correspondence with an instruction, and when an arbitrary video position in the display screen is instructed, the display information is read in response to the instruction to display the instruction It may be configured to display at the inner position.
Moreover, you may comprise so that the display timing of the display information on a display, a display position, and the magnitude | size of a display may be changed.

Alternatively, time data may be derived using the number of video frames handled in the processing by the moving image processing function 241, the moving speed of the athlete may be measured, and the speed display may be performed on the display.
In the third embodiment, the storage information stored in the memory card may be not the design data but the feature point information of the subject facility stored so as to be readable based on the designation information of the subject facility.

  The video display system and method disclosed herein, the mobile terminal system and method thereof, the mobile terminal device and method thereof, the server and the control program thereof, and the recording medium are used to display intended display information in the video being displayed in real time. It can be used in various display systems or display devices, information processing devices, such as sightseeing spots, public institutions, and the like that are to be displayed in a superimposed manner.

10, 10A mobile terminal system (video display system)
10B mobile terminal device (video display system)
20, 20A, 20B Terminal device 21 Radio control unit 22 Key (instruction means)
23 Memory 23 (part of display control means)
24 Central processing unit (part of display control means)
25 Camera 26 Display (display means)
30 Information communication network (communication line)
40 server 241 video processing function (part of display control means)
242 Object recognition function (part of display control means)
243 Information addition function (part of display control means)
244 Information processing function (part of display control means)
401 Network connection unit 402 Information processing function (part of display control means)
403 Movie processing function (part of display control means)
404 Object recognition function (part of display control means)
405 Information addition function (remaining part of display control means)

Claims (43)

Display means for displaying a video of the subject;
Instruction means for instructing a feature point in the video displayed on the display means;
A video display system comprising: display control means for displaying display information corresponding to the instruction in the video displayed on the display means based on an instruction from the instruction means. The display control means includes
First feature information output means for outputting feature point information corresponding to the instruction in the video;
Information requesting means for requesting feature point information of the subject;
Second feature information output means for outputting feature point information of the subject based on the request information output from the information request means;
The display corresponding to the instruction based on the feature point information corresponding to the instruction output from the first feature information output means and the feature point information of the subject output from the second feature information output means. 2. The video display system according to claim 1, wherein the information is displayed in a video displayed on the display means.   The subject is a subject facility, and the second feature information output means includes design data output means for storing design data of the subject facility, the design data output from the design data output means, and the The video display system according to claim 2, further comprising a calculation unit that calculates feature point information of the object facility based on the request information output from the information request unit.   The subject is a filming facility, and the second feature information output unit stores the request information output from the information requesting unit and the feature point information of the filming facility in association with each other. 3. The video display system according to claim 2, further comprising: a reading unit that reads out feature point information of the subject based on the request information output from the information request unit.   4. The video display system according to claim 3, wherein the design data output means is an external recording medium.   5. The video display system according to claim 4, wherein the information storage means is an external recording medium. Display means of a terminal device for displaying a video of a photographed facility;
Instruction means of the terminal device for instructing feature points in the video displayed on the display means;
A portable terminal system comprising: display control means for displaying display information corresponding to the instruction in an image displayed on the display means based on an instruction from the instruction means. The display control means includes
While the first feature information output means for outputting the feature point information corresponding to the instruction in the video and the information request means for requesting the feature point information of the subject facility are provided in the portable terminal device,
A second feature information output means for outputting feature point information of the subject facility based on the request information output from the information request means is provided in a server connected to the portable terminal device via a communication line. ,
Corresponding to the instruction based on the feature point information corresponding to the instruction output from the first feature information output means and the feature point information of the object facility output from the second feature information output means 8. The portable device according to claim 7, further comprising a control unit provided in the terminal device for generating display information to be displayed and displaying the generated display information in a video displayed on the display unit. Terminal system. The display control means includes
While the terminal device is provided with a facility for specifying the facility for outputting the designation for the facility to be photographed,
First feature information output means for outputting feature point information corresponding to the instruction in the video;
Second feature information output means for outputting feature point information of the subject facility based on the designation output from the subject facility designation means;
The instruction is based on the feature point information corresponding to the instruction output from the first feature information output means and the feature point information of the object facility output from the second feature information output means. A processing unit that generates corresponding display information and outputs video information in which the display information is superimposed on video information from the mobile terminal device is provided in a server connected to the terminal device via a communication line. 8. The portable terminal system according to claim 7, wherein   The second feature information output means includes design data output means for storing design data of the subject facility, the design data output from the design data output means, and the request information output from the information request means. The mobile terminal system according to claim 8, further comprising: an arithmetic unit that calculates the feature point information of the subject facility based on   The second feature information output means is an information storage means for storing the request information output from the information request means and the feature point information of the subject facility in association with each other, and is output from the information request means. The portable terminal system according to claim 8 or 9, further comprising a reading unit that reads out the feature point information of the object facility based on the request information.   The display control means includes an object recognition means for recognizing a feature point corresponding to the instruction, a moving image processing means for outputting the feature point information of the recognized feature point, and an information processing for outputting the feature point information of the display facility. Generating the display information based on the feature point information output from the video processing means and the feature point information output from the facility information processing means, and adding the generated display information to the video information 10. The portable terminal system according to claim 8, further comprising an information adding unit configured to perform the above-described information addition. Display means for displaying a video of the subject facility;
Instruction means for instructing a feature point in the video displayed on the display means;
First feature information output means for outputting feature point information corresponding to the instruction in the video;
Information request means for requesting feature point information of the object facility;
The feature point information corresponding to the instruction output from the first feature information output means and the feature point information of the subject facility returned from the server based on the request information output from the information request means. A portable terminal device comprising: display control means for displaying display information corresponding to the instruction on the basis of the video displayed on the display means. Display means for displaying a video of the subject facility;
Instruction means for instructing a feature point in the video displayed on the display means;
A facility specification means for specifying the object facility;
Display information corresponding to the instruction returned from the server based on the instruction information output from the instruction means and the designation information output from the subject facility designation means is displayed in the video displayed on the display means. And a display control means for displaying the portable terminal device.   The display control means includes first feature information output means for outputting feature point information corresponding to the instruction in the video, information request means for requesting feature point information of the object facility, and the first feature information output means. Based on the feature point information corresponding to the instruction output from the feature information output means and the feature point information of the object facility returned from the server in response to the request information output from the information request means 14. The portable terminal device according to claim 13, further comprising control means for displaying display information corresponding to the instruction in the video displayed on the display means.   The display control means includes an object recognition means for recognizing a feature point corresponding to the instruction, a moving image processing means for outputting the feature point information of the recognized feature point, and information for outputting the feature point information of the pre-photographed facility The display information is generated based on the processing means, the feature point information output from the moving image processing means, and the feature point information output from the information processing means, and the generated display information is displayed on the display means. 15. The portable terminal device according to claim 13, further comprising information adding means for adding to video information used for display. Display means for displaying a video of the subject facility;
Instruction means for instructing a feature point in the video displayed on the display means;
First feature information output means for outputting feature point information corresponding to the instruction in the video;
Facility designating means for designating the subject facility;
Design data output means for storing design data of the object facility;
The design data is read from the design data output means based on the designation information output from the facility designation means, and the feature point information of the object facility is output based on the read design data. Feature information output means;
Corresponding to the instruction based on the feature point information corresponding to the instruction output from the first feature information output means and the feature point information of the object facility output from the second feature information output means A portable terminal device comprising: display control means for displaying display information in an image displayed on the display means. Display means for displaying a video of the subject facility;
Instruction means for instructing a feature point in the video displayed on the display means;
First feature information output means for outputting feature point information corresponding to the instruction in the video;
Information request means for requesting feature point information of the object facility;
Information storage means for storing the request information output from the information request means and the feature point information of the subject facility in association with each other;
Second feature information output means for reading out the feature point information of the subject facility from the information storage means based on the request information output from the information request means;
Corresponding to the instruction based on the feature point information corresponding to the instruction output from the first feature information output means and the feature point information of the object facility output from the second feature information output means A portable terminal device comprising: display control means for displaying display information in an image displayed on the display means.   18. The portable terminal device according to claim 17, wherein the design data output means is an external recording medium.   19. The portable terminal device according to claim 18, wherein the information storage means is an external recording medium.   A recording medium that records the designated information of the subject facility and the design data in association with each other.   A recording medium that records the feature point information of the object facility and the read request information of the feature point information in association with each other. Storage means for recording the designated information of the subject facility and the design data in association with each other;
A server comprising: information processing means for calculating feature point information of the subject facility based on the designation information and design data read from the storage means. Storage means for recording the feature point information of the subject facility and the readout request information of the feature point information in correspondence with each other;
A server comprising: reading means for reading the storage means based on the read request information. First feature information output means for outputting feature point information corresponding to an instruction in the video;
Second feature information output means for outputting the feature point information of the subject facility based on the designation of the subject facility;
The instruction is based on the feature point information corresponding to the instruction output from the first feature information output means and the feature point information of the object facility output from the second feature information output means. And a processing unit that generates corresponding display information and outputs video information in which the display information is superimposed on video information from the terminal device. Take a picture of the subject with photography means,
Display the shot video on the display means,
Instruct the feature point in the image displayed on the display means with the instruction means,
A video display method characterized in that display information corresponding to an instruction is displayed in an image displayed on the display unit based on an instruction from the instruction unit. The display of the display information in the displayed video is
Output feature point information corresponding to the instruction,
Request feature point information of the subject,
Outputting the feature point information of the subject based on the required request information;
27. Display information corresponding to the instruction is displayed in the displayed video based on the feature point information corresponding to the instruction and the feature point information of the subject. Video display method.   The video display method according to claim 27, wherein the subject is a subject facility, and the feature point information of the subject is generated based on design data of the subject facility and the request information. .   The subject is a subject facility, and the feature point information of the subject is stored in association with the request information and feature point information of the subject facility, and is stored based on the request information. 28. The video display method according to claim 27, wherein the feature point information of the subject facility is read out. The subject is photographed by the photographing means of the terminal device,
Display the captured video on the display means of the terminal device,
Instructing the feature point in the video displayed on the display means with the instruction means of the terminal device,
An image display method for a portable terminal system, wherein display information corresponding to the instruction is displayed in an image displayed on the display means based on an instruction from the instruction means. The display of the display information in the displayed video is
Output feature point information corresponding to the instruction,
Request the feature point information of the subject from the terminal device to the server,
The feature point information of the subject is returned to the terminal device based on the request information requested by the server,
The display information corresponding to the instruction is displayed in the displayed image based on the feature point information corresponding to the instruction and the feature point information of the subject. Video display method for portable terminal system. The display of the display information in the displayed video is
Sending instruction information corresponding to the instruction and designation information for designating the subject from the terminal device to the server;
Generating display information corresponding to the instruction based on the instruction information and the designation information received by the server, and transmitting the display information to the terminal device to display the display information in the displayed video. 32. The video display method for a mobile terminal system according to claim 30, wherein   32. The portable terminal system according to claim 31, wherein the subject is a subject facility, and the feature point information of the subject is generated based on design data of the subject facility and the request information. Video display method.   The subject is a subject facility, and the feature point information of the subject is stored in association with the request information and the feature point information of the subject facility, and is stored based on the request information. 32. The video display method for a mobile terminal system according to claim 31, wherein the feature point information of the subject facility is read out. Record the design data of the subject facility and the designation for the subject facility in the storage means,
Display the video of the subject facility on the display means,
Instructing the feature point in the video displayed on the display means with the instruction means,
Outputting feature point information corresponding to the instructions in the video,
Specify the subject facility,
Based on the designation and design data read from the storage means, the feature point information of the subject facility is calculated,
Display information corresponding to the instruction is displayed in the displayed video based on the feature point information corresponding to the output instruction and the calculated feature point information of the subject facility. Display method for portable terminal device. Record the feature point information of the subject facility and the designation of the subject facility in the storage means,
Display the video of the subject facility on the display means,
Instructing the feature point in the video displayed on the display means with the instruction means,
While outputting feature point information corresponding to the instructions in the video,
Read the feature point information of the subject facility corresponding to the designation by designating the subject facility from the storage means,
Display information corresponding to the instruction is displayed in the displayed video based on the feature point information corresponding to the outputted instruction and the feature point information of the imaged facility read out. An image display method for a portable terminal device.   A control program for causing a computer to control the video display system according to any one of claims 1 to 6.   A control program for causing a computer to control the portable terminal system according to any one of claims 7 to 12.   A control program for causing a computer to control the mobile terminal device according to any one of claims 13 to 20.   A control program causing a computer to control the server according to any one of claims 23 to 25.   30. A control program for causing a computer to execute the video display method according to any one of claims 26 to 29.   35. A control program for causing a computer to execute the video display method of the mobile terminal system according to any one of claims 30 to 34.   37. A control program for causing a computer to execute the video display method for a portable terminal device according to claim 35 or 36.

Images