JP2008244660A - Image analysis system automatically acquiring related information from captured image, mobile terminal, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To specify a photographed image as a key to perform image analysis, and to distribute various information on the key image. <P>SOLUTION: If a user photographs an image using a mobile terminal 10, location information for the photographed object is computed, based on the location information, distance information, and direction information acquired by a GPS receiving portion 101, a photographing portion 102, and a direct information acquiring portion 108. Then, the computed location information is associated with the image data and stored at an image information memory 104a. When the mobile terminal 10 is in an idle status and has power fully, a control portion 100 of the mobile terminal 10 automatically transmits the image data and the location information for the photographed object of the image data stored at the image information memory 104a to an image analysis server 30. On the other hand, the image analysis server 30 performs image analysis based on the received image data and the received location information, and provides various information related to the image data. The mobile terminal 10 displays the provided information on a display 107. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image analysis system that automatically acquires related information from a captured image, a portable terminal, and the like.

For example, Reference 1 discloses a technique for analyzing image data captured by a mobile terminal having a capturing function and providing information of a type specified by a user.
On the other hand, the Internet usage rate of users has recently been increasing due to the introduction of full browser compatible devices and the start of information search / distribution services by telecommunications carriers. This is due to the problem that there are still few opportunities for “unintended users” to enjoy browsing the Internet.
JP 2003-283664 A

  The present invention has been made in view of the above circumstances, and performs image analysis by designating, as a key, an image photographed by a user “somehow” with a mobile terminal, that is, an image that has not been photographed for the purpose of retrieval. It is an object of the present invention to provide an image analysis system, a portable terminal, and the like that automatically distribute various information related to, and a site that posts the information.

In order to achieve the above object, an image analysis system according to the first aspect of the present invention includes:
An image analysis system comprising an image analysis server and a mobile terminal connected to the image analysis server via a network,
The image analysis server
An image information receiving unit for receiving image data and position information of a photographing target of the image data from the portable terminal;
Corresponding position information, feature information representing a feature in the vicinity of the position specified by the position information, and related information including specification information of a site on which information related to the position information and the feature information is posted An image-related information storage unit for storing;
When receiving the image data and the position information of the photographing target of the image data from the portable terminal, the feature information near the position specified by the received position information is acquired from the image related information storage unit, and the image A related information extraction unit for extracting related information stored in association with feature information most suitable for the data;
The related information extracted by the related information extracting unit, the related information transmitting unit for transmitting to the portable terminal,
With
The portable terminal is
An image information storage unit that stores the imaged image data and the position information of the imaging target of the image data in association with each other;
An image information transmission unit that transmits image data stored in the image information storage unit and position information of a photographing target of the image data to the image analysis server;
A related information receiving unit that receives related information related to the image data and the position information corresponding to the image data from the image analysis server;
A related information display unit that displays the related information received from the image analysis server in a selectable manner;
Is provided.

A portable terminal according to the second aspect of the present invention is
Further, when receiving the image data and the position information of the photographing target of the image data, the image data and the position information of the photographing target are analyzed, and based on the analysis result, the image data and the position information of the photographing target are analyzed. The mobile terminal connected via the network to the image analysis server that extracts the related information including the specific information of the site that publishes the information related to and returns the screen information for selectively displaying the related information. And
An image information storage unit that stores the imaged image data and the position information of the imaging target of the image data in association with each other;
An image information transmission unit that transmits image data stored in the image information storage unit and position information of a photographing target of the image data to the image analysis server;
A related information receiving unit that receives related information related to the image data and the position information corresponding to the image data from the image analysis server;
A related information display unit that displays the related information received from the image analysis server in a selectable manner;
Is provided.

  Further, in the portable terminal, the position information of the shooting target of the image transmitted to the image analysis server may be calculated based on shooting position information, distance information to the shooting target, and shooting direction information.

The portable terminal is
A usage status determination unit that determines whether the user is currently using the mobile terminal; and
A transmission capability determination unit that determines whether or not the image data and position information of a photographing target of the image data have predetermined power that can be transmitted to the image analysis server;
A transmission state determination unit that determines whether or not the image data and position information associated with the image data have been transmitted to the image analysis server;
In response to a user operation, a transmission permission / inhibition input unit for inputting transmission permission / inhibition information for designating whether or not the captured image may be transmitted to the image analysis server;
An image information storage control unit that stores transmission availability information input from the transmission availability input unit in association with each image in the image information storage unit;
Further comprising
The image information transmitting unit is determined by the usage status determining unit that the mobile terminal is not in use, and the mobile terminal has sufficient power to transmit the image data to the image analysis server. When it is determined by the transmission capability determination unit, it is determined by the transmission state determination unit that the image data stored in the image information storage unit is not transmitted to the image analysis server, and the corresponding transmission availability information Transmitting image data designated as transmittable to the image analysis server and position information corresponding to the image data;
You may do it.

Moreover, the said form terminal is
The mobile terminal further includes a state detection unit that detects a call state, an image output state, an audio output state, and an unoperated state,
When the state detection unit detects that the mobile terminal is not in a call state, no image is output, no sound is output, and an unoperated state continues for a predetermined time, The usage status changing unit determines that the mobile terminal is unused.
You may do it.

Moreover, in the said form terminal,
The image information storage unit further stores a transmission state flag that specifies whether the image data has been transmitted to the image analysis server in association with the image data, and the transmission state determination unit does not store the transmission state flag. When it is designated as transmission, the image transmission unit determines that the image data is transmission, and transmits the image data to the image analysis server and then has been transmitted to the transmission status flag. specify,
You may do it.

A method for controlling a mobile terminal according to the third aspect of the present invention includes:
Upon receiving the image data and the position information of the image capturing target of the image data, the image data and the position information of the image capturing target are analyzed, and based on the analysis result, the image data and the position information of the image capturing target are related. This is a method of controlling mobile terminals connected via a network to an image analysis server that extracts related information including specific information of a site on which information is posted and returns screen information for selectively displaying the related information. And
An image information storage unit that stores the imaged image data and the position information of the imaging target of the image data in association with each other;
An image information transmitting unit that transmits image data stored in the image information storage unit and position information of a photographing target of the image data to the image analysis server;
A related information receiving unit receiving related information related to the image data and the position information corresponding to the image data from the image analysis server;
A related information display step in which the information display unit displays the related information received from the image analysis server in a selectable manner;
Is provided.

Further, a program for controlling a mobile terminal according to the fourth aspect of the present invention is:
Upon receiving the image data and the position information of the image capturing target of the image data, the image data and the position information of the image capturing target are analyzed, and based on the analysis result, the image data and the position information of the image capturing target are related. The computer connected via the network to the image analysis server that extracts the related information including the specific information of the site where the information is posted and returns the screen information for selectively displaying the related information,
An image information storage unit that stores the captured image data and the position information of the imaging target of the image data in association with each other;
An image information transmission unit that transmits image data stored in the image information storage unit and position information of a photographing target of the image data to the image analysis server;
A related information receiving unit that receives related information related to the image data and the position information corresponding to the image data from the image analysis server;
A related information display unit for selectively displaying the related information received from the image analysis server;
As a function.

  According to the present invention, it is possible to perform image analysis by designating an image that has not been shot for the purpose of search as a key, and automatically distribute various information related to the key image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram illustrating a configuration example of an image processing system according to the present embodiment. As shown in FIG. 1, in this image analysis system, a plurality of portable terminals 10 and an image analysis server 30 are connected via a communication network 20. In FIG. 1, three portable terminals are shown for convenience, but the number of connected terminals is not particularly limited to this.
Each component of this system will be described below.

  A mobile terminal 10 according to the present embodiment is a mobile communication terminal device such as a mobile phone or a PHS (Personal Handyphone System), for example, and as shown in FIG. GPS information receiving unit 101, photographing unit 102, operation unit 103, storage unit 104, communication unit 105, terminal state detection unit 106, display unit 107, orientation information acquisition unit 108, audio output Unit 109.

  The control unit 100 includes a CPU (Central Processing Unit), a ROM (Read Only Memory) (not shown), a RAM (Random Access Memory) (not shown), and the like, and controls the mobile terminal 10 as a whole. . Specifically, the CPU performs control / arithmetic processing by executing a program stored in a ROM or a storage unit 104 described later. During this control / arithmetic processing, the control unit 100 temporarily stores various data in the RAM and uses it as a work area.

  The GPS information receiving unit 101 is composed of a receiving circuit or the like, receives radio waves from a GPS satellite (not shown), and measures the distance from the received radio waves, thereby determining the latitude and longitude where the mobile terminal is located, etc. Calculate location information.

  The photographing unit 102 includes a lens, an image sensor such as a CCD or a CMOS, and a drive circuit. The photographing unit 102 photoelectrically converts an optical image of a subject formed by the lens to generate a video signal. The photographing unit 102 also acquires the distance from the lens to the photographing target from the focal length of the lens.

  The operation unit 103 is a keypad that receives an operation input from a user, a character information input, and the like, and designates, for example, transmission permission / prohibition information described later at the time of image shooting.

  The storage unit 104 includes a nonvolatile memory such as an EEPROM, and various predetermined values including a program for controlling the portable terminal 10 and a transmission destination address (IP address, URL, etc.) of the image analysis server 30 described later. Remember. In addition, the storage unit 104 functions as an image information storage unit 104a that stores captured images and information related to the images.

  As shown in FIG. 4A, the image information storage unit 104a is information including an image file, position information, a transmission enable / disable flag, a transmission state flag, and the like. Among these, the image file is a file of image data photographed by the photographing unit 102. The position information is position information indicating the position of the object to be imaged of the image. The position information is indicated by, for example, latitude and longitude. The transmission enable / disable flag is information specified by the user, and specifies whether or not the image may be transmitted to the image analysis server 30 described later. For example, when the user designates that the image may be sent to the image analysis server 30, “send is possible” is stored in the send permission flag, and when it is designated that the send is not designated, “send is not possible”. Is saved. The transmission status flag is a flag indicating whether or not the image has been transmitted to the image analysis server 30 described later. For example, when the image is not transmitted to the image analysis server 30, “not transmitted” is stored in the transmission state flag, and when it is transmitted, “transmitted” is stored.

  The communication unit 105 includes a transmission / reception circuit and the like, and connects to the Internet via the base station by controlling radio wave transmission / reception by an antenna (not shown) for wireless communication with the base station (not shown). Perform data communication.

  The terminal state detection unit 106 includes various sensors and the like, and includes a remaining battery level (not shown), a charge state, a call state, an output state to the display unit 107 described later, and an output to the audio output unit 109 described later. The state, the non-operation period of the operation unit 103, and the like are detected.

  The display unit 107 includes a display device such as an LCD (Liquid Crystal Display), and displays image data, text data, and the like.

  The orientation information acquisition unit 108 includes a gyro and the like, and acquires orientation information that the mobile terminal is facing.

  The audio output unit 109 includes a D / A converter, a speaker, and the like, and outputs audio data.

  Next, the image analysis server 30 will be described. The image analysis server 30 is a server that analyzes the image automatically transmitted from the mobile terminal 10 and provides related information. As shown in FIG. 3, the image analysis server 30 includes a control unit 300, a storage unit 301, a communication unit 302, and the like.

  The control unit 300 includes a CPU (Central Processing Unit), a ROM (Read Only Memory) (not shown), a RAM (Random Access Memory) (not shown), and the like, and controls the entire image analysis server 30. Do. Specifically, the CPU performs processing such as image analysis by executing a program stored in a ROM or a storage unit 301 described later. During this control / arithmetic processing, the control unit 300 temporarily stores various data in the RAM and uses it as a work area.

  The storage unit 301 includes a large-capacity storage device such as a hard disk, and stores a program for the control unit 300 to analyze an image. The storage unit 301 also functions as an image-related information storage unit 301a that stores the captured image and the positional information of the target object of the captured image in association with each other.

For example, as illustrated in FIG. 4B, the image related information storage unit 301 a stores feature information, related site information, and summary information in association with position information. The position information is latitude and longitude information, and the feature information is, for example, a characteristic landmark existing around the position specified by the position information (for example, a store registered in GPS, This is information that identifies the facility. Related site information and summary information are information related to feature information and position information (related site information and summary information are referred to as related information). That is, the related site information stores URLs or the like of sites or blogs that post information related to position information and feature information. In the summary information, a simple explanation about the related site information is registered. For example, when the location information indicates the vicinity of Shiba Park 4-chome, Minato-ku, Tokyo (this is assumed to be latitude xx degrees and longitude yy degrees), and the feature information is a red tower image, URLs such as sites related to Tokyo Tower and blogs are stored. Then, a brief description of these sites is registered in the summary information as “Tokyo Tower site”, for example.
In this embodiment, related site information and summary information are stored as related information, but other information related to feature information and position information may be used.

  The communication unit 302 includes a communication interface and the like, and communicates with the mobile terminal 10. The communication unit 302 may include a modem device, an infrared communication device, and the like.

  Next, operations of the mobile terminal 10 and the image analysis server 30 having the above configuration will be described.

(Image accumulation)
First, a flow of capturing and storing images in the mobile terminal 10 will be described with reference to FIG. When the user operates the operation unit 103 to instruct the mobile terminal 10 to capture an image (step S101), the capturing unit 102 captures an image. Simultaneously with shooting, the control unit 100 calculates position information of the shooting target (step S102). That is, the shooting position is acquired from the GPS information receiving unit 101, the distance information from the shooting unit 102 to the shooting target is acquired, and the orientation of the mobile terminal 10 at the time of shooting is acquired from the orientation information acquiring unit 108. Then, the control unit 100 calculates the position information of the shooting target object from the shooting position information, shooting distance information, and orientation information. Next, the control unit 100 displays on the display unit 107 a selection menu that allows the user to specify whether or not the captured image may be transmitted to the image analysis server 30 (step S103). When the user operates the operation unit 103 and specifies transmission permission / inhibition information (whether or not transmission is allowed) (step S104), the control unit 100 creates a file from the image data captured in step S101, and captures the image. The information is stored in the image information storage unit 104a in association with the position information of the object and the transmission availability information. At this time, “not transmitted” is stored in the transmission status flag (step S105).

  The object to be photographed is not particularly intended for retrieval, and may be one that the user sees appropriately during normal use. For example, it may be a building seen in the city, an object, or any object that the user wants to photograph.

(Automatic distribution of related information)
As described above, the control unit 100 automatically transmits the image accumulated by performing steps S <b> 101 to S <b> 105 to the image analysis server 30 based on the state of the mobile terminal 10. Then, the image analysis server 30 distributes related information to the mobile terminal 10 that is the transmission source based on the transmitted image and position information. Hereinafter, the flow of automatic distribution of related information will be described with reference to FIG.

  The mobile terminal 10 executes the following automatic distribution process at a predetermined time interval or at a predetermined time. First, the control unit 100 determines whether or not the mobile terminal 10 is in an unused state (step S200). Here, based on the detection result from the terminal state detection unit 106, the control unit 100 is not in a call state, and there is no image or video in the output state on the display unit 107 (that is, not watching TV or the like). ) When it is determined that the audio output unit 109 does not output audio (that is, music is not being listened to), the operation unit 103 has not been operated for a predetermined time, or the like, the conditions are satisfied. It is determined that the mobile terminal 10 is in an unused state.

  Next, the control unit 100 determines whether or not the remaining battery level detected by the terminal state detection unit 106 is greater than or equal to a predetermined amount (step S201). When it is determined that the amount is equal to or less than the predetermined amount (step S201; No), the control unit 100 determines whether charging is being performed from the result detected by the terminal state detection unit 106 (step S202). If the battery is being charged (step S202; Yes) or the remaining battery level is equal to or greater than the predetermined amount (step S201; Yes), the control unit 100 advances the processing to step 203. Otherwise, that is, when the remaining battery level is equal to or less than the predetermined amount (step 201; No) and the battery is not being charged (step S202; No), the process returns to step S200.

  If it is determined in the above processing (steps S200 to S202) that the mobile terminal 10 is in an unused state and the battery is sufficiently supplied, then the strength of the received radio wave from the base station (not shown) is predetermined. It is determined whether or not the value is greater than or equal to the value (that is, whether or not the communication state is good) (step S203). If the strength of the radio wave is equal to or greater than the predetermined value (step S203; Yes), the control unit 100 indicates that the transmittable information is “possible” in the image data stored in the image information storage unit 104a, and no transmission is performed. For those for which the flag is “not transmitted”, the image data and the position information of the object to be imaged associated with the image data are transmitted to the image analysis server 30 (step S204). The address of the image analysis server 30 that is a transmission destination can be acquired by referring to the transmission destination address stored in the storage unit 104. After the transmission, the control unit 100 updates the transmission flag to “transmitted” (step S205). If the received radio wave intensity is equal to or less than the predetermined value (step S203; No), the process returns to step S200.

  When the control unit 300 of the image analysis server 30 receives the image data and the position information of the photographing object of the image (step S206), the control unit 300 performs image analysis using an existing method based on the information, Information is acquired (step S207). For example, the control unit 300 extracts a contour line including the center point of the image, and extracts features such as color information included in the contour line. On the other hand, the control unit 300 acquires feature information from the image-related information storage unit 301a based on the position information of the photographing object. Then, pattern matching is performed between the information extracted from the image and the feature information acquired based on the position information, and related information associated with the most suitable feature information, that is, site information related to the feature information. And get summary information.

  For example, assuming that image data obtained by photographing a red tower near longitude xx and latitude yy is received, the control unit 300 acquires several pieces of feature information near longitude xx and latitude yy from the image-related information storage unit 301a and receives them. Perform pattern matching with the image data. Among them, for example, if it is most suitable for the image 1 taken of Tokyo Tower, http: /tokyotower.xx.jp is acquired as related site information (see FIG. 4B). Then, “Tokyo Tower site” is acquired as summary information. The image analysis method is not limited to the above, and other methods may be used.

  Next, the control unit 300 lists the related site information and the corresponding summary information, and analyzes the analyzed image in the form of a web page or a mail having a selectable link that can be accessed by simply clicking the user. The data is transmitted as related information to the mobile terminal 10 that is the data transmission source via the communication unit 302 (step S208).

  On the mobile terminal 10 side, the web page or the mail containing the related information received via the communication unit 105 (step S209) is displayed on the display unit 107 (step S210). The related information automatic distribution processing is thus completed. The user can browse the site and information related to the captured image “somehow” by simply clicking on the displayed related information.

  In this way, the image analysis system according to the present embodiment automatically distributes related information that is relatively interesting to the user to the mobile terminal 10, thereby providing the Internet to users who do not have a clear purpose. Provide an opportunity to browse.

  In addition, since the mobile terminal 10 transmits an image when the mobile terminal 10 is in an unused state with a low load, there is also an advantage that it is not necessary to give a user a discomfort such as a heavy processing during use.

  Although the embodiments of the present invention have been described above, the present invention can be modified and applied in various forms and is not limited to the above embodiments.

  For example, in the above embodiment, the transmission state flag is stored in association with the image, but an untransmitted folder or a transmitted folder may be prepared in the storage unit 104. Then, only the image designated by the user to be transmitted to the image analysis server 30 is stored in the unsent folder, and when the image in the unsent folder is transmitted to the image analysis server 30, it is stored in the transmitted folder. You may make it move.

  In addition, the automatic transmission function is not always enabled, but the user may operate the operation unit 103 to make a specification for invalidating the automatic transmission function.

  Further, when the remaining power is equal to or greater than the predetermined amount, the image data is transmitted to the image analysis server 30. However, the transmission status flag is “not transmitted” and the transmission permission flag is “transmittable”. The power required for transmission may be calculated from the data amount of the image data, and may be transmitted when the portable terminal has a remaining power that is equal to or greater than the calculated power.

  In the above embodiment, the still image taken by the user is transmitted to the image analysis server 30, but a predetermined image (for example, the first and last frame images) is selected from the moving image. Alternatively, it may be transmitted to the image analysis server 30.

  Further, as a method for simply determining whether or not the mobile terminal 10 is in an unused state, there is a method for checking the open / closed state of the mobile terminal 10 and further checking the time of the non-operating state. For example, when the mobile terminal 10 is a foldable type or includes a flip that covers the operation unit, the terminal state detection unit 106 sets the state in which the operation unit is exposed to the user to “open” and is hidden Is detected as “closed”. In the case of “open”, it is further detected whether or not there has been no operation for a predetermined time since the previous operation.

  At this time, if the non-operation state continues for a predetermined time or more, the control unit 100 may determine that the mobile terminal 10 is in an unused state. Even when the open / close state is “closed”, the control unit 100 may determine that the mobile terminal 10 is in an unused state.

  In the above embodiment, the captured image stored in the mobile terminal 10 is transmitted to the image analysis server 30. However, in addition to the mobile terminal 10, the digital information that can calculate the position information of the shooting target portion is used. Images captured by a camera or the like may be stored in a personal computer, and the personal computer may automatically transmit these captured images to the image analysis server 30 to obtain related information.

  Further, in the above embodiment, the mobile terminal 10 calculates the position information of the shooting target. However, the mobile terminal 10 needs information (position information, orientation information) to calculate the position information of the shooting target. , Distance information, etc.) may be transmitted to the image analysis server 30, and the image analysis server 30 may calculate the position information of the object to be imaged.

  In the above embodiment, the related information based on both the image data and the position information of the photographing target of the image is distributed. However, the information related to the image data and the position information is distributed individually. You may make it do. In this case, store information or the like in the vicinity of the shooting location may be provided based on the position of the shooting location itself, not the position of the shooting target.

  In the above-described embodiment, the control program of the mobile terminal 10 and the image analysis server 30 has been described as being stored in advance in the storage unit or the like. However, a storage medium such as a flexible disk, a CD-ROM (Compact Disk Read-Only Memory), a DVD (Digital Versatile Disk), an MO (Magneto-Optical disk), or a USB memory is read into the mobile terminal 10 or the image analysis 30 server. Any suitable device may be provided, and the control program may be stored and distributed on these storage media, and the program read by the reader. And the apparatus which performs the above-mentioned processing operation may be comprised by installing the read program.

  Further, the control program may be stored in a disk device or the like of a predetermined server device on the communication network and downloaded to the mobile terminal 10 or the image analysis server 30. Furthermore, the above-described processing can also be achieved by starting and executing the program while transferring it via the communication network 20.

  It should be noted that each control unit of the mobile terminal 10 or the image analysis server 30 according to the above embodiment is not configured to be controlled by executing a program, but can also be realized by dedicated hardware.

It is a block diagram showing an example of composition of an image analysis system concerning an embodiment of the invention. It is a block diagram which shows the structural example of a portable terminal. It is a block diagram which shows the structural example of an image analysis server. (A) is a figure which shows the example of a data structure of an image information storage part, (B) is a figure which shows the example of a data structure of an image related information storage part. It is a flowchart of the accumulation process of an image. It is a flowchart of the automatic delivery process of related information.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Mobile terminal 100 Control part 101 GPS receiver 102 Image | photographing part 103 Operation part 104 Storage part 104a Image information storage part 105 Communication part 106 Terminal state detection part 107 Display part 20 Communication network 30 Image analysis server 300 Control part 301 Storage part 301a Image Related information storage unit 302 Communication unit

Claims (8)

An image analysis system comprising an image analysis server and a mobile terminal connected to the image analysis server via a network,
The image analysis server
An image information receiving unit for receiving image data and position information of a photographing target of the image data from the portable terminal;
Corresponding position information, feature information representing a feature in the vicinity of the position specified by the position information, and related information including specification information of a site on which information related to the position information and the feature information is posted An image-related information storage unit for storing;
When receiving the image data and the position information of the photographing target of the image data from the portable terminal, the feature information near the position specified by the received position information is acquired from the image related information storage unit, and the image A related information extraction unit for extracting related information stored in association with feature information most suitable for the data;
The related information extracted by the related information extracting unit, the related information transmitting unit for transmitting to the portable terminal,
With
The portable terminal is
An image information storage unit that stores the captured image data and the position information of the shooting target of the image data in association with each other;
An image information transmission unit that transmits image data stored in the image information storage unit and position information of a photographing target of the image data to the image analysis server;
A related information receiving unit that receives related information related to the image data and the position information corresponding to the image data from the image analysis server;
A related information display unit that displays the related information received from the image analysis server in a selectable manner;
Comprising
An image analysis system characterized by that. When receiving the image data and the position information of the shooting target of the image data, the image data and the position information are analyzed, and information related to the image data and the position information of the shooting target based on the analysis result A mobile terminal connected via a network to an image analysis server that extracts related information including specific information of a site that publishes and returns screen information for selectively displaying the related information,
An image information storage unit that stores the captured image data and the position information of the shooting target of the image data in association with each other;
An image information transmission unit that transmits image data stored in the image information storage unit and position information of a photographing target of the image data to the image analysis server;
A related information receiving unit that receives related information related to the image data and the position information corresponding to the image data from the image analysis server;
A related information display unit that displays the related information received from the image analysis server in a selectable manner;
Comprising
A portable terminal characterized by that. The position information of the shooting target of the image to be transmitted to the image analysis server is calculated based on shooting position information, distance information to the shooting target, and shooting direction information.
The mobile terminal according to claim 2, wherein: A usage status determination unit that determines whether the user is currently using the mobile terminal; and
A transmission capability determination unit that determines whether or not the image data and position information of a photographing target of the image data have predetermined power that can be transmitted to the image analysis server;
A transmission state determination unit that determines whether or not the image data and position information associated with the image data have been transmitted to the image analysis server;
In response to a user operation, a transmission permission / inhibition input unit for inputting transmission permission / inhibition information for designating whether or not the captured image may be transmitted to the image analysis server;
An image information storage control unit that stores transmission availability information input from the transmission availability input unit in association with each captured image in the image information storage unit;
Further comprising
The image information transmitting unit is determined by the usage status determining unit that the mobile terminal is not in use, and the mobile terminal has sufficient power to transmit the image data to the image analysis server. When it is determined by the transmission capability determination unit, it is determined by the transmission state determination unit that the image data stored in the image information storage unit is not transmitted to the image analysis server, and the corresponding transmission availability information Transmitting image data designated as transmittable to the image analysis server and position information corresponding to the image data;
The mobile terminal according to claim 2, characterized in that: The mobile terminal further includes a state detection unit that detects a call state, an image output state, an audio output state, and an unoperated state,
When the state detection unit detects that the mobile terminal is not in a call state, no image is output, no sound is output, and an unoperated state continues for a predetermined time, The usage status changing unit determines that the mobile terminal is unused.
The mobile terminal according to claim 4, wherein: The image information storage unit further stores a transmission state flag that specifies whether the image data has been transmitted to the image analysis server in association with the image data, and the transmission state determination unit does not store the transmission state flag. When it is designated as transmission, the image transmission unit determines that the image data is transmission, and transmits the image data to the image analysis server and then has been transmitted to the transmission status flag. specify,
The mobile terminal according to claim 4, wherein: Upon reception of the image data and the position information of the shooting target of the image data, the image data and the position information of the shooting target are analyzed, and based on the analysis result, the image data and the position information of the shooting target are determined. A method of controlling mobile terminals connected via a network to an image analysis server that extracts screen-related information that selectively displays related information including site-specific information for posting related information. Because
An image information storage unit that stores the imaged image data and the position information of the imaging target of the image data in association with each other;
An image information transmitting unit that transmits image data stored in the image information storage unit and position information of a photographing target of the image data to the image analysis server;
A related information receiving unit receiving related information related to the image data and the position information corresponding to the image data from the image analysis server;
A related information display step in which the information display unit displays the related information received from the image analysis server in a selectable manner;
A method for controlling a portable terminal. Upon receiving the image data and the position information of the image capturing target of the image data, the image data and the position information of the image capturing target are analyzed, and based on the analysis result, the image data and the position information of the image capturing target are related. The computer connected via the network to the image analysis server that extracts the related information including the specific information of the site where the information is posted and returns the screen information for selectively displaying the related information,
An image information storage unit that stores the captured image data and the position information of the imaging target of the image data in association with each other;
An image information transmission unit that transmits image data stored in the image information storage unit and position information of a photographing target of the image data to the image analysis server;
A related information receiving unit that receives related information related to the image data and the position information corresponding to the image data from the image analysis server;
A related information display unit for selectively displaying the related information received from the image analysis server;
A program that functions as

Images