JP2006285546A - Information providing system, database server, portable communication terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information providing system, and a portable communication terminal capable of quickly acquiring information relevant to an object in a landscape which a user wants. <P>SOLUTION: In the information providing system, the information relevant to the object in the landscape which the user wants to know about is acquired by using the portable communication terminal having a present position detecting function. It has a landscape image generating circuit generating an azimuth landscape image on the basis of positional information sent from the portable communication terminal, and the map database server retrieving attribute information associated with the azimuth image and sending the azimuth landscape image at the present position to the user via a network. The portable communication terminal 1 is provided with a display means 4 for selecting and displaying desired information from plural pieces of attribute information relevant to particular objects associated with landscape image candidates by selecting the landscape image candidates similar to the landscape including the particular objects on the basis of the azimuth landscape image sent from the database server 2. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention uses a mobile phone terminal with an imaging function and a location information function to connect to a map database server via a network and search a landscape image stored based on location information from the user. The present invention relates to an information providing system and a mobile communication terminal that search for a specified object image, specify an object, and provide detailed information about the specified object.

  In recent years, mobile phones with camera functions and GPS (Global Positioning System) functions have become very popular, and various information providing services have been provided. Among them, a navigation system that displays and guides map information of a current location is known as a service of a mobile phone with a GPS function. With this service, the user can see the displayed map information to know the current position, and can easily go to the destination while viewing the map information.

In recent years, there has been an increasing demand for photographing and observing an object in a landscape using a camera and binoculars and to know information about the observation object in the photographing direction and the observation direction. For this purpose, it is necessary to acquire not only position information but also azimuth information. Conventionally, azimuth sensors, for example, geomagnetic sensors, are generally used. As an example of a mobile communication terminal device using these azimuth sensor functions, a mobile phone (see Patent Document 1) that can accurately guide to a destination, and shooting information such as a shooting position and a shooting direction along with shooting information via a network. The system (refer patent document 2) etc. which transmit and transmit and provide the information regarding the target object in the landscape specified by imaging information is made | formed.
Japanese Patent No. 2003-299136 Japanese Patent No. 2002-175315

  However, the portable communication terminal device and camera with an orientation detection function proposed in the above-mentioned patent document requires a new orientation sensor for the device, which causes a problem that the size of the device increases and the cost also increases. In particular, a geomagnetic orientation sensor that is generally used has a problem that it does not operate normally in a building using a steel frame such as a building street or an underground shopping street.

In addition, it is very difficult to specify an object from among things in a landscape using a camera or binoculars even if the orientation information is acquired and the position of the object is specified.
The objective of this invention aims at proposing the information provision system and portable communication terminal which can acquire the information regarding the target object in the scenery which a user wants to obtain in a short time.

  Claim 1 of the present invention is an information providing system that acquires information on an object in a landscape that a user wants to know using a mobile communication terminal having a current position detection function, and is transmitted from the mobile communication terminal. A map that searches a landscape image generation circuit that generates a azimuth landscape image and attribute information associated with the azimuth image based on the obtained location information and transmits the azimuth landscape image at the current position to the user via the network The mobile communication terminal is associated with the landscape image candidate by selecting a landscape image candidate similar to the landscape including the specific object based on the orientation landscape image transmitted from the database server. It is characterized by comprising display means for selecting and displaying desired information from among a plurality of attribute information relating to the specified specific object.

  According to a third aspect of the present invention, there is provided position detection means for detecting a current position of the mobile communication terminal, three-dimensional map data in which position information, direction information and attribute information of the object are associated, and the position detection. A landscape image generating means for generating azimuth image data around the current position detected by the means from the three-dimensional map data, and an imaging for picking up the object around the current position and converting it into object image data And image recognition means for extracting a partial image corresponding to the object in the orientation image data based on the orientation image data and the object image data, and the image recognition means extracted And a display unit configured to display the attribute information associated with the partial image and the object image data.

Based on the position information transmitted from the mobile phone terminal, search for attribute information from a landscape image generation circuit that generates an omnidirectional landscape image and an attribute database associated with the omnidirectional image, and omnidirectional at the current position It has a map database server that sends landscape images to users via the network,
The mobile phone terminal selects a landscape image candidate similar to a landscape including the specific object based on the omnidirectional landscape image transmitted from the database server, thereby specifying the specific object associated with the landscape image candidate. And a display means for selecting and displaying desired information from among a plurality of attribute information.

According to the configuration of the present invention, it is possible to specify a shooting direction even in a building town or an underground shopping area that cannot be dealt with only by a direction sensor as in the prior art.
Furthermore, since the target image is searched after acquiring the similar landscape image for specifying the target object, the search range is narrowed down, and the search time can be greatly shortened.

  The information providing system of the present invention can acquire information on an object to be known in a landscape anytime and anywhere.

The best mode for carrying out the present invention will be described below with reference to the drawings.
<Example 1>
FIG. 1 is a block diagram illustrating a configuration of a mobile phone terminal with an imaging function and a position detection function and an information providing system according to Embodiment 1 of the present invention.

The information providing system of the present embodiment is composed of a mobile phone terminal 1 and a database server 2 cities. FIG. 2 is a configuration block diagram of a mobile phone as an example of a mobile phone terminal.
The cellular phone terminal 1 includes a digital camera 3, a GPS drive circuit 17, and a GPS antenna 18 as shooting functions.

  The digital camera 3 includes a CCD image sensor 301, an imaging lens for forming an optical image of a landscape on the image sensor 301, and an image sensor driving circuit 302 that drives the image sensor 301 and outputs an image of a subject. .

The image memory 7 stores image data output from the digital camera 3.
The display circuit 8 and the display 4 can directly display the captured image captured in the image memory 7 and the image data from the image sensor 302, and can display the current captured image in real time.

  The data memory 10 is provided with a map data area 101 for storing map data and a landscape image data area 102 for storing landscape image data described later. Map data and landscape image data are transmitted from the database server 2 and displayed on the display.

The communication circuit 11 is a circuit that communicates with the database server 2 and includes a connection adapter that can be connected to a public line, the Internet, wireless communication, or the like.
The release key 5 is an operation button that outputs a signal that causes the digital camera 3 to image a subject.

  When the operator depresses the release key 5, the CPU 6 sends a control signal to the image sensor driving circuit 302 to control the imaging operation and store the captured landscape image data in the image memory 7. The CPU 6 includes a program for controlling the entire mobile phone terminal, such as the communication circuit 11 and a power source (not shown).

  On the side surface on the same side as the display 14, a map operation key 12, a cursor key 13, a scroll key 14, a selection key 15, a search key 16, and the like are provided as operation units. In the present embodiment, description of functions related to the voice telephone is omitted.

The scroll key 14 is an operation key for inputting a direction and an amount for scrolling landscape image data described later displayed on the display 4.
The GPS antenna 18 and the GPS drive circuit 17 receive radio waves from GPS satellites, measure the latitude and longitude of the current position of the mobile phone terminal 1, and output the latitude and longitude values to the CPU 6.

  On the other hand, the database server 2 includes a communication circuit 21 that communicates with the mobile phone terminal 1, a cutout circuit 22, a landscape image generation circuit 23, a three-dimensional map database 24, and an image recognition circuit 26.

  The cutout circuit 22 specifies and cuts out the data size from the three-dimensional map data in response to a request for position data such as longitude and latitude, and reduction / enlargement from the operator, and converts the data into two-dimensional map data.

  The landscape image generation circuit 23 is based on the shooting position data (latitude, longitude) transmitted from the user, and is omnidirectional landscape in all directions centered on the position data from the map stored in the three-dimensional map database. Generate an image. Omnidirectional landscape images include live-action landscape images and CG landscape images generated from 3D map data. In the case of a 3D CG image, based on the current position information transmitted from the mobile phone terminal, if there is ground height, elevation angle data, etc., the camera viewpoint is determined based on them and the object is viewed from the camera viewpoint position. Various CG images can be generated and displayed according to the direction and distance.

  Create omnidirectional landscape image data. When there is no ground height or elevation data, landscape image data is created with a ground height of 1.5 m and an elevation angle of zero. In addition, examples of the live-action landscape image include an omnidirectional landscape image continuously captured in all directions at once and a panoramic image obtained by connecting a plurality of slide images. These live-action landscape images are stored in the 3D map database, and the landscape image generation circuit searches the map database based on the current position information so that it can be displayed on the mobile phone terminal. Process to a size that can be sent.

  In the 3D map database 24, a 3D computer graphic (CG) image, a landscape image (actual image), and the like are stored. The stored 3D CG image is stored as 3D CG data composed of polygon objects on the map of buildings and public facilities. These three-dimensional map data are linked to attribute databases such as building names, public facility names, address names, place names, signboards, and signs.

  In addition, the live-action image stored in the 3D map database is linked to the attribute database in the same manner as the CG image. Various attribute information stored in the attribute database includes position information (latitude, longitude, altitude) and azimuth information (azimuth, elevation angle, depression angle, etc.). Further, corresponding to the position information of these maps, there are names of things such as houses, buildings, addresses, public facilities such as schools and hospitals, and detailed information on things such as transportation.

  As an example of the stored photographed image, there is an omnidirectional panoramic image continuously imaged with respect to a field of view of 360 degrees in all directions from the viewpoint of position data (latitude, longitude). In addition, the object that can be photographed centering on the current position data, for example, famous landscape buildings, towers and tall buildings, slide landscape images created using public facilities such as parks and schools, or slide images continuously. It may be a panoramic image joined together. Each of these live-action landscape images is associated with attribute information such as position information (background, latitude, ground height), azimuth information (azimuth, elevation angle), and detailed information on things.

The thing attribute information database 25 is associated with the address and name of the thing, and explanation information about the thing (advertisement, history, origin, etc. for a company).
The image recognition circuit 26 performs a clipping process and a matching process for the specific object specified in the captured image transmitted from the user, and based on the object information in the attribute database stored in the database, the specific object Recognition process.
Next, processing operations when using the information providing service will be described based on the flowcharts of FIGS.

  Registration is required to use the map database server service. Therefore, in step S1, the user (user) transmits the user ID for service use registration, the mobile phone number, and the current imaging position data (longitude, latitude) acquired from the GPS satellite to the database server using the mobile phone.

  On the other hand, the database server receives the user ID, the mobile phone, and the current position data from the user in step S2. In step S3, the database server drives the clipping circuit based on the received position data to obtain the current data from the 3D map database. Cut out map data around the location. In step S4, the cut-out 3D map data is converted into 2D map data and transmitted to the mobile phone.

  In step S5, the two-dimensional map data transmitted from the server is received and stored in the map data area 101 of the data memory 10 in step S6. In step 7, the two-dimensional map displayed as a marker in the current position data is displayed on the display 4 of the mobile phone.

Returning to the flowchart of FIG. 3, in step S8, the operator operates the cursor key 13 to correct the current position data.
Since it is necessary to clearly know the location where the operator is present, this correction operation is preferably performed in advance before imaging.

  FIG. 5 is a map displayed on the display of the mobile phone. A marker indicating the current position is displayed. In the figure, a marker indicating position data acquired from GPS and a marker after correction are shown. Needless to say, this correction work is performed in advance in a place that the user can understand.

  Next, in step S9 of FIG. 3, the database server transmits the corrected position data to the database server. On the server side, in step S10, the corrected position data is received, and the landscape image generation circuit generates an omnidirectional image based on the three-dimensional map database.

Step S11 transmits the omnidirectional landscape image associated with the azimuth information at the current position of the user generated by the landscape image generation circuit to the user's mobile phone based on the user ID.
In step S12, the mobile phone receives the omnidirectional landscape image and stores it in the landscape image data area of the data memory in step S13.

  Next, in step S14, the user searches the omnidirectional landscape image transmitted from the database for a landscape image similar to the landscape viewed from the current position of the object to be known (hereinafter referred to as a landscape image candidate), select. The selection operation at this time is performed using the scroll key 14 of FIG. Furthermore, in step S15, the direction corresponding to the selected landscape image candidate can be acquired. Details of this orientation acquisition will be described later with reference to FIG.

  If the related attribute information is received together with the omnidirectional landscape image, the attribute information of a plurality of things related to it can be known if the direction is selected. The attribute information of the plurality of things may be displayed on the display of the mobile phone, and the user may select information from them.

Next, an operation for specifying an object based on a captured image and acquiring detailed information will be described.
Although the direction from the current position to the object has been specified, there are many cases where there are many things in the city before, and the object cannot be specified only by the direction information. In that case, the object is imaged using a camera, and the object is recognized based on the captured image. FIG. 6 shows the direction from the current position toward the object.

Here, a method for selecting an orientation from an omnidirectional landscape image will be described in detail.
FIG. 7 is a diagram for explaining an operation for displaying the developed image of the omnidirectional landscape image and the display screen of the mobile phone. FIG. 7A is an omnidirectional landscape image at a shooting position, and FIG. 7B is a captured image with an object designation frame captured by a camera-equipped mobile phone with the azimuth determined from the omnidirectional landscape image. is there·
The omnidirectional image is associated with shooting direction information such as shooting direction, elevation angle, and ground height. The operation of the horizontal scroll key 14 in FIG. 2 can continuously display the omnidirectional landscape image stored in the image memory on the display by pressing the right direction of the scroll key. In addition, the mobile phone terminal is provided with an azimuth detection circuit that counts the amount of scrolling and measures the azimuth data associated with the omnidirectional image clockwise with respect to the north reference. The corresponding azimuth data is displayed on the display. Moreover, the object to be photographed in the landscape is viewed, and a landscape image (landscape image candidate) similar to the landscape is selected from the omnidirectional landscape image. When a landscape image candidate is selected by the selection key 15, the azimuth is determined and stored in the data memory. In FIG. 5, the position is 96 degrees clockwise from the north, almost on the east side. In addition, the camera finder is provided with a designated frame for specifying an object, and an object in the landscape is put in the designated frame to capture a landscape image.

Next, a process for recognizing an object and obtaining detailed information based on the captured image will be described. FIG. 4 shows the operation flow.
In step S16, the object specified by the specified frame on the viewfinder is imaged, and a captured image is acquired. In step S17, the captured image specified in the specified frame and the azimuth information determined in step S15 in FIG. 3 are transmitted to the database server.

  In step S16, a search target is designated using a designation frame on the camera finder, and a captured image is acquired toward the target. In step S17, the captured image is transmitted to the database server via the network. As information transmitted together with the captured image, there are information on the selected landscape image candidate in addition to the orientation information and the position information acquired in step S15. Shooting information (shooting range, shooting distance, etc.) is also sent as necessary.

On the server side, the captured image is received in step S18. At the time of this access, the user ID inquires the registered ID in step S2 of FIG.
In step S19, processing for cutting out the search target image within the designated frame is performed.

In step S20, the extracted search target image is used as a template image, the omnidirectional landscape image is matched with reference to the azimuth information, and the target is recognized and specified.
In step S21, the name and detailed information regarding the identified object are transmitted to the user.

In step S22, the name and detailed information regarding the specified object are received on the mobile phone terminal side, and the user selects and displays them on the display of the mobile phone terminal as necessary.
FIG. 8 shows an example of the name of the thing displayed on the display and its detailed information.

<Example 2>
The second embodiment of the present invention is an embodiment in which the object recognition and identification process is performed in a mobile phone. The second embodiment is the same up to the processing operation up to step S15 in FIG.

FIG. 9 is a system configuration block diagram of the second embodiment.
The mobile phone terminal includes a recognition means 19, an image extraction circuit 21 that cuts out and extracts a search target specified by a specified frame, and an image search circuit that searches the extracted target image from candidate images of landscape images.

FIG. 10 is a processing flow of the second embodiment.
The processing contents from Step 116 to Step S119 in FIG. 10 correspond to Steps S19, S20, and S22 in FIG. 7, and the processing contents are the same, so the detailed description is omitted.

  Although the mobile phone with a camera function has been described as the first and second embodiments, the present invention is not limited thereto, and any mobile phone terminal device having an imaging function such as a device having a communication function in the camera may be used.

  As described above, if it is the best mode for carrying out the present invention, even if the mobile communication terminal does not have a direction sensor, if it is registered in the information providing service, the object in the landscape is specified. This makes it possible to obtain detailed information on things very efficiently.

  As described above, the information providing system according to the present invention can easily acquire information on an object in a landscape regardless of a place without a direction sensor, using a mobile phone terminal with a position information function. I can do it. Furthermore, if it is a mobile phone terminal with an imaging function, it is possible to obtain detailed information on a specific object on the spot simply by sending a captured image in which the object is specified and orientation information to the database side. It is very convenient.

1 is a conceptual diagram of a configuration of an information providing system according to a first embodiment of the present invention. 1 is a configuration block diagram of a mobile phone with an imaging function according to a first embodiment of the present invention. FIG. 1 is a flow chart of processing of an information providing system according to the best mode for carrying out the present invention. Processing flow diagram of information providing system according to Embodiment 1 of the present invention The figure explaining the current position acquired from GPS on the two-dimensional map displayed on the display of a mobile phone, and the corrected current position The figure explaining the direction (photographing direction) which looked at the specific target object from the present position on the displayed two-dimensional map The figure explaining the omnidirectional landscape image associated with the azimuth information transmitted from the database Example of detailed information stored in the database The conceptual block diagram of the information provision system concerning Example 2 of this invention Block diagram of a mobile phone terminal according to the second embodiment of the present invention Processing flow diagram of information providing system according to Embodiment 2 of the present invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Mobile phone terminal 2 Database server 3 Camera provided in the mobile phone terminal 4 Display 9 Direction detection circuit 14 Scroll key 17 GPS drive circuit 18 GPS antenna 23 Landscape image generation circuit 24 3D map database 25 Object attribute database information 19, 26 Image recognition circuit

Claims (11)

An information providing system for acquiring information about an object in a landscape that a user wants to know using a mobile communication terminal having a current position detection function,
Based on the position information transmitted from the mobile communication terminal, a landscape image generation circuit that generates an orientation landscape image and attribute information associated with the orientation image are searched, and the orientation landscape image at the current position is retrieved via the network. A map database server to send to users
The mobile communication terminal relates to a specific object associated with the landscape image candidate by selecting a landscape image candidate similar to a landscape including the specific object based on the orientation landscape image transmitted from the database server. An information providing system comprising display means for selecting and displaying desired information from a plurality of attribute information. The directional information in the attribute database is used as an imaging direction when the mobile communication terminal further includes an imaging unit, and an imaging direction when the imaging unit images an object in a landscape. Information providing system described. An information providing system that provides information about an object in a landscape using a mobile communication terminal,
Position detecting means for detecting a current position of the mobile communication terminal;
3D map data in which the position information of the object, the direction information and its attribute information are associated,
Landscape image generation means for generating azimuth image data around the current position detected by the position detection means from the three-dimensional map data;
Imaging means for imaging the object around the current position and converting it into object image data;
Image recognition means for extracting a partial image corresponding to the object in the orientation image data based on the orientation image data and the object image data;
The attribute information associated with the partial image extracted by the image recognition means;
Display means for displaying the object image data;
An information providing system characterized by comprising: From the azimuth image data generated by the landscape image generation means, further comprising a selection means for selecting a candidate image,
4. The information providing system according to claim 3, wherein the image recognition unit extracts the partial image from the candidate images selected by the selection unit. A position correcting means for correcting the current position detected by the position detecting means;
The information providing system according to claim 3, wherein the landscape image generation unit generates azimuth image data of the current position corrected by the position correction unit. The position detection means, the imaging means, and the display means;
Transmitting means for transmitting the current position detected by the position detecting means;
The mobile communication terminal according to claim 3, further comprising: a receiving unit that receives the omnidirectional image data and the attribute information displayed by the display unit. The 3D map data, the landscape image image generation means, the image recognition means, a reception means for receiving the current position, the azimuth image data generated by the landscape image generation means, and the 3D map data 6. The database server according to claim 3, further comprising transmission means for transmitting attribute information. Transmitting means for transmitting the position detection means, the imaging means, the image recognition means, the display means, the current position detected by the position detection means, and the partial image extracted by the image recognition means; The mobile communication terminal according to claim 3, further comprising: a receiving unit that receives the omnidirectional image data and the attribute information displayed by a display unit. The 3D map data;
The landscape image generation means, the current position, the reception means for receiving the partial image, the omnidirectional image data generated by the landscape image generation means, and the attribute information of the 3D map data are transmitted. The database server according to any one of claims 3 to 5, further comprising: a transmission unit configured to transmit. Position detection means for detecting the current position;
Position detection means for detecting the current position;
Imaging means for imaging a specific object in the landscape viewed from the current position;
The three-dimensional map data in which the position information of the specific object, the azimuth information and its attribute information are associated,
A landscape image generating means for generating a landscape image around the current position based on the three-dimensional map data and the current position;
A selection means for selecting a landscape image candidate similar to the specific object from the landscape image generated by the landscape image generation means,
An information providing system comprising: an orientation detection unit that detects an imaging direction of the imaging unit based on the landscape image generated by the landscape image generation unit and the landscape image candidate selected by the selection unit. The information according to claim 10, wherein the direction detection unit converts a position occupied by the landscape image candidate with respect to the landscape image into a direction on the three-dimensional map data, and sets the direction as the imaging direction. Offer system.

Images