JP2008092326A - Portable terminal, information display method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve operability in displaying information associated with a specific range among pieces of information associated with each position on a document. <P>SOLUTION: This portable terminal has: an image analysis part 11 which analyzes a photographed image of a document to extract an identification code and a position code embedded in the document; an identification information acquisition part 12 which decodes the identification code to acquire identification information; a positional information acquisition part 13 which decodes the position code to acquire positional information; a range information generation part 14 which corrects the positional information to generate range information indicating a range of an image to be displayed; a selection information acquisition part 15 which acquires selection information indicating selection content of classification of related images relevant to the document; a communication control part 17 which transmits the identification information, the range information and the selection information to receive a display image which is an image in the specific range of the related image and a display control part 16 which performs control so as to display the display image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a mobile terminal device, an information display method, and a program.

  Capturing the embedded data encoded from the substrate having the first image information, decoding the encoded embedded data, developing the alignment information, and the first information based on the alignment information A method of displaying the aligned second information is known (for example, see Patent Document 1).

  This patent document 1 discloses the following configuration as a first system for realizing this method. That is, the substrate having embedded data thereon is positioned under the semitransparent mirror. An image from the substrate is captured by an image capture device. The image capture device sends the captured image to a decoder, which decodes the image and determines a code from the captured image. The decoder sends the code to the image generator. The image generator processes the codes, generates / retrieves image information based on these codes, and sends this image information to the translucent mirror. An observer looking down at the translucent mirror sees the image generated by the image generator overlaid on the image from the substrate.

  Moreover, the following structure is disclosed as a second system for realizing this method. That is, the image from the substrate is reflected from the translucent mirror to the projection surface. Images from the substrate are also captured by an image capture device. The image capture device forwards this image to the decoder, which processes the image to decode the alignment information and determine whether further information should be generated. Based on the determination, the decoder passes the signal to the image generator and directs the image generator to generate an image. An image generator generates an image that is projected onto the projection surface. An observer viewing the projection surface sees the image from the substrate overlaid on the image generated by the image generator.

JP 2001-31733 A (FIGS. 8 and 9)

  Here, conventionally, when displaying information associated with a specific range among information associated with each position of the document, the specific range cannot be designated without moving the document. Accordingly, there is a problem that the operability when displaying such information is not good.

  The present invention has been made under the background as described above, and the object thereof is an operation for displaying information associated with a specific range among information associated with each position on a document. It is to improve sex.

The invention described in claim 1
Imaging means for imaging a subject;
Display means for displaying an image;
An acquisition means for acquiring the position information at the specific location from the image obtained by imaging the specific location on the document in which the position information is embedded at each position by the imaging means;
Control means for controlling the display means to display information associated with a specific range determined using the position information acquired by the acquisition means among information associated with each position on the document And a portable terminal device.
The invention described in claim 2
2. The apparatus according to claim 1, further comprising a determination unit that determines the specific range using the position information acquired by the acquisition unit and information related to a positional relationship between the imaging unit and the display unit. It is a portable terminal device of description.
The invention described in claim 3
2. The control unit according to claim 1, wherein the control unit controls the display unit to display the information associated with the specific range so as to be superimposed on an image of a portion within the specific range of the document. It is a portable terminal device.
The invention according to claim 4
2. The mobile terminal device according to claim 1, wherein the control unit controls the display unit to display information belonging to a classification selected by a user among information associated with the specific range. is there.
The invention described in claim 5
Imaging a specific location on the document with location information embedded at each location;
Obtaining the position information at the specific location from the image obtained by imaging;
Determining a specific range on the document using the acquired position information;
A method of displaying information associated with the specific range among information associated with each position on the document.
The invention described in claim 6
On the computer,
A function of acquiring the position information at the specific location from the image obtained by imaging the specific location on the document in which the location information is embedded at each position,
A program for realizing a function of displaying, on a display unit, information associated with a specific range determined using the acquired position information among information associated with each position on the document. .
The invention described in claim 7
The program according to claim 6, further causing the computer to realize a function of determining the specific range using the acquired position information and information regarding a positional relationship between the imaging unit and the display unit. .
The invention according to claim 8 provides:
The program according to claim 6, wherein the display function displays information associated with the specific range so as to be superimposed on an image of a portion of the document within the specific range.
The invention according to claim 9 is:
The program according to claim 6, wherein the display function displays information belonging to a classification selected by a user among information associated with the specific range.

In the invention of claim 1, when displaying information associated with a specific range among information associated with each position of the document, the specific range can be designated without moving the document. Compared to the case where this configuration is not provided, the operability when displaying such information is improved.
The invention of claim 2 has an effect that a specific range when a specific portion on a document is imaged can be determined according to the structure of the mobile terminal device.
According to the third aspect of the present invention, since an image of a part within a specific range of the document can be displayed in an overlapping manner, the displayed information can be grasped in relation to the original document.
The invention of claim 4 has an effect that information desired by the user among a plurality of types of information can be displayed.
According to the fifth aspect of the present invention, when displaying information associated with a specific range among information associated with each position of the document, the specific range can be determined regardless of the location to be imaged. As compared with the case where the present configuration is not provided, there is an effect that information in an appropriate range can be displayed.
According to the sixth aspect of the present invention, when displaying information associated with a specific range among information associated with each position of the document, the specific range can be determined regardless of the location to be imaged. As compared with the case where the present configuration is not provided, there is an effect that information in an appropriate range can be displayed.
The invention of claim 7 has an effect that a specific range when a specific part on a document is imaged can be determined according to the structure of the mobile terminal device.
According to the eighth aspect of the present invention, since an image of a part within a specific range of the document can be displayed in an overlapping manner, the displayed information can be grasped in relation to the original document.
The invention of claim 9 has an effect that information desired by the user among a plurality of types of information can be displayed.

The best mode for carrying out the present invention (hereinafter referred to as “embodiment”) will be described below in detail with reference to the accompanying drawings.
FIG. 1 shows the configuration of a computer system to which this embodiment is applied.
As shown in the figure, in this computer system, a mobile phone 10 and a server 20 are connected via a base station 80 and a network 90.

The mobile phone 10 is a general mobile phone and is owned by a general user. However, in the present embodiment, the mobile phone 10 is a so-called “mobile phone with camera” having an imaging function. In addition to the function of communicating by voice, it is assumed that a function for performing data communication with the server 20 is provided.
The server 20 is realized by a personal computer, a workstation, or another computer, and is installed in a management center or the like of a company that provides a service for the mobile phone 10.
The base station 80 is a device for connecting the mobile phone 10 to the network 90.
The network 90 is a network that exists between the server 20 and the base station 80. The network 90 includes a packet communication network for the mobile phone 10 and the Internet, but these are shown without distinction in the figure.

Next, the hardware configuration of the mobile phone 10 and the server 20 shown in FIG. 1 will be described with reference to FIG.
FIG. 2A is a diagram schematically illustrating an example of the hardware configuration of the mobile phone 10.
As shown in the figure, the mobile phone 10 includes a CPU (Central Processing Unit) 10a as a computing means, a memory 10b as a storage means, and a display mechanism 10c including a display such as an LCD (Liquid Crystal Display). And an input device 10d such as operation keys, a sound input mechanism 10e such as a microphone, and a sound output mechanism 10f such as a speaker. In addition, an imaging mechanism 10g including a camera for imaging a subject is provided. Furthermore, a radio circuit 10h and an antenna 10i for performing radio communication via the base station 80 are provided. Here, it is assumed that the wireless circuit includes a baseband LSI that performs signal processing of digital data transmitted and received wirelessly, although not shown.

FIG. 2B is a diagram schematically illustrating an example of the hardware configuration of the server 20.
As shown in the figure, the server 20 includes a CPU 20a that is a computing means, a main memory 20b that is a storage means, and a magnetic disk device (HDD: Hard Disk Drive) 20c. Here, the CPU 20a executes various types of software such as an OS (Operating System) and applications, and realizes the above-described functions. The main memory 20b is a storage area for storing various software and data used for execution thereof, and the magnetic disk device 20c is a storage area for storing input data for various software, output data from various software, and the like. .
Further, the server 20 includes a communication I / F 20d for performing communication with the outside, a display mechanism 20e including a video memory and a display, and an input device 20f such as a keyboard and a mouse.

By the way, in this embodiment, first, a specific portion on a document is photographed using the camera of the mobile phone 10. And the information relevant to the specific range determined according to the specific location is acquired from the server 20 and displayed on the display of the mobile phone 10.
Here, the document to be photographed is an image formed on a medium such as paper, but a code image representing identification information of the document is embedded so that related information can be specified. Also, a code image representing position information for specifying each position on the document is embedded so that the photographing location can be recognized.

Therefore, this code image will be described.
FIG. 3 is a diagram for explaining the code pattern that is the basis of the code image.
First, the bit pattern constituting the code pattern will be described.
FIG. 3A shows an example of bit pattern arrangement.
A bit pattern is the minimum unit of information embedding. Here, as shown in FIG. 3A, bits are arranged at two locations selected from nine locations. In the figure, black squares indicate positions where bits are arranged, and hatched squares indicate positions where bits are not arranged. There are 36 (= ₉ C ₂ ) combinations for selecting 2 locations out of 9 locations. Therefore, 36 kinds (about 5.2 bits) of information can be expressed by such an arrangement method.

By the way, the minimum square shown in FIG. 3A has a size of 2 dots × 2 dots at 600 dpi. Since the size of one dot at 600 dpi is 0.0423 mm, one side of this minimum square is 84.6 μm (= 0.0423 mm × 2). The larger the dots that make up the code pattern, the more likely it is to be noticeable. However, if it is too small, printing with a printer becomes impossible. Therefore, the above value is adopted as the dot size, which is larger than 50 μm and smaller than 100 μm. Thereby, it is possible to form dots of an optimum size that can be printed by the printer. That is, 84.6 μm × 84.6 μm is the minimum size that can be stably formed by the printer.
In addition, by setting the dot to such a size, one side of one bit pattern becomes about 0.5 mm (= 0.0423 mm × 2 × 6).

A code pattern composed of such bit patterns will be described.
FIG. 3B shows an example of the arrangement of code patterns.
Here, the minimum square shown in FIG. 3B corresponds to the bit pattern shown in FIG. In FIG. 3A, it has been described that 36 kinds of information can be expressed by one bit pattern. However, in this code pattern, one bit pattern has 32 kinds (5 bits) except for a synchronous code. It shall represent information.

The identification code obtained by encoding the identification information is embedded using 16 (= 4 × 4) bit patterns. Also, the X position code obtained by encoding the position information in the X direction and the Y position code obtained by encoding the position information in the Y direction are each embedded using four bit patterns. Further, a synchronization code for detecting the position and rotation of the code pattern is embedded in the upper left corner using one bit pattern.
Note that the size of one code pattern is approximately 2.5 mm because it is equal to the width of five bit patterns. In the present embodiment, a code image obtained by imaging the code pattern generated in this way is arranged on the entire sheet surface.

Although a specific encoding method of identification information and position information has not been described here, encoding is performed by the following method.
First, an RS (Reed-Solomon) code of a block coding method is used for encoding identification information. As described with reference to FIG. 3, in the present embodiment, information is embedded using a bit pattern that can represent 5-bit information. Accordingly, since an information error also occurs in units of 5 bits, an RS code having good coding efficiency is used in the block coding method. However, the encoding method is not limited to the RS code, and other encoding methods such as a BCH code can also be used.
In addition, an M-sequence code, which is a kind of pseudo-random sequence, is used for encoding position information. Here, the M sequence is a sequence of the maximum period that can be generated by a K-stage linear shift register, and has a sequence length of 2 ^K −1. Arbitrary consecutive K bits extracted from the M sequence have a property that they do not appear at other positions in the same M sequence. Therefore, the position information is encoded using this property.

Next, the outline | summary operation | movement in this Embodiment is demonstrated.
FIG. 4 is a diagram for explaining the outline operation of the present embodiment.
In the present embodiment, first, a user uses a mobile phone 10 to photograph a document and obtain an image (A). Thereby, the mobile phone 10 decodes the identification code and the position code embedded in the photographed image to obtain identification information and position information (B). At this time, the mobile phone 10 displays a screen for selecting a classification of an image related to the photographed image (hereinafter referred to as “related image”), and the user selects the classification on the screen (C ).

  Then, the information is transmitted to the server 20. Here, the server 20 stores a document image that is electronic data of a document to be photographed and a related image related to the document image. Therefore, the server 20 searches the database for document images and related images according to the received information (D). Then, an image in a range corresponding to the display of the mobile phone 10 is cut out from the searched image, and this is used as a display image (E). For example, if a certain part of a document is photographed from directly above, an image in a range existing immediately below the display is cut out to be a display image.

Then, the server 20 transmits the display image to the mobile phone 10, and the mobile phone 10 displays the display image on the display (F).
The processes from A to F are repeated according to the movement of the mobile phone 10 (G). That is, while the mobile phone 10 moves on the document, it is automatically controlled so that the document is photographed at a constant time interval. Or you may make it image | photograph by a user giving an instruction | indication of imaging | photography, moving the mobile telephone 10. FIG.

With such a schematic operation, an image as shown in FIG. 5 is displayed on the display of the mobile phone 10. Here, a map is illustrated as a document to be imaged.
FIG. 5A shows an example in which related images are synthesized and displayed on a map to be photographed. Since a related image is added and displayed on the original map, such a display form is hereinafter referred to as “additional type”. FIG. 4 illustrates the “additional type” because the document image which is electronic data of the original document and the related image related to the document image are combined into a display image.
On the other hand, FIG. 5B shows an example in which only a related image is displayed without displaying a map to be photographed. In the following, such a display form is referred to as a “perspective type” because it feels as if the map being photographed is seen through and a related image hidden behind the map is displayed.

Next, the functional configuration of the mobile phone 10 in the present embodiment will be described.
FIG. 6 is a block diagram showing a functional configuration of the mobile phone 10.
As shown in the figure, the mobile phone 10 includes an image analysis unit 11, an identification information acquisition unit 12, a position information acquisition unit 13, a range information generation unit 14, a selection information acquisition unit 15, a display control unit 16, And a communication control unit 17.

The image analysis unit 11 acquires an image captured by the imaging mechanism 10g, analyzes the image, and extracts an identification code and a position code.
The identification information acquisition unit 12 decodes the identification code extracted by the image analysis unit 11 and acquires the identification information.
The position information acquisition unit 13 decodes the position code extracted by the image analysis unit 11 and acquires position information.
The range information generation unit 14 corrects the position information acquired by the position information acquisition unit 13 based on the positional relationship between the display mechanism 10c and the imaging mechanism 10g, and indicates the range information corresponding to the display mechanism 10c at the time of shooting. Is generated.
The selection information acquisition unit 15 acquires information (hereinafter referred to as “selection information”) related to selection of a classification by the user when related images belonging to a plurality of classifications exist for the same document image.
The display control unit 16 performs control when the display mechanism 10c displays a related image or the like.
The communication control unit 17 performs control when the wireless circuit 10h communicates with the server 20 via the antenna 10i, the base station 80, and the network 90.

  These functions are realized by cooperation between software and hardware resources. That is, the program in which the CPU 10a realizes the functions of the image analysis unit 11, the identification information acquisition unit 12, the position information acquisition unit 13, the range information generation unit 14, the selection information acquisition unit 15, the display control unit 16, and the communication control unit 17. Are realized from the memory 10b.

Next, the functional configuration of the server 20 in the present embodiment will be described.
FIG. 7 is a block diagram showing a functional configuration of the server 20.
As illustrated, the server 20 includes a communication control unit 21, a search unit 22, an extraction unit 23, and a database 24.
The communication control unit 21 performs control when the communication I / F 20 d communicates with the mobile phone 10 via the base station 80 and the network 90.
The search unit 22 searches the database 24 based on the information received by the communication control unit 21 and returns the search result to the communication control unit 21.
The extraction unit 23 extracts a specific range from the search result obtained by the search unit 22.
The database 24 stores document images and related images so that they can be searched based on document identification information and classifications related to related images.

These functions are realized by cooperation between software and hardware resources. That is, the CPU 20a realizes these functions by reading a program that realizes the functions of the communication control unit 21, the search unit 22, and the extraction unit 23 from, for example, the magnetic disk device 20c into the main memory 20b.
The database 24 can be realized by using, for example, the magnetic disk device 20c.

Hereinafter, the contents of the database 24 will be specifically described. Here, too, a map is illustrated as a document.
First, in the database 24, a map image and a related image are stored for each map ID that is identification information for uniquely identifying a map. 7 exemplifies map images and related images for map IDs “1001”, “1002”,...
Here, the map image is electronic data obtained by digitizing a map to be photographed in advance.
The related image is electronic data obtained by digitizing other information related to the same region as that included in the map image.

Examples of the related image include images such as a bus route map, a train route map, a general road map, and an expressway map as shown on the left side in the database 24 of FIG.
By the way, these related images are for creating and storing other information related to the map image in advance as image data, but the storage method of the related images is not necessarily limited to this.
That is, a storage method as shown on the right side in the database 24 of FIG. Here, other information related to the map image is stored not as image data but as correspondence information between coordinate values and facility names. In this case, a related image is dynamically obtained by creating image data indicating that there is a facility corresponding to the position indicated by the coordinate value, as necessary.

Next, the operation of the present embodiment will be described in detail.
FIG. 8 is a sequence diagram showing operations of the mobile phone 10 and the server 20 in the present embodiment.
In the mobile phone 10, first, the image analysis unit 11 acquires an image captured by the imaging mechanism 10g (step 101). Then, the acquired image is analyzed (step 102).

Here, the processing performed by the image analysis unit 11 includes the following.
First, in the acquired image, there is a process of separating a document image (for example, an image of the map itself) and a code image. For example, if a code image is printed in a color that is not used to print a document, it is possible to perform separation by detecting this color.
Alternatively, an infrared sensor may be provided as an image sensor in the imaging mechanism 10g. In this case, the document is printed with visible toner, and the code image is printed with invisible toner having an absorption wavelength in the infrared region. Then, by irradiating the infrared light with the infrared LED and receiving the reflected light with the image sensor for infrared light, the image of the document and the code image can be separated.

  Next, there is a process for removing noise included in the cut code image. Here, the noise includes variations in CMOS sensitivity, noise generated by an electronic circuit, and the like. What processing is performed to remove noise should be determined according to the characteristics of the imaging mechanism 10g. For example, sharpening processing such as blurring processing or unsharp masking can be applied.

The next process is a process of detecting a dot pattern (position of a dot image) from an image. For example, the dot pattern portion and the background portion are separated by binarization processing, and the dot pattern can be detected from each binarized image position. When a binarized image contains a lot of noise components, for example, it is necessary to combine a filter process for determining a dot pattern based on the area and shape of the binarized image.
The detected dot pattern is converted into digital data on a two-dimensional array. For example, on a two-dimensional array, a position where there is a dot is converted to “1”, and a position where there is no dot is converted to “0”.

Next, there is processing for detecting a bit pattern composed of a combination of two dots shown in FIG. 3A from digital data. For example, the bit pattern can be detected by moving the boundary position of the block corresponding to the bit pattern on the two-dimensional array and detecting the boundary position so that the number of dots included in the block is two. .
When the bit pattern is detected in this way, the synchronization code is detected by referring to the type of the bit pattern. And an identification code and a position code are detected based on the positional relationship from a synchronous code.

Thereafter, the identification information acquisition unit 12 acquires the identification information by decoding the identification code, and passes it to the communication control unit 17 (step 103). The identification information is obtained by performing RS decoding processing on the identification code.
In addition, the position information acquisition unit 13 decodes the position code to acquire the position information, and transfers it to the range information generation unit 14 (step 104). The position information can be obtained by comparing the position of the read partial series with the M series used at the time of image generation.
Next, the range information generation unit 14 obtains a range of an image to be displayed on the display of the mobile phone 10 based on the position information acquired in step 104 and information on the positional relationship between the display mechanism 10c and the imaging mechanism 10g. The data is transferred to the communication control unit 17 (step 105). It is assumed that information regarding the positional relationship between the display mechanism 10c and the imaging mechanism 10g is stored in the memory 10b.

Here, how to obtain the range of the image to be displayed on the display of the mobile phone 10 will be specifically described.
FIG. 9A shows a state in which the mobile phone 10 is viewed horizontally from the top.
As shown in the figure, for example, the center point of the lens of the camera 18 is the origin O, the X axis is taken in the direction parallel to the horizontal frame of the display 19, and the Y axis is taken in the direction parallel to the vertical frame. A point at the upper left corner of the display 19 is A, a point at the upper right corner is B, a point at the lower left corner is C, and a point at the lower right corner is D.
On the other hand, when shooting a document with the mobile phone 10, coordinates are set on the document as follows.
That is, the center point of the shooting range 38 by the camera 18 of the mobile phone 10 is set as the origin O ′, the X axis is taken in the direction parallel to the horizontal frame of the shooting range 38, and the Y axis is taken in the direction parallel to the vertical frame. Then, points A ′, B ′, C ′, and D ′ corresponding to the points A, B, C, and D on the mobile phone 10 are obtained at the coordinates on the document. A range surrounded by these points A ′, B ′, C ′, and D ′ is set as a display range 39.

Returning to FIG. 8 again, the description of the operation of the present embodiment will be continued.
When the identification information is acquired by the identification information acquisition unit 12 and the range information is generated by the range information generation unit 14, the selection information acquisition unit 15 selects an option for the display control unit 16 to select a classification related to the related image. Is displayed. Then, under the control of the display control unit 16, the display mechanism 10c displays options for selecting a classification related to the related image (step 106).
Therefore, when the user selects a specific classification related to the related image, the selection information acquisition unit 15 acquires selection information indicating the selection content and passes it to the communication control unit 17 (step 107). At this time, the user also designates a display form when displaying an image on the mobile phone 10. Then, the selection information acquisition unit 15 includes the designation of the display form in the selection information and transfers it to the communication control unit 17.
Thereby, the communication control unit 17 controls the radio circuit 10h, and transmits identification information, range information, and selection information to the server 20 via the antenna 10i, the base station 80, and the network 90 (step 108).

Thereby, in the server 20, the communication I / F 20d receives the identification information, the range information, and the selection information, and passes these pieces of information to the search unit 22 via the communication control unit 21 (step 201).
Next, the search unit 22 searches the database 24 using the identification information and selection information as keys, and searches for related images (step 202). That is, the related image corresponding to the identification information and belonging to the classification indicated by the selection information is specified. For example, if the user takes a map with the map ID “1001” using the mobile phone 10 and selects “bus route map” as the related image to be displayed, the bus route map associated with the map ID “1001”. Identify the images. Further, if the user takes a map with the map ID “1002” using the mobile phone 10 and selects “facility name” as the related image to be displayed, the facility image associated with the map ID “1002” is displayed. (Images showing locations such as “C bank head office”, “△△ department store”) are identified.

Next, the search unit 22 determines whether the display form is “perspective type” or “additional type” (step 203).
Here, when it is determined that the display form is “perspective type”, the retrieved related image is directly transferred to the extraction unit 23. At this time, the range information passed from the communication control unit 21 is also passed to the extraction unit 23.
On the other hand, when it is determined that the display form is “additional type”, the database 24 is searched again using the identification information as a key, and a map image is searched (step 204). Then, the related image retrieved in step 202 is synthesized with this map image, and this synthesized image is delivered to the extraction unit 23 (step 205). At this time, the range information passed from the communication control unit 21 is also passed to the extraction unit 23.

  Then, the extraction unit 23 cuts out the range indicated by the range information from the image passed from the search unit 22 (step 206). Then, the extraction unit 23 returns this to the search unit 22 as a display image to be displayed on the mobile phone 10, and the search unit 22 delivers this display image to the communication control unit 21. Thereafter, the communication control unit 21 transmits the display image to the mobile phone 10 via the communication I / F 20d (step 207).

  Thereby, in the mobile phone 10, the communication control unit 17 acquires a display image via the antenna 10i and the radio circuit 10h (step 109). Then, the display image is transferred to the display control unit 16, and the display mechanism 10c displays the display image under the control of the display control unit 16 (step 110).

The embodiment has been described above.
In the present embodiment, the range information generation unit 14 in the mobile phone 10 performs processing for generating range information indicating the range of an image to be displayed based on position information acquired from a document. However, the generation of the range information can be performed by the server 20. In that case, the mobile phone 10 sends identification information, position information, selection information, and information for specifying its own model to the server 20. Here, it is assumed that the magnetic disk device 20c of the server 20 stores a correspondence relationship between the model of the mobile phone 10 and information on the structure thereof (particularly, the positional relationship between the display mechanism 10c and the imaging mechanism 10g). . Thereby, the server 20 acquires information related to the structure of the mobile phone 10 that transmitted the information, and specifies the range of the image to be cut out based on this information and the position information.

  In the present embodiment, after a document image and a related image are combined, a specific range is cut out from the combined image to generate a display image. However, the processing need not necessarily be performed in this order. That is, a specific range may be cut out from the document image, the same range may be cut out from the related image, and these may be combined to generate a display image.

Here, the background to the present embodiment will be briefly described.
Nowadays, many mobile phones are provided with a camera, and this camera is often used not only as a photographer but also as an input device such as a barcode or a QR code. For example, a QR code is printed on a map, and it is also possible to obtain information on facilities on the map by inputting this code.
However, these codes only represent information at a certain point on the map. That is, two-dimensional information utilizing the characteristics of the map cannot be expressed. As the two-dimensional information, for example, there is a bus route that connects two points, but such information cannot be expressed.
Therefore, in this embodiment, continuous two-dimensional information can be expressed by using a code that can embed continuous information.

By the way, in this Embodiment, the map was illustrated as a document and the bus route map and facility name which were effective when displayed on a map as a related image were illustrated. However, this embodiment can be applied to other documents.
For example, the document may be a text document written in a certain language, and the related image may be a result of translating the text document into another language.
It is also conceivable that a document is a drawing and a related image is a comment on the drawing.

In this embodiment, a camera-equipped mobile phone is assumed as a device that can shoot a document, display an image, and can be easily moved on the document. Any portable terminal device may be used as long as it has the following.
Furthermore, in this embodiment, the document image and the related image are acquired from a server such as a company. However, the present invention is not limited to this configuration. For example, the same function as that of the server may be realized by a personal computer that a general user has as in the case of the mobile phone. Alternatively, if the storage capacity of the mobile terminal device permits, the mobile terminal device may store the document image and the related image.

  The program for realizing the present embodiment can be provided not only by communication means but also by storing it in a recording medium such as a CD-ROM.

It is the figure which showed the structure of the computer system with which embodiment of this invention is applied. It is the figure which showed the hardware constitutions of the mobile telephone and server in embodiment of this invention. It is a figure for demonstrating the code pattern used by embodiment of this invention. It is a figure for demonstrating schematic operation | movement in embodiment of this invention. It is the figure which showed the example of the image displayed on a mobile telephone in embodiment of this invention. It is the block diagram which showed the function structure of the mobile telephone in embodiment of this invention. It is the block diagram which showed the function structure of the server in embodiment of this invention. It is the sequence diagram which showed the operation | movement of the mobile telephone and server in embodiment of this invention. It is a figure for demonstrating the production | generation process of the range information in embodiment of this invention.

Explanation of symbols

10 ... mobile phone, 20 ... server, 80 ... base station, 90 ... network

Claims (9)

Imaging means for imaging a subject;
Display means for displaying an image;
An acquisition means for acquiring the position information at the specific location from the image obtained by imaging the specific location on the document in which the position information is embedded at each position by the imaging means;
Control means for controlling the display means to display information associated with a specific range determined using the position information acquired by the acquisition means among information associated with each position on the document And a portable terminal device.   2. The apparatus according to claim 1, further comprising a determination unit that determines the specific range using the position information acquired by the acquisition unit and information related to a positional relationship between the imaging unit and the display unit. The portable terminal device described.   2. The control unit according to claim 1, wherein the control unit controls the display unit to display the information associated with the specific range so as to be superimposed on an image of a portion within the specific range of the document. Mobile terminal device.   2. The portable terminal device according to claim 1, wherein the control unit controls the display unit to display information belonging to a classification selected by a user among information associated with the specific range. Imaging a specific location on the document with location information embedded at each location;
Obtaining the position information at the specific location from the image obtained by imaging;
Determining a specific range on the document using the acquired position information;
And displaying information associated with the specific range among information associated with each position on the document. On the computer,
A function of acquiring the position information at the specific location from the image obtained by imaging the specific location on the document in which the location information is embedded at each position,
A program for realizing a function of displaying, on a display unit, information associated with a specific range determined using the acquired position information among information associated with each position on the document.   The program according to claim 6, further causing the computer to realize a function of determining the specific range by using the acquired position information and information on a positional relationship between the imaging unit and the display unit.   7. The program according to claim 6, wherein in the display function, information associated with the specific range is displayed so as to be superimposed on an image of a portion within the specific range of the document.   The program according to claim 6, wherein the display function displays information belonging to a category selected by a user among information associated with the specific range.

Images