JP2003244488A - Image information adding apparatus and system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display a character to be attached to an image on a display device by calculating the location information from a sign at a specific place by photographing it, and obtaining the same image as the field of view of a user from a database on the basis of this location information. <P>SOLUTION: In the image information adding apparatus provided with an optical information means for obtaining an angle of view from a focal distance and an image cutout size in the case of photographing from information of an image input means and a calculation means for calculating a photographer location information and a photographing direction on the basis of the angle of view and the image information adding system provided with a sign means having a pair of patterns and an information storage means storing a background image in the same direction as a direction for picking up the image of the sign means and attached information attached to this background image, the photographer location and the photographing direction are calculated from the photographed images of the pair of patterns, background image information acquired from the information storage means is worked on the basis of this result, the background image information of the same field of view is acquired, the attached information is acquired, and the background image of the same field of view and the attached information are displayed. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for obtaining incidental information of a specific location based on an image from an image input means, and an image information adding device for associating the incidental information with an image of an object to be picked up. Regarding the system.

[0002]

2. Description of the Related Art As a method for specifying a place where an image of an object is picked up by an image input means, a GPS is built in the image input device to specify the place, and a direction magnet is used to determine in which direction the image picked up is. It is known to specify that direction.

Further, in Japanese Patent Laid-Open No. 6-318231, a device for obtaining image information and a measuring means such as a position retrieve information from a database for a predetermined object, and an image input means inputs the image. The information from the database is added to the image to display the image on the video display.

[0004]

However, in the method of incorporating the GPS and the compass in the image pickup device as described above, there are problems such as an increase in the manufacturing cost of the device and an increase in the entire device.

Further, the method disclosed in Japanese Patent Laid-Open No. 6-318231 has a problem in that the device has a measuring means for measuring the position and the like, which makes the device complicated.

An object of the present invention is to photograph a sign placed in a place having a specific landscape and calculate position information from the sign without the need for GPS, compass, or position measuring means. Based on the calculated position information, the same image as the visual field of the user is obtained from the database, and the characters (characters, symbols, pictures, sounds) attached to the image are displayed on the display device.

[0007]

In order to solve the above problems, the present invention adopts the following configurations. An image input unit 3 for picking up an image of an object, an optical information unit 10 for obtaining an angle of view from a focal length at the time of shooting and an image cutout size from information of the image input unit, a position of a photographer based on the angle of view, and a shooting position. An image information adding device 1 including a distance / direction calculating means 8 for calculating a direction and an image display means 6 for displaying an image on a screen from the image input means through a signal processing circuit 4A,
Marking means 2 installed in a specific place with a pair of symbols having predetermined dimensions in the horizontal and vertical directions, background image information and the background image in the same direction as the direction in which the marking means is imaged. Image information addition including an information storage unit 12 that stores associated information associated with the image information, and a communication circuit network 11 that connects the communication control unit 9 and the information storage unit 12 provided in the image information addition device. System,
A background image acquired from the information storage means on the basis of the position information by calculating the photographer position and the photographing direction, which are the position information of the photographer, from the photographed images of the pair of symbols (two cross shapes) of the sign means 2. The information is processed 4B to obtain the background image information of the same field of view and the accompanying information of the background image, and the background image of the same field of view and the accompanying information are displayed on the image display means (of a famous tourist spot). It naturally includes disposing a person in the vicinity of the marking means installed in the photographing spot and displaying this person together with the background image and the accompanying information).

Further, an image inputting means for picking up an object, an optical information means for obtaining an angle of view from a focal length at the time of photographing and an image cutout size from information of the image inputting means, and a position of a photographer based on the angle of view. And an image information adding device including a distance / direction calculating means for calculating a shooting direction and an image display means for displaying an image on the screen from the image input means through a signal processing circuit, and the image information adding device is predetermined in the horizontal and vertical directions. Information storing a sign means installed at a specific place with a pair of symbols having different dimensions, background image information in the same direction as the direction in which the sign means is imaged, and accompanying information accompanying the background image An image information adding system comprising: a storage means; and a communication circuit network connecting the communication control section provided in the image information adding device and the information storage means, wherein the image information adding system comprises a pair of the marking means. The photographer position and the photographing direction, which are the photographer's position information, are calculated from the photographed image of the pattern, and the background image information obtained from the information storage means is processed based on the position information to obtain the background image information of the same field of view. Along with the acquisition, the accompanying information of the background image is acquired, and the object imaged by the image input means 3 (the background image taken out from the information storage means is not adopted, but the actually copied background image is adopted. And the accompanying information (the arrow pointing to the mountain in the background image, the building, etc. and its description) is displayed, and the combined image is displayed on the image display means. Is characterized by.

Further, in the image information adding system, instead of obtaining the position information of the photographer from the pair of symbols of the sign means, the sign information storage unit 31 for obtaining latitude and longitude information of the position of the sign means from the sign means. Is provided in the image information adding device, and the position information of the photographer is obtained from the latitude and lightness information from the sign information storage unit.

Further, in the image information adding system, the object imaged by the image input means 3 is compared with the processed object taken out from the information storage means 12 and processed to make a difference between the objects. It is determined whether or not there is (There is a case where the building in the background image is reconstructed or lost because the processing object is a past image, and it is determined whether it is different from the situation at the present time. It may be used to update the information based on the judgment, or if it is different, an actual image from the signal processing circuit 4A may be adopted without adopting the object to be processed).

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION An image information adding apparatus and system according to an embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an image information adding device and its system according to an embodiment of the present invention.

In FIG. 1, the image information adding system is
As a basic configuration, the image information adding device 1, a marking means 2 for drawing a pair of symbols and transmitting unique information, a communication circuit network 11, and the image information adding device 1 are accessible via the communication circuit network 11. The database 12 is composed of blocks.

Further, the image information adding device 1 includes an image pickup section 3
An image processing circuit 4A for a camera signal that corrects the image of the image pickup unit, an optical information unit 10 that obtains the angle of view by acquiring information of the image pickup unit, and the position of the photographer based on the optical information from the optical information unit. And a distance / direction calculation unit 8 for calculating the shooting direction,
The sign information storage unit 7 that communicates with the sign unit 2 to store the unique information of the sign unit 2, the communication control unit 9 that communicates with the communication circuit network 11, and the accompanying information and the video signal obtained from the communication control unit 9 From a signal processing circuit 4B for processing, an image synthesizing section 5 for synthesizing the output of the image processing circuit 4A for camera signals and the signal processing circuit 4B for video from the communication section 9, and a display section 6 for displaying the image. Composed.

Further explaining the marking means 2, a pair of symbols ((1) or (2) in FIG. 2) having recognizable information of predetermined horizontal and vertical dimensions (X, Y). As shown in FIG. 2, the cross-shaped symbols are a pair of symbols separated from each other by L. A pair of symbols is drawn on one marker 2 in (1) of FIG. 2 and an L symbol is shown in (2) of FIG. , One cross shape is drawn on each of the two spaced apart markers 2), and means for transmitting with unique information described later (30, 31, 32 in the description of FIG. 3 described later).
In a specific place (the specific place is a place where the background of the field of view in the case of being viewed from a proper direction in the north, south, east, and west and a proper distance from the database is previously stored in a database). It was installed. Further, the unique information of the marking means 2 is registered in the database 12.

The unique information transmitting means of the marking means 2 has the structure shown in FIG. In FIG. 3, reference numeral 31 is a storage unit in which unique information is stored. 30 is a storage unit 3
1 is composed of a modulator for reading out the unique information from the device 1 for high frequency modulation, a high frequency output unit for wirelessly transmitting the data, and an antenna unit. Reference numeral 32 is a power supply unit for constantly driving the above circuit. For example, it is composed of a solar cell, a charging unit, and a secondary battery. These blocks are paired with signs.

Further, the database 12 has a marking means 2
Specific information, an image of a place where the sign means 2 is installed (for example, a background image existing in the back of the sign), and accompanying information (for example, a proper noun name of the background image and a background image) Arrow, its description, voice, etc.) are registered.

Next, the operation of the image information adding device 1 will be described. The image information adding device 1 is, as shown in FIG.
It is composed of the constituent elements 3 to 9 described above, and includes the general constituent elements of the digital camera and adds the improved constituent elements thereto.

When the image information adding apparatus 1 takes a picture of the marker 2 whose dimensions are specified in the horizontal and vertical directions shown in FIG. 2 by the image pickup section 3, the image processing circuit 4A makes it possible to take a normal picture from the front. Since it can be determined that the shapes are similar to each other (according to the distance between the sign and the image pickup unit), the sign 2 is obtained from the optical information unit (focal length and image cutout size of the image pickup sensor) 10 obtained from the image pickup unit 3. It is possible to calculate the distance to the marker 2 by the arithmetic circuit 8 based on the standardized size and the distance when the object is standardized from the optical information unit 10 described above.

Referring to FIG. 4A, the reference angle of view is set to 2Q.
(In (2) of FIG. 4, the reference angle of view Q is the angle formed by the line connecting the lowermost end and the uppermost end of the size Y of the marker from the left reference point), the reference distance is M, and the standard symbol size of the marker is Y
And Then, by measuring the angle of view 2Q ′ and the size Y ′ of the marker in the image from the optical information 10 in the photographed image, one symbol of the marker (requires a pair of cross-shaped symbols shown in FIG. 4) The distance M ′ from which the photographer is present is M ′ = (M × Y × tanQ) / (Y ′ × tanQ ′). In addition, in (3) and (4) of FIG.
Indicates the size when the marker size Y distant from M ′ is brought to the distance M by the perspective method, and is an auxiliary dimension when calculating M ′ and is a value in the calculation process.

Then, as shown in FIG. 4, for a pair of symbols, the distances from the two marks are measured and calculated corresponding to the symbols, that is, known reference values M, Y, Q
In addition to Y ', Q'from the inside of the image based on the optical information of the captured image
It becomes possible to obtain M1 ′ and M2 ′ by measuring

Further, as shown in FIG. 5, when the mark 2 is photographed at an angle, the standard mark 2 is distorted by the perspective method because it becomes a two-dimensional image. By comparing the dimensions of the two patterns from this distortion, the angle with respect to the marker 2 with the photographer is calculated,
The actual distance is calculated by the arithmetic circuit 8 from the standardized distance from the optical information 10 and the above calculated angle, and the own position information with respect to the sign 2 is clarified.

Next, the calculation of the direction with respect to the marking means will be described. Using M1 'and M2' measured and calculated using the two markers (a pair of markers) shown in FIG. 5, the distance from each symbol is calculated. Therefore, at the same time, it is possible to narrow down the position satisfying the distance from the design to one point (the points distant from the pair of marks by M1 'and M2', respectively, are determined as one point of the doll mark shown in the figure). You can decide how far apart. Furthermore, the doll mark narrowed down to 1 point is also narrowed down to 1 point on the opposite side of the drawing with the sign as a mirror surface, but since the photographer sees the sign drawn from one side It is possible to specify that the direction in which the vehicle is present is one side. Such calculation processing is performed based on the optical information from the optical information means.
It is calculated by the direction calculation unit 8. This determines how far away the photographer is from the sign and from which direction.

If the distance between the symbols in the sign is L, a triangle having three sides L, M1 ', M2' is formed. If the distance L between the symbols is large as shown in FIG. The difference between M1 'and M2' in the distance between
Since it is dimensioned, the difference in length between Y1 'and Y2' becomes large from the perspective method, and M1 ', M from the above calculation formula of M'
The difference of 2'becomes large, and L, M1 ', M for identifying the position
A triangle with two sides of 2'is more clear and an accurate position can be calculated.

The communication control unit 9 stores the position information (distance to the sign and direction to the sign) and the sign-specific information (information shown in FIG. 3) received from the sign 2 in the storage unit 7 for communication. The network 11 is connected to a predetermined database 12.

In the database 12, a signal relating to the unique information of the sign 2 is selected from the unique information of the sign and the position information (position and direction) calculated by the image information adding device 1 by a signal from the communication control unit (for example, , A background image when the photographer observes the specific place where the sign is installed is selected) and transmitted to the image information adding apparatus 1.

The selected image and the data of its accompanying information are input to the signal processing circuit 4B, where the selected image is an angle-of-view image based on the above-mentioned position information, that is, the same image as the visual field of the user ( The background image of the sign specified by the position and direction) is processed. At this time, the image information adding device 1 can select and acquire a character (character, picture, sound, etc.) associated with the video selected in the database. In this way, the image and the additional information are obtained from the database 12, the processing circuit 4B of the image information adding device 1 processes the signal based on the position information, and the signal is displayed on the image display device 6. Here, as the character, for example, a specific character such as a well-known building name or a distant mountain name in a background image selected from a database having a matched field of view can be cited.

The image taken by the image information adding device 1 can be compared (or combined) with the comparison circuit (or combination circuit) 5 and displayed on the display device 6.
The image synthesizing (comparing) circuit 5 compares the signals from the signal processing circuits 4A and 4B and sends the image from 4A and the accompanying information from 4B to the display unit 6, or the image from 4B and the accompanying method. The image is transmitted or only the image from 4A is transmitted to appropriately transmit the image or the accompanying information.

When registering the location of the sign 2 in the database, the unique information of the sign 2 and the latitude, longitude and altitude of the place where the sign 2 is to be installed are entered. It is possible to specify the latitude, longitude and altitude of the place where the sign 2 is installed by communicating with the database from.

[0029]

According to the present invention, an inexpensive sign installed in a specific place is photographed, and relative position information from the sign is recorded as standard optical information in a digital camera or the like in the photographed image. By calculating from the pattern size of the sign, additional information other than the information of the actual experience at a specific place from one image, that is, stored in a database that is stored easily or in detail because it is limited to a specific place It is possible to convert the captured image into an image in the same direction and distance as the landscape of the visual field that the user is going to shoot, and display the virtual image on the display device.

Further, the additional information related to the actually photographed image is compared and synthesized with the virtual image and the symbol size of the sign, and only the additional information and the actual image are superimposed on the photographed image and displayed on the display device. It will be possible.

Also, the shooting location (a sign is installed)
The position information can be known by the imaging device without using the GPS, and a system for catching the GPS signal is unnecessary, and the device can be simplified. Further, since the accompanying information related to the shooting location can be obtained, there is an effect that the video editing can be easily performed on the spot without performing the video editing after the shooting.

Further, since the position information is obtained from the sign, by comparing the background image of the photographer with the image transmitted from the database, the shape and color of the building reflected in the background of the image are collated. It also becomes possible. Further, by comparing the images, there is an effect that it is possible to determine the difference between the images (change due to the history).

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image information adding device and its system according to an embodiment of the present invention.

FIG. 2 is a diagram showing a specific structure of a labeling means according to the present embodiment.

FIG. 3 is a block diagram showing a configuration of a unique information transmitting means included in the marking means according to the present embodiment.

FIG. 4 is a diagram illustrating position calculation according to the present invention.

FIG. 5 is a diagram illustrating another position calculation according to the present invention.

[Explanation of symbols]

1 Image information addition device 2 marking means 3 Imaging means 4A, 4B image processing circuit 5 Image composition comparison section 6 Image display device 7 Sign specific information storage 8 Distance / direction calculator 9 Communication control unit 10 Optical information section 11 Communication network 12 Database 30 Modulation section, high frequency output section and antenna section 31 Unique Information Storage / Storage Unit 32 power supply

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ identification code FI theme code (reference) G06T 3/00 300 G06T 3/00 300 5L096 7/60 150 7/60 150P 180 180B H04N 7/173 630 H04N 7/173 630 7/18 7/18 K F Term (reference) 5B057 AA20 BA02 BA19 CE08 CE09 CH12 DA16 DC03 DC08 5B075 KK07 ND08 ND16 NK06 PP12 PP22 PQ02 PQ20 PQ32 PQ69 5C022 AA01 AA11 AB68 AC01 AC13 AC42 AC02 5C02 AC01 5 DA09 EA01 EH01 FD01 GB12 GB14 HA16 5C064 BA07 BB10 BC18 BC23 BD02 BD08 5L096 CA02 FA66 FA67

Claims (5)

[Claims] 1. An image input unit for picking up an image of an object, an optical information unit for obtaining an angle of view from a focal length at the time of photographing and an image cutout size from information of the image input unit, and a position of a photographer based on the angle of view. And an image information adding device having a distance / direction calculating means for calculating a shooting direction and an image display means for displaying an image on the screen from the image input means through a signal processing circuit, and a predetermined horizontal and vertical direction. Information storing a sign means installed in a specific place with a pair of symbols having different dimensions, background image information in the same direction as the direction in which the sign means is imaged, and accompanying information accompanying the background image An image information adding system, comprising: a storage unit; and a communication circuit network connecting the communication control unit provided in the image information adding device and the information storage unit, wherein The photographer position and the photographing direction, which are the photographer's position information, are calculated from the photographed image of the pattern, and the background image information obtained from the information storage means is processed based on the position information to obtain the background image information of the same field of view. An image information adding system, characterized in that the background image of the same field of view and the incidental information are displayed on the image display means while acquiring the incidental information of the background image. 2. An image input means for picking up an image of an object, an optical information means for obtaining an angle of view from a focal length at the time of shooting and an image cutout size from information of the image input means, and a position of a photographer based on the angle of view. And an image information adding device having a distance / direction calculating means for calculating a shooting direction and an image display means for displaying an image on the screen from the image input means through a signal processing circuit, and a predetermined horizontal and vertical direction. Information storing a sign means installed in a specific place with a pair of symbols having different dimensions, background image information in the same direction as the direction in which the sign means is imaged, and accompanying information accompanying the background image An image information adding system, comprising: a storage unit; and a communication circuit network connecting the communication control unit provided in the image information adding device and the information storage unit, wherein The photographer position and the photographing direction, which are the photographer's position information, are calculated from the photographed image of the pattern, and the background image information obtained from the information storage means is processed based on the position information to obtain the background image information of the same field of view. Image information characterized by acquiring the incidental information of the background image together with the acquisition, synthesizing the object imaged by the image input unit and the incidental information, and displaying the combined image on the image display unit. Additional system. 3. The image information adding system according to claim 1, wherein instead of obtaining the position information of the photographer from the pair of symbols of the marking means, the latitude and longitude information of the marking means position is obtained from the marking means. An image information adding system, wherein a sign information storage unit is provided in the image information adding device, and position information of a photographer is obtained from latitude and lightness information from the sign information storage unit. 4. The image information adding system according to claim 1, wherein the object imaged by said image input means is compared with the processed object taken out from said information storage means and processed to compare the object. An image information adding system characterized by judging whether or not there is a difference between them. 5. An image input means for picking up an image of an object, an optical information means for obtaining an angle of view from a focal length at the time of photographing and an image cutout size from information of the image input means, and a photographer based on the angle of view. It has a function to calculate the position and shooting direction, and has a pair of symbols with predetermined dimensions in the horizontal and vertical directions from the photographed image of the pair of symbols in the marking means installed at a specific place. A position / direction calculation means for calculating a photographer position and a photographing direction, which are position information of the photographer, background image information in the same direction as the direction in which the sign means is imaged, and incidental information accompanying the background image are stored. A communication control unit that sends and receives information to and from the information storage unit, and processes background image information obtained from the information storage unit based on the position information to obtain background image information of the same field of view. An information processing circuit for acquiring the accompanying information of the background image, a comparison and synthesis circuit for comparing and synthesizing an image that has passed through the signal processing circuit from the image input means and a processed image that has passed through the information processing circuit, and the comparison. An image information adding device, comprising: an image display means for displaying an image from a synthesizing circuit on a screen.