JP6723018B2 - Object recording device and object recording program - Google Patents

Description

The present invention relates to an object recording device and an object recording program.

Conventionally, in order to manage roads and rivers, a system has been proposed in which image data, attribute data such as shooting time and shooting location, map data, etc. are stored in a management server or displayed. ..

For example, U.S. Pat. No. 6,037,049 discloses a mobile device in which the orientation of the mobile device relative to a target is based at least in part on one or more signals received from at least one sensor supported by the mobile device. Detecting, displaying a video image captured in a camera view of the mobile device while in the orientation, and the target at least partially obscured in the orientation and the camera view. Overlaying one or more visual indicators on the displayed video image based at least in part on the difference between and.

Further, in Patent Document 2 below, information acquisition means for acquiring position information of a target object, shooting position information, shooting direction information, and shooting angle of view information of a camera, and a target object obtained by this information acquisition means. Calculating means for calculating the position of the target object on the photographed image obtained by the camera based on the position information of the camera, the photographing position information of the camera, the photographing direction information and the photographing angle of view information, and the calculating means. A display unit for displaying information indicating the calculated position of the target object in the photographed image; and a judgment unit for judging whether or not the target object is within the photographing angle of view of the camera, The display unit displays the information indicating the direction in which the target is located when the determination unit determines that the target is not within the photographing angle of view of the camera. Is disclosed.

Special table 2014-505250 gazette Japanese Patent No. 3729161

However, in the above-mentioned conventional technique, although an image of an object (subject) such as a road or a river is displayed on the display screen, a description of the object (subject) and other related information (visual In the case of associating an indicator, information indicating a certain direction of a target object, etc., and superimposing and displaying the image on the image, the related information does not have coordinate information and can be displayed only on the image. Therefore, the related information cannot be superimposed and displayed on another image of the target taken from another direction.

An object of the present invention is to provide an object recording device and an object recording program capable of adding and recording information for identifying the related information to the related information of the object to be photographed and displayed. ..

In order to achieve the above object, one embodiment of the present invention is a target object recording apparatus, which is a shooting position determining unit that determines a shooting position of a target object that is being displayed by shooting or playing back, and displays by shooting or playing back. Direction detecting means for detecting a direction of the inside object from the photographing position, and determining a length of a vector pointing in the direction from the photographing position as a starting point based on a predetermined reference and generating the object vector And a related information setting means for setting related information of the target object, and a related information storage means for storing the related information in association with the target object vector.

The object recording apparatus has a coordinate axis, and sets a virtual three-dimensional space in which a starting point is a photographing position of the object and an end point of an object vector facing the object is associated with coordinates on the coordinate axis. Dimensional space setting means, display range specifying means for specifying a display range in the virtual three-dimensional space based on the shooting position and the shooting azimuth, tilt and angle of view of the camera used for shooting, and the virtual three-dimensional space. Related information specifying means for specifying the related information associated with the target object vector whose end point is included in the display range, and display means for displaying the specified related information by superimposing it on the image being displayed. It is preferable to further include

Further, the vector generation means, when the depression angle in the direction is equal to or more than a predetermined reference angle, the intersection point calculated from the straight line drawn from the starting point in the direction of the object vector and the equation of the preset horizon It is preferable to determine the length of the object vector based on the coordinates.

The vector generating means may determine the length of the object vector based on the height of the installation position of the camera when the depression angle in the direction is smaller than a predetermined reference angle.

Further, the object recording device corrects the length of the object vector based on a new object vector for the same object generated by the vector generating means on an image captured or reproduced from another direction. You may further provide the vector correction means to do.

Further, in the object recording apparatus, the display means sets the coordinates of the end point of the object vector in the virtual three-dimensional space as the coordinates of the related information and displays the projection point of the coordinates on a preset horizon. It may further include vector correction means for displaying and changing the length and/or direction of the object vector associated with the relevant information when the position of the projection point is changed via the input means.

It is preferable that the target object recording device is configured on a tablet terminal having a photographing function and a display function of a photographed image.

Another embodiment of the present invention is an object recording program, wherein a computer is used to determine a shooting position of a target object being displayed by shooting or reproducing, a shooting position determining unit, and a target object being displayed by shooting or playing. Direction detecting means for detecting a direction from the photographing position, vector generating means for determining the length of a vector pointing in the direction based on a predetermined reference, starting from the photographing position, and generating as an object vector, It is characterized by functioning as related information setting means for setting related information of an object, and related information storage means for storing the related information in association with the object vector.

Further, the object recording program has a computer, which has a coordinate axis, and a virtual three-dimensional space in which an end point of an object vector having a coordinate of a photographing position of the object as a starting point and facing the direction of the object is associated with coordinates on the coordinate axis. Virtual three-dimensional space setting means, a display range specifying means for specifying a display range in the virtual three-dimensional space based on the shooting position and the shooting direction, tilt and angle of view of the camera used for shooting, and the virtual Related information specifying means for specifying the related information associated with the object vector whose end point is included in the display range in the three-dimensional space, and display means for displaying the specified related information by superimposing it on the image being displayed. , And further function as.

According to the present invention, the information for identifying the relevant information is added to the relevant information of the object to be photographed and displayed and recorded, and the information is also superimposed on another image obtained by photographing the object from another direction. Can be displayed.

It is explanatory drawing of the example of the target recording device concerning embodiment. It is a functional block diagram of the object recording device concerning an embodiment. FIG. 3 is an explanatory diagram of a direction from a shooting position of an object detected by a direction detection unit according to the embodiment. It is explanatory drawing of a virtual three-dimensional space. It is explanatory drawing of the 1st correction and 2nd correction concerning embodiment. It is explanatory drawing of the usage example of the target recording device concerning embodiment. It is a flowchart of an operation example of the object recording device according to the embodiment. FIG. 8 is a flowchart of another operation example of the object recording device according to the embodiment.

Hereinafter, modes for carrying out the present invention (hereinafter, referred to as embodiments) will be described with reference to the drawings.

1(a) and 1(b) are explanatory views of the object recording apparatus according to the embodiment. In FIG. 1A, a photographing/display terminal 10 such as a tablet terminal having a photographing function and a reproduction/display function of a photographed image has a display screen 12 for displaying an image on the front side and a photographing target (object) on the back side. A camera 14 (see FIG. 1B) for shooting is provided.

The camera 14 does not have to be a stereo camera, and can be configured with a single camera (imaging device) as long as it can capture a moving image. The camera 14 can be composed of an image pickup device using a CCD, a CMOS or the like.

The photographing/display terminal 10 photographs the direction of the arrow A with the camera 14 and displays the image on the display screen 12 while photographing an image including a road, a river, a building and other objects (imaging objects). Alternatively, it is possible to store an image including an object photographed in the past and reproduce and display it on the display screen 12. The data of images taken in the past may be stored in an external server or the like, and the shooting/display terminal 10 may acquire the data via the Internet line or another appropriate communication line and reproduce and display the data. The shooting/display terminal 10 according to the present exemplary embodiment displays a target when displaying an image including an object being shot by the shooting/display terminal 10 or when playing back and displaying an image shot in the past. A process of setting related information related to an object and superimposing it on an image including an object is executed. A method of setting related information and a display example will be described later.

Note that the shooting/display terminal 10 is not limited to a tablet terminal, and a shooting function, a display function of a shot image, a function of communicating with an external server using an internet line or a telephone line, and a shooting direction. As long as it has a function of detecting, a function of detecting one's own inclination using a gyro, a function of specifying one's position using GPS, etc., it can be used as the photographing/display terminal 10.

Furthermore, MMS (Mobile Mapping System (Mobile Mapping System)), which is a vehicle-mounted laser measuring device, is used as a photographing device (camera 14), and the acquired single photograph or the entire circumference image is used by a computer or the like. May be configured to be displayed on an appropriate screen (display screen 12).

The related information includes, for example, past repair points of roads and peripheral equipment, damage status, identification codes of roads and rivers (national road numbers, river codes, etc.), positions of kiloposts set on roads and rivers, roads. Examples include, but are not limited to, names of surrounding buildings, facilities occupying the buildings, prefectures, municipalities, lot numbers, and the like. Any information related to roads, rivers, buildings and other objects can be set as related information.

When setting the related information, in order to specify the display position of the related information, an object vector representing the position of the object related to the related information is set. The method of setting the object vector will be described later.

FIG. 2 shows a functional block diagram of an example of the object recording apparatus according to the embodiment. In FIG. 2, the object information display device includes a photographing position determination unit 16, a direction detection unit 18, a vector generation unit 20, a related information setting unit 22, a virtual three-dimensional space setting unit 24, a display range specifying unit 26, and related information specification. It is configured to include a unit 28, a display control unit 30, a vector correction unit 32, a communication unit 34, a storage unit 36, and a CPU 38. The object recording device includes a CPU 38, a ROM, a RAM, a non-volatile memory, an I/O, a communication interface, and the like, and is configured as a computer that controls the entire device and performs various calculations. And a program that controls the processing operation of the CPU 38.

The shooting position determination unit 16 determines the shooting position of the shooting target (target object) being displayed (at the same time as shooting) or by being played back. That is, the coordinates of the shooting position (the position of the camera 14) when the camera 14 of the shooting/display terminal 10 is shooting an object are determined. The object that determines the shooting position is an element of the image (road, building, etc.) designated by the user with an appropriate input means (pointing device such as mouse, keyboard, touch panel, etc.) in the image displayed on the display screen 12. Are all elements displayed as images, such as buildings, natural objects such as mountains and trees). The shooting position determination unit 16 is configured to specify the coordinates of the shooting position based on, for example, the position information acquired by the GPS receiver during shooting or the position information acquired by the GPS receiver included in the image data being reproduced. You can An IMU (inertial measurement device) may be combined with the GPS receiver. Examples of the coordinates of the photographing position include, but are not limited to, three-dimensional coordinates such as latitude, longitude, and altitude. The coordinates of the specified shooting position are stored in the storage unit 36.

The direction detection unit 18 includes a shooting direction detection unit 18a and a tilt detection unit 18b, and detects the direction from the shooting position of the target object being displayed during shooting or playback.

The shooting direction detection unit 18a detects a shooting direction (direction of the optical axis of the camera 14) when the camera 14 of the shooting/display terminal 10 is shooting a shooting target. The shooting direction can be detected by, for example, a magnetic sensor. That is, the shooting direction detection unit 18a receives the output from the magnetic sensor and calculates the shooting direction based on the mounting direction of the magnetic sensor, the optical axis direction of the camera 14, and the like. The calculated shooting direction is stored in the storage unit 36.

The tilt detecting unit 18b tilts the camera 14 (shooting screen) while the camera 14 of the shooting/display terminal 10 is shooting a shooting target, that is, the camera 14 rotates about an optical axis (rolls) and is orthogonal to the optical axis. The angle of rotation (pitching) around the horizontal axis (horizontal axis extending to the left and right when the shooting direction is the front when there is no rolling and tilting by that angle when there is rolling) is detected. Note that rolling and pitching are shown in FIG. The tilt of the camera 14 is obtained by, for example, the tilt detector 18b receiving an output from the acceleration sensor and calculating the tilt. The storage unit 36 stores the tilt of the camera 14 calculated by the tilt detection unit 18b.

The direction detection unit 18 stores the direction from the photographing position of the target object displayed on the display screen 12, which is designated by the user by an appropriate input unit (pointing device such as mouse, keyboard, touch panel, etc.), in the storage unit 36. It is detected based on the shooting direction, the tilt, and the angle of view of the camera 14 read from the. The above-mentioned direction detected by the direction detection unit 18 is stored in the storage unit 36. Note that the angle of view represents the range of the scene photographed by the camera 14 in terms of angles, and the horizontal angle of view, the vertical angle of view, and the diagonal angle of view are known in advance as values unique to the camera 14. It is stored in the storage unit 36, and the direction detection unit 18 reads out from the storage unit 36 and uses it.

Here, the position of the target object whose direction from the shooting position is detected can be a pixel or a pixel region (a region formed by a set of continuous pixels) included in the image of the target object. Reference numeral 18 detects the direction of the pixel or pixel area from the shooting position. The pixel or pixel region is a pixel or pixel region as designation information input by the input unit when the user designates an object on the screen by the input unit. The designation information can be, for example, a pixel or a pixel area included in the image of the object, which is designated by a touch panel or a pointing device. Further, a configuration may be used in which the above-mentioned pixel or pixel area is designated by a keyboard.

The direction specifying unit 18 receives the input designation information, and determines which pixel or pixel region on the display screen 12 the pixel or pixel region designated by the input means is. When the designation information is the pixel area, the direction detection unit 18 sets in advance the number of pixels forming the pixel area, the pixel continuation method, the central pixel for detecting the direction, and the like. The direction detection process is executed based on this setting. The pixel serving as the center for detecting the direction is a pixel in which the direction from the shooting position to the center pixel is the direction detected by the direction detecting unit 18.

3(a) and 3(b) are explanatory views of the direction from the photographing position of the object detected by the direction detection unit 18. 3A is a projection view on the XY plane in the above direction, and FIG. 3B is a projection view on a plane W parallel to a straight line facing the above direction and orthogonal to the XY plane. Although XYZ represents coordinate axes, the coordinate axes are not limited to this, and arbitrary coordinate axes can be used.

In FIG. 3A, the shooting position of the object P (position of the camera 14) determined by the shooting position determination unit 16 is indicated by C. Further, the angle of (the projection on the XY plane of) the photographing range of the camera 14 is indicated by α. The direction from the shooting position C to the object P shown in FIG. 3A corresponds to the direction projected on the XY plane, that is, the azimuth. Further, in FIG. 3B, the direction from the photographing position C to the object P corresponds to the elevation angle or the depression angle.

Each point (pixel) in the image captured by the camera 14 and displayed on the display screen 12 has a direction as a combination of the azimuth and elevation/depression angle (elevation angle or depression angle) shown in FIGS. 3(a) and 3(b). And one-to-one correspondence. The direction detecting unit 18 detects, with respect to the pixel or pixel region in the target object designated by the user, the shooting direction detected by the shooting direction detecting unit 18a and the rolling and pitching of the camera 14 detected by the tilt detecting unit 18b. It can be calculated from the angle of view of the camera 14 by a known method. The elevation/depression angle can also be calculated by a known method from the rolling and pitching of the camera 14 detected by the tilt detection unit 18b and the angle of view of the camera 14.

The vector generation unit 20 uses the shooting position determined by the shooting position determination unit 16 as a starting point, and the length of the vector pointing in the direction detected by the direction detection unit 18 for the object specified by the user is based on a predetermined reference. Determined as a target object vector. Here, the predetermined criteria are as follows. First, when the depression angle in the above direction is equal to or greater than the predetermined reference angle, the target is calculated based on the coordinates of the intersection calculated from the straight line drawn from the starting point in the direction of the object vector and the preset equation of the ground plane. Determine the length of the object vector. That is, the distance between the starting point and the intersection is set as the length of the object vector. In addition, the intersection point becomes the end point of the object vector. The predetermined reference angle can be set from an empirical value, but can be, for example, 20 to 40 degrees, and is preferably 30 degrees. The vector generation unit 20 also stores the generated object vector in the storage unit 36.

If the depression angle in the direction is smaller than the predetermined reference angle (including the case of horizontal and elevation angles), the length of the object vector is determined based on the height of the installation position of the camera 14. In this case, the length of the object vector can be set to, for example, twice the height of the installation position of the camera 14, but the length is not limited to this. For example, a configuration may be set in which the coefficient by which the height of the installation position is multiplied can be set for each type of the camera 14 (a tablet terminal that the user holds by hand for shooting, a vehicle-mounted MMS, etc.). In this case, a point having the length determined above from the starting point of the object vector becomes the ending point of the object vector.

The related information setting unit 22 sets the related information of the object. The content of the related information is as described above. The related information setting unit 22 accepts the designation of the target object performed by the user with an appropriate input means (pointing device such as a mouse, keyboard, touch panel, etc.) on the screen, and inputs the related information about the designated target object. Can be configured to display an input screen for. For example, when the user inputs designation information for designating an object from the input means, the related information setting unit 22 receives the designation information. The designation information can be, for example, a pixel or a pixel area included in the image of the object, which is designated by a touch panel or a pointing device. Further, a configuration may be used in which the above-mentioned pixel or pixel area is designated by a keyboard. The related information setting unit 22 sets the input screen in association with the pixel or the pixel area that is the designated information, and the display control unit 30 displays the input screen on the display screen. When the user inputs the content of the related information regarding the object on the input screen, the related information setting unit 22 receives the related information and stores it in the storage unit 36. The related information setting unit 22 also sets a graphic (hereinafter sometimes referred to as a tag) for displaying such related information. It is preferable that this tag includes characters, figures, etc. for displaying the content of the related information.

Since the object specifying process is the same as the case of the direction detecting unit 18, the direction detecting process by the direction detecting unit 18 and the related information setting process by the related information setting unit 22 are performed simultaneously or continuously. You may execute it.

Next, the related information setting unit 22 reads the target object vector generated by the vector generation unit 20 for the specified target object from the storage unit 36, and stores the target object vector in the storage unit 36 in association with the related information of the corresponding target object. .. In this case, the related information setting unit 22 and the storage unit 36 function as a related information storage unit that stores the related information associated with the object vector.

It is preferable that the related information of the target object set by the related information setting unit 22 is displayed in the vicinity of the target object, for example, at a position where the end point of the target object vector is included in the figure of the tag.

The virtual three-dimensional space setting unit 24 sets a virtual three-dimensional space including the target object vector. Here, the virtual three-dimensional space is a virtual space, that is, a space set on a computer, having coordinate axes representing coordinates corresponding to the coordinates (for example, latitude, longitude, and altitude) of the real space. Further, the virtual three-dimensional space setting unit 24 associates the target object vector with the coordinates in the virtual three-dimensional space based on the coordinate information that the target object vector has as an attribute. The coordinate information that the object vector has as an attribute can be, for example, the coordinates of the end point of the object vector or the coordinates of the start point and the end point. Further, the coordinate axes in the virtual three-dimensional space and the coordinate axes representing the coordinates of the shooting position determined by the shooting position determination unit 16 may be the same (for example, unified in latitude, longitude, or altitude), or different coordinate axes may be used. Each may be set and the coordinates may be converted as necessary. The virtual three-dimensional space setting unit 24 stores the information about the virtual three-dimensional space such as the coordinate axis of the virtual three-dimensional space and the object vector associated with the coordinates in the virtual three-dimensional space in the storage unit 36.

FIG. 4 shows an explanatory diagram of the virtual three-dimensional space. In FIG. 4, coordinate axes α1, α2, and α3 are defined in the virtual three-dimensional space set on the computer. This coordinate axis can be set arbitrarily, and as described above, for example, may be (X, Y, Z) coordinates (mathematical coordinate system), (latitude, longitude, elevation) coordinates (earth coordinate system), or the like.

In the virtual three-dimensional space, an object vector Vp having coordinate information (for example, coordinates of an end point or coordinates of a start point and an end point) as attributes, and coordinate axes α1, α2, α3 in the virtual three-dimensional space based on the coordinate information. It is associated with the coordinates it represents. Here, “associated with the coordinate represented by the coordinate axis” means that the object vector Vp is arranged at a position corresponding to the coordinate information of the object vector Vp based on the coordinate axes α1, α2, and α3. Say that. The related information associated with the target object vector Vp is also included in the virtual three-dimensional space. In the example of FIG. 4, the related information is displayed as “tag” and is arranged at a position including the end point of each object vector Vp.

The display range specifying unit 26 is used for the tilt of the camera 14 used for shooting and the shooting position determined by the shooting position determination unit 16, the shooting direction detected by the shooting direction detection unit 18a, and the tilt detected by the tilt detection unit 18b. The display range in the virtual three-dimensional space is specified based on the angle of view of the camera 14. The above-mentioned shooting position, shooting direction, tilt and angle of view of the camera 14 are read from the storage unit 36 and used by the display range specifying unit 26. The display range specifying unit 26 sets the shooting position, shooting direction, and tilt of the camera 14 used for shooting when setting a screen (see FIG. 4) for perspective projection of a virtual three-dimensional space as a known computer graphics process. And the size and direction of the screen (direction in which the normal line of the screen faces) are determined based on the angle of view of the camera 14. The display range is a range in which the virtual three-dimensional space is perspectively projected on the screen. The display range in this case is the display range when viewed from the shooting position. The coordinates of the photographing position include plane coordinates such as latitude and longitude and the altitude described above. The altitude may be the altitude value determined by the photographing position determination unit 16, or may be a preset altitude value if there is a large error in the detected value. The specified display range, that is, the size and direction of the screen is stored in the storage unit 36, and the display range specifying unit 26 displays the display screen 12 on the display screen 12.

Since the image acquired by the camera 14 can also be specified based on the shooting position, the shooting direction, the tilt and the angle of view of the camera 14, the object vector and the like associated with the virtual three-dimensional space are It is possible to superimpose and display via the display range specified by the display range specifying unit 26. The display control unit 30 performs such superimposed display processing.

In FIG. 4, the display range is used for shooting position (for example, the viewpoint of the camera 14 determined by latitude, longitude, and altitude, represented by (x, y, z) in the example of FIG. 4), shooting direction, and shooting. The display range specifying unit 26 specifies the tilt of the camera 14 and the angle of view of the camera 14. In the example of FIG. 4, the display range is shown as a rectangular area. As described above, this display range is set as a screen for perspective projection of the virtual three-dimensional space, and the size of the screen is determined based on the shooting position, the shooting direction, the tilt of the camera 14 used for shooting, and the angle of view of the camera 14. Size and direction are determined.

The related information specifying unit 28 specifies the related information associated with the object vector whose end point is included in the display range in the virtual three-dimensional space specified by the display range specifying unit 26. This processing is performed by, for example, an object vector whose end point is perspectively projected on the screen, that is, plane coordinates such as latitude and longitude included in a range (display range) of the virtual three-dimensional space perspectively projected on the screen as coordinates of the end point, and This is performed by specifying an object vector having a height (elevation) as an object vector in the shooting range. By specifying the target object vector, the related information specifying unit 28 can also specify the related information associated with the target object vector. The target object vector generated in advance may be configured to communicate with an external server by the communication unit 34 and be acquired from the server. In this case, during the shooting of the shooting target, the coordinates of the shooting position determined by the shooting position determination unit 16 are transmitted to the server, and the coordinates included in the fixed range (the fixed range wider than the shooting range) including the transmitted coordinates are set. The object vector having the coordinates of the end point can be returned from the server. The object vector returned from the server is stored in the storage unit 36. Further, the object vector may not be received from the server during shooting, but the object vector having the coordinates of the end point included in the area that is the candidate of the shooting range may be stored in the storage unit 36 in advance.

In FIG. 4, the related information specifying unit 28 determines that the coordinates of the end point (plane coordinates such as latitude and longitude and/or height (elevation)) are in the range of the virtual three-dimensional space perspectively projected on the screen indicating the display range. Identify the included object vector. Next, the related information specifying unit 28 reads out the related information related to the specified target object vector from the storage unit 36 and causes the display control unit 30 to display the related information on the display screen 12 as a tag. In the example of FIG. 4, the tags I and II associated with the object vector whose display range includes the coordinates of the end point are displayed on the display screen 12, and are associated with the object vector whose display range does not include the coordinates of the end point. Tag III is not displayed.

The display control unit 30 displays the image being captured or reproduced by the camera 14 of the photographing/display terminal 10 on the display screen 12 and is associated with the image to the object vector specified by the related information specifying unit 28. The related information is read from the storage unit 36, superimposed, and displayed on the display screen 12. In this case, it is preferable to display the related information such that a part of the image of the related information overlaps with the end point of the object vector (the point is included in the image of the related information). In addition, the object vector may or may not be displayed. It is preferable that the display be switched between display and non-display based on an instruction input by the user.

By the operations of the display range specifying unit 26, the related information specifying unit 28, and the display control unit 30 described above, when an object photographed at a certain photographing position C1 is photographed from another photographing position C2, the certain photographing position C1 It is possible to display the related information of the target object set in the image captured in step 1 on the target object in the image captured from the other capturing position C2 via the virtual three-dimensional space. This is because the same target exists in the display range when viewed from the certain shooting position C1 and the other shooting position C2 specified by the display range specifying unit 26, and the target vector associated with the virtual three-dimensional space. This is because the same related information is specified and displayed via.

The vector correction unit 32 corrects the length of the target object vector based on a new target object vector for the same target object generated by the vector generation unit 20 on an image captured or reproduced from another direction. This correction process is hereinafter referred to as the first correction.

In the vector correction unit 32, the display control unit 30 sets the coordinates of the end point of the object vector in the virtual three-dimensional space as the coordinates of the related information on the display screen 12 in advance in the virtual three-dimensional space of these coordinates. When the position of the projection point is changed via the input means while displaying the projection point on the horizontal plane, the length and/or direction of the object vector associated with the related information is corrected. To do. This correction process is hereinafter referred to as the second correction.

5(a) and 5(b) are explanatory diagrams of the first correction and the second correction. FIG. 5A is an example of the first correction, and FIG. 5B is an example of the second correction.

FIG. 5A shows an object P designated by the user when the object 14 is photographed by the camera 14 from a certain point I on the road R and the image is displayed on the display screen 12, for example. An object vector Vp1 of length L1 starting from the point I generated by the vector generation unit 20 is shown. Next, when the same object P is photographed by the camera 14 from the point II different from the point I and the image is displayed on the display screen 12, when the user specifies the object, the vector The generation unit 20 generates the object vector Vp2 having the length L2 starting from the point II. In this case, if the lengths L1 and L2 and the directions of the two object vectors Vp1 and Vp2 for the same object P are set correctly, the end points E1 and E2 of the object vectors Vp1 and Vp2 are the object P. It should match in position. However, if there is an error in the length or the direction, as shown in FIG. 5A, the end points E1 and E2 of the object vectors Vp1 and Vp2 are displaced. Therefore, the vector correction unit 32 sets a surface F including the object vector Vp2 generated later in time (for example, a surface orthogonal to the road is mentioned, but is not limited thereto), and the surface F is set. And an intersection point C between the object vector Vp1 and the object vector Vp1 set earlier in time are obtained, and the length of the starting point of the object vector Vp1 and this intersection point C becomes the new length Lc of the object vector Vp1. The vector Vp1 is corrected (first correction). In addition, a plane including the object vector Vp1 set earlier in time is set, an intersection of this plane and the object vector Vp2 generated later in time is obtained, and the start point of the object vector Vp2 and this intersection The object vector Vp2 may be corrected so that the length becomes a new length of the object vector Vp2.

FIG. 5B shows an object P designated by the user when the object P is photographed by the camera 14 from a certain point I on the road R and the image is displayed on the display screen 12, for example. An object vector Vp1 of length L1 starting from the point I generated by the vector generation unit 20 is shown. Further, the related information Rp of the object set by the related information setting unit 22 is displayed on the display screen 12. At this time, the vector correction unit 32 instructs the display control unit 30 to project the projection point of the coordinates of the end point of the object vector Vp1 onto the ground plane previously set in the virtual three-dimensional space at the projection point S1 at the position of the related information Rp. To be displayed. In the example of FIG. 5B, it is assumed that the projection point S1 is out of the road R and the user determines that it is inappropriate as the position of the related information of the target object (not shown). In this case, when the user moves the position of the projection point S1 on the road R through an appropriate input means (a pointing device such as a mouse, a keyboard, a touch panel, etc.), it corresponds to the projection point S2 after the movement. The length of the object vector is corrected to the length (L2) (second correction) to obtain the corrected object vector Vp2. Here, the "length corresponding to the projection point S2" is a point on the object vector Vp1, and the point at which the projection point on the horizon coincides with the projection point S2 after the movement and the object. It is the length of the starting point of the vector Vp1.

Note that the second correction includes not only the correction for shortening the length of the target object vector shown in the example of FIG. 5B, but also the correction for increasing the length. Further, the correction is not limited to the change of the length, and may be the correction of changing the direction of the object vector by moving the projection point.

As a result of the first correction and the second correction described above, the vector correction unit 32 stores the information of the object vector whose length and/or direction has been changed in the storage unit 36.

The communication unit 34 is configured by an appropriate interface, and is used by the CPU 38 for exchanging data (relevant information of an object to be photographed) with an external server or the like via a wireless or wired communication line.

The storage unit 36 is composed of a hard disk device, a non-volatile memory such as a solid state drive (SSD), etc., and stores the above-mentioned various information and the information necessary for each processing performed by the object recording device, such as the operation program of the CPU 38. Let The storage unit 36 includes a digital versatile disc (DVD), a compact disc (CD), a magneto-optical disc (MO), a flexible disc (FD), a magnetic tape, an electrically erasable and rewritable read-only memory ( EEPROM), flash memory, etc. may be used. The storage unit 36 also includes a random access memory (RAM) that mainly functions as a work area of the CPU 38, and a read-only memory (ROM) that stores control programs such as BIOS and other data used by the CPU 38. It is suitable.

The function of the storage unit 36 may be set in an external server.

6(a) and 6(b) are explanatory views of a usage example of the object recording apparatus according to the embodiment. In the example shown in FIGS. 6A and 6B, the MMS 102 (imaging device) that is the laser measuring device mounted on the vehicle 100 corresponds to the camera 14, and an appropriate reproducing device 104 such as a computer displays a display screen. Equivalent to 12. Further, the photographing position is acquired by GPS, and the photographing direction and the tilt of the camera are acquired by IMU.

In FIG. 6A, while the vehicle 100 is traveling on the road R, the MMS 102 takes a single-photograph image of the road and the road periphery. At this time, the object (a street light in this example) P is also photographed. In FIG. 6A, the object P is photographed at three photographing positions I, II, and III on the road R. Note that, in the example of FIG. 6A, the distances between the shooting positions I, II, and III are described as being far apart, but as in the case of moving image shooting, the shooting positions I, II, and III are continuously spaced at a short time or distance. It is also possible to adopt a configuration for photographing.

In FIG. 6B, the image acquired in FIG. 6A is reproduced and displayed on the display screen of an appropriate reproducing device 104 such as a computer. I, II, and III in the drawing indicate three points (imaging positions) on the road R where the object P is imaged.

In FIG. 6B, when the user specifies the object P from an appropriate input means during image reproduction and display, the vector generation unit 20 generates the object vector for the object P. In FIG. 6B, the target object vector set at the shooting position I is represented by Vp1, and the target object vector set at the shooting position II is represented by Vp2. The target vector Vp2 is associated with the related information Rp set by the related information setting unit 22 for the object P and displayed on the display screen 12. In this case, the object vectors such as Vp1 and Vp2 are not displayed on the display screen 12.

Here, for example, when the image at the shooting position III is reproduced and displayed, the related information Rp associated with the object vector Vp2 set at the shooting position II is displayed. In this processing, the display range specifying unit 26 specifies the display range in the virtual three-dimensional space from the photographing position III, and the related information specifying unit 28 sets the coordinates of the end point of the object vector Vp2 in the virtual three-dimensional space in this display range. It is determined whether or not it exists, and if it exists, the display control unit 30 reads out the related information Rp associated with the object vector Vp2 from the storage unit 36, superimposes it on the image being reproduced and displayed, and displays it on the display screen 12. This is done by displaying.

Note that the related information Rp can also be displayed at the shooting position I by the same processing as in the case of the shooting position III.

Further, the length of the object vector Vp2 can be corrected (first correction) by the object vector Vp1, for example. In this process, the vector correction unit 32 sets a surface that includes the target object vector Vp1 and is orthogonal to the road R, obtains the intersection of this surface and the target object vector Vp2, and determines the start point of the target object vector Vp2 (imaging position II. ) And this intersection point are corrected by correcting the object vector Vp2 so that the object vector Vp2 has a new length. By this correction, the length of the object vector Vp2, that is, the coordinates of the end point is changed, so the display position of the related information Rp associated with this on the display screen 12 is also changed. The display control unit 30 performs the display position changing process.

In the example of FIG. 6B, the length of the target object vector Vp2 is corrected by the target object vector Vp1, but the length of the target object vector Vp1 may be corrected by the target object vector Vp2. In this case, the related information Rp is associated with the object vector Vp1.

Further, although the example of the first correction is shown in the example of FIG. 6B, the above-described second correction can be executed.

Further, in the example of FIG. 6B, the object vector Vp3 having the shooting position III as the starting point is also shown. The target object vector Vp3 is a target object vector whose target is the damaged portion B on the road R. Based on the instruction input by the user, the related information setting unit 22 can set and relate the related information to the object vector Vp3.

Further, in the example of FIG. 6A, an example in which a single photograph image of the road and the road periphery is taken by the MMS 102 is shown, but it may be an all-surrounding image. Alternatively, the above-described processing may be performed using a tablet terminal or the like while performing shooting and playback display at the same time.

FIG. 7 shows a flow of an operation example of the object recording apparatus according to the embodiment. In FIG. 7, the shooting position determination unit 16 determines the shooting position of the target object (specified by the user) that is being displayed at the same time as shooting, or is being displayed by reproduction (S1).

Next, the direction detection unit 18 displays the object (pixels or pixel areas included in the image of the object) which is being displayed at the same time as the shooting or is being displayed by the reproduction and which is designated by the user from the shooting position. The direction of is detected (S2).

The vector generation unit 20 uses the shooting position determined by the shooting position determination unit 16 as a starting point, and sets the length of the vector pointing in the direction detected by the direction detection unit 18 with respect to the target object designated by the user as described above. It is determined based on the standard and is generated as an object vector (S3).

Next, the related information setting unit 22 sets the related information of the object (S4). In this process, as described above, the display control unit 30 displays the input screen set in association with the target object (pixels or pixel regions included in the image of the target object) specified by the user from the input unit. This is executed by the related information setting unit 22 receiving and displaying the content of the related information regarding the object displayed and input by the user on the input screen. The related information received by the related information setting unit 22 is stored in the storage unit 36 (S5).

FIG. 8 shows a flow of another operation example of the object recording apparatus according to the embodiment. In FIG. 8, first, the virtual three-dimensional space setting unit 24 sets a virtual three-dimensional space including the target object vector (S11).

Next, the display range specifying unit 26 specifies the display range in the virtual three-dimensional space based on the shooting position, the shooting direction, the tilt of the camera 14, and the angle of view of the camera (S12).

The related information specifying unit 28 specifies the related information related to the object vector whose end point is included in the display range in the virtual three-dimensional space specified by the display range specifying unit 26 (S13), and the display control unit 30 , The image being photographed or reproduced by the camera 14 of the photographing/display terminal 10 is displayed on the display screen 12, and the relevant information associated with the object vector specified by the relevant information specifying unit 28 is stored in the storage unit. It is read from 36, superposed, and displayed on the display screen 12 (S14).

The above-described program for executing the steps of FIGS. 7 and 8 can be stored in a recording medium, or the program may be provided by communication means. In that case, for example, the program described above may be regarded as an invention of a “computer-readable recording medium recording a program” or an invention of a “data signal”.

10 shooting/display terminal, 12 display screen, 14 camera, 16 shooting position determination unit, 18 direction detection unit, 20 vector generation unit, 22 related information setting unit, 24 virtual three-dimensional space setting unit, 26 display range specifying unit, 28 Related information specifying unit, 30 display control unit, 32 vector correction unit, 34 communication unit, 36 storage unit, 38 CPU, 100 vehicle, 102 MMS, 104 playback device.

Claims (7)

Photographing position determining means for determining the photographing position of the object being displayed by photographing or reproducing,
Direction detecting means for detecting the direction of the object being displayed from the photographing position by photographing or reproducing,
A vector generation unit that determines the length of a vector pointing in the direction based on a predetermined reference, starting from the shooting position, and generates the target vector.
Related information setting means for setting related information of the object,
Related information storage means for storing the related information in association with the object vector,
A virtual three-dimensional space setting unit that has a coordinate axis and sets a virtual three-dimensional space in which the end point of the object vector starting from the photographing position and pointing in the direction is associated with the coordinate on the coordinate axis.
Display range specifying means for specifying a display range in the virtual three-dimensional space based on the shooting position and the shooting direction, tilt, and angle of view of the camera used for shooting;
Related information specifying means for specifying the related information associated with the object vector whose end point is included in the display range in the virtual three-dimensional space;
Display means for displaying the specified related information by superimposing it on the image being displayed,
Equipped with
The vector generation means, when the depression angle in the direction is equal to or greater than a predetermined reference angle, the coordinates of the intersection calculated from the straight line drawn from the starting point in the direction of the object vector and the preset equation of the ground plane are set. An object recording device that determines the length of the object vector based on the above . The object according to claim 1, wherein the vector generation unit determines the length of the object vector based on the height of the installation position of the camera when the depression angle in the direction is smaller than a predetermined reference angle. Object recording device. Further comprising a vector correction means for correcting the length of the object vector based on a new object vector for the same object generated by the vector generation means on an image photographed or reproduced from another direction. Item 1. The object recording apparatus according to Item 1 or 2 . Photographing position determining means for determining the photographing position of the object being displayed by photographing or reproducing,
Direction detecting means for detecting the direction of the object being displayed from the photographing position by photographing or reproducing,
A vector generation unit that determines the length of a vector pointing in the direction based on a predetermined reference, starting from the shooting position, and generates the target vector.
Related information setting means for setting related information of the object,
Related information storage means for storing the related information in association with the object vector,
A virtual three-dimensional space setting unit that has a coordinate axis and sets a virtual three-dimensional space in which the end point of the object vector starting from the photographing position and pointing in the direction is associated with the coordinate on the coordinate axis.
Display range specifying means for specifying a display range in the virtual three-dimensional space based on the shooting position and the shooting direction, tilt, and angle of view of the camera used for shooting;
Related information specifying means for specifying the related information associated with the object vector whose end point is included in the display range in the virtual three-dimensional space;
Display means for displaying the specified related information by superimposing it on the image being displayed,
The display means displays the projection point of the coordinates on the preset horizon, using the coordinates of the end point of the object vector in the virtual three-dimensional space as the coordinates of the related information, and the above-mentioned via the input means. Vector correction means for correcting the length and/or direction of the object vector associated with the related information when the position of the projection point is changed ,
Bei El, the object recording apparatus. The object recording apparatus according to any one of claims 1 to 4 , which is configured on a tablet terminal having a photographing function and a display function of a photographed image. Computer,
Photographing position determining means for determining the photographing position of the object being displayed by photographing or reproducing,
Direction detecting means for detecting the direction of the object being displayed from the photographing position by photographing or reproducing,
A vector generation unit that determines the length of a vector pointing in the direction based on a predetermined reference by using the shooting position as a starting point and generates the target vector.
Related information setting means for setting related information of the object,
Related information storage means for storing the related information in association with the object vector,
A virtual three-dimensional space setting means having a coordinate axis and setting a virtual three-dimensional space in which the end point of the object vector starting from the photographing position and pointing in the direction is associated with the coordinate on the coordinate axis;
Display range specifying means for specifying a display range in the virtual three-dimensional space based on the shooting position and the shooting direction, tilt, and angle of view of the camera used for shooting.
Related information specifying means for specifying the related information associated with the object vector whose end point is included in the display range in the virtual three-dimensional space,
Display means for displaying the specified related information by superimposing it on the image being displayed,
To function as,
The vector generation means, when the depression angle in the direction is equal to or greater than a predetermined reference angle, the coordinates of the intersection calculated from the straight line drawn from the starting point in the direction of the object vector and the preset equation of the ground plane are set. An object recording program having a function of determining the length of the object vector based on the above . Computer,
Photographing position determining means for determining the photographing position of the object being displayed by photographing or reproducing,
Direction detecting means for detecting the direction of the object being displayed from the photographing position by photographing or reproducing,
A vector generation unit that determines the length of a vector pointing in the direction based on a predetermined reference by using the shooting position as a starting point and generates the target vector.
Related information setting means for setting related information of the object,
Related information storage means for storing the related information in association with the object vector,
A virtual three-dimensional space setting means having a coordinate axis and setting a virtual three-dimensional space in which an end point of an object vector having a shooting position of the object as a starting point and facing the direction of the object is associated with the coordinates on the coordinate axis;
Display range specifying means for specifying a display range in the virtual three-dimensional space based on the shooting position and the shooting direction, tilt, and angle of view of the camera used for shooting.
Related information specifying means for specifying the related information associated with the object vector whose end point is included in the display range in the virtual three-dimensional space,
Display means for displaying the specified related information by superimposing it on the image being displayed,
The display means displays the projection point of the coordinates on the preset horizon, using the coordinates of the end point of the object vector in the virtual three-dimensional space as the coordinates of the related information, and the above-mentioned via the input means. Vector correction means for correcting the length and/or direction of the object vector associated with the relevant information when the position of the projection point is changed,
And then to function, the pair Zobutsu recording program.