JPWO2013161028A1 - Image display device, image display method, image display program, and recording medium - Google Patents

Abstract

The present invention provides an image display technique that allows a user to intuitively and easily grasp which feature in the front image has been recognized by the image recognition processing. When a road sign 21 to be recognized is recognized, a target marker 31 is immediately displayed around the image portion of the road sign 21 and a moving object 32 is displayed near the target marker 31 at the same time. The The moving object 32 continues to be displayed while moving toward the position of the lower left corner, which is the extracted image display position. When the moving object 32 reaches the extracted image display position in the lower left corner, an extracted image 33 that is generated by extracting and enlarging the image portion of the road sign 21 in the area in the target marker 31 is displayed. It is possible to recognize the position of the road sign in the display coordinates x1-y1 of the front image and clearly indicate what the image portion is.

Description

  The present invention relates to an image display device, a navigation device, an image display method, an image display program, and a recording medium that extract and display an image portion of a feature recognized in a front image of a moving body.

  In recent years, a camera mounted on a moving body is used to capture an image of the surroundings, and among many display objects displayed in the image, image recognition is performed on features such as road signs that should particularly alert the user, A technique for registering the feature and its actual position with respect to map information stored in the navigation device has been proposed.

  For example, in the technique described in Patent Document 1, a road sign or the like extracted from a captured image of a vehicle-mounted camera, a travel distance of the vehicle from the imaging location, an elapsed time since imaging, and the like are appropriately adjusted. By displaying together with a map image for car navigation, it is easy for the user to recognize when and what kind of image the device has extracted.

Japanese Patent No. 4354217

  However, in the image recognition processing, it is common to display only the recognition result without displaying the processing content itself during the processing step. For example, when a plurality of road signs are displayed in the forward image at a certain time, even if one road sign is recognized by the image recognition process, the user intuitively grasps which road sign is recognized. It is not easy to do. Especially for the driver who is driving, it is not possible to closely check the road signs that appear to move while driving and the road signs that result from recognition on the display screen of limited size. It was difficult, and it was difficult to determine whether the road sign of the recognition result is related to the road that is currently running.

  In the above prior art, although the rough relative relationship between the road sign itself of the recognition result and the own vehicle at that time is displayed, it is not possible to grasp from which position in the captured image it is extracted and displayed, so the road that is currently running Cannot immediately determine whether it is related to Therefore, there has been a demand for an image display technique that allows the user to intuitively and easily grasp which feature in the front image is recognized by the image recognition processing.

  The problem to be solved by the present invention includes the above-described problem as an example.

  In order to solve the above-described problem, the invention according to claim 1 is an image display device, which is an image pickup device that picks up a front image from a moving moving body, a display device that displays the front image, and the front image. Recognizing means for recognizing a predetermined feature in the image, and superimposed display so as to move the object from the position where the predetermined feature is recognized in the display coordinates of the front image on the display means toward the predetermined position. First object superimposing means for causing image information to be superposed and displayed at the predetermined position in the display coordinates of the front image, on the basis of the image portion of the predetermined feature that has been image-recognized. And comprising.

  In order to solve the above-mentioned problem, the invention according to claim 6 is a navigation device, wherein the image pickup means for picking up a front image from a moving moving body, the front image and a map image around the current position of the moving body. Display means for simultaneously displaying, a recognition means for recognizing a predetermined feature in the front image, and a position where the predetermined feature is image-recognized in display coordinates of the front image on the display means, Generated on the basis of the second object superimposing means for superimposing and displaying the object to move toward the actual position of the predetermined feature in the display coordinates of the map image, and the image portion of the predetermined feature recognized as an image A second notification image superimposing unit that superimposes and displays the notification image on the actual position in the display coordinates of the map image.

  In order to solve the above-mentioned problem, an invention according to claim 7 is an image display method executed by an image display device, the imaging step of capturing a front image from a traveling moving body, and a display for displaying the front image. A recognition step for recognizing a predetermined feature in the front image, and an object from a position where the predetermined feature is recognized in the display coordinates of the front image in the display step toward a predetermined position. A first object superimposing step for superimposing and displaying the image so as to move, and a notification image generated based on the image portion of the predetermined feature that has been image-recognized at the predetermined position in the display coordinates of the front image. 1 notification image superimposing step.

  In order to solve the above-mentioned problem, an invention according to claim 8 is an image display method executed by a navigation device, the imaging step of capturing a front image from a traveling moving body, the front image and the moving body. A display step for simultaneously displaying a map image around the current position, a recognition step for recognizing a predetermined feature in the front image, and the predetermined feature in the display coordinates of the front image in the display step. A second object superimposing step of superimposing and displaying an object so as to move from the position where the image is recognized toward the actual position of the predetermined feature in the display coordinates of the map image; and And a second notification image superimposing step of superimposing and displaying the notification image generated based on the image portion on the actual position in the display coordinates of the map image.

  In order to solve the above-mentioned problem, an invention according to claim 9 is an image display program, and causes the image processing method according to claim 7 or 8 to be executed by an image processing device or a navigation device.

  In order to solve the above-mentioned problem, the invention according to claim 10 is a recording medium, and the image display program according to claim 9 is stored so as to be readable by the image processing device or the navigation device.

It is a perspective view which shows the structural example of the vehicle carrying the navigation apparatus which is embodiment of the image display apparatus of this invention. It is a block diagram which shows the hardware structural example of a navigation apparatus. It is a figure which shows an example of the display of the vehicle window scenery seen from the cab, and the display corresponding to this. It is a figure showing the example of a display at the time of displaying the front image which the front camera of the attitude | position state shown in FIG. 3 imaged on the display. It is an example which showed the display timing of the road sign in the front image, the target marker, the moving object, and the extraction image in time series. It is a figure showing the example of a display mode of an extraction picture at the time of misrecognizing. It is a figure showing the example of a display of a registration selection screen. It is a flowchart showing the control content which CPU of a navigation apparatus main body performs. It is a figure showing the example of a display of an icon image. It is a figure showing the case where reliability is changed by repeating image recognition. It is a figure explaining the example of efficiency improvement in the case of repeating image recognition. It is a figure showing the modification which displays a front image and a map image together on a display.

  Hereinafter, one of the embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a perspective view showing a configuration example of a vehicle equipped with a navigation device which is an embodiment of an image display device of the present invention. In FIG. 1, the navigation device S is provided with a single front camera 1 at a front side position of a room mirror 101 installed on the ceiling of a vehicle V as a moving body. The imaging direction Dc of the front camera 1 is attached in a posture capable of imaging the front in the traveling direction of the host vehicle V. The front camera 1 does not remove the direction Dv of the vehicle axis passing through the center front-rear direction of the host vehicle V from the imaging range within the movable range of the posture.

  FIG. 2 is a block diagram illustrating a hardware configuration example of the navigation device S. As illustrated in FIG. In FIG. 2, the navigation device S includes a front camera 1, a navigation device body 2, a display 3, and an operation unit 4.

  The front camera 1 has a function of capturing a front image in the traveling direction of the host vehicle described above using, for example, a CCD image sensor and outputting a corresponding signal to a CPU (described later) of the navigation apparatus body 2. In the example of the present embodiment, the front camera 1 continuously captures a plurality of image frames in a time series with a sufficiently short time period, thereby capturing a front image in the form of a moving image.

  The display 3 corresponds to a display unit, and is composed of, for example, an LCD panel, and a forward image captured by the front camera 1 based on an image signal input from a graphic controller (described later) of the navigation device body 2 is described later. It has a function of displaying a notification image and a composite image in which a moving object is superimposed.

  The operation unit 4 has a function of detecting various input operations from the user. Although not particularly shown, the display 3 may be substituted for an operation unit as a so-called touch panel display.

  The navigation device body 2 has a CPU 11, a storage device 12, a GPS 13, and a graphic controller 14.

  The CPU 11 has a function of controlling the navigation device S as a whole by performing various calculations according to the operation of a predetermined program and exchanging information with other units and outputting various control instructions.

  The storage device 12 includes a ROM 12a, a RAM 12b, and a storage medium 12c. The ROM 12a is an information storage medium in which various processing programs and other necessary information are written in advance. The RAM 12b is an information storage medium on which information necessary for executing the various programs is written and read. The storage medium 12c is a non-volatile information storage medium such as a flash memory or a hard disk.

  The GPS 13 has a function of measuring the current position of the vehicle V and acquiring current position information, and detecting and registering the actual position of a predetermined feature (detailed later) with respect to map information stored in advance. Have.

  The graphic controller 14 has a function of acquiring image data from a video RAM (not shown) and the GPS 13 under the control of the CPU 11 and displaying an image signal based on the image data on the display 3. As described above, the graphic controller 14 in the example of the present embodiment displays an image signal based on synthesized image data in which a notification image or a moving object described later is superimposed on a front image captured by the front camera 1. 3 is output. In the example of the present embodiment, the graphic controller 14 causes the display 3 to display an image signal at an arbitrary timing regardless of the time period when the front camera 1 captures a front image in the moving image format.

  FIG. 3 is a diagram illustrating an example of a vehicle window landscape viewed from the cab and a display on the display 3 corresponding thereto. In the illustrated example, the host vehicle V is traveling on a substantially straight road, and shows a state in which the host vehicle V follows the forward vehicle Vf that is in front of the traveling direction. The front camera 1 provided on the back side (traveling direction side) of the room mirror 101 captures a forward image including the straight road and the forward vehicle Vf. At this time, as described above, the imaging range of the front camera 1 includes the vehicle axis direction Dv of the host vehicle V without removing it. A display 3 installed next to the steering 103 on the instrument panel 102 displays a front image captured by the front camera 1.

  Here, in the navigation apparatus S of the example of the present embodiment, the road sign installed on the road on which the host vehicle V is traveling, particularly the road sign that restricts the action of the host vehicle V such as speed limit or parking prohibition is described above. Image recognition is performed from the front image as a predetermined feature. Then, the image portion of the road sign displayed at the recognition position is extracted as an extracted image, and the actual position on the map is registered in the map information of the GPS 13 together with the extracted image. In this way, by registering and accumulating extracted images of road signs actually installed on the running road in the map information corresponding to the actual position, it was practically not covered by the ready map information Learn and update in detail the location and content of various road signs.

  However, the recognition accuracy of the current image recognition technology is not yet perfect under various environmental conditions, and ultimately the user may need to judge whether the recognition result is correct and registered in the map information. However, in the image recognition process, since it is common to display only the recognition result without displaying the processing content itself during the processing step, it may make the determination of the user difficult. For example, when a plurality of road signs are displayed in the forward image at a certain time, even if one road sign is displayed as a recognition result by the image recognition processing, the user can determine which road sign is recognized. It is not easy to grasp intuitively. Especially for the driver who is driving, it is not possible to closely check the road signs that appear to move while driving and the road signs that result from recognition on the display screen of limited size. It was difficult, and it was difficult to determine whether the road sign of the recognition result is related to the road that is currently running.

  Therefore, in this embodiment, the extracted image of the recognition result in the front image and the recognition of the extracted image so that the user can intuitively and easily grasp which road sign in the front image has been recognized by the image recognition processing. The moving object indicating the correspondence between the position and the display position is individually superimposed and displayed. Hereinafter, display examples of such extracted images and moving objects will be sequentially described.

  First, FIG. 4 shows a display example when a front image captured by the front camera 1 in the posture state shown in FIG. 3 is displayed on the display 3. In the example of the present embodiment shown in the figure, the front image is displayed with the entire display coordinates XY set in the display area of the display 3, that is, the display coordinates XY of the display 3 are the display coordinates of the front image. It matches x1-y1. In the example shown in FIG. 4, two road signs 21 and 22 are displayed as display objects in addition to the preceding vehicle Vf in the front image.

  In the example of the navigation device S of the present embodiment, a road sign 21 with a maximum speed limit is set in advance as a feature to be image-recognized. That is, in the image recognition process of the navigation device S, a feature having an appearance common to a road sign with a maximum speed limit, which is a circular shape with a red border with a numeral written in the center, is recognized and detected. For example, the road sign 21 in the illustrated example is a road sign indicating a maximum speed limit of 40 km. Basically, the image recognition process of the navigation device S scans the entire display coordinates x1-y1 of the front image and this road sign. 21 is recognized and detected. The maximum speed limit road sign 21 corresponds to a predetermined feature described in each claim. The feature to be recognized is not limited to a road sign, and any other feature such as a traffic light, a post, a store signboard, or the like can be set.

  When the navigation apparatus S recognizes the road sign 21, the target marker 31, the moving object 32, and the extracted image 33 are superimposed and displayed within the display coordinates x1-y1 of the front image at predetermined positions and timings, respectively. To do.

  The target marker 31 is a substantially rectangular frame-shaped graphic image that is displayed for a relatively short time in an arrangement surrounding an image portion corresponding to the road sign 21 that has been image-recognized. The target marker 31 corresponds to a marker described in each claim.

  The moving object 32 moves on a straight path (see the broken line in the figure; not displayed on the actual display 3) from the approximate center position of the image portion of the road sign 21 that has been image-recognized to the extracted image display position described later. It is a graphic image to be displayed, and in the example of this embodiment, is shown by a meteor-shaped graphic with a tail. The moving object 32 corresponds to an object described in each claim.

  The extracted image 33 is an enlarged / reduced image obtained by extracting (copying) the image portion of the road sign 21, in this example, the image portion of the region surrounded by the target marker 31, and enlarging or reducing it to a predetermined size. In the example of the present embodiment, the image is superimposed and displayed at the extracted image display position set at the lower left corner of the display coordinates x1-y1 of the front image. The extracted image 33 corresponds to the notification image and the enlarged / reduced image described in each claim, and the extracted image display position corresponds to the predetermined position described in each claim.

  FIG. 5 is an example showing the display timing of the road sign 21, the target marker 31, the moving object 32, and the extracted image 33 in time series in the display coordinates x1-y1. First, in FIG. 5A, when the recognition target road sign 21 is recognized as an image, the target marker 31 is immediately displayed around the image portion of the road sign 21, and at the same time, a moving object near the target marker 31. 32 is displayed.

  Next, as shown in FIG. 5B, the display of the target marker 31 disappears at the timing when the road sign 21 is removed from the target marker 31 as the host vehicle V moves. On the other hand, the moving object 32 continues to be displayed while moving toward the position of the lower left corner, which is the extracted image display position.

  Then, as shown in FIG. 5C, when the moving object 32 reaches the extracted image display position in the lower left corner, the image portion of the road sign 21 that is in the area in the target marker 31 is extracted and enlarged and generated. The extracted image 33 is displayed. Although not particularly illustrated, only the moving object 32 is erased immediately after this, and the extracted image 33 is continuously displayed as a still image for a certain period. During this time, the entire front image continues to change as the host vehicle V moves, but the extracted image 33 of the road sign 21 is continuously displayed as a still image, so that it can be clearly distinguished from the front image that is the background.

  By performing the series of display modes as described above in a short time, the navigation device S recognizes the position of the road sign in the display coordinates x1-y1 of the front image, and what is the content of the image portion. Can be clearly shown. In other words, the user who has seen the image recognizes the road sign 21 of the image portion surrounded by the target marker 31 by the navigation device S, and the image portion extracted therefrom is enlarged and displayed at a predetermined extracted image display position. You can grasp this intuitively. At this time, the user can confirm at which position the image-recognized road sign 21 is image-recognized in the display coordinates x1-y1 of the forward image and what kind of content it is. It is easy to determine whether or not the road is related to a running road. For example, although not particularly shown, a road sign of another road different from the road on which the host vehicle V is traveling may be captured in the front image if the place has good visibility, and the road sign recognized by the navigation device S is imaged. 21 is extracted and displayed in the above-described display mode, so that the user who sees it can determine whether or not the extracted and displayed road sign 21 is installed corresponding to the road on which the vehicle V is traveling. Becomes easy.

  Further, in the image recognition process performed by the navigation device S of the present embodiment, it is possible to calculate the reliability of image recognition for the feature in the front image by itself. The certainty factor is an evaluation value indicating the coincidence with the target characteristic object that should be image-recognized originally, that is, the reliability of the image recognition (and extraction). A known method may be used for the calculation of the certainty factor, and detailed description thereof is omitted here. For example, as shown in FIG. 6, a signboard 23 (characters are written in the center) similar in appearance to the road sign 21 (maximum speed-restricted road sign 21 with a number in the center) that should be recognized. The navigation device S may recognize an image of the red octagonal shaped object with a red border. However, in this case, since they do not exactly match, a certain degree of certainty is calculated. In addition, in the forward image captured at night or in the rain, the uncertainties remain in the image recognition even with respect to the maximum speed limit road sign 21 that is the original recognition target, so the certainty level may be lowered.

  In this embodiment, when the certainty level at the time of such image recognition is low to some extent, as shown in the figure, the outer periphery of the extracted image 33 is surrounded by a broken line to make the display mode different from the normal case, thereby allowing the image to the user. Notifies the low confidence level of recognition. Further, as shown in FIG. 7, a registration selection screen 34 that prompts the user to select and input whether or not such an extracted image 33 with low certainty should be registered in the map information is displayed. At this time, considering the case where the user is a driver of the vehicle V, when there is no selection input operation and a predetermined time has passed, default selection is automatically performed with a preset selection candidate. In the example shown in the figure, the option “Yes” for selecting registration is set as a default, and is displayed surrounded by a thick frame.

  FIG. 8 is a flowchart showing the control contents executed by the CPU 11 of the navigation apparatus body 2 in order to realize the operation mode described above. This flow is executed when the front camera 1 starts capturing a front image.

  In FIG. 8, first, in step S5, a front image (image frame) captured by the front camera 1 at that time is acquired and displayed.

  The process proceeds to step S10, where the entire image frame of the forward image acquired in step S5 is scanned to recognize the recognition target road sign 21 (abbreviated as a sign in the figure), and the image portion of the road sign 21 is extracted. In this image recognition process, the certainty factor is also calculated. Moreover, the procedure of this step S10 corresponds to the extraction means and the extraction step described in each claim.

  Proceeding to step S15, it is determined whether or not the certainty factor when the road sign 21 is image-recognized in step S10 is equal to or higher than the medium level range. If the certainty factor is in the low level range, the determination is not satisfied, and the process returns to step S5 and the same procedure is repeated.

  On the other hand, if the certainty factor is greater than or equal to the medium level range, the determination is satisfied, and the routine goes to the next Step S20.

  In step S20, the target marker 31 is displayed so as to surround the image portion of the road sign 21 recognized in step S10. Thereafter, the target marker 31 continues to be displayed until the road sign 21 moves in the display coordinates x1-y1 of the image frame and moves away from the target marker 31. The procedure of step S20 corresponds to the display control means described in each claim.

  Moving to step S25, the center position of the display area of the road sign 21 in the display coordinates x1-y1 of the image frame is detected.

  Moving to step S30, the moving object 32 is moved and displayed from the center position detected in step S25 to the extracted image display position (the position of the lower left corner in this example). When the moving object 32 reaches the extracted image display position, the process proceeds to the next step S35. The procedure of step S30 corresponds to the display control means and the display control process described in each claim.

  In step S35, it is determined whether or not the certainty factor when the road sign 21 is image-recognized in step S10 is in a high level range. If the certainty factor is in the high level range, the determination is satisfied, and the routine goes to Step S40.

  In step S40, the extracted image 33 of the road sign 21 recognized in step S10, that is, the image portion in the range surrounded by step S20 is enlarged / reduced to a prescribed size, and the enlarged / reduced image is normally displayed as it is at the extracted image display position. . The procedure of step S40 corresponds to the display control means and the display control process described in each claim.

  The process proceeds to step S45, where the actual position of the road sign 21 that has been image-recognized by collating with the current position information of the GPS 13 is detected, and the actual position is registered in the map information together with the extracted image 33. And it returns to said step S5 and repeats the same procedure.

  On the other hand, if it is determined in step S35 that the certainty level is in a medium level range that does not satisfy the high level, the determination is not satisfied and the process proceeds to step S50.

  In step S50, the road sign extraction image 33 extracted in step S10 is enlarged / reduced to a predetermined size, and a broken line is superimposed on the outer periphery of the enlarged / reduced image and displayed at the extracted image display position. That is, as shown in FIG. 6, the extracted image 33 is displayed with low confidence. The procedure of step S50 also corresponds to the display control means and the display control process described in each claim.

  In step S55, the registration selection screen 34 shown in FIG. 7 is displayed in the display coordinates x1-y1 of the image frame.

  In step S60, it is determined whether or not the user has performed an input operation corresponding to “Yes” for registration permission via the operation unit 4. When the input operation of “Yes” is performed, the determination is satisfied, and the process proceeds to Step S45, and the extracted image 33 and the actual position are registered in the map information.

  On the other hand, if the “Yes” input operation is not performed, the determination is not satisfied, and the routine goes to Step S65.

  In step S <b> 65, it is determined whether or not the user has performed an input operation corresponding to “No” for which registration is not permitted via the operation unit 4. If an input operation of “No” is performed, the determination is satisfied, the process returns to step S5, and the same procedure is repeated.

  On the other hand, if the “No” input operation is not performed, the determination is not satisfied, and the routine goes to Step S70.

  In step S70, it is determined whether or not a predetermined waiting time has elapsed since the registration selection screen 34 was displayed in step S55. If the standby time has not yet elapsed, the determination is not satisfied and the routine returns to step S60 and the same procedure is repeated.

  On the other hand, if the standby time has elapsed, the determination is satisfied, and the routine goes to Step S75.

  In step S75, it is determined whether or not the default setting is such that registration permission is automatically selected. If the automatic registration is set to be permitted, the determination is satisfied, and the process proceeds to step S45 where the extracted image 33 and the actual position are registered in the map information.

  On the other hand, when it is set not to permit automatic registration, the determination is not satisfied, and the process returns to step S5 and the same procedure is repeated.

  As described above, in the navigation device S of the above-described embodiment, the display 3 (corresponding to the display unit) that displays the front image taken from the vehicle V (corresponding to the moving body) and the highest of the front images. An extracted image 33 generated based on the procedure of step S10 (corresponding to extraction means) for extracting an image of the speed limit road sign 21 (corresponding to a predetermined feature) and the extracted image of the maximum speed limit road sign 21 extracted. (Corresponding to a notification image) is displayed at the extracted image display position (corresponding to a predetermined position) of the display 3, and includes a procedure of Step S40 and Step S50 (corresponding to display control means). 3, a moving object 32 (moving toward the extracted image display position from the position where the image of the road sign 21 with the maximum speed limit in 3 is extracted. To display the equivalent) to the object.

  In addition, in the image display method executed by the navigation device S of the above embodiment, the navigation device S (image) including the display 3 (corresponding to the display unit) that displays a forward image taken from the vehicle V (corresponding to a moving object). (Corresponding to a display device) is an image display method executed by the procedure of step S10 (corresponding to an extraction process) for extracting an image of a road sign 21 (corresponding to a predetermined feature) of the maximum speed limit in the forward image. ), And an extracted image 33 (corresponding to a notification image) generated based on the extracted image of the road sign 21 with the maximum speed limit extracted, at an extracted image display position (corresponding to a predetermined position) of the display 3; Step S50 (corresponding to the display control step), and in the procedure of Step S30, the road of the maximum speed limit on the display 3 To display a moving object moving toward the extraction image display position from the image extracted location of identification 21 (corresponding to the object).

  In this way, the navigation device S has the entertainment property of the moving object 32 as to which position of the road sign 21 in the display coordinates x1-y1 of the forward image is recognized and the image portion is extracted as the extracted image 33. It can be shown by expression. In addition, the extracted image 33 fixedly displayed at a certain extracted image display position can be clearly distinguished from the forward image that continues to change with the movement of the host vehicle V in the background. Thereby, the moving display of the moving object 32 can clearly and geometrically express the correspondence between the image recognition position in the display coordinates x1-y1 of the front image and the extracted image 33 displayed at the extracted image display position. As a result, the user can intuitively and easily grasp which feature in the front image is recognized by the image recognition process, and convenience is improved.

  As an object to be displayed between the image recognized position and the extracted image display position, in addition to the moving object 32 as described above, the correspondence between the image portion of the road sign 21 and the extracted image 33 is a straight line connecting two points. A relationship may be indicated.

  In addition to the above-described configuration, the feature in the front image that is the recognition target is a road sign 21, and the extracted image 33 expands or contracts the extracted image portion of the road sign 21 to a predetermined size. The scaled image is included.

  In this way, even when the road sign 21 that is actually present at any distance from the host vehicle V is image-recognized, the image portion can be indicated by the extracted image 33 of a certain scale, and the user can recognize the image. It becomes easy to confirm the contents of the road sign 21.

  In the image recognition process in the above embodiment, only the feature corresponding to the road sign 21 with the maximum speed limit set as the recognition target is recognized. The present invention is not limited to this, and further recognizes images up to the speed limit number described on the road sign 21 of the maximum speed limit, and displays an icon image that abstractly represents the contents of the image portion instead of the extracted image portion itself. It may be displayed. For example, in FIG. 9, the enlarged / reduced image of the extracted image portion is displayed at the lower left corner of the display coordinates x1-y1 of the front image, and the icon image 35 is displayed side by side to the right. The icon image 35 in the illustrated example displays a newly generated geometric figure corresponding to the content of the recognized road sign 21 and also includes a character string “40 km speed limit” of the content indicated by the road sign 21. It is added. By displaying such an icon image 35, the user can easily confirm the contents of the road sign 21 whose image has been recognized. Note that the content of the icon image 35 may be changed according to the certainty or reliability of the image recognition. For example, when it is possible to recognize that the road sign 21 is the maximum speed limit but not the center number, the figure of the road sign 21 in which “?” Is displayed instead of the number is displayed in the center. (Not shown). Also, only the icon image may be displayed at the extracted image display position without displaying the enlarged / reduced image, or may be displayed by switching from the middle. The icon image 35 also corresponds to a notification image described in each claim.

  In addition to the above-described configuration, the procedure of step S10 calculates a certainty factor when the image of the road sign 21 with the maximum speed limit is extracted, and the procedure of steps S40 and S50 includes the extracted image 33. Are displayed according to the certainty factor.

  In this way, the user can intuitively grasp the degree of certainty at the time of image recognition due to the difference in the display mode of the extracted image 33, whether or not to register in the map information, etc. Judgment becomes easy. In the above embodiment, only the display mode of the extracted image 33 is changed according to the level of certainty. However, for example, the display mode of the target marker 31 and the moving object 32 may also be changed. Further, in the above embodiment, after displaying the dedicated registration selection screen 34, the user is allowed to select and input whether or not the extracted image 33 can be registered in the map information. However, the present invention is not limited to this. In addition, the display 3 may be configured by a touch panel display so that the user can refuse registration in the map information by touching the display area of the extracted image 33 at an arbitrary timing.

  In addition to the configuration described above, when the image of the maximum speed limit road sign 21 is extracted in the procedure of step S10, the vicinity of the position where the image of the maximum speed limit road sign 21 is extracted on the display 3 The procedure of step S20 (corresponding to display control means) for displaying the target marker 31 (corresponding to the marker) is performed.

  In this way, the position of the road sign 21 recognized in the display coordinates x1-y1 of the front image can be more clearly indicated by the target marker 31. In the above embodiment, the target marker 31 is displayed for a relatively short time. However, the target marker 31 may be continuously displayed following the movement of the road sign 21 accompanying the movement of the host vehicle V. In this embodiment, the target marker 31 is displayed, but the display of the target marker is arbitrary. Even when the target marker is not displayed, since the moving object 32 is moved and displayed from the center position of the road sign 21 recognized as an image to the extracted image display position in the present invention, it is possible to know which image is recognized.

  In addition, when the processing capability of the navigation apparatus S is sufficient, the image recognition process can be frequently performed in a short cycle. As a result, the degree of certainty may change each time image recognition is performed a plurality of times while the host vehicle V approaches the same road sign 21. For example, if the surroundings are dark at night, as shown in FIG. 10A, even if the road sign 21 located far away is not clear as shown in FIG. It may be more clearly visible due to the influence of headlights. In this case, the certainty factor when the image recognition is performed in FIG. 10B is greater than the certainty factor when the image recognition is performed in FIG. As described above, the display mode of the extracted image 33 and the moving object 32 may be changed in real time in accordance with the certainty of the image recognition that changes with time. On the contrary, since the reliability of image recognition is improved due to the approach, the certainty of the current recognition may decrease with respect to the previous recognition. In this case, the moving object 32 may be expressed as disappearing in the middle. Good.

  When the image recognition process is repeatedly performed in a short cycle as described above, it is necessary to perform image recognition by scanning the entire display coordinates x1-y1 of the front image every time the image recognition process is performed on the same road sign 21. There is no. For example, as shown in FIG. 11, the road sign 21 is predicted to move in the direction toward the outer circumference along a half line (see the broken line arrow in the figure) arranged substantially radially around the vanishing point P. The

  Here, the vanishing point P is the infinitely distant movement position of the host vehicle V in the traveling direction at that time in the forward image (the vehicle V in the display coordinates x1-y1 in the forward image displayed at that time). This point corresponds to a planned position that is reached when the vehicle travels in the traveling direction as it is. This point is always approximately the horizon and the vehicle axis direction Dv when the host vehicle V is traveling on a flat and straight road. It is located on the intersection with the approximate extension line.

  Such vanishing point P is obtained by a known method, and the same road at the time of the next image recognition process at a separation distance on display coordinates x1-y1 corresponding to the vehicle speed of the host vehicle V with respect to the road sign 21 once image-recognized. The display position 41 of the sign 21 can be predicted. Therefore, at the time of the next image recognition, by recognizing the image by widening the scanning range at the predicted position 41 and its surrounding area (for example, the shaded area 42 shown in the figure), the time until the recognition position is determined is greatly reduced. it can.

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit and technical idea of the present invention.

(1) When displaying a front image and a map image together on the display In the above embodiment, the front image is displayed on the entire display coordinates XY of the display 3, but the present invention is not limited to this. For example, a map image of the GPS 13 may be displayed together with the front image on the display 3, and in this case, the extracted image 33 may be superimposed and displayed at a position on the map image corresponding to the actual position.

  In FIG. 12, a map image is displayed on the left side and a front image is displayed on the right side of the entire display coordinates XY of the display 3, and the display coordinates x2-y2 are displayed in the display area of the map image. Are respectively set with display coordinates x1-y1. Although only a part of the front image is displayed at the display coordinates x1-y1, the entire front image may be compressed and displayed at the display coordinates x1-y1. The map image is displayed around the current position of the host vehicle V on an appropriate scale.

  In this modification, as shown in the drawing, the road sign 21 is recognized in the display coordinates x1-y1 of the front image, and the extracted image 33 obtained by reducing the image portion to an appropriate size is used as the actual position of the road sign 21. Is superimposed and displayed at a position on the display coordinates x2-y2 of the map image corresponding to. The moving object 32 is moved and displayed across the recognition position in the front image and the display position in the map image.

  In the above, the control procedure corresponding to the step S30 for moving and displaying the moving object 32 corresponds to the display control means and the display control step described in each claim, and corresponds to the steps S40 and S50 for displaying the extracted image 33. The control procedure corresponds to the display control means and the display control step described in each claim.

  As described above, in the navigation device S of the present modification, the display 3 (display unit) that simultaneously displays a front image taken from the vehicle V (corresponding to a moving body) and a map image around the current position of the vehicle V. And a control procedure (corresponding to the extraction means) corresponding to step S10 for extracting the image of the road sign 21 (corresponding to a predetermined feature) of the maximum speed limit in the front image, and the extracted In steps S40 and S50, the extracted image 33 (corresponding to the notification image) generated based on the image of the maximum speed limit road sign 21 is displayed at the actual position of the maximum speed limit road sign 21 in the map image. Corresponding control procedure (corresponding to display control means), and in the control procedure corresponding to step S30 (corresponding to display control means) Wherein the maximum speed limit position where the image obtained by extracting the road sign 21 of displaying the moving object (corresponding to the object) to move toward the actual position of the maximum speed limit of the road sign 21 in said map image.

  Further, in the image display method executed by the navigation device S of the present modification, the display 3 (at the same time displaying a forward image taken from the vehicle V (corresponding to a moving object) and a map image around the current position of the vehicle V). Step S10 is an image display method executed by the navigation device S including the display unit), and extracts an image of the maximum speed limit road sign 21 (corresponding to a predetermined feature) in the front image. An extracted image 33 (corresponding to a notification image) generated based on the corresponding control procedure (corresponding to the extracting step) and the extracted image of the road sign 21 with the maximum speed limit extracted is the road with the maximum speed limit in the map image. Control step (corresponding to a display control step) corresponding to step S40 and step S50 to be displayed at the actual position of the sign 21, In the control procedure, a moving object (object) that moves from the position where the image of the road sign 21 with the maximum speed limit on the display 3 is extracted to the actual position of the road sign 21 with the maximum speed limit in the map image. Display).

  In this way, the navigation device S has the entertainment property of the moving object 32 as to which position of the road sign 21 in the display coordinates x1-y1 of the forward image is recognized and the image portion is extracted as the extracted image 33. It can be shown by expression. Further, the extracted image 33 displayed in the display coordinates x2-y2 of the map image can clearly indicate the actual position of the road sign 21 at the display position on the map image. Thereby, the moving display of the moving object 32 can clearly and geometrically express the correspondence between the image recognition position in the display coordinates x1-y1 of the front image and the actual position in the display coordinates x2-y2 of the map image. . As a result, the user can intuitively and easily grasp which feature in the front image is recognized by the image recognition process, and convenience is improved.

  In addition to those already described above, the methods according to the above-described embodiments and modifications may be used in appropriate combination.

1 Front camera 2 Navigation device body 3 Display (corresponding to the display)
4 Operation unit 11 CPU
12 Storage device 13 GPS
14 Graphic controller 21 Maximum speed limit road sign (corresponding to a specified feature)
31 Target marker (equivalent to marker)
32 Moving objects (equivalent to objects)
33 Extracted image (equivalent to notification image, enlarged / reduced image)
34 Registration selection screen 35 Icon image S Navigation device V Vehicle

The present invention relates to an image display device , an image display method , an image display program, and a recording medium that extract and display an image portion of a feature recognized in a front image of a moving body.

In order to solve the above-mentioned problem, the invention according to claim 1 is an image display device, which simultaneously displays a forward image taken from a moving body and a map image around the current position of the moving body, An extraction unit that extracts an image of a predetermined feature in the front image, and a notification image that is generated based on the feature image extracted by the extraction unit corresponds to an actual position of the extracted feature Display control means for displaying at a position in the map image, the display control means in the display unit, the position in the front image from which the image of the feature is extracted, and the existence of the feature An object indicating the correspondence with the position in the map image corresponding to the position is displayed.

In order to solve the above-mentioned problem, an invention according to claim 7 is an image display method executed by an image display device , wherein a front image taken from a moving object and a map image around the current position of the moving object are simultaneously displayed. An image display method executed by an image display device including a display unit for displaying, wherein an extraction step of extracting an image of a predetermined feature in the front image, and an image of the feature extracted by the extraction step A display control step of displaying a notification image generated based on the position in the map image corresponding to the actual position of the extracted feature, and the display control step includes the feature in the display unit. An object indicating the correspondence between the position in the front image from which the image of the image is extracted and the position in the map image corresponding to the actual position of the feature is displayed.

In order to solve the above-mentioned problem, the invention described in claim 8 is an image display program, and the image display method described in claim 7 is executed by the image display apparatus.

In order to solve the above-mentioned problem, the invention according to claim 9 is a recording medium, and the image display program according to claim 8 is stored so as to be readable by the image display device .

Claims (10)

A display unit for displaying a front image taken from a moving body;
Extraction means for extracting an image of a predetermined feature in the front image;
Display control means for displaying a notification image generated based on the extracted image of the predetermined characteristic object at a predetermined position of the display unit;
With
The display control means displays an object moving from the position where the image of the predetermined feature on the display unit is extracted toward the predetermined position. The predetermined feature in the front image is a road sign;
The image display device according to claim 1, wherein the notification image includes a scaled image obtained by scaling the extracted image of the predetermined feature to a predetermined size.   The image display device according to claim 1, wherein the notification image includes an icon image that abstractly represents the content of the extracted image of the predetermined feature. The extraction means calculates a certainty factor when the image of the predetermined feature is extracted;
The image display device according to any one of claims 1 to 3, wherein the display control means displays at least one of the object and the notification image in a manner corresponding to the certainty factor.   The display control unit displays a marker in the vicinity of the position where the image of the predetermined feature is extracted on the display unit when the extraction unit extracts the image of the predetermined feature. The image display device according to claim 1. A display unit that simultaneously displays a front image taken from a moving object and a map image around the current position of the moving object;
Extraction means for extracting an image of a predetermined feature in the front image;
Display control means for displaying a notification image generated based on the extracted image of the predetermined feature in the actual position of the predetermined feature in the map image;
With
The display control means displays an object that moves from the position where the image of the predetermined feature on the display unit is extracted toward the actual position of the predetermined feature in the map image. apparatus. An image display method executed by an image display device including a display unit that displays a front image taken from a moving body,
An extraction step of extracting an image of a predetermined feature in the front image;
A display control step of displaying a notification image generated based on the extracted image of the predetermined feature object at a predetermined position of the display unit;
Including
The display control step displays an object moving from the position where the image of the predetermined characteristic object is extracted on the display unit toward the predetermined position. An image display method executed by a navigation device including a display unit that simultaneously displays a front image captured from a moving object and a map image around the current position of the moving object,
An extraction step of extracting an image of a predetermined feature in the front image;
A display control step of displaying a notification image generated based on the extracted image of the predetermined feature in the actual position of the predetermined feature in the map image;
Including
The display control step displays an object that moves from the position where the image of the predetermined feature on the display unit is extracted toward the actual position of the predetermined feature in the map image. Display method.   An image display program for causing the image display method according to claim 7 or 8 to be executed by an image display device or a navigation device. 10. A recording medium in which the image display program according to claim 9 is stored so as to be readable by the image display device or the navigation device.

Images