JP2005004410A - Display device for road sign etc. and image display method for image of road sign etc. - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a display device for road signs etc., and a display method for images of road signs etc., that has improved convenience by setting a display period etc. for an image of a road sign etc., displayed by a display means to a period suitable to a user and also enable an image for car navigation and the image of the road sign etc., to be viewed at the same time. <P>SOLUTION: The display device 1 for road signs etc., is equipped with a road sign etc. extraction means 7 of extracting a road sign etc., from an image picked up by an on-vehicle camera 4, a recording means 15 of recording the image of the extracted road sign etc., a display means 3 for displaying car navigation images, and image output means 8 and 16 of editing the car navigation image and the road sign image etc., recorded in the recording means 15 so that they can be displayed on the same screen of the display means 15 at the same time. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a display device for road signs and the like and an image display method for images of road signs and the like that can extract road signs and the like from images captured by an in-vehicle camera and display them on display means.
[0002]
[Prior art]
When driving a car, the driver needs to visually recognize a road sign or a road display (hereinafter referred to as a road sign or the like) to determine the road condition. However, since the driver needs to visually recognize road signs and the like in a short period of time, an inexperienced driver can easily overlook road signs and the like, and there is a risk of misrecognizing even if not overlooked. For this reason, there is a problem that an inexperienced driver may get lost or perform an act in violation of the Road Traffic Law. In order to solve such a problem, today, a road sign or the like is photographed with an in-vehicle camera and recorded on a recording means. A technology has been developed that allows a driver to recognize a road sign or the like by outputting the voice. (For example, refer to Patent Document 1 and Patent Document 2.)
[0003]
[Patent Document 1]
JP-A-5-174294 [Patent Document 2]
Japanese Patent Laid-Open No. 6-036183
[Problems to be solved by the invention]
However, in the prior art, it has been unclear when the captured image such as a road sign is the captured image. Also, in the prior art, how long the acquired image such as a road sign is continuously displayed on the display means, how many times the image is output by voice, and how long the image such as the road sign is recorded in the recording means is sufficient. Has not been studied.
[0005]
For example, in the invention disclosed in Japanese Patent Application Laid-Open No. 06-036183, the period of the road sign image to be recorded on the recording means is determined by the recording capacity of the recording means. When a new road sign etc. image is recorded on the recording means whose recording capacity is full, the new road sign etc. image is overwritten on the old road sign etc. An update has been made. As described above, it is not desirable from the viewpoint of the convenience of the driver that the recording time and the display time of the image such as the road sign depend on the hardware limit of the recording capacity of the recording means.
[0006]
Furthermore, the mainstream of today's display means is a display device of a car navigation system, and it is desired to simultaneously display an image for navigation and an image such as a road sign on the screen of the same display device.
[0007]
The present invention has been made in view of the above-described problems, and the display period of an image such as a road sign displayed on the display means or the warning period in which a warning is given by the warning means is set as a suitable period for the user. The present invention provides a display device for road signs and the like, and a method for displaying images such as road signs, which can improve the convenience of the user and can simultaneously view the car navigation image and the road sign image.
[0008]
[Means for Solving the Problems]
In order to solve the above-mentioned problem, the invention according to claim 1 includes a road sign etc. extracting means for extracting a road sign etc. from an image taken by an in-vehicle camera, a road sign etc. extracted by the road sign etc. extracting means Image editing so that the recording means for recording the image, the display means for displaying the car navigation image, the car navigation image and the image such as the road sign recorded in the recording means can be displayed on the display means simultaneously. It is a display device for road signs and the like provided with image output means for performing and outputting to the display means.
[0009]
In the present invention, a display sign or the like means a road sign and a road display. A road sign refers to a sign board that displays regulations or instructions regarding road traffic (Article 2, Paragraph 1, Item 15 of the Road Traffic Law). In addition, road marking is a sign that displays regulations or instructions regarding road traffic, and means road lines, paint, intentions, lines, symbols or characters drawn on the road (Article 2 of the Road Traffic Law) 1 item 16).
[0010]
According to a second aspect of the present invention, in the road sign display device according to the first aspect, the image output means is configured to reduce a size of the navigation image in a horizontal direction or a vertical direction. An image of the road sign or the like is arranged and output to the display means.
[0011]
The invention according to claim 3 is the road sign display device according to claim 1 or 2, further comprising distance calculation means for calculating a travel distance from a place where the road sign or the like is imaged. The image is edited so that the travel distance calculated by the distance calculation unit for each image of the road sign or the like can be displayed together with the image of the road sign or the like, and is output to the display unit.
[0012]
According to a fourth aspect of the present invention, in the road sign display device according to the fourth aspect, the image output means displays the road sign or the like whose travel distance calculated by the distance calculation means is a predetermined distance or more. The image editing is performed so that it is not displayed on the screen.
[0013]
The invention according to claim 5 is the road sign display device according to claim 1 or 2, further comprising an elapsed time calculation means for calculating an elapsed time from when the road sign or the like is imaged. The means is characterized in that the elapsed time calculated by the elapsed time calculation means for each of the road sign etc. images is edited and output to the display means so that it can be displayed together with the road sign etc. image.
[0014]
The invention according to claim 6 is the road sign display device according to claim 4, wherein the image output means is the road sign or the like in which the elapsed time calculated by the elapsed time calculation means has exceeded a predetermined time. Image editing is performed so that the image is not displayed on the display means.
[0015]
According to a seventh aspect of the present invention, in the road sign display device according to any one of the first to sixth aspects, when a road sign or the like is extracted by the road sign extraction means, the driver is notified. A warning means for warning is provided.
[0016]
The invention according to claim 8 is the road sign display device according to claim 7, wherein the warning means issues a warning by sound or voice.
[0017]
The invention according to claim 9 is the road sign display device according to any one of claims 1 to 8, wherein the display means has a touch panel function, and the road sign displayed on the display means. When the image is touched more than a certain level, the display means identifies the touched image such as a road sign, and the image output means performs image editing so that the identified image such as the road sign is not displayed on the display means. And the recording means deletes the identified image such as a road sign.
According to a tenth aspect of the present invention, in the road sign display device according to the ninth aspect, the image output means enlarges or reduces the road sign image in accordance with an operation of a touch panel of the display means. The image editing is performed so that the image can be displayed on the display means.
According to an eleventh aspect of the present invention, an image of a road sign or the like indicating a road sign or the like is extracted from an image captured by a vehicle-mounted camera, and the extracted image of the road sign or the like is recorded in a recording unit and recorded in the recording unit It is an image display method for displaying an image of a road sign or the like that is edited and displayed on the display means so that the road sign or the like image and the car navigation image can be simultaneously displayed on a display means.
[0018]
According to a twelfth aspect of the present invention, in the image display method for an image of a road sign or the like according to the eleventh aspect, a space formed by reducing the size of the navigation image in a horizontal direction or a vertical direction in the image editing. An image of the road sign or the like is arranged with respect to
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, a display device for road signs and the like according to the present invention will be described with reference to the drawings.
[0020]
As shown in FIG. 1, the road sign display device 1 is connected to a car navigation device 2, a monitor 3, an in-vehicle camera 4, and a position information detection unit 5.
[0021]
The road information (navigation signal) retrieved by the car navigation device 2 is input to a road sign display device 1 (specifically, an image correction unit 8 described later), and the monitor 3 is displayed via the road sign display device 1. Is output.
[0022]
The monitor 3 has a touch panel function. When a user operates (touches) a touch panel (not shown), control information corresponding to the operation content is displayed on the road sign display device 1 (specifically, a microcomputer 7 described later). ). An image corrected by the image correction unit 8 is displayed on the monitor 3.
[0023]
The in-vehicle camera 4 is installed in front of the vehicle so as to capture the traveling direction of the vehicle, and the captured image (video) is always output to the display device 1 such as a road sign.
[0024]
The position information detection unit 5 is a device that detects the position information of the vehicle, and the display device 1 such as a road sign calculates the travel distance of the vehicle based on the position information. In this embodiment, the position information detection unit 5 is connected to the display device 1 such as a road sign. However, a time information detection unit is provided instead of the position information detection unit 5 so that the time information is displayed on the road sign display device 1. It is good also as a structure that the microcomputer 7 calculates elapsed time by sending to.
[0025]
The road sign display device 1 includes a microcomputer 7, an image correction unit 8, a sound / audio output unit 9, an A / D decoder 10, an input frame memory 11, a work memory 12, an output frame memory 13, A D / A encoder 14, an external storage device 15, and an image composition unit 16 are provided.
[0026]
The A / D decoder 10 converts an image input from the in-vehicle camera 4 from analog data to digital data and records it in the input frame memory 11.
[0027]
The microcomputer 7 receives a user control signal input via the touch panel and the position information detected by the position information detection unit 5. The microcomputer 7 extracts an image such as a road sign from an image recorded in the work memory 12 via the data bus 17 and records position information where the image such as a road sign is captured in the external storage device 15. Further, the microcomputer 7 outputs an image such as a road sign to the output frame memory 13, and the current vehicle position information detected by the position information detection unit 5 and the position information of the road sign image stored in the external storage device 15. And the travel distance is obtained and output to the output frame memory 13. Further, the microcomputer 7 outputs a sound / sound output command to the sound / sound output unit 9 when an image such as a road sign is extracted and displayed on the monitor 3. Further, the microcomputer 7 issues an image correction command to the image correction unit 8 and causes the image correction unit 8 to correct the car navigation image.
[0028]
The D / A encoder 14 has a role of converting an image such as a road sign and travel distance information recorded in the output frame memory 13 into analog data and outputting the analog data to the image composition unit 16.
[0029]
The navigation signal of the car navigation device 2 is input to the image correction unit 8. The image correction unit 8 corrects the car navigation image (navigation image) in accordance with an image correction command from the microcomputer 7. FIGS. 2A to 2C show examples of correcting a car navigation image. In accordance with an image correction command, the car navigation image is compressed in the vertical direction as shown in FIG. Or compressed in the horizontal direction as shown in FIG. In addition, the image correction unit 8 performs mask processing 19 on a part of the car navigation image without compressing the car navigation image in accordance with an image correction command from the microcomputer 7, thereby converting the car navigation image into a car navigation image. The image is corrected so that an image such as a road sign displayed so as to be superimposed can be easily seen.
[0030]
The image composition unit 16 displays an image such as a road sign and travel distance information on the car navigation image corrected by the image correction unit 8 as shown in FIGS.
[0031]
The sound / speech output unit 9 warns (notifies) the driver with a specific sound or sound that an image such as a road sign has been detected in response to a sound / speech output command from the microcomputer 7. If the microcomputer 7 is provided with a function (artificial intelligence function) for judging the meaning content of the extracted image such as a road sign, the sound / voice is output in accordance with the sound / voice output command of the microcomputer 7. The output unit 9 may read out the name and description of a road sign or the like.
[0032]
The road sign display device 1 described above shows a case where the car navigation device 2 does not perform processing equivalent to the processing performed by the image correction unit 8. When an image processing unit (not shown) of the car navigation device 2 performs an image correction process such as a road sign, the image correction unit 8 is not provided, and the image processing unit of the car navigation device 2 performs the image correction process. It is good. Further, when the image captured by the in-vehicle camera 4 is digital data 14, the A / D decoder 10 is not necessary, and a digital processing unit that performs digital synthesis processing may be provided before the D / A encoder. Is possible. Furthermore, instead of the microcomputer 7, dedicated hardware such as a DSP or an ASIC may be used.
[0033]
Next, display processing of an image such as a road sign in the microcomputer 7 will be described using the flowchart shown in FIG. The display processing of an image such as a road sign is composed of three major processes including a road sign recognition / recording process, a record organizing process, and an additional information processing / image output process.
[0034]
First, the microcomputer 7 outputs an image correction command to the image correction unit 8 to cause the image correction unit 8 to correct the car navigation screen (step S1). Next, the microcomputer 7 detects the road sign etc. from the image captured by the in-vehicle camera 4 in consideration of the appearance information such as the road sign set in advance, and determines the position / size on the screen, etc. An image such as a sign is extracted (step S2). At this time, the microcomputer 7 may determine the meaning of the extracted road sign or the like. For example, the microcomputer 7 preliminarily holds appearance information such as a road sign and its meaning content as sample data, thereby specifying the meaning content such as a corresponding road sign from the appearance of the detected road sign image. It becomes possible. In addition, a method of detecting a road sign or the like based on an artificial shape, for example, a geometric shape that does not exist in nature from an image captured by the in-vehicle camera 4, or a method of detecting by color information is used. May be.
[0035]
The process of detecting a road sign or the like from the screen imaged by the in-vehicle camera 4 is a technique known to those skilled in the art, and is disclosed in Japanese Patent Laid-Open Nos. 5-174294 and 6-36183. Since this is a technique, description here is omitted.
[0036]
After that, the microcomputer 7 cuts out the extracted image such as a road sign, and performs processing such as lightness / color tone adjustment, shaping processing such as enlargement / reduction, image compression, etc. as necessary, and outputs the frame memory 13 and the external storage. The device 15 is caused to record an image such as a road sign (step S3). The image such as a road sign stored in the output frame memory 13 and the external storage device 15 does not necessarily have to be an image captured by the in-vehicle camera 4. You may synthesize it on the spot.
[0037]
Further, if necessary, the microcomputer 7 outputs a sound / sound output command to the sound / sound output unit 9 and causes the sound / sound output unit 9 to issue a warning by sound / sound. The warning by sound / voice can surely notify the driver that a road sign or the like has been detected. When a warning is given by voice, the microcomputer 7 may generate a name / meaning content such as a road sign by voice.
[0038]
Thereafter, the microcomputer 7 detects the travel distance for each road sign or the like extracted from the position information detection unit 9 based on the position information, and stores it in the output frame memory 13 in association with each road sign or the like image (step S4).
[0039]
Moreover, the microcomputer 7 determines the operation content etc. which the user operated with the touch panel from the control signal input by the touch panel (step S5). For example, when the user touches a specific road sign etc. screen for a certain time or more with the touch panel, the microcomputer 7 interprets that the user wants to delete the corresponding road sign etc. image, and the road The image composition unit 16 is instructed to delete the image such as a sign from the monitor 3 and the corresponding image such as a road sign is deleted from the external storage device 15. This process is not necessarily performed after the road sign recognition / recording process (steps S1 to S4). When deleting images such as road signs according to the touch time of the touch panel, the other series of processes may be stopped while this process is being executed, so the process is performed independently of the other processes. You may do it.
[0040]
Thereafter, the microcomputer 7 determines the travel distance of the road sign etc. image already displayed on the monitor 2 and instructs the image composition unit 16 to delete the road sign etc. image showing a value of a certain distance or more from the monitor 3. At the same time, the corresponding image such as a road sign is deleted from the external storage device 15. When the time information detection unit is used instead of the position information detection unit 5 and the elapsed time is displayed on the monitor 3 in association with the image such as a road sign, the image such as the road sign that has passed a predetermined time is deleted. Control may be performed.
[0041]
The microcomputer 7 deletes the old road sign etc. image based on the mileage, elapsed time, etc. determined in advance by the user etc., so that the user wants the road sign etc. image to be displayed on the monitor 3 only for the desired period, It can be stored in the external storage device 15.
[0042]
If the microcomputer 7 can determine the meaning content of the road sign, etc., the road condition etc. is automatically judged by the time-dependent change of the extracted road sign etc., and the old road sign not suitable for the road condition It is also possible to set to delete the same image. For example, when the microcomputer determines that the speed limit has been changed from an image such as a road sign, it is possible to set to delete the old road sign image. In addition, when a road sign indicating the same content is extracted at regular intervals, such as a road sign indicating a speed limit, the microcomputer 7 determines that the similar information has been updated, and the old road sign It is also possible to set to delete the same image.
[0043]
Thereafter, the microcomputer 7 acquires the current position of the vehicle from various signals (information signal, vehicle control signal, etc.) from the position information detector 5 and other vehicles (step S7), and displays the acquired current position on the monitor 3 By comparing the position information for each image such as a road sign, the current travel distance is obtained and updated and recorded in the external storage device 15, and the travel distance information displayed on the monitor 3 is updated (step S8). ). A database can be constructed in the external storage device 15 or the like by linking images such as road signs and travel distances. Finally, the microcomputer 7 extracts an appropriate number of images such as road signs from the constructed database, or extracts only images such as special road signs and displays the images such as road signs on the monitor 3 (step S9). .
[0044]
The number and types of images to be displayed on the monitor 3 may be set in advance by the user. Moreover, when displaying, you may change the display size of images, such as a road sign, by automatic adjustment or manual adjustment by a user. Furthermore, when the characters are small and difficult to read, the user can enlarge the image such as a road sign by touching the corresponding road sign image, or enlarge the entire image by operating a separately provided joystick or the like. It may be possible to reduce the size. Further, the display position and display order of images such as road signs may be changed by direct or indirect instructions from the user.
[0045]
As described above, by using the road sign display device 1 according to the present invention, an image such as a road sign can be displayed on the monitor 3 together with the image for the car navigation system, so that the user only looks at the monitor. Thus, it is possible to acquire car navigation information and information such as road signs.
[0046]
Further, by displaying the distance traveled and the elapsed time in association with each road sign image on the monitor 3 or storing them in the external storage device 15, the user can effectively manage the road sign image based on the information associated with the user. -Since it can be utilized, it is possible to improve convenience for the user.
[0047]
In addition, since it is possible to adjust the period for deleting the image such as the road sign from the monitor 3 or the external storage device 15 using the travel distance or the elapsed time, the image such as the road sign is optimal according to the user's preference. It is possible to display on the monitor 3 only for a period, and it is also possible to set the number of times of warning and the period by the sound / voice output unit 9.
[0048]
【The invention's effect】
By using the road sign display device according to the present invention, an image such as a road sign can be displayed together with the car navigation image on the display means. Therefore, the user can only view the car navigation information, the road sign, etc. by looking at the display means. It becomes possible to acquire information.
[0049]
Further, the road sign etc. image on the display means based on the information associated with the user by associating the travel distance and the elapsed time with each road sign etc. image and displaying them on the display means or storing them in the storage means. Since it is possible to adjust the display period and the like, it is possible to improve user convenience.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a display device for road signs and the like according to the present invention.
FIGS. 2A and 2B are diagrams showing an image corrected by an image correction unit, in which FIG. 2A shows an image obtained by longitudinally compressing an image for car navigation, and FIG. (C) shows an image obtained by performing mask processing on a part of the car navigation image.
FIG. 3 is a flowchart showing a display process of road signs and the like of the display apparatus for road signs and the like according to the present invention.
[Explanation of symbols]
1 Display device for road signs, etc. 2 Car navigation device 3 Monitor (display means, warning means)
4 vehicle-mounted camera 5 position information detection unit 7 microcomputer (road sign etc. extraction means, distance calculation means, elapsed time calculation means)
8 Image correction unit (image output means)
9 Sound / Voice output (Warning means)
12 Work memory 15 External storage device (recording means)
16 Image composition unit (image output means)

Claims (12)

Road sign etc. extracting means for extracting road signs etc. from the image taken by the in-vehicle camera,
Recording means for recording an image such as a road sign extracted by the road sign etc. extracting means;
Display means for displaying a car navigation image;
Image output means for performing image editing so that the car navigation image and the image such as the road sign recorded in the recording means can be simultaneously displayed on the display means, and outputting to the display means. Display device for road signs. The image output means arranges the road sign image in a space formed by reducing the size of the navigation image in a horizontal direction or a vertical direction, and outputs the image to the display means. Item 4. A display device for road signs and the like according to item 1. A distance calculating means for calculating a travel distance from a place where the road sign or the like is imaged;
The image output unit edits an image so that the travel distance calculated by the distance calculation unit for each image such as a road sign can be displayed together with the image such as the road sign, and outputs the edited image to the display unit. Item 3. A display device for road signs, etc. according to item 1 or item 2. 5. The road according to claim 4, wherein the image output unit performs image editing so that the road sign or the like whose travel distance calculated by the distance calculation unit is not less than a predetermined distance is not displayed on the display unit. Signs and other display devices. An elapsed time calculating means for calculating an elapsed time from when the road sign or the like was imaged,
The image output means edits an image so that the elapsed time calculated by the elapsed time calculation means for each image such as a road sign can be displayed together with the image such as the road sign, and outputs the edited image to the display means. The display device for a road sign or the like according to claim 1 or 2. The image output means performs image editing so that the display means does not display an image of the road sign or the like whose elapsed time calculated by the elapsed time calculating means has exceeded a predetermined time. 4. A display device for road signs and the like according to 4. The road sign according to any one of claims 1 to 6, further comprising warning means for warning a driver when a road sign or the like is extracted by the road sign or the like extraction means. Display device. 8. The road sign display device according to claim 7, wherein the warning means issues a warning by sound or voice. The display means has a touch panel function,
When the road sign image displayed on the display means is touched more than a certain value, the display means identifies the touched road sign image, and the image output means identifies the identified road sign image. The road sign according to any one of claims 1 to 8, wherein image editing is performed so that the display means does not display the image, and the recording means deletes the identified image such as the road sign. Equivalent display device. The image output means performs image editing so that an image such as a road sign can be enlarged or reduced and displayed on the display means in accordance with an operation of a touch panel of the display means. Road sign display device. An image of a road sign or the like indicating a road sign or the like is extracted from an image captured by an in-vehicle camera, the extracted image of the road sign or the like is recorded in a recording unit, and the road sign or the like image recorded in the recording unit and An image display method for an image of a road sign or the like, wherein the image is edited so that a car navigation image can be simultaneously displayed on a display means and displayed on the display means. 12. The road sign according to claim 11, wherein in the image editing, the image such as the road sign is arranged in a space formed by reducing a size of the navigation image in a horizontal direction or a vertical direction. An image display method for equal images.

Images