JPH0636183A - Image recorder - Google Patents

Abstract

PURPOSE:To make it possible to easily confirm road guides and marks, etc., and to relax the tension of a driver to his misjudgment. CONSTITUTION:This image recorder is provided with a frame image storage part 3 for storing the image fetched from a video camera 1 for a fixed time, a detection part 6 detecting prescribed information or road information on road guides and marks, etc., from the image stored by the frame image storage part 3, a moving image storage part 5 for storing the before and after image including the image where the road information is detected when the information is detected by the detection part 6 and a display part 12 displaying the image stored by the moving image storage part 5 as a moving image or a static image by a frame unit.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image recording apparatus capable of holding and displaying a moving image captured by a video camera mounted on a vehicle for a certain period of time and compensating for the misjudgment of a road guide plate or a sign. .

[0002]

2. Description of the Related Art In recent years, navigation devices have started to be used to notify a driver of a vehicle of a current position, a traveling direction, and the like. However, the driver must identify and determine the road guides and signs that can be known while driving. Generally, when traveling on an unguided road, information obtained from the road guide or the sign is required, which may cause tension to be careful not to miss the road guide or the sign.

By the way, PRL85-73 P35-46 is used as a technique for detecting and recognizing characters in a scene image, and proc. IARP Workshop on MAchine Vision Applic is used as a technique for detecting and recognizing road signs.
ation Nov.28-30,1990, Tokyo P417-420, Hei 3 Jyosho Zendai 3D-6. These technologies are for obtaining road information, and are not for supplementing the misjudgment of road signs and signs.

[0004]

As described above, when traveling on an unguided road, it is necessary to be careful so as not to overlook the road guide and road signs. However, it does not relieve the driver's tension about the failure to see the road guidance.

The present invention has been made in view of the above points, and provides an image recording apparatus capable of easily confirming road guides, signs, etc., and relieving the driver's strain of being overlooked. The purpose is to

[0006]

SUMMARY OF THE INVENTION According to the present invention, first storage means for storing an image captured from a video camera for a certain period of time, and predetermined information from the image stored by the first storage means. A second detecting means for detecting and a second means for, when the predetermined information is detected by the detecting means, accumulating images before and after including the image in which the information is detected.
The storage means and the display means for displaying the image stored by the second storage means.

[0007]

According to such a structure, even if the driver misses the road guide or the sign, the image including the road guide or the sign can be displayed on the display for confirmation.
Therefore, according to the present invention, the driver's tension with respect to unsightlyness such as road guidance is relieved, and even if he / she fails to see the roadway, he / she can confirm the roadway guidance without panicking, so that safe driving can be accelerated.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the arrangement of the image recording apparatus according to the embodiment. In the embodiment, a moving image of a traveling direction is captured by a video camera from a moving vehicle, and when a specific character or symbol or color information is detected from the captured moving image, the moving images before and after that are detected. An image recording device capable of supplementing an oversight of a road guide plate or a sign by holding and displaying for a certain time will be described as an example.

As shown in FIG. 1, the image storage device includes a video camera 1, an A / D conversion unit 2, a frame image storage unit 3,
The image compression unit 4, the moving image storage unit 5, the detection unit 6, the recognition unit 7,
Information comparison unit 8, notification unit 9, input unit 10, image expansion unit 1
1, a display unit 12, a coordinate input unit 13, a memory unit 14, and a control unit 15.

The video camera 1 is for picking up a moving image in the traveling direction from a moving vehicle and is, for example, a color CCD camera. The video camera 1 is installed on the roof of a vehicle, a hood, or the like, and captures an image in the traveling direction of a vehicle without blurring, for example, with an electronic shutter of 1/1000 seconds, and for example R
Color separation into three primary colors G, B (red, green, blue), and output to the A / D converter 2 as electric signals. The A / D conversion unit 2 is an R / G converter for moving images captured by the video camera 1.
The B component is each quantized into an 8-bit digital image, for example.

The frame image storage section 3 includes an A / D conversion section 2
The moving image quantized by, that is, one frame of a time-series continuous image captured at 1/30 second intervals, for example, is temporarily stored.

The image compression unit 4 processes the image stored in the frame image storage unit 3 into, for example, the international standard JPEG (Joint).
The image is compressed using an image compression processing technique based on the Photographic Experts Group) system and stored in the moving image storage unit 5. That is, since the digital image information has a large amount of information, a huge storage device is required without compression processing. For example, if the image size is 512 x 512 pixels and 1 byte / pixel, the R, G, and B color components are 768 Kbytes / frame, and the image data for 1 second is 30 times as much (30 frames) as approximately 23 Mbytes. Become. Therefore, if the image is not compressed, the storage capacity required for even a few minutes of information becomes enormous. The above-mentioned JPEG method is basically a still image compression technique, but since it can be processed at high speed, it can be pseudo-used for moving picture compression processing. As a moving image compression technology, the international standard MPEG (Mo
tion Picture Experts Group) method may be used. An image decompression unit 11, which will be described later, performs a process that is the reverse of the compression process performed by the image compression unit 4, and decompresses the compressed image. For such image compression / decompression technology, for example, Nikkei Electronics 1990.10.15 No.511.
It is described in P116-142. The moving image storage unit 5 is for storing a compressed continuous digital image for a certain period of time, and is composed of a storage device such as a hard disk.

The detection unit 6 processes the image information stored in the frame image storage unit 3, detects a road guide board and a road sign by extracting specific color information and line segment detection, and further guides the road. Detects characters and symbols written on boards and road signs.

The recognizing unit 7 recognizes the characters and symbols detected by the detecting unit 6 by comparing them with a reference pattern by, for example, a pattern matching method. For detecting and recognizing specific information on such roads, for example, proc. IARP Works
hop on MAchine Vision Application Nov.28-30,1990, T
The techniques described in okyo P417-420 and the 3rd Journal of Hei3 Jyosho can be used.

The information comparison unit 8 compares the recognition information (road information) about the characters and symbols obtained by the recognition unit 7 with preset specific characters and symbols (setting information),
It is determined whether or not the matching information exists, and if so, the notification unit 9 is notified of that fact. In response to the notification from the information comparison unit 8, the notification unit 9 outputs a notification message to the driver by, for example, a voice rule synthesis technique.

The input unit 10 uses the coordinate data input from the coordinate input unit 13 to generate characters and symbols for setting information by using, for example, an online character recognition technique, a keyboard and menu selection. The data shown is input to the information comparison unit 8. The image decompression unit 11 decompresses the compressed image stored in the moving image storage unit 5 into a displayable state and outputs it to the display unit 12.

The display unit 12 is a display device that displays the moving images stored in the moving image storage unit 5 in various forms. The coordinate input unit 13 is a coordinate input device, such as a transparent tablet, which is installed so as to overlap the display surface of the display unit 12. Since the display unit 12 and the coordinate input device 13 are overlapped with each other, a menu or the like displayed on the display unit 12 can be directly designated by using a finger, a stylus pen, or the like.

The memory unit 14 is a general-purpose memory for temporarily storing information. The memory unit 14 is provided with a road information detection flag indicating that road information has been detected, a pointer indicating an image stored in the moving image storage unit 5, and the like. The control unit 15 outputs various instructions to the above-mentioned units,
For example, a CPU that controls the operation of the entire apparatus. The control unit 14 sets the road information detection flag in the memory unit 14.

Next, the operation of the embodiment will be described with reference to the flow chart shown in FIG. here,
When the video camera 1 captures a moving image in the traveling direction from a vehicle traveling on an expressway, and when the interchange guide plate information is detected from the captured moving image, the moving images before and after that are retained for a certain period of time. An explanation will be given by taking as an example a process of supplementing an oversight of a guide board by displaying the information and obtaining notification information of an interchange confirmation used by the user in advance.

When the image storage device is activated, the control unit 15
Initializes each unit and sets the road information detection flag in the memory unit 14 to "0" indicating that road information is not detected (step S1).

An image of the traveling direction from the running vehicle is first captured by the video camera 1 at an interval of 1/30 second, and R.
Color separation into three primary colors of G and B, and A /
It is sent to the D conversion unit 2. The A / D conversion unit 2 uses, for example, R
Each of G and B is quantized into an 8-bit grayscale image and stored in the frame image storage unit 3 (step S2). This image is, of course, a moving image that is updated every 30 seconds.
The frame unit of / 30 seconds can be considered as a still image.

The stored image is compressed by the image compression unit 4 by, for example, the international standard JPEG system and stored in the moving image storage unit 5 (steps S3 and S4).
The accumulated moving image is updated by deleting the old frame every time a new frame is accumulated, and always accumulates moving image information for a fixed time. The accumulation of images in the moving image accumulation unit 5 is repeated until the road information detection flag becomes "1" (step S5), that is, until the road information is detected from the photographed image.

The moving picture in this case is mapped in the moving picture storage unit 5 as a moving picture first storage area as shown in FIG. 3, for example. The image compressing section 4 manages the latest frame and the oldest frame of the frame images which are continuously accumulated with a pointer. When a new frame is registered, the pointer of the latest frame is moved to the registration position and the pointer of the oldest frame is also incremented by one. When the pointer exceeds the moving image first storage area, the image compression unit 4 moves the pointer to the very beginning of the area. By rotating the pointer in the moving image first storage area in this way, between the areas indicated by the pointers of the oldest frame and the latest frame in the area,
There is always continuous moving image information at fixed time intervals.

On the other hand, in parallel with the above-mentioned moving image storage processing, the detection unit 6 uses the specific color information for the frame image stored in the frame image storage unit 3 to detect road information. Is performed (step S6). In many cases, the highway interchange guide plates to be detected are limited to green. The detection unit 6 detects a green region indicating an image portion of the interchange guide board by using, for example, a color clustering technique.

The color space of the R, G, B3 primary colors is represented by a three-dimensional space as shown in FIG. 4, and when each is 8 bits, each component can take a value from 0 to 255. Therefore, the ideal green color is when R = 0, G = 255, and B = 0. Normally, the color of the interchange guide plate taken from the video camera 1 is not the ideal green color, so the detection unit 6
Determines that the space within a certain distance from the ideal green, as indicated by the diagonal lines in FIG. 4, is green.

With such a color clustering technique, only the green region is detected from the image. The detected green region is pre-formed by a Hough transform, such as a guide plate.
The tongue-shaped area can be detected. Such a plate-like area detection technique is described in Technical Bulletin Vol.88 No.14 P9-16. The detection unit 6 checks the vertical / horizontal size ratio of the area on the green plate, and if there is no large error, detects it as a guide plate area candidate (step S6).

If the detection is confirmed (step S
7) The latest frame pointer in which the frame is recorded is stored in the memory unit 14 (step S8), and the road information detection flag is set to "1" (step S).
9).

Here, if the setting information is not input in advance (step S10), whether the predetermined number of frames (n) has been recorded in the first moving image recording area of the moving image storage unit 5 from the stored pointer. Frame images are sequentially accumulated while being discriminated. When an image of a predetermined frame is recorded in the first moving image recording area from the stored pointer (step S1
1), the control unit 15 takes out a predetermined frame image before and after the stored pointer and sets it as one scene in the moving image second storage area of the moving image storage unit 5 shown in FIG.
Yes (step S12). Further, the latest scene frame pointer in which the scene is recorded is stored in the memory unit 14.

When the scene image is stored in the moving image second storage area, the road information detection flag is set to "0" (step S13). In the same manner, the process returns to step S2 to accumulate the frame image and the scene image.

Up to n pieces of scene information can be stored,
Every time a new scene is accumulated, the old scene is deleted and updated, and a fixed number of scene information is always accumulated. Like the frame, the scene is also managed by the pointer of the latest scene and the pointer of the oldest scene, and when a new scene is registered, the pointer of the latest scene is moved there and the oldest scene is moved. The pointer of the scene is also advanced by one. When the pointer exceeds the moving image second storage area, the pointer is moved to the very beginning of the area. By rotating the pointer in the second moving image storage area in this way, a fixed number of continuous areas are always present in the area between the oldest scene pointer and the latest scene pointer in the area. Scene information exists.

By the way, the image expansion unit 11 reads the image of the compressed scene stored in the moving image second storage area of the moving image storage unit 5, performs expansion processing, and outputs it to the display unit 12. For example, as shown in FIG. 5, the display unit 12 displays a multi-screen image (scene) of the frame in which it is determined that the interchange guide plate has been detected. Further, it may be displayed as a moving image instead of a still image.

On the display screen shown in FIG. 5, an arbitrary frame image among the frames displayed on the multi-screen is displayed through the coordinate input unit 13 installed so as to overlap the display surface of the display unit 12. The screen is enlarged and displayed by touching. At this time, the control unit 15 determines a designated frame from the coordinate information obtained by touching, enlarges the frame, and outputs the frame to the display unit 12.

By touching a menu such as forward or rewind on the display screen, the front or rear frame is sequentially displayed from the currently displayed frame, and a guide plate, a sign or the like is displayed. You can retrieve the image that is the easiest to check the contents of.

Next, a process of setting the setting information in advance and obtaining the notification information of the road information confirmation will be described. For example, the input unit 10 recognizes a character handwritten by the coordinate input unit 13 using the online character recognition technique, and obtains road information of the interchange guideboard. Enter (Setting information). This setting information is registered in the information comparison unit 8.

When road information is detected from an image of a frame taken by the video camera 1 (steps S6, S
7) When the control unit 15 determines that the setting information is registered in advance (step S10), it instructs the detection unit 6 and the recognition unit 7 to recognize the road information (step S14).

The detection unit 6 detects the character information in the detected area of the interchange guide plate. Such character detection techniques are described in PRL 85-73 P35-46. The detected character string is recognized by the recognition unit 7 by a recognition technique such as pattern matching and is output as a corresponding character code.

The information comparing section 8 searches the character information obtained by the recognizing section 7 with the preset character information of "Urawa" and compares whether or not a matching character exists (step S15). . If there is a matching character, that is, if the desired road information is obtained, the number (representing the distance to the "Urawa" interchange) following the "Urawa" character on the information board is finally given. Is notified to the driver by the notification unit 10 by, for example, voice rule synthesis (step S16).

In the above embodiment, the video camera 1 is described as a color CCD camera,
Not only the CCD camera but also other image pickup devices such as an image pickup tube can be considered. The shutter speed is set in relation to the moving speed of the vehicle, and is not limited to 1/1000 second. Needless to say, the image capturing interval is not limited to 1/30 second. The color separation is not limited to the three primary colors of R, G, and B, and various color spaces such as hue, brightness, and saturation can be considered.

Further, the number of quantization bits of an image is not limited to 8 bits, and the image compression method is JPEG or MPEG.
Not limited to the method. The moving image storage unit 5 is not limited to the hard disk device, but may be an optical medium such as a magneto-optical disk device or a semiconductor memory device such as a RAM.

In the above embodiment, the input unit 10
In the above, the processing is executed based on the coordinate data input from the coordinate input unit 13, but a voice recognition function may be provided so that the data can be input based on the voice data.

[0041]

As described above, according to the present invention, from a moving image captured by a video camera mounted on a vehicle,
When information that matches specific characters, symbols, color information, etc. is detected, the moving images before and after that are held for a certain period of time and displayed, so that even if the road guide or sign is missed, it is displayed on the display. Can be displayed and confirmed.
Therefore, the driver's tension with respect to the failure to see the road guide or the like is relieved, and the road guide or the like can be confirmed without hesitation even if the driver fails to see the road, so that there is an effect of promoting safe driving.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an image recording apparatus according to an embodiment of the present invention.

FIG. 2 is a flowchart for explaining the operation of the embodiment.
To.

FIG. 3 is a diagram for explaining information stored in a moving image storage unit 5.

FIG. 4 is a diagram for explaining a color area in an R, G, B color space.

FIG. 5 is a diagram showing an example of a multi-screen display displayed on a display unit 12.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Video camera, 2 ... A / D conversion part, 3 ... Frame image storage part, 4 ... Image compression part, 5 ... Moving image storage part, 6 ... Detection part, 7 ... Recognition part, 8 ... Information comparison part , 9 ... Notification unit, 10
Input unit, 11 image expansion unit, 12 display unit, 13 coordinate input unit, 14 memory unit, 15 control unit.

Claims (1)

[Claims] 1. A first storage unit for storing an image captured from a video camera for a certain period of time, a detection unit for detecting predetermined information from the image stored by the first storage unit, and the detection. Second accumulating means for accumulating images before and after the image in which the predetermined information is detected by the means, and an image accumulated by the second accumulating means An image recording device, comprising: