JP3945333B2 - Imaging display system - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an imaging display system that displays on a display device image frames output at predetermined time intervals from an imaging device attached to a moving body.
[0002]
[Prior art]
As devices that support driving of a vehicle such as an automobile, an in-vehicle camera and an in-vehicle display device that displays an image captured by the in-vehicle camera are used. For example, the rear of the vehicle is photographed with an in-vehicle camera, and the garage is placed while checking the rear of the vehicle with the in-vehicle display device. In addition, the left and right front and / or left and right rear of the vehicle are photographed by the in-vehicle camera, and the course is changed while confirming the left and right front and / or the left and right rear by the in-vehicle display device.
[0003]
[Problems to be solved by the invention]
The vehicle-mounted camera or the vehicle-mounted display device may be in a fixed state (hereinafter referred to as “freeze”) due to a problem that the image displayed on the vehicle-mounted display device does not change. When the display on the in-vehicle display device freezes, even if the vehicle is moving, the displayed image does not change, so driving cannot be supported. However, since there is no method for detecting a freeze and there is no method for notifying the occurrence of the freeze, the driver may continue driving without noticing the freeze.
[0004]
The present invention has been made in view of such circumstances, and an object thereof is to provide an imaging display system capable of detecting and notifying that an output image of an imaging apparatus has not changed (freeze).
[0005]
In addition, another object of the present invention is to provide an imaging display system capable of performing freeze notification more accurately in consideration of the state of another imaging device.
[0006]
Another object of the present invention is to provide an imaging display system that can detect a stop of a moving body and perform freeze notification more accurately.
[0007]
Another object of the present invention is to provide an imaging display system capable of outputting a notification image to a display device to quickly notify a freeze.
[0008]
Another object of the present invention is to provide an imaging display system capable of promptly informing a freeze in a state where an output message of an imaging device is displayed while synthesizing and displaying a notification message on a display device. To do.
[0009]
[Means for Solving the Problems]
An imaging display system according to a first aspect of the present invention is an imaging display system that displays image frames output at predetermined time intervals from an imaging device attached to a moving body on a display device, and detects difference between successive image frames. And means for notifying that the image frame output from the imaging device has not changed when no difference is detected.
[0010]
An imaging display system according to a second aspect of the present invention is an imaging display system that displays image frames output from a plurality of imaging devices attached to a moving body at predetermined time intervals on a display device, and images differences between successive image frames. Difference detection means for detecting each device, and notification means for notifying that an image frame output from the some imaging device is not changed when no difference is detected in some imaging devices. Features.
[0011]
The imaging display system according to a third aspect of the present invention is the first or second aspect of the invention, further comprising speed detecting means for detecting the moving speed of the moving body, wherein the difference detecting means is configured such that the moving speed of the moving body is equal to or lower than a predetermined speed. The feature is that the detection of the difference should be stopped.
[0012]
An imaging display system according to a fourth invention is characterized in that, in any one of the first to third inventions, the notification means is configured to output a notification image and a display instruction for the notification image to the display device. To do.
[0013]
In any one of the first to third inventions, the imaging display system according to a fifth invention is capable of composite display in which the display device displays characters superimposed on an image frame, and the notification means includes a notification message and It is characterized in that a combined display instruction for the notification message is output to the display device.
[0014]
In the first invention, the difference detection means detects a difference between successive image frames output from the imaging device. For example, every time an image frame is output from the imaging device, a difference from the previously output image frame is detected. For example, the difference is detected by detecting a difference in luminance values of pixels corresponding to sampling points set on the image frame. If no difference is detected, the display on the display device may be fixed without changing (hereinafter referred to as “freeze”), so the image frame output from the imaging device changes by the notification means. Notify that you have not. For example, a notification message is displayed on the display device or a notification sound is output from a speaker.
[0015]
In the second invention, the difference detection means detects a difference between successive image frames output from the respective imaging devices. An imaging device in which no difference is detected may be frozen. However, for example, when the imaging device is attached to a vehicle (moving body) and the vehicle is stopped, it is considered that no difference is detected in all the imaging devices. When a difference is not detected by some imaging devices, it is highly likely that the some imaging devices are frozen, so that an image frame output from some imaging devices is not changed by the notification unit. Is notified. For example, if only one of the three imaging devices has not been detected, a notification is made that this one image frame has not changed.
[0016]
In the third invention, the moving speed of the moving body is detected by the speed detecting means. For example, when the imaging device is attached to a vehicle (moving body), the moving speed of the vehicle is detected from vehicle speed pulses sent to a speedometer or speed data based on the vehicle speed pulses. When the moving speed of the vehicle is equal to or lower than the predetermined speed, the difference detection unit stops detecting the difference. For example, when the vehicle is stopped, a difference between successive image frames is often not detected, and it is difficult to distinguish from freeze. However, when the vehicle is moving, a difference between successive image frames should be detected, and the freeze can be reliably detected. In the second invention, if a difference in the image frame is detected due to the passing of a passerby near the stationary moving body, the moving body is considered to be moving, but in the third invention, It is possible to reliably detect the stop of the moving body based on the moving speed. The predetermined speed is not limited to the case where the speed is zero, and includes a low speed with little change in the image frame.
[0017]
In the fourth invention, when a difference between successive image frames is not detected, the notification means outputs a notification image and a display instruction for the notification image to the display device, and the notification image is displayed on the display device. When a speaker is installed in the vehicle (moving body) or a notification LED (light emitting diode) is installed on an instrument panel or the like, a notification sound is output from the speaker or the notification LED is turned on by the notification means. It is also possible. A driver of a vehicle (moving body) who notices a freeze by notification performs a resetting operation such as turning on the power of the imaging device or the imaging display system. It is also possible to include a reset operation instruction in the notification image.
[0018]
In the fifth invention, when the difference between successive image frames is not detected, the notification means outputs a notification message and a combined display instruction of the notification message to the display device, and the display device displays the notification message on the image frame. Overlaid. The driver of the vehicle (moving body) sees the notification message while looking at the image frame. Even when the erroneous detection of freeze or the freeze is temporary, it is possible to continue to view the image frame output from the imaging device.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the present invention will be specifically described with reference to the drawings showing embodiments thereof. In this description, an automobile (moving body) including four cameras (imaging devices) and a display device that displays images captured by the cameras will be described as an example. The video imaged by the camera includes, for example, 30 image frames per second.
[0020]
FIG. 1 shows an example of an imaging display system according to the present invention. The imaging display system includes four cameras (imaging devices) 10, 20, 30, 40 using a CCD (charge coupled device), a display device 60 such as a liquid crystal display or a CRT (cathode-ray tube) display, and a camera. 10, 20, 30, 40 and a monitoring unit 50 to which the display device 60 is connected. The cameras 10, 20, 30, 40, the display device 60, and the monitoring unit 50 can be connected by a unique wiring or can be connected via an in-vehicle LAN (local area network).
[0021]
The monitoring unit 50 includes first and second frame buffers 12 and 14, 22 and 24, 32 that temporarily store image frame data (hereinafter referred to as frame data) input from the cameras 10, 20, 30, and 40, respectively. And 34, 42 and 44. The first frame buffers 12, 22, 32, and 42 are connected to the cameras 10, 20, 30, and 40 and the second frame buffers 14, 24, 34, and 44, respectively, and the second frame buffers 14, 24, 34, and 44 are The display device 60 is connected.
[0022]
The first frame buffers 12, 22, 32, and 42 store frame data newly input from the cameras 10, 20, 30, and 40 (hereinafter, the current frame data), and the second frame buffers 14, 24, Reference numerals 34 and 44 respectively store the frame data (hereinafter referred to as the previous frame data) previously input from the cameras 10, 20, 30 and 40 and stored in the first frame buffers 12, 22, 32 and 42, respectively. The The frame data previously stored in the second frame buffer 14, 24, 34, 44 is sent to the display device 60 and displayed.
[0023]
The first and second frame buffers 12 and 14, 22 and 24, 32 and 34, 42 and 44 are connected to the comparison units 16, 26, 36 and 46, respectively. The comparison units 16, 26, 36 and 46 compare the current and previous frame data stored in the first and second frame buffers 12 and 14, 22 and 24, 32 and 34, 42 and 44, respectively. It operates as a means for detecting the difference between the two (difference detecting means). For example, the brightness value of each pixel in the current and previous image frames is compared to detect a difference between both image frames. In this description, the luminance values of the pixels corresponding to the sampling points set on the frame are compared.
[0024]
The update of the first frame buffers 12, 22, 32, 42 and the second frame buffers 14, 24, 34, 44 and the comparison by the comparison units 16, 26, 36, 46 are obtained from the cameras 10, 20, 30, 40. It is executed in response to input of frame data.
[0025]
The comparison units 16, 26, 36, 46 are connected to the determination unit 52, and send the detection result of the difference between the current and previous frame data to the determination unit 52. If no difference in the frame data is detected, there is a possibility that freezing has occurred. The determination unit 52 confirms whether a difference is detected for all the cameras. When no difference is detected for all the cameras, the determination unit 52 determines that no difference is detected because the vehicle is stopped. For example, when a difference is not detected for only one camera, the determination unit 52 determines that the camera is frozen.
[0026]
The determination unit 52 is connected to the display processing unit 54 and sends a detection signal to the display processing unit 54 when a freeze is detected. The display processing unit 54 is connected to the display device 60 and outputs a warning message (notification message) and a warning message display instruction to the display device 60 when receiving a detection signal from the determination unit 52. The caution message is stored in a ROM (read only memory) (not shown) of the display processing unit 54. The determination unit 52 and the display processing unit 54 operate as means (notification means) for notifying that the image frames output from the some cameras have not changed when no difference is detected by some cameras.
[0027]
In the present description, the display device 60 can perform composite display (superimpose display) in which characters (caution messages) are displayed in an overlapping manner on an image frame, and the display processing unit 54 can display warning messages and superimpose displays. An instruction is sent to the display device 60 to display a caution message in a superimpose manner. The detection signal output from the determination unit 52 includes information on the camera in which the freeze is detected. When the display device 60 displays images of a plurality of cameras, the display processing unit 54 detects the freeze. Instruct the camera to superimpose and display a warning message on the image portion of the camera. For example, as shown in FIG. 2, when the display screen 60a of the display device 60 is divided into four areas according to the camera, a caution message is superimposed on the area corresponding to the camera where the freeze is detected. To instruct.
[0028]
Next, freeze detection and attention display using the imaging display system according to the present invention will be described. An example of a freeze detection procedure is shown in FIG. 3, and an example of a caution display procedure is shown in FIG.
[0029]
After the current frame data is input from the camera (S10), the current and previous frame data are compared by the comparison unit (S12). In the example of FIG. 1, the current frame data input from the cameras 10, 20, 30, and 40 are stored in the first frame buffers 12, 22, 32, and 42, respectively. The previous frame data is stored in the second frame buffers 14, 24, 34, 44. The comparison units 16, 26, 36, and 46 compare the current and previous frame data stored in the first and second frame buffers 12 and 14, 22 and 24, 32 and 34, 42 and 44, respectively.
[0030]
When there is no difference between the current and previous frame data (S14: NO), the comparison units 16, 26, 36, and 46 detect the fixation (freeze) of the video displayed on the display device 60 (S16). If there is a difference in the previous frame data (S14: YES), it is determined that the video is not fixed. The fixed detection result of the video is sent to the determination unit 52.
[0031]
If there is a camera in which video fixation is detected (S30: YES in FIG. 4) and video fixation is not detected for all cameras (S32: NO), the determination unit 52 detects video fixation. The determined camera is determined to be frozen, the display processing unit 54 is instructed to display a caution message, and the caution message is displayed on the display device 60 (S34).
[0032]
Further, the determination unit 52 has a camera in which video fixation is detected (S30: YES), and if the video fixation of all cameras is detected (S32: YES), the vehicle is stopped. It is determined that the video is fixed to, and no caution message is displayed. Further, the determination unit 52 does not display a caution message when there is no camera in which video fixation is detected (S30: NO).
[0033]
Thereafter, each time frame data is input from the cameras 10, 20, 30, 40 to the monitoring unit 50, the processes shown in FIGS. 3 and 4 are repeated. However, the processing shown in FIGS. 3 and 4 can be repeated every time frame data is input, for example, three times.
[0034]
In the above-described embodiment, the freeze is detected based on the output image frames of the cameras 10, 20, 30, and 40. However, it is also possible to detect the freeze in consideration of the moving speed of the vehicle. FIG. 5 shows an example of an imaging display system that detects the freeze in consideration of the moving speed of the vehicle. The imaging display system shown in FIG. 5 is obtained by connecting a vehicle speed sensor 62 that detects the moving speed of a vehicle (moving body) to the imaging display system shown in FIG. The vehicle speed sensor 62 is connected to the determination unit 52 and sends to the determination unit 52 vehicle speed pulses corresponding to the rotation of the wheels or speed data based on the vehicle speed pulses. In this description, the speed data is sent to the determination unit 52. If the speed data is zero or less than a predetermined speed, the determination unit 52 determines that the vehicle is stopped and stops detecting the freeze.
[0035]
An example of a caution display procedure is shown in FIG. The determination unit 52 detects the stop of the vehicle based on the speed data sent from the vehicle speed sensor 62 (S20). When the vehicle is stopped (S20: YES), it is natural that the output images of the cameras 10, 20, 30, and 40 are fixed without being fluctuated, so that the detection of freeze is not performed. At this time, it is of course possible to stop the processing (S12, S14, S16) shown in FIG. When the vehicle is moving (S20: NO), a caution display process is performed as in FIG. 4 (S30, S32, S34).
[0036]
In the above-described embodiment, a plurality of cameras 10, 20, 30, and 40 are used. However, for example, a single camera of the imaging display system using the vehicle speed sensor 62 shown in FIG. . FIG. 6B shows an example of a caution display procedure when one camera is used. In the example of FIG. 6B, the video fixing detection (S32) of all the cameras shown in FIG. 6A is not performed, and the video fixing is detected while the vehicle is moving (S20: NO). (S30: YES), a warning message is displayed (S34).
[0037]
In the above-described embodiment, one camera performs shooting in one direction. However, for example, an optical element such as a prism may be used to perform shooting in two directions in the left and right directions. In this case, it is possible to divide the sampling points of the image frame into two areas for left and right, and detect the fixed display image for each area. Further, the first and second frame buffers can be divided into a left buffer and a right buffer, and it is possible to detect the fixed display image for each buffer. Alternatively, it is of course possible to process the images in the left and right directions as one image.
[0038]
In the embodiment described above, the cameras 10, 20, 30, 40 and the display device 60 are connected to the monitoring unit 50. However, the monitoring unit 50 is provided inside the display device 60, and the cameras 10, 20, 30, 40 are displayed. A configuration for connecting to the device 50 is also possible. Further, the second frame buffers 14, 24, 34, and 44 are not directly connected to the display device 60, but are connected to the display processing unit 54, and the display processing unit 54 performs a superimpose display process of the caution message. It is also possible to output an image frame on which is superimposed on the display device 60.
[0039]
In the above-described embodiment, an automobile is described as an example of a moving body. However, the present invention can be applied to an arbitrary moving body such as a vehicle such as a train or a ship or an airplane.
[0040]
【The invention's effect】
According to the first invention, when a difference between successive image frames output from the imaging device is not detected, it can be notified that the display screen may be frozen. It is possible to promptly notify the display screen freeze and promptly prompt the recovery process from the freeze.
[0041]
According to the second aspect of the invention, by reporting the freeze when the difference between successive image frames output from some of the imaging devices is not detected, a false report of the freeze when no difference is detected due to the stop of the moving body Knowledge can be prevented. Freeze notification can be performed more accurately.
[0042]
According to the third invention, when the moving speed of the moving body becomes equal to or lower than the predetermined speed, the detection of the difference between the image frames is stopped. Can be prevented. Freeze notification can be performed more accurately.
[0043]
According to the fourth invention, by displaying the notification image on the display device, it is possible to promptly notify the driver or the like of the vehicle of the freeze.
[0044]
According to the fifth aspect, by composing and displaying the notification message on the display device, it is possible to promptly notify the vehicle driver or the like of the freeze while the image frame is displayed.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating an example of an imaging display system according to the present invention.
FIG. 2 is a diagram showing an example of a display screen of the display device.
FIG. 3 is a flowchart illustrating an example of a freeze detection procedure.
FIG. 4 is a flowchart illustrating an example of a caution display procedure;
FIG. 5 is a block diagram showing another example of the imaging display system according to the present invention.
FIG. 6 is a flowchart showing another example of a caution display procedure;
[Explanation of symbols]
10, 20, 30, 40 Camera (imaging device)
16, 26, 36, 46 Comparison part (difference detection means)
50 monitoring unit 52 determination unit (informing means)
54 Display processing section (notification means)
60 display device 62 vehicle speed sensor (speed detection means)

Claims (5)

In an imaging display system for displaying image frames output at predetermined time intervals from an imaging device attached to a moving body on a display device,
A difference detecting means for detecting a difference between successive image frames;
An imaging display system comprising: notifying means for notifying that an image frame output from an imaging device has not changed when no difference is detected. In an imaging display system that displays image frames output at predetermined time intervals from a plurality of imaging devices attached to a moving body on a display device,
A difference detecting means for detecting a difference between successive image frames for each imaging device;
An imaging display system comprising: an informing means for informing that an image frame output from the some imaging device has not changed when a difference is not detected by some imaging devices. Comprising speed detecting means for detecting the moving speed of the moving body;
3. The imaging display system according to claim 1, wherein the difference detecting means is configured to stop detecting the difference when the moving speed of the moving body is equal to or lower than a predetermined speed. The imaging display system according to claim 1, wherein the notification unit is configured to output a notification image and a display instruction for the notification image to the display device. The display device is capable of composite display for displaying characters superimposed on an image frame,
The imaging display system according to claim 1, wherein the notification unit is configured to output a notification message and a combined display instruction of the notification message to the display device.

Images