JP4596257B2 - Diagnostic equipment for vehicles - Google Patents

Description

  The present invention relates to a vehicle diagnostic apparatus for diagnosing operations of a display and input operation means.

  As the vehicle travels, the current position is detected by GPS (Global Positioning System), etc., and the current position is displayed on the display device together with the road map, and an appropriate route from the current position to the destination is displayed. An in-vehicle navigation device that is set and guided by a display device, an audio output device, or the like contributes to an efficient and safe operation of the user.

  In recent years, the functions of electronic devices mounted on vehicles such as in-vehicle navigation have become more sophisticated. There are many things that cannot be identified.

  In view of this, a failure diagnosis device has been devised that sequentially stores information on electronic devices in a large-capacity storage device and analyzes the recorded information when investigating the cause to identify the cause (see Patent Document 1).

  Further, as a diagnostic device for input operation means such as a display or a touch panel, when the same coordinate value is continuously input for a predetermined time, a touch input for determining that an element corresponding to the coordinate value is defective A formula data processing device has been devised (see Patent Document 2).

JP 2005-043138 A Japanese Patent Laid-Open No. 05-298027

  In the example of Patent Document 1, the electrical signal of each electronic device can be recorded, but there is a problem that it is impossible to record how the output image of the display device looks to the user. For example, when a video signal is output from the control unit of the in-vehicle navigation device, but is not normally displayed on the screen due to a failure of the display device, it may not be recorded as a failure. Furthermore, it may not be detected even if operated by the user due to a failure of the input operation means, or it may remain detected even if the user does not operate, and whether the detected switch input information is actually operated by the user There is a problem that cannot be recorded.

  In the example of Patent Document 2, failure detection is possible when an element fails in an on state, but there is a problem that failure detection is not possible when an element fails in an open state. There is also a problem that failure detection cannot be performed when an abnormality occurs in the display on the display.

  Against the background of the above problems, an object of the present invention is to provide a vehicle diagnostic apparatus capable of accurately diagnosing operations of a display device and input operation means.

Means for Solving the Problems and Effects of the Invention

The present invention provides a vehicle diagnostic apparatus for solving the above-described problems. That is, a display device that is mounted on a vehicle and displays various types of information on the screen based on display instructions from the display control unit, an imaging unit that captures the screen display content of the display device at a predetermined timing, and the content of the display instructions Display content comparing means for comparing the screen display contents taken and display diagnosis means for diagnosing whether the operation of the display device is correctly performed based on the result of the display content comparison It is configured as a vehicle diagnostic device.

  In the present invention, the operation of the display is diagnosed by monitoring the display with a photographing means such as a camera. Specifically, a display instruction from the display control unit and an image obtained by photographing the display screen of the display are periodically recorded, for example. Then, by analyzing these recorded items, there is a problem in the operation of the display by checking whether there is a contradiction between the display instruction and the display screen of the display, that is, the display screen is in accordance with the display instruction. Can be determined.

In addition , a vehicle diagnostic apparatus for solving the above problems includes an input operation unit that is mounted on a vehicle and performs various input operations, an input operation detection unit that detects contents of the input operation, and an input operation at a predetermined timing. The operation of the input operation means is correctly performed based on the result of the input operation comparison, the image pickup means for photographing the surrounding area and the surroundings, the input operation comparison means for comparing the content of the detected input operation with the photographed input operation. And an input operation diagnosing means for diagnosing whether or not the vehicle is diagnosed.

  According to the present invention, an operation diagnosis of an input operation unit is performed by monitoring the input operation unit such as an operation switch by an imaging unit such as a camera. Specifically, the content of the detected input operation and an image obtained by capturing the input operation are periodically recorded, for example. Then, these recorded ones are analyzed, and it is determined whether there is a problem in the operation of the input operation means by checking whether there is a contradiction between the content of the detected input operation and the photographed input operation. be able to.

In addition , a vehicle diagnostic apparatus for solving the above problems includes a display device that is mounted on a vehicle and displays various information on a screen based on display instructions from a display control unit, and input operation means that performs various input operations. Input operation detecting means for detecting the contents of the input operation, screen display contents of the display at a predetermined timing, photographing means for photographing the input operation means and its surroundings, contents of the display instruction, and photographed screen display Display content comparing means for comparing the contents, display diagnostic means for diagnosing whether the operation of the display is correctly performed based on the result of the display content comparison, contents of the detected input operation and photographed input Input operation comparison means for comparing operations, and input operation diagnosis means for diagnosing whether the operation of the input operation means is performed correctly based on the result of the input operation comparison Configured as a vehicle diagnostic device for.

  The present invention performs operation diagnosis of input operation means such as a display and an operation switch by monitoring input operation means such as a display and an operation switch by a photographing means such as a camera. Specifically, a display instruction from the display control unit and an image obtained by photographing the display screen of the display are periodically recorded, for example. Then, it is possible to determine whether or not there is a problem in the operation of the display device by analyzing these recorded data and checking whether there is a contradiction between the display instruction and the display screen of the display device. Moreover, the content of the detected input operation and an image obtained by capturing the input operation are periodically recorded, for example. Then, these recorded ones are analyzed, and it is determined whether there is a problem in the operation of the input operation means by checking whether there is a contradiction between the content of the detected input operation and the photographed input operation. be able to.

In addition , the vehicle diagnostic apparatus of the present invention can be configured to include display diagnostic timing generation means for generating a diagnostic timing for the operation of the display.

  With the above configuration, it is possible to diagnose the operation of the display without affecting other processing of the vehicle diagnostic device or increasing the processing load of the vehicle diagnostic device.

Further, the display diagnostic timing generating means in the vehicle diagnostic system of the present invention includes a display instruction change detection means for detecting a change in the content of the display instruction, the case where a change in the content of the display instruction is detected, the indicator It is possible to adopt a configuration in which the operation diagnosis timing is used.

  According to the above configuration, since the diagnosis of the operation of the display device is not performed unless a change in the content of the display instruction is detected, it affects other processing of the vehicle diagnostic device and increases the processing load of the vehicle diagnostic device. Therefore, it is possible to diagnose the operation of the display without any problem. In addition, the operation of the display can be diagnosed in real time.

Further , the display diagnosis timing generation means in the vehicle diagnostic apparatus of the present invention can be configured to generate the diagnosis timing of the operation of the display by a predetermined operation of the input operation means.

  With the above configuration, since the operation of the display device is not diagnosed unless a predetermined operation of the input operation means is performed, it affects other processing of the vehicle diagnostic device and increases the processing load of the vehicle diagnostic device. Therefore, it is possible to diagnose the operation of the display without any problem. In addition, it is possible to diagnose the operation of the display device at a timing desired by the user.

In addition , the vehicle diagnostic apparatus of the present invention includes a date / time information acquisition unit that acquires date / time information including at least a time, a display instruction storage unit that stores the contents of a display instruction together with the date / time information, and a captured screen display content. A display content diagnosis means for storing information together with information, and the display diagnosis means compares the display instruction with the screen display content taken after the time when the display instruction is stored to diagnose the operation of the display. The structure to do can be taken.

  Since the screen display changes after the display instruction during normal operation, the display instruction may be compared with the screen display content captured after the display instruction is stored. In addition, since the date and time data can be acquired from the vehicle diagnostic device or other in-vehicle devices as described later, the manufacturing cost of the vehicle diagnostic device does not increase. With the above configuration, the operation of the display can be diagnosed in a relatively short processing time. It is also possible to diagnose the operation of the display device in real time or at the timing desired by the user.

Further , the vehicle diagnostic apparatus of the present invention has screen display content extracting means for extracting a portion where the screen display content is photographed from the data photographed by the photographing means, and the photographing content storage means is for extracting the extracted portion. A memorizing configuration can be taken.

  With the above configuration, only the data necessary for diagnosis of the operation of the display device is extracted and stored, so that it is possible to suppress an increase in the storage capacity of the photographing content storage means, and the manufacturing cost of the vehicle diagnostic device can be reduced. .

In addition , the vehicle diagnostic apparatus of the present invention can have a configuration having an input operation diagnosis timing generation means for generating a diagnosis timing of the operation of the input operation means.

  With the above configuration, it is possible to diagnose the operation of the input operation means without affecting other processes of the vehicle diagnostic apparatus and without increasing the processing load of the vehicle diagnostic apparatus.

Further , the input operation diagnosis timing generation means in the vehicle diagnostic apparatus of the present invention can be configured to generate the diagnosis timing of the operation of the input operation means when an input operation is detected.

  With the above configuration, since the operation of the input operation means is not diagnosed unless an operation of the input operation means is detected, it affects other processing of the vehicle diagnostic device and increases the processing load of the vehicle diagnostic device. The operation of the input operation means can be diagnosed without any problems. It is also possible to diagnose the operation of the input operation means in real time.

The input operation diagnosis timing generating means in the vehicle diagnostic system of the present invention may be the structure for generating a timing for diagnosis of the operation of the input operation means by a predetermined operation of the input operation means.

  According to the above configuration, the operation of the input operation means is not diagnosed unless there is a predetermined operation of the input operation means, so that it affects the other processes of the vehicle diagnostic device more than the embodiment of the invention of claim 9. In addition, it is possible to diagnose the operation of the input operation means without increasing the processing load of the vehicle diagnostic apparatus. It is also possible to diagnose the operation of the input operation means in real time or at the timing desired by the user.

Moreover , the input operation means in the vehicle diagnostic apparatus of the present invention can be configured as a touch panel that is installed on the display screen of the display and performs a predetermined input by a user input operation on the display screen.

  Recently, a touch panel is often used for the purpose of space saving or efficient arrangement of operation switches. With the above configuration, it is possible to diagnose the operation of the touch panel.

In addition , the vehicle diagnostic apparatus of the present invention includes date / time information acquisition means for acquiring date / time information including at least time, input operation content storage means for storing contents of detected input operations together with date / time information, and photographed input Input operation storage means for storing the operation together with date and time information, and the input operation diagnosis means compares the content of the input operation with an input operation photographed after the time when the content of the input operation was stored. A configuration for diagnosing the operation of the input operation means can be taken.

  Since the screen display changes after the input operation during normal operation, the input operation may be compared with the screen display content captured after the input operation is stored. In addition, since the date and time data can be acquired from the vehicle diagnostic device or other in-vehicle devices as described later, the manufacturing cost of the vehicle diagnostic device does not increase. With the above configuration, the operation of the input operation means can be diagnosed in a relatively short processing time. It is also possible to diagnose the operation of the input operation means in real time or at the timing desired by the user.

The vehicle diagnostic system of the present invention has an input operation extracting means for extracting a portion input operation means from the photographed by photographing means data is taken, the input operation storage means storing the extracted portion The structure to do can be taken.

  With the above configuration, only the data necessary for diagnosing the operation of the input operation means is extracted and stored, so that it is possible to suppress an increase in the storage capacity of the input operation content storage means, thereby reducing the manufacturing cost of the vehicle diagnostic device. Can be reduced.

Moreover , the imaging | photography means in the diagnostic apparatus for vehicles of this invention can take the structure included and attached to the indicator installed for the vehicle room lamp or the rear-seat passenger | crew.

  The display device included in the vehicle diagnostic apparatus of the present invention is often attached near the center panel between the driver seat and the passenger seat. In addition, there is a vehicle in which a display device is attached to the rear ceiling of the vehicle or between the driver's seat and the back of the passenger seat so that a rear passenger can view a navigation screen, a television, and the like. With the above configuration, the camera or the like can be photographed at a position where the display device included in the vehicle diagnostic apparatus can be photographed without newly increasing the installation location of the vehicle accessories and without reducing the comfort in the vehicle. Means can be attached.

The imaging unit in the vehicle diagnostic system of the present invention may take a suspicious person intrusion detection camera configured to detect an intruder to front monitoring camera or a vehicle for photographing the front of the vehicle.

  In recent years, for example, to maintain an appropriate inter-vehicle distance, to maintain a driving lane such as a so-called lane keep, or to detect a behavior of a vehicle traveling ahead and appropriately cope with the change in the behavior For this reason, there is a vehicle equipped with a front monitoring camera for photographing the front of the vehicle. In recent years, an increase in the number of vehicles equipped with antitheft devices in order to prevent such damage due to frequent car theft and car theft. As one of the antitheft devices, there is a suspicious person intrusion detection camera for detecting approach or intrusion to a vehicle. By using both of these cameras, the configuration of the vehicle diagnostic apparatus of the present invention can be realized at low cost.

  The purpose of providing a vehicle diagnostic apparatus capable of accurately diagnosing the operation of the display and the input operation means is to monitor the display and the input operation means with a camera mounted in the vehicle. And the operation diagnosis of the input operation means.

  Hereinafter, embodiments of the present invention will be described with reference to examples shown in the drawings. FIG. 1 shows the overall configuration of the vehicle diagnostic apparatus of the present invention. The vehicle diagnostic apparatus includes an in-vehicle navigation apparatus (hereinafter abbreviated as a navigation apparatus) 100 and a camera 25. The display 10 of the navigation device 100 is installed in the center of the instrument panel in the front of the vehicle interior, and the camera 25 as the photographing means of the present invention can photograph the vehicle front direction including the navigation device 100 such as the center of the ceiling. It is installed in a proper place. In the example of FIG. 1, since the area 25 indicated by a broken line in the camera 25 is an imageable range, the installation position of the display device 10 may be within that range.

  The camera 25 is a known CCD camera or the like, for example, and can be used without an increase in cost if it is shared with a suspicious person intrusion detection camera for detecting an intruder into the vehicle 40 or a front monitoring camera for photographing the front of the vehicle 40. be able to. Further, if it is mounted in the room lamp or in the rear seat display, it can be easily attached to the vehicle 40, and design restrictions can be reduced.

  In the embodiment of the present invention, the operation of the display included in the in-vehicle navigation device is diagnosed. However, in addition to the in-vehicle navigation device, the operation of the display such as the in-vehicle audio device is diagnosed. It is good also as a structure.

  FIG. 2 is a block diagram showing the overall configuration of the vehicle-mounted navigation device. The on-vehicle navigation device 100 includes a position detector 1, a map data input device 6, an operation switch group 7, a remote control (hereinafter referred to as remote control) sensor 11, a voice synthesis circuit 24 for voice guidance, a speaker 15, and a nonvolatile memory 9. , A display device 10, a hard disk device (HDD) 21, a control circuit 8 connected thereto, and a remote control terminal 12.

  The position detector 1 includes a known geomagnetic sensor 2, a gyroscope 3, a distance sensor 4, and a GPS receiver 5 that detects the position of the vehicle 40 based on radio waves from a satellite. Since these sensors 2, 3, 4, and 5 have errors of different properties, they are configured to be used while being complemented by a plurality of sensors. Depending on the accuracy, some of the above-described sensors may be used, and further, a steering rotation sensor or a wheel sensor of each rolling wheel, such as a vehicle speed sensor 23, may be used.

  As the operation switch group 7 which is the input operation means of the present invention, for example, a touch panel 22 integrated with the display 10 or a mechanical switch is used. The touch panel 22 is pressed when an electric circuit is wired on the screen of the display 10 in the X-axis direction and the Y-axis direction through a gap called a spacer on a glass substrate and a transparent film, and the user touches the film. Since the voltage value changes due to short-circuiting of a part of the wiring, a so-called resistance film method for detecting this as a two-dimensional coordinate value (X, Y) is widely used. In addition, using a so-called capacitance method, a substance that receives an electrical signal is applied to the glass surface of a transparent conductive substrate, and a change in capacitance is detected by the sensor as an electrical signal when a finger is brought close to the glass surface. Also good.

  In addition to the mechanical switch and the touch panel 22, a pointing device such as a mouse or a cursor may be used. It is also possible to input various instructions using the microphone 31 and the voice recognition unit 30. In this method, a voice signal input from the microphone 31 is processed by a known voice recognition technique in the voice recognition unit 30 and converted into an operation command corresponding to the result. The voice recognition unit 30 amplifies the voice signal input from the microphone 31 to an appropriate level, and after A / D converting the amplified voice signal, the voice is recognized by a voice recognition algorithm such as a known hidden Markov model. An audio signal processing device (DSP: Digital Signal Processor) for identifying, a memory or the like in which reference data for identifying audio is stored, and the DSP converts the audio signal into numerical information corresponding to the audio. After being converted, it is sent to the control circuit 8. Various instructions can be input through the operation switch group 7, the remote control terminal 12, and the microphone 31.

  The transceiver 13 is a device for receiving road traffic information from, for example, a VICS (Vehicle Information and Communication System) center 14.

  Further, by communicating with an ETC (Automatic Toll Collection System, ETC: Electronic Toll Collection) vehicle-mounted device 16, it is possible to capture the fee information received by the ETC vehicle-mounted device 16 from the ETC road device into the navigation device 100. Further, the ETC vehicle-mounted device 16 may be connected to an external network and communicate with the VICS center 14.

  The control circuit 8 which is a display diagnosis means, input operation detection means, input operation diagnosis means, date and time information acquisition means, display diagnosis timing generation means, display instruction change detection means, and input operation diagnosis timing generation means of the present invention The computer is configured as a known CPU 81, ROM 82, RAM 83, I / O 84 as an input / output circuit, and a bus line 85 for connecting these components. The CPU 81 controls the navigation program 21p and data stored in the HDD 21. The CPU 81 controls the reading / writing of data to / from the HDD 21. The A / D conversion unit 86 includes a well-known A / D (analog / digital) conversion circuit, and converts analog data input from the position detector 1 or the like to the control circuit 8 into digital data that can be calculated by the CPU 81, for example. It is. The ROM 82 may store a program for performing a minimum necessary operation among the navigation functions when the HDD 21 fails.

  The image processing unit 87 which is a display content comparison unit, an input operation comparison unit, a screen display content extraction unit, and an operation content extraction unit of the present invention includes an image processing circuit that analyzes an image by a known technique such as pattern recognition. Composed.

  In addition to the navigation program 21p, the HDD 21 which is the display instruction storage means, photographing content storage means, input operation content storage means, and input operation storage means of the present invention includes so-called map matching data for improving the accuracy of position detection, road Map data 21m including road data representing the connection of is stored. The map data 21m stores predetermined map image information for display and road network information including link information and node information. The link information is predetermined section information constituting each road, and includes position coordinates, distance, required time, road width, number of lanes, speed limit, and the like. The node information is information that defines an intersection (branch road) and the like, and includes position coordinates, the number of right / left turn lanes, a connection destination road link, and the like. In addition, data indicating whether or not traffic is possible is set in the inter-link connection information.

  In addition, in the HDD 21, auxiliary information for route guidance, entertainment information, and other data can be written independently by the user and stored as user data 21u. The user data 21 u can be rewritten by operating the operation switch group 7, the touch panel 22 and the remote control terminal 12, or by inputting voice from the microphone 31. Further, data and various information necessary for the operation of the navigation device 100 may be stored as the database 21d.

  The map data 21m and user data 21u can be added / updated from the storage medium 20 via the map data input device 6. The storage medium 20 is generally a CD-ROM or DVD based on the amount of data, but may be another medium such as a memory card. Moreover, you may use the structure downloaded from an external network.

  The non-volatile memory 9 is constituted by a rewritable semiconductor memory such as an EEPROM (Electrically Erasable & Programmable Read Only Memory) or a flash memory, and information and data necessary for the operation of the navigation device 100. Is remembered. The nonvolatile memory 9 is configured to retain the stored contents even when the accessory switch of the vehicle 40 is turned off, that is, the navigation device 100 is turned off.

  Further, information and data necessary for the operation of the navigation device 100 may be stored in the HDD 21 instead of the nonvolatile memory 9. Furthermore, information and data necessary for the operation of the navigation device 100 may be stored separately in the nonvolatile memory 9 and the HDD 21.

  The display 10 is formed of a known color liquid crystal display, and includes a dot matrix LCD (Liquid Crystal Display) and a driver circuit (not shown) for performing LCD display control. The driver circuit uses, for example, an active matrix driving method in which a transistor is attached to each pixel so that the target pixel can be reliably turned on and off, and is based on a display command and display screen data sent from the control circuit 8. To display. An organic EL (ElectroLuminescence) display or a plasma display may be used as the display.

  The speaker 15 is connected to a known voice synthesis circuit 24 connected to the I / O 84 of the control circuit 8, and the digital voice data stored in the nonvolatile memory 9 or the HDD 21 is converted into an analog voice by the voice synthesis circuit 24 according to a command from the navigation program 21 p. The one converted to is sent. Note that the speech synthesis method includes a recording / editing method in which speech waveforms are stored as they are or after being encoded and connected as necessary.

  The vehicle speed sensor 23 includes a rotation detection unit such as a known rotary encoder. For example, the vehicle speed sensor 23 is installed in the vicinity of the wheel mounting unit to detect the rotation of the wheel and send it to the control circuit 8 as a pulse signal. In the control circuit 8, the number of rotations of the wheel is converted into the speed of the vehicle 40 to calculate an expected arrival time from the current position of the vehicle 40 to a predetermined place, or an average vehicle speed for each travel section of the vehicle 40. To do.

The external device connection device 26 includes an input / output circuit and a connector for connecting to external devices such as sensors and other control devices so as to be able to transmit data. It has a configuration capable of realizing at least one of the following functions.
(1) Connection of a data backup device for backing up the contents stored in the nonvolatile memory 9 or the HDD 21.
(2) In-vehicle LAN (Local Area Network) communication interface circuit for data transmission with other in-vehicle devices.
(3) External network connection via the mobile phone 17.

  In addition, data acquisition from the vehicle speed sensor 23 or connection to the ETC vehicle-mounted device 16 may be performed via the external device connection device 26.

  With this configuration, when the navigation program 21p is activated by the CPU 81 of the control circuit 8, the navigation device 100 allows the user to operate the operation switch group 7, the touch panel 22, the remote control terminal 12, or the voice from the microphone 31. When the route guidance processing for displaying the destination route on the display 10 is selected from the menu displayed on the display 10 by the input, the following processing is performed.

  That is, when the user selects a point from a point or facility search or address search on the map, a registered point set by the user, etc. and sets it as a destination, the position detector 1 determines the current position of the vehicle 40, A process for obtaining an optimum guide route from the current position to the destination is performed. Then, the guidance route is displayed on the road map on the display 10 so as to guide the user to the appropriate route. As a method for automatically setting an optimum guide route, a method such as the Dijkstra method is known. In addition, at least one of the display 10 and the speaker 15 provides a guidance according to an operation or a message according to an operation state.

  FIG. 3 is a flowchart for explaining processing for recording an image photographed by the camera 25. This process is included in the navigation program 21p and is repeatedly executed together with other processes. First, the initial value of navigation device side information (hereinafter abbreviated as navigation side information) is recorded in the navigation side information recording area of the HDD 21 together with the time information at that time (S100). Here, the navigation side information refers to a detection state of whether or not the operation switch group 7 or the touch panel 22 is pressed in the control circuit 8, and a display command or display for screen display sent from the control circuit 8 to the display unit 10. It is data.

  Next, it is determined whether the ignition switch (IG) is on or the accessory (ACC) switch is on. When the ignition switch is in the off state and the accessory switch is in the off state (S101: No), the navigation device 100 is not activated, and thus the system waits until the navigation device 100 is activated.

  On the other hand, when the ignition switch is on or the accessory switch is on (S101: Yes), since the navigation device 100 is activated, it is determined whether there is a change in the navigation side information. Here, the change in the navigation side information means a display command for screen display sent from the control circuit 8 to the display 10 when a change in the operation state of the operation switch group 7 or the touch panel 22 is detected in the control circuit 8 or This is when the display data changes. If there is a change in the navigation side information (S102: Yes), the navigation side information is recorded in the navigation side information recording area of the HDD 21 together with the time information at that time (S103). On the other hand, if there is no change (S102: No), nothing is done and the process proceeds to the next step (S104).

  For example, when a pressed state is detected from a state where the operation switch group 7 or the touch panel 22 is not pressed, the navigation side information is recorded as a change in the navigation side information.

  FIG. 8 shows an example of the navigation side information recording area of the HDD 21. The recording date and time when the navigation side information is recorded, image information including a display command or display data for screen display, and switch information including operation contents of the operation switch group 7 or the touch panel 22 detected by the control circuit 8.

Any of the following methods may be used to acquire the date / time information that is the basis of the recording date / time.
(1) A method including RTC (Real Time Counter) for the control circuit 8 or the CPU 81 of the navigation device 100 to keep time, and generating date and time information by RTC based on the date and time set at the time of factory shipment, for example.
(2) A method for acquiring date and time information through communication with other on-vehicle equipment.
(3) A method of using date and time information included in the GPS signal acquired by the GPS receiver 5.
(4) A method of acquiring date / time information from facilities / equipment outside the vehicle using a communication line.

  Next, for example, it is determined whether it is time to record an image taken by the camera 25 such as a cycle of 0.5 seconds. If not (S104: No), the process returns to step S101 and waits for a change in navigation side information. .

  On the other hand, when it is time to record an image captured by the camera 25 (S104: Yes), a frame image (still image) at that time is taken into the navigation device 100 (S105), and the image processing unit 87 uses the frame image from the frame image. Only the portion where the display 10 and the operation switch group 7 are photographed is extracted (S106), and recorded in the extracted image recording area of the HDD 21 together with the time information at that time (S107).

  FIG. 4 shows the entire frame image taken by the camera 25. FIG. 5 shows only the portion where the display 10 and the operation switch group 7 are taken from the frame image. In the extraction method, when the shooting range of the camera 25 is fixed, the portion where the display 10 and the operation switch group 7 are shot is shot at substantially the same position in the frame image. A predetermined range of the image is set in advance and a method for extracting the predetermined range, or a reference image obtained by photographing only the display 10 and the operation switch group 7 is stored in the HDD 21 or the nonvolatile memory 9 in advance. There is a method of extracting a portion determined to match the reference image from the frame image using the above pattern determination technique or the like.

  FIG. 9 shows an example of recorded contents in the extracted image recording area of the HDD 21. The recording date and time when the captured image is recorded and the captured image.

  Although the image extraction is performed in the navigation device 100, a circuit corresponding to the image processing unit 87 may be included in the camera 25.

  The operation diagnosis process using the navigation side information and the image photographed by the camera 25 recorded in the HDD 21 will be described with reference to the flowchart of FIG. This process is included in the navigation program 21p. The execution method of this process may be repeatedly executed together with other processes, or may be executed by shifting to a specific operation mode such as an “operation diagnosis mode” different from the normal operation mode of the navigation device 100.

  First, the navigation side information recorded at the beginning of the navigation side information recording area and the extracted image recorded at the beginning of the extracted image recording area recorded in the HDD 21 are read (S201), and the screen display is performed from the navigation side information. For example, the image information which is the display command or the display data is transferred to the PV which is an internal work area secured in the HDD 21, for example, and the switch information regarding the operation of the operation switch group 7 or the touch panel 22 is selected from the navigation side information. It moves to PS which is an area (S202).

  Then, the next navigation side information is read (S203). Time information is transferred from the read navigation side information to another internal work area PT (S204).

  PV and PS are image information and switch information to be determined from now. PT is time information when the navigation side information is updated next. That is, a captured image captured by the camera 25 after the time when the image information PV and the switch information PS are recorded and before the time PT when the next navigation side information is updated, and the image information PV and the switch information PS. And compare.

  The relationship between the captured image, the image information PV, and the switch information PS will be described with reference to FIG. The camera 25 captures images at a constant period and records the captured images in the extracted image recording area. For example, when the user presses a key included in the operation switch group 7 or the touch panel 22 at time t2, key-on information indicating that the key has been pressed is recorded as navigation information at time t2 in the navigation side information recording area. Since the user presses the key at the photographing timing at time t3, the state is photographed and recorded in the extracted image recording area as a photographed image at time t3. When the user removes his / her finger from the key at time t5, key-off information indicating that the user has stopped pressing the key is recorded as navigation information at time t5. Thereafter, an operation for changing the screen display of the display device 10 according to the pressed key is performed, and a display command and display screen data are recorded as navigation information at time t7. The display screen after the change is photographed at the photographing timing at time t8 and recorded as a photographed image at time t8.

  When the time information of the captured image is compared with the time information PT, and the time information PT is earlier (the PT is smaller) (S205: No), the time information PT is included than the time information of the captured image. Since the navigation side information is old, the process returns to step S202, the navigation side information recorded next to the currently read navigation side information is read, and the image information PV, switch information PS, and time information PT are updated.

  On the other hand, when the time information of the captured image and the time information PT are the same time, or the time information PT is a later time (the time information PT is larger) (S205: Yes), the next step (S206) Proceed to That is, if the time information of the currently read captured image is before the time information PT that is the time information when the navigation side information is updated next, the image information PV and the switch information PS include the current determination condition and The content recorded before the time information of the captured image is included.

  First, image information is compared. If the captured image and the image information PV match and match (S206: Yes), it is determined that the display 10 is operating normally, and the process proceeds to the next step (S208). If they do not match (S206: No), for example, if the screen is not displayed at all, or if a screen different from the navigation side information is captured, it is determined that the display 10 is operating abnormally, Predetermined processing such as sending out a voice message notifying the abnormal operation is performed (S207).

  The comparison of image information is performed as follows, for example. In the examples of FIGS. 8 and 9, the screen information (such as “Destination setting”) and the button name (“Search”) for displaying the destination setting screen are displayed in the image information (PV) of the navigation information with the data number 2. , “Setting”, etc.). The photographed image to be compared at this time is the photographed image B included in the data number 102 due to the recording date and time. Then, by using a technique such as pattern recognition, it is possible to compare image information by examining whether or not the captured image B includes character information or a character pattern corresponding to the screen title and button name. Then, when the captured image B includes the display information included in the image information PV in excess of a predetermined ratio, it is determined that the captured image B and the image information PV match.

  In the example of FIG. 10, this corresponds to the comparison between the image information PV included in the navigation information at time t7 and the captured image at time t8, or the comparison between the image information PV included in the navigation information at time t16 and the captured image at time t17. To do.

  Next, switch information is compared. When the read photographed image shows a situation where the switch is operated (S208: Yes), it is compared with the navigation side information. If the switch operation status of the photographed image matches the switch information PS (S209: Yes), it is determined that the operation switch group 7 or the touch panel 22 is operating normally, and the next photographed image is read (S213). On the other hand, when the switch operation status of the captured image does not match the switch information PS of the navigation side information (S209: No), the operation switch group 7 or the touch panel 22 is in an abnormal state (open failure) in which the operation corresponding to the operation cannot be performed. And a predetermined process such as sending out a voice message informing the operation abnormality is performed (S210).

  When the switch operating state is not shown in the photographed image (S208: No), it is checked whether the switch information PS includes information on operating the switch. If there is no information on operating the switch in the switch information PS (S211: No), it is determined that the operation switch group 7 or the touch panel 22 is operating normally, and the next captured image is read (S213). . On the other hand, when there is information indicating that the switch is operated in the switch information PS even though the state of operating the switch is not shown in the photographed image (S211: Yes), the operation switch group 7 or the touch panel 22 is displayed. It is determined that there is an abnormal state (short circuit failure) in which some operation is performed even though the operation is not performed, and predetermined processing such as sending a voice message informing the operation abnormality is performed (S212).

  For example, the determination as to whether or not the state of operating the switch is shown in the photographed image is performed as follows. In the example of FIGS. 8 and 9, the switch information PS with the data number 1 includes the information that the destination setting button is pressed, so that the captured image included in the data number 101 of FIG. Compare with A. The photographed image A should have been photographed when the user is pressing the “destination setting” button assigned to the operation switch group 7 or the touch panel 22. The position of the “set destination” button is known in advance. Therefore, in the captured image, if it is possible to detect a state where at least a part of the characters of “destination setting” is invisible in a skin-colored object determined as a finger in an area where the “destination setting” button is captured, It can be detected that “Destination setting” is pressed.

  In the example of FIG. 10, this corresponds to the comparison between the switch information PS included in the navigation information at time t2 and the captured image at time t3, or the comparison between the switch information PS included in the navigation information at time t11 and the captured image at time t12. To do.

  After the captured image is read (S213), the process returns to step S205, and the processing from step S205 onward is repeated for all of the recorded navigation information and captured image.

  In the example of FIGS. 3 and 6, the operation diagnosis is performed after the captured image and the navigation side information are once recorded in the HDD 21. However, the operation diagnosis can be performed in real time using an image recognition technique. Hereinafter, a description will be given with reference to the flowchart of FIG. This process is included in the navigation program 21p and is repeatedly executed together with other processes.

  First, when the ignition switch (IG) is on or the accessory (ACC) switch is on and the navigation device 100 is activated (S300: Yes), it is determined whether there is any change in the navigation side information. If there is a change in the side information (S301: Yes), the navigation side information is acquired (S302).

  Next, a frame image (still image) photographed by the camera 25 is taken into the navigation device 100 (S303), and the image processing unit 87 extracts only the portion where the display 10 and the operation switch group 7 are photographed from the frame image. (S304).

  Then, image information which is a display command or display data for screen display is transferred from the navigation side information to, for example, PV which is an internal work area secured in the HDD 21, and the operation switch group 7 or the touch panel is selected from the navigation side information. The information regarding the operation 22 is transferred to the internal work area PS (S305).

  The subsequent abnormality determination processing (after S306) is the same as that after step S206 in FIG. 6 except that there is no processing for reading the next photographed image, and therefore detailed description thereof will be omitted.

  Although the embodiments of the present invention have been described above, these are merely examples, and the present invention is not limited to these embodiments, and the knowledge of those skilled in the art can be used without departing from the spirit of the claims. Various modifications based on this are possible.

The figure which shows the whole structure of the diagnostic apparatus for vehicles. The block diagram which shows the structure of a vehicle-mounted navigation apparatus. The flowchart for demonstrating the process which records the image image | photographed with the camera. The figure which shows the example of the image image | photographed with the camera. The figure which shows the example which extracted the indicator and its periphery from the image image | photographed with the camera. The flowchart for demonstrating an operation | movement diagnostic process. The flowchart for demonstrating another example of an operation | movement diagnostic process. The figure which shows an example of the recording content of the navigation information recording area. The figure which shows an example of the recording content of an extraction image recording area. The figure for demonstrating the relationship between the imaging | photography timing of a camera, and navigation information recording timing.

Explanation of symbols

7 Operation switch group (input operation means)
8 control circuit (display diagnosis means, input operation detection means, input operation diagnosis means, date and time information acquisition means, display diagnosis timing generation means, display instruction change detection means, input operation diagnosis timing generation means)
9 Non-volatile memory 10 Display 21 Hard disk device (display instruction storage means, photographing content storage means, input operation content storage means, input operation storage means)
22 Touch panel (input operation means)
25 Camera (photographing means)
40 Vehicle 87 Image processing unit (display content comparison means, input operation comparison means, screen display content extraction means, input operation extraction means)
100 Car navigation system

Claims (11)

An input operation means mounted on the vehicle for performing various input operations;
Input operation detecting means for detecting the content of the input operation;
Photographing means for photographing the input operation means and its surroundings at a predetermined timing;
Input operation comparison means for comparing the content of the detected input operation with the photographed input operation;
Input operation diagnostic means for diagnosing whether the operation of the input operation means is performed correctly based on the result of the input operation comparison;
A vehicle diagnostic apparatus characterized by comprising: A display mounted on a vehicle and displaying various information on a screen based on a display instruction from a display control unit;
Input operation means for performing various input operations;
Input operation detecting means for detecting the content of the input operation;
The screen display content of the display at a predetermined timing, and the imaging means for imaging the input operation means and its surroundings,
Display content comparing means for comparing the content of the display instruction with the captured screen display content;
A display diagnostic means for diagnosing whether the operation of the display is correctly performed based on the result of the display content comparison;
Input operation comparison means for comparing the content of the detected input operation with the photographed input operation;
Input operation diagnostic means for diagnosing whether the operation of the input operation means is performed correctly based on the result of the input operation comparison;
A vehicle diagnostic apparatus characterized by comprising: As a display diagnosis timing generation means for generating a timing for diagnosis of the operation of the display, including a display instruction change detection means for detecting a change in the content of the display instruction,
The vehicular diagnostic apparatus according to claim 2, wherein when a change in the content of the display instruction is detected, the operation timing of the display is diagnosed. Date and time information acquisition means for acquiring date and time information including at least the time;
Display instruction storage means for storing the contents of the display instruction together with the date and time information;
Photographing content storage means for storing the photographed screen display content together with the date and time information;
The said display diagnostic means compares the said display instruction with the screen display content image | photographed after the time when the said display instruction was memorize | stored, The operation | movement of the said display is diagnosed . Vehicle diagnostic device. Screen display content extraction means for extracting a portion where the screen display content is photographed from data photographed by the photographing means;
The vehicle diagnostic apparatus according to claim 4, wherein the photographing content storage unit stores the extracted portion . The vehicle diagnosis according to any one of claims 2 to 5, wherein the input operation means is a touch panel that is installed on a display screen of the display and performs a predetermined input by a user input operation on the display screen. apparatus. The vehicular diagnostic apparatus according to claim 1, further comprising an input operation diagnosis timing generation unit configured to generate a timing for diagnosis of the operation of the input operation unit. The vehicle diagnostic apparatus according to claim 7, wherein the input operation diagnosis timing generation unit generates a diagnosis timing of the operation of the input operation unit when the input operation is detected . The vehicle diagnostic apparatus according to claim 7 or 8, wherein the input operation diagnosis timing generation means generates a diagnosis timing of the operation of the input operation means by a predetermined operation of the input operation means . Date and time information acquisition means for acquiring date and time information including at least the time;
Input operation content storage means for storing the content of the detected input operation together with the date and time information;
Input operation storage means for storing the photographed input operation together with the date and time information;
The input operation diagnosis unit compares the content of the input operation with an input operation photographed after the time when the content of the input operation was stored, and diagnoses the operation of the input operation unit. 10. The vehicle diagnostic apparatus according to any one of 9 above. An input operation extracting means for extracting a portion where the input operation means is photographed from data photographed by the photographing means;
The vehicle diagnostic apparatus according to claim 10, wherein the input operation storage unit stores the extracted portion .

Images