JP2013041483A - On-vehicle camera control unit, on-vehicle camera control system and on-vehicle camera system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an on-vehicle camera control unit that sets a processing mode by itself according to circumstances of a vehicle by setting the processing mode on the basis of vibration detection information and performs processing according to the set processing mode, and further to provide an on-vehicle camera control system, an on-vehicle camera system and the like.SOLUTION: An on-vehicle camera control unit 100 includes: an image acquisition section 110 for acquiring infrared image data from an infrared image pickup section 210; a vibration detection information acquisition section 130 for acquiring vibration detection information from a vibration detection section, sensor 240; and a processing section 120 for performing processing on the basis of the infrared image data. The processing section 120 sets one processing mode out of a plurality of processing modes including a first processing mode and a second processing mode having a processing execution rate lower than that of the first processing mode as a processing mode for use on the basis of the vibration detection information and performs detection processing of an object in accordance with the set processing mode for use.

Description

  The present invention relates to an in-vehicle camera control device, an in-vehicle camera control system, an in-vehicle camera system, and the like.

  In recent years, in order to cope with the worsening and more sophisticated crime, crime prevention measures have been strengthened and various crime prevention items have been desired.

  For example, with regard to crime prevention in residences, it is described in Patent Document 1 that, when unauthorized opening / closing by a suspicious person to a door or window of a residence is detected, an alarm such as an imaging by an infrared camera or a buzzer is given to notify the resident. There is such a conventional technology.

  However, simply increasing the number of crime prevention items increases the cost of the crime prevention system as the crime prevention system increases, resulting in an increase in the cost burden on the user.

  On the other hand, a vehicle-mounted night vision device that captures and displays a near-infrared image in front of the vehicle body with an in-vehicle infrared camera, detects a pedestrian from the captured image, and displays the detection result superimposed on the near-infrared image. Has been put to practical use. As a pedestrian detection technique used in such a night vision device, there is a conventional technique described in Patent Document 2.

  In the prior art of Patent Document 2, a search is made as to whether or not a part having the same shape as the shape of the predetermined part in the detection target is continuously present in the target, and the part having the same shape is continuously present. In this case, it is determined that the object is an artificial object, and a pedestrian is recognized from the remaining objects excluding the object.

  Under such circumstances, the present applicant is examining a method of using a vehicle-mounted camera for both intruder prevention use in a residence and pedestrian detection use on a road.

JP 2002-150438 A JP 2005-352974 A

  When using an in-vehicle infrared camera for both pedestrian detection and prevention of intruders on the road, the processing of image data appropriate for each application is different. The user needs to set each processing mode when the vehicle is parked.

  According to some aspects of the present invention, the in-vehicle camera control that sets the processing mode according to the vehicle situation by setting the processing mode based on the vibration detection information, and performs the processing according to the set processing mode. An apparatus, an in-vehicle camera control system, an in-vehicle camera system, and the like can be provided.

  In addition, according to some aspects of the present invention, an in-vehicle camera control device, an in-vehicle camera control system, and an in-vehicle camera system capable of monitoring an object to be monitored according to the traveling state of the vehicle while suppressing power consumption. Etc. can be provided.

One aspect of the present invention is an image acquisition unit that acquires infrared image data from an infrared image capturing unit;
A vibration detection information acquisition unit that acquires vibration detection information from the vibration detection unit; and a processing unit that performs processing based on the infrared image data. The processing unit includes a first processing mode and the first processing mode. Among the plurality of processing modes including the second processing mode with a lower execution rate of processing based on the infrared image data, one processing mode is set as a usage processing mode based on the vibration detection information, and the set usage The present invention relates to an in-vehicle camera control device that executes an object detection process based on the infrared image data according to a processing mode.

  In one aspect of the present invention, it is possible to appropriately switch processing according to the usage processing mode set based on the vibration detection information.

  In one aspect of the present invention, when the processing unit determines that vibration is not detected based on the vibration detection information, the processing unit sets the use processing mode to the second processing mode, and You may perform the process for crime prevention based on infrared image data.

  Thereby, when the vibration is not detected, it is possible to perform a security process in the second processing mode.

  In one aspect of the present invention, when it is determined that vibration is detected based on the vibration detection information, the processing unit sets the use processing mode to the first processing mode, and the infrared image Alert display information indicating whether or not the object has been detected may be generated based on the data, and display image data including the alert display information may be generated.

  Thereby, when vibration is detected, it is possible to generate display image data to be displayed on the presentation unit or the like.

  In one aspect of the present invention, the image acquisition unit acquires visible image data from a visible image capturing unit, and the processing unit adds the alert display information to the visible image data, thereby Display image data may be generated.

  Accordingly, it is possible to generate image data in which alert display information is added to visible image data as display image data, and an image that is easy for the user to view can be provided.

  In one aspect of the present invention, the processing unit sets the visible image capturing unit to an operation-off state or a power saving state when it is determined that vibration is not detected based on the vibration detection information. May be.

  Thereby, when vibration is not detected, it is possible to suppress power consumption by setting the visible image capturing unit to the operation-off state or the power-saving state.

  In the aspect of the invention, the processing unit may be configured such that when the usage processing mode is set to the second processing mode, the usage processing mode is set to the first processing mode. In comparison, the acquisition rate of the infrared image data in the image acquisition unit may be set low.

  Thereby, since the acquisition rate of infrared image data can be set lower in the second processing mode than in the first processing mode, it is possible to suppress power consumption and the like.

  In the aspect of the invention, the image acquisition unit may be configured such that when the usage processing mode is set to the second processing mode, the usage processing mode is set to the first processing mode. The infrared image data may be acquired by the infrared image capturing unit set at a position or a direction different from.

  This makes it possible to change the position and direction of the infrared image capturing unit in the first processing mode and the second processing mode, and it is possible to capture an area in accordance with the mode.

  In one aspect of the present invention, the processing unit is used when the usage processing mode is set to the second processing mode and the usage processing mode is set to the first processing mode. The target object may be detected using a template to be detected.

  As a result, since a common template can be used in the first processing mode and the second processing mode, the amount of retained data can be reduced, processing can be shared, and the like.

  In one aspect of the present invention, the image acquisition unit acquires the infrared image data at a given time interval, and the processing unit includes the first infrared image data acquired at a first timing, The security process including a comparison process with second infrared image data acquired at a second timing different from the first timing may be performed.

  Thereby, comparison processing of infrared image data acquired at different timings as security processing can be performed, and intruders can be detected by simple processing.

  In one aspect of the present invention, the processing unit sets the infrared image capturing unit to an operation-off state when it is determined that vibration is not detected based on the vibration detection information. The security process may be performed after the elapse of time.

  As a result, when switching to the second processing mode, the infrared image capturing unit can be in an operation-off state for a certain period of time, so that power consumption can be suppressed.

  In one aspect of the present invention, the crime prevention process includes a residential crime prevention process and a vehicle crime prevention process, and the processing unit sets the use process mode to the second treatment mode and the crime prevention process. When it is determined that vibration is detected based on the vibration detection information during execution of the residential crime prevention process, the crime prevention process may be switched to the vehicle crime prevention process.

  Thereby, it becomes possible to suppress theft of the vehicle when the vehicle is parked.

  Further, in one aspect of the present invention, the processing unit is determined that the infrared image capturing unit is set to a position or direction for home crime prevention, and vibration is detected based on the vibration detection information. In this case, the vehicle crime prevention process may be executed as the crime prevention process.

  Thereby, it becomes possible to suppress theft of the vehicle when the vehicle is parked.

  In one aspect of the present invention, the image acquisition unit acquires visible image data from a visible image capturing unit, and the processing unit sets the visible image capturing unit to an operation-off state in the residential crime prevention process, In the vehicle crime prevention process, the visible image capturing unit may be set to an operation on state.

  As a result, it is possible to reduce power consumption by turning off the visible image capturing unit in the residential crime prevention process, and in the vehicle crime prevention process, by turning on the visible image capturing unit in the operation on state, It becomes possible to acquire information on the current position.

  In one aspect of the present invention, the vibration detection unit is a vibration detection unit for correcting image blur caused by vibration during travel of a vehicle that occurs in an image acquired by the image acquisition unit, and the processing unit The use processing mode setting process may be performed based on the vibration detection information from the vibration detection unit.

  As a result, the vibration detection unit used for the setting process of the usage process mode can be used for image blur correction, and therefore the system configuration can be simplified.

  In the aspect of the invention, the vibration detection unit may detect vibration of a motor that drives the vehicle.

  Thereby, since the vibration of the motor can be detected, it becomes possible to determine the driving state of the electric motor in a hybrid car, for example.

  Another aspect of the present invention is based on an image acquisition unit that acquires infrared image data from an infrared image capturing unit, a vibration detection information acquisition unit that acquires vibration detection information from a vibration detection unit, and the infrared image data. A plurality of processing modes including a first processing mode and a second processing mode having a lower execution rate of processing based on the infrared image data than the first processing mode. An in-vehicle camera control system that sets one processing mode as a use processing mode based on the vibration detection information and executes a target detection process based on the infrared image data according to the set use processing mode. Involved.

  Moreover, the other aspect of this invention is related with the vehicle-mounted camera system containing the vehicle-mounted camera control apparatus in any one of said, and the said infrared image imaging part.

The system configuration example of this embodiment. 2A is a plan view showing attachment positions of the infrared image capturing unit and the visible image capturing unit, and FIG. 2B is a side view of FIG. 2A. 2 is a hardware system configuration example of the present embodiment. 4A to 4D are explanatory diagrams of processing when the vehicle is in a traveling state. FIG. 5A and FIG. 5B are explanatory diagrams of alert images. FIG. 6A to FIG. 6C are explanatory diagrams of residential crime prevention processing. FIG. 7A to FIG. 7C are explanatory diagrams of vehicle crime prevention processing. The flowchart for demonstrating the process of this embodiment. The flowchart for demonstrating a template process.

  Hereinafter, this embodiment will be described. First, a system configuration example of the present embodiment will be described, and then features of the present embodiment will be described. Finally, the flow of processing of this embodiment will be described using a flowchart. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

1. System Configuration Example First, FIG. 1 shows a configuration example of an in-vehicle camera control device of this embodiment and an in-vehicle camera system including the same.

  The in-vehicle camera control device 100 includes an image acquisition unit 110, a processing unit 120, a vibration detection information acquisition unit 130, and a storage unit 150. Examples of the in-vehicle camera system 200 including the in-vehicle camera control device 100 include an in-vehicle camera system including an infrared image capturing unit 210, a visible image capturing unit 220, a presentation unit 230, a sensor 240, and the like. The in-vehicle camera control device 100 and the in-vehicle camera system 200 including the same are not limited to the configuration in FIG. 1, and various modifications such as omitting some of these components or adding other components. Implementation is possible. For example, some or all of the functions of the in-vehicle camera control device 100 may be realized by the infrared image capturing unit 210 or the visible image capturing unit 220. Further, some or all of the functions of the in-vehicle camera control device 100 according to the present embodiment may be realized by a server connected by communication.

  Next, processing performed in each unit will be described.

  First, the image acquisition unit 110 acquires infrared image data from the infrared image imaging unit 210 and acquires visible image data from the visible image imaging unit 220.

  Then, the processing unit 120 performs various processes using data obtained from the storage unit 150 in accordance with the vibration detection information acquired by the vibration detection information acquisition unit 130.

  Furthermore, the vibration detection information acquisition unit 130 acquires vibration detection information (information for determining whether vibration is detected) at the sensor 240.

  The functions of the image acquisition unit 110, the processing unit 120, and the vibration detection information acquisition unit 130 can be realized by hardware such as various processors (CPU and the like), ASIC (gate array and the like), programs, and the like.

  The storage unit 150 stores a database and serves as a work area for the processing unit 120 and the like, and its function can be realized by a memory such as a RAM or an HDD (hard disk drive).

  The infrared image capturing unit 210 captures an infrared image. The infrared image includes a far-infrared image and a mid-infrared image. When the human body is included in the detection target, the infrared imaging unit 210 may be sensitive to a far infrared wavelength region of 8 to 12 μm, and it is necessary to obtain sensitivity at a longer distance (approximately 200 m away). If there is, it is desirable to use a pyroelectric infrared sensor camera as the infrared image capturing unit 210. Using a pyroelectric sensor camera, thermal images (infrared images) from a long distance to a short distance can be acquired with a single sensor film, and the body temperature of the human body and the boundary line of other stationary objects can be clearly displayed. Therefore, it is optimal for human body extraction, and costs are significantly lower than other infrared sensors.

  In this embodiment, the infrared image capturing unit 210 is used for crime prevention purposes such as prevention of intrusion of a house. In this case, the monitoring area is an area including a window and a door of the house. Therefore, in order to detect an intruder into the monitoring area, it is desirable that the infrared imaging unit 210 has sensitivity in the far infrared wavelength region of 8 to 12 μm as described above. Therefore, the same infrared image pick-up part can also be shared at the time of driving | running | working and a house crime prevention. Furthermore, compared to detecting pedestrians while the vehicle is running, when detecting an intruder during crime prevention in a residence, the background image does not change in time, and only the presence or absence of the intruder needs to be detected. Therefore, image processing is very simple as compared to when traveling. In addition, since the time for the intruder to pass through the imaging area is longer than that for the pedestrian, it is possible to detect the intruder while setting the frame rate low and performing thinning imaging. It is possible to reduce the amount.

  On the other hand, the visible image capturing unit 220 captures a visible image. The visible image capturing unit 220 may be a camera using a CCD or CMOS as an image sensor, and may be a monochrome camera or a color camera. When a monochrome camera is used, a grayscale image is captured as a visible image, and when a color camera is used, a color image is captured as a visible image.

  The infrared image capturing unit 210 and the visible image capturing unit 220 may include a device (processor) used for image processing or the like. In the present embodiment, infrared image data or visible image data is output as it is to the in-vehicle camera control device 100, but the present invention is not limited to this. For example, a part of the processing unit of the in-vehicle camera control device 100 may be provided in the infrared image capturing unit 210 or the visible image capturing unit 220. In this case, information after image processing is performed on the infrared image or the visible image is output to the in-vehicle camera control device 100.

  Next, FIG. 2A and FIG. 2B show examples of attachment positions of the infrared image capturing unit, the visible image capturing unit, and the like.

  As shown in FIGS. 2A and 2B, the infrared image capturing unit 210 has a pan angle formed by the horizontal component direction HLL1 of the optical axis of the lens and the vehicle body centerline CL1 as PA1, and the optical axis of the lens. It is mounted in the bumper or grill of the vehicle body so that the tilt angle formed by the horizontal line HL1 between the vertical component direction PLL1 and the ground is TA1. Here, the infrared image capturing unit 210 may include an electric motor that swings the infrared image capturing unit 210 vertically and horizontally. In this case, the rotation angle can be freely adjusted so that the road ahead or the side of the road can be imaged.

  As shown in FIGS. 2A and 2B, the visible image capturing unit 220 has PA2 as the pan angle formed by the horizontal component direction HLL2 of the optical axis of the lens and the vehicle body centerline CL2, and the optical axis of the lens. It is attached to the upper part of the front window so that the tilt angle formed by the horizontal line HL2 between the vertical component direction PLL2 and the ground is TA2. Here, similarly to the infrared image capturing unit 210, the visible image capturing unit 220 may include an electric motor that swings the visible image capturing unit 220 up and down and left and right. In this case, the rotation angle can be freely adjusted so that the road ahead or the side of the road can be imaged.

  In FIG. 2A, PA1 and PA2 are 0 ° or more for easy understanding. However, when imaging the front in the vehicle body traveling direction, PA1 = PA2 = 0 ° and TA1 = TA2. Is desirable. Further, the infrared image capturing unit 210 and the visible image capturing unit 220 may be installed on the side surface of the vehicle body, or a plurality of each of the image capturing units may be installed.

  Further, the infrared image capturing unit 210 and the visible image capturing unit 220 may be provided with a selector switch in the vehicle so that the driver can arbitrarily determine the shooting direction and the shooting position. The selector switch can turn ON / OFF driving of the infrared image capturing unit 210 or the visible image capturing unit 220 and change the position and direction of the infrared image capturing unit 210 and the visible image capturing unit 220. Note that the selector switch is omitted in FIGS. 1, 2A, and 2B.

  Next, the presentation unit 230 shows the processing result of the in-vehicle camera control device 100 to the user. The presentation unit 230 may be an alarm device or a display.

  When the presenting unit 230 is an alarm device, the alarm device issues an alarm (sound) based on the output signal obtained from the processing unit 120 when the presence or absence of a pedestrian or an intruder into the residence is detected. If the horn and front light of the vehicle body are used as an alarm device, the horn can be sounded or the front light can be blinked when an intruder is detected in the residence, and there is no need to install new equipment. There is.

  Further, when the presentation unit 230 is a display, the display displays an image indicating these positions when the processing unit 120 detects a preceding vehicle, a pedestrian, or an intruder into the residence, or detects An image generated by superimposing a contour image of an object on a visible image may be acquired from the processing unit 120 and displayed.

  In the present embodiment, a gyro sensor is used as the sensor 240. The gyro sensor has low power consumption and can detect minute vibrations. In addition, the sensor 240 may be an acceleration sensor, a GPS sensor, or the like.

  FIG. 3 shows a hardware configuration diagram of this embodiment. First, the in-vehicle camera control device 100 of FIG. 1 can be realized by the CPU 450 and the memory 470 of FIG. Here, the image acquisition unit 110, the processing unit 120, and the vibration detection information acquisition unit 130 in FIG. 1 can be realized by the CPU 450 in FIG. 1, and the storage unit 150 in FIG. 1 can be realized by the memory 470 in FIG.

  The in-vehicle camera system 200 in FIG. 1 includes the infrared camera 410, the CPU 450, and the memory 470 in FIG. Here, the infrared image capturing unit 210 in FIG. 1 is realized by the infrared camera 410 in FIG. 3 and includes an electric motor 420.

  Further, the visible image capturing unit 220 in FIG. 1 is realized by the visible camera 430 in FIG. 3 and includes an electric motor 440. Furthermore, the presentation unit 230 in FIG. 1 can be realized by the display 490 in FIG. 3. 1 can be realized by the gyro sensor 460, the GPS 480, or the like shown in FIG.

2. In the above embodiment, the in-vehicle camera control device 100 includes an image acquisition unit 110 that acquires infrared image data from the infrared image capturing unit 210, a processing unit 120 that performs processing based on infrared image data, A vibration detection information acquisition unit 130 that acquires vibration detection information from a vibration detection unit (for example, sensor 240). And the process part 120 performs the detection process of the target object based on infrared image data according to the process mode (usage process mode) set based on the vibration detection information.

  The processing mode is a general term for the processing state of the in-vehicle camera control device 100, and means a series of processing performed by the in-vehicle camera control device 100 depending on the set processing mode. Furthermore, the processing mode actually used when controlling the in-vehicle camera control device is referred to as a usage processing mode. In the present embodiment, the processing mode has at least a first processing mode and a second processing mode, and the second processing mode is a mode in which the execution rate of processing based on infrared image data is lower than that in the first processing mode. . Here, the execution rate represents the amount of processing executed per unit time in the processing unit 120, and a low execution rate means that the processing amount in the processing unit 120 per unit time is small. To do. Therefore, for example, when the processing unit 120 is realized by a CPU or the like, the second processing mode has a smaller load on the CPU than the first processing mode, which is advantageous in terms of power saving.

  Specifically, it can be considered that the execution rate of the second processing mode is made lower than that of the first processing mode by the following method. For example, when the first processing mode and the second processing mode acquire the same number of infrared image data per unit time, the object detection processing is performed on all image data in the first processing mode. In the second processing mode, the object detection process is performed on some infrared image data extracted at predetermined time intervals. Alternatively, the object detection process may be performed after setting the infrared image data acquisition rate (corresponding to the number of infrared image data acquisitions per unit time) in the second process mode to be lower than that in the first process mode.

  The first processing mode may be a travel mode (a processing mode executed when the vehicle is traveling) in a narrow sense, and the second processing mode is a security mode (a processing mode for executing a security processing described later) in a narrow sense. ). As will be described later, the processing unit 120 can select at least one of the crime prevention mode and the traveling mode, and can switch to another processing mode.

  Thereby, it becomes possible to perform processing according to the use processing mode set based on the vibration detection information. For example, if the running state of the vehicle is determined based on the vibration detection information, different processing can be performed depending on whether the vehicle is in a running state or the vehicle is in a non-running state. The vibration detection by the vibration detection unit is, for example, determined that the vibration is detected when a sensor 240 (for example, a gyro sensor) provided on the vehicle body continuously detects a vibration having a frequency of 1 Hz to 20 Hz for a predetermined time. May be.

  Further, when it is determined that the vibration is not detected based on the vibration detection information, the processing unit 120 sets the use processing mode to the second processing mode and performs a security process based on the infrared image data. Also good.

  Here, the process for crime prevention is a process for detecting an object at the time of crime prevention, and corresponds to a process for detecting an intruder into the residence, for example, at the time of crime prevention against the entry of the residence. Specifically, as will be described later, the object detection processing by comparing the first infrared image data acquired at the first timing and the second infrared image data acquired at the second timing is security. It can be considered as a process. The first timing and the second timing may be two consecutive timings as the acquisition timing of the infrared image data. For example, the latest infrared image data is compared with the infrared image data one timing before. Alternatively, the first timing may be a given reference timing. For example, the first infrared image data acquisition timing after the start of the security process can be considered as the first timing. In this case, the first infrared image data corresponds to the fixed image data for monitoring (background image data) in which no intruder exists, and the fixed image data for monitoring is compared with the latest infrared image data. Will do.

  Thereby, when vibration is not detected, it is possible to perform security processing using infrared image data. The case where the vibration is not detected is, for example, a case where the vehicle is stopped. Specifically, a case where the vehicle is parked in the user's home parking lot can be considered. In this case, the monitored object becomes an intruder into a residence or the like, and an optimal process is performed for these monitoring. As the security process, not only the object detection process but also the process of presenting the infrared image data captured by the infrared image capturing unit 210 or detection information based on the infrared image data in the presenting unit 230 may be performed.

  Here, the detection information is information indicating that the monitoring target is reflected in the infrared image. For example, as an example of the detection information, there is character information that informs the user that there is an intruder into a residence or the like, and a command that causes the presentation unit 230 to generate a warning sound.

  In this case, what is necessary is just to show to the presentation part 230, when an intruder etc. exist in the monitoring area. Therefore, in the non-running state, the power consumption of the in-vehicle camera control device 100 can be suppressed as compared with the case where the presentation unit 230 always provides the presentation.

  Here, the monitoring area is an area that needs to be monitored in order to detect the monitoring target of the in-vehicle camera system 200. For example, the monitoring area is detected by the infrared image capturing unit 210 or the visible image capturing unit 220. It means the range to be imaged. In the example of FIG. 6A to be described later, this is a range including the door DR and the window WD of the house captured by the infrared image capturing unit 210.

  Further, the detection information may be information indicating that the monitoring target is not reflected. In this case, when the vehicle is in the non-running state, the in-vehicle camera control device 100 performs the presentation in the presentation unit 230 in principle at all times.

  Further, when it is determined that vibration is detected based on the vibration detection information, the processing unit 120 sets the use processing mode to the first processing mode, and whether or not an object is detected based on the infrared image data. May be generated, and display image data including the generated alert display information may be generated.

  Here, the display image data is data representing a display image. The display image is an image for informing the user of the presence of the detected object. For example, when the first processing mode is a travel mode (a mode in which the vehicle is in a travel state), the display image is an image for informing the user of the presence of a monitoring target such as a pedestrian or a preceding vehicle. Yes, it refers to the image itself displayed on the presentation unit 230. In this case, as the alert display information, an image representing the outline of the monitoring object or an image representing a symbol for notifying that the monitoring object is nearby can be considered. That is, the display image data including the alert display information is an image generated by superimposing a contour image of a pedestrian or a preceding vehicle on an infrared image. Further, a warning sound or the like may be emitted simultaneously with displaying the display image.

  Thereby, image data emphasizing the presence of the monitoring object can be generated and presented to the user, and the user can be alerted to the monitoring object. If the first processing mode is the traveling mode, the monitoring object is a pedestrian or a preceding vehicle, and optimal processing is performed for these monitoring. For example, an outline image of a pedestrian or a preceding vehicle is displayed to the user. It can be presented to call attention to the monitored object.

  In addition, when it is determined that the vehicle is in a running state, infrared image data is acquired, so that the monitoring target object corresponding to the situation around the vehicle such as that there are many pedestrians or many other vehicles. It is possible to monitor.

  For example, FIG. 4A shows a state where the vehicle CA1 is traveling and the traveling mode is set. At this time, the infrared image capturing unit 210 provided in the vehicle CA1 captures the front of the vehicle and captures the preceding vehicle CA2, the pedestrian WP1, the person WP2 riding the bicycle, and the like.

  Here, the infrared image data is information representing an infrared image (thermal image) captured by the infrared image capturing unit 210. The infrared image is as described above. For example, FIG. 4C illustrates an infrared image IIM captured by the infrared image capturing unit 210 provided in the vehicle CA1 illustrated in FIG. 4A as an example of the infrared image. In the infrared image IIM, a pedestrian WP1, a preceding vehicle CA2, a person WP2 riding a bicycle, and the like are projected darkly at portions where the temperature is high. An infrared image is not suitable for identifying a landscape or the like, but it is easy to identify a temperature distribution or the like, and is useful for discriminating between a background and a monitoring object.

  However, display images, such as contour images and images in which contour images are superimposed on infrared images, look very different from the scenery that the user actually sees. It may be difficult to use properly. For example, when the first processing mode is the traveling mode, even if an outline image or the like is presented, there is a possibility that the user cannot immediately reflect the driving of the vehicle.

  Therefore, the image acquisition unit 110 may acquire visible image data from the visible image capturing unit 220. Then, the processing unit 120 may perform display image data generation processing by adding alert display information to the visible image data.

  Here, as the display image data in which the alert display information is added to the visible image data, image data corresponding to an image in which an image representing the contour of the target object is superimposed on the visible image, or the like can be considered.

  The visible image data is information representing a visible image captured by the visible image capturing unit 220. The visible image is as described above. For example, as an example of the visible image, a visible image VIM captured by the visible image capturing unit 220 provided in the vehicle CA1 illustrated in FIG. 4A is illustrated in FIG. The visible image VIM also shows a pedestrian WP1, a preceding vehicle CA2, a person WP2 riding a bicycle, and the like. Since the visible image VIM has almost no difference from the scenery seen by the user, it is easy to grasp the entire image including the scenery at a glance. On the other hand, it is difficult to distinguish a landscape and a monitoring object using only a visible image. Therefore, in the present embodiment, the monitoring object is distinguished using the infrared image having the characteristics as described above.

  Further, a specific example of a display image generated using a visible image is shown in FIG. When generating the display image CIM, for example, based on the characteristics of the temperature distribution represented by the infrared image IIM shown in FIG. WP2) and other than people (background and preceding car CA2) are distinguished. Then, a display image CIM is generated by superimposing alert images (ALIM1 and ALIM2), which will be described later, so as to surround a portion where a person is located on the visible image. By presenting such an image, the user is warned to pay attention to the pedestrian.

  Here, the alert image is an image for notifying the user of the presence of the monitoring target, and means an image that is a part of the traveling display image displayed on the presentation unit 230. Examples of the alert image include an exclamation mark, a schematic diagram showing a pedestrian and a preceding vehicle, and an emphasis character indicating that a danger is imminent. Note that the alert image is included in the alert display information as described above.

  Specifically, images ALIM1 and ALIM2 shown in FIG. 4D correspond to alert images. In addition, an exclamation mark such as ALIM3 displayed on the head of the pedestrian WP1 on the image CIM1 shown in FIG. 5A, or a walk on the image CIM2 shown in FIG. There are an image ALIM4 for emphasizing the outline of the person WP1, and a band image such as ALIM5 for emphasizing that the danger is imminent using characters and symbols.

  As a result, an alert image is superimposed on a visible image that matches the scenery that the user actually sees to generate a display image for traveling, so that even when using a display image, the user does not feel uncomfortable. In addition, it is possible to call attention to the monitoring object. Furthermore, by using the alert image, the user can easily distinguish the background from the monitoring target.

  In addition, when it is determined that vibration is not detected based on the vibration detection information, the processing unit 120 may set the visible image capturing unit 220 to the operation off state or the power saving state.

  Thereby, when the vibration is not detected (when the second processing mode is set), the visible image capturing unit 220 can be set to the operation-off state or the power saving state, so that power consumption can be suppressed. become. As described above, in the second processing mode in which vibration is not detected, it is assumed that the vehicle is stopped and an intruder is detected. In that case, it is sufficient if the presence / absence of an intruder can be presented to the user, and it is not always necessary to provide a visible image that is easy to see for the user. Therefore, the visible image capturing unit 220 may be set to an operation off state or a power saving state.

  In addition, when the usage processing mode is set to the second processing mode, the processing unit 120 sets the acquisition rate of infrared image data in the image acquisition unit 110 as compared to the case where the usage processing mode is set to the first processing mode. It may be set low.

  Accordingly, it is possible to suppress the consumption of power used for driving the infrared image capturing unit 210 and image processing of infrared image data. It is known that intruders in a residence or the like often stay in a certain place in order to see the state of the residence, and the transit time of the monitoring area is longer than that of a pedestrian or the like. In addition, when the vehicle is parked, the background image does not change with time, and therefore image processing is performed more than when processing other than crime prevention processing is performed (for example, when the vehicle is in a traveling state). It is easy and there is no need to set a high frame rate.

  In addition, when the usage processing mode is set to the second processing mode, the image acquisition unit 110 sets the infrared image capturing unit 210 set to a position or direction different from that when the usage processing mode is set to the first processing mode. Thus, infrared image data may be acquired.

  As a result, it is possible to acquire images obtained by capturing appropriate areas in the second processing mode and in the first processing mode, respectively. As described above, the first processing mode is a mode in which the execution rate of processing is higher than that in the second processing mode, and typically a traveling mode is assumed. In the traveling mode, it is assumed that the vehicle moves forward, and therefore it is necessary to capture a monitoring object such as a pedestrian or an obstacle in front. Therefore, in the traveling mode, the infrared image capturing unit 210 is set to a position or direction where the front of the vehicle can be imaged. On the other hand, in the second processing mode (crime prevention mode in a narrow sense), since a person who is going to enter the residence becomes a monitoring target, a place that is likely to be an entry route to the residence (for example, a window) or the like is entered. It is necessary to take an image of an area where it is not desirable to do so (such as in the premises of your home). Therefore, in the crime prevention mode, the infrared image capturing unit 210 is set to a position or a direction in which the region as described above can be captured.

  In addition, the processing unit 120 may perform the object detection process in the second process mode using a template used for the object detection when the use process mode is the first process mode. This may be said to use a common template in both the first processing mode and the second processing mode.

  This makes it possible to reduce the template data stored in the storage unit 150 and the like, and to use a common template in the first processing mode and the second processing mode, so that the processing can be shared. This is because, when the first processing mode and the second processing mode are the traveling mode and the crime prevention mode, it is assumed that the person is included in the monitored object during the traveling and the crime prevention. This is because (for example, temperature data) is the same when observed during traveling and when observed during crime prevention.

  Further, the image acquisition unit 110 acquires infrared image data at given time intervals. The processing unit 120 includes a security process including a comparison process between the first infrared image data acquired at the first timing and the second infrared image data acquired at the second timing different from the first timing. Processing may be performed.

  Here, the first infrared image data and the second infrared image data refer to infrared image data representing infrared images captured at different timings. For example, infrared image data for a certain timing 1 is acquired, this is used as first infrared image data, and infrared image data acquired at every predetermined period thereafter is used as second infrared image data for comparison processing. Also good. In addition, the infrared image data acquired at the timing next to the first infrared image data is the second infrared image data, and the infrared image data acquired at the timing next to the second infrared image data is the first infrared image data. Comparison processing may be performed assuming that the image data is infrared image data.

  Further, as a specific example of the security process, the monitoring target is detected when there is a difference between the two infrared image data as a result of the comparison process between the first infrared image data and the second infrared image data. May be determined. For example, in the example of FIG. 6A, the infrared image captured by the infrared image capturing unit 210 at a certain timing t is IIM1 illustrated in FIG. 6B, and the infrared image captured at timing (t + 1) is illustrated in FIG. It is referred to as IIM2 shown in (C). In this case, first, the infrared image data indicating the infrared image IIM1 is stored in the storage unit as the first infrared image data, and when the infrared image data indicating the infrared image IIM2 is acquired, the infrared image data of the IIM2 is acquired. Is compared with the first infrared image data as the second infrared image data. As a result, in this example, it is assumed that the monitoring target object is not reflected in the infrared image IIM1, but the appearance of an intruder IV2 is confirmed in the infrared image IIM2. In this case, in FIG. 6A, at timing t, it is assumed that the person IV who was at the position P1 outside the monitoring area moved to the position P2 and entered the monitoring area before the timing (t + 1). This person IV is recognized as an intruder IV2. Then, it is determined that there is a difference between the first infrared image data and the second infrared image data, and the presenting unit 230 presents that there is an intruder.

  Thereby, when the second processing mode is set, the monitoring target is detected using only the infrared image data, and when the monitoring target is detected, the presence of the intruder is notified to the user. It becomes possible.

  Further, when it is determined that vibration is not detected based on the vibration detection information, the processing unit 120 sets the infrared image capturing unit 210 to an operation-off state, and performs a security process after a given time has elapsed. You may go.

  As a result, the infrared image capturing unit 210 can be turned off for a given time, so that power consumption can be suppressed. This is assumed to be performed at the time of switching from the traveling mode to the crime prevention mode, for example. In other words, it is considered that the user is near the vehicle (actually in the vehicle) before switching. Therefore, when the mode is switched, the user is near the vehicle or the residence, so the security processing by the system does not necessarily have a high priority. In such a case, even if the infrared image capturing unit 210 is turned off for a certain period of time, it is considered that there is no particular problem.

  The security process may include a residential security process and a vehicle security process. And the process part 120 sets the process to perform as a process for crime prevention, when the use process mode is set to the 2nd process mode, and a vibration is detected based on vibration detection information at the time of execution of a residential crime prevention process. You may switch from a crime prevention process to a vehicle crime prevention process.

  Here, the residential crime prevention process refers to a process for the purpose of detecting an intruder or the like into a residence or a facility. On the other hand, the vehicle crime prevention process refers to a process aimed at detecting a vehicle theft or the like.

  As a result, it is possible to perform vehicle crime prevention processing or the like when there is a possibility that the vehicle has been stolen while performing residential crime prevention processing. This assumes a case where the vehicle itself is stolen while monitoring a residence or the like. Specifically, when the house crime prevention process is being executed, if vibration is detected (the vehicle may be traveling), there is a high possibility that the vehicle has been stolen. In other words, even though the vehicle was stopped (parked) and monitoring by the security process was started, the vehicle was driven without taking regular procedures, and the driver in this case stealed instead of the user. It is supposed to be something.

  In addition, when the infrared image capturing unit 210 is set at the position or direction for residential crime prevention, when the vibration is detected based on the vibration detection information, the processing unit 120 performs the vehicle crime prevention process as the crime prevention process. You may go.

  As a result, when there is a possibility that the vehicle has been stolen, it is possible to perform vehicle crime prevention processing. Although the vehicle has been stopped (parked) and monitoring by the crime prevention process is being started, the vehicle crime prevention process is executed based on the situation that the vehicle has been run without taking a regular procedure. It is the same. However, it may be determined from the position or direction of the infrared image capturing unit 210 that the vehicle is stopped, not from the processing contents of the use processing mode and the security process. Specifically, this is the case as shown in FIG. In FIG. 7A, the in-vehicle camera system originally mounted on the vehicle CA performed the home crime prevention process and monitored the neighborhood of the residence at the point P1, but the vehicle CA was stolen by the theft. , It shows a situation where he has run away without completing the home crime prevention process. In such a case, it is not necessary to continue the residential crime prevention process, and it is necessary to take measures against vehicle theft.

  Here, the position or direction for home crime prevention is, for example, the position or direction of the infrared image capturing unit 210 that can capture a location that the user wants to monitor, such as a door or window of a house or facility. A plurality of combinations are stored in advance in the storage unit 150 or the like for the position or direction for home crime prevention, and the user operates the selector switch or the like to identify the position or direction, and automatically captures infrared images under the control of the in-vehicle camera control device 100. The unit 210 may be moved. In addition, the user may manually move and arrange the position or direction of the infrared image capturing unit 210 and input to the in-vehicle camera control device 100 that the arrangement has been completed.

  A specific example is shown in FIG. FIG. 6A shows a state of processing when the second processing mode (security mode) is set, and shows a state where the vehicle CA is parked on the parking in front of the residence. . In FIG. 6A, the position and direction of the infrared image capturing unit 210 are adjusted so that the image can be captured of the door DR and the window WD where an intruder is likely to enter a residence or the like.

  In addition, the processing unit 120 may set the visible image capturing unit 220 to the operation-off state in the residential crime prevention process, and may set the visible image capturing unit 220 to the operation-on state in the vehicle crime prevention process.

  Thereby, when the residential crime prevention process is being performed, while the unnecessary visible image capturing unit 220 is turned off and power consumption is reduced, the visible image capturing unit 220 is turned on in the vehicle crime prevention process, The location where the vehicle is currently located can be imaged and presented to the user. When the vehicle is stolen, it is necessary to notify the user or the like of the current position of the vehicle. This is because a visible image that can be easily identified at first glance is more useful than an infrared image. A specific example is shown in FIG. By setting the visible image capturing unit 220 to the operation-on state, it is possible to acquire a visible image as illustrated in FIG. Since the visible image capturing unit 220 is provided in the vehicle, the acquired visible image is an image obtained by capturing a position corresponding to the current position of the vehicle (for example, in front of the vehicle). Therefore, the position of the stolen vehicle can be estimated by acquiring a visible image. In particular, if a signboard representing a place name, a facility, or the like is captured as shown in FIG. 7B, the position can be easily specified. Even if the acquired visible image is displayed on the display in the vehicle, it is not transmitted to the user or the like, and there are few advantages. Therefore, in this case, it is desirable to transmit visible image information to an external device using some communication means. As the external device, for example, a mobile phone owned by the user can be considered. When the in-vehicle camera system has communication means, the information transmitted to the external device is not limited to a visible image, and may be character information as shown in FIG. Good.

  In addition, the vibration detection unit (for example, the sensor 240) may be a vibration detection unit for correcting image blur caused by vibration during traveling of the vehicle that occurs in the image acquired by the image acquisition unit 110. And the process part 120 performs the setting process of a use process mode based on the vibration detection information from a vibration detection part.

  Thereby, using the common vibration detection unit (sensor 240), determination of whether to set the usage processing mode (for example, whether to set the first processing mode or the second processing mode, that is, for example, whether it is a running state) Etc.) and image blur correction can be performed. As shown in FIG. 2, both the infrared image capturing unit 210 and the visible image capturing unit 220 are mounted on the vehicle. For this reason, when the vehicle vibrates, the imaging unit also vibrates, and blurring occurs in the acquired image. Since an image with blurring is an image that is difficult for the user to see, it is preferable to correct the blurring. At this time, since it is assumed that the blur is caused by the vibration of the vehicle, the image blur correction can be performed if the degree of the vibration is known. Since the vibration detection unit for use processing mode setting processing is mounted on the vehicle in the same manner as the imaging unit, it can detect vibrations in the imaging unit. It is possible to use. This eliminates the need to prepare separate vibration detection units for the two processes, thereby simplifying the configuration.

  The vibration detection unit may detect vibration of a motor that drives the vehicle.

  As a result, the vibration of the motor can be detected by the vibration detector. For example, in a hybrid car or the like that uses motive power using gasoline or the like in combination with electric power, the vibration of the motor is detected by providing a vibration detection unit in an electric motor driven by electric power. When the vehicle is running, vibrations due to multiple factors, such as vibrations caused by driving the engine and vibrations received from the road surface, can be detected, but vibrations caused by driving the motor can be detected by installing a vibration detector on the motor. it can. For example, the motor may be used at the time of starting / acceleration requiring a large torque, and in that case, the motor vibrates violently to obtain a large power. This can be clearly distinguished from the driving after sufficient speed has been obtained. Therefore, by detecting the vibration of the motor by the vibration detection unit, it is possible to determine not only whether the vehicle is running but also whether it is starting or accelerating. It is also possible to set different usage processing modes for those other than those and perform appropriate processing according to each.

  Note that the in-vehicle camera control device of this embodiment may be realized by a program. In this case, the in-vehicle camera control device according to the present embodiment is realized by a processor such as a CPU executing a program. Specifically, a program stored in the information storage medium is read, and a processor such as a CPU executes the read program. Here, the information storage medium (computer-readable medium) stores programs, data, and the like, and functions as an optical disk (DVD, CD, etc.), HDD (hard disk drive), or memory (card type). It can be realized by memory, ROM, etc. A processor such as a CPU performs various processes of this embodiment based on a program (data) stored in the information storage medium. That is, the information storage medium includes a program (a program for causing the computer to execute the processing of each unit) for causing a computer (an apparatus including an operation unit, a processing unit, a storage unit, and an output unit) to function as each unit of the present embodiment. Remembered.

3. Processing Flow Hereinafter, the processing flow of this embodiment will be described with reference to the flowchart of FIG. Here, as the first processing mode and the second processing mode, a traveling mode and a crime prevention mode are assumed.

  First, in step 1 (S1), parameters necessary for processing are initialized. Next, in step 2 (S2), in order to grasp the state of the vehicle body, vibration of the vehicle body is detected. If vibration is detected, the vehicle is running or stopped, and if vibration is not detected, it is determined that the vehicle is parked.

  When vibration of the vehicle body is detected, the infrared image capturing unit 210 captures the front in the vehicle body traveling direction, and captures a thermal image in the processing region (S3). Here, when the infrared image capturing unit 210 includes a human body as a detection target, it is sufficient that the infrared imaging unit 210 has sensitivity in a far infrared wavelength region of 8 to 12 μm. A pyroelectric infrared sensor camera is even better.

  In step 4 (S4), the visible image capturing unit 220 images the front in the vehicle body traveling direction, and captures the visible image in the processing region. Here, the visible image capturing unit 220 may use a CCD or CMOS as an image sensor, and may be a monochrome camera or a color camera. The visible image is a gray image when a monochrome camera is used, and a color image when a color camera is used. In this embodiment, a case where a color camera capable of acquiring color information is used as the visible image capturing unit 220 will be described. In addition, as for the order of step 3 (S3) and step 4 (S4), as long as the space | interval between steps is short, whichever is first. In some cases, both may be processed in parallel.

  Next, in step 5 (S5), candidate areas are extracted from the thermal image obtained in step 3 (S3). That is, the temperature region representing the feature of the object is extracted as a lump. In the present embodiment, the detection target object is described as a preceding vehicle and a pedestrian (person). When a target other than this is set as a detection target, the temperature information of the target may be added as a feature amount to the processing target in this step.

  First, in the case of a preceding vehicle body, using the fact that the muffler region becomes a high temperature of about 100 ° C., a region having temperature information of, for example, 80 ° C. or more is searched from the thermal image, and the position information and temperature of that region are searched. Detect information. On the other hand, in the case of a pedestrian, by utilizing the fact that the surface temperature of the exposed face becomes a medium temperature of about 30 ° C., a region having temperature information of about 30 ° C. is searched from the thermal image, The position information and temperature information of the area are detected. Here, among the results detected in each region, the position information may be, for example, the barycentric coordinate position of the lump, and the temperature information may be, for example, the average temperature of the lump. In step 6 (S6), a position and temperature list in which the position information and temperature information of each region obtained in step 5 (S5) are associated is created, and the process proceeds to the subsequent processes.

  Next, in step 7 (S7), based on the list created in step 6 (S6), it is determined whether or not temperature information indicating the feature amount of the preceding vehicle or pedestrian that is the detection target is extracted. If no candidate object is extracted in the list, it is determined that there is no preceding vehicle or pedestrian as a detection target in the image captured by the infrared image capturing unit 210 this time, Return to Step 3 (S3) for the next processing. On the other hand, if one or more candidate objects are extracted from the list, the process proceeds to the next step (S8) to detect the object from the visible image.

  In step 8 (S8), based on the list created in step 6 (S6), a detection area for subject detection processing is set in the visible image. Next, in step 9 (S9), a template used for the template matching process is selected for the detection region set in step 8 (S8). This template selection process will be described with reference to FIG.

  First, in step 801 (S801), based on the temperature information in the list created in step 6 (S6), a template candidate to be used in subsequent processing is selected. Specifically, a template having a value within the range of the average temperature information ± ΔT of the candidate area is set as a template candidate used for detecting an object in one candidate area. For example, the temperature information of the template is 100 ± 3 ° C. to be selected as the template used for the preceding vehicle presence candidate region. However, since the high temperature portion is special in a normal driving environment, an area of 80 ° C. or higher may be selected as a template in advance. On the other hand, the template used for the pedestrian presence candidate region is, for example, the temperature information of the template is 28 ± 3 ° C. Considering that the high temperature part is affected by the outside air temperature, it is necessary to consider that the temperature of the same object changes within a predetermined range. In this example, an object having medium temperature information is a pedestrian, but an animal template may be selected as a candidate in this step in order to distinguish it from an animal (such as a dog).

  Next, in step 802 (S802), for each detection area set in step 7 (S7), density information (light / dark information) and color information are extracted from the visible image as feature amounts. For example, in the case of a black-and-white camera, only the density information is obtained, but the size and type of the object in the detection area are estimated from the density information using density values having an average density value or a density histogram peak or edge information. What is necessary is just to extract the feature-value which can be performed. Further, for example, in the case of a color camera, color information is obtained in addition to density information. Therefore, it is only necessary to extract whether or not a red region corresponding to the tail lamp of the preceding vehicle body exists in the detection region.

  In step 803 (S803), an optimal template is determined for each detection region set in the visible image based on the results of step 801 (S801) and step 802 (S802). Perform template matching in the image.

  That is, in step 10 (S10) in FIG. 8, template matching processing is performed in the visible image using the template selected in step 8 (S8), and the presence or absence of the final target object in each detection region and The position is detected. In step 9 (S9), a vehicle body template is selected for the preceding vehicle presence candidate region, and a person template is selected for the pedestrian presence candidate region. In this step, when the preceding vehicle is detected from the preceding vehicle presence candidate area, the presence / absence of the preceding vehicle is determined by the correlation calculation of the density value (gray value) in the preceding vehicle existence candidate area and the body template using the body template. The detailed position is detected. Further, when detecting a pedestrian from the pedestrian presence candidate area, the presence / absence of the pedestrian and its detailed position are detected by correlation calculation between the pedestrian existence candidate area and the density value in the human template. In this embodiment, the template matching process is performed by correlation calculation using density values. However, the present invention is not limited to this. For example, an edge image is held as a template, and the final target object is determined by this correlation value. The presence / absence and its position may be detected.

  Next, in step 11 (S11), it is determined whether or not the detection process has been performed for all the areas in the list created in step 6 (S6). If all the areas in the list have not been processed, the process returns to step 8 (S8) to set the detection area in order to switch the processing area. If the processing for all the areas in the list has been completed, the process proceeds to the next step in order to output the processing results up to that point.

  Finally, in step 12 (S12), the detection result is output to the presentation unit 230 and the like, and the process returns to step 2 (S2) to perform the subsequent processing. Specifically, in order to control, for example, steering, accelerator, brake, etc., or to issue an alarm such as the presence or absence of a pedestrian or the risk of collision, the detection result of this process is displayed as an alarm or image in the presentation unit 230. Present.

  Next, a description will be given of a method in which the vehicle body stops at the residence and is used for crime prevention in the residence. In step 2 (S2), if there is no vibration, it is determined that the vehicle is stopped. In step 13 (S13), driving of the infrared camera and the visible camera is stopped. In step 14 (S14), only the infrared camera is rotated so that a preset image acquisition region can be imaged. The direction of the infrared camera may be manually set. In step 15 (S15) and step 16 (S16), a timer is set, and after a predetermined time has passed, the infrared camera is driven to capture the set front and obtain an initial screen of the thermal image (S17). The timer is set because it is not necessary to acquire an image immediately even if the infrared camera is set to the house, and it is possible to be vigilant even if image acquisition is started after a certain period of time. is there.

  In general, a window, a door, or the like can be considered as a place for detecting crime prevention in a house. Here, when detecting a person as an intruder, any person having sensitivity in the far-infrared wavelength region of 8 to 12 μm may be used, and it can be shared with the infrared camera installed on the vehicle body, and there is no problem. Furthermore, compared to detection of pedestrians while the vehicle is traveling, the detection of crime prevention in the house has a fixed background image and only needs to detect the presence or absence of an intruder, so that image processing is very simple. In addition, since the imaging time and the movement of the intruder are slower than those of the pedestrians, thinned imaging is possible, and the amount of electricity consumed during imaging is reduced.

  In step 17 (S17), after a predetermined time has elapsed, the front is imaged by the infrared camera, and the image is captured. Here, the sensitivity of the infrared camera may be reduced. This is because the distance between the vehicle body and the object (window / door) is short, and there is little blurring of the outline due to the overlap of the background portion and the image of the intruder. Further, in step 18 (S18), a thermal image at time t + i is captured.

  Next, in step 19 (S19), the temperature feature amount of the thermal image obtained in step 17 (S17) and the thermal image in step 18 (S18) are compared. In general, in the case of an intruder, by utilizing the fact that the surface temperature of the exposed face is about 30 ° C., a region having temperature information of about 30 ° C. is searched from the thermal image, and the temperature Detect the presence of information. If there is no temperature information, it is determined in step 21 (S21) whether the infrared camera is in the initial position (pedestrian detection position used during traveling). If it is not the initial value, it is in the intruder detection position. In other words, since the processing in the security mode is being performed, the security processing is continued and a thermal image is taken again after several seconds.

  If the infrared camera has been changed to the initial position in step 21 (S21), the process returns to step 1 (S1) to start from the initial setting. In step 19 (S19), if a region having an intruder surface temperature of about 30 ° C. can be detected from the comparison processing of the two thermal images, in step 20 (S20), the detection result is displayed on the presentation unit 230 and the like. Is output, and the process returns to Step 1 (S1) where the subsequent processing is performed.

  With this configuration, when a vehicle theft occurs at midnight, the difference in the image is greatly generated in step 19 (S19), so that the detection result is obtained for the presentation unit 230 or an externally connected device. It is output and the vehicle theft can be detected. Further, after the theft, it is assumed that vibration is detected in S2, and in that case, processing in the running mode is performed, so the infrared camera is reset, the visible camera and the infrared camera are simultaneously driven, and the current running position is reached. A corresponding visible image or the like can be acquired. As described above, it is desirable that the visible image is transmitted to the external device.

  Although the present embodiment has been described in detail as described above, it will be easily understood by those skilled in the art that many modifications can be made without departing from the novel matters and effects of the present invention. Accordingly, all such modifications are intended to be included in the scope of the present invention. For example, a term described at least once together with a different term having a broader meaning or the same meaning in the specification or the drawings can be replaced with the different term in any part of the specification or the drawings. Further, the configurations and operations of the in-vehicle camera control device, the in-vehicle camera control system, and the in-vehicle camera system are not limited to those described in the present embodiment, and various modifications can be made.

100 on-vehicle camera control device, 110 image acquisition unit, 120 processing unit,
130 vibration detection information acquisition unit, 150 storage unit, 200 in-vehicle camera system,
210 infrared image capturing unit, 220 visible image capturing unit, 230 presenting unit,
240 sensors, 410 infrared cameras, 420 electric motors,
430 Visible camera, 440 Electric motor, 460 Gyro sensor,
470 memory, 490 display

Claims (17)

An image acquisition unit for acquiring infrared image data from the infrared image capturing unit;
A vibration detection information acquisition unit for acquiring vibration detection information from the vibration detection unit;
A processing unit for performing processing based on the infrared image data;
Including
The processor is
One processing mode is used based on the vibration detection information from among a plurality of processing modes including a first processing mode and a second processing mode having a lower execution rate of processing based on the infrared image data than the first processing mode. An in-vehicle camera control device, which is set as a processing mode and executes object detection processing based on the infrared image data according to the set use processing mode. In claim 1,
The processor is
When it is determined that vibration is not detected based on the vibration detection information, the use processing mode is set to the second processing mode, and security processing based on the infrared image data is performed. In-vehicle camera control device. In claim 2,
The processor is
When it is determined that vibration is detected based on the vibration detection information, the use processing mode is set to the first processing mode, and whether or not the object is detected based on the infrared image data is determined. An in-vehicle camera control device that generates alert display information to be displayed and generates display image data including the alert display information. In claim 3,
The image acquisition unit
Visible image data is acquired from the visible image capturing unit,
The processor is
The in-vehicle camera control device, wherein the display image data is generated by adding the alert display information to the visible image data. In claim 4,
The processor is
An in-vehicle camera control device, wherein when it is determined that vibration is not detected based on the vibration detection information, the visible image capturing unit is set to an operation off state or a power saving state. In any one of Claims 1 thru | or 5,
The processor is
When the usage processing mode is set to the second processing mode, the infrared image data is acquired by the image acquisition unit compared to when the usage processing mode is set to the first processing mode. An in-vehicle camera control device characterized in that the rate is set low. In any one of Claims 1 thru | or 6.
The image acquisition unit
When the usage processing mode is set to the second processing mode, the infrared image capturing unit set to a position or direction different from that when the usage processing mode is set to the first processing mode The in-vehicle camera control device is characterized in that the infrared image data is acquired by the above. In any one of Claims 1 thru | or 7,
The processor is
When the use process mode is set to the second process mode, the object is detected using a template used when the use process mode is set to the first process mode. An in-vehicle camera control device. In any of claims 2 to 8,
The image acquisition unit
Acquiring the infrared image data at given time intervals;
The processor is
The security process including a comparison process between the first infrared image data acquired at the first timing and the second infrared image data acquired at a second timing different from the first timing is performed. An in-vehicle camera control device. In any one of Claims 2 thru | or 9.
The processor is
When it is determined that vibration is not detected based on the vibration detection information, the infrared image capturing unit is set to an operation-off state, and the security process is performed after a given time has elapsed. In-vehicle camera control device. In any of claims 2 to 10,
The security process includes a residential security process and a vehicle security process,
The processor is
When the use processing mode is set to the second processing mode and it is determined that vibration is detected based on the vibration detection information while the residential security processing is being executed as the security processing, the security processing A vehicle-mounted camera control device characterized by switching processing to the vehicle crime prevention processing. In claim 11,
The processor is
When it is determined that the infrared image capturing unit is set in the position or direction for residential crime prevention and vibration is detected based on the vibration detection information, the vehicle crime prevention process is performed as the crime prevention process. An in-vehicle camera control device characterized by being executed. In claim 11 or 12,
The image acquisition unit
Visible image data is acquired from the visible image capturing unit,
The processor is
The in-vehicle camera control device characterized in that the visible image capturing unit is set in an operation-off state in the residential crime prevention process, and the visible image capturing unit is set in an operation-on state in the vehicle security process. In any one of Claims 1 thru | or 13.
The vibration detector is
A vibration detection unit for correcting image blur caused by vibrations during running of the vehicle, which occurs in an image acquired by the image acquisition unit;
The processor is
An in-vehicle camera control device that performs setting processing of the use processing mode based on the vibration detection information from the vibration detection unit. In any one of Claims 1 thru | or 14.
The vibration detector is
A vehicle-mounted camera control device that detects vibrations of a motor that drives a vehicle. An image acquisition unit for acquiring infrared image data from the infrared image capturing unit;
A vibration detection information acquisition unit for acquiring vibration detection information from the vibration detection unit;
A processing unit for performing processing based on the infrared image data;
Including
The processor is
One processing mode is used based on the vibration detection information from among a plurality of processing modes including a first processing mode and a second processing mode having a lower execution rate of processing based on the infrared image data than the first processing mode. A vehicle-mounted camera control system that is set as a processing mode, and that performs an object detection process based on the infrared image data in accordance with the set use processing mode.   An in-vehicle camera system comprising the in-vehicle camera control device according to claim 1 and the infrared image capturing unit.