JP2016133932A - On-vehicle device, on-vehicle system, image data processing method, and image data processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently leave an appropriate circumstantial evidence when an accident occurs in a vehicle.SOLUTION: An on-vehicle device 30 includes: a dangerous state determination section 38 for determining whether a vehicle is in danger on the basis of data to be obtained from sensors (a sensor group) mounted on the vehicle; and an image synthesis section 40 for synthesizing image data captured by multiple cameras (a camera group) mounted on the vehicle from the point of time of detection that the danger is coming near in driving the vehicle, and generating synthetic image data for enabling the recognition of a state in a direction where the danger comes near and the recognition of states in directions other than the direction where the danger comes near.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an in-vehicle device, an in-vehicle system, an image data processing method, and an image data processing program.

  Conventionally, a drive recorder mounted on a vehicle such as an automobile always records a certain direction (for example, forward) of the vehicle with a camera. Further, the conventional drive recorder stores videos before and after the collision based on the detection result of an acceleration sensor or the like that detects whether or not the vehicle collides with an obstacle or the like (see, for example, Patent Document 1).

  Conventionally, in the case of playing back images shot by a plurality of cameras provided in a vehicle, if an event is detected by a sensor or the like during video acquisition, a video oriented in the direction of the event is generated and played back. A technique is known (for example, refer to Patent Document 2).

JP 7-277230 A JP 2014-134912 A

  However, in the above-mentioned Patent Document 2, for example, a collision direction image is generated at the time of a collision. However, there are cases where the situation evidence at the time of the collision cannot be accurately shown only by the collision direction image. For example, if a collision occurs due to an event that occurred in the opposite direction of the collision direction, or if it was a complex event due to an event that occurred in multiple directions, the situation should be accurately indicated only by the event in the collision direction. I can't. Further, the above-mentioned Patent Document 2 merely shows a technique related to reproduction.

  In one aspect, the present invention provides an in-vehicle device, an in-vehicle system, an image data processing method, and an image data processing program that can efficiently leave appropriate situation evidence when an accident occurs in a vehicle. Objective.

  In one aspect, the in-vehicle device includes: a detection unit that detects whether or not the vehicle is in danger based on data obtained from a sensor mounted on the vehicle; and the vehicle by the detection unit during driving of the vehicle. From the time when it is detected that the danger is imminent, the image data taken by a plurality of cameras mounted on the vehicle is synthesized, and the state of the danger imminent and the danger Composite image data capable of recognizing a state in a direction other than the approaching direction, or a composite image capable of recognizing the state in the direction in which the danger is approaching and the state in a direction other than the direction in which the danger is approaching A processing unit that transmits the image data to another device that generates data.

  Appropriate situational evidence in the event of a vehicle accident can be efficiently left.

It is a figure showing roughly the composition of the information processing system concerning one embodiment. FIG. 2A is a diagram illustrating a hardware configuration of the in-vehicle system, and FIG. 2B is a diagram illustrating a hardware configuration of the in-vehicle device. It is a figure which shows arrangement | positioning of a camera and a sensor. It is a figure which shows the hardware constitutions of a server. It is a functional block diagram of a vehicle-mounted apparatus and a server. FIG. 6A is a diagram illustrating a data structure of the vehicle DB, and FIG. 6B is a diagram illustrating a data structure of the composite image DB. It is a figure which shows the data structure of all the vehicle DB. It is a flowchart which shows the process of a vehicle-mounted apparatus. It is a figure which shows an example of synthetic | combination image data. It is a functional block diagram of the vehicle-mounted apparatus and server which concern on a modification.

  Hereinafter, an embodiment of an information processing system will be described in detail with reference to FIGS.

  FIG. 1 schematically shows a configuration of an information processing system 100 according to an embodiment. The information processing system 100 includes one or more in-vehicle systems 20 and a server 10. The in-vehicle system 20 and the server 10 are connected to a network 80 such as the Internet.

  FIG. 2A shows a hardware configuration of the in-vehicle system 20. The in-vehicle system 20 is a system provided in a vehicle such as an automobile, and as shown in FIG. 2A, a camera group 28, a sensor group 26, an in-vehicle instrument 24 as an acquisition device, an in-vehicle device 30, .

  The camera group 28 has a plurality of cameras 21. For example, as shown in FIG. 3, a plurality of cameras 21 are provided, one for each of the front side, the rear side, the left side, and the right side of the vehicle. However, the number and arrangement of the cameras 21 may be appropriately changed according to the shape of the vehicle. The camera 21 may be, for example, a camera provided in advance in the vehicle to assist driving or may be a retrofitted camera. In addition, each camera shall be a wide-angle camera which can image | photograph the range shown with a broken line in FIG. 3, for example.

  The sensor group 26 includes a plurality of sensors 23. The sensor 23 may be any sensor that can detect the approach of an obstacle, such as a clearance sonar or a millimeter wave radar. Note that the camera group 28 may be used as the sensor group 26 when the approach of the obstacle can be detected from the captured image of the camera group 28. For example, as shown in FIG. 3, it is assumed that a total of eight sensors 23 are provided, two on each of the front side, the rear side, the left side, and the right side of the vehicle. The sensor 23 may be a sensor provided in advance in the vehicle for improving safety, or may be a retrofitted sensor.

  The vehicle-mounted instrument 24 includes sensors that measure the state of the vehicle. For example, the vehicle-mounted instrument 24 includes a speed sensor, an acceleration sensor, a sensor that detects an angle (steering angle) of a steering wheel, a sensor that detects ON / OFF of a brake, and a sensor that detects an operating state of a blinker (direction indicator). Etc. The vehicle-mounted instrument 24 may include a camera capable of photographing the inside of the vehicle, a GPS sensor, a gyro sensor, and the like.

  The in-vehicle device 30 acquires information from the camera group 28, the sensor group 26, and the in-vehicle instrument 24, processes the information, and transmits various data to the server 10. FIG. 2B shows the hardware configuration of the in-vehicle device 30. As shown in FIG. 2B, the in-vehicle device 30 includes a CPU (Central Processing Unit) 190, a ROM (Read Only Memory) 192, a RAM (Random Access Memory) 194, and a storage unit (here, an HDD (Hard Disk Drive)). 196, a network interface 197, a portable storage medium drive 199, and the like. Each part of the in-vehicle device 30 is connected to the bus 198. The camera group 28, the sensor group 26, and the vehicle-mounted instrument 24 are connected to the bus 198 via an interface unit (not shown). The CPU 190 executes a program (including an image data processing program) stored in the ROM 192 or the HDD 196 or a program (including an image data processing program) read by the portable storage medium drive 199 from the portable storage medium 191. Thus, the in-vehicle device 30 is caused to function as each unit shown in FIG. Details of each part in FIG. 5 will be described later.

  Returning to FIG. 1, the server 10 collects and manages data transmitted from one or more in-vehicle systems 20. FIG. 4 shows the hardware configuration of the server 10. As shown in FIG. 4, the server 10 includes a CPU 90, a ROM 92, a RAM 94, a storage unit (HDD) 96, a network interface 97, a portable storage medium drive 99, and the like. Each part of the server 10 is connected to a bus 98. The CPU 90 causes the server 10 to function as each unit in FIG. 5 by executing a program stored in the ROM 92 or the HDD 96 or a program read from the portable storage medium 91 by the portable storage medium drive 99.

  Next, functions of the in-vehicle device 30 and the server 10 will be described with reference to FIG.

  As illustrated in FIG. 5, the in-vehicle device 30 includes an image acquisition unit 32, a vehicle information acquisition unit 34, a detection result acquisition unit 36, a dangerous state determination unit 38 as a detection unit, and a processing unit by the CPU 190 executing a program. Function as an image composition unit 40 and a communication unit 42. 5 also shows a vehicle DB 50 and a composite image DB 52 stored in the HDD 196 and the like.

  The image acquisition unit 32 acquires image data captured by the camera group 28 and stores it in the vehicle DB 50. In addition, the image acquisition part 32 shall store the image data (still image data) cut out for every frame from the moving image data image | photographed in the camera group 28 in vehicle DB50.

  The vehicle information acquisition part 34 stores the vehicle information detected in the vehicle-mounted instrument 24 in vehicle DB50. The image data acquired by the image acquisition unit 32 and the vehicle information acquired by the vehicle information acquisition unit 34 at the same time as the image data are stored in the vehicle DB 50 in an associated state.

Here, the data structure of vehicle DB50 is demonstrated based on Fig.6 (a). As shown in FIG. 6A, the vehicle DB 50 has fields such as “time”, “image”, “speed”, “acceleration”, “handle”, “brake”, and “winker”. In the “time” field, information on the time (date and time) when the image data and the vehicle information are acquired is stored. In the “image” field, four image data (one frame) acquired by the camera group 28 (four cameras 21) are stored. The “speed” field stores the vehicle speed (km / h), and the “acceleration” field stores the vehicle acceleration (m / s ² ). The “handle” field stores information (ie, steering angle information) indicating how much (how many times) the handle has been rotated from the neutral position, and the “brake” field stores the brakes. Is stored (ON / OFF). In the “winker” field, whether or not a winker is used (left / right / none) is stored. In addition to the above, the vehicle DB 50 stores vehicle position information (e.g., obtainable from a GPS sensor mounted on the vehicle), traveling direction information (e.g., obtainable from a gyro sensor, etc.), and the like. It may be. The vehicle DB 50 may store image data obtained by photographing the inside of the vehicle (for example, image data that can confirm the driver's state). In the vehicle DB 50, the number of information (records) that can be stored is determined in advance. For this reason, after the predetermined number of information is stored in the vehicle DB 50, the old information is sequentially deleted.

  Returning to FIG. 5, the detection result acquisition unit 36 acquires the detection result of the possibility of collision by the sensor group 26. The detection result acquisition unit 36 transmits the acquired result to the dangerous state determination unit 38.

  The dangerous state determination unit 38 determines whether or not the vehicle is approaching danger based on the detection result of the sensor group 26 received from the detection result acquisition unit 36. The determination result of the dangerous state determination unit 38 is transmitted to the image composition unit 40.

  The image composition unit 40 synthesizes image data (image data acquired at the same time) stored in the vehicle DB 50 based on the determination result of the dangerous state determination unit 38 to generate composite image data. The image composition unit 40 transmits the generated composite image data to the communication unit 42. Further, when the communication unit 42 cannot communicate with the server 10, the image composition unit 40 stores the composite image data in the composite image DB 52.

  Here, the composite image DB 52 has a data structure as shown in FIG. As shown in FIG. 6B, the composite image DB 52 stores the composite image data in association with the time of shooting.

  Returning to FIG. 5, the communication unit 42 exchanges image data and information with the communication unit 12 of the server 10 by directly connecting to the network 80 or by connecting to the network 80 via a portable terminal. In the present embodiment, the communication unit 42 transmits the composite image data generated by the image composition unit 40 and vehicle information (information such as speed and acceleration) stored in the vehicle DB 50 to the server 10 via the network 80. To do.

  Next, functions of the server 10 will be described. The server 10 functions as the communication unit 12 and the information processing unit 14 illustrated in FIG. 5 when the CPU 90 executes the program. The communication unit 12 acquires composite image data and vehicle information related thereto from the communication unit 42 of the in-vehicle device 30 via the network 80 and transmits the composite image data to the information processing unit 14.

  The information processing unit 14 stores the composite image data and vehicle information acquired from the communication unit 12 in the all vehicle DB 16. Here, the entire vehicle DB 16 has a data structure as shown in FIG. As shown in FIG. 7, the all-vehicle DB 16 has a “vehicle” field newly provided in comparison with the vehicle DB 50 (FIG. 6A), and the “movie” field is replaced with an “image” field. Is provided. The “vehicle” field stores vehicle identification information, and the “moving image” field stores moving image data obtained from the acquired combined image data. Other fields are the same as those of the vehicle DB 50 in FIG.

  Next, processing of the in-vehicle system 20 (in-vehicle device 30) will be described in detail along the flowchart of FIG. 8 with reference to other drawings as appropriate. The process of FIG. 8 is a process that is executed, for example, when the driver of the vehicle turns on the ignition key or when the power source of the in-vehicle device 30 is turned on by the driver. That is, the process of FIG. 8 is a process executed while the driver is driving the vehicle.

  In the process of FIG. 8, first, in step S10, the image acquisition unit 32 acquires image data from the camera 21, the detection result acquisition unit 36 acquires the detection result of the sensor 23, and the vehicle information acquisition unit 34 performs in-vehicle instrumentation. The vehicle state is acquired from 24. Note that the processing in step S10 is executed for each frame of the camera 21.

  Next, in step S12, the image acquisition unit 32 stores the acquired image data in the vehicle DB 50, and the vehicle information acquisition unit 34 stores the acquired vehicle information in the vehicle DB 50. In this case, as in each record (row) in FIG. 6, the image data and the vehicle information are stored in the vehicle DB 50 in association with the acquisition time.

  Next, in step S14, the dangerous state determination unit 38 determines whether or not the vehicle is in danger based on the detection result of the sensor acquired from the detection result acquisition unit 36. For example, if another vehicle is approaching from the front or rear of the vehicle, the determination in step S14 is affirmed and the process proceeds to step S16. On the other hand, if the determination in step S14 is negative, the process returns to step S10.

  When the determination in step S14 is affirmed and the process proceeds to step S16, the image composition unit 40 synthesizes the image data stored immediately before in the vehicle DB 50 to generate composite image data. For example, as shown in FIG. 3, when four cameras 21 are provided in the vehicle, as the composite image data, the composite image data overlooking the vehicle as shown in FIG. Is generated. This composite image data can be said to be composite image data that can recognize a state in a direction in which danger is imminent and a state in a direction other than that direction. For example, in the case where a rear-end collision is caused by the other vehicle A, if only the front image data remains, the event of the rear-end collision by the other vehicle A does not remain as image data. On the other hand, by generating the composite image data as shown in FIG. 9, the vehicle has turned the steering wheel to avoid danger due to an obstacle ahead, and another vehicle A collided there from behind. A series of events can be grasped. At the same time, the movement of the other vehicle B in the opposite lane can be grasped at the same time. Note that if the vehicle information related to the composite image data includes information that the vehicle has suddenly decelerated, it can be understood that the other vehicle A in the rear has collided with the company due to the rapid deceleration.

  Next, in step S <b> 18, the communication unit 42 determines whether or not it is connected to the network 80. When the determination here is affirmed, the communication unit 42 communicates the communication unit of the server 10 with the combined image data generated by the image combining unit 40 and the vehicle information stored in the vehicle DB 50 associated with each other. 12 to send. Thereafter, the process proceeds to step S10.

  On the other hand, if the determination in step S18 is negative, that is, if the communication unit 42 is not connected to the network 80, the process proceeds to step S22. In step S22, the image composition unit 40 stores the generated composite image data in the composite image DB 52. In this case, it is assumed that the composite image data is stored in the composite image DB 52 in association with the time when the image data that is the basis of the generated composite image data is acquired. After the process of step S22, the process returns to step S10. Note that the composite image data stored in the composite image DB 52 is appropriately stored in the server 10 together with vehicle information (stored in the vehicle DB 50) corresponding to the composite image data when the communication unit 42 is connected to the network 80. Shall be sent.

  Note that the information processing unit 14 of the server 10 sequentially acquires the composite image data and the vehicle information by repeatedly executing the process of step S20 of FIG. In this case, the information processing unit 14 stores the acquired composite image data and vehicle information in the all vehicle DB 16. Note that the information processing unit 14 stores a plurality of composite image data having the same vehicle identification information and continuous times in the all vehicle DB 16 as continuously connected moving image data (FIG. 7). (See “ABCD”).

  Through the above processing, when the vehicle is in an accident, composite image data (moving image data) indicating a state where the vehicles before and after the vehicle are viewed from above are accumulated in the all vehicle DB 50 of the server 10. For this reason, when there is an accident, the user or the like can analyze the cause of the accident or the like in detail by confirming the moving image data of the entire vehicle DB 50. In accordance with this, the cause of the accident or the like can be analyzed in more detail by checking the vehicle information stored in the all vehicle DB 50.

  As described above in detail, according to the present embodiment, the in-vehicle device 30 determines whether or not the vehicle is in danger based on data obtained from the sensor 23 (sensor group 26) mounted on the vehicle. The image data taken by a plurality of cameras 21 (camera group 28) mounted on the vehicle from the time when it is detected that the danger state is approaching when the vehicle is driven is synthesized with the dangerous state determination unit 38, And an image composition unit 40 that generates composite image data capable of recognizing a state in a direction approaching danger and a state in a direction other than the direction approaching danger. As a result, composite image data (video data) that can confirm the state of the direction in which the danger is approaching and the state in the direction other than the direction in which the danger is approaching when the danger is imminent or an accident occurs is used as the situation evidence. Can leave. With this composite image data (moving image data), in the event of an accident, in addition to the situation of the accident, it is possible to confirm the indirect cause of the accident and the surrounding situation when the accident occurs. In the present embodiment, since the composite image data is not always generated, the processing amount can be reduced as compared with the case where the composite image data is always generated. That is, according to the present embodiment, it is possible to efficiently leave an appropriate situation proof of an accident.

  Further, in the present embodiment, since the state where the danger is imminent is a state where the obstacle has approached the vehicle more than a predetermined distance, the image data is obtained from an appropriate timing that the obstacle has approached the vehicle more than a predetermined distance. Can be synthesized.

  In the present embodiment, a vehicle DB 50 that stores image data captured by a plurality of cameras and vehicle information (for example, speed and acceleration) at the timing of capturing the image data in association with each other is provided. Thereby, since the state of the vehicle when an accident occurs can be confirmed, it becomes possible to perform a more detailed analysis of the cause of the accident.

  Further, in the present embodiment, when the image composition unit 40 generates composite image data, the composite image data is transmitted in real time to a device (server 10) not mounted on the vehicle (see step S20 in FIG. 8). As a result, even if a situation such as a fire occurs due to an accident and the in-vehicle device 30 is broken, the data that is the situation evidence of the accident is evacuated to the server 10 outside the vehicle, thereby suppressing the disappearance of the situation evidence. be able to.

  In the above-described embodiment, as illustrated in FIG. 9, the case has been described in which the composite image data is image data as seen from directly above the vehicle. However, the present invention is not limited to this. For example, various composite image data such as composite image data overlooking the periphery of the vehicle by 360 ° and composite image data covering the field of view from the driver are generated. May be.

  In the above embodiment, the case where the vehicle information obtained from the vehicle-mounted instrument 24 is associated with the image data and the composite image data has been described. However, the vehicle information need not necessarily be associated.

  Moreover, although the case where the data stored in all the vehicle DB16 was moving image data was demonstrated in the said embodiment, it is not restricted to this, You may be still image data. Moreover, although the said embodiment demonstrated the case where image data was always memorize | stored in vehicle DB50, not only this but you may make it not memorize | store image data in vehicle DB50. In this case, the storage capacity of the HDD 196 of the in-vehicle device 30 can be reduced.

  In addition, although the said embodiment demonstrated the case where image data was synthesize | combined in the vehicle-mounted apparatus 30, it is not restricted to this. For example, as illustrated in FIG. 10, the image composition unit 40 and the composite image DB 52 may be omitted from the in-vehicle device 30, and the information processing unit 14 of the server 10 may have a function as a composition processing unit. In this case, when the danger state determination unit 38 determines that the danger is imminent, the communication unit 42 transmits the image data and the vehicle information to the server 10 in association with each other. Then, the information processing unit 14 of the server 10 may synthesize the received image data to generate composite image data. Even if it does in this way, the effect similar to the said embodiment can be acquired. In the case of FIG. 10, including the communication unit 42, other composite image data that can recognize a state in a direction other than the direction in which danger is imminent and the direction in which danger is imminent is generated from a plurality of image data. A function as a processing unit for transmitting image data to the apparatus (server 10) is realized.

  In the above embodiment, the case where the composite image data is transmitted to the server 10 has been described. However, the present invention is not limited to this. For example, a database storing composite image data or the like is stored in a storage device provided in a casing having high durability or fire resistance provided in the vehicle, and the composite image data generated by the in-vehicle device 30 in the database. Etc. may be stored. Thereby, even if an accident occurs and the vehicle is in flames, it is possible to suppress the disappearance of the situation evidence. Further, since it is not necessary to provide the server 10 or the network 80, there is an advantage that the cost required for system construction can be reduced and the communication cost or the like becomes unnecessary.

  The above processing functions can be realized by a computer. In that case, a program describing the processing contents of the functions that the processing apparatus should have is provided. By executing the program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium (except for a carrier wave).

  When the program is distributed, for example, it is sold in the form of a portable recording medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) on which the program is recorded. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. Further, each time the program is transferred from the server computer, the computer can sequentially execute processing according to the received program.

  The above-described embodiment is an example of a preferred embodiment of the present invention. However, the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention.

In addition, the following additional remarks are disclosed regarding description of the above embodiment.
(Additional remark 1) Based on the data obtained from the sensor mounted in the vehicle, the detection part which detects whether the danger is approaching the vehicle,
From the time when the detection unit detects that the vehicle is in danger while the vehicle is in operation, the image data taken by a plurality of cameras mounted on the vehicle is synthesized, and the danger is approached. Generating composite image data capable of recognizing a state in a direction other than the direction in which the danger is approaching and a direction in a direction other than the direction in which the danger is approaching, or a state other than the state in the direction in which the danger is approaching and the direction in which the danger is approaching An in-vehicle device comprising: a processing unit that transmits the image data to another device that generates composite image data capable of recognizing a state of a direction.
(Additional remark 2) The memory | storage part which associates and memorize | stores the image data image | photographed by these cameras, and at least one of the information on the state of the said vehicle at the timing which image | photographed this image data, and the information on the state inside the said vehicle The in-vehicle device according to appendix 1, further comprising:
(Additional remark 3) The said process part transmits this synthetic | combination image data to the apparatus which is not mounted in the said vehicle, when the said synthetic | combination image data is produced | generated, The vehicle-mounted apparatus of Additional remark 1 or 2 characterized by the above-mentioned.
(Additional remark 4) The said composite image data is data which can recognize the state which looked down at the said vehicle from the top, The vehicle-mounted apparatus in any one of Additional remarks 1-3 characterized by the above-mentioned.
(Supplementary note 5) The in-vehicle device according to any one of supplementary notes 1 to 4, wherein the state in which the danger is approaching is a state in which an obstacle has approached the vehicle more than a predetermined distance.
(Appendix 6) a sensor mounted on the vehicle;
A plurality of cameras mounted on the vehicle;
An in-vehicle system comprising: the in-vehicle device according to any one of appendices 1 to 5.
(Additional remark 7) The vehicle-mounted system of Additional remark 6 further provided with the acquisition apparatus which acquires the information on the state of the said vehicle, and the information on the state inside the said vehicle.
(Supplementary Note 8) Based on data obtained from sensors mounted on the vehicle, it is detected whether or not the vehicle is in danger.
A state in which the danger is approaching by combining image data taken by a plurality of cameras mounted on the vehicle from the time when it is detected that the danger is approaching the vehicle during driving And an image data processing method, wherein one or a plurality of computers execute a process of generating composite image data capable of recognizing a state in a direction other than the direction in which the danger is approaching.
(Supplementary Note 9) The image data captured by the plurality of cameras and at least one of the information on the state of the vehicle and the information on the internal state of the vehicle at the timing when the image data is captured are stored in the storage unit in association with each other. The image data processing method according to appendix 8, wherein the one or more computers execute the process.
(Supplementary note 10) The image data processing method according to supplementary note 8 or 9, wherein the one or more computers execute the process of transmitting the composite image data to a device not mounted on the vehicle.
(Additional remark 11) The said synthetic image data is data which can recognize the state which looked down at the said vehicle from the top, The image data processing method in any one of Additional remark 8-10 characterized by the above-mentioned.
(Supplementary note 12) The image data processing method according to any one of supplementary notes 8 to 11, wherein the state in which the danger is approaching is a state in which an obstacle has approached the vehicle by a predetermined distance or more.
(Additional remark 13) Based on the data obtained from the sensor mounted in the vehicle, it is detected whether or not the vehicle is in danger.
A state in which the danger is approaching by combining image data taken by a plurality of cameras mounted on the vehicle from the time when it is detected that the danger is approaching the vehicle during driving And an image data processing program for generating a composite image data capable of recognizing a state in a direction other than the direction in which the danger is approaching.
(Supplementary Note 14) The image data captured by the plurality of cameras and at least one of the information on the state of the vehicle and the information on the internal state of the vehicle at the timing when the image data is captured are stored in the storage unit in association with each other. The image data processing program according to appendix 13, wherein the computer executes the process.
(Supplementary note 15) The image data processing program according to supplementary note 13 or 14, wherein the computer is caused to execute a process of transmitting the composite image data to a device not mounted on the vehicle.
(Additional remark 16) The said synthetic image data is data which can recognize the state which looked down at the said vehicle from the top, The image data processing program in any one of Additional remarks 13-15 characterized by the above-mentioned.
(Supplementary note 17) The image data processing program according to any one of supplementary notes 13 to 16, wherein the state in which the danger is approaching is a state in which an obstacle has approached the vehicle more than a predetermined distance.

20 Vehicle-mounted system 21 Camera 23 Sensor 24 Vehicle-mounted instrument (acquisition device)
30 On-vehicle device 38 Dangerous state determination unit (detection unit)
40 Image composition unit (processing unit)
42 Communication unit (processing unit)

Claims (8)

A detection unit that detects whether or not the vehicle is in danger based on data obtained from a sensor mounted on the vehicle;
From the time when the detection unit detects that the vehicle is in danger when the vehicle is in operation, the image is taken by combining image data taken by a plurality of cameras mounted on the vehicle. Generating composite image data capable of recognizing a state in a direction other than the direction in which the danger is approaching and a direction in a direction other than the direction in which the danger is approaching, or a direction other than the state in the direction in which the danger is approaching and the direction in which the danger is approaching And a processing unit that transmits the image data to another device that generates composite image data that can recognize the state of the vehicle.   The image processing apparatus further includes a storage unit that stores image data captured by the plurality of cameras and at least one of information on a state of the vehicle and information on an internal state of the vehicle at a timing when the image data is captured. The in-vehicle device according to Item 1.   The in-vehicle device according to claim 1, wherein, when the composite image data is generated, the processing unit transmits the composite image data to a device that is not mounted on the vehicle.   The in-vehicle device according to any one of claims 1 to 3, wherein the composite image data is data capable of recognizing a state in which the vehicle is viewed from above.   The in-vehicle device according to any one of claims 1 to 4, wherein the danger is approaching a state where an obstacle has approached the vehicle more than a predetermined distance. A sensor mounted on the vehicle;
A plurality of cameras mounted on the vehicle;
An in-vehicle system provided with the in-vehicle device according to any one of claims 1 to 5. Based on data obtained from sensors mounted on the vehicle, it is detected whether the vehicle is in danger or not,
A state in which the danger is approaching by combining image data taken by a plurality of cameras mounted on the vehicle from the time when it is detected that the danger is approaching the vehicle during driving And an image data processing method, wherein one or a plurality of computers execute a process of generating composite image data capable of recognizing a state in a direction other than the direction in which the danger is approaching. Based on data obtained from sensors mounted on the vehicle, it is detected whether the vehicle is in danger or not,
A state in which the danger is approaching by combining image data taken by a plurality of cameras mounted on the vehicle from the time when it is detected that the danger is approaching the vehicle during driving And an image data processing program for generating a composite image data capable of recognizing a state in a direction other than the direction in which the danger is approaching.

Images