US10306207B2 - Information processing system - Google Patents
Information processing system Download PDFInfo
- Publication number
- US10306207B2 US10306207B2 US15/324,414 US201515324414A US10306207B2 US 10306207 B2 US10306207 B2 US 10306207B2 US 201515324414 A US201515324414 A US 201515324414A US 10306207 B2 US10306207 B2 US 10306207B2
- Authority
- US
- United States
- Prior art keywords
- priority
- unit
- arbitration
- bus
- sensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
- 230000010365 information processing Effects 0.000 title claims abstract description 22
- 238000012544 monitoring process Methods 0.000 claims description 24
- 238000012545 processing Methods 0.000 claims description 19
- 238000004891 communication Methods 0.000 claims description 14
- 230000015654 memory Effects 0.000 claims description 14
- 238000001514 detection method Methods 0.000 claims description 10
- 230000005856 abnormality Effects 0.000 claims description 8
- 238000005259 measurement Methods 0.000 claims 3
- 238000000034 method Methods 0.000 abstract description 5
- 230000008569 process Effects 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 13
- 230000002093 peripheral effect Effects 0.000 description 12
- 238000003384 imaging method Methods 0.000 description 4
- 238000003860 storage Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000010420 art technique Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K31/00—Vehicle fittings, acting on a single sub-unit only, for automatically controlling vehicle speed, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator
- B60K31/0008—Vehicle fittings, acting on a single sub-unit only, for automatically controlling vehicle speed, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator including means for detecting potential obstacles in vehicle path
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
- B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
- B60R1/28—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with an adjustable field of view
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/31—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles providing stereoscopic vision
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60T—VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL; ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
- B60T8/00—Arrangements for adjusting wheel-braking force to meet varying vehicular or ground-surface conditions, e.g. limiting or varying distribution of braking force
- B60T8/17—Using electrical or electronic regulation means to control braking
- B60T8/171—Detecting parameters used in the regulation; Measuring values used in the regulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/36—Handling requests for interconnection or transfer for access to common bus or bus system
- G06F13/362—Handling requests for interconnection or transfer for access to common bus or bus system with centralised access control
-
- G06K9/00805—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/165—Anti-collision systems for passive traffic, e.g. including static obstacles, trees
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
- B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
- B60R2300/107—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using stereoscopic cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/20—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used
- B60R2300/202—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used displaying a blind spot scene on the vehicle part responsible for the blind spot
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/20—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used
- B60R2300/207—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used using multi-purpose displays, e.g. camera image and navigation or video on same display
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/802—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying vehicle exterior blind spot views
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/8066—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring rearward traffic
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60T—VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL; ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
- B60T2210/00—Detection or estimation of road or environment conditions; Detection or estimation of road shapes
- B60T2210/30—Environment conditions or position therewithin
- B60T2210/32—Vehicle surroundings
Definitions
- the present invention relates to a system for processing information.
- Patent Literature 1 As a background art of the technical field.
- the patent publication discloses that “to suitably decide a priority order in the case of using a bus between devices and to sufficiently transfer data between devices even when any device is connected to the bus in an information processor connecting plural devices on the same bus.
- the bus arbitrating means 110 selects either the existent device 102 or the extended device 105 and applies the use right of the bus 101 in accordance with the priority order decided by a priority order deciding means 109 based on device information obtained from an existent device information storing means 104 and an extended device information storing means 107 (see Abstract).
- Patent Literature 1 JP-A-5-53975
- An object is to positively process sensor information in an information processor in which plural sensors are connected.
- information can be transmitted based on information of priority which has been previously decided.
- priority which has been previously decided.
- An information processing system including plural sensors, plural buses respectively connected to the plural sensors, an arbitration unit performing arbitration of the plural buses and a priority determination unit determining the priority of each of the plural sensors, in which the arbitration unit performs arbitration of the plural buses in accordance with the determination result of the priority determination unit.
- processing can be performed suitably and positively in real time in accordance with conditions of the sensor such as a dangerous state.
- conditions of the sensor such as a dangerous state.
- another sensor can be used by flexibly changing the sensor having the priority according to the sensor state, which improves robustness.
- FIG. 1 is a view showing a configuration example of the entire information processing system according to the present invention.
- FIG. 2 is a view showing a first configuration example for performing arbitration of respective units in accordance with the priority by using plural buses.
- FIG. 3 is a view showing a second configuration example for performing arbitration of respective units in accordance with the priority by using plural buses.
- FIG. 4 is a view showing a third configuration example for performing arbitration of respective units in accordance with the priority by using plural buses.
- FIG. 5 is a view showing the detailed configuration of a stereo camera and the correspondence between the stereo camera and an arbitration unit.
- FIG. 6 is a view showing an example in which bus priority is changed in accordance with environment of an own vehicle and peripheral environment.
- Embodiment 1 an information processing system for processing plural sensor signals will be explained. Recently, a moving object detection device which detects moving objects such as a pedestrian and a vehicle by using plural imaging units (cameras) such as a stereo camera is practically used. Plural cameras including one for monitoring the front direction of the vehicle, one for monitoring the periphery of the vehicle or the rear direction of the vehicle may be provided. Respective functions are described here.
- the front camera functions as a front monitoring camera for detecting existence of an obstacle or a preceding vehicle existing within about 100 meters and is used also as a sensor having a function of an automatic braking or following the preceding vehicle by measuring the distance and relative speed.
- the peripheral camera is used as a periphery monitoring sensor for assisting parking when stopping the vehicle.
- a display is installed at a position where a driver can see video of a camera attached in the rear of the vehicle such as in a car interior, and the video of the rear is displayed on the display.
- the driver can check a dead angle which is difficult to be seen by eyes by the display.
- Respective cameras have respective functions as sensors such as the front camera for controlling the vehicle and the camera for assisting the driving, therefore, priorities of processing differ according to the situation when these functions are realized on the same platform.
- the present invention provides a scheme capable of changing the priority of a communication line such as a bus so that the camera playing the role of control is preferentially processed in such conditions.
- FIG. 1 shows a configuration example of the entire information processing system according to the present invention.
- the information processing system shown in FIG. 1 is configured by including a controller 101 receiving information from respective sensors and controlling the respective sensors, a stereo camera 102 monitoring the front direction within about 100 meters, a communication unit 103 as a C2X module performing communication between an own vehicle and the other vehicle or communication (C2X (Car to X) communication) between the own vehicle and communication devices installed on a road and so on, a peripheral camera 104 having a function of detecting an obstacle or a moving object by outputting video obtained by imaging the periphery of the own vehicle on a display installed at a position where the video can be seen by a driver and by performing image processing to the inputted video, a navigation system 105 having a function of displaying a current location of the own vehicle on a map by using map information and GPS receiving information to thereby navigate to a destination set by a user and a display 106 displaying video obtained by respective cameras, map information by the navigation system 105 ,
- FIG. 2 shows a first configuration example for performing arbitration of respective units in accordance with the priority by using plural buses.
- the arbitration between the stereo camera 102 and the peripheral camera 104 will be explained as an example.
- the controller 101 has a general configuration including a CPU 201 , a memory 202 , an address conversion unit 203 and a memory 204 which are mutually connected through a bus 20 , further including a priority determination unit 205 and an arbitration unit 206 .
- a memory 207 and a first sensor 208 connected to the arbitration unit 206 through a bus 21 correspond to the peripheral camera 104
- a memory 209 and a second sensor 210 connected to the arbitration unit 205 through a bus 22 correspond to the stereo camera 102 .
- the priority determination unit 205 determines the priority based on information from respective sensors. For example, when receiving information indicating that an obstacle which may collide is detected from the second sensor 209 , the priority determination unit 205 determines the former as information with high priority also by referring to information from the first sensor 207 .
- the arbitration unit 206 performs arbitration so that the access is made to the bus which should be preferentially connected. In the above example, the access is made to the bus 22 which should be preferentially connected based on the determination result from the priority determination unit 205 and desired video and so on are read out from the memory 208 to be displayed on the display 106 . In this case, for example, priorities are assigned to high-order bit addresses seen from the CPU, and the arbitration unit 206 controls access to the bus in accordance with the priorities.
- the priority determination unit 205 determines the priority based on information from respective sensors
- the arbitration unit 206 performs arbitration so that the access is made to the optimum bus based on the determination result and reads necessary information appropriately from the memory of the next stage, thereby transmitting suitable sensor information and displaying necessary information to the user in accordance with situation change.
- FIG. 1 The case where two sensors are included is cited as an example and explained in FIG. 2 , however, the present invention is not limited to this.
- sensors not shown in FIG. 1 such as a sensor for information concerning other vehicles and roads obtained from the communication unit 103 , a vehicle speed monitoring sensor which monitors speed of the own vehicle and a travelling direction monitoring sensor which monitors a travelling direction of the own vehicle, which can be applied in the same manner by providing buses corresponding to respective sensors.
- the memory 207 and the first sensor 208 correspond to the peripheral camera 104
- the memory 209 and the second sensor 210 connected to the arbitration unit 205 through the bus 22 correspond to the stereo camera 102
- the present invention is not limited to this and can be applied to a system having some plural sensors. The same applies to other embodiments explained below.
- Embodiment 2 is an example of an information processing system which can give priority to information of a particular sensor further positively in addition to Embodiment 1. The details will be explained with reference to FIG. 3 . Concerning components which are the same as those of Embodiment 1, the components of Embodiment 1 are invoked and description is omitted, and different components will be mainly explained.
- FIG. 3 shows a second configuration example for performing arbitration of respective units in accordance with the priority by using plural buses.
- a bus 30 is newly provided to the second sensor 210 and the arbitration unit 206 in addition to the components of Embodiment 1.
- the bus 30 is a dedicated bus not shared with other sensors and not for large capacity data such as image data.
- the priority is updated by the dedicated bus according to the result from the second sensor 210 , thereby changing the priority without delay, positively transmitting and controlling information of the sensor which is particularly important as the sensor for detecting the existence of an obstacle in front of the own vehicle or a preceding vehicle without being interrupted by processing of other sensors just before the vehicle collides an obstacle.
- the dedicated bus 30 is provided with respect to the second sensor 210 and the arbitration unit 206 is shown here, the present invention is not limited to this and can be appropriately applied to sensors determined to be highly important in advance.
- Embodiment 3 is an example of an information processing system including a diagnostic unit for determining whether the sensor can correctly operate due to failure or environment or not, in which, when it is determined that the sensor does not correctly operate, another sensor is allowed to preferentially operate to thereby improve the robustness of the entire system.
- the system will be explained with reference to FIG. 4 .
- Concerning components which are the same as those of Embodiment 1 and 2, the components of Embodiment 1 and 2 are appropriately invoked and description is omitted, and different components will be mainly explained.
- FIG. 4 shows a third configuration example for performing arbitration of respective units in accordance with the priority by using plural buses.
- a diagnostic unit 401 is newly added to the components of Embodiment 2, and the diagnostic unit 401 is connected to the second sensor 210 and the bus 30 .
- the diagnostic unit 401 detects an abnormality of the second sensor 210
- the diagnostic unit 401 notifies the arbitration unit 206 of the abnormality of sensor processing.
- the arbitration unit 206 rejects a priority request from the second sensor 210 and gives preference to another sensor. According to the process, another sensor can substitute for part of functions, for example, when the sensor is out of order, therefore, the reliability of the entire system is improved.
- the diagnostic unit 401 In the case where an abnormality of the sensor is detected by the diagnostic unit 401 , the user may be informed of the abnormality by display on the display 106 or voice from a not-shown voice output unit, and abnormality information may be transmitted to the outside by the communication unit 103 .
- the diagnostic unit 401 is connected to the second sensor 2 is shown in the Embodiment, the present invention is not limited to this.
- the diagnostic units may be appropriately used for other sensors respectively as well as one diagnostic unit may be connected with respect to plural sensors.
- Embodiment 4 is a more specific example of an information processing system using the stereo camera, in which the bus priority is determined by an instruction from a function unit of the stereo camera which makes collision determination, thereby changing the priority efficiently by a small number of circuits.
- FIG. 5 shows a configuration example mainly showing the detailed configuration of the stereo camera 102 and the correspondence between the stereo camera 102 and the arbitration unit 206 .
- the stereo camera 102 can calculate positional aberration (parallax) of the same object (three-dimensional object) such as a pedestrian on plural images imaged at the same time by template matching and can measure a distance of the object (three-dimensional object) based on the calculated parallax, which includes a camera 501 as a first imaging unit and a camera 502 as a second imaging unit to obtain plural images.
- the stereo camera 102 is further configured by appropriately including a distance image generation unit 503 which generates a distance image by using images outputted from the camera 501 and the camera 502 , a three-dimensional object detection unit 504 which detects a three-dimensional object based on the generated distance image and making collision determination based on the position, distance and so on, a RAM 209 which is a temporary storage area, a ROM 505 which stores programs and various initial values and an external IF 506 which is a communication means for notifying a control system such as a brake or the user of the recognition state of the camera.
- a distance image generation unit 503 which generates a distance image by using images outputted from the camera 501 and the camera 502
- a three-dimensional object detection unit 504 which detects a three-dimensional object based on the generated distance image and making collision determination based on the position, distance and so on
- a RAM 209 which is a temporary storage area
- a ROM 505 which stores programs and various initial values
- the three-dimensional object detection unit 504 detects a position of the three-dimensional object or a distance from the own vehicle by generating a histogram of, for example, distances in the vertical direction with respect to a road of the generated distance image and by calculating a peak value thereof, which makes collision determination based on the distance between the detected three-dimensional object (obstacle) and the own vehicle.
- the bus 30 is provided between the three-dimensional object detection unit 504 and the arbitration unit 206 for changing the priority more efficiently, thereby receiving information indicating that the distance with respect to the obstacle is close immediately by the three-dimensional object detection unit 504 through the bus 30 .
- the arbitration unit 206 can perform arbitration so as to give priority to the bus of the stereo camera 102 by changing the bus priority by the not-shown priority determination unit.
- an image processing unit for determining collision with respect to the obstacle by a known method is arranged instead of the three-dimensional object detection unit 504 in the components to configure the system.
- Embodiment 5 is an example in which the priority is flexibly changed by environment of the own vehicle or peripheral environment.
- FIG. 6 shows an example of the above.
- the peripheral monitoring camera in the case where the vehicle speed is less than 30 km/h and the front monitoring camera is out of order, the peripheral monitoring camera is fixed in a state of having the priority and video from the peripheral monitoring camera is displayed on the display, and in the case where the vehicle speed is 30 km/h or more and the front monitoring camera is out of order, display is performed on the display while appropriately changing the priority based on information from various sensors other than the front monitoring camera.
- the priority is fixed or changed appropriately according to the vehicle speed or failure information as described above, thereby switching of display which is highly convenient for the user.
- the present invention is not limited to this.
- the peripheral camera is fixed in the state of having priority and video from the peripheral monitoring camera is displayed when the vehicle speed is less than 30 km/h, and that the priority is not fixed and appropriately changed while giving priority to the front monitoring camera when the vehicle speed is 30 km/h or more as shown in FIG. 6 ( b ) .
- the initial setting corresponding to not only environment of the own vehicle such as the vehicle speed but also peripheral environment may be performed.
- the fixing of the bus and so on to be preferentially accessed may be switched in accordance with the information.
- the front monitoring camera capturing a direction where the obstacle which may collide exists can be fixed in the state of having priority.
- the present invention is not limited to the above embodiments and includes various modification examples.
- the above embodiments have been explained in detail for making the present invention easy to understand, and are not always limited to embodiments having all components explained above. It is possible to perform addition, deletion and replacement of other components with respect to part of components of embodiments. It is also possible to suitably combine respective embodiments in a scope of the gist of the invention.
- part or all of the above respective components, functions, processing units, processing means and so on may be realized by hardware, for example, by designing them as an integrated circuit.
- the above respective components, functions and so on may also be realized by software by a processor interpreting and executing a program for realizing respective functions.
- Information of programs, tables, files and so on for realizing respective functions can be stored in memories such as a flash memory or storages such as a memory card.
- control lines and information lines which are considered to be necessary are shown for convenience of explanation, and all control lines and information lines in products are not always shown. It may be considered that almost all components are mutually connected in actual fact.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Mechanical Engineering (AREA)
- Theoretical Computer Science (AREA)
- Transportation (AREA)
- General Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Chemical & Material Sciences (AREA)
- Combustion & Propulsion (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Traffic Control Systems (AREA)
- Bus Control (AREA)
Abstract
Description
- 1 bus
- 101 controller
- 102 stereo camera
- 103 communication unit
- 104 communication camera
- 105 navigation system
- 106 display
Claims (10)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014139293A JP2016018295A (en) | 2014-07-07 | 2014-07-07 | Information processing system |
JP2014-139293 | 2014-07-07 | ||
PCT/JP2015/065970 WO2016006368A1 (en) | 2014-07-07 | 2015-06-03 | Information processing system |
Publications (2)
Publication Number | Publication Date |
---|---|
US20170180701A1 US20170180701A1 (en) | 2017-06-22 |
US10306207B2 true US10306207B2 (en) | 2019-05-28 |
Family
ID=55064010
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/324,414 Active 2035-09-28 US10306207B2 (en) | 2014-07-07 | 2015-06-03 | Information processing system |
Country Status (4)
Country | Link |
---|---|
US (1) | US10306207B2 (en) |
EP (1) | EP3168750B1 (en) |
JP (1) | JP2016018295A (en) |
WO (1) | WO2016006368A1 (en) |
Families Citing this family (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR102462502B1 (en) * | 2016-08-16 | 2022-11-02 | 삼성전자주식회사 | Automated driving method based on stereo camera and apparatus thereof |
CN114627664A (en) | 2016-08-26 | 2022-06-14 | 松下电器(美国)知识产权公司 | Three-dimensional information processing method and three-dimensional information processing device |
JP6332384B2 (en) * | 2016-09-29 | 2018-05-30 | マツダ株式会社 | Vehicle target detection system |
KR20190098411A (en) * | 2018-02-14 | 2019-08-22 | 현대자동차주식회사 | Vehicle and controlling method of vehicle |
JP6847885B2 (en) * | 2018-03-20 | 2021-03-24 | 株式会社東芝 | Information processing equipment, information processing methods and programs |
CN109017802B (en) * | 2018-06-05 | 2020-12-25 | 长沙智能驾驶研究院有限公司 | Intelligent driving environment perception method and device, computer equipment and storage medium |
US20230254365A1 (en) | 2020-06-24 | 2023-08-10 | Nippon Telegraph And Telephone Corporation | Information processing device, information processing method, and program |
WO2023145690A1 (en) * | 2022-01-26 | 2023-08-03 | キヤノン株式会社 | Image processing system, moving body, image capture system, image processing method, and storage medium |
WO2023145693A1 (en) * | 2022-01-26 | 2023-08-03 | キヤノン株式会社 | Image-processing system, mobile object, image-processing method, and storage medium |
Citations (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0553975A (en) | 1991-08-28 | 1993-03-05 | Matsushita Electric Ind Co Ltd | Bus controller |
JP2002271350A (en) | 2001-03-06 | 2002-09-20 | Fujitsu Ltd | Data processing system and common bus access method |
US20030171865A1 (en) | 2000-05-23 | 2003-09-11 | Martin Moser | Method and device for co-ordinating multiple driving system devices of a vehicle |
US20040001019A1 (en) * | 2002-05-10 | 2004-01-01 | Terumi Nakazawa | Apparatus for controlling vehicle |
US20050111701A1 (en) * | 2003-11-26 | 2005-05-26 | Hitachi, Ltd. | Monitoring system |
US20050143887A1 (en) | 2003-12-26 | 2005-06-30 | Fuji Jukogyo Kabushiki Kaisha | Vehicle driving assist system |
US20060271286A1 (en) * | 2005-05-27 | 2006-11-30 | Outland Research, Llc | Image-enhanced vehicle navigation systems and methods |
US20070154068A1 (en) * | 2006-01-04 | 2007-07-05 | Mobileye Technologies, Ltd. | Estimating Distance To An Object Using A Sequence Of Images Recorded By A Monocular Camera |
JP2008207627A (en) | 2007-02-23 | 2008-09-11 | Auto Network Gijutsu Kenkyusho:Kk | In-vehicle imaging system, imaging device, and display control device |
US20080240506A1 (en) * | 2007-03-30 | 2008-10-02 | Aisin Aw Co., Ltd. | Feature information management apparatuses, methods, and programs |
US20090248270A1 (en) * | 2008-03-25 | 2009-10-01 | Fuji Jukogyo Kabushiki Kaisha | Vehicle drive assist system |
US20100134325A1 (en) * | 2008-11-28 | 2010-06-03 | Fujisu Limited | Image processing apparatus, image processing method, and recording medium |
JP2010136295A (en) | 2008-12-08 | 2010-06-17 | Xanavi Informatics Corp | Imaging display control device and method, and car navigation system and camera system using imaging display control device |
JP2011114850A (en) | 2009-11-30 | 2011-06-09 | Yupiteru Corp | Vehicle-mounted video recorder, reproducing apparatus, and program |
US20110234749A1 (en) * | 2010-03-28 | 2011-09-29 | Alon Yaniv | System and method for detecting and recording traffic law violation events |
US20120078498A1 (en) * | 2009-06-02 | 2012-03-29 | Masahiro Iwasaki | Vehicular peripheral surveillance device |
EP2439714A1 (en) | 2009-06-04 | 2012-04-11 | Toyota Jidosha Kabushiki Kaisha | Vehicle surrounding monitor device and method for monitoring surroundings used for vehicle |
US20130070095A1 (en) * | 2011-09-16 | 2013-03-21 | Harman International Industries, Incorporated | Fast obstacle detection |
WO2013132951A1 (en) | 2012-03-09 | 2013-09-12 | 日立オートモティブシステムズ株式会社 | Distance calculation device and distance calculation method |
US20130250114A1 (en) * | 2010-12-01 | 2013-09-26 | Magna Electronics Inc. | System and method of establishing a multi-camera image using pixel remapping |
US20130282238A1 (en) | 2011-11-16 | 2013-10-24 | Flextronics Ap, Llc | Monitoring state-of-health of processing modules in vehicles |
US20140015494A1 (en) * | 2011-03-29 | 2014-01-16 | Panasonic Corporation | In-vehicle charging device |
US20140118341A1 (en) * | 2012-10-25 | 2014-05-01 | Fujitsu Limited | Image processing apparatus and method |
US20140169630A1 (en) * | 2011-08-02 | 2014-06-19 | Nissan Motor Co., Ltd. | Driving assistance apparatus and driving assistance method |
US20150029012A1 (en) * | 2013-07-26 | 2015-01-29 | Alpine Electronics, Inc. | Vehicle rear left and right side warning apparatus, vehicle rear left and right side warning method, and three-dimensional object detecting device |
US20150213719A1 (en) * | 2012-12-28 | 2015-07-30 | Hyundai Mobis Co., Ltd. | Lateral control apparatus of vehicle and control method of the same |
US20150274062A1 (en) * | 2014-03-27 | 2015-10-01 | Jet Optoelectronics Co., Ltd. | Vehicle monitoring system |
-
2014
- 2014-07-07 JP JP2014139293A patent/JP2016018295A/en active Pending
-
2015
- 2015-06-03 EP EP15819626.1A patent/EP3168750B1/en active Active
- 2015-06-03 WO PCT/JP2015/065970 patent/WO2016006368A1/en active Application Filing
- 2015-06-03 US US15/324,414 patent/US10306207B2/en active Active
Patent Citations (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0553975A (en) | 1991-08-28 | 1993-03-05 | Matsushita Electric Ind Co Ltd | Bus controller |
US20030171865A1 (en) | 2000-05-23 | 2003-09-11 | Martin Moser | Method and device for co-ordinating multiple driving system devices of a vehicle |
JP2002271350A (en) | 2001-03-06 | 2002-09-20 | Fujitsu Ltd | Data processing system and common bus access method |
US20040001019A1 (en) * | 2002-05-10 | 2004-01-01 | Terumi Nakazawa | Apparatus for controlling vehicle |
US20050111701A1 (en) * | 2003-11-26 | 2005-05-26 | Hitachi, Ltd. | Monitoring system |
US20050143887A1 (en) | 2003-12-26 | 2005-06-30 | Fuji Jukogyo Kabushiki Kaisha | Vehicle driving assist system |
JP2005196276A (en) | 2003-12-26 | 2005-07-21 | Fuji Heavy Ind Ltd | Apparatus for supporting operation for vehicle |
US20060271286A1 (en) * | 2005-05-27 | 2006-11-30 | Outland Research, Llc | Image-enhanced vehicle navigation systems and methods |
US20070154068A1 (en) * | 2006-01-04 | 2007-07-05 | Mobileye Technologies, Ltd. | Estimating Distance To An Object Using A Sequence Of Images Recorded By A Monocular Camera |
JP2008207627A (en) | 2007-02-23 | 2008-09-11 | Auto Network Gijutsu Kenkyusho:Kk | In-vehicle imaging system, imaging device, and display control device |
US20080240506A1 (en) * | 2007-03-30 | 2008-10-02 | Aisin Aw Co., Ltd. | Feature information management apparatuses, methods, and programs |
US20090248270A1 (en) * | 2008-03-25 | 2009-10-01 | Fuji Jukogyo Kabushiki Kaisha | Vehicle drive assist system |
US20100134325A1 (en) * | 2008-11-28 | 2010-06-03 | Fujisu Limited | Image processing apparatus, image processing method, and recording medium |
JP2010136295A (en) | 2008-12-08 | 2010-06-17 | Xanavi Informatics Corp | Imaging display control device and method, and car navigation system and camera system using imaging display control device |
US20120078498A1 (en) * | 2009-06-02 | 2012-03-29 | Masahiro Iwasaki | Vehicular peripheral surveillance device |
EP2439714A1 (en) | 2009-06-04 | 2012-04-11 | Toyota Jidosha Kabushiki Kaisha | Vehicle surrounding monitor device and method for monitoring surroundings used for vehicle |
JP2011114850A (en) | 2009-11-30 | 2011-06-09 | Yupiteru Corp | Vehicle-mounted video recorder, reproducing apparatus, and program |
US20110234749A1 (en) * | 2010-03-28 | 2011-09-29 | Alon Yaniv | System and method for detecting and recording traffic law violation events |
US20130250114A1 (en) * | 2010-12-01 | 2013-09-26 | Magna Electronics Inc. | System and method of establishing a multi-camera image using pixel remapping |
US20140015494A1 (en) * | 2011-03-29 | 2014-01-16 | Panasonic Corporation | In-vehicle charging device |
US20140169630A1 (en) * | 2011-08-02 | 2014-06-19 | Nissan Motor Co., Ltd. | Driving assistance apparatus and driving assistance method |
US20130070095A1 (en) * | 2011-09-16 | 2013-03-21 | Harman International Industries, Incorporated | Fast obstacle detection |
US20130282238A1 (en) | 2011-11-16 | 2013-10-24 | Flextronics Ap, Llc | Monitoring state-of-health of processing modules in vehicles |
WO2013132951A1 (en) | 2012-03-09 | 2013-09-12 | 日立オートモティブシステムズ株式会社 | Distance calculation device and distance calculation method |
US20150015673A1 (en) | 2012-03-09 | 2015-01-15 | Hitachi Automotive Systems, Ltd. | Distance calculator and distance calculation method |
US20140118341A1 (en) * | 2012-10-25 | 2014-05-01 | Fujitsu Limited | Image processing apparatus and method |
US20150213719A1 (en) * | 2012-12-28 | 2015-07-30 | Hyundai Mobis Co., Ltd. | Lateral control apparatus of vehicle and control method of the same |
US20150029012A1 (en) * | 2013-07-26 | 2015-01-29 | Alpine Electronics, Inc. | Vehicle rear left and right side warning apparatus, vehicle rear left and right side warning method, and three-dimensional object detecting device |
US20150274062A1 (en) * | 2014-03-27 | 2015-10-01 | Jet Optoelectronics Co., Ltd. | Vehicle monitoring system |
Non-Patent Citations (1)
Title |
---|
Extended European Search Report dated Feb. 2, 2018 for the European Application No. 15819626.1. |
Also Published As
Publication number | Publication date |
---|---|
EP3168750B1 (en) | 2020-04-08 |
WO2016006368A1 (en) | 2016-01-14 |
EP3168750A4 (en) | 2018-03-07 |
US20170180701A1 (en) | 2017-06-22 |
JP2016018295A (en) | 2016-02-01 |
EP3168750A1 (en) | 2017-05-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10306207B2 (en) | Information processing system | |
CN108447302B (en) | Information processing apparatus and recording medium | |
JP6137081B2 (en) | Car equipment | |
EP3416368B1 (en) | Calibration system | |
US10183621B2 (en) | Vehicular image processing apparatus and vehicular image processing system | |
JP6936679B2 (en) | Information processing equipment and programs | |
US20160272024A1 (en) | Overhead view for hitch connection | |
US10152890B2 (en) | On-vehicle camera device | |
US11003922B2 (en) | Peripheral recognition device, peripheral recognition method, and computer readable medium | |
US10796167B2 (en) | Periphery recognition device | |
CN109823340A (en) | It is a kind of control vehicle parking method, control equipment | |
JP4848644B2 (en) | Obstacle recognition system | |
US20200353866A1 (en) | Parking Support Device | |
US10354148B2 (en) | Object detection apparatus, vehicle provided with object detection apparatus, and non-transitory recording medium | |
US11455840B2 (en) | Method, device and computer-readable storage medium with instructions for processing data in a motor vehicle for forwarding to a back end | |
JP2018139120A (en) | Information processing system | |
US20230168104A1 (en) | Image processing device | |
US20200342758A1 (en) | Drive assistance device, drive assistance method, and recording medium in which drive assistance program is stored | |
CN111766866A (en) | Information processing device and automatic travel control system including the same | |
EP3865815A1 (en) | Vehicle-mounted system | |
JP2020042846A (en) | Information processing system | |
JP2018136917A (en) | Information processing apparatus, information processing method, and program | |
WO2016002418A1 (en) | Information processing device | |
JP7145227B2 (en) | sign recognition device | |
JP2020042846A5 (en) |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: HITACHI AUTOMOTIVE SYSTEMS, LTD., JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MATONO, HARUKI;AOKI, TOSHIYUKI;MUTO, YOSHIYUKI;SIGNING DATES FROM 20161130 TO 20161201;REEL/FRAME:040875/0018 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: HITACHI ASTEMO, LTD., JAPAN Free format text: CHANGE OF NAME;ASSIGNOR:HITACHI AUTOMOTIVE SYSTEMS, LTD.;REEL/FRAME:056299/0447 Effective date: 20210101 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |