DE102019007001B4 - VEHICLE IMAGE PROCESSING DEVICE, VEHICLE IMAGE PROCESSING METHOD, PROGRAM and STORAGE MEDIUM - Google Patents

Abstract

A vehicle image processing device (1), comprising:a plurality of buffers (2) configured to accumulate portions of image data (G) individually and sequentially inputted from a plurality of cameras (8) mounted in a vehicle (9 ) are installed and each record different surrounding areas of the vehicle (9) in order to link the parts of image data (G) with the cameras (8); a processor (3) which is configured to process the buffer (2) based on status information (S) for determining driving conditions of the vehicle (9), and for acquiring the portion of image data (G) from the selected buffer (2) to perform image processing thereon; a signal line (4) which contains the portions of image data (G) in the buffers (2) to the processor (3); and a transfer controller (5) configured to output the portion of image data (G) in the buffer (2) required by the processor (3) to the signal line (4) by selecting based on the status information (S), wherein the status information (S) includes a vehicle speed (V), and the processor (3) carries out the selection of the buffer (2) based on a comparison of the vehicle speed (V) and threshold values (L), the threshold values (L) indicating a start of switching. Threshold (Ls) and a switching completion threshold (Le), in which a difference from the switching start threshold (Ls) is a first value, the processor (3) is configured to carry out the image processing on a predetermined number of pieces of the image data (G) per unit time, and when the vehicle speed (V) is between the switching start threshold (Ls) and the switching completion threshold (Le), the processor (3) switches the buffers (2) to be selected such that when the Vehicle speed (V) approaches the switching completion threshold (Le) from the switching start threshold (Ls), the predetermined number of pieces of image data (G) on which the image processing is performed per unit time before the vehicle speed (V) reaches the switching start threshold. Threshold (Ls) is reached, is replaced with the predetermined number of pieces of image data (G) after the vehicle speed (V) has reached the switching completion threshold (Le).

Description

Technical field of the invention

The present invention relates to outdoor environment detection performed with a plurality of cameras installed in a vehicle.

Background of the invention

In recent years, many technologies have been developed in which a plurality of cameras installed in a vehicle are used to perform external environment detection regarding the vehicle, and thus braking depending on the situation is automatically applied to the vehicle with the result will ensure that driving assistance and automatic driving of vehicles advances. In the above-described technologies, since not only the present vehicle but also people around it, other vehicles and the like move, images captured by the cameras must be processed in real time for feedback from the external environment detection for controlling alarms, braking and the like. However, if a processor is provided in each of the cameras, then the cost of the vehicle will increase significantly.

To solve such a problem, Patent Document 1 discloses that in a configuration in which a plurality of stereo cameras with I/Os are individually connected and in which a CPU (Image Processing IC) uses a RAM to perform image processing on captured images as needed To perform image data, a situation assessment/image selection section determines which of the images recorded with the stereo cameras is sent to an image processing section according to one

Support mode is output, and thus the image processing section (a pre-processing section and the subsequent sections) performs the image processing. In this way, the need to use a computing device whose processing capability is high and the like can be avoided, and thus the cost can be reduced.

Similarly, Patent Literature 2 discloses that a CPU (calculation section) which can perform front detection processing for the image information of a front camera inputted into an input section and rear detection processing for the image information of a rear camera, respectively executed as the front detection processing and the back detection processing Selects and performs processing based on the state information of the present vehicle (information indicating forward movement and backward movement and driving state). In this way, compared to the case where CPUs for front detection processing and back detection processing are provided separately, their configuration can be simplified.

Furthermore, Patent Literature 3 discloses that a part of captured images obtained by imaging the external environment of a vehicle is selected as a target area according to conditions of the vehicle including the driving environment and the driving state of the vehicle, and image processing is performed on the selected target area , and thus a processing load can be reduced compared to the case where image processing is performed on the entire captured images.

Patent Document 4 discloses providing an image pickup device and an image pickup method for displaying the optimal observation area without depending on the conditions under which the vehicle enters the road or the installation situation of the plurality of cameras. In addition, a provision of a plurality of peripheral image capture means which obtain images of the periphery of a vehicle is disclosed, the peripheral image capture means which receives the images displayed to the driver of the vehicle and the image area of the images obtained by the peripheral image capture means, that are displayed to the driver of the vehicle can be selected.

Patent Document 5 discloses an image processor for a vehicle having a plurality of cameras and a monitor that displays a panoramic image derived from a plurality of images generated by the plurality of cameras. The processor includes input buffers for storing camera image data captured by corresponding camera modules and output buffers for storing a panoramic image. The camera image data for generating the panoramic image is retrieved from the input buffers. The camera image data is stored in an output buffer using a reconfigured address translation table. The reconfigured address conversion table is an address conversion table obtained by reconfiguring an address conversion table assuming that the camera modules are free from vehicle installation errors. The reconfiguration is carried out based on parameters which are used to correct vehicle installation errors of camera modules.

Patent Document 6 discloses an image processing device that includes stereo cameras and detects the distance to a target in the vicinity of a vehicle. The image processing device includes sensors installed in the vehicle; determines a vehicle situation by combining detection signals from the sensors and switches from image data from a stereo camera for driving support to image data from a stereo camera for door opening/closing support; performs distortion corrections unique to the plurality of stereo cameras on the selected image data so that a correlation calculation common to the plurality of stereo cameras can be performed; performs the correlation calculation on two sets of image data from the single stereo camera; and detects the distance to the target using a calculation result of the correlation calculation.

Patent Document 7 discloses an image processor that produces a more natural synthesized image from camera images captured by a plurality of cameras that capture the surroundings of a vehicle. A parameter storage section stores a plurality of image synthesis parameter groups that represent the correspondence between the camera images and the synthesized image and have different spatial or temporal resolution relationships. A parameter selection section selects the image synthesis parameter group according to the output of a vehicle motion detection section for detecting the movement of the vehicle such as the traveling speed and direction. An image synthesis section generates the synthesized image from the camera images according to the selected image synthesis parameter group.

Patent Document 8 discloses a traffic signal detection device including a narrow-angle camera, a wide-angle camera having a larger viewing angle than the narrow-angle camera, and a traffic signal detector configured to detect a traffic signal from at least one narrow-angle image captured by the narrow-angle camera and one captured by the wide-angle camera captured wide-angle image. The traffic signal detector selects, as a detection result, a traffic signal detected from the narrow-angle image or a traffic signal detected from the wide-angle image based on a deceleration start area in which a vehicle should start deceleration, the deceleration start area being from a position of a Traffic signal with respect to the vehicle and a speed of the vehicle is calculated to stop the vehicle at a stopping position with respect to the traffic signal.

Patent literature

• Patent specification 1: JP 2013 - 93 013 A
• Patent specification 2: JP 2018 - 95 067 A
• Patent specification 3: JP 2015 - 210 584 A
• Patent specification 4: US 2008/0143833 A1
• Patent specification 5: US 2006/0197761 A1
• Patent specification 6: US 2013/0088578 A1
• Patent specification 7: EP 1 150 252 A2
• Patent specification 8: EP 3 306 589 A1

Summary of the invention

However, in Patent Document 1, all image data generated by shooting with each stereo camera is temporarily input to the CPU to perform image selection with the situation judgment/image capture section. Thus, the CPU must perform processing for picking up all the image data at a stage before the above-described image selection, and accordingly an operation time allocated to the image processing is reduced. Similarly, in Patent Document 2, the CPU receives inputs from all the image information of the front camera and the rear camera, performs correction processing (deformation correction) on all the image information that has been input, and thereafter performs the selection described above. Thus, for example, the CPU must perform processing for picking up all image data and processing of image information which has not been selected, and thus an operation time allocated to image processing is reduced accordingly. As described above, if the amount of processing performed by a processor for image processing is increased to process captured images in real time, then the processor must have a higher processing capability, with the result that the cost of the vehicle is unlikely to be sufficiently reduced.

In view of the above situation, an object of at least one embodiment of the present invention is to provide a vehicle image processing apparatus which can perform image processing of images captured by a plurality of cameras more quickly while reducing the cost of a vehicle.

• Eine Fahrzeugbildverarbeitungsvorrichtung gemäß zumindest einem Ausführungsbeispiel der vorliegenden Erfindung umfasst: eine Vielzahl von Puffern, die konfiguriert sind, um Teile von Bilddaten, die von einer Vielzahl von Kameras einzeln und sequenziell eingegeben sind, die in einem Fahrzeug installiert sind, zu akkumulieren, um die Teile von Bilddaten mit den Kameras zu verknüpfen; einen Prozessor, der konfiguriert ist, um den Puffer basierend auf den Zustandsinformationen des Fahrzeugs auszuwählen, und um den Teil von Bilddaten aus dem ausgewählten Puffer zu akquirieren, um diesbezüglich eine Bildverarbeitung durchzuführen; eine Signalleitung zum Übertragen der Teile von Bilddaten in den Puffern zu dem Prozessor; und eine Transfersteuerung, die konfiguriert ist, um den Teil von Bilddaten in dem Puffer, der von dem Prozessor benötigt wird, an die Signalleitung auszugeben.

According to the present invention, there are provided a vehicle image processing apparatus according to independent claim 1, a vehicle image processing method according to independent claim 13, a vehicle image processing program according to independent claim 14, and a computer-readable storage medium according to independent claim 15. Advantageous modifications can be found in the dependent patent claims.

• A vehicle image processing apparatus according to at least one embodiment of the present invention includes: a plurality of buffers configured to accumulate pieces of image data individually and sequentially inputted from a plurality of cameras installed in a vehicle to form the pieces to link image data to the cameras; a processor configured to select the buffer based on the state information of the vehicle and to acquire the portion of image data from the selected buffer to perform image processing thereon; a signal line for transmitting the pieces of image data in the buffers to the processor; and a transfer controller configured to output to the signal line the portion of image data in the buffer required by the processor.

In the configuration of (1) described above, the processor for image processing does not perform any processing on the image data until the image data is transferred with the transfer control from the buffer in which the image data captured by each of the cameras is stored, and it performs image processing only on the image data acquired through the transmission. In this way, the load on the processor can be reduced and thus a greater amount of operating time can be allocated to image processing. In addition, the processor determines, for example, from the image data of the individual cameras assigned in the buffers based on the status information of the vehicle, such as the vehicle speed and the switching position, the buffers which must be set to the target for the image processing in order to be adjusted to the target for to limit the cameras set to image processing to at least a part thereof, and thus the number of pieces of image data that need to be processed per unit time can be reduced.

As described above, the image processing processor sets the image data of the cameras determined based on the state information of the vehicle to the target for image processing, and does not perform any processing until the image data set to the target has been acquired, with the result that environmental detection (the detection of a person and an object) around the vehicle can be carried out more quickly by the cameras. Thus, providing the processor for each of the cameras and using an expensive processor with better processing capability can be avoided, and thus the cost of the vehicle can be reduced.

(2) In some embodiments, according to the configuration of (1) described above, the processor determines at least one of the buffers as a selection target based on the state information, and sets a selection frequency of the at least one buffer included in the selection target larger than selection frequencies of the others Buffer to sequentially select the at least one buffer included in the selection target.

In the configuration of (2) described above, the processor determines from the cameras one or more cameras (one or more buffers) set to the target for image processing, acquires the image data from the one or more buffers corresponding to the one or the plurality of cameras designated to perform the image processing sequentially, thereby performing the image processing on a predetermined number of pieces of image data per unit time. As described above, the cameras set to the target for image processing are restricted to at least a part of the cameras, and thus the number of pieces of image data to be processed per unit time can be reduced, with the result that Environment detection (detection of movement of a person and an article) around the vehicle can be performed by the cameras in a satisfactory manner in real time.

(3) In some embodiments, according to the configuration of (1) and (2) described above, the vehicle image processing apparatus further includes: a detection unit having the buffers and the transfer controller provided therein; a working buffer configured to store the portion of image data output by the transfer controller to the signal line and subjected to image processing performed by the processor; and a processor unit having the processor and the working memory provided therein, and wherein the signal line is configured to be able to interconnect the buffers in the acquisition unit and the working buffer in the processor unit.

In the configuration of (3) described above, the portions of image data of each camera managed with the buffers are transmitted through the signal line for connecting the detection unit and the processor unit, and transferred to the working buffer provided in the processor unit and subjected to image processing in the processor. The management and transfer processing of the buffers in the detection unit are separated from the image processing in the processing unit in terms of hardware in this way, and thus, as described above, environmental detection (the detection of a person and an article) around the vehicle can be performed the cameras are carried out in a satisfactory manner in real time.

(4) In some embodiments, according to the configuration of (1) to (3) described above, the state information includes a vehicle speed, and the processor performs the selection of the buffer based on the comparison of the vehicle speed and threshold values.

In general, there is an upper limit on the number of pieces of image data on which a processor can perform image processing per unit time. Thus, if the pieces of image data of the cameras are processed sequentially as the number of cameras (buffers) is increased, then the number of pieces of image data per camera subjected to image processing at a unit time is reduced. If this number is reduced, a longer delay in image processing occurs with respect to the pieces of image data sequentially supplied from each of the cameras, and thus a time delay in environment recognition by the image processing of the cameras is increased.

In the configuration of (4) described above, the processor makes a selection based on the comparison of the vehicle speed and the threshold values, such as determining the buffers which are set as the target for the image processing. If the vehicle speed is increased, then the time delay in environment detection by the image processing is more likely to become severe, whereas if the vehicle speed is reduced, the time delay is more likely to be within an acceptable range. Thus, when the vehicle speed is reduced, even if a larger number of cameras are set to the image processing target, driving assistance for the vehicle can be performed appropriately. Therefore, the cameras set to the target for image processing are selected by the relationship with the vehicle speed, and thus even the processor whose processing capability is relatively low can provide driving assistance to the vehicle through the environment recognition using the cameras in one be carried out in a satisfactory manner in real time.

(5) In some embodiments according to the configuration of (4) described above, the cameras include one or more of the cameras for recording the surroundings of the vehicle, the thresholds include a first threshold, and if the vehicle speed is less than the first threshold, then selects the processor selects the buffer in which the portion of image data recorded with the one or more cameras for recording the entire surroundings of the vehicle is accumulated.

For example, if the vehicle is stopped before it has been moved or is in a state (state in which driving has been started) in which the vehicle is moved at a very low speed due to a creeping phenomenon or the like The vehicle can be moved to different positions depending on a steering angle. Thus, with the near-field cameras, which have a relatively wide viewing angle in order to be able to appropriately record a near field or the like, it is necessary to monitor the entire surroundings of the present vehicle (at least the entire orientation in a horizontal direction).

In the configuration of (5) described above, when the above-described state (state in which driving has been started) has been detected with the first threshold value, the processor provides images captured by one or more of the cameras (such as one or more of the near-field cameras) required to obtain images around the vehicle (the entire orientation) are recorded on the target for image processing. In this way, driving assistance corresponding to conditions of the vehicle can be carried out in a satisfactory manner in real time by monitoring with the cameras.

For example, if the cameras, such as a total of four cameras for recording the front, back, left side and right side of the vehicle are installed to obtain the entire environment, then the number of cameras aimed at the target for the Image processing settings are more likely to be increased. If the images from the cameras are sequential are switched one-to-one and then the image processing is performed when the number of cameras is increased, then the processing cycle of the image data in the individual cameras (individual buffers) is lengthened. However, since the vehicle is moving at a very low speed, as long as the image processing is permissible (not problematic) as the driving assistance for the vehicle, the image processing can be performed on the images captured by the individual cameras.

(6) In some embodiments according to the configuration of (4) and (5) described above, the cameras include a camera for recording an area in front of the vehicle and a camera for recording an area behind the vehicle, the state information further includes a switching position of one Transmission of the vehicle, the threshold values include the first threshold value and a second threshold value which is greater than the first threshold value, and when the vehicle speed is equal to or greater than the first threshold value and less than the second threshold value, and the shift position indicates forward movement, the processor only selects the buffer in which the part of image data taken by the camera for recording the area in front of the vehicle is accumulated, whereas when the switch position indicates a backward movement, the processor only selects the buffer in which the part of image data captured by the camera for capturing the area behind the vehicle is accumulated.

For example, in a state (low-speed traveling state) in which the vehicle can be quickly moved in a lateral direction with respect to the traveling direction depending on the steering angle, it is necessary to use the near-field cameras (described above) and the like in order to detect an area in the The direction of travel of the present vehicle is monitored extensively.

In the configuration of (6) described above, when the above-described state (low-speed running state) is detected with the first threshold value and the second threshold value, the processor provides images taken with the cameras for taking images in the traveling direction of the vehicle, on the target for image processing. In this way, driving assistance corresponding to conditions of the vehicle can be performed in a satisfactory manner in real time by monitoring with the cameras.

(7) In some embodiments according to the configuration of (4) to (6) described above, the cameras include a far-field camera that captures a far-field area in the traveling direction of the vehicle, and a near-field camera that captures a near-field area in the traveling direction of the vehicle, and which has a wider viewing angle than the far-field camera, the thresholds include the second threshold, and if the vehicle speed is equal to or greater than the second threshold, then the processor selects the buffer in which the portion of image data captured by the far-field camera is accumulated whereas when the vehicle speed is less than the second threshold, the processor selects the buffer in which the portion of image data captured by the near-field camera is accumulated.

In a driving condition in which the vehicle speed is relatively so high that the vehicle cannot quickly turn in one direction by steering, the time required for feedback (e.g. automatic braking in the vehicle or a warning to the driver) depends on the Environment detection (the detection of a person and an article) through image processing is permitted to the driver of the vehicle or the like, further shortened. Thus, it is necessary to monitor a distant area from the vehicle in order to detect a target (a person and an article) in a distant area earlier.

In the configuration of (7) described above, when the above-described driving state is detected with the second threshold, the processor sets images captured by the far-field cameras for photographing a distant area in the traveling direction of the vehicle as the target for image processing a. As described above, for example, images captured with a smaller number of cameras such as one camera are adjusted to the target for image processing, and thus driving assistance can be provided corresponding to conditions of the vehicle by monitoring with the cameras in a satisfactory manner carried out in real time.

(8) In some embodiments, according to the configuration of (4) to (7) described above, the threshold values include a switching start threshold and a switching completion threshold, in which a difference from the switching start threshold is a first value, the processor is configured to perform the image processing on a predetermined number of pieces of the image data per unit time, and when the vehicle speed is between the switching start threshold and the switching completion threshold, the processor switches the buffers to be selected such that when the vehicle speed approaches the switching completion threshold from the switching start threshold, the predetermined number of pieces of image data on which the image processing is performed per unit time, before the vehicle speed reaches the switching start threshold, is replaced with the predetermined number of pieces of image data after the vehicle speed reaches the switching completion threshold.

When the cameras (buffers) set to the target for image processing are switched accurately at a predetermined threshold, for example, if the vehicle speed obtained from the vehicle includes an error, appropriate buffers corresponding to the vehicle speed around the threshold are not selected , with the result that appropriate environmental detection is unlikely to be performed.

According to the configuration of (8) described above, the cameras (buffers) set to the target for the image processing of the processor are gradually switched from the previous stage to be at the predetermined threshold values (such as the first threshold value described above and the second threshold value) to be switched completely. In this way, even if the vehicle speed obtained from the vehicle has an error, the external environment detection regarding the vehicle can be performed appropriately.

(9) In some embodiments, according to the configuration of (8) described above, when the vehicle speed reaches the switching start threshold, the processor first switches the camera corresponding to the buffer, which must be selected when the vehicle speed reaches the switching completion threshold , and the buffer corresponding to the camera, which is linked to at least one recording direction.

According to the configuration of (9) described above, the external environment detection regarding the vehicle can be performed appropriately even if the vehicle speed obtained from the vehicle has an error.

(10) In some embodiments according to the configuration (1) to (9) described above, the vehicle image processing device further includes: an acquiring section that acquires characteristic information including an instruction for the monitoring direction at a specific location, and the processor selects the buffers based on the driving position and the property information.

According to the configuration of (10) described above, the processor sets the image data of the camera for photographing the area in the monitoring direction indicated by the property information to the target for image processing. Although it may be necessary to monitor the specific direction in a focused manner in the vehicle traveling in various places, such as a place where accidents frequently occur, the cameras set to the target for image processing switched based on the property information, and thus the driving assistance can be carried out, which is safer.

(11) In some embodiments according to the configuration of (1) to (10) described above, the vehicle represents a commercial vehicle, the state information includes the vehicle speed and the steering angle, the processor determines whether a turn of the vehicle is a turn based on the state information to the right or a left turn, and when the vehicle turns right, the processor selects the buffer in which the portion of image data is accumulated by capturing at least an area on a right side in a forward direction and an area on a left side in a backward direction with respect to the traveling direction, whereas when the vehicle turns left, the processor selects the buffer in which the portion of image data obtained by capturing at least an area on the left side in the forward direction is accumulated and an area on the right side in the backward direction with respect to the traveling direction.

For example, since a commercial vehicle such as a forklift has a short wheelbase and a large steering angle, it is necessary to pay attention to the lateral direction compared to the forward direction with respect to the traveling direction when driving is started in a state in which a steering wheel is turned. In particular, it is necessary to not only the right side in the forward direction with respect to the traveling direction, but also the left side in the reverse direction with respect thereto at the time of turning right, and not only the left side in the forward direction with respect to the direction of travel, but also the right side in the reverse direction with respect to the Simultaneously monitor the time of a left turn.

According to the configuration of (11) described above, the processor determines the turning direction based on the vehicle speed and the steering angle, and selects the cameras set to the target for image processing based on the determination result. As described above, according to the turning direction (steering angle) of the vehicle, the image data from not only one side such as the right side but also the opposite side (left side) is set to the target for image processing, and thus at the time a lateral turn while paying attention to the direction opposite to the direction of travel, for example, the detection of a person and an article is performed with the result that driving assistance can be performed, which is safer.

(12) In some embodiments according to the configuration of (11) described above, the cameras include a right-side recording camera for recording an area on the right side of the vehicle and a left-side recording camera for recording an area on the left side thereof, and when the vehicle is after when the vehicle turns right, the processor performs the image processing on the area of a part on a left side of the parts of image data captured by the right-side capture camera and the left-side capture camera individually, whereas when the vehicle turns left, the processor performs the image processing in the area of a part on a right side of the parts of image data individually captured by the right-side pickup camera and the left-side pickup camera.

According to the turning direction (steering angle) of the vehicle, the image data from not only one side such as the right side but also the opposite side (left side) is set to the target for image processing, and thus the number of cameras that are on the target for image processing is increased, with the result that the frequency at which image processing is performed by the processor on the image data accumulated in each of the buffers (cameras) is reduced (the cycle is reduced).

According to the configuration of (12) described above, the processor performs the image processing on the area of a part of each part of image data acquired when the vehicle turns. For example, at the time of a right turn, the camera on the right processes the area (the left side of the image) of only the front of the vehicle, and the camera on the left processes the area (the left side of the image) of only the rear of the vehicle in a limited manner according to the steering angle, that is, the image processing is performed only on the areas of parts with the result that the lengthening of the acquisition cycle of the image data from each of the buffers can be reduced. In this way, even if the number of cameras is increased, driving assistance can be performed in a satisfactory manner in real time, which is safer.

(13) In some embodiments according to the configuration of (1) to (12) described above, the state information includes the steering angle and the shift position of the transmission of the vehicle, and the processor performs image processing in an area of one based on the steering angle and the shift position Part contained in the parts of image data acquired by the buffers.

According to the configuration of (13) described above, based on the steering angle and the shift position, a part of each piece of image data is extracted to be adjusted to the target for image processing. In this way, in each part of image data, image processing can be dispensed with in the sub-area that does not include the vehicle's travel path predicted based on the steering angle and the switching position. Thus, the load of image processing on each piece of image data can be reduced, and thus the number of pieces of image data on which image processing can be performed per unit time can be increased. Thus, performance time required at high vehicle speed or the like can be met.

(14) In some embodiments according to the configuration of (1) to (13) described above, the cameras include a first camera for photographing an area in a first direction of the vehicle and a second camera for recording an area in a direction different from the first direction Direction, and the vehicle image processing device further includes a setting section that sets the shooting parameter for each of the cameras based on the brightness of the pieces of image data captured by the first camera and the second camera individually.

According to the configuration of (14) described above, the front light and the Backlight of the cameras set on the target for image processing is detected so that the shooting parameters for the cameras are adjusted, and thus an influence caused by brightness variations caused by sunlight can be minimized, with the result that stable environment detection can be carried out.

(15) A vehicle image processing method according to at least one embodiment of the present invention is performed by a processor of a computer, the computer comprising: a plurality of buffers configured to accumulate portions of image data obtained from a plurality of cameras individually and sequentially are entered that are installed in a vehicle to associate the pieces of image data with the cameras; the processor configured to select the buffer based on the state information of the vehicle including a vehicle speed, and to acquire the portion of image data from the selected buffer to perform image processing thereon; a signal line for transmitting the pieces of image data in the buffers to the processor; and a transfer controller configured to output to the signal line the portion of image data in the buffer required by the processor, and wherein the vehicle image processing method includes a step of performing selection of the buffer based on a comparison of vehicle speed and threshold values .

According to the configuration of (15) described above, the same effects as described in (1) above are achieved.

(16) A vehicle image processing program according to at least one embodiment of the present invention instructs a processor of a computer to perform a step of performing selection of a buffer based on the comparison of a vehicle speed and threshold values, and the computer includes: a plurality of buffers configured are to accumulate pieces of image data individually and sequentially inputted from a plurality of cameras installed in a vehicle to associate the pieces of image data with the cameras; the processor configured to select the buffer based on state information of the vehicle including the vehicle speed, and to acquire the portion of image data from the selected buffer to perform image processing thereon; a signal line for transmitting the pieces of image data in the buffers to the processor; and a transfer controller configured to output to the signal line the portion of image data in the buffer required by the processor.

According to the configuration of (16) described above, the same effects as described in (1) above are achieved.

(17) A storage medium according to at least one embodiment of the present invention is a computer-readable storage medium that stores the vehicle image processing program described in (16) above.

According to the configuration of (17) described above, the vehicle image processing program stored in the storage medium is executed, and thus the same effects as described in (1) above are achieved.

According to at least one embodiment of the present invention, there is provided a vehicle image processing apparatus which can perform image processing on images captured by a plurality of cameras more quickly while reducing the cost of a vehicle.

Brief description of the drawings

• 1 is a diagram schematically showing the hardware configuration of a vehicle image processing apparatus according to an embodiment of the present invention;
• 2 is a diagram showing an installation example of a plurality of cameras in a vehicle according to the embodiment of the present invention;
• 3 is a diagram showing the functional blocks of the vehicle image processing apparatus according to the embodiment of the present invention;
• 4 is a flowchart showing a method of determining buffers of a selection target according to the embodiment of the present invention and selecting cameras according to a vehicle speed;
• 5 is a diagram showing an order in which the buffers (cameras) are selected when the vehicle speed is less than a first threshold (V < L1) according to the embodiment of the present invention;
• 6 is a diagram showing an order in which the buffers (cameras) are selected when the vehicle speed is equal to or greater than that according to the embodiment of the present invention first threshold and is less than a second threshold (L1 ≤ V <L2);
• 7 is a diagram showing an order in which the buffers (cameras) are selected when the vehicle speed is equal to or greater than the second threshold (L2 < V) according to the embodiment of the present invention;
• 8th is a diagram for illustrating the gradual switching of the buffers around the first threshold when the vehicle speed is increased according to the embodiment of the present invention;
• 9 is a diagram for illustrating the gradual switching of the buffers around the second threshold when the vehicle speed is increased according to the embodiment of the present invention;
• 10 is a diagram showing directions and areas monitored when a forklift vehicle turns according to the embodiment of the present invention;
• 11 is a flowchart showing a method for determining the buffers when the forklift vehicle turns according to the embodiment of the present invention;
• 12 is a diagram showing a travel path in image data set to the target for image processing according to the embodiment of the present invention; and
• 13 is a diagram for illustrating a case in which the vehicle image processing apparatus according to the embodiment of the present invention determines the backlit state of the vehicle.

Detailed description of the invention

Some embodiments of the present invention will be described below with reference to accompanying drawings. However, the dimensions, materials, shapes, relative arrangements and the like of constituent components described by way of example or shown in drawings are not intended to limit the scope of the present invention. They only represent simple descriptive examples.

For example, an expression of relative or absolute arrangement such as "in one direction", "along one direction", "parallel", "orthogonal", "centered", "concentric" and "coaxial" should not be construed as meaning only the arrangement is displayed in a strictly literal sense, but also includes a state in which the arrangement is relatively displaced by a deviation or by an angle or a distance, whereby the same function can be achieved.

For example, an expression of a same state, such as "the same, the same, the same," "same," and "uniform" should not be construed as indicating only the state in which the feature is, strictly speaking, the same also includes a condition in which there is a deviation or a difference that can still achieve the same function.

Further, for example, an expression of a shape such as a rectangular shape or a cylindrical shape should not be construed as only a geometrically strict shape, but also include a shape with unevenness or chamfered corners within the range in which the same effect can be achieved.

On the other hand, an expression such as "comprising", "comprising", "having", "containing" and "constituting" does not intend to exclude other components.

1 is a diagram schematically showing the hardware configuration of a vehicle image processing device 1 according to an embodiment of the present invention. 2 is a diagram showing an installation example of a plurality of cameras 8 in a vehicle 9 according to the embodiment of the present invention. 3 is a diagram showing the functional blocks of the vehicle image processing apparatus 1 according to the embodiment of the present invention.

The vehicle image processing device 1 (hereinafter, as necessary, the image processing device 1) is a device for performing image processing on images (image data G) individually captured with a plurality of cameras 8 installed (included) in the vehicle 9, which is a general one Automobile or a commercial vehicle, such as a forklift. As in 1 As shown, the cameras 8 in the image processing device 1 are individually connected with separate wires or the like, and the images captured by the individual cameras 8 are input to the image processing device 1. The number of cameras 8 is arbitrary and is preferably determined according to the purpose thereof and the specifications (such as a viewing angle and a distance that can perform shooting) of the cameras. In the Individual cameras 8, images recorded with the cameras 8 are continuously input into the image processing device 1.

According to the in 1 and 2 In the exemplary embodiment shown, each of the cameras 8 generates a predetermined number of images per second. According to the in 2 In the embodiment shown, a total of six cameras 8 are installed in the vehicle 9, and the individual cameras are installed to record in the circumferential direction of the vehicle 9. In particular, four of the six cameras 8 are near-field cameras 81 for near-field recording, and two cameras are far-field cameras 82 for far-field recording. The near-field camera 81 has a wider viewing angle than the far-field cameras in order to be able to appropriately record a relatively close field. In contrast, the far-field camera 82 has a more restricted viewing angle than the near-field camera 81 in order to be able to appropriately record a relatively distant field. Thus, the four near-field cameras 81 are separately installed at four locations of the front, rear, left side and right side of the vehicle 9 to completely capture the entire perimeter of the vehicle 9 with the four near-field cameras 81. The two far-field cameras 82 are separately installed at two locations of the front (where the driver's vision is provided) and the rear of the vehicle 9 to record distant areas in the forward and backward directions.

So that the captured images which are input from the cameras 8 installed in the vehicle 9 as described above are sequentially shown in 1 shown, are processed in real time, the image processing device 1 comprises a plurality of buffers 2, a processor 3, a signal line 4 and a transfer controller 5.

The configurations of the same are described individually.

The buffers 2 are storage sections configured to accumulate pieces of image data G input from the cameras 8 installed in the vehicle 9 for each of the cameras 8 individually and sequentially. As in 1 As shown, a dedicated buffer 2 is provided for each of the cameras 8, and thus the image data G is managed (stored) for each of the cameras 8. According to the in 1 In the embodiment shown, each of the buffers 2 represents a queue (FIFO), and thus the pieces of image data G are taken out in an order in which the pieces of image data G are input into the buffers 2. Each of the buffers 2 can have a ring buffer structure. Each of the buffers 2 may be set to have such a capacity that only a small number of pieces of image data G can be stored, and a memory having a smaller capacity is used so that the cost can be reduced. According to the in 1 In the embodiment shown, an A/D converter 22 converts the analog information of the image captured by each of the cameras 8 into digital information, and thus the image data G is accumulated in each of the buffers 2. The transfer controller 5, which will be described below, manages the individual buffers 2 to accumulate (store) the image data G in the individual buffers 2.

The processor 3 selects the buffer 2 among the above-described buffers 2 based on, for example, the state information S of the vehicle 9, such as a vehicle speed V and a shift position Sp, and acquires the image data G from the selected buffer to perform image processing. Specifically, each processor 3 operates (such as computing data) according to the instruction of a program (image processing program) loaded in a main storage device to sequentially acquire the image data G from any one of the buffers 2, and to perform image processing (such as for example, the capture of a person and an article) on the image data G that has been acquired. As in 1 shown, the number of processors 3 is smaller than the number of buffers 2 in order to reduce the cost of the vehicle 9. According to the in 1 In the exemplary embodiment shown, the processor 3 contained in the image processing device 1 has a CPU shown in the figure.

In particular, the processor 3 is operated according to the image processing program, which has an in 3 function shown to determine at least one of the buffers 2 as a selection target among the buffers 2 based on the state information S of the vehicle 9, and sets the selection frequencies of the buffers 2 included in the selection target larger than the selection frequencies of the other buffers 2, which are not included in the selection target, to sequentially select one or more of the buffers 2 included in the selection target. In particular, for example, by setting the selection frequencies of the buffers 2 not included in the selection target to a significantly low frequency such as zero, they can be prevented from being selected by the processor 3. According to the in 3 In the embodiment shown, the image processing program instructs a computer to realize: a selection target buffer determining section 71 which determines the buffers 2 of the selection target; an image data acquiring section 72 which sequentially selects one or more of the buffers 2 designated and included in the selection target selects, and which acquires the image data G of the selected buffers 2; and an image processing section 73 that performs image processing on the acquired image data G.

Since in general the number of pieces of image data G on which a processor 3 can perform image processing per unit time is limited when the image data G from all cameras 8 are monitored in real time, for example, a processor 3 must be provided which has an excellent Has processing capability, or the processor 3 must be provided for each of the cameras 8. However, there is generally a priority (importance) in the direction that needs to be monitored according to driving conditions, and, for example, the need for monitoring the direction opposite to the driving direction at the time of driving is small. Thus, each time the driving conditions of the vehicle 9 are determined based on the state information S of the vehicle 9, the cameras 8 that need to be set to the target for image processing are at least a part of the cameras 8 installed in the vehicle 9 are installed, limited according to the details of driving assistance being performed, with the result that the processor 3 performs the image processing. In this way, even if a plurality of buffers 2 are included in the selection target, the number of buffers 2 set to the target for image processing is minimized, the frequency at which the processor 3 acquires the image data G becomes , for each of the buffers 2 is increased (the cycle in which the image processing is performed is shortened), with the result that environment detection using the cameras 8 can be performed in real time.

The signal line 4 is a transfer path for transmitting the image data G in each of the above-described buffers 2 to the processor 3. The signal line 4 may be configured to transmit the image data G serially, or may be configured to transmit the image data G to be transmitted in parallel. Thus, the image data G in each of the buffers 2 is transmitted through the signal line 4 to be transmitted to the processor 3.

The transfer controller 5 can communicate with the processor 3 for image processing described above, manages the accumulation of the image data G in the buffers 2, and outputs the image data G required by the processor 3 to the signal line 4. In other words, the transfer controller 5 appropriately stores the image data G in each of the buffers 2, while managing the communication with the cameras 8 and memory addresses to appropriately store the communicated image data G in each of the buffers 2. As described above, the processor 3 determines the buffers 2 set to the selection target based on the state information S of the vehicle 9, and each time the processor 3 sequentially selects the buffer 2 therein, the image data G of the selected buffer becomes 2 acquired. Here, the transfer controller 5 outputs the image data G of the required buffer 2 to the signal line 4 when the processor 3 specifies the buffer 2 to be acquired to the transfer controller 5 to require the image data G, and thus the transfer controller 5 transmits the image data G the processor 3. The processor 3 may perform the above-described requirement (control signal) for the transfer control 5 through the signal line 4, or it may perform it through a control signal line provided separately.

As described above, the transfer of the image data G to the processor 3 is performed by the transfer controller 5, and thus the processor 3 can assign an operation time to the image processing without performing the transfer processing on the image data G. Thus, the processor 3 can allocate a larger proportion of the processing capability to the image processing, and thus, even if the processor 3 has a relatively low processing capability, the processor 3 can appropriately perform real-time driving assistance by the individual cameras 8.

In short, in the image processing device 1 configured as described above, since the transfer controller 5 performs the transfer processing on the image data G, the image processing processor 3 does not perform any processing on the image data G until the image data G is transferred from the buffer 2 with the transfer controller 5. in which the image data G recorded with each of the cameras 8 is stored, has been transferred, and it performs image processing only on the image data G acquired through the transfer. In addition, the processor 3 determines, for example, from the image data G of the individual cameras 8 accumulated in the buffers 2, based on the status information S of the vehicle 9, the buffers 2, which must be set to the target for the image processing in order to be set to the target for the image processing set cameras 8 to at least a part of the cameras 8, and thus the number of parts of image data that have to be processed per unit time is reduced.

As described above, the processor 3 sets the image data G of the cameras 8 determined based on the state information S of the vehicle 9 to the target for image processing and does not perform any processing until the image data G set to the target has been acquired are, with the result that environmental detection (the detection of a person and an article) around the vehicle 9 can be carried out more quickly by the cameras 8. Thus, providing the processor 3 for each of the cameras 8 and using an expensive processor 3 with better processing capability can be avoided, and thus the cost of the vehicle 9 can be reduced.

In some embodiments, the image processing device 1, as in 1 shown, further comprising: a detection unit 11 (board) in which a plurality of the above-described buffers 2 and the transfer controller 5 are provided; an image processing buffer 6 (work buffer) which stores the image data G outputted to the signal line 4 by the transfer controller 5 and on which the processor 3 performs image processing; a processor unit 12 (board or card), in which the processor 3 and the image processing buffer 6 are provided. In this case, the signal line 4 described above can connect the buffers 2 in the detection unit 11 and the image processing buffer 6 in the processor unit 12 to each other. The detection unit 11 described above may be formed with a programmable logic device such as an FPGA.

In other words, the image data G, which is transferred with the transfer controller 5 according to the requirement of the processor 3, is stored in the image processing buffer 6 provided in the processor unit 12. Then, the processor 3 performs image processing on the image data G stored in the image processing buffer 6. The image processing buffer 6 of the in 1 The embodiment shown is a queue (FIFO), and the processor 3 performs image processing on one or more pieces of image data G stored in the image processing buffer 6 in an order in which the pieces of image data G are stored in the image processing buffer 6.

According to the in 1 In the embodiment shown, the image processing device 1 acquires the above-described state information S of the vehicle 9 from a vehicle state monitoring section 14 such as an ECU installed in the vehicle 9. Specifically, the vehicle condition monitoring section 14 collects the condition information S of the vehicle 9 including the vehicle speed and the shift position Sp through a vehicle network 15 (such as a CAN) installed in the vehicle 9, and stores the condition information S into a buffer (condition information buffer 62) for storage of the status information S provided in the processor unit 12. Collecting the status information S with the vehicle status monitoring section 14 and storing the status information S in the status information buffer 62 are carried out cyclically, and thus the last status information S is stored in the status information buffer 62. Thus, the processor 3 accesses the status information buffer 62 described above in order to be able to acquire the last status information S of the vehicle 9, and determines the buffers 2 which are set to the selection target based on the acquired status information S.

Although in the in 1 In the embodiment shown, the vehicle condition monitoring section 14 is provided in the other unit on the vehicle side (control unit 13), the control unit 13 and the processor unit 12 may be connected to the vehicle network 15, or they may be connected to separate wires.

According to the configuration described above, the image data G of the individual cameras 8 managed with the buffers 2 is transmitted through the signal line 4 for connecting the detection unit 11 and the processor unit 12, is transmitted to the image processing buffer 6 provided in the processor unit 12, and is subjected to image processing in the Processor 3 suspended. The management and transfer processing of the buffers 2 in the detection unit 11 and the image processing in the processor unit 12 are separated in hardware in this way, and thus, as described above, the environment detection (detection of a movement of a person and an article) can be carried out the vehicle 9 around through the cameras 8 can be carried out in a satisfactory manner in real time.

Some embodiments regarding a method for determining the buffers 2 of the selection target described above will then be given with reference to 4 until 10 described. 4 is a flowchart showing the method of determining the buffers 2 of the selection target according to the embodiment of the present invention, and the cameras 8 are selected according to the vehicle speed V. 5 is a diagram showing an order in which the buffers 2 (cameras 8) are selected when the vehicle speed V is smaller than a first threshold L1 (V < L1) according to the embodiment of the present invention. 6 is a diagram showing an order in which the Buffer 2 (cameras 8) are selected when the vehicle speed V according to the embodiment of the present invention is equal to or greater than the first threshold L1 and less than a second threshold L2 (L1 ≤ V < L2). 7 is a diagram showing an order in which the buffers 2 (cameras 8) are selected when the vehicle speed V is equal to or greater than the second threshold L2 (L2 ≤ V) according to the embodiment of the present invention. 8th is a diagram for illustrating the gradual switching of the buffers around the first threshold L1 when the vehicle speed V is increased according to the embodiment of the present invention. 9 is a diagram for illustrating the gradual switching of the buffers around the second threshold L2 when the vehicle speed V is increased according to the embodiment of the present invention. 10 is a diagram showing directions and areas monitored when a forklift vehicle turns according to the embodiment of the present invention. 11 is a flowchart showing a method for determining the buffers 2 when the forklift vehicle turns according to the embodiment of the present invention.

In the following description, it is assumed that the processor 3 is operated according to the instruction of the image processing program described above to achieve realization. It is also assumed that a relationship L1 < L2 < L3 holds according to the first threshold L1, the second threshold L2 and a third threshold L3.

According to some embodiments, the state information S of the vehicle 9 includes the vehicle speed V. As in 4 shown, the processor 3 can perform the selection of the buffers 2 described above based on the comparison of the vehicle speed V and predetermined threshold values L. According to the in 1 until 4 In the embodiment shown, the processor 3 determines one or more of the buffers 2 set to the above-described selection target according to the command of the image processing program for the selection target buffer determining section 71 based on the comparison of the vehicle speed V and a plurality of threshold values L (L1 to L3 ). Specifically, the processor 3 (selection target buffer determining section 71) checks in step S1 of 4 whether a driver intends to move the vehicle 9 or not by checking whether the shift position Sp is not in park. According to the commands of the selection target buffer determining section 71 and the image data acquiring section 72, when the shift position Sp is not in parking (P), in step S2 and the subsequent steps, the selection of the cameras 8 (buffer 2) is made corresponding to the vehicle speed V carried out.

In general, there is an upper limit to the number of pieces of image data on which a processor can perform image processing per unit time. Thus, when the pieces of image data G of the cameras 8 are processed sequentially as the number of cameras 8 (buffer 2) is increased, the number of pieces of image data on which image processing is performed per unit time in each of the cameras 8 becomes reduced. If this number is reduced, a longer delay occurs in image processing with respect to the pieces of image data G sequentially supplied from each of the cameras 8, and thus a time delay in environment recognition by the image processing of the cameras is increased. However, if the vehicle speed V is increased, then the time delay in environment detection by the image processing is more likely to become severe, whereas if the vehicle speed V is reduced, the time delay is more likely to be within an acceptable range. In other words, when the vehicle speed V is reduced even if a larger number of cameras 8 are set to the image processing target, driving assistance for the vehicle 9 can be performed appropriately. Thus, according to the present embodiment, the buffers 2 are switched according to the vehicle speed V, and thus the image processing is performed on images taken with cameras 8 having a higher priority according to driving assistance among the cameras 8 installed in the vehicle 9 are installed.

Specifically, for example, when the vehicle 9 is stopped before moving or is in a state (state in which driving is started) in which the vehicle 9 is moving at a very low speed due to a creeping phenomenon or the like The vehicle 9 can be moved to different positions depending on a steering angle. Thus, it is taken into account that with the near-field cameras 81, which have a relatively wide viewing angle in order to be able to appropriately record a near-field or the like, it is necessary to monitor the entire surroundings of the present vehicle.

Thus, the cameras include 8 according to some in 2 shown embodiments one or more of the cameras 8 (in 2 the four near-field cameras 81) for recording the surroundings of the vehicle 9. The threshold values L described above, which are compared with the vehicle speed V, include the first threshold value L1. In this case, the processor 3, as in 4 shown, when the vehicle speed V is less than the first threshold value L1 (V < L1), the buffers 2 select in which the image data G is accumulated, which is recorded with the one or more of the cameras 8 for recording the entire surroundings of the vehicle have been.

If according to the in 4 shown embodiment in step S2 V < L1 applies, then in step S3, as in 5 shown, external environment detection is carried out by image processing on the image data G of the near-field cameras 81. In other words, the processor 3 sets as the selection target the four buffers 2 (2a to 2d) in which the image data G inputted from the four near-field cameras 81 one by one are accumulated. Here, for example, although the order in which the buffers 2 of the selection target are selected is arbitrary, the selection as in 5 shown, carried out. Since according to the in 5 In the exemplary embodiment shown, the processor 3 performs image processing on the image data G obtained by recording with the near-field cameras 81 in the order of a first near-field camera 81a (the front of the vehicle 9), a second near-field camera 81b (the right side of the vehicle 9), a third near-field camera 81c (the rear of the vehicle 9), and a fourth near-field camera 81d (the left side of the vehicle 9) (see steps S31 to S34 in 5 ), the pieces of image data G are acquired sequentially while selecting the buffers 2 in the order of a first buffer 2a, a second buffer 2b, a third buffer 2c, a fourth buffer 2d, the first buffer 2a, ....

As in the in 1 In the embodiment shown, in order to obtain the entire environment, the number of cameras tuned to the target for image processing is more likely to be increased. Thus, if the images of the cameras 8 are sequentially switched one by one and then the image processing is performed when the number of cameras 8 is increased, the processing cycle of the image data G in the individual cameras 8 (individual buffers 2) is lengthened. However, since the vehicle 9 is moving at a very low speed, the image processing can be carried out on the images recorded with the individual cameras 8 as long as the image processing is permitted (not problematic) as driving support for the vehicle 9.

It is considered that even when the vehicle speed V is equal to or greater than the first threshold L1, for example, in a state (low-speed traveling state) in which the vehicle 9 can be quickly moved in a lateral direction with respect to the traveling direction depending on the steering angle , it is necessary to use the near-field cameras 81 and the like to monitor a wide area in the traveling direction of the present vehicle.

Thus, according to some other embodiments, such as in 2 shown, the cameras 8 at least the cameras 8 (in 2 the two near-field cameras 81 are installed on the front and rear of the vehicle 9) for individually recording an area in front of the vehicle 9 and an area behind the vehicle 9. The status information S of the vehicle 9 further includes the switching position Sp of the transmission of the vehicle 9 The threshold values L described above, which are compared with the vehicle speed V, include the first threshold value L1 and the second threshold value L2, which is greater than the first threshold value L1. If in this case, as in 4 and 6 shown, the vehicle speed V is equal to or greater than the first threshold L1 and less than the second threshold L2 (L1 ≤ V < L2), and the switching position Sp indicates forward movement, then the processor 3 can select the buffer 2 in which the Image data G is accumulated with the camera 8 (in 2 the first near-field camera 81a) has been recorded for recording the area in front of the vehicle 9, whereas when the switch position Sp indicates a backward movement, the processor 3 can select the buffer 2 in which the image data G obtained with the camera 8 (in 2 the third near-field camera 81c) has been recorded for recording the area behind the vehicle 9.

If according to the in 4 In the exemplary embodiment shown in the above-described step S2, a relationship V < L1 does not apply and in the following step S4 L1 ≤ V < L2 applies, then in step S5 the external environment detection, which is different from that in the above-described step S3, is carried out by the Image processing is carried out on the image data G of the near-field cameras 81. Like in particular 6 shown, the processor 3 (selection target buffer determining section 71) checks the switching position Sp in step S61. Then, if the switching position Sp indicates the forward movement, in step S62 the first buffer 2a in which the image data G of the first near-field camera 81a for shooting a Near field area in front of the vehicle 9 are accumulated, set to the selection target. If in On the contrary, the switch position Sp indicates the backward movement, then in step S63, the third buffer 2c in which the image data G of the third near-field camera 81c for shooting a near-field area behind the vehicle 9 is accumulated is set to the selection target.

In contrast, in a driving state in which the vehicle speed is relatively so high that the vehicle cannot quickly turn in one direction by steering, the time allowed for feedback (e.g. automatic braking in the vehicle 9 or a warning to the driver). ) from the environmental recognition (the detection of a person and an article) through the image processing to the driver of the vehicle 9 or the like is further shortened. Thus, it is considered that a remote area is being monitored by the vehicle 9, and thus it is necessary to detect a target (a person and an article) in a remote area earlier.

Thus, according to some embodiments, such as in 2 shown, the cameras 8: the far-field cameras 82 for recording a far-field area in the direction of travel of the vehicle 9; and the near-field cameras 81, which record the near-field area in the direction of travel of the vehicle 9, and which have a wider viewing angle than the far-field cameras 82. The threshold values L described above, which are compared with the vehicle speed V, include the second threshold value L2. As in 4 and 7 shown, processor 3 can use buffers 2 (2e and 2f in 1 ), in which the image data G accumulated with the far-field cameras 82 are accumulated, select when the vehicle speed V is equal to or greater than the second threshold L2, whereas when the vehicle speed V is less than the second threshold L2, the processor 3 the buffers 2 (2a to 2d in 1 ) can select in which the image data G that was recorded with the near-field cameras 81 is accumulated.

If according to the in 4 In the exemplary embodiment shown in step S4 described above, a relationship of V < L2 does not apply and in the following step S6 L2 ≤ V < L3 (or L2 ≤ V) applies, then in step S7 the external environment recognition is carried out by image processing on the image data G Far-field cameras 82 carried out. Like in particular 7 As shown, the processor 3 (selection target buffer determining section 71) checks the switching position Sp in step S71. Then, when the switching position Sp indicates the forward movement, the processor 3 (the same as described above) sets the fifth buffer 2e as the selection target in step S72 , in which the image data G of the first far-field camera 82a for recording the far-field area in front of the vehicle 9 are accumulated. In contrast, if the switch position Sp indicates the backward movement, then the processor 3 sets the sixth buffer 2f as the selection target in which the image data G of the second far-field camera 82b for shooting a far-field area behind the vehicle 9 is accumulated in step S73.

According to the configuration described above, the processor 3 performs selection based on the comparison of the vehicle speed V and the threshold values L, for example, determining the buffers 2 which are set to the target for image processing. As described above, the cameras 8 (buffers 2) set to the target for image processing are selected by the relationship with the vehicle speed V, and thus even the processor 3, whose processing capability is relatively low, can provide driving assistance for that Vehicle through environment detection using the cameras in a satisfactory manner in real time.

According to some embodiments, as in 8th and 9 shown, the above-described threshold values L, which are compared with the vehicle speed V, include a switching start threshold Ls and a switching completion threshold Le, in which a difference with the switching start threshold Ls represents a first value W. In this case, if the vehicle speed V is between the switching start threshold Ls and the switching completion threshold Le, then the processor 3 can switch the buffers 2 to be selected such that when the vehicle speed V is at the switching completion threshold Le from the switching start threshold Ls approximates, a predetermined number of pieces of image data G on which the image processing is performed per unit time before the vehicle speed V reaches the switching start threshold Ls is replaced with the predetermined number of pieces of image data G after the vehicle speed V reaches the switching completion -Threshold Le has been reached. In other words, a ratio of the camera 8 (in 8th the first near-field camera 81a), whose priority (importance) is high, is gradually increased according to the vehicle speed V.

When the cameras 8 (buffers 2) set to the target for the image processing of the processor 3 are accurately switched at a predetermined threshold L (the first threshold L1 or the second threshold L2), for example, that obtained from the vehicle 9 Vehicle speed V an error then appropriate buffers 2 corresponding to the vehicle speed V are not selected around the threshold L, with the result that the appropriate environment detection is unlikely to be performed. Thus, according to the present embodiment, the buffers 2 set to the selection target are gradually switched.

Here, since the processor 3 performs image processing on the predetermined number of pieces of image data G per unit time, the predetermined number of pieces of image data G is formed with image data G acquired from one or more buffers 2 of the selection target. The switching start threshold Ls is the threshold L which provides an opportunity to switch a part of the buffers 2 that acquire the predetermined parts of image data G to one of the one or more buffers 2 selected by the processor 3, when the vehicle speed V reaches the switching completion threshold Le. The switching completion threshold Le is, for example, the first threshold L1 or the second threshold L2 described above, and the switching start threshold Ls is a value obtained by increasing or decreasing the switching completion threshold Le only by the first value W . In particular, for example, if the switching completion threshold Le is the first threshold L1 described above, then the switching start threshold Ls is Le+W (where Le+W < L2) or Le-W. For example, if the switching completion threshold Le is the second threshold L2 described above, then the switching start threshold Ls is Le+W or Le-W (where Le-W > L1). Between the switching start threshold Ls and the switching completion threshold Le can also be made, as in 8th and 9 shown, one or more threshold values (stepwise threshold value Lm) may be provided, so that the switching described above is carried out more gradually.

According to the in 8th In the embodiment shown, the switching completion threshold Le is the first threshold L1 described above, and the switching start threshold Ls is a value obtained by reducing the first threshold L1 only by the first value W (Ls=Le-W). If in this case, according to in 1 until 3 In the exemplary embodiment shown, the vehicle speed V is less than the first threshold value L1 (Le), then the processor 3 selects the four buffers 2 (2a to 2d) corresponding to the four near-field cameras 81 (see. 5 ) sequentially, whereas if the vehicle speed V is equal to or greater than the first threshold L1, the processor 3 only selects the first buffer 2a corresponding to the first near-field camera 81a, which records the area in front of the vehicle 9 (see Fig. 6 ).

Thus, in a case where the shift position Sp indicates the forward movement and the vehicle speed V is sequentially increased from zero, the vehicle speed V is smaller than the shift start threshold Ls (V < Ls), then the processor selects 3 (image data acquisition section 72) selects the first buffer 2a, the second buffer 2b, the third buffer 2c and the fourth buffer 2d sequentially, then returns to the first buffer 2a again and repeatedly performs the selection in the same order. Thereafter, when the vehicle speed V reaches the switching start threshold Ls, then the processor 3 (the same as described above) replaces at least one of the second buffer 2b, the third buffer 2c and the fourth buffer 2d among the buffers 2 (2a to 2d), which have so far been included in the selection target, with the first buffer 2a.

If here the vehicle speed V, as in 8th shown, the switching start threshold Ls is reached, then the processor 3 can control the camera 8 (in 8th the first near-field camera 81a for recording the area ahead) corresponding to the buffer 2 (in 8th only the first buffer 2a), which must be selected when the vehicle speed V reaches the switching completion threshold Le, and the buffer 2 (in 8th the third buffer 2c) corresponding to the camera 8 (in 8th the third near-field camera 81c for recording the area behind), which is least linked to the recording direction, switch.

According to the in 8th In the exemplary embodiment shown, the processor 3 switches the third buffer 2c corresponding to the third near-field camera 81c for recording the area behind the vehicle 9 to the first buffer 2a. In other words, when the vehicle speed V reaches the switching completion threshold Le (first threshold L1), a state in which the image data G in the four directions of the front, rear, left side and right side of the vehicle 9 has hitherto occurred the target for image processing has been set has been changed to a state in which only the area ahead is monitored, and thus the backward direction is opposite to the forward direction, then the image data G in the backward direction is the least important among the left side, the right side and the rear of the vehicle 9. If the vehicle speed V reaches the start of the switchover Threshold Ls is reached, then the processor 3 switches the third buffer 2c.

According to the in 8th In the embodiment shown, the stepwise threshold Lm is provided between the shift start threshold Ls and the shift completion threshold Le, and when the vehicle speed V is increased from the shift start threshold Ls to reach the stepwise threshold Lm (V = Lm), then the second buffer 2b corresponding to the second near-field camera 81b for recording an area on the right side of the vehicle 9 is switched to the first buffer 2a. Then, when the vehicle speed V is further increased to reach the switching completion threshold Le (V = Le), the processor 3 switches the fourth buffer 2d corresponding to the fourth near-field camera 81d for shooting an area on the left side of the vehicle 9 the first buffer 2a. In this way, at the first threshold L1, the processor 3 sets the first buffer 2a corresponding to the first near-field camera 81a to the selection target to perform image processing. Thus the ratio of the camera becomes 8 (in 8th the first near-field camera 81a), whose importance is high, is gradually increased according to the vehicle speed V.

On the other hand, the switching completion threshold Le is according to FIG 9 In the embodiment shown, the above-described second threshold L2, and the switching start threshold Ls is a value obtained by reducing the second threshold L2 only by the first value W (Ls=Le-W). In this case, the processor selects 3 according to in 1 until 3 shown embodiment, as described above, when the vehicle speed V is equal to or greater than the first threshold value L1 and less than the second threshold value L2, only the first buffer 2a corresponding to the first near-field camera 81a for recording the area in front of the vehicle 9 (see . 6 ), whereas when the vehicle speed V is equal to or greater than the second threshold L2, the processor 3 only selects the fifth buffer 2e corresponding to the first far-field camera 82a for recording the area in front of the vehicle 9 (see Fig. 7 ).

Thus, the processor 3 selects only the first buffer 2a in a case where the shift position Sp indicates the forward movement and the vehicle speed V is sequentially increased from the first threshold L1 when the vehicle speed V is smaller than the shift start threshold Ls ( V < Ls). Thereafter, the processor 3 includes (adds) the fifth buffer 2e into the current selection target and reduces the selection frequency of the first buffer 2e constituting the current selection target when the vehicle speed V reaches the switching start threshold Ls. According to the in 9 In the embodiment shown, among three continuous pieces of image data G on which image processing is performed, the first buffer 2a of the last piece of image data G is replaced with the fifth buffer 2e, and thus the selection frequency of the first buffer 2a is reduced. Furthermore, when the vehicle speed V is increased so that the gradual threshold Lm is reached, the last two parts among the three continuous parts of image data G are replaced with the fifth buffer 2e. Furthermore, when the vehicle speed V is increased to reach the switching completion threshold Le, then only the fifth buffer 2e is selected. In this way, a ratio of the camera is 8 (in 9 the first far-field camera 82a), whose importance is high, is gradually increased according to the vehicle speed V.

According to the above discussion, although the case in which the vehicle speed V is increased has been described, the same applies to a case in which, when the vehicle speed V is reduced, the vehicle speed V is determined by the switching start threshold Ls, the step threshold Lm and the switching completion threshold Le runs in this order.

According to the configuration described above, the cameras 8 (buffers 2) set to the image processing target of the processor 3 are gradually switched from the previous stage to be fully switched with the predetermined threshold values L (such as the first threshold value L1 and the second threshold L2 described above). In this way, even if the vehicle speed V obtained from the vehicle 9 has an error, the external environment detection regarding the vehicle 9 can be performed appropriately.

According to some embodiments, the image processing device 1 may further include a feature information acquisition section 74 (acquiring section) that acquires feature information including an instruction for the monitoring direction at a specific location, and the processor 3 selects the buffer 2 based on the feature information. In the vehicle 9, which travels in different places, there is a place (specific place) in which the specific direction is preferably in a focused manner as the driving assistance is monitored, and there is, for example, a place where accidents in which a person jumps out from the right direction often occur. Thus, the image processing device 1 acquires the above-described feature information to include in the selection target the buffer 2 corresponding to the camera 8 for shooting an area in the monitoring direction indicated by the feature information. Here, the selection frequency of the buffer 2 corresponding to the camera 8 for shooting the area in the monitoring direction indicated by the property information can also be increased over that of the other buffers 2 included in the selection target.

In particular, the image processing device 1 (image processing program) can also, as in 3 shown, in addition to the above-described property information acquisition section 74, include a position acquisition section 75 which acquires the traveling position of the vehicle 9. According to the in 3 In the embodiment shown, the above-described feature information corresponds to the specific location, and when the traveling position of the vehicle 9 reaches the specific location corresponding to the feature information, then the buffer 2 corresponding to the camera 8 for recording the area in the monitoring direction indicated by the feature information is in included in the selection target. The property information acquisition section 74 and the position acquisition section 75 may be realized by the operation of the processor 3 according to the instruction of the image processing program, or they may be realized using hardware. According to the in 3 In the embodiment shown, the instruction for the surveillance direction is input to the selection target buffer designating section 71, and thus the buffer 2 corresponding to the camera 8 for shooting the area in the surveillance direction is included in the selection target.

The driving position of the vehicle 9 can be acquired by a self-position detection technology, such as GPS (“Global Positioning System”) or SLAM (“Simultaneous Localization and Mapping”), or the driving position (position information) can be acquired from outside by communicating with one installed on a road Beacon station, communication with a base station in a mobile communication network, or communication with the outside using a wireless communication technology such as near field communication using RFID (“Radio Frequency IDentification”). The specific location and property information are stored in the database in advance, and the information thereof can be acquired by performing communication with the outside as needed, or the image processing device 1 can store this database. Alternatively, in the image processing device 1, a device such as a beacon station may be installed near the specific location, and the buffer 2 corresponding to the camera 8, which when a communication is received from the device such as a beacon station, based on the information of the monitoring direction included in the property information obtained through this communication, an area in the direction thereof may be included in the selection target.

According to the configuration described above, the processor 3 sets the image data G of the camera 8 for photographing the area in the monitoring direction indicated by the property information to the target for image processing. Although it may be necessary to monitor the specific direction in a focused manner in the vehicle 9 traveling in various places, such as a place where accidents frequently occur, the cameras 8 pointing to the target for the image processing is set, switched based on the characteristic information, and thus the driving assistance can be performed, which is safer.

For example, according to some embodiments, if the vehicle 9 is a commercial vehicle, then the state information S of the vehicle 9 includes the vehicle speed V and a steering angle A, and the processor 3 may determine based on the state information S whether the turn of the vehicle 9 is a right turn or turning left. Then, when the vehicle 9 turns right, the processor 3 can select the buffer 2 in which the image data G is accumulated by capturing at least a right-side area in the forward direction and a left-side area in the forward direction backward direction with respect to the traveling direction, whereas the vehicle 9 turns left, the processor can select the buffer 2 in which the image data G is accumulated by capturing at least an area on the left in the forward direction and an area are preserved on the right side in the reverse direction with respect to the direction of travel.

For example, as in 10 shown, a commercial vehicle, such as a forklift, has a short wheelbase and a large steering angle, it is required to be directed to the lateral direction to pay sufficient attention to the driving direction compared to the forward direction when starting driving in a state in which a steering wheel is operated. In particular, it is necessary to have not only the right side in the forward direction with respect to the traveling direction, but also the left side in the reverse direction with respect thereto at the time of a right turn, and not only the left side in the forward direction with respect to the driving direction direction of travel, but also the right side in the reverse direction with reference to this at the time of a left turn.

However, as described above, if the image data G obtained by shooting an area on the side of the direction opposite to the traveling direction is also set to the target for image processing, the number of cameras 8 with respect to which the processor 3 which performs image processing is simply increased, and thus the selection frequencies of the buffers 2 included in the selection target are reduced (the selection cycle is lengthened).

Thus, according to some embodiments, such as in 10 and 11 shown, the cameras 8 are a right-side recording camera (in 2 the second near-field camera 81b) for recording the area on the right side of the vehicle 9 and a left-side recording camera (in 2 the fourth near-field camera 81d) for recording the area on the left side of the same. The processor 3 is operated according to the command of the image processing program for the image processing section 73, and thus the image processing can be carried out at the time of a right turn in the area (subarea Gp, in 10 the left half of the image data G) of a part of the left side of the image data G which are individually recorded with the right-side recording camera and the left-side recording camera described above, whereas at the time of turning to the left, the image processing is carried out on the sub-area Gp (in 10 the right half of the image data G) can be performed on the right side of the image data G individually captured with the right-side capture camera and the left-side capture camera described above. In this way, driving assistance, which is safer, can be performed in a satisfactory manner.

11 corresponds to a transition from step S2 to step S3 in 4 , and if in step S2 of 4 the vehicle speed V is smaller than the first threshold value L1 described above (V < L1), then in step S21 of 11 the steering angle A is checked, and if the steering angle A is equal to or smaller than a steering angle threshold At (A ≤ At), then the process goes to step S3 of 4 , as already described. In contrast, if in step S21 of 11 the steering angle A is greater than the steering angle threshold At (A > At), then the steering angle A is checked in step S22. Then, when the steering angle A indicates a right turn, in step S23, image processing is performed on the left half area of the image data G individually captured with the right-side pickup camera and the left-side pickup camera described above. In contrast, if the steering angle A indicates a left turn, then in step S24, image processing is performed on the right half area of the image data G individually captured by the right-side pickup camera and the left-side pickup camera described above.

According to the configuration described above, the processor 3 determines the turning direction based on the vehicle speed V and the steering angle A, and selects the cameras 8 set to the target for image processing based on the determination result. As described above, according to the turning direction (steering angle) of the vehicle, the image data G from not only one side such as the right side but also the opposite side (left side) is set to the target for image processing, and thus at the time a lateral turn while paying attention to the direction opposite to the driving direction is also performed, for example, the detection of a person and an article is performed with the result that the driving assistance can be performed, which is safer.

The following is with reference to 12 and 13 another embodiment is described. 12 is a diagram showing a travel path in the image data G set to the target for image processing according to the embodiment of the present invention. 13 is a diagram for illustrating a case in which the vehicle image processing device 1 according to the embodiment of the present invention determines the backlit state of the vehicle 9.

According to some exemplary embodiments, the status information S of the vehicle 9 described above includes the steering angle A and the switching position Sp. As in 12 As shown, the processor 3 is operated according to the command of the image processing program for the image processing section 73 to perform image processing on the area (subarea Gp) of a part included in the image data G obtained from the buffers 2 are acquired based on the steering angle A and the switching position Sp. Although in an in 12 In the camera image shown, a horizon h is shown in a distant area and a road r is curved in the rightward direction when the vehicle 9 travels along the road r, it is considered that the need for automatic detection processing in the areas other than the travel path to carry out is not high. As described above, the image processing is performed on only the portion Gp of the image data G, and thus the amount of information of each portion of image data G on which the image processing is performed can be reduced, with the result that the performance time of the image processing on one portion of image data G is reduced.

For example, according to some embodiments, as in 12 shown, the path on which the vehicle 9 travels can be estimated based on the steering angle A and the shift position Sp, and the sub-area Gp of the image data G can be determined based on the estimation result. According to the in 12 In the embodiment shown, the processor 3 performs the image processing on the estimated travel path, and, for example, performs the automatic detection processing only on the estimated travel path. For example, if the threshold L compared with the vehicle speed V described above includes the third threshold L3, and the vehicle speed V as in 4 shown is equal to or greater than the third threshold value L3, then the processor 3 can, based on the result of a prediction of the travel path based on the steering angle A and the switching position Sp, carry out the image processing on the sub-area Gp that is in the acquired by the buffers 2 Image data G is included. Although the prediction of the traveling path can be performed by a known method, the prediction of the traveling path can be performed taking into account the size of the present vehicle, the position at which the camera 8 is mounted, the angle (optical axis angle) at which the camera 8 is mounted is, and the internal parameters of the camera 8 are carried out. In this way, for example, when the vehicle speed V exceeds the second threshold 12 and is further increased, the number of cameras 8 set to the target for image processing is restricted, and further, the area of each piece of acquired image data G , on which the image processing is performed, is reduced, with the result that the number of pieces of image data G on which the processor 3 can perform the image processing per unit time is increased, and thus the environment recognition can be performed in real time.

According to some other embodiments, for example, the areas of an upper portion and a part of the left and right sides of the image data G, which are determined according to the vehicle speed V and the steering angle A, are removed from the target of the image processing, and thus based on the vehicle speed V , the steering angle A and the switching position Sp, the sub-area Gp of the image data G on which the image processing is to be carried out is determined. For example, it is in 12 It can be seen from the steering angle A that the vehicle 9 is traveling to turn right, and thus, taking into account the vehicle speed V, areas of an upper portion and a left side of the captured image can be removed from the target for image processing.

According to the configuration described above, based on the steering angle A and the shift position Sp, a part of each piece of image data G is extracted to be adjusted to the target for image processing. In this way, in each part of image data G, the image processing on the part area which does not include the travel path of the vehicle predicted based on the steering angle A and the shift position Sp can be omitted. Thus, the load of image processing on each piece of image data G can be reduced, and thus the number of pieces of image data G on which image processing can be performed per unit time can be increased. Consequently, a performance time required at a high vehicle speed V or the like can be coped with.

According to some embodiments, as in 13 shown, the cameras 8 include a first camera 8a for recording an area in one direction of the vehicle 9 and a second camera 8b for recording an area in a direction different from the one direction. As in 3 As shown, the image processing device 1 may further include an adjustment section 76 which sets shooting parameters for each of the cameras 8 based on the brightness of the image data G individually captured by the first camera 8a and the second camera 8b described above. The processor 3 described above may be operated according to the command of the image processing program to realize the setting section 76, or the setting section 76 may be realized using another type of hardware. According to the in 13 The exemplary embodiment shown is the first camera 8a first far-field camera 82a, and the second camera 8b is the second far-field camera 82b, which captures an area in a direction opposite to the one direction captured by the first camera 8a. However, the present invention is not limited to the present embodiment. According to some other embodiments, the first camera 8a and the second camera 8b need not be of the same type as long as the shooting directions between the first camera 8a and the second camera 8b are different, and for example, the first camera 8a is the far-field camera 82, and the second Camera 8b is the near field camera.

As described above, since the entire surroundings of the vehicle 9 are photographed with the cameras 8 in an outdoor area depending on a positional relationship with the sunlight, some cameras 8 are illuminated from behind and some cameras 8 are illuminated from the front, and when the If recording parameters are the same for all cameras, then you probably have a camera that cannot obtain images of suitable image quality (e.g. brightness). Thus, according to the present embodiment, for example, every time cameras 8 other than the cameras 8 set to the target for image processing are temporarily included in the selection target, the brightness of the image data G is evaluated such that an outdoor environment is light and the like is estimated, and thus the recording parameters for the cameras 8 are automatically adjusted as required, with the result that in any environment, the image data G of a suitable image quality can be obtained from the individual cameras 8.

Specifically, according to the embodiment described above, if the vehicle speed V is equal to or greater than the first threshold L1, then the environment detection is performed only with the camera 8 (the first near-field camera 81a or the first far-field camera 82a) for photographing the area in the traveling direction, and for example, this camera 8 is the first camera 8a described above. Here, the processor 3 acquires the image data G captured with the camera 8 (the third near-field camera 81c or the second far-field camera 82b) for capturing the area in the direction opposite to the direction of travel or the like cyclically (for example, once every 10 seconds). For example, this camera 8 is the second camera 8b described above. Then, the brightness of the image data G from each of the first camera 8a and the second camera 8b is evaluated.

Thereafter, for example, the average brightness (first average brightness I1) of a plurality of pieces of image data G formed only with images captured with the first camera 8a and the average brightness (second average brightness 12) of a plurality of pieces of image data G , which are formed with images recorded with the first camera 8a and the second camera 8b, are evaluated individually. For example, among pieces of image data G acquired continuously with time, the second average brightness 12 of a plurality of continuous pieces of image data G containing the image data G obtained by recording the area of the direction opposite to the traveling direction is, and the first average brightness I1 of a plurality of continuous pieces of image data G which do not contain the image data G in the opposite direction can be evaluated. Then, if I1<12, it is determined that the camera 8 for shooting the area in the traveling direction is in the backlit state at that time, and thus the setting of the shooting parameters for the first camera 8a is changed such that, for example, Image data G generated by the cameras 8 in the backlit state can be made bright, and a suitable recording can therefore be carried out. In contrast, when I1 > 12, the setting is changed so that the brightness of the first camera 8a is reduced.

According to the in 13 In the exemplary embodiment shown, the brightness of the image data G of the first camera 8a is relatively low due to the rear light, and the brightness of the image data G of the second camera 8b is relatively high due to the front light. Thus, a relationship of I1 < 12 holds, and thus the setting section 76 determines that the first camera 8a for the traveling direction is illuminated from behind to change the setting of the shooting parameters, and thereby make the image data G of the first camera 8a bright .

According to the configuration described above, the front light and the back light of the cameras 8 set to the target for image processing are detected so that the shooting parameters for the cameras 8 are adjusted, and thus an influence caused by changes in the cameras 8 can be minimized brightness generated by sunlight, with the result that stable environmental detection can be carried out.

The present invention is not limited to the embodiments described above, and includes embodiments that by adding variations to the above-described embodiments, and embodiments obtained by combining these embodiments.

The property information acquiring section 74 described above represents an example of the acquiring section.

The programs described above constitute software for instructing a computer to implement individual functional portions described below, and may be stored in a computer-readable storage medium.

Claims (15)

Vehicle image processing device (1), comprising: a plurality of buffers (2) configured to accumulate portions of image data (G) individually and sequentially input from a plurality of cameras (8) installed in a vehicle (9) and each having different surrounding areas of the vehicle (9) in order to link the parts of image data (G) to the cameras (8); a processor (3) configured to select the buffer (2) based on state information (S) for determining driving conditions of the vehicle (9), and to extract the portion of image data (G) from the selected buffer (2). acquire in order to carry out image processing in this regard; a signal line (4) which transmits the pieces of image data (G) in the buffers (2) to the processor (3); and a transfer controller (5) which is configured to output the part of image data (G) in the buffer (2) required by the processor (3) to the signal line (4) by selecting based on the status information (S), wherein the status information (S) includes a vehicle speed (V), and the processor (3) carries out the selection of the buffer (2) based on a comparison of the vehicle speed (V) and threshold values (L), wherein the threshold values (L) include a switching start threshold (Ls) and a switching completion threshold (Le), in which a difference to the switching start threshold (Ls) is a first value, the processor (3) is configured to perform image processing on a predetermined number of pieces of the image data (G) per unit time, and when the vehicle speed (V) is between the switching start threshold (Ls) and the switching completion threshold (Le), the processor (3) switches the buffers (2) to be selected in such a way that when the vehicle speed (V) approaches the switching completion threshold Threshold value (Le) approaches the switching start threshold (Ls), the predetermined number of pieces of image data (G) on which the image processing is performed per unit time before the vehicle speed (V) reaches the switching start threshold (Ls). the predetermined number of pieces of image data (G) is replaced after the vehicle speed (V) reaches the switching completion threshold (Le). Vehicle image processing device Claim 1 , wherein the processor (3) determines at least one buffer (2) of the buffers (2) as a selection target based on the status information (S) and a selection frequency of the at least one buffer (2) contained in the selection target is greater than selection frequencies of the other buffers ( 2) sets to sequentially select the at least one buffer (2) contained in the selection target. Vehicle image processing device (1). Claim 1 or 2 , further comprising: a detection unit (11) which has the buffers (2) and the transfer controller (5) provided therein; a work buffer (6) configured to store the portion of image data (G) output by the transfer controller (5) to the signal line (4) and subjected to image processing performed by the processor (3); and a processor unit (12) which has the processor (3) and the working buffer (6) provided therein, the signal line (4) being configured to connect the buffers (2) in the detection unit (11) and the working buffer (6). to be able to connect to each other in the processor unit (12). Vehicle image processing device (1) according to one of Claims 1 until 3 , wherein the cameras (8) comprise one or more of the cameras recording an environment of the vehicle (9), the threshold values (L) comprise a first threshold value (L1), and if the vehicle speed (V) is less than the first threshold value (L1) is, the processor (3) selects the buffer (2) in which the part of image data (G) which is recorded with the one or more of the cameras (8) recording the entire surroundings of the vehicle (9) is accumulated. Vehicle image processing device (1) according to one of Claims 1 until 4 , where the cameras (8) are a camera covering an area in front of the vehicle (9), and a camera that records an area behind the vehicle (9), the status information (S) further includes a switching position (Sp) of a transmission of the vehicle (9), the threshold values (L ) the first threshold value (L1) and a second threshold value (L2), which is greater than the first threshold value (L1), and if the vehicle speed (V) is equal to or greater than the first threshold value (L1) and less than the second Threshold value (L2), and the switching position (Sp) indicates a forward movement, the processor (3) only selects the buffer (2) in which the part of image data (G) recorded with the camera is accumulated Area in front of the vehicle (9), whereas if the switching position (Sp) indicates a backward movement, the processor (3) only selects the buffer (2) in which the part of image data (G) recorded with the camera is accumulated is which takes up the area behind the vehicle (9). Vehicle image processing device (1) according to one of Claims 1 until 5 , wherein the cameras (8) are a far-field camera (82), which records a far-field area in a direction of travel of the vehicle (9), and a near-field camera (81), which records a near-field area in the direction of travel of the vehicle (9) and which has a wider viewing angle than the far-field camera (82), the threshold values (L) include the second threshold value (L2), and if the vehicle speed (V) is equal to or greater than the second threshold value (L2), the processor (3) uses the buffer ( 2) selects in which the part of image data (G) recorded with the far-field camera (82) is accumulated, whereas if the vehicle speed (V) is less than the second threshold value (L2), the processor (3) uses the buffer (2) selects in which the part of image data (G) recorded with the near-field camera (81) is accumulated. Vehicle image processing device (1) according to one of Claims 1 until 6 , wherein when the vehicle speed (V) reaches the switching start threshold (Ls), the processor (3) first switches the camera corresponding to the buffer (2) which must be selected when the vehicle speed (V) reaches the switching completion threshold (Le) is reached, and to the buffer (2) corresponding to the camera, which is linked to at least one recording direction. Vehicle image processing device (1) according to one of Claims 1 until 7 , further comprising: an acquiring section (74) comprising property information with an instruction for a monitoring direction at a specific location, wherein the processor (3) selects at least one of the buffers (2) based on a travel position and the property information. Vehicle image processing device (1) according to one of Claims 1 until 8th , wherein the vehicle (9) is a commercial vehicle, the status information (S) includes the vehicle speed (V) and a steering angle (A), the processor (3) determines based on the status information (S) whether the vehicle (9 ) is a right turn or a left turn, and when the vehicle (9) turns right, the processor (3) selects the buffer (2) in which the portion of image data (G) obtained by recording is accumulated at least an area on a right side in a forward direction and an area on a left side in a backward direction with respect to the traveling direction, whereas when the vehicle (9) turns left, the processor (3) selects the buffer (2). , in which the part of image data (G) obtained by recording at least a region on the left side in the forward direction and a region on the right side in the backward direction with respect to the traveling direction is accumulated. Vehicle image processing device (1). Claim 9 , wherein the cameras (8) include a right-side recording camera (81b) which records an area on the right side of the vehicle (9) and a left-side recording camera (81d) which records an area on the left side thereof, and if the vehicle (9) turns right, the processor (3) performs the image processing on an area of a part on a left side of the pieces of image data (G) obtained with the right-side recording camera (81b) and the left-side recording camera (81d). are individually captured, whereas when the vehicle (9) turns left, the processor (3) performs the image processing on an area of a part on a right side of the pieces of image data (G) recorded with the right-side recording camera (81b) and are recorded individually using the left-hand recording camera (81d). Vehicle image processing device (1) according to one of Claims 1 until 10 , wherein the status information (S) includes the steering angle (A) and the switching position (Sp) of the transmission of the vehicle (9), and the processor (3) estimates a path based on the steering angle (A) and the switching position (Sp). which the vehicle (9) drives, and based on a result of the estimation, performs the image processing on an area of a part contained in the parts of image data (G) acquired from the buffers (2). Vehicle image processing device (1) according to one of Claims 1 until 11 , wherein the cameras (8) comprise a first camera (8a), which records an area in a first direction of the vehicle (9), and a second camera (8b), which records an area in a direction different from the first direction , and the vehicle image processing device (1) further has a setting section (76) which, based on a brightness of the parts of image data (G) individually captured by the first camera (8a) and the second camera (8b), determines a capture parameter for adjusts each of the cameras (8). Vehicle image processing method performed by a processor (3) of a computer, the computer comprising: a plurality of buffers (2) configured to accumulate pieces of image data (G) individually and sequentially input from a plurality of cameras (8) installed in a vehicle (9) to the pieces to link image data (G) to the cameras (8); wherein the processor (3) is configured to select the buffer (2) based on state information (S) of the vehicle (9) including a vehicle speed (V), and to select the portion of image data (G) from the selected buffer (2) to acquire in order to carry out image processing in this regard; a signal line (4) which transmits the pieces of image data (G) in the buffers (2) to the processor (3); and a transfer control (5) which is configured to output the part of image data (G) required by the processor (3) in the buffer (2) to the signal line (4), wherein the vehicle image processing method comprises a step for performing the selection of the buffer (2) based on a comparison of the vehicle speed (V) and threshold values (L), wherein the status information (S) includes a vehicle speed (V), and the processor (3) carries out the selection of the buffer (2) based on a comparison of the vehicle speed (V) and threshold values (L), wherein the threshold values (L) include a switching start threshold (Ls) and a switching completion threshold (Le), in which a difference to the switching start threshold (Ls) is a first value, the processor (3) is configured to perform image processing on a predetermined number of pieces of the image data (G) per unit time, and when the vehicle speed (V) is between the switching start threshold (Ls) and the switching completion threshold (Le), the processor (3) switches the buffers (2) to be selected in such a way that when the vehicle speed (V) approaches the switching completion threshold Threshold value (Le) approaches the switching start threshold (Ls), the predetermined number of pieces of image data (G) on which the image processing is performed per unit time before the vehicle speed (V) reaches the switching start threshold (Ls). the predetermined number of pieces of image data (G) is replaced after the vehicle speed (V) reaches the switching completion threshold (Le). Vehicle image processing program instructing a processor (3) of a computer to perform a step of performing a selection of a buffer (2) based on a comparison of a vehicle speed (V) and threshold values (L), the computer comprising: a plurality of buffers (2 ), which are configured to accumulate pieces of image data (G) input individually and sequentially from a plurality of cameras (8) installed in a vehicle (9) to share the pieces of image data (G). to link the cameras (8); wherein the processor (3) is configured to select the buffer (2) based on state information (S) of the vehicle (9) including the vehicle speed (V), and to select the portion of image data (G) from the selected buffer (2) to acquire in order to carry out image processing in this regard; a signal line (4) which transmits the pieces of image data (G) in the buffers (2) to the processor (3); and a transfer controller (5) which is configured to output the part of image data (G) required by the processor (3) in the buffer (2) to the signal line (4), the status information (S) being a vehicle speed (V ), and the processor (3) performs the selection of the buffer (2) based on a comparison of the vehicle speed (V) and thresholds (L), the thresholds (L) comprising a switch start threshold (Ls) and a switch completion threshold (Le), in which a difference to the switching start threshold (Ls) is a first value, the processor (3) is configured to carry out the image processing processing on a predetermined number of pieces of the image data (G) per unit time, and when the vehicle speed (V) is between the switching start threshold (Ls) and the switching completion threshold (Le), the processor (3) selects the buffers (3) to be selected 2) switches such that when the vehicle speed (V) approaches the switching completion threshold (Le) from the switching start threshold (Ls), the predetermined number of pieces of image data (G) on which the image processing is performed per unit time , before the vehicle speed (V) reaches the switching start threshold (Ls), is replaced with the predetermined number of pieces of image data (G) after the vehicle speed (V) reaches the switching completion threshold (Le). Computer-readable storage medium that contains the vehicle image processing program Claim 14 saves.

Images