JP2022058835A - Image processing apparatus and image processing method, and image processing program and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of reducing a processing load on a transmission destination or the like as compared with a case where image information corresponding to an imaging result in each vehicle is always transmitted.

SOLUTION: An image processing apparatus comprises: a camera 1 whose image capturing range is in front of the own vehicle; a processing unit 2 for determining whether or not a relationship between a forward vehicle appearing within the image capturing range and the own vehicle has become a specific relationship that an image capturing result of the camera 1 can be employed as the image capturing result indicating a surrounding situation of the own vehicle; and an interface 3 for externally transmitting image data corresponding to the image capturing result of the camera 1 corresponding to a timing at which the relationship between the forward vehicle and the own vehicle has become the specific relationship.

SELECTED DRAWING: Figure 2

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present application belongs to the technical fields of image processing devices and image processing methods, as well as image processing programs and recording media. More specifically, the image processing device and the image processing method for processing the image information corresponding to the image pickup result captured with the front of the moving body such as a vehicle as the image pickup range, the program for the image processing device, and the program are recorded. It belongs to the technical field of recording media.

In recent years, image information corresponding to the image pickup result of a camera mounted on a general vehicle and whose imaging range is in front of the vehicle is collected from each vehicle by a server device via a network such as the Internet, and detected based on the image information. Research and development is being conducted on a traffic information providing system that provides the image information itself to other vehicles and the like together with information indicating road conditions. In such a traffic information providing system, it is necessary for the server device to evaluate whether or not each of the collected image information is useful image information for the driver of the vehicle. Examples of the prior art document that discloses such a technique include the following Patent Document 1. In the technique disclosed in Patent Document 1, a predetermined image analysis process is applied to each image information collected in the server device as described above, and it is analyzed that, for example, a large number of vehicles have not moved for a certain period of time. If so, it is determined that it is useful image information indicating that it is a traffic jam.

Japanese Unexamined Patent Publication No. 2017-21745

However, in the technique described in Patent Document 1, it is necessary to perform the image analysis processing on all of the collected image information, and therefore, especially when a large number of vehicles are concentrated due to traffic congestion. However, there is a problem that the processing load as a server device becomes an issue.

Further, in this case, for example, it is conceivable to have a configuration in which the result of performing the above image analysis processing in each vehicle is transmitted to the server device. However, even in this case, since it is necessary to constantly execute the image recognition processing and the like necessary for each vehicle, there is also a problem that the processing load in each vehicle increases as a result.

Therefore, the present application has been made in view of each of the above problems, and one example of the problem is the transmission destination of the image information, etc., as compared with the case where the image information corresponding to the image pickup result in each vehicle is constantly transmitted. It is an object of the present invention to provide an image processing apparatus and an image processing method capable of reducing the processing load in the image processing apparatus, a program for the image processing apparatus, and a recording medium on which the program is recorded.

In order to solve the above problem, the invention according to claim 1 is a distance between an imaging means whose imaging range is in front of a moving object and another moving object and the moving object which are reflected in the imaging range. A determination means for determining whether or not the distance is equal to or greater than a preset threshold distance, and a recording means for recording image information corresponding to the imaging result corresponding to the timing when the distance is determined to be equal to or greater than the threshold distance. And.

In order to solve the above problems, the invention according to claim 7 is an image processing performed in an image processing apparatus including an image pickup means having an image pickup range in front of a moving body, a determination means, and a transmission means. In the method, whether or not the imaging step of imaging the imaging range by the imaging means and the distance between another moving object and the moving object reflected in the imaging range are equal to or more than a preset threshold distance. It includes a determination step of determining whether or not the image is determined by the determination means, and a transmission step of transmitting image information corresponding to the image pickup result corresponding to the timing when the distance becomes equal to or greater than the threshold distance to the outside by the transmission means.

In order to solve the above problems, the invention according to claim 8 is any of claims 1 to 7, excluding the image pickup means, in which a computer connected to an image pickup means having an image pickup range in front of the moving body is used. It is made to function as the image processing apparatus according to the first item.

In order to solve the above problems, the invention according to claim 9 makes the image processing program according to claim 8 readable by a computer connected to an image pickup means having an image pickup range in front of the moving body. It has been recorded.

It is a block diagram which shows the outline structure of the image processing apparatus which concerns on embodiment. It is a block diagram which shows the outline structure of the drive recorder which concerns on Example. It is a flowchart which shows the image processing which concerns on Example. It is a figure which exemplifies the effect of the image processing which concerns on an Example, (a) is a figure which exemplifies the result in the case of the prior art, (b) is a figure which illustrates the effect of the image processing which concerns on an Example. be.

Next, a mode for carrying out the present application will be described with reference to FIG. Note that FIG. 1 is a block diagram showing an outline configuration of the image processing apparatus according to the embodiment.

As shown in FIG. 1, the image processing apparatus S according to the embodiment includes an image pickup means 1, a determination means 2, and a recording means 2-1.

In this configuration, the image pickup means 1 is an image pickup means whose imaging range is in front of the moving body.

Then, the determination means 2 may use the image pickup result of the image pickup means as the image pickup result indicating the situation around the moving body in the relationship between the other moving body and the moving body reflected in the image pickup range of the image pickup means 1. Determine if the relationship is as specific as possible.

As a result, the recording means 2-1 records the image information corresponding to the image pickup result of the image pickup means 1 corresponding to the timing when the relationship determined by the determination means 2 becomes the specific relationship.

As described above, according to the operation of the image processing apparatus S according to the embodiment, the relationship with other moving objects reflected in the front imaging range captures the imaging result as an imaging result indicating the surrounding situation. Image information corresponding to the imaging result corresponding to the timing of the specific relationship that can be used is recorded. Therefore, the processing load of the image processing apparatus can be reduced as compared with the case where the image processing is always performed.

Next, specific examples corresponding to the above-described embodiments will be described with reference to FIGS. 2 to 4. The examples described below are examples in which the present application is applied to image processing in a drive recorder provided in a vehicle as an example of the "moving body" according to the present application. In the following description, the vehicle equipped with the drive recorder is simply referred to as "own vehicle".

Further, FIG. 2 is a block diagram showing an outline configuration of a drive recorder according to an embodiment, FIG. 3 is a flowchart showing image processing according to the embodiment, and FIG. 4 illustrates an effect of the image processing according to the embodiment. It is a figure etc. At this time, in FIG. 2, for each of the constituent members of the embodiment corresponding to each constituent member in the image processing apparatus S according to the embodiment shown in FIG. 1, the same member number as each constituent member in the image processing apparatus S is used. ing.

As shown in FIG. 2, the drive recorder DR according to the embodiment includes a camera 1 composed of a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor) image pickup element, a CPU, a RAM (Random Access Memory), and the like. A processing unit 2 composed of a ROM (Read Only Memory) or the like, an interface 3, an operation unit 4 composed of an operation button or a touch panel, and a recording unit 5 composed of an HDD (Hard Disc Drive) or SSD (Solid State Drive) or the like. For example, it is composed of a sensor unit 6 including an acceleration sensor, an inter-vehicle distance sensor, and the like, and a display 7 including a liquid crystal display and the like. At this time, the camera 1 corresponds to an example of the imaging means 1 according to the embodiment, the processing unit 2 corresponds to an example of the determination means 2 and the recording means 2-1 according to the embodiment, respectively, and the interface 3 corresponds to the embodiment. Corresponds to an example of the transmission means 3 according to the above. Further, the processing unit 2 corresponds to an example of the "first determination means", an example of the "second determination means", and an example of the "detection means" according to the present application, respectively.

In the above configuration, the camera 1 has an image pickup range in front of the own vehicle, and under the control of the processing unit 2, the camera 1 captures the image pickup range cyclically (that is, cyclically), and corresponds to the image pickup result. The image data is output to the processing unit 2. This image data corresponds to an example of "image information" according to the present application. On the other hand, the interface 3 exchanges data with an external server device (not shown) via a network such as the Internet, or with a navigation device (not shown) mounted on another vehicle under the control of the processing unit 2. It controls the exchange of data by so-called vehicle-to-vehicle communication between vehicles. In the following description, the external server device (not shown) or the navigation device (not shown) mounted on another vehicle is simply referred to as a “server device or the like”. Further, when the operation by the user for designating the operation in the drive recorder DR is performed in the operation unit 4, the operation unit 4 generates an operation signal corresponding to the operation and outputs it to the processing unit 2. Further, the sensor unit 6 detects the acceleration applied to the own vehicle in, for example, the triaxial directions, and outputs the acceleration data indicating the detected acceleration to the processing unit 2. Further, the sensor unit 6 detects the inter-vehicle distance between the other vehicle existing in front of the own vehicle and the own vehicle, and outputs the distance data indicating the detected inter-vehicle distance to the processing unit 2. In the following description, the other vehicle is simply referred to as a "front vehicle". As a result, the processing unit 2 causes the recording unit 5 to non-volatileally record the image data output from the camera 1. Then, the processing unit 2 reads out necessary image data and transmits it to the server device or the like via the interface 3 by the image processing according to the embodiment described below. Further, the processing unit 2 detects the acceleration or speed of the own vehicle based on the acceleration data output from the sensor unit 2, and the front vehicle and the own vehicle based on the distance data output from the sensor unit 2. Detects the distance between vehicles. Further, the processing unit 2 controls the processing as the drive recorder DR including the image processing according to the embodiment based on the operation signal output from the operation unit 4.

Next, the image processing according to the embodiment will be specifically described with reference to FIGS. 2 and 3. As the flowchart is shown in FIG. 3, the image processing according to the embodiment is started, for example, at regular time intervals after the power switch (not shown) as the drive recorder DR is turned on. When the image processing according to the embodiment is started, the processing unit 2 first acquires congestion (congestion) information including, for example, a point in front of the own vehicle (step S1). This step S1 may be presumed to be congested, for example, when the speed of the own vehicle becomes a certain speed or less, and the congestion may be caused by, for example, VICS (registered trademark) (Vehicle Information Communication System) or vehicle-to-vehicle communication. The information itself may be obtained. When the congestion information is acquired in step S1, it is presumed that the rear part of another vehicle existing in front of the own vehicle is reflected in the image pickup range of the camera 1. Therefore, the processing unit 2 next detects whether or not the vehicle in front has started, that is, whether or not the state of the vehicle in front has changed from the stopped state to the moving state (running state) (step S2). As step S2, for example, the processing unit 2 may detect that the vehicle in front has started when the inter-vehicle distance gradually increases from a certain value, or the image itself of the vehicle in front captured by the camera 1 may be detected. When the image becomes smaller after analysis, it may be detected that the vehicle in front has started. Further, the start of the vehicle in front may be detected by performing so-called edge detection processing on the image. After that, the processing unit 2 determines whether or not the start of the vehicle in front is actually detected in step S2 (step S3). Then, if the vehicle in front does not exist in the first place due to, for example, the accuracy of the congestion information acquired in step S1 is low, the start is not detected (step S3: NO), so that the processing unit 2 returns to step S1 and described above. Repeat the process. On the other hand, when the start of the vehicle in front is detected in the determination of step S3 (step S3: YES), the processing unit 2 then determines whether or not the own vehicle has started, that is, the state of the own vehicle changes from the stopped state to the starting state. It detects whether or not it has changed (step S4). As step S4, for example, the processing unit 2 detects that the vehicle has started when the vehicle has a significant forward acceleration or speed based on the acceleration data output from the sensor unit 6. Can be configured. After that, the processing unit 2 determines whether or not the start of the own vehicle is actually detected in step S4 (step S5). Then, when the start of the own vehicle is not detected yet (step S5: NO), the processing unit 2 returns to the above step S4 and waits for the detection of the start of the own vehicle. On the other hand, when the start of the own vehicle is detected in the determination of step S5 (step S5: YES), the vehicle in front has started before that (see step S3: YES). In the meantime, in the server device or the like obtained from the drive recorder DR according to the embodiment, the image data corresponding to the image data obtained by imaging the state with the camera 1 is congested, for example, around the own vehicle. There is a distance (interval) that can be effectively used as image data indicating the situation of (traffic). That is, the server device and the like have an image in which the situation around the own vehicle is easily recognized due to the space (the space corresponding to the above distance (interval)) in which the vehicle in front starts first (see step S3: YES). You will be able to get the data. Therefore, the processing unit 2 records image data corresponding to the image pickup result captured at a time before (for example, 0.5 seconds before) a preset time from the timing when the start of the own vehicle is detected in step S5. It is read from No. 5 and the read image data is transmitted (uploaded) to the server device or the like via the interface 3 (step S6).

After that, the processing unit 2 determines whether or not to end the image processing according to the embodiment for reasons such as the power of the drive recorder DR being turned off (step S7). In the determination of step S7, when the image processing according to the embodiment is terminated (step S7: YES), the processing unit 2 terminates the image processing as it is. On the other hand, in the determination of step S7, when the image processing according to the embodiment is continued (step S7: NO), the processing unit 2 returns to the above step S1 and repeats the above-mentioned processing.

As described above, according to the image processing according to the embodiment, the distance from the vehicle in front that is reflected in the image pickup range in front is the interval at which the image pickup result can be used as the image pickup result indicating the surrounding situation. Image data corresponding to the imaging result corresponding to the timing (see step S5: YES in FIG. 3) is transmitted to an external server device or the like. Therefore, the processing load on the processing unit 2 and the destination of the image data can be reduced as compared with the case where the image data is constantly transmitted from the drive recorder DR, and the image data corresponds to the image data at the destination. The imaging result can be usefully used for processing related to, for example, congestion determination. More specifically, this effect will be described with reference to FIG. 4. For example, the image pickup result G captured by the camera 1 of the own vehicle in a state where there is no space between the vehicle and the vehicle in front is illustrated in FIG. 4 (a), for example. As described above, the vehicle C in front is shown very close, and even if the server device or the like acquires the image data corresponding to the image pickup result G in this state, it is usefully used for determining, for example, the congestion situation around the vehicle. It is not possible. On the other hand, in the image pickup result G captured by the camera 1 in a state where the distance from the front vehicle C is sufficiently large (see step S5: YES), the front vehicle C is, for example, as illustrated in FIG. 4 (b). It is reflected farther than in the case illustrated in FIG. 4 (a). Therefore, in this case, the distance between the own vehicle and the vehicle in front C is such that the image data corresponding to the image pickup result G can be effectively used for determining the situation around the own vehicle in the server device or the like. Vacant. Therefore, if the server device or the like acquires image data corresponding to the image pickup result in the state illustrated in FIG. 4B, it is possible to determine the congestion situation in the lane in which the vehicle is located or in another lane around the vehicle. It can be usefully used for.

Further, since the image data corresponding to the imaging result corresponding to the time when the vehicle in front is in the moving state and the own vehicle changes from the stopped state to the moving state is transmitted to the external server device or the like (FIG. 3, step S5: (Refer to YES), the processing load on the processing unit 2 and the destination of the image data can be reduced, and at the destination, the imaging result that accurately indicates the congestion situation in the surrounding area can be beneficially utilized. ..

Further, the image pickup result corresponding to the time when the own vehicle changes from the stopped state to the moving state (see FIG. 3 step 5: YES) after the vehicle in front changes from the stopped state to the moving state (see FIG. 3 step S3: YES). Since the corresponding image data is transmitted to an external server device or the like, it is possible to reduce the processing load on the processing unit 2 and the destination of the image data, and at the same time, the situation of congestion in the surrounding area can be further improved. It is possible to make useful use of the imaging results shown accurately.

Furthermore, when determining whether or not the vehicle in front is in a moving state based on the imaging result of the vehicle in front (see step S2 in FIG. 3), it is ensured that the vehicle in front is in a moving state. It can be detected and the necessary image data can be transmitted to the outside.

Further, when determining whether or not the own vehicle has changed from the stopped state to the moving state based on the acceleration or speed of the own vehicle (see step S4 in FIG. 3), it is determined that the own vehicle has changed from the stopped state to the moving state. It can be reliably detected and the necessary image data can be transmitted to the outside.

In the above-described embodiment, the start of each of the vehicle in front and the vehicle is detected to determine whether or not the distance between the vehicle in front and the vehicle is large (see steps S2 to S5 in FIG. 3). Here, for example, when both the vehicle in front and the own vehicle are present in a traffic jam traveling at a low speed, the distance between the vehicles may not be widened easily. Therefore, in such a case, for example, the time from when the vehicle in front changes from the stopped state to the moving state until the own vehicle changes from the stopped state to the moving state is empirical, experimental, or regional (for example, at that point). The image data corresponding to the image pickup result corresponding to the time when the preset threshold time or more is exceeded according to the specificity of the traffic jam may be transmitted to an external server device or the like. In this case as well, the processing load on the processing unit 2 and the transmission destination of the image data can be reduced, and the imaging result that accurately indicates the congestion situation in the surrounding area can be beneficially utilized at the transmission destination. can. Furthermore, the distance of the vehicle in front when the vehicle in front changes from the stopped state to the moving state and then the vehicle changes from the stopped state to the moving state is empirical, experimental, or regional (for example, at that point). The image data corresponding to the image pickup result corresponding to the time point when the distance becomes equal to or more than the preset threshold distance according to the specificity of the traffic jam may be transmitted to an external server device or the like. In this case as well, the processing load on the processing unit 2 and the transmission destination of the image data can be reduced, and the imaging result that accurately indicates the congestion situation in the surrounding area can be beneficially utilized at the transmission destination. can.

Further, in the above-described embodiment, the case where the embodiment is applied to the image processing in the drive recorder DR mounted on the own vehicle has been described, but in addition to this, the image processing in the drive recorder mounted on the motorcycle or the railway vehicle has been described. The embodiment can also be applied to.

Further, in addition to the above-described embodiment, for example, when the own vehicle is waiting for a signal and the vehicle in front disappears from the imaging range of the own vehicle, the timing of step S5: YES may be set. In this case, the image data in a state where the front view is open can be transmitted to an external server device or the like.

Further, a program corresponding to the flowchart shown in FIG. 3 is recorded on a recording medium such as an optical disk or a hard disk, or acquired via a network such as the Internet, and read out to a general-purpose microcomputer or the like. By executing this, it is possible to make the microcomputer or the like function as the processing unit 2 according to the embodiment.

1 Imaging means (camera)
2 Judgment means (processing unit)
2-1 Recording means 3 Interface DR drive recorder S Image processing device C Front vehicle G Imaging result

In order to solve the above problem, the invention according to claim 1 is a distance between an imaging means whose imaging range is the front of a moving object and another moving object and the moving object which are reflected in the imaging range. Records image information corresponding to the imaging result of the imaging means at the timing when it is determined that the distance is equal to or greater than the preset threshold distance and the determination means for determining whether or not the distance is equal to or greater than the preset threshold distance. It is equipped with a recording means.

In order to solve the above problems, the invention according to claim 7 is an image processing performed in an image processing apparatus including an image pickup means having an image pickup range in front of a moving body, a determination means, and a transmission means. In the method, whether or not the imaging step of imaging the imaging range by the imaging means and the distance between another moving object and the moving object reflected in the imaging range are equal to or more than a preset threshold distance. A determination step of determining whether or not the image is determined by the determination means, and a transmission step of transmitting image information corresponding to the image pickup result of the image pickup means to the outside by the transmission means at the timing when the distance becomes equal to or greater than the threshold distance. include.

Claims (9)

An imaging means whose imaging range is in front of the moving object,
A determination means for determining whether or not the distance between the moving body and another moving body reflected in the imaging range is equal to or larger than a preset threshold distance.
A recording means for recording image information corresponding to the imaging result corresponding to the timing when the distance is determined to be equal to or greater than the threshold distance.
An image processing device characterized by comprising. In the image processing apparatus according to claim 1,
An image processing apparatus characterized in that the threshold distance is set in advance as a distance at which the image pickup result can be used as an image pickup result indicating a situation around the moving body. In the image processing apparatus according to claim 1 or 2.
An image processing apparatus further comprising a transmission means for transmitting the image information corresponding to the image pickup result corresponding to the timing to the outside. The image processing apparatus according to any one of claims 1 to 3.
The determination means is
The first determination means for determining whether or not the moving body has changed from the stopped state to the moving state,
A second determination means for determining whether or not the other moving body has changed from the stopped state to the moving state,
Equipped with
The recording means corresponds to the timing when the distance when the moving body changes from the stopped state to the moving state after the other moving body changes from the stopped state to the moving state becomes equal to or more than the threshold distance. An image processing device characterized by recording the image information corresponding to an image pickup result. In the image processing apparatus according to claim 4,
The second determination means is an image processing apparatus characterized in that it determines whether or not the other moving object has changed from a stopped state to a moving state based on the image pickup result in which the other moving object is reflected. .. In the image processing apparatus according to claim 4 or 5.
The first determination means further includes a detection means for detecting the acceleration or speed of the moving body.
The first determination means is an image processing device that determines whether or not the moving body has changed from a stopped state to a moving state based on the detected acceleration or velocity. An image processing method executed in an image processing apparatus including an image pickup means having an image pickup range in front of a moving body, a determination means, and a transmission means.
An imaging step in which the imaging range is imaged by the imaging means, and
A determination step of determining whether or not the distance between the moving object and another moving object reflected in the imaging range is equal to or greater than a preset threshold distance by the determining means.
A transmission step of transmitting image information corresponding to the image pickup result corresponding to the timing when the distance becomes equal to or more than the threshold distance to the outside by the transmission means.
An image processing method comprising. A computer connected to an imaging means whose imaging range is in front of the moving object,
An image processing program comprising the function as the image processing apparatus according to any one of claims 1 to 7, excluding the image pickup means. A recording medium according to claim 8, wherein the image processing program is readablely recorded by a computer connected to an image pickup means having an image pickup range in front of the moving body.

Images