JP7276255B2 - Image processing device, program, and image processing system - Google Patents

Description

The present disclosure relates to image processing apparatuses, programs, and image processing systems.

Patent Document 1 discloses a display device for a vehicle driving assist device. The display device highlights the preceding vehicle by displaying the symbol representing the preceding vehicle in front of the own lane and the symbol representing the vehicle outside the own lane in different colors.

Japanese Patent No. 4779355

The display device described in Patent Document 1 may not be able to guide the line of sight of the vehicle occupant to a specific target in the displayed image. In one aspect of the present disclosure, it is preferable to provide an image processing device, a program, and an image processing system capable of guiding the line of sight of a vehicle occupant to a specific target in a displayed image.

(1) One aspect of the present disclosure is an image processing device (3) that displays images (57, 65, 67) on display devices (15, 17, 19) provided in a vehicle (2). The image processing device acquires first information representing the position of a target (59) existing around the vehicle and second information representing whether or not the target is a target to be emphasized (59A). and an image creation unit (37) configured to create an image representing said target at said position represented by said first information.

When the second information indicates that at least a part of the target represented in the image is the target to be emphasized, the image creation unit may determine that the target to be emphasized is within the depth of field. Create the image (57, 65) represented as such.

An image processing device, which is one aspect of the present disclosure, can guide the line of sight of a vehicle occupant to a target to be emphasized in an image to be created.
(2) Another aspect of the present disclosure is a program that causes a computer to function as an image processing device (3). The image processing device includes first information representing the position of a target (59) existing around the vehicle (2), and second information representing whether or not the target is a target to be emphasized (59A). and an image configured to create an image (57, 65, 67) representing said target at said location represented by said first information. a production unit (37).

When the second information indicates that at least a part of the target represented in the image is the target to be emphasized, the image creation unit may determine that the target to be emphasized is within the depth of field. Create the image (57, 65) represented as such.

A program, which is another aspect of the present disclosure, causes a computer to function as an image processing device. The image processing device can guide the line of sight of the vehicle occupant to the target to be emphasized in the created image.

(3) Another aspect of the present disclosure is an image processing system (41, 3) comprising an information providing device (41) and an image processing device (3). The information providing device includes first information representing the position of a target (59) existing around the vehicle (2), and second information representing whether or not the target is a target to be emphasized (59A). and configured to provide

The image processing device comprises: an information acquisition unit (35) configured to acquire the first information and the second information from the information providing device; an image creation unit (37) configured to create an image (57, 65, 67) representing the.

When the second information indicates that at least a part of the target represented in the image is the target to be emphasized, the image creation unit may determine that the target to be emphasized is within the depth of field. Create the image (57, 65) represented as such.

An image processing system, which is another aspect of the present disclosure, can guide the line of sight of a vehicle occupant to a target to be emphasized in an image to be created.

1 is a block diagram showing the configuration of an in-vehicle system; FIG. It is a block diagram showing the functional composition of an in-vehicle system. 4 is a flowchart showing processing executed by an image processing apparatus; FIG. 4 is an explanatory diagram showing an example of a first image; FIG. 4 is an explanatory diagram showing an example of a first image; FIG. 10 is an explanatory diagram showing an example of a second image; FIG. 10 is an explanatory diagram showing an example of a second image; FIG. 11 is an explanatory diagram showing an example of a third image; FIG. 11 is an explanatory diagram showing an example of a third image; FIG. 11 is an explanatory diagram showing an example of a third image; FIG. 12 is an explanatory diagram showing changes in the second image when the priority of the target to be emphasized changes; FIG. 10 is an explanatory diagram showing processing for changing the display range of the first image;

Exemplary embodiments of the present disclosure are described with reference to the drawings.
<First Embodiment>
1. Configuration of In-vehicle System 1 The configuration of the in-vehicle system 1 will be described with reference to FIGS. 1 and 2. FIG. An in-vehicle system 1 is mounted in a vehicle 2 .

As shown in FIG. 1, the in-vehicle system 1 includes an image processing device 3, an automatic driving system 5, a locator ECU 7, a travel control ECU 9, a surrounding monitoring sensor 11, a meter display 15, and a CID (Center Information Display). 17 , HUD 19 , operation device 21 , body ECU 23 , direction indicator SW (switch) 25 , and driver monitor 27 . Each component of the in-vehicle system 1 is communicably connected to each other via a communication bus 29 .

The image processing apparatus 3 includes a microcomputer having a CPU 31 and a semiconductor memory such as RAM or ROM (hereinafter referred to as memory 33).
Each function of the image processing device 3 is realized by the CPU 31 executing a program stored in a non-transitional substantive recording medium. In this example, the memory 33 corresponds to a non-transitional substantive recording medium storing programs. Also, by executing this program, a method corresponding to the program is executed. Note that the image processing apparatus 3 may include one microcomputer, or may include a plurality of microcomputers.

The image processing device 3 includes an information acquisition unit 35, an image creation unit 37, and an image output unit 39, as shown in FIG. The function of each component provided in the image processing apparatus 3 will be described later.
The automatic driving system 5 includes an automatic driving ECU 41 and a driving support ECU 43 . The automatic driving ECU 41 is an in-vehicle ECU that realizes an automatic driving function that automatically performs driving operations on behalf of the occupant of the vehicle 2 . The automatic driving ECU 41 enables autonomous driving at automatic driving level 3 or higher defined by the Society of Automotive Engineers of America, for example.

The automatic driving ECU 41 includes an environment recognition unit 45, an action determination unit 47, and an operation execution unit 49, as shown in FIG.
The environment recognition unit 45 acquires locator information and map information from the locator ECU 7 . Locator information and map information will be described later. The environment recognition unit 45 acquires detection information from the perimeter monitoring sensor 11 . Detection information will be described later.

The environment recognition unit 45 recognizes the driving environment of the vehicle 2 based on the locator information, map information, and detection information. The running environment includes, for example, the position and speed of a target 59 existing around the vehicle 2 . Examples of targets 59 include other vehicles, pedestrians, bicycles, animals, fixed objects, lane boundaries, and the like. Further, the running environment includes, for example, the position and shape of the lane on which the vehicle 2 is running.

The environment recognition unit 45 produces first information and second information. The first information is information representing the position of the target object 59 existing around the vehicle 2 . The second information is information indicating whether or not the target 59 whose position is indicated by the first information is the emphasis target target 59A. The emphasis target object 59A is a target object to which the occupant of the vehicle 2 should pay attention.

Examples of the target object 59A to be emphasized include a preceding vehicle that the vehicle 2 is following, another vehicle that is meandering, and a vehicle that is traveling in the lane after the change when the vehicle 2 is planning to change lanes. other vehicles in the vehicle, surrounding vehicles communicating with the vehicle 2, surrounding vehicles recognized by the automatic driving system 5 that should avoid running side by side, and surrounding vehicles that have cut in and entered the vehicle recognized by the automatic driving system 5. be done.

The environment recognition unit 45 sets the priority to the emphasis target object 59A when there are two or more emphasis target targets 59A. The higher the need for attention of the vehicle 2, the higher the priority. The second information further includes priority.

The environment recognition unit 45 can set the priority based on the behavior of the vehicle 2, for example. The environment recognition unit 45 acquires later-described direction indication information from the body ECU 23, for example. Based on the direction indication information, the environment recognition unit 45 searches for a target on the changed route when the vehicle 2 changes the route. If a target exists on the changed course, the environment recognition unit 45 sets the priority of that target higher than that of other targets.

The behavior determination unit 47 generates a planned travel line for the vehicle 2 based on the travel environment recognized by the environment recognition unit 45 . The operation execution unit 49 performs acceleration/deceleration control, steering control, etc. of the vehicle 2 so that the vehicle 2 travels along the planned travel line generated by the action determination unit 47 in cooperation with the travel control ECU 9 . The operation execution unit 49 creates the third information. The third information is information indicating whether or not the vehicle 2 is currently in the automatic driving state. The automatic driving state is a state in which automatic driving is being performed.

The automatic driving ECU 41 corresponds to an information providing device that provides the image processing device 3 with the first information and the second information. A system including the automatic driving ECU 41 and the image processing device 3 corresponds to an image processing system.
The driving assistance ECU 43 is an in-vehicle ECU that realizes a driving assistance operation for assisting the driving operation of the occupant of the vehicle 2 . The driving assistance ECU 43 enables advanced driving assistance or partial automatic driving control, for example, about automatic driving level 2 defined by the Society of Automotive Engineers of America.

The locator ECU 7 includes a GNSS (Global Navigation Satellite System) receiver, an inertial sensor, and the like. The locator ECU 7 combines the positioning signal received by the GNSS receiver, the measurement result of the inertial sensor, the vehicle speed information output to the communication bus 29, and the like, and sequentially measures the position, traveling direction, and the like of the vehicle 2. The locator ECU 7 sequentially outputs locator information to the communication bus 29 . The locator information is information representing the position, traveling direction, and the like of the vehicle 2 .

The locator ECU 7 further includes a map database 51 . The map database 51 is mainly composed of a large-capacity storage medium. A storage medium stores a large amount of map data. Map data includes three-dimensional map data and two-dimensional map data. Three-dimensional map data is so-called high-precision map data. The 3D map data contains information necessary for advanced driving assistance and automatic driving. The 3D map data includes 3D shape information of roads, detailed information of each lane, and the like.

The locator ECU 7 reads map data around the current position of the vehicle 2 from the map database 51 . The locator 7ECU provides map information together with locator information to the driving support ECU 43, the automatic driving ECU 41, and the like. The map information is information including map data read from the map database 51 .

In addition, the in-vehicle system 1 acquires locator information and map information from a user terminal such as a smartphone or a navigation device instead of the locator ECU 7, and provides the acquired locator information and map information to the driving support ECU 43, the automatic driving ECU 41, and the like. You may

The travel control ECU 9 is an electronic control device that mainly includes a microcontroller. The travel control ECU 9 generates vehicle speed information indicating the current travel speed of the vehicle 2 based on detection signals from wheel speed sensors provided at the hub portions of the wheels of the vehicle 2 . The traveling control ECU 9 sequentially outputs vehicle speed information to the communication bus 29 .

The travel control ECU 9 has at least functions of a brake control ECU and a drive control ECU. The travel control ECU 9 brakes each wheel of the vehicle 2 based on one of an operation command based on the driving operation of the occupant of the vehicle 2, a control command from the driving support ECU 43, and a control command from the automatic driving ECU 41. Force control and output control of the power source of the vehicle 2 are continuously performed.

The surroundings monitoring sensor 11 is an autonomous sensor that monitors the surrounding environment of the vehicle 2 . The surroundings monitoring sensor 11 can detect targets existing around the vehicle 2 . The surroundings monitoring sensor 11 provides detection information to the driving support ECU 43, the automatic driving ECU 41, and the like via the communication bus 29. The detection information is information representing the position, speed, etc. of the detected target.

The perimeter monitoring sensor 11 includes, for example, a camera unit 53, a millimeter wave radar 55, and the like. The camera unit 53 may be configured to include a monocular camera or may be configured to include a compound eye camera. The camera unit 53 can photograph, for example, the front range, the side range, and the rear range of the vehicle 2 . The detection information includes, for example, at least one of imaging data generated by imaging the surroundings of the vehicle 2 by the camera unit 53 and an analysis result of the imaging data.

The millimeter wave radar 55 emits millimeter waves or quasi-millimeter waves toward the surroundings of the vehicle 2 . The millimeter wave radar 55 receives reflected waves reflected by the target. The detection information includes detection results of the millimeter wave radar 55, for example. The perimeter monitoring sensor 11 may further include a lidar, a sonar, and the like.

The meter display 15 and the CID 17 each have a configuration mainly including a liquid crystal display or an OLED (Organic Light Emitting Diode) display, for example. The meter display 15 and the CID 17 display various images on the display screen based on the control signal and video data acquired from the image processing device 3 . The meter display 15 is installed in front of the driver's seat of the vehicle 2, for example. The CID 17 is installed above the center cluster of the vehicle 2, for example. The CID 17 has a touch panel function. The CID 17 detects, for example, a touch operation, a swipe operation, or the like on the display screen by an occupant of the vehicle 2 or the like.

The HUD 19 projects the light of the image formed in front of the occupant of the vehicle 2 onto the windshield based on the control signal and video data acquired from the image processing device 3 . An occupant of the vehicle 2 perceives the light of the image reflected by the windshield toward the interior of the vehicle 2 . Thus, the HUD 19 displays a virtual image in the space ahead of the windshield. The occupant of the vehicle 2 visually recognizes the virtual image displayed by the HUD 19 while superimposing it on the foreground of the vehicle 2 .

Note that the meter display 15, the CID 17, and the HUD 19 correspond to display devices that the vehicle 2 has.
The operation device 21 is an input unit that receives a user operation by an occupant of the vehicle 2 or the like. For example, user operations related to activation and deactivation of the automatic driving function are input to the operation device 21 . Examples of the operating device 21 include a steering switch provided on the spoke portion of the steering wheel, an operating lever provided on the steering column portion, and a voice input device for recognizing the utterance content of the occupant of the vehicle 2 .

The body ECU 23 is an electronic control device that mainly includes a microcontroller. The body ECU 23 has at least the function of controlling the operation of the lighting device mounted on the vehicle 2 . Body ECU 23 is electrically connected to direction switch 25 . The direction indicator switch 25 is a lever-shaped operating portion provided on the steering column.

The direction switch 25 can accept a leftward operation and a rightward operation. When the direction indicator switch 25 receives a leftward operation, the body ECU 23 causes the left direction indicator to start blinking. When the direction indicator switch 25 receives a right direction operation, the body ECU 23 causes the right direction indicator to start blinking.

The body ECU 23 provides the direction indication information to the automatic driving ECU 41 and the like via the communication bus 29 . The direction indication information is information representing the content of the operation performed on the direction indication switch 25 .
The driver monitor 27 includes a near-infrared light source, a near-infrared camera, and a control unit for controlling them. The driver monitor 27 is installed, for example, on the upper surface of the steering column or the upper surface of the instrument panel. The driver monitor 27 is attached so that the near-infrared camera faces the headrest portion of the driver's seat.

The driver monitor 27 uses a near-infrared camera to photograph the head of the occupant of the vehicle 2 irradiated with near-infrared light from the near-infrared light source. An image captured by the near-infrared camera is image-analyzed by the control unit. The control unit extracts information (hereinafter referred to as occupant information) representing the position of the eye point and the line-of-sight direction of the occupant of the vehicle 2 from the captured image. The control unit provides the extracted occupant information to the image processing device 3 and the like via the communication bus 29 .

2. 3. Processing Executed by Image Processing Apparatus 3 The processing repeatedly executed by the image processing apparatus 3 at predetermined time intervals will be described with reference to FIGS. 3 to 12. FIG. In step 1 of FIG. 3 , the information acquisition unit 35 acquires first information and second information from the environment recognition unit 45 . The first information is information representing the positions of targets existing around the vehicle 2 . The second information is information indicating whether or not the target whose position is indicated by the first information is the emphasis target target 59A. The information acquisition unit 35 also acquires the third information from the behavior determination unit 47 . The third information is information indicating whether or not the vehicle 2 is currently in the automatic driving state.

In step 2, the image creation unit 37 determines whether or not the vehicle 2 is currently in the automatic driving state based on the third information acquired in step 1 above. If it is determined that the vehicle is in the automatic driving state, the process proceeds to step 3. If it is determined that the vehicle is not in the automatic driving state, the process ends.

In step 3, the image forming unit 37 recognizes the targets corresponding to the enhancement target 59A among the targets whose positions are represented by the first information. The image creation unit 37 recognizes the target, which is determined as the enhancement target target 59A in the second information acquired in step 1, as the enhancement target target 59A. The image creation unit 37 calculates the number of recognized enhancement target objects 59A. If the number of emphasis target objects 59A is 1, the process proceeds to step 4; If the number of emphasis target objects 59A is two or more, the process proceeds to step 5. If the number of emphasis target objects 59A is 0, the process proceeds to step 6;

In step 4 , image creation unit 37 creates first image 57 . An example of the first image 57 is shown in FIG. A first image 57 is an image representing a target 59 . In the example shown in FIG. 4, target 59 is another vehicle. The first image 57 is, for example, a composite image simulating a photographed image photographed by a camera from a fixed photographing position. The shooting position is, for example, a position above and behind the vehicle 2 . The position of the target 59 in the first image 57 is the position of the target represented by the first information.

A portion of first image 57 is depth of field region 61 . Targets 59 that are within depth of field region 61 are represented as being in depth of field. That is, targets 59 within the depth-of-field region 61 appear more in focus than targets 59 outside the depth-of-field region 61 .

In first image 57 , enhanced target 59 A is within depth of field region 61 . Therefore, in the first image 57, the target object 59A to be emphasized is represented as being in the depth of field. In the first image 57, for example, the emphasis target object 59A is emphasized.

Examples of emphasizing expressions include adding highlights, adding animation processing, adding other parts, and the like. Animation processing includes, for example, blinking animation, enlargement animation, and the like. Other components include, for example, spotlights, ripple marks, arrows, and the like. In addition, as the emphasized expression, for example, the target 59 that is not the emphasized target 59A (hereinafter referred to as the normal target 59B) is represented in a different color.
For example, when only one target 59 exists in the depth of field area 61, the enhancement target 59A does not have to be emphasized.

In the first image 57, the normal target 59B may or may not exist within the depth of field region 61. FIG. In the example shown in FIG. 4, the normal target 59B exists within the depth of field region 61. In the example shown in FIG. The normal target 59B in the depth of field region 61 is not, for example, emphasized as described above. The normal target 59B in the depth of field region 61 is, for example, rendered inconspicuous. Examples of less conspicuous expressions include mosaic expression, tone down, pixel reduction, addition of animation processing, addition of other parts, and the like. As processing of the animation, for example, reduction of the normal target 59B can be mentioned. Another component is, for example, a mask that normally hides the target 59B.

In the first image 57, for example, outside the depth-of-field region 61, there is usually a target 59B. A normal target 59B that is outside the depth of field region 61 is represented as being outside the depth of field. That is, the normal target 59B outside the depth-of-field region 61 appears out of focus compared to the enhancement target 59A. The normal target 59B outside the depth of field region 61 is not, for example, emphasized as described above.

The first image 57 includes the vehicle 2, for example, as shown in FIG. The vehicle 2 is, for example, within the depth of field region 61 . In this case, in the first image 57 the vehicle 2 appears to be in focus compared to the normal target 59B outside the depth of field region 61 . The vehicle 2 may exist outside the depth of field region 61, for example. Also, the first image 57 may have a depth of field area 61 different from the depth of field area 61 . The vehicle 2 may, for example, be present in a depth of field area different from the depth of field area 61 .

The first image 57 includes, for example, a representation of behavior of the vehicle 2 . The display representing the behavior of the vehicle 2 includes, for example, a display in which the stop lamps and turn lamps of the vehicle 2 represented in the first image 57 light up or blink. The indication that the stop lamp of the vehicle 2 is turned on in the first image 57 indicates that the vehicle 2 is braked. The blinking display of the turn lamp of the vehicle 2 displayed in the first image 57 indicates that the vehicle 2 changes course to the right or left. The vehicle 2 braking and the vehicle 2 turning left or right correspond to the behavior of the vehicle 2 .
For example, when the stop lamps and turn lamps of the vehicle 2 are lit or blinked, the vehicle 2 may be the emphasis target object 59A.

The first image 57 may be as shown in FIG. The first image 57 shown in FIG. 5 is basically the same as the first image 57 shown in FIG. However, first image 57 shown in FIG. 5 further includes warning 63 . The warning 63 includes the content to be conveyed to the occupants of the vehicle 2 . The content of the warning 63 is, for example, the content regarding the emphasis target object 59A. In the first image 57 the warning 63 is outside the depth of field region 61 . Alert 63 appears to be in focus.

In step 5 , image creation unit 37 creates second image 65 . An example of the second image 65 is shown in FIG. The second image 65 is basically the same as the first image 57 . However, in the second image 65 , two or more enhancement target objects 59A are within the depth of field region 61 . For example, in the second image 65 , all enhancement target objects 59 A identified as enhancement target objects 59 A by the second information are within the depth of field region 61 . Therefore, in the second image 65, the two or more emphasis target objects 59A are represented as if they are in the depth of field.

In the second image 65, for example, two or more emphasis target objects 59A are respectively emphasized. For example, the second image 65 is represented with a deeper depth of field than the first image 57 . When there is a difference in priority between two or more enhancement target objects 59A, in the second image 65, for example, the enhancement target object 59A appears to be more focused as its priority is higher. be done. Further, in the second image 65, for example, the emphasis target object 59A is emphasized more remarkably as its priority is higher.

The second image 65 may be as shown in FIG. A second image 65 shown in FIG. 7 is divided into a divided image 65A and a divided image 65B. The split image 65A includes a depth of field area 61A. Split image 65B includes depth of field region 61B. For example, the vertical position of the depth of field region 61A and the vertical position of the depth of field region 61B are different.

In split image 65A, enhancement target 59A is within depth of field region 61A. In split image 65B, enhancement target 59A is within depth of field region 61B. In each of the divided image 65A and the divided image 65B, for example, the emphasis target object 59A is emphasized.

The divided image 65A and the divided image 65B are arranged so that the depth of field is narrower than in the case of providing one depth-of-field area including all the enhancement target objects 59A present in the divided image 65A and the divided image 65B. expressed. Therefore, the effect of guiding the line of sight of the occupant of the vehicle 2 to the emphasis target object 59A is even higher.

In step 6 , image creation unit 37 creates third image 67 . An example of the third image 67 is shown in FIG. A third image 67 shown in FIG. 8 is basically the same as the first image 57 . However, in the third image 67 shown in FIG. 8, all the normal targets 59B are represented as being in focus. Also, the third image 67 does not include the enhancement target object 59A.

The third image 67 may be as shown in FIG. A third image 67 shown in FIG. 9 is basically the same as the first image 57 . However, in the third image 67 shown in FIG. 9 , the normal target 59B corresponding to the vehicle ahead of the vehicle 2 exists in the depth of field region 61 .

The third image 67 may be the one shown in FIG. A third image 67 shown in FIG. 10 is basically the same as the first image 57 . However, in the third image 67 shown in FIG. 10 , the vehicle 2 exists within the depth of field region 61 . A normal target 59B may also be present in the depth of field region 61 .

In step 7, the image output unit 39 outputs control signals and video data corresponding to the image created in any one of steps 4 to 6 to the display device. The display device is one or more of meter display 15 , CID 17 and HUD 19 . The display device displays the image created in any of steps 4-6 above. An occupant of the vehicle 2 can see the displayed image.

The image processing device 3 repeats the processing of steps 1 to 7 described above. The priority of the emphasis target object 59A represented in the second image 65 may change with the passage of time. In the example shown in FIG. 11, the second image 65 represents two enhanced targets 59A. At time T1, of the two emphasis target objects 59A, the emphasis target object 59A in the right lane was the priority target object 59M. The priority target 59M is the emphasis target 59A with the highest priority.

The second image 65 at time T1, for example, represents the priority target 59M as being more in focus than the enhancement target 59A on the left. Also, in the second image 65 at time T1, for example, the priority target 59M is emphasized more significantly than the emphasis target 59A on the left side.

At time T2, the left direction indicator of the vehicle 2 started blinking. The content of the direction indication information is the content indicating that the direction indication switch 25 has been operated in the left direction. Based on the direction indication information, the environment recognition unit 45 sets the left emphasis target 59A of the two emphasis targets 59A as the priority target 59M. The emphasis target 59A in the right lane is no longer the priority target 59M.

The second image 65 at time T2, for example, shows the new priority target 59M to be more in focus than the emphasis target 59A in the right lane. Also, in the second image 65 at time T2, for example, the new priority target 59M is emphasized more significantly than the emphasis target 59A in the right lane.

The image processing device 3 repeats the processing of S1 to S7 described above. The emphasis target object 59A represented in the first image 57 or the second image 65 may change over time. For example, normal target 59B may change to enhanced target 59A. In addition, the emphasis target target 59A may change to the normal target 59B. Also, the enhancement target object 59A that has been out of the angle of view until then may be newly represented in the first image 57 or the second image 65 . Also, the emphasis target object 59A that has been displayed in the first image 57 or the second image 65 may disappear.

The image creation unit 37 changes the first image 57 and the second image 65 in accordance with the change in the enhancement target object 59A. For example, the image creation unit 37 changes the position, size, shape, focus position, etc. of the depth of field area 61 so that the changed emphasis target object 59A is included in the depth of field area 61 . The focus position is the most clearly visible part of the depth of field area 61 . The image creation unit 37 preferably seamlessly changes the position, size, shape, focus position, etc. of the depth of field area 61 .

The image creation unit 37 can change the display range of the first image 57 or the second image 65 according to the positions, number, etc. of the emphasis target object 59A. For example, the image creation unit 37 creates either the first image 57 with a wide display range shown on the left side of FIG. 12 or the first image 57 with a narrow display range shown on the right side of FIG. It can be selected according to the number and the like. Image creation unit 37 preferably seamlessly changes the display range of first image 57 or second image 65 .

3. Effects of Image Processing Apparatus 3 (1A) The image processing apparatus 3 acquires first information and second information. The image processing device 3 creates an image representing the target 59 at the position represented by the first information. If the second information indicates that at least part of the target 59 represented in the image is the enhancement target 59A, the image processing device 3 adjusts the enhancement target 59A so that it is in the depth of field. create a first image 57 or a second image 65 represented by .

In the first image 57 or the second image 65, the emphasis target 59A is represented as if it were in the depth of field, so it is easy to attract the attention of the occupants of the vehicle 2. FIG. Therefore, the image processing device 3 can guide the line of sight of the occupant of the vehicle 2 to the enhancement target object 59A in the first image 57 or the second image 65 .

(1B) The image processing device 3 can create an image that includes the vehicle 2 represented as being in the depth of field and a representation of the behavior of the vehicle 2 . An occupant of the vehicle 2 can easily understand the behavior of the vehicle 2 by viewing the image. Further, the occupant of the vehicle 2 can understand the relationship between the behavior of the vehicle 2 and the emphasis target object 59A by viewing the image.

(1C) The image processing device 3 can display a warning 63 in the portion of the image that is represented as being outside the depth of field. Since the area around the warning 63 is shown to be out of focus, the warning 63 is easily conspicuous.

(1D) In the first image 57 or the second image 65, when the target object 59A to be enhanced is changed, the image processing device 3 displays the changed target object 59A to be enhanced so that it is in the depth of field. so that the first image 57 or the second image 65 is changed. Therefore, even when the emphasis target object 59A changes, the image processing device 3 can maintain the above effect (1A).

(1E) The second information represents the priority of the emphasis target object 59A. The image processing device 3 displays the enhancement target object 59A in the first image 57 or the second image 65 such that the higher the priority, the better the focus. The image processing device 3 can further guide the line of sight of the occupant of the vehicle 2 to the emphasis target object 59A having a high priority in the first image 57 or the second image 65 .

(1F) When the priority of the target object 59A to be emphasized represented in the first image 57 or the second image 65 is changed, the image processing device 3 converts the representation of the target object 59A to the priority after the change. change the expression according to Therefore, even when the priority of the emphasis target object 59A changes, the image processing device 3 can maintain the effect of (1E).
<Other embodiments>
Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments, and various modifications can be made.

(1) The image processing device 3 can reduce the texture resolution and the number of display polygons of the portion of the image other than the depth of field region 61 compared to the depth of field region 61 . In this case, the calculation load of the image processing device 3 can be reduced.

(2) The priority of the emphasis target object 59A may be set based on information other than direction indication information. For example, the image processing device 3 sets the priority of the emphasis target object 59A based on the occupant information acquired from the driver monitor 27 .

The occupant information is, for example, information representing the line-of-sight direction of the occupant of the vehicle 2 . The image processing device 3 gives higher priority to the emphasis target object 59A in the direction not viewed by the occupant of the vehicle 2 than to the emphasis target object 59A in the line-of-sight direction of the occupant of the vehicle 2 . In this case, the image processing device 3 can guide the line of sight of the occupant of the vehicle 2 to the emphasis target object 59A, which the occupant of the vehicle 2 may not have noticed.

(3) The image processing apparatus 3 and techniques described in the present disclosure were provided by configuring a processor and memory programmed to perform one or more functions embodied by a computer program. It may also be implemented by a dedicated computer. Alternatively, the image processing apparatus 3 and techniques described in this disclosure may be implemented by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the image processing apparatus 3 and techniques thereof described in this disclosure are a combination of a processor and memory programmed to perform one or more functions and a processor configured by one or more hardware logic circuits. It may also be implemented by one or more dedicated computers configured in combination. Computer programs may also be stored as computer-executable instructions on a computer-readable non-transitional tangible storage medium. The method of realizing the function of each part included in the image processing device 3 does not necessarily include software, and all the functions may be realized using one or a plurality of pieces of hardware.

(4) A plurality of functions possessed by one component in the above embodiment may be realized by a plurality of components, or a function possessed by one component may be realized by a plurality of components. . Also, a plurality of functions possessed by a plurality of components may be realized by a single component, or a function realized by a plurality of components may be realized by a single component. Also, part of the configuration of the above embodiment may be omitted. Moreover, at least part of the configuration of the above embodiment may be added or replaced with respect to the configuration of the other above embodiment.

(5) In addition to the image processing device 3 described above, a system having the image processing device 3 as a component, a program for causing a computer to function as the image processing device 3, and a non-transitional device such as a semiconductor memory recording this program The present disclosure can also be implemented in various forms such as a physical recording medium, an image processing method, an image creation method, an image display method, and the like.

2 Vehicle 3 Image processing device 15 Meter display 17 CID 19 HUD 35 Information acquisition unit 37 Image creation unit 39 Image output unit 41 Automated driving ECU 45 Environment Recognition unit 47 Action determination unit 49 Operation execution unit 57 First image 59 Target 59A Target to be emphasized 59B Normal target 59M Priority target 61, 61A, 61B ... depth of field area, 63 ... warning, 65 ... second image, 67 ... third image

Claims (7)

An image processing device (3) configured to display images (57, 65, 67) on display devices (15, 17, 19) of a vehicle (2),
It is configured to acquire first information representing the position of a target (59) existing around the vehicle and second information representing whether or not the target is a target to be emphasized (59A). an information acquisition unit (35),
an image creation unit (37) configured to create an image representing the target at the location represented by the first information;
with
When the second information indicates that at least a part of the target represented in the image is the target to be emphasized, the image creation unit may determine that the target to be emphasized is within the depth of field. creating said image (57, 65) represented as
the image creation unit is configured to create the image including the vehicle represented as being in a depth of field and a representation representing behavior of the vehicle;
The image processing device , wherein the display representing the behavior of the vehicle is a display in which a stop lamp or a turn lamp of the vehicle lights up or blinks . The image processing device according to claim 1,
An image processing device, wherein the image creation unit is configured to display a warning (63) on portions of the image that are represented as being outside the depth of field. The image processing device according to claim 1 or 2 ,
In the image, when the object to be emphasized has changed, the image creation unit changes the image so that the object to be emphasized after the change is represented in the depth of field. Device. The image processing device according to any one of claims 1 to 3 ,
the second information further represents the priority of the target to be emphasized;
The image processing device, wherein the image creation unit is configured to represent the target to be emphasized in the image so as to be more in focus as the priority is higher. The image processing device according to claim 4 ,
When the priority of the target object to be emphasized represented in the image is changed, the image creation unit changes the expression of the target object to be emphasized to an expression corresponding to the changed priority. processing equipment. A program that causes a computer to function as an image processing device (3),
The image processing device is
so as to obtain first information representing the position of a target (59) existing around the vehicle (2) and second information representing whether or not the target is a target to be emphasized (59A) a configured information acquisition unit (35);
an image creation unit (37) configured to create an image (57, 65, 67) representing the target at the location represented by the first information;
with
When the second information indicates that at least a part of the target represented in the image is the target to be emphasized, the image creation unit may determine that the target to be emphasized is within the depth of field. creating said image (57, 65) represented as
the image creation unit is configured to create the image including the vehicle represented as being in a depth of field and a representation representing behavior of the vehicle;
The display representing the behavior of the vehicle is a display in which a stop lamp or a turn lamp of the vehicle lights up or blinks.
program. An image processing system (41, 3) comprising an information providing device (41) and an image processing device (3),
The information providing device includes first information representing the position of a target (59) existing around the vehicle (2), and second information representing whether or not the target is a target to be emphasized (59A). and configured to provide
The image processing device is
an information acquisition unit (35) configured to acquire said first information and said second information from said information providing device;
an image creation unit (37) configured to create an image (57, 65, 67) representing the target at the location represented by the first information;
with
When the second information indicates that at least a part of the target represented in the image is the target to be emphasized, the image creation unit may determine that the target to be emphasized is within the depth of field. creating said image (57, 65) represented as
the image creation unit is configured to create the image including the vehicle represented as being in a depth of field and a representation representing behavior of the vehicle;
The display representing the behavior of the vehicle is a display in which a stop lamp or a turn lamp of the vehicle lights up or blinks.
image processing system.

Images