JP5584561B2 - Image processing apparatus, image display system, and image display method - Google Patents

Description

  The present invention relates to a technique for displaying an image on a passenger vehicle.

  In passenger vehicles such as buses, a passenger entrance is usually provided on the side of the vehicle body. On the other hand, since the side area of a passenger vehicle is likely to be a blind spot from a crew member (typically a driver), a technique that allows the crew member to appropriately confirm safety when the passenger gets on and off is desired.

  In order to respond to such a demand, an image display system is provided in which a camera for photographing the vicinity of the entrance / exit is arranged on a bus, and an image obtained by the camera is displayed on a display device provided so as to be visible from a driver's seat. It has been proposed (see, for example, Patent Document 1).

JP 2002-172978 A

  By the way, in passenger vehicles such as buses, in order to increase passenger boarding / exiting efficiency, a plurality of entrances separated from each other are often provided on one side surface. In this case, it is desirable that the crew members can appropriately confirm safety when passengers get on and off at all of the plurality of entrances.

  In order to cope with this, it is conceivable to simply provide a camera at each of a plurality of entrances. However, in this case, the number of cameras required for the image display system increases, and the cost of the image display system increases.

  It is also conceivable to provide a wide-angle camera that can capture the entire lateral region of the passenger vehicle at once, and display a captured image obtained by this wide-angle camera on a display device. However, in this case, since the state of a wide range of the entire side area is displayed on the display device, the range that the crew should check for safety becomes wide, and there is a possibility that the crew's attention may be distracted. .

  The present invention has been made in view of the above-described problems, and provides a technology that allows a crew member to appropriately confirm the safety of passenger boarding / exiting in a passenger vehicle having a plurality of entrances separated from each other on one side surface. The purpose is to do.

  In order to solve the above-mentioned problem, the invention of claim 1 is an image processing device for generating a display image to be displayed on a display device mounted on a vehicle, wherein the vehicle is provided with a plurality of boarding / exiting devices arranged apart from each other. A passenger vehicle having a mouth on one side, an acquisition means for acquiring a photographed image obtained by a camera capable of photographing a range including the vicinity of all of the plurality of entrances and exits; A generating unit configured to generate a plurality of partial images each indicating the vicinity of the mouth; and an output unit configured to output and display a display image including the plurality of partial images side by side on the display device.

  The invention of claim 2 is an image display system that is mounted on a vehicle and displays a display image, the vehicle being a passenger vehicle having a plurality of entrances spaced apart from each other on one side surface. And an acquisition means for acquiring a captured image obtained by a camera capable of capturing a range including the vicinity of all of the plurality of entrances, and a plurality of partial images respectively indicating the vicinity of the plurality of entrances from the captured image. Generating means for generating the image, and display means for displaying a display image including the plurality of partial images arranged side by side.

Further, the invention of claim 3 is an image display method for displaying a display image in a vehicle, wherein the vehicle is a passenger vehicle having a plurality of entrances arranged at a distance from each other on one side surface, Acquiring a captured image obtained by a camera capable of capturing a range including the vicinity of all of the plurality of entrances; and generating a plurality of partial images respectively indicating the vicinity of the plurality of entrances from the captured image. And a step of displaying a display image including the plurality of partial images side by side.
According to a fourth aspect of the present invention, in the image processing apparatus according to the first aspect of the present invention, the image processing apparatus further includes a detection device that detects an object existing in the vicinity of the plurality of entrances and exits, and the output unit includes the plurality of portions. Of the image, only a partial image corresponding to the entrance / exit where the object is detected is displayed.
Further, the invention of claim 5 is the image processing apparatus according to claim 1, further comprising a composite image generation unit that generates an overhead image that is a composite image overlooking the passenger vehicle from directly above. The plurality of partial images and the overhead image are displayed together.

  According to the first to third aspects of the present invention, a plurality of partial images each showing the vicinity of the plurality of entrances and exits are generated from the captured image of the camera, and a display image including the plurality of partial images arranged is displayed on the display device. For this reason, the crew member can direct attention to the vicinity of each of the plurality of entrances and exits, and can appropriately check the safety of passengers getting on and off.

FIG. 1 is a block diagram showing the configuration of the image display system. FIG. 2 is a diagram illustrating a passenger vehicle as viewed from above. FIG. 3 is a diagram for explaining a method of generating a composite image. FIG. 4 is a diagram illustrating transition of operation modes of the image display system. FIG. 5 is a diagram illustrating transition of the position of the virtual viewpoint in the surrounding confirmation mode. FIG. 6 is a diagram illustrating an example of a composite image generated in the surrounding confirmation mode. FIG. 7 is a diagram illustrating an example of a composite image generated in the surrounding confirmation mode. FIG. 8 is a diagram illustrating an example of a display image displayed in the back mode. FIG. 9 is a diagram illustrating display mode transition in the front mode. FIG. 10 is a diagram illustrating an example of a display image displayed in the overhead view mode. FIG. 11 is a diagram illustrating an example of a display image displayed in the getting-on / off mode. FIG. 12 is a diagram illustrating a region corresponding to a partial image displayed in the getting-on / off mode. FIG. 13 is a diagram illustrating a flow of processing for generating a display image in the getting-on / off mode. FIG. 14 is a diagram for explaining a region to be cut out in the captured image of the left side camera. FIG. 15 is a diagram illustrating an example of a display image displayed in the side mirror mode.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<1. System configuration>
FIG. 1 is a block diagram illustrating a configuration of an image display system 120 according to the present embodiment. The image display system 120 is mounted on a passenger vehicle that transports a large number of passengers, and has a function of generating an image showing the state of the periphery of the passenger vehicle and displaying it in the passenger compartment. Hereinafter, the passenger vehicle will be described as a bus. A crew member (typically a driver) who is a user of the image display system 120 can grasp the state of the periphery of the passenger vehicle almost in real time by using the image display system 120.

  As shown in FIG. 1, the image display system 120 mainly includes an image processing device 100 that generates a display image showing a state around a passenger vehicle, and a display device 20 that displays various types of information to crew members. Yes.

  The display device 20 includes a liquid crystal display and the like, and displays various display images showing the state of the periphery of the passenger vehicle generated by the image processing device 100. The display device 20 is disposed at a position that is visible from the driver's seat 90 of the passenger vehicle (see FIG. 2).

  The image processing apparatus 100 includes a main body unit 10 that is an ECU (Electronic Control Unit) having a function of generating an image, a photographing unit 5 that photographs the periphery of a passenger vehicle, and a changeover switch 6 that can be operated by a crew member. Yes. The main body 10 generates a display image to be displayed on the display device 20 based on a captured image obtained by photographing the periphery of the passenger vehicle with the photographing unit 5. The main body 10 is arranged at a predetermined position of a passenger vehicle different from the display device 20.

  The photographing unit 5 is electrically connected to the main body unit 10 and operates based on a signal from the main body unit 10. The photographing unit 5 includes a plurality of in-vehicle cameras, specifically, a front camera 51, a left side camera 52, a right side camera 53, and a back camera 54. Each of the in-vehicle cameras 51 to 54 includes a lens and an image sensor and electronically acquires an image. These four vehicle-mounted cameras 51-54 are each arrange | positioned at four surfaces of front, back, left, and right of a passenger vehicle.

  FIG. 2 is a diagram showing the passenger vehicle 9 as viewed from above. As shown in the figure, the passenger vehicle 9 has a long shape in the front-rear direction. The passenger vehicle 9 is a right-hand drive vehicle, and includes two entrances 91 and 92 for passengers to get on and off the left side surface 9L opposite to the driver's seat 90. Each of the entrances 91 and 92 is provided with a door, which is opened and closed according to an instruction from the crew.

  These two entrances 91 and 92 are spaced apart from each other. The front entrance / exit 91 is disposed at substantially the same position as the driver's seat 90 in the front-rear direction of the passenger vehicle 9. On the other hand, the rear entrance 92 is disposed between the center in the front-rear direction of the passenger vehicle 9 and the rear tire. When a passenger gets on and off using these two entrances 91 and 92, the passenger is present in the side area on the left side of the passenger vehicle 9. However, this left side region is difficult to confirm directly from the driver's seat 90. In particular, the area near the rear entrance / exit 92 is likely to be a blind spot because it is away from the driver's seat 90.

  The front camera 51 is provided at the center upper portion of the front surface 9F of the passenger vehicle 9 in the left-right direction. The optical axis 51a of the front camera 51 is directed in the straight direction of the passenger vehicle 9 in plan view. The back camera 54 is provided at the upper center in the left-right direction on the rear surface 9 </ b> B of the passenger vehicle 9. The optical axis 54a of the back camera 54 is directed in the direction opposite to the straight traveling direction of the passenger vehicle 9 in plan view.

  Further, the left side camera 52 is provided at the upper center in the front-rear direction on the left side surface 9L of the passenger vehicle 9. The optical axis 52a of the left side camera 52 is directed to the left side of the passenger vehicle 9 along the left-right direction of the passenger vehicle 9 in plan view. Further, the right side camera 53 is provided at the upper center in the front-rear direction on the right side surface 9 </ b> R of the passenger vehicle 9. The optical axis 53a of the right side camera 53 is directed to the right side of the passenger vehicle 9 along the left-right direction of the passenger vehicle 9 in plan view.

  Fish-eye lenses are employed as the lenses of these in-vehicle cameras 51 to 54, and each of the in-vehicle cameras 51 to 54 is a wide-angle camera having an angle of view θ of 180 degrees or more. For this reason, it is possible to capture the entire periphery of the passenger vehicle 9 by using these four in-vehicle cameras 51 to 54.

  The left side camera 52 can photograph the entire side region on the left side of the passenger vehicle 9 including the range from the front end to the rear end of the passenger vehicle 9. Therefore, the left side camera 52 can capture a range including both the vicinity of the front entrance / exit 91 and the vicinity of the rear entrance 92 at a time.

  Returning to FIG. 1, the changeover switch 6 receives an instruction to change the display content of the display device 20 from the crew. The changeover switch 6 is disposed in the vicinity of the driver's seat 90 separately from the main body 10 so that the crew can easily operate.

  The main body 10 of the image processing apparatus 100 includes a control unit 1 that controls the entire apparatus, an image acquisition unit 41 that acquires a captured image from the imaging unit 5, and an image generation unit 3 that generates an image based on the captured image. And an image output unit 42 that outputs an image to the display device 20.

  The image acquisition unit 41 receives captured images obtained by shooting from each of the plurality of in-vehicle cameras 51 to 54 included in the shooting unit 5.

  The image generation unit 3 is a hardware circuit that can perform various types of image processing. The image generation unit 3 processes a captured image acquired by the image acquisition unit 41 and generates a display image used for display on the display device 20. The image generation unit 3 includes a captured image cutout unit 31, a composite image generation unit 32, and a display image generation unit 33 as main functions.

  The captured image cutout unit 31 cuts out a partial area of the captured image acquired by the shooting unit 5 and generates a partial image using the cutout area as one image. The photographed image cutout unit 31 can cut out a plurality of areas included in one photographed image and make them partial images. The captured image cutout unit 31 can also perform image processing such as distortion correction and enlargement / reduction on the generated partial image.

  The composite image generation unit 32 generates a composite image indicating a region around the passenger vehicle 9 viewed from an arbitrary virtual viewpoint based on the plurality of captured images acquired by the plurality of on-vehicle cameras 51 to 54 of the imaging unit 5. . A method by which the composite image generation unit 32 generates a composite image will be described later.

  The display image generation unit 33 generates a display image used for display on the display device 20 using the partial image generated by the captured image cutout unit 31 and the composite image generated by the composite image generation unit 32. The display image includes at least one of a partial image and a composite image.

  The image output unit 42 outputs the display image generated by the image generation unit 3 to the display device 20. Thereby, a display image showing at least a part of the area around the passenger vehicle 9 is displayed on the display device 20.

  The control unit 1 is a computer including a CPU, a RAM, a ROM, and the like. Various control functions of the control unit 1 are realized by the CPU performing arithmetic processing according to a predetermined program. For example, the control unit 1 performs control to change the content of the display image displayed on the display device 20 in accordance with the change of the operation mode of the image display system 120. At this time, the control unit 1 instructs the image generation unit 3 on the type of image to be generated, parameters necessary for image generation, and the like.

  Further, when the changeover switch 6 is operated by the crew, a predetermined signal is input to the control unit 1. Based on this signal, the control unit 1 can perform control according to the instruction of the crew.

  The main body 10 of the image processing apparatus 100 further includes a nonvolatile memory 40, a signal input unit 43, and a card reading unit 44, which are connected to the control unit 1.

  The nonvolatile memory 40 is a flash memory or the like that can maintain stored contents even when the power is turned off. The nonvolatile memory 40 stores vehicle data 4a together with a program (firmware) that realizes the function of the control unit 1. The vehicle data 4a indicates information unique to the passenger vehicle 9 such as the vehicle body size.

  The signal input unit 43 receives signals from various devices provided in the passenger vehicle 9. A signal from the outside of the image display system 120 is input to the control unit 1 via the signal input unit 43. In the present embodiment, a signal from the shift sensor 81 is input to the control unit 1 via the signal input unit 43. From the shift sensor 81, the operation position of the shift lever of the transmission of the passenger vehicle 9, ie, “P (parking)”, “D (forward)”, “N (neutral)”, “R (reverse)”, etc. The shift position is input.

  The card reading unit 44 reads a memory card MC that is a portable recording medium. The card reading unit 44 includes a card slot in which the memory card MC can be attached and detached, and reads data recorded on the memory card MC installed in the card slot. Data read by the card reading unit 44 is input to the control unit 1. The memory card MC is a flash memory or the like capable of storing various data, and the image display system 120 can use various data stored in the memory card MC. For example, it is possible to update a program (firmware) that realizes the function of the control unit 1 by reading a program stored in the memory card MC.

<2. Image composition processing>
Next, a method in which the composite image generation unit 32 of the image generation unit 3 generates a composite image that shows how the periphery of the passenger vehicle 9 is viewed from an arbitrary virtual viewpoint based on the plurality of captured images obtained by the imaging unit 5. Will be described. FIG. 3 is a diagram for explaining a method of generating a composite image.

  When the front camera 51, the left side camera 52, the right side camera 53, and the back camera 54 of the photographing unit 5 are simultaneously photographed, four photographed images P1 showing the front, left side, right side, and rear of the passenger vehicle 9, respectively. ~ P4 is acquired. That is, the four captured images P1 to P4 acquired by the imaging unit 5 include information indicating the entire periphery of the passenger vehicle 9 at the time of shooting.

  Next, each pixel of the four captured images P1 to P4 is projected onto a three-dimensional curved surface TS in a virtual three-dimensional space. The three-dimensional curved surface TS has, for example, a substantially hemispherical shape (a bowl shape), and a center portion (a bottom portion of the bowl) is defined as a position where the passenger vehicle 9 exists. A correspondence relationship is determined in advance between the positions of the pixels included in the captured images P1 to P4 and the positions of the pixels of the solid curved surface TS. For this reason, the value of each pixel of the solid curved surface TS can be determined based on this correspondence and the value of each pixel included in the captured images P1 to P4.

  The correspondence between the positions of the pixels of the captured images P1 to P4 and the positions of the pixels of the three-dimensional curved surface TS is determined by the arrangement of the four in-vehicle cameras 51 to 53 in the passenger vehicle 9 (inter-distance, ground height, optical axis angle). Etc.) The table data indicating the correspondence relationship is included in the vehicle data 4a stored in the nonvolatile memory 40.

  Further, the shape and size of the vehicle body included in the vehicle data 4a are used, and a polygon model indicating the three-dimensional shape of the passenger vehicle 9 is virtually constructed. The configured model of the passenger vehicle 9 is arranged in a substantially hemispherical center portion defined as the position of the passenger vehicle 9 in the three-dimensional space in which the three-dimensional curved surface TS is set.

  Furthermore, the virtual viewpoint VP is set by the control unit 1 for the three-dimensional space where the three-dimensional curved surface TS exists. The virtual viewpoint VP is defined by the viewpoint position and the viewing direction, and is set at an arbitrary viewpoint position corresponding to the vicinity of the passenger vehicle 9 in this three-dimensional space in an arbitrary viewing direction.

  Then, according to the set virtual viewpoint VP, a necessary area on the three-dimensional curved surface TS is cut out as an image. The relationship between the virtual viewpoint VP and a necessary area in the three-dimensional curved surface TS is determined in advance, and is stored in advance in the nonvolatile memory 40 or the like as table data. On the other hand, rendering is performed with respect to the polygon model in accordance with the set virtual viewpoint VP, and the resulting image of the two-dimensional passenger vehicle 9 is superimposed on the cut out image. Thereby, the composite image which shows a mode that the passenger vehicle 9 and the periphery of the passenger vehicle 9 were seen from arbitrary virtual viewpoints VP will be produced | generated.

  For example, when a virtual viewpoint VPa is set in which the viewpoint position is directly above the center of the passenger vehicle 9 and the visual field direction is directly below, the passenger vehicle 9 is looked down at the passenger vehicle 9 from directly above the passenger vehicle 9. 9 and a composite image CPa indicating the area around the passenger vehicle 9 is generated. Further, as shown in the figure, when a virtual viewpoint VPb is set in which the viewpoint position is the left rear of the position of the passenger vehicle 9 and the visual field direction is the substantially forward direction of the passenger vehicle 9, the passenger vehicle 9 is viewed from the left rear. A composite image CPb indicating the passenger vehicle 9 and a region around the passenger vehicle 9 is generated so as to overlook the entire periphery.

  In the case of actually generating a composite image, it is not necessary to determine the values of all the pixels of the three-dimensional curved surface TS, and only the values of the pixels in the area necessary corresponding to the set virtual viewpoint VP are photographed. By determining based on the images P1 to P4, the processing speed can be improved.

  In the image display system 120, by using such a function of the composite image generation unit 32, a composite image viewed from an arbitrary viewpoint around the passenger vehicle 9 can be generated and displayed on the display device 20.

<3. Operation mode>
Next, the operation mode of the image display system 120 will be described. FIG. 4 is a diagram illustrating transition of operation modes of the image display system 120. The image display system 120 has three operation modes: a surrounding confirmation mode M1, a front mode M2, and a back mode M3. These operation modes can be switched by the control of the control unit 1 according to the operation of the crew and the traveling state of the passenger vehicle 9.

  The surrounding confirmation mode M <b> 1 is an operation mode in which an animation is performed so as to go around the passenger vehicle 9 while looking down at the passenger vehicle 9. The front mode M2 is an operation mode for displaying a display image mainly showing the front and side of the passenger vehicle 9 that is required when moving forward. Further, the back mode M3 is an operation mode for displaying a display image mainly showing the rear of the passenger vehicle 9 that is required at the time of reverse.

  When the image display system 120 is activated, it first enters the front mode M2. In the case of the front mode M2, when the changeover switch 6 is continuously pressed for a predetermined time (for example, 1 second) or more, the surrounding confirmation mode M1 is switched. When switched to the surrounding confirmation mode M1, an animation expression is made so as to go around the passenger vehicle 9. When a predetermined time (for example, 6 seconds) has elapsed after the end of the animation expression, the mode is automatically switched to the front mode M2.

  In the case of the front mode M2, when the position of the shift lever input from the shift sensor 81 becomes “R (reverse)”, the mode is switched to the back mode M3. That is, when the speed change device of the passenger vehicle 9 is operated to the “R (reverse)” position, the passenger vehicle 9 is in a reverse state, so that it can be switched to the back mode M3 mainly showing the rear of the passenger vehicle 9. . In the case of the back mode M3, when the position of the shift lever is other than “R (reverse)”, the mode is switched to the front mode M2.

  Hereinafter, display modes on the display device 20 in each of the surrounding confirmation mode M1, the back mode M3, and the front mode M2 will be described.

<4. Surrounding confirmation mode>
First, the display mode of the display device 20 in the surrounding confirmation mode M1 will be described. In the surrounding confirmation mode M1, a composite image showing the entire periphery of the passenger vehicle 9 is displayed. Then, the virtual viewpoint VP for the composite image is changed stepwise and continuously, and animation is expressed.

  Specifically, the virtual viewpoint VP is set so as to look down at the passenger vehicle 9, and the virtual viewpoint VP is continuously moved so as to go around the passenger vehicle 9 as shown in FIG. The virtual viewpoint VP is initially set behind the passenger vehicle 9 and then circulates around the passenger vehicle 9 in the clockwise direction. When the virtual viewpoint VP moves to the rear again through the left side, the front side, and the right side of the passenger vehicle 9 in this way, the virtual viewpoint VP moves to just above the passenger vehicle 9. In this manner, a plurality of synthesized images are continuously generated with the virtual viewpoint VP being moved. For example, as shown in the composite image CP11 shown in FIG. 6, the composite image CP12 shown in FIG. 7, etc., a composite image in which the entire periphery of the passenger vehicle 9 is viewed from the viewpoint of looking down at the passenger vehicle 9 is generated.

  The plurality of generated composite images are sequentially output from the image output unit 42 as they are as display images. The display device 20 displays the images continuously in time. As a result, an animation expression is made such that the passenger vehicle 9 circulates around the passenger vehicle 9 while looking down at the passenger vehicle 9.

  A crew member can visually confirm the situation of the entire circumference of the passenger vehicle 9 from the viewpoint of looking down at the passenger vehicle 9 by visually recognizing such an animation expression, and intuitively check the obstacles and passengers around the passenger vehicle 9. The positional relationship with the vehicle 9 can be grasped.

<5. Back mode>
Next, the display mode of the display device 20 in the back mode M3 will be described. FIG. 8 is a diagram illustrating an example of the display image DP3 displayed on the display device 20 in the back mode M3.

  The display image DP3 includes a bird's-eye view image BP1 that is a composite image obtained by bird's-eye view of the passenger vehicle 9, a partial image BP2 obtained by cutting out a partial area of the image captured by the back camera 54, and a field-of-view icon C3 that indicates a field-of-view range. Includes side by side.

  Since the two images BP1 and BP2 can be viewed in the back mode M3, the crew can check the situation around the passenger vehicle 9 in the traveling direction together with the entire periphery of the passenger vehicle 9 at a glance. .

<6. Front mode>
Next, the display mode of the display device 20 in the front mode M2 will be described. In the front mode M2, there are three display modes having different display modes, specifically, an overhead view mode, an getting-on / off mode, and a side mirror mode. The contents included in the display image displayed on the display device 20 are different for each display mode.

  As shown in FIG. 9, these display modes are switched by the control of the control unit 1 in the order of the overhead view mode M21, the getting on / off mode M22, and the side mirror mode M23 each time the crew member presses the changeover switch 6. When the changeover switch 6 is pressed in the side mirror mode M23, the operation returns to the overhead view mode M21 again.

<6-1. Overhead mode>
The bird's-eye view mode M21 is a display mode that is generally used when moving forward. FIG. 10 is a diagram illustrating an example of the display image DP21 displayed on the display device 20 in the overhead view mode M21.

  The display image DP21 includes a bird's-eye view image FP1 that is a composite image obtained by bird's-eye view of the passenger vehicle 9, a partial image FP2 obtained by cutting out a partial area of the image captured by the front camera 51, and a field-of-view icon C21 that indicates a field-of-view range. Includes side by side.

  In the bird's-eye view mode M21, since such two images FP1 and FP2 can be viewed, the crew can check the situation in front of the passenger vehicle 9 in the traveling direction together with the entire periphery of the passenger vehicle 9 at a glance. .

<6-2. Boarding mode>
The boarding / alighting mode M22 is a display mode that is used by the crew to confirm the safety of passengers getting on and off. FIG. 11 is a diagram illustrating an example of the display image DP21 displayed on the display device 20 in the getting-on / off mode M22.

  The display image DP22 includes a first partial image SP1 obtained by cutting out a partial area of the image captured by the left side camera 52, and a second partial image SP2 obtained by extracting another partial area of the image captured by the left side camera 52. And a visual field icon C22 indicating the visual field range.

  As shown in FIG. 12, the first partial image SP1 shows an area SA1 in the vicinity of the front entrance / exit 91. On the other hand, the second partial image SP2 shows a region SA2 in the vicinity of the rear entrance 92.

  In the boarding / alighting mode M22, such two partial images SP1 and SP2 can be viewed, so that the crew can confirm the situation in the vicinity of both the front boarding gate 91 and the rear boarding port 92 at a glance. In the display device 20, only the area in the vicinity of each of the two entrances 91 and 92 is displayed instead of the entire left side area of the passenger vehicle 9. For this reason, the crew can appropriately pay attention to the vicinity of each of the two entrances 91 and 92, which are areas to be confirmed, without being distracted. As a result, the crew can appropriately confirm the safety of passengers getting on and off.

  The two partial images SP1 and SP2 included in the display image DP22 in the getting-on / off mode M22 are generated by being cut out from the same captured image obtained by the left side camera 52.

  FIG. 13 is a diagram showing a flow of processing for generating the display image DP22 in the getting-on / off mode M22.

  First, the image acquisition unit 41 acquires a captured image obtained by one shooting of the left side camera 52 from the left side camera 52. This captured image shows both the area SA1 near the front entrance / exit 91 and the area SA2 near the back entrance 92 (step S11).

  Next, as shown in FIG. 14, the captured image cutout unit 31 cuts out a region A1 corresponding to the region SA1 in the vicinity of the front entrance / exit 91 from the captured image P2 of the left side camera 52, thereby An image SP1 is generated. Further, the photographed image cutout unit 31 cuts out a region A2 corresponding to the region SA2 in the vicinity of the rear entrance 92 from the photographed image P2 of the same left side camera 52 to generate the second partial image SP2 (step S2). S12).

  By the way, in the photographed image acquired using the fisheye lens, the image of the subject is shown in a compressed state in the peripheral area. For this reason, the size shown in the captured image P2 is different between the area SA1 near the front entrance / exit 91 and the area SA2 near the back entrance 92. Specifically, the area SA1 in the vicinity of the front entrance / exit 91 that is separated from the position of the left side camera 52 is shown smaller than the area SA2 in the vicinity of the rear entrance / exit 92. For this reason, the area A1 indicating the area SA1 near the front entrance / exit 91 is cut out smaller than the area A2 indicating the area SA2 near the rear entrance / exit 92. Even if the sizes of the two areas A1 and A2 thus cut out are different, the sizes of the two areas SA1 and SA2 that are actual subjects corresponding to the two areas A1 and A2 are the same.

  Next, the display image generation unit 33 generates a display image DP22 (see FIG. 11) including the first partial image SP1 and the second partial image SP2 side by side (step S13). The generated display image DP22 is output from the image output unit 42 to the display device 20. Thereby, the display image DP22 is displayed on the display device 20, and the first partial image SP1 and the second partial image SP2 are displayed on the same screen (step S14).

  The processing shown in FIG. 13 is repeatedly executed at a predetermined cycle (for example, 1/60 second) under the control of the control unit 1. Thereby, in the display apparatus 20, the situation of the vicinity of both the front entrance / exit 91 and the back entrance 92 is shown in substantially real time.

  As described above, in the image display system 120, the partial image SP <b> 1 showing the area SA <b> 1 near the front entrance / exit 91 and the partial image SP <b> 2 showing the area SA <b> 2 near the rear entrance 92 are one on-vehicle camera 52. It is generated by cutting out from the obtained photographed image. For this reason, the state of the vicinity of both the front entrance / exit 91 and the rear entrance / exit 92 can be shown to a crew member, without arrange | positioning a vehicle-mounted camera in each of the front entrance / exit 91 and the rear entrance / exit 92.

<6-3. Side mirror mode>
The side mirror mode M23 is a display mode for displaying a range that is substantially the same as the range reflected on the side mirror of the passenger vehicle 9. FIG. 15 is a diagram illustrating an example of the display image DP23 displayed on the display device 20 in the side mirror mode M23.

  The display image DP23 includes a partial image MP1 obtained by cutting out a partial area of the photographed image of the left side camera 52, a partial image MP2 obtained by cutting out a partial area of the photographed image of the right side camera 53, and a visual field indicating a visual field range. It is included side by side with the icon C23. The partial image MP1 shows a range substantially similar to the range shown on the left side mirror. On the other hand, the partial image MP2 shows a range substantially the same as the range shown on the right side mirror.

  In the side mirror mode M23, such two images MP1 and MP2 can be viewed, so that the crew can check the display device 20 in a range that is substantially the same as the range shown in the side mirror.

<7. Modification>
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications are possible. Below, such a modification is demonstrated. All forms including those described in the above embodiment and those described below can be combined as appropriate.

  For example, in the image display system 120, a detection device that detects an object such as a clearance sonar is provided, and an object that exists in the vicinity of the two entrances 91 and 92, or an approaching object that approaches the two entrances 91 and 92. May be detected. And the display position of 1st partial image SP1 and 2nd partial image SP2 may be fixed, and you may make it display only the partial image corresponding to the entrance / exit 91,92 which detected the object and the approaching object. In this way, the crew can instantly grasp which of the entrances 91 and 92 is near the object or approaching object.

  In the above embodiment, the display mode is switched to the boarding / alighting mode M22 by operating the changeover switch 6 by the crew member. However, when the passenger vehicle stops at a predetermined stop such as a bus stop, You may make it switch to boarding / alighting mode M22 without.

  The fact that the passenger vehicle has stopped at the stop can be determined, for example, by communicating between the wireless communication devices provided in each of the passenger vehicle and the stop and based on the communication result. Further, it may be determined that the passenger vehicle has stopped at the stop based on the opening of the door provided at the entrance / exit of the passenger vehicle.

  Further, in response to the departure of the passenger vehicle from the stop, the display mode may be switched from the getting on / off mode M22 to another display mode (for example, the overhead view mode M21). Whether the passenger vehicle has departed from the stop can be determined based on, for example, the traveling speed of the passenger vehicle. Further, when the traveling speed of the passenger vehicle exceeds a predetermined speed (for example, 10 km / h), the display itself on the display device 20 may be stopped in order to concentrate the crew on driving.

  Moreover, in the said embodiment, although 1st partial image SP1 and 2nd partial image SP2 were displayed on the display apparatus 20 in boarding / alighting mode M22, it is a synthesized image which overlooks the passenger vehicle 9 from right above. You may make it display a bird's-eye view image collectively.

  In the above embodiment, a bus is described as an example of a passenger vehicle. However, a passenger vehicle having a plurality of entrances spaced apart from each other, such as a tram or a cable car, on one side. The technology described above can be applied to any passenger vehicle.

  Further, in the above embodiment, the passenger vehicle is provided with two entrances / exits on one side surface. However, even if the passenger vehicle is provided with three or more entrances / exits on one side surface, it is described above. The applied technology is applicable. In any case, the passenger vehicle may be provided with a camera capable of photographing the range including the vicinity of all of the plurality of entrances and exits where the passenger boarding and exiting should be monitored.

  Further, in the above-described embodiment, it has been described that various functions are realized in software by the arithmetic processing of the CPU according to the program. However, some of these functions are realized by an electrical hardware circuit. Also good. Conversely, some of the functions realized by the hardware circuit may be realized by software.

DESCRIPTION OF SYMBOLS 9 Passenger vehicle 20 Display apparatus 31 Captured image cut-out part 33 Display image generation part 41 Image acquisition part 42 Image output part 52 Left side camera 90 Driver's seat 91 Front entrance / exit 92 Rear entrance / exit

Claims (5)

An image processing device that generates a display image to be displayed on a display device mounted on a vehicle,
The vehicle is a passenger vehicle having a plurality of entrances on one side surface that are spaced apart from each other.
An acquisition means for acquiring a captured image obtained by a camera capable of capturing a range including the vicinity of all of the plurality of entrances;
Generating means for generating a plurality of partial images respectively indicating the vicinity of the plurality of entrances and exits from the captured image;
An output means for outputting and displaying a display image including the plurality of partial images side by side on the display device;
An image processing apparatus comprising: An image display system mounted on a vehicle and displaying a display image,
The vehicle is a passenger vehicle having a plurality of entrances on one side surface that are spaced apart from each other.
An acquisition means for acquiring a captured image obtained by a camera capable of capturing a range including the vicinity of all of the plurality of entrances;
Generating means for generating a plurality of partial images respectively indicating the vicinity of the plurality of entrances and exits from the captured image;
Display means for displaying a display image including the plurality of partial images side by side;
An image display system comprising: An image display method for displaying a display image in a vehicle,
The vehicle is a passenger vehicle having a plurality of entrances on one side surface that are spaced apart from each other.
Obtaining a captured image obtained by a camera capable of capturing a range including the vicinity of all of the plurality of entrances;
Generating a plurality of partial images respectively indicating the vicinity of the plurality of entrances and exits from the captured image;
Displaying a display image including the plurality of partial images side by side;
An image display method comprising: The image processing apparatus according to claim 1.
A detection device for detecting an object present in the vicinity of the plurality of entrances;
Further comprising
The output means displays only a partial image corresponding to the entrance / exit where the object is detected among the plurality of partial images. The image processing apparatus according to claim 1.
A composite image generation unit that generates an overhead image that is a composite image of overhead viewing the passenger vehicle from above;
Further comprising
The output unit displays the plurality of partial images and the overhead image together.

Images