JP2021111843A - Video playback device, record playback device, video playback method, and video playback program - Google Patents

Abstract

To check the status of an event properly by using a bird's-eye view video.SOLUTION: A video playback device 200 includes: an information acquisition unit 141 that acquires a plurality of pieces of shooting data taken by a plurality of cameras that shoot surroundings of a vehicle on the basis of an event that has occurred on the vehicle, and information indicating an event detection direction with respect to the vehicle; a bird's-eye view video generation unit 143 that performs viewpoint conversion processing for the plurality of pieces of shooting data acquired by the information acquisition unit 141 so as to make the shoot data look as if viewed from above the vehicle, and changes boundaries for synthesizing a plurality of videos according to the event detection direction to generate a composite bird's-eye view video; and a playback control unit 142 that displays the bird's-eye view video generated by the bird's-eye view video generation unit 143 on a display unit 130.SELECTED DRAWING: Figure 9

Description

The present invention relates to a video reproduction device, a recording / reproduction device, a video reproduction method, and a video reproduction program.

Conventionally, there is known a device that displays or records an image taken by a camera for a bird's-eye view image of a vehicle as an image for a drive recorder (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2014-134912

The bird's-eye view image as described in Patent Document 1 is generated by synthesizing a plurality of images that have undergone viewpoint conversion processing. When synthesizing a plurality of images, the images may be distorted at the boundary of the composition and may not be displayed properly. When an event occurs in a vehicle, depending on the direction in which the event occurs, an image showing the event may be displayed on or near the boundary as described above, and the event status may not be confirmed properly. ..

The present invention has been made in view of the above, and provides a video playback device, a recording / playback device, a video playback method, and a video playback program capable of appropriately confirming the status of an event using a bird's-eye view video. The purpose.

The video reproduction device according to the present invention acquires a plurality of shooting data taken by a plurality of cameras that shoot the surroundings of the vehicle based on an event generated for the vehicle, and information indicating an event detection direction for the vehicle. The information acquisition unit and the plurality of shooting data acquired by the information acquisition unit are subjected to viewpoint conversion processing as seen from above the vehicle, and the boundary where the plurality of images are combined is set according to the event detection direction. It includes a bird's-eye view image generation unit that generates a bird's-eye view image that has been changed and synthesized, and a playback control unit that displays the bird's-eye view image generated by the bird's-eye view image generation unit on the display unit.

The recording / playback device according to the present invention includes an event detection unit that detects an event for the vehicle based on the acceleration applied to the vehicle, and a plurality of cameras that photograph the surroundings of the vehicle based on the event detected by the event detection unit. A plurality of shooting data taken, an information acquisition unit that acquires information indicating an event detection direction for the vehicle, and a plurality of shootings for a predetermined period regarding the detected event when the event detection unit detects an event for the vehicle. The recording control unit that records the data and the plurality of shooting data recorded by the recording control unit are subjected to viewpoint conversion processing as seen from above the vehicle, and the boundary at which the plurality of images are combined is detected as an event. It includes a bird's-eye view image generation unit that generates a bird's-eye view image that is changed according to the direction and synthesized, and a playback control unit that displays the bird's-eye view image generated by the bird's-eye view image generation unit on the display unit.

The video reproduction method according to the present invention acquires a plurality of shooting data taken by a plurality of cameras that shoot the surroundings of the vehicle based on an event generated for the vehicle, and information indicating an event detection direction for the vehicle. The information acquisition step to be performed and the plurality of shooting data acquired in the information acquisition step are subjected to viewpoint conversion processing as seen from above the vehicle, and the boundary for synthesizing the plurality of images is determined according to the event detection direction. It includes a bird's-eye view image generation step of generating a bird's-eye view image that has been changed and synthesized, and a playback control step of displaying the bird's-eye view image generated in the bird's-eye view image generation step on the display unit.

The video reproduction program according to the present invention acquires a plurality of shooting data taken by a plurality of cameras that shoot the surroundings of the vehicle based on an event generated for the vehicle, and information indicating an event detection direction for the vehicle. The information acquisition step to be performed and the plurality of shooting data acquired in the information acquisition step are subjected to viewpoint conversion processing as seen from above the vehicle, and the boundary for synthesizing the plurality of images is determined according to the event detection direction. The computer is made to execute a bird's-eye view image generation step of generating a bird's-eye view image that has been changed and synthesized, and a playback control step of displaying the bird's-eye view image generated in the bird's-eye view image generation step on the display unit.

According to the present invention, the situation of the event can be appropriately confirmed by using the bird's-eye view image.

FIG. 1 is a block diagram showing an example of a recording / reproducing device according to the first embodiment of the present invention. FIG. 2 is an explanatory diagram showing an example of a bird's-eye view image. FIG. 3 is a diagram showing an example of a bird's-eye view image in which a part of the composite boundary is changed. FIG. 4 is a diagram showing another example of a bird's-eye view image in which a part of the composite boundary is changed. FIG. 5 is a diagram showing another example of a bird's-eye view image in which a part of the composite boundary is changed. FIG. 6 is a diagram showing another example of a bird's-eye view image in which a part of the composite boundary is changed. FIG. 7 is a flowchart showing a flow of recording processing in the recording / playback apparatus according to the first embodiment. FIG. 8 is a flowchart showing a flow of reproduction processing in the recording / reproduction apparatus according to the first embodiment. FIG. 9 is a block diagram showing an example of the video reproduction device according to the second embodiment.

Hereinafter, embodiments of a video playback device, a recording / playback device, a video playback method, and a video playback program according to the present invention will be described with reference to the drawings. The present invention is not limited to this embodiment. In addition, the components in the following embodiments include those that can be easily replaced by those skilled in the art, or those that are substantially the same.

[First Embodiment]
The configuration of the recording / reproducing device according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing an example of the configuration of a recording / reproducing device according to each embodiment of the present invention. The recording / reproducing device 100 shown in FIG. 1 is a so-called drive recorder, and may be a portable device that can be used in the vehicle in addition to the device installed in the vehicle. Further, the recording / reproducing device 100 may be realized by sharing the configuration with, for example, a navigation device, an infotainment system, or the like provided in the vehicle.

The recording / reproducing device 100 includes a photographing unit 10, a recording unit 20, an operation unit 30, a display unit 40, an acceleration sensor 50, a GPS (Global Positioning System) receiving unit 60, and a control unit 70.

The photographing unit 10 includes a front camera 11, a rear camera 12, a right side camera 13, and a left side camera 14. The front camera 11 is arranged at the front of the vehicle and photographs the surroundings centered on the front of the vehicle. The rear camera 12 is arranged at the rear of the vehicle and photographs the surroundings centered on the rear of the vehicle. The right-side camera 13 is arranged on the right side of the vehicle and photographs the surroundings centered on the right side of the vehicle. The left-side camera 14 is arranged on the left side of the vehicle and photographs the periphery centered on the left side of the vehicle.

The front camera 11, the rear camera 12, the right side camera 13, and the left side camera 14 output the captured video as shooting data to the shooting data acquisition unit 71 of the control unit 70. The photographing unit 10 photographs all directions of the vehicle by the front camera 11, the rear camera 12, the right side camera 13, and the left side camera 14.

The recording unit 20 stores various data such as event data recorded when an event is detected. The recording unit 20 stores, for example, shooting data for a predetermined period based on the detection of an event as event data. The recording unit 20 can be realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or a solid state drive. The recording unit 20 may be composed of a plurality of different memories or the like.

The operation unit 30 receives various operations on the control unit 70. Various operations include operations such as starting playback of event data and starting saving of event data. The operation unit 30 outputs an operation signal corresponding to the received operation to the operation control unit 77. The operation unit 30 can be realized by, for example, a physical switch or a touch panel provided on the display unit 40.

The display unit 40 displays various images. The display unit 40 displays, for example, an image in which the event data recorded by the recording unit 20 is reproduced. The display unit 40 is a display including, for example, a liquid crystal display (LCD) or an organic EL (Organic Electro-Luminescence) display.

The acceleration sensor 50 detects the acceleration applied to the vehicle. The acceleration sensor 50 detects, for example, the acceleration applied to the vehicle while parking. The acceleration sensor 50 can be realized by, for example, a three-axis acceleration sensor. The acceleration sensor 50 outputs acceleration information regarding the detected acceleration to the acceleration information acquisition unit 79.

The GPS receiving unit 60 is composed of a GPS receiving circuit, a GPS receiving antenna, and the like, and receives GPS signals. The GPS receiving unit 60 outputs the received GPS signal to the position information acquisition unit 82.

In the control unit 70, for example, a program (for example, a program according to the present invention) stored in a storage unit (not shown) is executed by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like with the RAM or the like as a work area. It is realized by. Further, the control unit 70 is a controller, and may be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The control unit 70 includes a shooting data acquisition unit (information acquisition unit) 71, a buffer memory 72, a shooting data processing unit 73, a recording control unit 74, a playback control unit 75, a bird's-eye view image generation unit 76, and operation control. It has a unit 77, a display control unit 78, an acceleration information acquisition unit 79, an event detection unit 80, an event detection direction acquisition unit (information acquisition unit) 81, and a position information acquisition unit 82. In FIG. 1, each unit realized by the control unit 70 is described as being connected to each other via a bus 83 for easy understanding.

The shooting data acquisition unit 71 acquires various shooting data from the outside. The shooting data acquisition unit 71 acquires shooting data from, for example, the shooting unit 10. The shooting data acquisition unit 71 acquires, for example, shooting data obtained by shooting the periphery of a vehicle that is running, stopped, or parked by the shooting unit 10. The shooting data acquisition unit 71 outputs, for example, the shooting data acquired from the shooting unit 10 to the buffer memory 72 and the display control unit 78. The shooting data acquisition unit 71 may acquire shooting data including audio data acquired by the shooting unit 10 or a microphone (not shown) arranged at another position in addition to the shooting data shot by the shooting unit 10. The shooting data acquisition unit 71 outputs a plurality of acquired shooting data to the buffer memory 72.

The buffer memory 72 is an internal memory of the control unit 70 that temporarily stores the shooting data acquired by the shooting data acquisition unit 71. Specifically, the buffer memory 72 temporarily stores the shooting data for a certain period of time acquired by the shooting data acquisition unit 71 while updating it.

The shooting data processing unit 73 executes various processes on the shooting data temporarily stored in the buffer memory 72. The shooting data acquisition unit 71 converts, for example, the shooting data temporarily stored in the buffer memory 72 into an arbitrary file format such as the MP4 format. The shooting data processing unit 73 generates, for example, shooting data as a data file for a certain period of time from the shooting data temporarily stored in the buffer memory 72. Specifically, the shooting data processing unit 73 generates shooting data for 60 seconds as a data file from the shooting data temporarily stored in the buffer memory 72. The shooting data processing unit 73 outputs the generated shooting data to the recording control unit 74. Further, the shooting data processing unit 73 outputs the generated shooting data to the display control unit 78. The period of the shooting data generated as a data file is set to 60 seconds as an example, but the period is not limited to this.

The recording control unit 74 stores various data in the recording unit 20. The recording control unit 74 controls, for example, to have the recording unit 20 record the shooting data filed by the shooting data processing unit 73. For example, when the event detection unit 80 detects an event, the recording control unit 74 stores the shooting data related to the video including the event in the recording unit 20 as event data.

The reproduction control unit 75 reproduces various data recorded in the recording unit 20. The reproduction control unit 75 reproduces, for example, the shooting data recorded as a data file in the recording unit 20. The reproduction control unit 75 reproduces, for example, the event data stored in the recording unit 20. The reproduction control unit 75 reproduces arbitrary shooting data and event data according to the control signal corresponding to the operation of the operation unit 30 output from the operation control unit 77. When the reproduction control unit 75 reproduces the shooting data and the event data, the reproduction control unit 75 causes the display unit 40 to display the bird's-eye view image generated by the bird's-eye view image generation unit 76.

The bird's-eye view image generation unit 76 performs viewpoint conversion processing on a plurality of shooting data recorded in the recording unit 20 as seen from above the vehicle, and synthesizes a plurality of shooting data that have been subjected to the viewpoint conversion processing. Generate a bird's-eye view image with. In the present embodiment, the bird's-eye view image generation unit 76 can generate a bird's-eye view image with the upper part of the vehicle as a virtual viewpoint.

The operation control unit 77 receives an operation signal related to an operation received from a user or the like from the operation unit 30. The operation control unit 77 receives operation signals related to operations such as the start of reproduction of shooting data and the start of recording of shooting data. The operation control unit 77 outputs a control signal corresponding to the received operation signal to the recording control unit 74 or the reproduction control unit 75. In this case, the recording control unit 74 and the reproduction control unit 75 execute the operation according to the control signal.

The display control unit 78 displays various images on the display unit 40. The display control unit 78 displays, for example, the shooting data and the event data reproduced by the reproduction control unit 75 on the display unit 40. Specifically, the display control unit 78 displays a video on the display unit 40 by outputting a video signal to the display unit 40. The display control unit 78 displays the shooting data on the display unit 40 by outputting the video signal related to the shooting data and the event data recorded in the recording unit 20 to the display unit 40, for example.

The acceleration information acquisition unit 79 acquires acceleration information regarding the acceleration applied to the vehicle from the acceleration sensor 50. Specifically, the acceleration information acquisition unit 79 acquires acceleration information regarding the acceleration applied to the vehicle while parking. The acceleration information acquisition unit 79 outputs the acquired acceleration information to the event detection unit 80.

The event detection unit 80 detects an event for the vehicle. The method by which the event detection unit 80 detects an event for the vehicle is arbitrary, but as an example, the event detection unit 80 detects an event based on the detection result of the acceleration sensor 50. In this case, the event detection unit 80 detects an acceleration equal to or greater than a threshold value in the acceleration detected by the acceleration sensor 50 as an event. The acceleration detected by the event detection unit 80 as an event is set to a threshold value at which an impact corresponding to an accident such as an object colliding with a vehicle is detected.

When the event detection unit 80 detects an event, the event detection direction acquisition unit 81 acquires the event detection direction for the vehicle. The event detection direction acquisition unit 81 detects the direction in which another object that has generated the acceleration detected as an event collides with the vehicle based on the acceleration information for each axis in the acceleration sensor 50 when the event is detected. do. The information indicating the event detection direction acquired by the event detection direction acquisition unit 81 is recorded in association with the event data. The event detection direction is the direction in which the event occurs when viewed from the vehicle. The event detection direction acquisition unit 81 may acquire the event detection direction not only by the acceleration information from the acceleration sensor 50 but also by voice, video, or a combination thereof.

The position information acquisition unit 82 receives a GPS signal from the GPS reception unit 60. The position information acquisition unit 82 calculates the current position information based on the GPS signal. The position information acquisition unit 82 and the GPS reception unit 60 are not limited to GPS, and may be compatible with other types of positioning satellite systems.

The bird's-eye view image generated by the bird's-eye view image generation unit 76 will be described with reference to FIG. FIG. 2 is an explanatory diagram showing an example of the bird's-eye view image P. The bird's-eye view image P includes a front image P1, a rear image P2, a right side image P3, and a left side image P4. Further, the bird's-eye view image P includes the composite boundaries B1, B2, B3, and B4. The own vehicle icon M is displayed in the center of the bird's-eye view image P. The own vehicle icon M indicates a form in which the vehicle is viewed from above.

The bird's-eye view image generation unit 76 basically generates a bird's-eye view image P in which the display direction is headed up, that is, the front side of the vehicle is set upward in the screen. The bird's-eye view image generation unit 76 may generate a bird's-eye view image P in which the display direction is set to a direction other than the heading-up, that is, the front side of the vehicle is set to a direction other than the upper side in the screen, depending on various conditions.

When the bird's-eye view image generation unit 76 starts displaying the bird's-eye view image, the bird's-eye view image is synthesized by performing viewpoint conversion processing so that the peripheral image acquired by the shooting data acquisition unit 42 can be seen from the virtual viewpoint located directly above the vehicle. To generate.

The bird's-eye view image generation unit 76 acquires the shooting data acquisition unit 71 and performs viewpoint conversion processing on the plurality of shooting data recorded in the recording unit 20 as if the vehicle was viewed from above. More specifically, the bird's-eye view image generation unit 76 generates an image that has undergone viewpoint conversion processing based on a plurality of shooting data captured by the front camera 11, the rear camera 12, the right side camera 13, and the left side camera 14. The method of the viewpoint conversion process may be any known method and is not limited.

The bird's-eye view image generation unit 76 performs a cutting process for cutting out an image in a predetermined range from a plurality of captured data that have undergone the viewpoint conversion process. The bird's-eye view image generation unit 76 cuts out the front cutout range from the shooting data of the front camera 11 that has undergone the viewpoint conversion process. The bird's-eye view image generation unit 76 cuts out the rear cutout range from the shooting data of the rear camera 12 that has undergone the viewpoint conversion process. The bird's-eye view image generation unit 76 cuts out the right side cutout range from the shooting data of the right side camera 13 that has undergone the viewpoint conversion process. The bird's-eye view image generation unit 76 cuts out the left side cutout range from the shooting data of the left side camera 14 that has undergone the viewpoint conversion process.

The front cutout range is a range for cutting out the front image P1. The front cut-out range is the range from the front end of the vehicle to the front. The rear cutout range is a range in which the rear image P2 is cut out from a plurality of captured data subjected to the viewpoint conversion process. The rear cutting range is the range from the rear end of the vehicle to the rear. The right side cutout range is a range in which the right side image P3 is cut out from a plurality of captured data subjected to the viewpoint conversion process. The right side cutout range is the range from the right side to the right side of the vehicle. The left side cutout range is a range in which the left side image P4 is cut out from a plurality of captured data subjected to the viewpoint conversion process. The left side cutout range is the range from the left side to the left side of the vehicle.

The bird's-eye view image generation unit 76 performs a compositing process for synthesizing a plurality of images cut out by the bird's-eye view image generation unit 76. The boundary between the front cutout range (front image P1) and the right side cutout range (right side image P3) is the composite boundary B1. The boundary between the front cutout range (front image P1) and the left side cutout range (left side image P4) is the composite boundary B2. The boundary between the rear cutout range (rear image P2) and the right side cutout range (right side image P3) is the composite boundary B3. The boundary between the rear cutout range (rear image P2) and the left side cutout range (left side image P4) is the composite boundary B4.

For example, the composite boundary B1 is set from the right front corner of the own vehicle icon M to the front side of the right side of the bird's-eye view image P. The composite boundary B2 is set from the left front corner of the own vehicle icon M to the front side of the left side of the bird's-eye view image P. The composite boundary B3 is set from the right rear corner of the own vehicle icon M to the rear side of the right side of the bird's-eye view image P. The composite boundary B4 is set from the left rear corner of the own vehicle icon M to the rear side of the left side of the bird's-eye view image P.

When the shooting data other than the event data is reproduced, or when the event data having no event detection direction is reproduced, the bird's-eye view image generation unit 76 has the front image P1 and the rear image P2 as shown in FIG. The right side image P3 and the left side image P4 are cut out so as to have the same shape and the same area symmetrically in the vertical direction, and the composite boundaries B1, B2, B3, B4 is set to generate a standard bird's-eye view image P. In other words, the bird's-eye view image generation unit 76 performs the cutting process with a preset initial value.

When the event data having the event detection direction is reproduced, the bird's-eye view image generation unit 76 changes the composite boundaries B1, B2, B3, and B4 according to the event detection direction. Specifically, the bird's-eye view image generation unit 76 can change the composite boundaries B1, B2, B3, and B4 so as not to overlap with the acquired event detection direction. Further, the bird's-eye view image generation unit 76 can change the composite boundaries B1, B2, B3, and B4 so that the display area of the image corresponding to the acquired event detection direction becomes large. Further, the bird's-eye view image generation unit 76 can move the composite boundary between the image corresponding to the acquired event detection direction and the adjacent image in the direction opposite to the event detection direction. The "direction opposite to the event detection direction" may include the opposite direction for one of the front-back direction and the left-right direction when the event detection direction is decomposed. For example, when the event detection direction is the right front direction, the opposite direction of the event detection direction may include each of the left rear direction, the left direction, and the rear direction.

FIG. 3 is a diagram showing an example of a bird's-eye view image in which a part of the composite boundary is changed. In FIG. 3, the case where the acquired event detection direction D1 is in front of the vehicle is taken as an example. When the event detection direction D1 is detected, the bird's-eye view image generation unit 76 identifies the image corresponding to the detected event detection direction D1. In the example of FIG. 3, the bird's-eye view image generation unit 76 can specify the front image P1 that reflects the front in the event detection direction D1. The bird's-eye view image generation unit 76 can change the composite boundaries B1 and B2 so that the display area of the front image P1 corresponding to the event detection direction D1 becomes large. The bird's-eye view image generation unit 76 may change only one of the composite boundaries B1 and B2. The bird's-eye view image generation unit 76 can move the composite boundaries B1 and B2 of the right side image P3 and the left side image P4 adjacent to the front image P1 in the direction opposite to the event detection direction D1. Specifically, the bird's-eye view image generation unit 76 can move so as to incline the outer peripheral sides of the composite boundaries B1 and B2 to the rear, respectively. The arrows on the bird's-eye view image P in FIGS. 3 to 6 indicate the moving direction of the composite boundary.

FIG. 4 is a diagram showing another example of a bird's-eye view image in which a part of the composite boundary is changed. In FIG. 4, an example is taken in which the acquired event detection direction D2 is in the front right direction with respect to the vehicle. In this case, the bird's-eye view image generation unit 76 can specify the front image P1 among the front image P1 and the right side image P3 showing the right front direction, which is the event detection direction D2. The bird's-eye view image generation unit 76 can change the composite boundary B1 so that the display area of the front image P1 corresponding to the event detection direction D2 becomes large. For example, the bird's-eye view image generation unit 76 can move the outer peripheral side of the composite boundary B1 so as to tilt backward as shown in FIG. 4 as the direction opposite to the event detection direction D2. In this case, the composite boundary B1 is arranged from the right front corner of the own vehicle icon M toward the right.

FIG. 5 is a diagram showing another example of a bird's-eye view image in which a part of the composite boundary is changed. In FIG. 5, as in FIG. 4, the case where the acquired event detection direction D2 is in the front right direction with respect to the vehicle is taken as an example. The bird's-eye view image generation unit 76 may specify the right side image P3 among the front image P1 and the right side image P3 as an image showing the right front direction. In this case, the bird's-eye view image generation unit 76 can change the composite boundary B1 so that the display area of the right side image P3 becomes large. In the example shown in FIG. 5, the bird's-eye view image generation unit 76 moves so as to tilt the outer peripheral side of the composite boundary B1 to the left as the direction opposite to the event detection direction D2. In this case, the composite boundary B1 is arranged from the right front corner of the own vehicle icon M toward the front.

FIG. 6 is a diagram showing another example of a bird's-eye view image in which a part of the composite boundary is changed. In FIG. 6, the case where the acquired event detection direction D2 is on the front right side with respect to the vehicle is taken as an example. As shown in FIG. 6, when the right side image P3 is specified as an image reflecting the right front direction, the bird's-eye view image generation unit 76 changes the composite boundaries B1 and B3 so that the display area of the front image P3 becomes large. Can be done. In the example shown in FIG. 6, the bird's-eye view image generation unit 76 moves so as to tilt the outer peripheral side of the composite boundary B1 to the left as the direction opposite to the event detection direction D2. Further, the bird's-eye view image generation unit 76 moves so as to tilt the outer peripheral sides of the composite boundaries B1 and B3 to the left as opposite directions to the event detection direction D2. In this case, the composite boundary B1 is arranged from the right front corner of the own vehicle icon M toward the front. Further, the composite boundary B3 is arranged from the right front corner portion of the own vehicle icon M toward the rear. In the case shown in FIG. 6, the right side image P3 of the bird's-eye view image P is displayed in the entire area in the front-rear direction. As a result, the behavior of the object R when the object R moves back and forth to the right side of the own vehicle, for example, when the object R collides with the own vehicle from the front right direction and moves in the rear right direction. Can be confirmed properly. The alternate long and short dash line in FIG. 6 shows the movement progress of the object R.

After the composite boundaries B1, B2, B3, and B4 of the bird's-eye view image P are changed, for example, when an operation to reset the change is performed to the operation control unit 77, the bird's-eye view image generation unit 76 operates. Based on the signal from the control unit 77, the composite boundaries B1, B2, B3, and B4 can be returned to the standard state shown in FIG. Further, when an operation is performed to individually change the positions of the composition boundaries B1, B2, B3, and B4 of the bird's-eye view image P, the bird's-eye view image generation unit 76 synthesizes the image based on the signal from the operation control unit 77. The positions of the boundaries B1, B2, B3, and B4 can be changed individually.

Next, the flow of the recording process in the recording / reproducing device 100 will be described with reference to FIG. 7. FIG. 7 is a flowchart showing a flow of recording processing in the recording / reproducing device 100 according to the first embodiment. Here, a case where loop recording processing is performed will be described.

The recording / playback device 100 starts event detection and loop recording (step S101). More specifically, the recording / playback device 100 detects an event by the event detection unit 80 based on the detection result of the acceleration sensor 50. The recording / playback device 100 generates loop recording / photographing data for each image for a predetermined period from the photographing data recorded in the buffer memory 72 by the photographing data processing unit 73. The recording / playback device 100 causes the recording control unit 74 to record the loop recording / photographing data in the recording unit 20. The recording / playback device 100 proceeds to step S102.

The recording / playback device 100 determines whether or not an event has been detected by the event detection unit 80 (step S102). More specifically, when the event is detected by the event detection unit 80 (Yes in step S102), the recording / playback device 100 proceeds to step S103. If the event detection unit 80 does not detect an event (No in step S102), the recording / playback device 100 proceeds to step S105.

When an event is detected (Yes in step S102), the recording / playback device 100 acquires the event detection direction by the event detection direction acquisition unit 81 (step S103). The recording / playback device 100 proceeds to step S104.

The recording / playback device 100 saves event data, which is shooting data before and after event detection, and an event detection direction (step S104). More specifically, the recording / playback device 100 generates shooting data including at least the time when an event occurs from the loop recording shooting data recorded in the recording unit 20 by the shooting data processing unit 73 as event data. The recording / playback device 100 stores event data in the recording unit 20 by the recording control unit 74. The recording / playback device 100 stores information indicating the event detection direction together with the event data. The recording / playback device 100 stores the event data in an overwrite-protected state. The recording / playback device 100 proceeds to step S105. If the event detection direction cannot be acquired even though the event is detected in step S103, the event data is saved without acquiring the event detection direction in step S104.

The recording / playback device 100 determines whether or not to end the event detection and loop recording (step S105). More specifically, the recording / reproducing device 100 determines that the event detection and loop recording are terminated under arbitrary conditions such as when the operation control unit 77 outputs the termination operation information or the operation of the vehicle is terminated. (Yes in step S105), the process ends. If the operation control unit 77 does not output the end operation information, the recording / reproducing device 100 determines that the event detection and the loop recording are not ended (No in step S105), and executes the process of step S102 again.

Next, the flow of the reproduction process in the recording / reproducing device 100 will be described with reference to FIG. FIG. 8 is a flowchart showing a flow of reproduction processing in the recording / reproducing device 100 according to the first embodiment.

The recording / playback device 100 determines whether or not there is a playback instruction for event data (step S201). When the operation control unit 77 acquires the operation information indicating the event data reproduction instruction, the recording / reproduction device 100 determines that the event data reproduction instruction is given (Yes in step S201), and proceeds to step S202. When the operation control unit 77 does not acquire the operation information indicating the playback instruction of the event data, the recording / playback device 100 determines that there is no playback instruction of the event data (No in step S201), and ends this process or ends the process. The process of step S201 may be executed again.

When it is determined that there is an event data reproduction instruction (Yes in step S201), the recording / reproduction device 100 determines whether or not there is an event detection direction associated with the event data for which reproduction is instructed (step S202). ). More specifically, when the recording / reproducing device 100 stores information indicating the event detection direction together with the event data to be reproduced, it determines that there is an event detection direction (Yes in step S202), and proceeds to step S203. If the information indicating the event detection direction is not stored together with the event data to be reproduced, the recording / reproduction device 100 determines that there is no event detection direction (No in step S202), and proceeds to step S205.

When it is determined that there is an event detection direction (Yes in step S202), the recording / playback device 100 generates a bird's-eye view image P (see FIGS. 3 to 6) corresponding to the event detection direction, and displays the generated bird's-eye view image P. It is displayed on the unit 40 (step S203), and the process proceeds to step S204.

The recording / reproducing device 100 determines whether or not to end the reproduction of the event data (step S204). When the event data is reproduced to the end, or when the operation of stopping the reproduction is performed, the recording / reproducing device 100 determines that the reproduction of the event data is finished (Yes in step S204), and ends the process. If the above does not apply, the recording / playback device 100 determines that the reproduction of the event data is not completed (No in step S204), and executes the process of step S204 again.

When it is determined in step S202 that there is no event detection direction (No in step S202), the recording / playback device 100 generates a standard bird's-eye view image P (see FIG. 2), and the generated bird's-eye view image P is displayed on the display unit 40. Display (step S205) and proceed to step S206.

The recording / reproducing device 100 determines whether or not to end the reproduction of the event data (step S206). When the event data is reproduced to the end, or when the operation of stopping the reproduction is performed, the recording / reproducing device 100 determines that the reproduction of the event data is finished (Yes in step S206), and ends the process. If the above does not apply, the recording / playback device 100 determines that the reproduction of the event data is not completed (No in step S206), and executes the process of step S206 again.

As described above, in the present embodiment, the acquired plurality of shooting data are subjected to viewpoint conversion processing as seen from above the vehicle, and the composite boundary for synthesizing the plurality of images is changed according to the event detection direction. By generating the synthesized bird's-eye view image P and displaying the generated bird's-eye view image P on the display unit, it is possible to suppress the arrangement of the composite boundary in the direction corresponding to the event detection direction. This makes it possible to appropriately confirm the status of the event using the bird's-eye view video.

In the present embodiment, since the composite boundary is changed so as not to overlap with the acquired event detection direction, when the event status is confirmed using the bird's-eye view video P, the event status can be appropriately confirmed by the overlap of the composite boundaries. It can be suppressed from disappearing.

In the present embodiment, when the bird's-eye view video P is generated, the composite boundary is changed so that the display area of the video showing the direction corresponding to the acquired event detection direction becomes large, so that the event status is confirmed more appropriately. be able to.

In the present embodiment, the composite boundary between the video corresponding to the acquired event detection direction and the adjacent video is changed so as to move in the direction opposite to the event detection direction. Therefore, in the situation where the event occurs, the display unit 40 While making effective use of the display space of, it is possible to emphasize and display the image showing the event.

[Second Embodiment]
The video reproduction device 200 according to the present embodiment will be described with reference to FIG. FIG. 9 is a block diagram showing an example of the video reproduction device 200 according to the second embodiment.

The video reproduction device 200 reproduces, for example, the shooting data stored in the recording / reproduction device 100 according to the first embodiment. The video reproduction device 200 includes a recording unit 110, an operation unit 120, a display unit 130, and a control unit 140. The video reproduction device 200 is an electronic device including, for example, a personal computer, a smartphone, a tablet terminal, a navigation device, and the like.

The recording unit 110 stores event data, information indicating an event detection direction, loop recording data, and the like recorded by the recording / reproducing device 100 and the like. The recording unit 110 is, for example, a semiconductor memory element such as a flash memory or a recording unit such as a memory card. Alternatively, it may be an external recording unit wirelessly connected via a communication device (not shown).

The operation unit 120 can accept various operations on the video reproduction device 200. For example, the operation unit 120 can accept an operation of reproducing the event data or the loop recording data recorded in the recording unit 110. For example, the operation unit 120 can accept an operation of erasing the event data recorded in the recording unit 110. The operation unit 120 outputs the operation information to the operation control unit 144 of the control unit 140.

The operation unit 120 is a touch panel or the like provided on the display unit 130, and an operation of reproducing event data stored in the recording unit 110 is performed by touching a thumbnail image displayed on the display unit 130 to perform an event. Select data and give playback instructions.

The display unit 130 is a display including, for example, a liquid crystal display or an organic EL display. The display unit 130 displays an image based on the image signal output from the display control unit 145 of the control unit 140. The display unit 130 displays the image recorded on the recording unit 110.

The control unit 140 is, for example, an arithmetic processing unit (control device) composed of a CPU or the like. The control unit 140 loads the stored program into the memory and executes the instructions included in the program. The control unit 140 includes an internal memory (not shown), and the internal memory is used for temporary storage of data in the control unit 140 and the like. The control unit 140 includes an information acquisition unit 141, a reproduction control unit 142, a bird's-eye view image generation unit 143, an operation control unit 144, and a display control unit 145, which are connected to the bus 146.

The information acquisition unit 141 acquires a plurality of shooting data taken by a plurality of cameras that shoot the surroundings of the vehicle based on an event that has occurred on the vehicle, and information indicating an event detection direction for the vehicle.

The reproduction control unit 142 acquires the selected event data or loop recording data from the recording unit 110 and reproduces the data. The reproduction control unit 142 controls to reproduce the event data or the loop recording data recorded in the recording unit 110 based on the control signals of the selection operation and the reproduction operation output from the operation control unit 144. More specifically, the reproduction control unit 142 causes the display unit 130 to display the bird's-eye view image generated by the bird's-eye view image generation unit 143 when reproducing the shooting data.

The bird's-eye view image generation unit 143 performs viewpoint conversion processing on a plurality of shooting data acquired by the information acquisition unit 141 as if viewed from above the vehicle, and changes the boundary for synthesizing the plurality of images according to the event detection direction. And generate a composite bird's-eye view image. The bird's-eye view image generation unit 143 changes the boundary for synthesizing a plurality of images so as not to overlap with the event detection direction acquired by the information acquisition unit 141. The bird's-eye view image generation unit 143 changes the boundary for synthesizing a plurality of images so that the display area of the image corresponding to the event detection direction acquired by the information acquisition unit 141 is large among the plurality of images. The bird's-eye view image generation unit 143 changes the boundary between the image corresponding to the event detection direction acquired by the information acquisition unit 141 and the adjacent image among the plurality of images so as to move in the direction opposite to the event detection direction. ..

The operation control unit 144 acquires the operation information of the operation received by the operation unit 120. For example, the operation control unit 144 acquires the selection operation information indicating the video data selection operation, the playback operation information indicating the video data reproduction operation, or the erasing operation information indicating the video data erasing operation, and outputs the control signal. do.

The display control unit 145 controls the display of video data on the display unit 130 based on the control signal from the reproduction control unit 142. The display control unit 145 outputs a video signal for outputting the video data to the display unit 130.

The flow of the reproduction process in the video reproduction device 200 is the same as the flow of the reproduction process in the recording / reproduction device 100 shown in FIG.

In this way, the video playback device 200 that reproduces the shooting data stored in the recording / playback device 100 also performs viewpoint conversion processing on the acquired plurality of shooting data as seen from above the vehicle, and performs a plurality of viewpoint conversion processes. The composite boundary for synthesizing the images is changed according to the event detection direction to generate the composite bird's-eye view image P, and the generated bird's-eye view image P is displayed on the display unit. Therefore, it is possible to suppress the arrangement of the composite boundary in the direction corresponding to the event detection direction. This makes it possible to appropriately confirm the status of the event using the bird's-eye view video.

Although the recording / reproducing device 100 and the video reproducing device 200 according to the present invention have been described so far, they may be implemented in various different forms other than the above-described embodiment.

Each component of the illustrated recording / reproducing device 100 and the video reproducing device 200 is a functional concept and does not necessarily have to be physically configured as shown in the drawing. That is, the specific form of each device is not limited to the one shown in the figure, and all or part of each device is functionally or physically dispersed or integrated in an arbitrary unit according to the processing load and usage status of each device. You may.

The configuration of the recording / playback device 100 and the video playback device 200 is realized, for example, by a program loaded in a memory as software. In the above embodiment, it has been described as a functional block realized by cooperation of these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

The above-mentioned components include those that can be easily assumed by those skilled in the art and those that are substantially the same. Further, the above configurations can be combined as appropriate. Further, various omissions, substitutions or changes of the configuration can be made without departing from the gist of the present invention.

In the above, the case where the loop recording process is performed has been described, but the same applies to the case where shooting is started after the event is detected.

B1, B2, B3, B4 ... Composite boundary, D1, D2 ... Event detection direction, M ... Own vehicle icon, P ... Bird's-eye view image, P1 ... Front image, P2 ... Rear image, P3 ... Right side image, P4 ... Left side Video, R ... Object, 10 ... Shooting unit, 11 ... Front camera, 12 ... Rear camera, 13 ... Right side camera, 14 ... Left side camera, 20,110 ... Recording unit, 23 ... Setting operation unit, 30,120 ... Operation unit, 40, 130 ... Display unit, 42, 71 ... Shooting data acquisition unit, 50 ... Accelerometer, 60 ... GPS receiving unit, 70, 140 ... Control unit, 72 ... Buffer memory, 73 ... Shooting data processing unit, 74 ... Recording control unit, 75,142 ... Playback control unit, 76,143 ... Bird's-eye view image generation unit, 77,144 ... Operation control unit, 78,145 ... Display control unit, 79 ... Accelerometer information acquisition unit, 80 ... Event detection Unit, 81 ... Event detection direction acquisition unit, 82 ... Position information acquisition unit, 83, 146 ... Bus, 100 ... Recording / playback device, 141 ... Information acquisition unit, 200 ... Video playback device

Claims (7)

An information acquisition unit that acquires a plurality of shooting data taken by a plurality of cameras that shoot the surroundings of the vehicle based on an event that has occurred on the vehicle, and information indicating an event detection direction for the vehicle.
A bird's-eye view of the plurality of captured data acquired by the information acquisition unit, which is subjected to viewpoint conversion processing as seen from above the vehicle, and the boundary for synthesizing the plurality of images is changed according to the event detection direction. A bird's-eye view video generator that generates video, and
A playback control unit that displays the bird's-eye view image generated by the bird's-eye view image generation unit on the display unit, and
Video playback device equipped with. The bird's-eye view image generation unit changes the boundary for synthesizing the plurality of images so as not to overlap with the event detection direction acquired by the information acquisition unit.
The video playback device according to claim 1. The bird's-eye view image generation unit changes the boundary for synthesizing the plurality of images so that the display area of the image corresponding to the event detection direction acquired by the information acquisition unit is large among the plurality of images.
The video playback device according to claim 1 or 2. The bird's-eye view image generation unit changes the boundary between the image corresponding to the event detection direction acquired by the information acquisition unit and the adjacent image among the plurality of images so as to move in the direction opposite to the event detection direction. do,
The video playback device according to any one of claims 1 to 3. An event detection unit that detects an event for the vehicle based on the acceleration applied to the vehicle,
An information acquisition unit that acquires a plurality of shooting data captured by a plurality of cameras that photograph the surroundings of the vehicle based on an event detected by the event detection unit, and information indicating an event detection direction for the vehicle.
When the event detection unit detects an event for the vehicle, a recording control unit that records a plurality of shooting data for a predetermined period regarding the detected event, and a recording control unit.
The plurality of captured data recorded by the recording control unit are subjected to viewpoint conversion processing as seen from above the vehicle, and the boundary for synthesizing the plurality of images is changed according to the event detection direction and synthesized. A bird's-eye view image generation unit that generates a bird's-eye view image, a playback control unit that displays the bird's-eye view image generated by the bird's-eye view image generation unit on the display unit, and a playback control unit.
A recording / playback device including. An information acquisition step of acquiring a plurality of shooting data taken by a plurality of cameras that shoot the surroundings of the vehicle based on an event that has occurred on the vehicle, and information indicating an event detection direction for the vehicle.
The plurality of shooting data acquired in the information acquisition step are subjected to viewpoint conversion processing as seen from above the vehicle, and the boundary for synthesizing the plurality of images is changed according to the event detection direction and synthesized. A bird's-eye view image generation step to generate a bird's-eye view image,
A playback control step for displaying the bird's-eye view image generated in the bird's-eye view image generation step on the display unit, and
Video playback method including. An information acquisition step of acquiring a plurality of shooting data taken by a plurality of cameras that shoot the surroundings of the vehicle based on an event that has occurred on the vehicle, and information indicating an event detection direction for the vehicle.
The plurality of shooting data acquired in the information acquisition step are subjected to viewpoint conversion processing as seen from above the vehicle, and the boundary for synthesizing the plurality of images is changed according to the event detection direction and synthesized. A bird's-eye view image generation step to generate a bird's-eye view image,
A playback control step for displaying the bird's-eye view image generated in the bird's-eye view image generation step on the display unit, and
A video playback program that lets your computer run.

Images