JP7418038B2 - System, control method, program, etc. - Google Patents

Description

The present invention relates to, for example, a system, a control method, a program, etc.

In conventional literature, there is a method in which video recorded by a drive recorder is played back on a personal computer or a so-called smartphone, and displayed on the personal computer's display screen or the smartphone's display screen (Patent Document 1). Furthermore, if a display screen is formed on the drive recorder, some drive recorders display images on the display screen.

Japanese Patent Application Publication No. 2016-14310

The method described in Patent Document 1 is still not sufficient.

The aim is to provide something better than before.

The purpose of the invention of the present application is not limited to this, but we also intend to acquire rights through divisional applications, amendments, etc. for structures that aim to obtain effects from the parts of the structures disclosed in this specification, drawings, etc. . For example, the present specification discloses a problem in which the phrase ``can be done'' is replaced with ``the problem is.'' Each problem is described as independent, and we intend to obtain rights for the structure to solve this problem independently through divisional applications, amendments, etc. Even if the problem is understood implicitly from the description, the applicant intends to claim a part of the structure described in the specification in an amendment or divisional application. .

(1) It is preferable that the system is capable of displaying images taken in a vehicle and is equipped with means for processing the image data for the display.

By processing video data for display, it is possible to provide something better than before.

The displayable system may be, for example, a personal computer, a smartphone, a drive recorder, or the like. Processing of video data for display includes, for example, processing for video playback, processing for video data transmission, processing for adjusting video brightness, processing for changing the video playback format, and reducing the amount of video data. It is better to use processing.

(2) The video data for display may be processed to adjust the brightness of the video based on instructions from the user.

The brightness of the image can be adjusted based on instructions from the user.

The system may be provided with instruction input means such as hardware buttons and software buttons, and instructions may be received from the user using these instruction input means, or images may be displayed. A menu may be displayed on the screen, and an instruction input means for selecting an instruction from the user may be provided in the menu. The brightness may be adjusted so that the dark parts of the entire video or part of the video become brighter, or the brightness may be adjusted so that the bright parts of the entire video or part of the video become darker.

(3) The above video contains bright and dark areas in one frame, and the brightness is adjusted so that the brightness adjustment amount of the dark areas is relatively larger than the brightness adjustment amount of the bright areas. It is preferable to process the video data for the above display to be adjusted.

The difference between the brightness of dark areas and bright areas is reduced, making the image easier to see as a whole.

It is preferable that the bright area be an area with brightness equal to or higher than a threshold value within one frame, and the dark area be an area with brightness less than the threshold value. It is also possible to provide a detection means for detecting a bright part that is equal to or higher than a threshold value, to make the detected part a bright part, and to make an undetected part a dark part. Alternatively, a detection means for detecting a dark portion below the threshold value may be provided, and the detected portion may be defined as a dark portion, and the undetected portion may be defined as a bright portion. The brightness adjustment amount may be adjusted brightly by adjusting the gain of the video data. The brightness may be adjusted so that both bright and dark areas become brighter, such that the brightness adjustment amount for the dark area is relatively larger than the brightness adjustment amount for the bright area. The brightness may be adjusted so that only dark areas become bright. It is even better if a means for adjusting this threshold value is provided. The means for adjusting the threshold value may be a means for adjusting the threshold value by inputting a user's instruction.

(4) The above video includes a video portion inside the vehicle interior and a video portion outside the vehicle interior in one frame, and the video data for the above display is processed to adjust the brightness of the video portion inside the vehicle interior. It's good to do that.

Video data with the brightness adjusted for the video portion of the vehicle interior can be obtained.

A means for detecting an image portion inside the vehicle interior may be provided, and the detected portion may be defined as an image portion inside the vehicle interior, and the undetected portion may be defined as an image portion outside the vehicle interior. Further, a means for detecting an image portion outside the vehicle may be provided, the detected portion may be set as the image portion outside the vehicle, and the undetected portion may be set as the image portion inside the vehicle. It is preferable to use edge detection to detect the video portion inside the vehicle interior. It is best to adjust the brightness by adjusting the gain of the video data. The brightness may be adjusted so that both the video portion inside the vehicle interior and the video portion outside the vehicle interior are bright, or the brightness may be adjusted so that only the video portion inside the vehicle interior is bright.

(5) Processing the video data for the above display to brighten the captured video so that the brightness adjustment amount inside the vehicle is relatively larger than the brightness adjustment amount for the video portion outside the vehicle interior. good.

It is thought that the brightness of the image area outside the vehicle interior is often brighter than the brightness of the image area inside the vehicle interior, so the amount of brightness adjustment inside the vehicle interior is greater than the amount of brightness adjustment for the image area outside the vehicle interior. By adjusting the brightness of the captured image so that the image is relatively large, the difference between the brightness of the image inside the vehicle and the brightness of the image outside the vehicle becomes smaller, making the image as a whole easier to see.

It is preferable to provide a means for detecting an image portion inside the vehicle interior, and make the detected portion the image portion inside the vehicle interior, and the portion not detected as the image portion outside the vehicle interior. Further, it is preferable to provide a means for detecting an image portion outside the vehicle interior, and use the detected portion as the image portion outside the vehicle interior, and the undetected portion as the image portion inside the vehicle interior. It is preferable to use edge detection to detect the video portion inside the vehicle interior. The brightness adjustment amount may be adjusted brightly by adjusting the gain of the video data. The brightness is set so that both the image area inside the vehicle interior and the image area outside the vehicle interior are brighter, so that the amount of brightness adjustment inside the vehicle interior is relatively larger than the amount of brightness adjustment for the image area outside the vehicle interior. The brightness may be adjusted so that only the video portion inside the vehicle is bright. The means for detecting the video portion inside the vehicle interior may, for example, accept a selection of the region of the vehicle interior portion in the video from the user and detect the selected portion as the video portion inside the vehicle interior, or may, for example, automatically detect the selected portion as the video portion inside the vehicle interior. Alternatively, a video portion inside the vehicle may be detected. As an automatic detection process, for example, when the movement of the vehicle is detected, parts of the image that change in response to the movement are detected as outside the vehicle, and areas that do not change in the image in response to the movement are detected as the interior of the vehicle. It is recommended that the process be performed as follows. In particular, it is preferable to perform a process of detecting an area in the video where there is movement in accordance with the speed of movement of the vehicle as outside the vehicle, and an area in which there is no movement in accordance with the speed of movement of the vehicle as inside the vehicle.

(6) It is preferable to include a first instruction input means for giving an instruction from the user to adjust the brightness inside the vehicle interior.

The brightness inside the vehicle can be adjusted based on instructions from the user.

The system may be provided with hardware buttons and software buttons, and these buttons may be used as the first instruction input means, or a menu may be displayed on the screen that displays the image, and that menu may be used as the first instruction input means. It can also be used as a means. If a menu is used as the first instruction input means, the user may be able to select an instruction from the menu. Alternatively, the keyboard of the personal computer may be used as the first instruction input means, or the keypad may be displayed on the display screen of the personal computer, the display screen of a smartphone, the display screen of a drive recorder, etc. The pad may be used as the first instruction input means. It is preferable to adjust the brightness in the vehicle interior based on a user's instruction given by the first instruction input means.

(7) It is preferable to provide a second instruction input means for receiving an instruction from the user without performing the above-mentioned brightness adjustment.

It is possible to prevent the brightness from being adjusted based on instructions from the user. By combining the first instruction input means and the second instruction input means, it is possible to switch between an instruction to perform brightness adjustment and an instruction not to perform brightness adjustment.

A hardware button or a software button may be used as the second instruction input means in the system, or a menu may be displayed on a screen displaying an image, and the menu may be used as the second instruction input means. If a menu is used as the second instruction input means, instructions from the user may be selected from the menu. Alternatively, the keyboard of the personal computer may be used as the second instruction input means, or the keypad may be displayed on the display screen of the personal computer, the display screen of a smartphone, the display screen of a drive recorder, etc. The pad may be used as the second instruction input means. When an instruction is given from the second instruction input means, the instruction from the second instruction input means is stopped, and when an instruction is given from the second instruction input means, the instruction from the first instruction input means is stopped. It would be good if you could.

(8) It is preferable to process the video data for the above display so that the first window that displays the video and the second window that includes the operation system for operating the displayed video are separated.

The first window that displays the video and the second window that includes an operation system that operates the displayed video can be separate.

For example, by displaying the first window that displays the video over the entire display screen, the video is displayed larger and becomes easier to see. The first window that displays the video and the second window that includes the operation system that operates the displayed video may be displayed on the same screen or on different screens, but preferably It is best to display it on a different screen. A switching instruction input means may be provided for switching between an instruction to display the first window and the second window on the same screen and an instruction to display the first window and the second window on different screens. Depending on the command from the switching instruction input means, the first window and the second window can be displayed on the same screen, or the first window and the second window can be displayed on different screens. For example, it is recommended to provide a software switching button on the screen or a hardware switching button on the system. The display can be switched according to instructions from the switching button. The operating system for operating the displayed video is, for example, one that gives commands to control video playback, such as starting video playback, stopping playback, pausing playback, fast forwarding, and rewinding.

(9) The display screen where the first window for displaying the video is displayed and the display screen where the second window containing the operation system for manipulating the video is displayed are separate. It is recommended to process video data for

The display screen on which the first window for displaying the video is displayed and the display screen on which the second window including the operation system for operating the video is displayed can be separate.

For example, a display screen on which a first window for displaying images is displayed is placed in a position that is easily visible to those viewing the images, and a display screen on which a second window containing an operation system for operating the images is displayed. It is recommended that the image be placed in a position where it is easy for the person operating the video to operate it. For example, when images are being obtained from driving by a driver managed by an administrator, the administrator operates the images while looking at the display screen where a second window is displayed, and When a driver or the like views the video on the display screen where the first window is displayed, the administrator can move the display screen where the second window is displayed to a position where it is easy to see so that the video can be easily operated. This allows drivers and others managed by an administrator to move the display screen on which the first window is displayed to a position where it is easier to view the video. The display screen on which the first window for displaying the video is displayed is preferably larger than the display screen on which the second window including the operation system for operating the video is displayed.

(10) The video data for display may be processed to generate an area for displaying the video in a second window that includes an operation system for operating the displayed video.

By doing this, the image will also be displayed in the second window, allowing you to operate while viewing the image. When the second window containing the operation system for operating the displayed image is displayed, the first window is displayed on the display screen by the user giving a command to display the first window for displaying the image. You can also make it appear. At that time, the first window may be displayed on the entire display screen.

(11) It is preferable to process the video data for display so as to display accompanying information of the video in the first window for displaying the video.

By doing this, the accompanying information of the video can be viewed along with the video.

The accompanying information of the video may be, for example, vehicle speed information, acceleration information, position information, etc. The accompanying information may be displayed on the video, or may be displayed in an area other than the area where the video is displayed in the first window for displaying the video. For example, when images are obtained from driving by a driver managed by an administrator, the administrator operates the images while looking at the second window, and the driver, etc. managed by the administrator View the video displayed in window 1. Videos viewed by managed drivers display incidental information such as vehicle speed and acceleration information when they are driving, so by viewing this incidental information they can better understand how they are driving. Can be clearly understood.

(12) It is preferable to process the video data for display so as to reduce the amount of data when the time is far from the time when the recording command is given.

It is possible to reduce the amount of data when the time is far from the time when the recording command is given than the amount of data near the time when the recording command is given.

The recording command may be generated, for example, when some kind of trigger is given to the vehicle that takes the video, or some kind of event is applied to the vehicle. For example, an event may occur when the brakes of a vehicle are stepped on or an impact is applied to the vehicle, and a trigger may be applied to the vehicle. Further, the system may be provided with software buttons and hardware buttons, and recording commands may be given by operating these buttons. It is possible to reduce the amount of data when there is a time difference between the time when a recording command is given and a later time, or when there is a time difference between the time when a recording command is given and a time before a previous time. The amount of data may also be reduced. The period including the time when the recording order is given is the first period, the second period is at least one of the periods after the first period or before the start of the first period, and the first period is the first period. It is preferable to reduce the amount of data in the second period than the amount of data in the period. The first period starts after a certain period of time has passed after the recording command is given, the second period starts after the end of the first period, and the amount of data in the second period is greater than the amount of data in the first period. may be decreased.

The amount of data may be reduced by increasing the compression rate of the video data, lowering the frame rate of the video data, or thinning out the number of pixels in the video. It is preferable to reduce the amount of data when video data is obtained by photographing in a vehicle. A rule for reducing the amount of data may be recorded, indicating by what method the amount of data was reduced.

(13) The video data for display may be processed so as to transmit video data with a reduced amount of data.

By doing so, it becomes possible to transmit reduced video data. Since the amount of video data to be transmitted is reduced, the transmission time is shortened.

The amount of data is preferably the amount of data per unit time. It is preferable to transmit the video data based on a mobile phone communication standard such as LTE (Long Term Evolution) or 4G. When capturing images in a vehicle using a drive recorder or the like, it is preferable to reduce the amount of image data obtained by shooting and transmit it from the drive recorder to a server or the like.

(14) It is preferable to process the video data for display so as to reduce the amount of video data to be transmitted compared to the amount of video data recorded on the recording medium in response to the recording command.

The amount of video data to be transmitted can be reduced compared to the amount of video data to be recorded.

The recording medium may be a hard disk or semiconductor memory built into the recording device, or may be a portable recording medium such as an SD card or an xD picture card.

(15) The video data for display may be processed so as to change the playback speed of the captured video based on the speed of the vehicle.

The playback speed of captured video can be changed based on the speed of the vehicle. The speed of the vehicle is preferably the speed of the vehicle that captured the video. The speed of the vehicle is preferably information based on vehicle speed information, particularly vehicle speed information. It is preferable that the speed information of the vehicle is recorded in association with the video data on a medium in which video data representing the captured video is recorded. The vehicle speed information may be obtained based on GPS (global positioning system) information, vehicle speed pulse information, etc. that is recorded on a recording medium together with the video data in association with the video data representing the constantly recorded video. The playback speed of the shot video may be changed by changing the time required to shoot the video and the time required to play the video. It is preferable to change the playback speed by changing the frame rate of the video at the time of shooting and the frame rate at the time of playback.

(16) It is preferable to process the video data for the above display so as to have a function of slowing down the video playback speed when the vehicle speed is slow compared to the video playback speed when the vehicle speed is fast.

In this way, the image when the vehicle is moving slowly can be clearly seen, and the image when the vehicle is moving quickly is difficult to see. The playback speed of videos taken when the vehicle is slow can be slowed down, making it easier to check videos taken when the vehicle is slow. Since the playback speed of the video when the vehicle is fast is not slow, the playback time of the entire video can be shortened. For example, when the speed of the vehicle is slow, it is particularly preferable to set the time when the vehicle is stopped, and in this way, the image while the vehicle is stopped can be viewed. It is preferable that the speed of the vehicle is high when the speed of the vehicle is faster than a threshold value. When determining whether the speed of a vehicle is slow, it is preferable to determine whether the speed of the vehicle is slow if other vehicles are captured in the video and the speed is slow compared to those other vehicles. In particular, in both control (16), control may be performed to slow down the video playback speed when the vehicle is slow compared to other vehicles. It is preferable to slow down the playback speed of the captured video by making the time required for reproducing the video longer than the time required for capturing the video. The function of slowing down the vehicle may be a function of increasing the number of frames per unit time of the video during playback compared to the number of frames per unit time of the video during shooting. The vehicle may be a vehicle that is photographing a video. For example, when the speed of a vehicle is slow so that information about a stopped vehicle can be obtained, the playback speed may be set to normal speed. When the speed of the vehicle is high, the playback speed may be set to n times the normal speed. It is preferable that n be a value larger than 1.

(17) It is preferable to process the video data for the above display so as to have a function of increasing the video playback speed when the vehicle speed is slow compared to the video playback speed when the vehicle speed is fast.

In this way, images when the vehicle is moving at a high speed can be clearly viewed, and images when the vehicle is at a slow speed are difficult to view. You can see the video slowly when the vehicle is moving at high speed. For videos taken while the vehicle being photographed is parked, skip playback is recommended because the videos are the same picture. If the video is taken while the vehicle is traveling at high speed, it is best to play it back slowly because the vehicle is moving fast and the video flows quickly. It is a good idea to fast-forward information that is not very necessary, such as images obtained when the vehicle is slow, such as when the vehicle is stopped due to a traffic signal. Browsing can be simplified. It is preferable that the speed of the vehicle is considered to be slow when the speed of the vehicle is slow below a threshold value. When determining whether the speed of a vehicle is slow, it is preferable to determine whether the speed of the vehicle is slow if surrounding vehicles are captured in the video and the speed is slow compared to those surrounding vehicles. In particular, for both controls in (17), it is recommended to increase the video playback speed when the vehicle is slow compared to other vehicles. The function for increasing the video playback speed may be a function that increases the playback speed of the shot video by making the time required for video playback shorter than the time required for video shooting. In particular, the function of increasing the video playback speed may be a function that reduces the number of frames per unit time of the video during playback compared to the number of frames per unit time of the video during shooting. For example, when the speed of the vehicle is high, the reproduction speed may be set to the normal speed or less than 1 times. When the speed of the vehicle is slow, the playback speed may be set to n times the normal speed. It is preferable that n be a value larger than 1.

(18) It is preferable to process the video data for the above display so as to increase the playback speed of the video of the front of the vehicle or the rear of the vehicle.

Playback time can be shortened. In particular, since the video of the front of the vehicle or the rear of the vehicle may not feel strange even if the playback speed is increased, the video playback time can be shortened without causing any discomfort. It is preferable to increase the playback speed of images in front of the vehicle or behind the vehicle without receiving a command to speed up the playback from the user. The playback speed will automatically increase. Whether the image is of the front of the vehicle or the rear of the vehicle may be determined, for example, by image analysis.

(19) It is preferable to process the video data for the above display so as to slow down the playback speed of the video in the lateral direction of the vehicle.

You can slowly view images in the horizontal direction. In particular, if the playback speed of the image in the lateral direction of the vehicle is fast, the image may flow and cause a sense of discomfort, but since the playback speed of the image in the lateral direction of the vehicle can be slowed down, this can be avoided, for example. . To slow down the playback speed, for example, it is preferable to make the time required for playback longer than the time required for shooting. Whether or not the image is a lateral view of the vehicle may be determined, for example, by analyzing the image.

It is preferable that the images taken in the vehicle include at least one of an image in front of the vehicle or an image behind the vehicle, and an image in the lateral direction of the vehicle. It is assumed that video data representing such a video is recorded on a recording medium. When an image of the front or rear of the vehicle is captured by a first camera that captures an image of the front or rear of the vehicle, it is recommended that the playback speed of the image obtained by such first camera be increased. . When a lateral image of the vehicle is captured by a second camera that captures an image of the vehicle in the lateral direction, it is preferable to slow down the playback speed of the image obtained by such second camera. When images are captured by a camera that can capture at least one of the front or rear images of the vehicle and the lateral images of the vehicle, the display area is changed in response to a command to change the display area of the images captured by such a camera. It is recommended that It is preferable to increase the playback speed when displaying images of the front or rear of the vehicle. Furthermore, when a lateral image of the vehicle is being displayed, the playback speed may be slowed down. When at least one image of the front or rear of the vehicle and an image of the lateral direction of the vehicle are displayed, the ratio of the display portion of the image of at least one of the front or rear of the vehicle to the display portion of the image of the lateral direction of the vehicle. It is preferable that the video playback speed be determined based on this. For example, when the display portion of at least one of the front and rear images of the vehicle is larger than the display portion of the image in the lateral direction of the vehicle, the video playback speed may be increased. On the other hand, when the display portion of the image in the lateral direction of the vehicle is larger than the display portion of the image in at least one of the front and rear directions of the vehicle, it is preferable to slow down the video playback speed.

(20) It is preferable to process the video data for the above display so that the captured video is played back at the normal playback speed based on a normal playback speed command given when the video is not played back at the normal playback speed.

By doing this, the video can be viewed at normal playback speed. The normal playback speed command may be given by the user using a button or the like. Further, it is preferable that the occurrence occurs based on a video scene when there is some kind of impact on the vehicle. Furthermore, it is preferable to generate the information based on information recorded in association with the video (for example, so-called historical information such as acceleration information, speed information, information on when the brake was pressed, and information on when the turn signal was activated). The normal playback speed is preferably such that the time required for shooting and the time required for playback are the same. For example, if you are shooting at a rate of 1 frame every 1/30 seconds, it is a good idea to play back at a rate of 1 frame every 1/30 seconds.

(21) It is preferable to process the video data for the above display so that the area for displaying the information regarding the vehicle obtained while driving the vehicle is enlarged.

Since the area for displaying information about the vehicle becomes larger, the information about the vehicle becomes easier to see. The information regarding the vehicle obtained while driving the vehicle may be data other than video data representing images captured by the vehicle. The vehicle-related information obtained while driving the vehicle may be, for example, the vehicle-related information obtained while driving the vehicle for photographing the video. Information about the vehicle is information obtained by the vehicle that takes the video (for example, map information indicating the location of the vehicle, vehicle acceleration information, speed information, information on whether the brakes are pressed, information on whether the turn signal is activated, etc.). good. It is preferable that the area for displaying information about the vehicle has at least two sizes, a small area and a large area, and a function to make the area larger is preferable. It is preferable that the area in which information about the vehicle is displayed is switched from a small area to a large area in response to a command given from a button or the like that gives a command such as an area expansion command. A default size is determined for the area for displaying information about the vehicle, and it is preferable to make the area larger than the default size. The region related to the vehicle may be enlarged to a predetermined size in response to a command from the user or a command from the system. It would be good to have a function other than the function of enlarging a window by dragging it. It is preferable to enlarge the area related to the vehicle in response to a command from a button such as an area expansion command. When an area for displaying images shot in a vehicle and an area for displaying information about the vehicle are defined in a window, it is preferable to make the area larger than the default size of the area for displaying information about the vehicle. Although it is best not to change the size of the area where the video is displayed, it is especially good to change it.

(22) It is preferable to process the video data for the above display so that the area for displaying information about the vehicle is larger than the area for displaying images shot in the vehicle.

Since the area for displaying information about the vehicle is larger than the area for displaying images, the information about the vehicle is easier to see than the image.

Information about the vehicle includes information obtained by the vehicle that captures the video (for example, map information indicating the location of the vehicle, vehicle acceleration information, speed information, information on when the brake is pressed, information on when the turn signal is activated, etc.) information). Normally, the area that displays information about the vehicle is smaller than the area that displays images shot in the vehicle, and the image shot in the vehicle is displayed in response to a command from the user or a command from the system. It is preferable to make the area in which information regarding the vehicle is displayed larger than the area. Images shot in a vehicle are displayed in a relatively large area, and information about the vehicle is displayed in a relatively small area, and due to a command from the user or a command from the system, the image is displayed in a relatively large area. It is preferable to display the video taken by the vehicle in a smaller area, and to display the information about the vehicle that was displayed in the smaller area in a larger area. The images taken in the vehicle are displayed in a relatively large area, and information about the vehicle is displayed in a relatively small area, and the images taken in the vehicle are displayed in a relatively small area. It is preferable to display information regarding the vehicle in an area larger than the relatively large area to be displayed. The images taken in the vehicle are displayed in a relatively large area, and information about the vehicle is displayed in a relatively small area, and the images taken in the vehicle are displayed in a relatively small area. It is better to display it in an even smaller area than a relatively small area. It is only necessary that the information regarding the vehicle be displayed in a relatively larger area than the video shot in the vehicle.

(23) It is preferable to process the video data for the above display so that the vehicle acceleration for each time is displayed three-dimensionally according to the time at which the video was taken.

By doing this, it becomes easier to see the acceleration. The acceleration of the vehicle may be the acceleration of the vehicle that captured the video. It is preferable to display the acceleration of the vehicle in three dimensions by displaying a three-dimensional space with degrees of freedom in each of the width, depth, and height directions as a plane. The data representing the acceleration may be recorded on the recording medium in association with video data representing one video. It is preferable that video data representing one video is included in one video file. It is preferable to display in three dimensions the entire acceleration represented by data recorded on a recording medium that is recorded in association with video information representing one video. It is preferable that the origin representing the acceleration of the vehicle at the start of imaging coincides with the point in time representing the acceleration of the vehicle at the end of imaging. It is preferable to display part of the acceleration of the vehicle from the start to the end of shooting.

(24) The forward acceleration of the vehicle is displayed based on the first axis of the three-dimensional graph, the lateral acceleration of the vehicle is displayed based on the second axis of the three-dimensional graph, and the vertical acceleration of the vehicle is displayed based on the second axis of the three-dimensional graph. It is preferable to process the video data for the above display so as to display the image based on the third axis of the three-dimensional graph.

By doing this, it becomes easier to see the acceleration. The first axis, the second axis, and the third axis are preferably perpendicular to each other. Vehicle acceleration can be understood relatively easily. It is preferable that the first axis is defined from the back to the front or from the front to the back, the second axis is defined in the width direction (rightward or leftward), and the third axis is defined upward or downward. It is preferable that the first axis is the X axis, the second axis is the Y axis, and the third axis is the Z axis.

(25) It is preferable to process the video data for the above display so that information regarding the vehicle is displayed superimposed on the video.

It becomes easier to view information about the vehicle. Information about the vehicle includes information obtained by the vehicle that captures the video (for example, map information indicating the location of the vehicle, vehicle acceleration information, speed information, information on when the brake is pressed, information on when the turn signal is activated, etc.) information). When displaying vehicle-related information superimposed on a video, it is preferable to erase the area in which the vehicle-related information is displayed. When displaying vehicle-related information superimposed on a video, it is preferable to display the video in a larger area than when the vehicle-related information is not superimposed on the video.

(26) It is preferable to process the video data for the above display so that the image of the car is displayed within the frame, and information about the acceleration of the vehicle is displayed within the frame in which the image of the car is displayed.

Information on the acceleration of the vehicle is displayed within the frame where the image of the car is displayed, making it relatively easy to understand the acceleration of the vehicle. It is preferable to display the acceleration in the direction of travel of the vehicle on the front side of the vehicle represented by the image. Furthermore, it is preferable to display the lateral acceleration of the vehicle on the side (left or right) of the vehicle represented by the image. Furthermore, it is preferable to display the vertical acceleration of the vehicle above or below the vehicle represented by the image. It is preferable to display information on the acceleration of the vehicle within the frame in which the image of the vehicle is displayed so that the arrow points in the positive direction of acceleration. For example, when acceleration is occurring in the direction of travel, the arrow points in the direction of travel, when acceleration occurs in the backward direction, the arrow points toward the rear, and when acceleration occurs in the left direction, the arrow points toward the left. When acceleration is occurring in the right direction, the arrow points to the right, when acceleration occurs in the upward direction, the arrow points upward, and when acceleration occurs in the downward direction, the arrow points downward. It's good to do that.

(27) In the above system whose performance does not satisfy a predetermined condition, it is preferable to process the video data for the display so as to notify that the system does not have the ability to process the video data for the display.

It can be seen that there is no ability to process video data for display. In a system whose performance does not meet predetermined conditions, for example, it will be possible to notify that there is no problem with the video or the device that takes the video because it does not have the ability to process video data. For systems in which the computer or CPU (Central Processing Unit) of the system does not satisfy a predetermined condition, it is preferable to notify that the system does not have the ability to process video data for display. The predetermined condition may be a performance that causes dropped frames in the process of displaying images shot in a vehicle.

(28) The video data for display may be processed so as to be played back from a predetermined playback position in response to a command to move the playback position of the video.

By doing this, for example, it becomes possible to skip the video portion up to a predetermined playback position and play it back. You can quickly play back the part of the video you want to check. The predetermined playback position may be a position that satisfies specific conditions. The location that satisfies the specific condition may be a location that includes a specific scene or a predetermined location. For example, the location where a specific scene is included may be determined by analyzing a video shot in a vehicle, such as the location of a scene that includes a specific subject such as a specific person, object, or building. It is preferable to determine the position where a specific scene is included by video analysis in advance before playing the video. In particular, it is advisable to store the determined position in association with the video. The predetermined position may be determined based on information regarding the vehicle. For example, it may be the location of a scene where a specific situation that occurred in the vehicle, such as sudden braking or sudden start, occurs. Information about the vehicle includes information obtained by the vehicle that captures the video (for example, map information indicating the location of the vehicle, vehicle acceleration information, speed information, information on when the brake is pressed, information on when the turn signal is activated, etc.) information).

(29) The video data for the above display may be processed so as to display the current playback position and the predetermined playback position.

The current playback position and the predetermined playback position are known. When reproducing a video according to a predetermined reproduction command in response to a video movement command, it is possible to know which portion of the entire video is to be reproduced. For example, it is preferable to display a playback bar, a seek bar, etc. indicating the playback position of the video, and to display a mark such as a line indicating a predetermined playback position on such a bar.

(30) Change the display mode of a predetermined playback position determined based on video analysis and the display mode of a predetermined playback position determined based on information about the vehicle obtained at the time of shooting the video. It is preferable to process the video data for the above display.

By looking at the display of the playback position, it can be determined whether the playback position is a predetermined playback position determined based on video analysis or a predetermined playback position determined based on information about the vehicle obtained when the video was shot. The predetermined playback position determined based on video analysis may be, for example, a position obtained by analyzing a video scene that includes a scene of a predetermined person or building. The playback position determined based on information about the vehicle is determined from so-called historical information such as map information indicating the vehicle's position, vehicle acceleration information, speed information, information on when the brake was pressed, and information on when the turn signal was activated. It is best to choose a position where it can be obtained. The information regarding the vehicle may be acquired at the time of shooting the video, and may be obtained separately from the video. For example, the color (e.g., red) and shape (e.g., triangle) of a mark displayed at a predetermined playback position determined based on video analysis, and the mark displayed at a playback position determined based on information about the vehicle. It is preferable to change the color (for example, green) and shape (for example, circular) of the mark displayed.

(31) It is preferable to process the above-mentioned video data for display so as to restrict the transmission of video data representing images taken in a vehicle.

By doing so, when a function for transmitting video data is provided, the amount of video data to be transmitted can be relatively reduced. When the system is installed in a vehicle and the system installed in the vehicle transmits video data, it is preferable to restrict the transmission of video data. Furthermore, when transmitting video data from a system installed in a vehicle through a telephone line, it is preferable to restrict the transmission of video data. It is preferable to limit the transmission of video data by limiting the number of transmissions. It is better to detect whether the number of transmissions has been reached. Furthermore, it is preferable to limit the transmission of video data by limiting the amount of video data to be transmitted. It is a good idea to detect whether the data limit has been reached. When the transmission of video data is restricted, it is preferable to transmit the video data using a transmission method that does not require transmission costs or is inexpensive.

(32) It is preferable to process the video data for display as described above so as to detect the amount of video data transmitted representing the video shot in the vehicle.

The amount of video data sent can be detected. When transmitting video data representing video captured by each of a plurality of vehicles from each vehicle, it is preferable to detect the amount of video data transmitted from each vehicle. It is preferable to have a function to display the detected transmission amount. In particular, it is preferable to have a function of displaying a list of transmission amounts of each of a plurality of vehicles. It is preferable to display the list in descending order of transmission amount or in order of transmission amount. For example, you can get a ranking of the amount of transmission and find out which vehicles transmit the most amount. In a company that has multiple vehicles, it is preferable to detect the amount of video data that is transmitted from each vehicle, representing video captured by each of the multiple vehicles. Among the vehicles owned by the company, it can be seen which vehicle sends the most amount of video data.

(33) Perform a communication speed test before recording video data representing images taken in a vehicle, and process the video data for the above display so that the video data is transmitted under the conditions obtained based on the communication speed test. It's good to do that.

Video data can be transmitted under conditions obtained based on communication speed tests. For example, it is preferable to determine the video recording resolution, frame rate, bit rate, etc. based on the transmission speed based on a communication speed test, and then transmit the video data. If a communication error occurs during communication, it is recommended to perform the communication speed test again, change the conditions, and then transmit the video data. The process of transmitting the video data under the conditions obtained based on the communication speed test may be a process of processing the original video data so that the rate of the video data is lowered as the communication speed becomes relatively slower.

(34) When an image taken in a vehicle differs from an image that can be visually recognized by a human, the above-mentioned display is displayed so as to at least either notify or record that there is an image that is different from an image that can be visually recognized by a human. It is recommended to process video data for

By doing this, it can be seen that the captured video is different from the video that humans can actually see. For example, it can be seen that the image that was visible to the driver of the vehicle and the recorded image are different. Therefore, for example, when a video of an accident or the like occurring is played back, it can be seen that the video may not have been visible to the driver. As a result, for example, the police, court, insurance company, etc. can use this notification or recorded information to determine whether or not the driver was negligent at the time of the accident. For example, if a polarizing filter can be attached to a camera that takes images, it is possible to record or record the image so that it can be seen whether the image was taken with the polarizing filter attached or without the polarizing filter attached. It is preferable to perform at least one of the notifications. For example, it is preferable to install a switch on the camera that takes the image and detect the state of the switch to determine whether the image was taken with a polarizing filter attached. Furthermore, it is preferable to detect whether the image was shot with a polarizing filter attached by analyzing the shot image. For example, it would be a good idea to analyze the video and detect whether the video was taken with a polarizing filter attached, based on the reflection of the dashboard on the window glass.
In addition, for example, when the image taken in a vehicle is not an image that can be visually recognized by a human being, the video data for the above display may be processed so as to notify that the image is not obtained in a state that is visible to a human being. You may also do so.

(35) It is preferable to process the video data for the above display so as to communicate with a videophone device.

The system can now be used as a videophone device. When the system is installed in a vehicle, the system installed in the vehicle can be used as a videophone device. The system is preferably equipped with a function for transmitting and receiving video data to and from a videophone, and a function for transmitting and receiving audio data to and from the videophone. It is preferable that the system is equipped with a function for recording onto a recording medium, and audio data, video data, etc. indicating the contents of a videophone call are recorded on the recording medium.

(36) It is preferable to process the video data for the above display so that a plurality of videos related to the movements of a specific person are displayed in succession.

By doing so, multiple images related to the movements of a specific person can be displayed in succession. For example, a series of videos of a specific person, such as footage before a specific person gets into a vehicle and drives the vehicle, footage of the specific person driving the vehicle, and footage after the specific person drives the vehicle, are It is best to display the video continuously as a video representing the action. You can check whether a specific person is behaving appropriately. For example, it would be good to display a driver's movements throughout the day in succession, such as a video of him before leaving the office, a video of him driving, and a video of him returning to the office. For example, when multiple videos are represented by multiple video data, those multiple video data are identified by an ID that identifies a specific person, and the multiple data identified by that ID are arranged in the order of recording time of the video data. It is best to display them consecutively.
The function of continuously displaying multiple videos related to the movements of a specific person is often used to identify a person through image recognition, such as whether a person in a captured video resembles an image of a person registered in advance. It is often possible to play back parts of a video in which a specific person is shown in succession, or to play back different files of the video in which the specific person is shown in chronological order, or to It is often useful to extract and play a portion of a single file of a video, or to play a combination of these, especially when video files shot at different times of the day are played continuously. It is a good idea to process it.

(37) Display the video data representing the video shot in the vehicle so that the video data recorded on the first recording medium and the video data recorded on the second recording medium are different video data. It is recommended to process video data for

By doing this, different images can be obtained. The video data recorded on the first recording medium and the video data recorded on the second recording medium are preferably shot at the same time. For example, video data representing a processed image of the celestial image may be recorded on a first recording medium, and video data representing an image closer to the raw image than the processed image may be recorded on a second recording medium. The processed video is preferably a partially cut out video or a coordinate transformed video. A video that is close to a raw video is preferably a RAW video or a compressed video. It is preferable that video data representing the front video is recorded on the first recording medium, and video data representing the rear video is recorded on the second recording medium. It is preferable that the video data recorded on the first recording medium and the video data recorded on the second recording medium are reproduced with their shooting times synchronized. It is preferable that the video represented by the video data recorded on the first recording medium be displayed in the first window, and the video represented by the video data recorded on the second recording medium be displayed in the second window. . It is preferable to synchronize and display the video displayed in the first window and the video displayed in the second window. Information about what kind of processing was performed on the processed video, such as the frame extraction range applied to the RAW video, contents of coordinate system transformation, etc., is recorded, and the range of the processed video that corresponds to the RAW video is recorded. It is best to display it as a broken line. When the system is a drive recorder installed in a vehicle, it is preferable that the drive recorder generate a processed video and a video that is closer to the raw video than the processed video. Further, when the system is a personal computer or the like rather than a drive recorder installed in a vehicle, it is preferable that the personal computer generates a processed video and a video that is closer to the raw video than the processed video.

(38) A button is provided in the system that gives a command to upload to the server, and the video data for the above display is uploaded to the server in response to a command from the button. It is better to process it.

In response to commands from the button, video data representing images taken in the vehicle can be uploaded to the server. The system is preferably installed in the vehicle. If the system has a function to record video data, it is a good idea to send the video data to the server before recording. Also, it is preferable to send the video data to the server while recording. Furthermore, it is preferable to read out the recorded video data and send it to the server. It is preferable to record the position of the video data when the command from the button was given, and display the position where the command from the button was given along with a map using an icon or the like when playing the video data. When such an icon is clicked, the video represented by the uploaded video data may be displayed. It is preferable to display a video represented by video data stored in a storage location specified by a URL (uniform resource locator).

(39) It is preferable to process the video data for display as described above so that the video is shot in the vehicle based on a shooting command from outside the vehicle where the video is shot.

Photography can now be performed in the vehicle based on a photographing command from outside the vehicle. For example, when the system and a mobile phone that can communicate with the system are both in the vehicle, a server outside the vehicle sends a shooting command to the mobile phone, and the mobile phone that receives the shooting command is placed inside the vehicle. It is preferable to send an imaging command to the system, and the system that receives the imaging command to start imaging. For example, a smart speaker is installed at the business office of a company that owns a vehicle, and the smart speaker is used to send trigger data for a shooting command to a specific driver's cell phone. It is preferable to send the trigger data for the photographing command from the telephone to the system installed in the vehicle driven by the particular driver. When recording video data obtained by shooting based on trigger data, it is recommended to set a trigger name. It is preferable to display the set trigger name when playing the video. For example, the trigger name may be displayed as "Voice photography trigger from manager A to driver X". For example, a lot of trigger data is sent from various managers to a popular driver or a driver who drives dangerously.

(40) The video data for display may be processed so as to display the recording frame rate and the display frame rate of the video shot in the vehicle.

You can see the frame rate of video recording and the frame rate of display. A frame complement function may also be provided. Data representing the frame rate of the video is preferably recorded in association with the video data.

(41) The video data for display may be processed so as to give operation commands to the robot based on the playback of the video taken in the vehicle.

The robot will move based on the video playback. For example, while replaying a scene where an impact occurs, it would be a good idea to give a robot, such as a communication robot, a command to move so that it is likely to fall over. The robot behaves like it's about to fall over. It is best to give a motion command that causes the robot to fall over. It is also a good idea to give a motion command for a fallen robot to get up. Furthermore, it would be good if the robot could give warnings about driving, such as scolding the driver by saying, ``Don't drive dangerously,'' or giving movement commands. It would be a good idea to keep a robot at a driving school or other place and have the robot scold the child instead of the driving school instructor. It would also be a good idea to have robots at bus companies and transport companies and have them scold them in place of the bus company's and transport company's operation managers. If the operation manager scolds the robot, there will be an uproar, but if the robot scolds the robot, it won't have much of an effect on human relationships. When you want the robot to perform an action, the system sends an action command to the robot that describes the desired action, and the robot receives the action command to cause the robot to operate.

(42) The video data for display may be processed so as to output a file for presentation software regarding the video shot in the vehicle.

You will get the presentation software file. There is a file based on the presentation software, and it is good to output a file that evaluates the driving content of the vehicle based on the video and displays the score of the vehicle's driving.

(43) The image taken in the vehicle is a spherical image, and it is preferable to process the video data for the above display so that the spherical image is displayed in the virtual reality space.

You can view spherical images in virtual reality space. It is recommended that spherical images be captured using a camera that can capture images in all directions: upward, downward, left, right, forward, and backward.

In the virtual reality space, it is preferable to arrange a screen such as an application for playing video, a PC viewer, etc. as part of a plane (board) or a gentle cylindrical surface (or a gentle spherical surface) in a three-dimensional space. can. When the two-screen display button on the PC viewer is pressed, it is preferable that a display is created with the PC viewer operation section on one side and the playback video on the other side.

It is advisable to provide a display button that expands (texture mapping) a spherical image over the entire interior of the sphere, starting from the place where the person is located in the VR space. By pressing this button, you can view the video as if you were actually inside the car. It is best to display the ball at the back of the window. Further, it is preferable to hide all windows so that the display state is as if the user is completely inside the car. It would be a good idea to be able to switch between the two companies using a button on the controller. It is also a good idea to display the ball in front of the window. It is a good idea to change your position to the outside of the ball, move forward, go outside the ball, and operate the operation window. For example, it is a good idea to expand the image outside the sphere while it is outside the sphere. In this way, the image can be viewed from outside the sphere. Instead of developing it into a perfect sphere, it is better to map it onto a curved surface that is set to correct distortions in optical systems such as lenses. It is preferable to have a function to make the spherical surface on which the image is projected semi-transparent. It would be a good idea to make it semi-transparent by pressing a button on the controller. It is best to change the origin position and the radius of the sphere by moving the left and right sticks of the controller up and down. Thumbnail displays, map displays, acceleration graphs, etc. may be arranged as independent panels in the three-dimensional space inside the sphere. It is preferable to configure the playback control functions such as playback, stop, and next event using a controller.

(44) A system is installed in the vehicle, and the video data for display is preferably processed so that the system operates based on audio data transmitted from outside the vehicle.

By doing so, the system installed in the vehicle can be operated based on audio data transmitted from outside the vehicle.

It is a good idea to operate a system such as a drive recorder using a smartphone etc. in the vehicle. If the vehicle is near your home and you can receive radio waves such as WiFi, you can operate the vehicle by transmitting voice data from a smartphone, smart speaker, communication robot, dedicated device, etc. in your home. When you say "Stream video from my dash recorder" to a voice recognition device (smartphone, smart speaker, communication robot, etc.), the dash recorder will turn on and the video from "your" dash recorder will be streamed via wireless such as WiFi. It is a good idea to play the video on a TV. If the vehicle's engine is off, it is best to use battery power.

When you say, "Play the video from yesterday's drive recorder," search for yesterday's drive recorder video on the drive recorder or the storage device (smartphone or PC) that stores the data already transferred from the drive recorder, and if there is one, play it. For example, you can say, ``We couldn't find yesterday's drive recorder footage.'' It is a good idea to play back the video from the closest drive recorder instead, such as "I will play the video from the drive recorder from the day before yesterday instead." If you say, "Play the video from the dash recorder you just called," it will play the most recent video from the dash recorder.

When playing back the drive recorder, it is preferable to play back the audio recorded along with the drive record video, but it is also possible to provide a function to play back other music or the like as BGM. If you say, ``Set the background music to ___,'' it would be a good idea to change it to the drive recorder audio and play the song along with the video on a music streaming service such as Spotify. It is best to perform digest playback, focusing on the parts where events occur. It would be good to be able to specify the playback time, such as ``Play the video on the drive recorder in about 5 minutes.'' At this time, for example, the videos may be played back at a specified time. It is preferable that unplayed videos are played back with priority.

When a voice saying "Save this video" is recognized while the video on the drive recorder is "playing", the video file being played is transferred from the drive recorder to a storage device (smartphone, PC, etc.) via WiFi etc. . You can also upload it to video posting SNS and share it.

When a voice saying "Save this video" is recognized while the video on the drive recorder is "recording", the record is transferred from the drive recorder to a storage device (smartphone, PC, etc.) via WiFi etc. from a predetermined time before that point. It is preferable to record the video on the drive recorder using the already recorded video and the video data from that point in time until a predetermined time later as an audio instruction trigger.

When a voice saying "Send video" is recognized while the drive recorder is "recording" video, the recorded video from a predetermined time before that point and the video data from that point are sent to the server, etc. using LTE, etc. It's a good idea to send it. It is a good idea to upload and share the video on SNS. When the voice saying "Stop video transmission" is recognized, it is preferable to stop transmitting the video to the server or the like. If a voice saying ``Send me the video up to this point'' is recognized while the video on the drive recorder is ``recording,'' it is a good idea to upload the recorded video from a predetermined time before that point to the video server using LTE, etc. It is a good idea to upload and share the video on SNS. When you say ``Start live streaming'' in front of the drive recorder, it is a good idea to start live distribution of the drive recorder video.

(45) There are multiple videos taken in the vehicle, and it is preferable to process the video data for the above display so that the display mode of the thumbnail image is changed for each video.

By doing this, when there are multiple videos, the display mode of the thumbnail image can be changed for each video.

It would be good to have a function to change the thumbnail display method and display range "for each individual thumbnail." For example, when you right-click on an individual thumbnail, a pop-up menu may be displayed that allows you to select how to display the clicked thumbnail. The types of thumbnail display methods may include, for example, fisheye display, panoramic display, enlarged flat display, ring-shaped display, and dome-shaped display. It is also recommended to provide original, twin cut, panoramic cut, and angle cut. You can change the video display range by left-clicking on each thumbnail. It is a good idea to provide menus for "rotate 180 degrees" and "rotate 90 degrees" in the right-click menu. In the case of two cameras, it is recommended to provide "camera switching".

It is preferable to have a function to save the changed thumbnail display method and display range.It is preferable to configure the system so that the changed state can be maintained. It would be good to have a function to undo (cancel) and redo (re-execute) changes in the thumbnail display method and display range. It would be good to have a function that allows the thumbnail display method and display range to be the same among thumbnails. It is a good idea to draw a selection frame around the thumbnail when left-clicked while holding down the shift key. When there are multiple selection frames, right-click any thumbnail and change the display method selected in the pop-up menu for all thumbnails with selection frames. Also, when there are multiple selection frames and you drag one thumbnail to the left, it is a good idea to change the status of all the thumbnails in the range displayed by that thumbnail in real time. Note that the selection frame is preferably released (selected state is released) when the left click on a portion other than the thumbnail is released. It is preferable to provide a row for displaying thumbnails that are taken at the same time (for example, the same frame) but have different thumbnail display methods or display ranges. For example, with a 360 degree x 2 camera, a row of first thumbnails with a 360 degree front fisheye display and a second row of thumbnails with a 360 degree rearward fisheye display are created, so that thumbnails from the same time are in the same column. It is best to draw it in It is preferable to have a function that allows the number of thumbnail rows to be expanded arbitrarily. To add more thumbnails, create a space between the lines in the thumbnail display area, right-click the space, and select "Insert Row" from the pop-up menu that appears. It is also a good idea to provide items such as "Delete top row" and "Delete bottom row" in this pop-up menu. It is recommended that rows or columns that exceed the display range of the screen can be scrolled using a scroll bar. For example, create 10 thumbnail rows with a 360 degree x 2 camera, and from the top, front fisheye display, rear fisheye display, panoramic display centered on the front, panoramic display centered on the back, panoramic display centered on the bottom. Enlarged flat display, enlarged flat display with the upper part stopped, ring-shaped display with the front as the most visible part, ring-shaped display with the back as the most visible part, dome-shaped display with the front as the most visible part, back It is best to display it like a dome display with the most visible part. These types of displays at the same time will be displayed in the same column. You can also scroll left and right to move the columns left and right, allowing you to see thumbnails of the entire page.

It is preferable to provide a function to play back the image of a thumbnail when a thumbnail is selected by left-double-clicking or the like, and it is particularly preferable that the playback starts in the display method and display range set as the thumbnail. In the pop-up menu that appears when you right-click in the playback window, there is an option to "Set this state as thumbnail." When this is selected, the display method and video within the display range at that time are set as the thumbnail of that video. It would be good to have a function to do so. Thumbnail display is a function that creates an area to display the original image corresponding to the right-clicked thumbnail, and clearly indicates the display area of the current thumbnail in that original image (surrounding it with a line, filling it with semi-transparent color, etc.). It is a good idea to provide In this way, it is possible to easily grasp which part of the original image is set as the thumbnail.

In relation to "Improving image quality and connection accuracy," it would be good to have a function that clearly indicates "parts where images are connected." For example, you can draw a red line on the stitched part. As the stitching accuracy increases, the joints become harder to see. You don't want to use seams in commercial photography, but if it becomes difficult to see, you might end up cropping across the seams. It may be a problem if a broken joint is discovered after delivery, such as after commercial printing, but such problems can be prevented.

In relation to ``reducing reflections on the main body'', it is good to have a ``function that clearly shows the reflected parts of the main body''. For example, the area corresponding to the main body portion may be shaded or filled in semi-transparently. In connection with "automatic face recognition & blurred shooting", it is recommended to provide "an area for displaying whether or not the image has been blurred" on the viewer screen. It is preferable to display an icon indicating that the video being played is a blurred image. Additionally, it is preferable to display an icon or the like indicating that the image is a blurred image in the list of file names. Related to "wireless live streaming", it has a function to record along with live streaming, stores information that allows you to know which recorded video is being streamed, and performs live streaming when playing recorded video on a viewer. It is preferable to have a function to display whether or not it has been done. It is a good idea to simply record what was live-streamed and what was not as separate files, and to store information as to whether or not it was live-streamed in the file properties. In particular, within one file, it is preferable to record the live streaming range and the non-live streaming range in a distinguishable manner, and display the two areas on the timeline so that they can be distinguished. In connection with "time-lapse photography," it would be a good idea to add information that identifies the video as time-lapse photography and record it, and to have a function that displays whether or not it was time-lapse photography when played back on the viewer. . It would be good to have a function that handles a series of time-lapse images as one set. Regarding the "SmartDevice", it would be good to have a function to add and record information on the video, such as from which device the recording start/stop recording instruction/setting instruction came, and a function to display this information on the viewer. It would be good if the "SmartDevice" had the ability to use voice recognition to operate the parts of the viewer that correspond to buttons, especially the ability to change the display method and display range. It is also preferable to have a function to change the recording method and recording range using voice recognition.

(46) When communicating video data representing images taken in a vehicle using a device installed in the vehicle, in addition to changing the communication speed depending on the type of communication contract, it is necessary to communicate using a different communication method. It is preferable to process the video data for the above display.

By doing this, it becomes possible to communicate using different communication methods depending on the type of communication contract.

For example, in a drive recorder for corporations, it would be good to have a function that supports various communication rate plans depending on how the corporation uses the vehicle.

Examples of features that would be good for low-speed, flat-rate, unlimited services that cost 300 yen to 500 yen per month include the following. For example, it may be a function that supports SIMs that cannot perform high-speed communication and have a speed of about 200 kbps. With 200 kbps, it is possible to send audio and still images (jpeg) at speeds faster than real time. Even at monaural 64 kbps, the sound quality is sufficient to determine whether or not the sound is an accident sound. Therefore, past data can be transmitted in a shorter time than real time. In addition, if you use a function that calculates the bit rate for a shorter time than the actual time by combining the past data from the current (recent history, etc.) communication bit rate or SIM contract bit rate, compresses it at that bit rate, and sends it. good. Furthermore, it would be good to have a function that sends the past part as a compressed file and the current part by streaming (IP phone may also be used). Furthermore, it is best to use a SIM that has a data limit of up to a certain amount of data for a certain period of time, such as 366MB for three days. For example, it would be good to have a function that displays the remaining transferable capacity on the drive recorder and viewer. As a display example, it is recommended to use 10MB of the 366MB used (356MB remaining until December 26th). As you near your data limit, it's a good idea to limit the types of information you send. For example, it would be better to change so that only audio data is sent without sending photo data. A low-speed dedicated SIM that can perform burst transfer (burst function: outputs about several Mbps for the first 1-2 seconds) is capable of sending several photos (or even just one photo) at different points in time related to the event. ), then send the audio data. It is best to transfer photos in bursts.

The following functions may be suitable for the night-time special plan. An example of a ``night-specific plan'' is that between 6:00 a.m. and 10:00 p.m., the maximum download speed is 200 kbps, but during the night between 10 p.m. and 6:00 p.m., there is a plan that allows high-speed communication with a maximum download speed of 375 Mbps and a maximum uplink speed of 50 Mbps. be. It would be good to have a function that allows you to set times when high-speed communication is possible or times when communication is restricted, and to upload videos all at once during times when high-speed communication is possible. In particular, it would be good to have a function that automatically turns on the power to the drive recorder of a parked vehicle with the engine off during times when high-speed communication is possible, and starts transmitting video data.

The following functions may be used to support the uplink specialized plan. An example of an "uplink specialized plan" is to keep the data communication speed of downlink communication as low as a maximum of 200 kbit/s, but instead ensure the data communication speed of uplink communication for transmitting data at a maximum of 50 Mbit/s. I have a plan. Since it seems possible to secure the downlink IP phone bandwidth, it would be a good idea to have a function from the server that limits the downlink communication to instruction commands and IP phone communication. In other words, even if a drive recorder has a function of receiving video from another drive recorder, for example, it is preferable to limit that function.

The following functions may be suitable for corporate share plans. Especially for corporate plans, it is recommended to use a data sharing plan that allows 50 GB to be shared among 20 lines, for example.

It would be a good idea to have a function to obtain information from the management screen provided by the SIM provider via scraping or API, count the SIM usage status on the drive recorder, obtain this information at the center, and display it on the viewer. It would be good to have a function to compare and display information. It is advisable to provide a function that displays on the viewer the total amount of packet usage and remaining usable amount of the drive recorder that is sharing packets. It would also be helpful to have a function that issues an alert when the remaining capacity is getting low, and a graph of predicted capacity usage. It would be good to have a function that displays the individual packet usage and remaining usable amount of packet sharing devices in a ranking format on the viewer. For example, by looking at the event status of a vehicle with a higher-ranked drive record, it is easy to verify whether the driver is driving poorly or if the drive record settings are incorrect. For companies that have set up multiple sharing groups, it would be a good idea to have a function that allows you to compare and display the usage status among the sharing groups. In this way, it can be used as a reference for optimizing the SIM allocation of each drive recorder to each group. It is also possible to use automatic analysis using AI to issue advice to move a specific SIM to another group. It is a good idea to display the file capacity in the viewer's display target selection field.

When using an SMS-compatible SIM, you can do the following. It would be good to have a function that, when a trigger occurs, generates a string with the importance of the trigger factor, such as information about the degree of maximum G, and sends it to the center phone number via SMS. It is preferable that the center (for example, a viewer) determines whether or not to send the transmission request instruction to the drive recorder based on the SMS data. It is preferable that the center determines information such as the type of data to be transmitted, such as video, and the transmission range to the drive recorder, and sends a transmission request instruction to the drive recorder.

If your SIM supports voice calls, you can do the following. It would be a good idea to equip the drive recorder with a voice call function and equip the center with a function to transfer outside calls made to the center (sales office). The center should have a function that obtains the operating status of the drive recorder using LTE, etc., and prevents outside line transfers under set conditions, such as when the vehicle is in motion. It is also a good idea to equip the drive recorder with a function that calls the center using the appropriate method based on the situation of the drive recorder, such as comparing charges between IP phones and voice calls, the other party to call, past driving history, etc. It would be good to have a function that allows calls to be made by voice in the event of a strong shock, or by IP phone in the case of a weak shock, and a function to communicate with the center using VoIP and connect to an outside line by telephone.

When using public wireless LAN and LTE together, it is recommended to do the following. It is equipped with the ability to connect to public wireless LANs, sending restricted data using LTE, and sending unrestricted or loosely-restricted video data using public wireless LANs. It would be good to have a function that allows you to upload from convenience stores' free WiFi, etc.

(47) It is preferable to process the video data for display so as to obtain information regarding the posture of the device held by the user's hand, and change the display range of the video data based on the information regarding the posture.
In this way, the user can change the display range of captured video data by changing the posture of the device held in hand. In particular, the device is a smartphone, the image is from a celestial camera, and when the smartphone is tilted or rotated based on information from the smartphone's gyro and G sensor regarding the viewpoint of the celestial camera, the viewpoint of the image is switched in that direction. good.
(48) The program may be readable by a computer that controls a computer of the system described in any one of (1) to (47) above.

By processing video data for display, it is possible to provide something better than before.

The displayable system may be, for example, a personal computer, a smartphone, a drive recorder, or the like. Processing of video data for display includes, for example, processing for video playback, processing for video data transmission, processing for adjusting video brightness, processing for changing the video playback format, and reducing the amount of video data. It is better to use processing.

(49) A method for controlling a system capable of displaying images shot in a vehicle, which may include a means for processing image data for the display.

By processing video data for display, it is possible to provide something better than before.

The displayable system may be, for example, a personal computer, a smartphone, a drive recorder, or the like. Processing of video data for display includes, for example, processing for video playback, processing for video data transmission, processing for adjusting video brightness, processing for changing the video playback format, and reducing the amount of video data. It is better to use processing.

The inventions shown in (1) to (47) above can be combined arbitrarily. For example, at least a part of the structure of at least one invention described in (2) and subsequent ones may be added to all or part of the structure of the invention shown in (1). In particular, it is recommended that the invention be an invention in which at least a part of the structure of at least one invention following (2) is added to the invention shown in (1), (48), or (49).

Alternatively, arbitrary configurations may be extracted from the inventions shown in (1) to (47) and the extracted configurations may be combined. The applicant of this application intends to obtain rights to the invention containing these features.

According to this invention, by processing video data for display, it is possible to provide something superior to the conventional art.

The effects of the invention of the present application are not limited to these, but the effects obtained from the parts of the structure disclosed in the specification, drawings, etc. are also disclosed, and the rights to the structure that produces the effects have also been acquired through divisional applications, amendments, etc. have the intention to For example, in this specification, a portion where it is written as “can be done” is a description that clearly indicates the effect to be achieved, and there are also portions that show an effect even if there is no description that “can be done.” Further, even without such a description, there are effects that can be understood from the configuration.

FIG. 3 is a perspective view of the drive recorder seen from the back. This shows the inside of a vehicle equipped with a drive recorder. FIG. 2 is a block diagram showing the electrical configuration of a drive recorder. 3 is a time chart of event recording video data and transmission video data. It is a flowchart which shows a transmission processing procedure. This is an example of a recording setting window. 1 is a block diagram showing the electrical configuration of a personal computer. FIG. 3 is a flowchart showing a processing procedure of a personal computer. 3 is a flowchart showing a reproduction processing procedure. 3 is a flowchart showing a brightness correction processing procedure. It is a flowchart which shows the brightness stop processing procedure. This is an example of a playback window. This is an example of a playback window. This is an example of an image before brightness correction. This is an example of an image after brightness correction. This is an example of an image before brightness correction. This is an example of an image after brightness correction. This is an example of an image before brightness correction. This is an example of an image after brightness correction. This is an example of a brightness correction curve. This is an example of a brightness correction curve. This is an example of a processing procedure for displaying a separate window. This is an example of a playback window. This is an example of a playback window. This is an example of a processing procedure for displaying an image on a separate display screen. This is an example of a playback window and a video display window displayed on separate display screens. This figure shows how to use a playback window and a video display window that are displayed on separate display screens. It is an example of a flowchart showing a reproduction processing procedure. It shows the relationship between vehicle speed, playback speed, and video time. It is an example of a flowchart showing a reproduction processing procedure. It is an example of a flowchart showing a video analysis processing procedure. It is an example of a flowchart showing a video analysis processing procedure. It is an example of a flowchart showing a reproduction processing procedure. It is an example of a flowchart showing a reproduction processing procedure. It is an example of a flowchart showing a processing procedure of a personal computer. This is an example of a playback window. This is an example of a playback window. This is an example of an acceleration graph. This is an example of a playback window. This is an example of a playback window. It is an example of a flowchart showing a processing procedure of a personal computer. It is an example of a flowchart showing a video analysis processing procedure. It is an example of a flowchart showing a processing procedure of a personal computer. This is an example of a playback window. It is an example of the flowchart which shows the 1st skip playback processing procedure. It is an example of the flowchart which shows the 2nd skip playback process procedure. It is an example of the flowchart which shows a transmission processing procedure. It is an example of the flowchart which shows a ranking display processing procedure. This is an example of a ranking display. It is an example of the flowchart which shows a recording processing procedure. It is an example of the flowchart which shows a recording processing procedure. It is an example of a flowchart showing a reproduction processing procedure. This is an example of a flowchart showing a videophone recording processing procedure. It is an example of a flowchart showing a reproduction processing procedure. It is an example of the flowchart which shows a recording processing procedure. It is an example of a flowchart showing a reproduction processing procedure. It is an example of the flowchart which shows a transmission processing procedure. It is an example of a flowchart showing a reproduction processing procedure. It is an example of the flowchart which shows the processing procedure of a smartphone. It is an example of the flowchart which shows the processing procedure of a reproduction|regeneration apparatus and a robot. It is an example of a flowchart showing the processing procedure of the virtual reality device. It is an example of a flowchart showing the processing procedure of a drive recorder, a smartphone, and a television device. This is an example of a processing procedure for displaying thumbnail images. This is an example of thumbnail image display. (A) to (E) are examples of thumbnail images.

[First example]
1 to 6 show a first embodiment, which concerns a drive recorder.

FIG. 1 is a perspective view of a drive recorder 1 (an example of a system) viewed diagonally from the rear. An SD card insertion slot 10 is formed on one side of the housing of the drive recorder 1. A display 11 is formed on almost the entire back surface of the housing of the drive recorder 1, and a plurality of operation buttons 12 are provided on both left and right sides with the display 11 in between. A joint rail 13 is provided on the top surface of the drive recorder 1. Although not shown in Figure 1, a camera lens is provided on the front of the drive recorder. A DC jack of the drive recorder 1 is formed on the side surface not shown in FIG. 1, and a speaker and an HD (high definition) output terminal are formed on the bottom surface of the drive recorder 1.

A camera including a lens captures an image in front of a vehicle, for example. Further, preferably, images of the interior of the vehicle and the exterior of the vehicle (the front of the vehicle) may be captured. The DC jack is a jack for connecting to a DC power source via a power cable. The SD card insertion slot 10 is used to insert an SD card [this is an example of a memory card, and may be other memory cards such as an xD picture card, or other memory such as a USB (Universal Serial Bus) memory other than a memory card]. It is an insertion port for A speaker outputs sound or voice. The HD output terminal is a terminal for connecting to other information equipment via a cable. The joint rail 13 is used to attach a joint for mounting the drive recorder 1 on a vehicle. The display 11 displays various images such as images captured by the camera of the drive recorder 1. The operation button 12 is used by the user to input various commands to the drive recorder 1.

FIG. 2 shows a view of the vehicle in which the drive recorder 1 is mounted, viewed from the inside of the vehicle toward the front.

A room mirror 4 is provided near the center of the upper part of a windshield 3 of a vehicle. A drive recorder 1 is fixed to the vehicle by a joint at a position on the passenger seat side (on the left side when facing the front in FIG. 2) adjacent to the rearview mirror 4.

A DC jack of a drive recorder 1 is connected to a cigarette lighter socket 5 via a power cable 6. When the accessory power source of the vehicle is turned on, power is supplied from the cigarette lighter socket 5 to the drive recorder 1.

FIG. 3 is a block diagram showing the electrical configuration of the drive recorder 1.

The drive recorder 1 includes a controller 20. This controller 20 has a CPU (Central Processing Unit) 20a that controls the entire operation of the drive recorder 1, a ROM (Read Only Memory) 20b that stores the operation programs of the drive recorder 1, and data etc. temporarily. A RAM (Random Access Memory) 20c for storage and a timer 20d are included. The controller 20 has functions such as a GPS information processing program, a video processing program, and a communication processing program, as will be described later.

The drive recorder 1 also includes a GPS (Global Positioning System) receiver 16, a camera 17, an SD card reader/writer 14, and an acceleration sensor 18. The GPS receiver 16 detects the position of the drive recorder 1, and the position of the vehicle in which the drive recorder 1 is mounted can be known. The camera 17 images the front of the vehicle in which the drive recorder 1 is mounted, as described above. When the SD card 23 is inserted into the SD card slot 10 as described above, the SD card reader/writer 14 reads the data recorded on the inserted SD card 23 and transfers the data to the SD card 23. Write. The acceleration sensor 18 detects the vertical, horizontal, and longitudinal acceleration applied to the drive recorder 1, and can determine the vertical, horizontal, and longitudinal acceleration of the vehicle in which the drive recorder 1 is installed. becomes.

GPS data representing the position of the drive recorder 1 output from the GPS receiver 16, video data representing the video imaged by the camera 17, and a signal representing the acceleration detected by the acceleration sensor 18 are input to the controller 20, respectively. . By executing the GPS information processing program, the controller 20 can record the GPS data output from the GPS receiver 16 onto the SD card 23 inserted into the SD card insertion slot 10. Furthermore, by executing the video processing program, the controller 20 can record video data obtained from the camera 17 in the SD card 23 in association with time.

Furthermore, the drive recorder 1 includes a speaker 15 that outputs the above-mentioned audio and the like, a display 11 that displays images and the like, operation buttons 12, a communication circuit 19, and an LTE module 21.

By outputting audio data etc. from the controller 20 to the speaker 15, audio etc. are output from the speaker 15, and by outputting video data etc. from the controller 20 to the display 11, images etc. are displayed on the display 11. Various commands from the operation button 12 are input to the controller 20.

The communication circuit 19 functions as a communication means for wirelessly communicating with external devices such as servers, personal computers, smartphones, tablet terminals, and the like. The controller 20 has a function of transmitting data such as video to an external device via the communication circuit 19 by executing a communication processing program. As the communication circuit 19, it is preferable to use one that complies with short-range wireless communication standards such as the WiFi standard and Bluetooth (registered trademark). The short-range wireless communication standard can be applied, for example, to communication between work vehicles (forklifts, etc.) operating within a premises and external equipment. Mobile communication system standards can be applied, for example, to communications between vehicles and external devices that move within a wider area.

The LTE (Long Term Evolution) module 21 is a communication circuit that complies with mobile phone communication standards. Instead of the LTE module 21, a communication circuit compliant with other mobile communication system standards such as 4G may be used. The LTE module 21 can transmit video data captured by the camera 17 to an external server or the like.

As will be described later, the operation program of the drive recorder 1 may be recorded in advance on the SD card 23, or may be received by the LTE module 21 via the Internet or the like and recorded on the SD card 23, or may be recorded on the SD card 23. The program may be inserted into a personal computer or the like, connected to the Internet via the personal computer, downloaded from the personal computer, and recorded on the SD card 23.

FIG. 4 is an example of a time chart when an event occurs in a vehicle in which the drive recorder 1 is mounted.

In this embodiment, constant recording and event recording are possible. Continuous recording is one in which recording continues, for example, while the drive recorder 1 is powered on, until the recording area of the SD card 23 for constant recording runs out. If it is set to allow overwriting of video data for constant recording, when the recording area for constant recording runs out, the video data obtained by constant recording will overwrite the recording area for constant recording. In event recording, video data obtained by shooting is sequentially stored temporarily in RAM20c, etc. for a certain period of time, such as 20 seconds, and is repeatedly overwritten unless an event occurs, and when an event occurs, it is stored in RAM20c, etc. The temporarily stored video data is read out and recorded in the event recording recording area of the SD card 23. When it is deemed that there has been some change in the vehicle in which the drive recorder 1 is installed (for example, detection of acceleration above the threshold, detection of speed above the threshold, detection of sudden braking, etc.), the drive - When it is assumed that an instruction to record an event has been given to the recorder 1 (for example, an instruction from the operation button 12), an event is generated from the controller 20 and the event recording is performed.

In this embodiment, when an event occurs, video data obtained by the camera 17 during the period before and after the event is recorded on the SD card 23 (a recording medium, which is an example of a portable recording medium). At the same time, the LTE module 21 transmits the information to the server, etc. In particular, in this embodiment, the amount of video data for transmission (the amount of data per unit time) that the LTE module 21 sends to the server etc. is reduced as time passes from the time the event occurs.

When an event (which is an example of a recording command) occurs at time t3 (YES in step 41 in FIG. 5), data is temporarily stored in the RAM 20c from time t1 (for example, 10 seconds before time t3) before time t3 when the event occurred. The stored video data is read and recorded in the event recording area of the SD card 23 as event recording video data. Further, video data obtained by photographing during a period from time t3 to time t5 (for example, 10 seconds after time t3) is also recorded in the event recording area of the SD card 23 as event recording video data. Video data captured during a period from time t1 to time t5 (for example, 20 seconds) before and after time t3 when the event occurs will be recorded in the event recording area of the SD card 23 as event recording video data. The video data for event recording, for example, represents video with a high frame rate and relatively high quality, with a cycle of 30 frames per second, and one frame is based on HD (high definition). Recording (eg, effective vertical resolution of 1080 lines) is performed.

Furthermore, when an event occurs at time t3, the video data temporarily stored in the RAM 20c is read from time t1 before time t3 when the event occurs, and from time t1 until time t2 after time t1 is stored in the RAM 20c. The controller 20 generates video data whose data amount is reduced from the data amount of the video data temporarily stored. The LTE module 21 transmits the video data whose data amount has been reduced to a server etc. as video data for transmission. From time t2 to time t3 when the event occurs, the LTE module 21 transmits the same video data as the event recording video data to the server etc. as the transmission video data. From time t3 to time t4, the LTE module 21 generates the same video data as the event recording video data from the video data captured by the camera 17, and sends the generated video data to the server, etc. as the video data for transmission. Send. From time t4 to time t5, the controller 20 generates video data whose data amount is reduced compared to the data amount of the event recording video data in the same way as from time t1 to time t2, and the video data whose data amount is reduced is generated. The LTE module 21 transmits the data to a server etc. as video data for transmission. The amount of video data for the period from time t2 to t4 before and after time t3 when the event occurs becomes relatively large, and the amount of video data for the period from time t1 to time t2 and the period from time t4 to time t5 becomes relatively large. [For example, it is preferable that one frame of video is produced per second, and one frame has the number of pixels based on VGA (Video Graphics Array).] The amount of video data representing each image is relatively small. From time t2 to time t4, the video data has a larger amount of data than the video data for transmission between time t1 and time t2 or the video data for transmission between time t4 and time t5. If there is, it is not necessarily necessary to use the same amount of data (data amount per unit time) as the video data for event recording as the video data for transmission. Furthermore, the amount of video data for transmission from time t1 to time t2 may be different from the amount of video data for transmission from time t4 to time t5. Furthermore, the amount of video data for transmission from time t2 to time t3 may be different from the amount of video data for transmission from time t3 to time t4.

Videos before and after an event occur are considered to be relatively important, as they may depict the scene of an accident. Since the amount of video data for the period before and after the occurrence of an event is large, it is possible to transmit the video of important scenes and leave important video on a server, etc. Since the amount of data is reduced for video data after a certain period of time has elapsed from the occurrence of an event and video data from a certain period of time before, the transmission time can be shortened, and if there is a cost for data transmission, the cost can be suppressed.

In the example shown in FIG. 4, data before and after the occurrence of an event are generated as the video data for transmission, but data before the occurrence of the event or data after the occurrence of the event may be generated as the video data for transmission. Further, data obtained after a certain period of time has elapsed or before a certain period of time from the occurrence of an event may be generated as video data for transmission. Furthermore, although the period of the video represented by the transmission data and the period of the video represented by the event recording video data are assumed to be the same, they may not be the same. In any case, it is sufficient to reduce the amount of data for which time has elapsed since the time when a recording command was given, such as when an event occurred.

FIG. 5 is a flowchart showing the transmission processing procedure of the drive recorder 1.

When an event occurs as described above (step 41), the controller 20 reads the video data temporarily stored in the RAM 20c, and the controller 20 generates event recording video data as described above. The controller 20 records the generated event recording video data in the event recording area of the SD card 23 (step 42).

The event recording video data recorded on the SD card 23 is read out, and the low data amount video representing the video of the period from time t1 to time t2 and from time t4 to time t5 is generated from the read event recording video data as described above. The controller 20 generates the data and high data amount video data representing the video for the period from time t2 to time t4 as video data for transmission (step 43) (the data amount is larger than the amount of video data recorded on the recording medium). (This is an example of a process for reducing the amount of video data to be transmitted.) The LTE module 21 transmits the generated transmission video data (which is an example of video data with a reduced data amount) to a server or the like (step 45).

In the process shown in FIG. 5, video data for transmission is generated from video data for event recording, but the above-described event recording data and video data for transmission may be generated in parallel. For example, for the period from time t1 to time t3 when the event occurs, event recording video data and transmission video data are simultaneously generated using video data temporarily stored in the RAM 20c, and the event recording video data and transmission video data are generated at the same time. After t3, the video data for event recording and the data for transmission may be generated simultaneously using the video data obtained by photographing with the camera 17.

FIG. 6 is an example of a recording setting window for setting the amount of video data for transmission.

As described with reference to FIGS. 4, 5, etc., in addition to constant recording, event recording and video data transmission are performed. Using the recording setting window 30 shown in FIG. 6, the user makes settings such as constant recording, event recording, and video data transmission.

The SD card 23 to be inserted into the drive recorder 1 is inserted into, for example, an SD card slot of a personal computer (not shown), and the recording setting program recorded on the SD card 23 is read out. Then, a recording setting window 30 shown in FIG. 6 is displayed on the display screen of the personal computer.

The recording setting window 30 includes a recording method selection frame 30a, a recording resolution selection frame 30b, a constant recording frame number selection frame 30c, an event recording frame number selection frame 30d, an event recording time selection frame 30e, a transmission setting frame 30f, and a first frame. It includes a transmission resolution setting frame 30g, a first transmission frame number setting frame 30h, a second transmission resolution setting frame 30i, a second transmission frame number setting frame 30j, a save instruction button 30k to an SD card, and the like. Other frames, buttons, etc. are formed in the recording setting window 30, but their explanation will be omitted.

The recording method selection frame 30a is a frame for setting the type of recording. By pulling down, a menu appears that allows the user to set the type of recording, such as continuous recording only or both continuous recording and event recording, and the user selects the desired recording type. The recording resolution selection frame 30b is a frame for setting the resolution of one frame of continuous recording. By pulling down a menu, a menu appears in which various resolutions can be set, and the user selects the desired resolution. When event recording is also set, the resolution of one frame of event recording is set to be the same as the resolution of one frame of constant recording. Of course, the resolution of event recording may be set higher than the resolution of constant recording. The constant recording frame number selection frame 30c is a frame for setting the number of frames per second (frame rate) in constant recording. By pulling down a menu, a menu appears in which various number of frames can be set, and the user selects the desired resolution. The event recording frame number selection frame 30d is a frame for setting the number of frames per second (frame rate) in event recording. By pulling down a menu, a menu appears in which various number of frames can be set, and the user selects the desired resolution.

The event recording time selection frame 30e is a frame for setting a recording time when an event occurs. By pulling down, a menu appears that allows the user to set various times before and after the event occurs, and the user sets the desired time. It may be possible to set the time to record before or after the event occurs, or to set the time to record from a certain period of time after the event occurs, or the time to record backwards from a certain time before the event. You can. The transmission setting frame 30f is a frame for setting whether or not to generate transmission video data and transmit it to a server or the like as described above. By pulling down, a menu of "valid" or "invalid" appears, and "valid" is selected when transmitting video data is to be transmitted, and "invalid" is selected when not transmitting video data for transmitting. As described with reference to FIG. 4, the first transmission resolution setting frame 30g is a frame for setting the resolution of the video data for transmission during the period from t2 to t4 in the vicinity of the event occurrence, sandwiching the event occurrence time t3. By pulling down, a menu representing various resolutions appears, and the user selects the desired resolution. As explained with reference to FIG. 4, the first transmission frame number setting frame 30h is the number of frames per second ( This is the frame for setting the frame rate. By pulling down, a menu representing various number of frames appears, and the user selects the desired number of frames. The second transmission resolution setting frame 30i is set after (for example, the period from time t4 to t5) or before (for example, time t1 This is a frame for setting the resolution of video data for transmission during the period from t2 to t2). By pulling down, a menu representing various resolutions appears, and the user selects the desired resolution. The second transmission frame number setting frame 30j is set after (e.g., the period from time t4 to t5) or before (e.g., time This is a frame for setting the number of frames per second (frame rate) of video data for transmission during the period from t1 to t2). By pulling down, a menu representing various number of frames appears, and the user selects the desired number of frames. The save to SD card instruction button 30k is clicked to save the contents set as described above to the SD card 23 inserted into the SD card slot of the personal computer.

The contents set using the recording setting window 30 are saved in the SD card 23, and when the SD card 23 is inserted into the SD card insertion slot 10 of the drive recorder 1, the SD card is saved as described above. Event recording video data is recorded in 23, and transmission data is generated and transmitted.

The number of drive recorders for communication is increasing, and when you want to send video data such as 4K or 8K, the communication charges will increase if you send a large amount of video data, so it is necessary to reduce the amount of data as described above. communication charges can be reduced. The drive recorder only needs to send video data of scenes of accidents or dangerous driving, and does not need to send video of normal scenes. If you want to send the video data where the event occurred, the amount of data will increase even if you send only the video data where the event occurred, so you may want to reduce the amount of data further. For this reason, it is best to send video data with a large amount of data before and after the moment when an event occurs so that a clear video is obtained, and to send video data with a small amount of data or coarse video data at other times. An event occurs when the vehicle's acceleration exceeds a threshold, and 20 seconds before and after the event are recorded on the SD card. Since the amount of data increases when you want to send the video immediately on the spot, you can use clear video for the front and back parts, and VGA for the scene where the accident occurred, which greatly reduces the amount of data you need to send. Continuous recording records all the time, and event recording records the 20 seconds before and after the event occurs. Continuous recording overwrites the oldest video data. A constant recording area and an event recording area are separated by the same SD. There are ways to reduce the amount of data when sending it via communication, such as reducing the frame rate, reducing the resolution, and increasing the compression rate.

There are drive recorders that can set event recording and constant recording separately, and drive recorders that cannot. For example, one video data with a large amount of data is set to an effective vertical resolution of 1080 lines, and the other video data with a small amount of data is set to VGA, or data is shot with two cameras and one camera has an effective vertical resolution of 1080 lines. It is also possible to obtain a large amount of video data and obtain a small amount of video data to be converted into VGA by the other camera. The sent data may also be recorded in SD data. Video data for transmission may be generated from video data recorded on the SD card. Constantly recorded data may also be transmitted. Basically, if it is sent via communication, it will be recorded on the server, so there is no need to record it on the SD card 23 of the drive recorder 1.

A backup of the transmitted video data may be recorded on the SD card 23. In addition, information (rules) about how the generated video data was sent as the video data for transmission is stored in the header of each frame of the video data for transmission, or is recorded in the header of the SD card 23. Alternatively, the data generated according to the rules may be recorded in the SD card 23. Information that video data different from the video data recorded on the SD card 23 of the drive recorder 1 is being transmitted is stored in the header of the video file that stores the transmitted video data, the header of the SD card 23, etc. Good too. It is preferable to record video data (raw data) representing a clear image on the SD card 23, or to send processed video data. It is preferable to process the video data for display so as to acquire information regarding the posture of the device held by the user's hand, and change the display range of the video data based on the information regarding the posture. In this way, the user can change the display range of captured video data by changing the posture of the device held in hand. In particular, the device is a smartphone, the image is from a celestial camera, and when the smartphone is tilted or rotated based on information from the smartphone's gyro and G sensor regarding the viewpoint of the celestial camera, the viewpoint of the image is switched in that direction. good.

[Second example]
FIGS. 7 to 21 show a second embodiment, which concerns reproduction of video captured by the drive recorder 1. FIG. In this embodiment, the brightness of the image captured by the drive recorder 1 is adjusted (for example, adjusted brightly) based on instructions from the user. When an image of a bright area (for example, an image of the outside of the vehicle) is captured, the amount of brightness adjustment for the image of the dark area is set to be higher than that of the image of the bright area based on instructions from the user. Make it relatively larger than the brightness adjustment amount.

FIG. 7 is a block diagram showing the electrical configuration of a personal computer 50 (which is an example of a system).

As described above, the video recorded by the drive recorder 1 can be played back using the personal computer 50. However, the data may be reproduced on the drive recorder 1 without using the personal computer 50.

The entire operation of the personal computer 50 is controlled by a CPU (Central Processing Unit) 58.

The personal computer 50 is provided with a display device 52. This display device 52 is controlled by a display control device 51 which is controlled by a CPU 58. The personal computer 50 also includes an SSD (solid state drive) 53 that is accessed by the CPU 58 and an SD card reader/writer that reads data recorded on the SD card 23 and writes data etc. to the SD card 23. Contains 54.

Furthermore, the personal computer 50 includes an input device 55 such as a keyboard and a mouse, a memory 56 such as a RAM, and a communication circuit 57 connected to a network such as the Internet.

An operating program, which will be described later, is received by the communication circuit 57 of the personal computer 50 via the Internet and installed on the personal computer 50. The CPU 58 controls each section by reading and executing the installed program. The operating program is stored in a recording medium such as the SD card 23, and the operating program may be read from such a recording medium and installed in the personal computer 50.

When the SD card 23 removed from the drive recorder 1 is inserted into the SD card slot of the personal computer 50 and the detection program recorded on the SD card 23 is started, the video recorded on the SD card 23 will be displayed. The CPU 58 reads data (for example, constantly recorded video data, event recording video data), and detects a video portion inside the vehicle interior and a video portion outside the vehicle interior. The image portion inside the vehicle interior and the image portion outside the vehicle interior can be determined by, for example, edge detection between the windshield 3 and the dashboard, and image analysis. The CPU 58 records data indicating the boundary between the detected video portion inside the vehicle interior and the video portion outside the vehicle interior, for example, in the header of the video file in which the video data is stored. A detection device (which is an example of a means for detecting) that detects a video portion inside a vehicle interior receives, for example, a selection of a region of a vehicle interior portion in a video from a user, and detects the selected portion as a video portion inside the vehicle interior. Alternatively, for example, the video portion inside the vehicle may be automatically detected. As an automatic detection process, for example, when the movement of the vehicle is detected, parts of the image that change in response to the movement are detected as outside the vehicle, and areas that do not change in the image in response to the movement are detected as the interior of the vehicle. It is recommended that the process be performed as follows. In particular, it is preferable to perform a process of detecting an area in the video where there is movement in accordance with the speed of movement of the vehicle as outside the vehicle, and an area in which there is no movement in accordance with the speed of movement of the vehicle as inside the vehicle.

When the playback program installed in the personal computer 50 is activated by the user, the display control device 51 displays a playback window 100 shown in FIG. 12 on the display screen of the display device 52 of the personal computer 50 under the control of the CPU 58. Display.

Referring to FIG. 12, a menu bar 101 is formed at the top of a playback window 100, and a video display area 111 is formed below this menu bar 101. Furthermore, a map image display area 121, a vehicle information display area 131, and a play list display area 141 are formed below the video display area 111.

The menu bar 101 displays various items, and by selecting each item, a pull-down menu is displayed and detailed commands can be given. The menu bar 101 includes the text strings ``File,'' ``View,'' ``Play,'' ``Tool,'' and ``Information,'' a folder icon 102, a camera icon 103, a printer icon 104, and a reel icon. 105, an FD (flexible disk) icon 106, a log icon 107, and a gear icon 108 are displayed. By pulling down the strings ``File,'' ``Display,'' ``Playback,'' ``Tool,'' and ``Information,'' a pull-down menu corresponding to each item will appear, allowing detailed commands to be given. . A folder is specified by clicking the folder icon 102, a still image is converted by clicking the camera icon 103, printing is performed by clicking the printer icon 104, and a reel icon 105 is clicked. By clicking, video conversion is performed, by clicking the FD (flexible disk) icon 106, the data is backed up, by clicking the log icon 107, it is converted to log data, and by clicking the gear icon By clicking 108, the recording settings are changed.

The video display area 111 is an area for displaying video recorded by the drive recorder 1 or the like. The map image display area 121 is an area that displays an image of a map near the shooting location of the video displayed in the video display area 111. By transmitting data representing the vehicle's position from the personal computer 50 to a map server existing on the Internet, and transmitting data representing a map near the vehicle's position from the map server to the personal computer 50, the map image display area is An image of the map is displayed on 121. The vehicle information display area 131 is an area that displays information such as the speed of the vehicle at the time of video recording. The vehicle information display area 131 includes a vehicle status display area 132 that displays the status of the vehicle to which the drive recorder 1 that recorded the video displayed in the video display area 111 is attached, and a vehicle status display area 132 that displays the status of the vehicle, such as video playback, stop, etc. an operation button 133 that gives an operation command for displaying an image inside the vehicle interior; an interior emphasis button (an example of a first instruction input means) 134 that gives an instruction for processing to brightly display an image inside the vehicle interior; a normal display button 135 (an example of a second instruction input means) for giving instructions for normal display processing; a time display area 136 for displaying the recording time of the video displayed in the video display area 111; It includes an acceleration graph display area 138 that displays the acceleration of the vehicle in a graph form. The play list display area 141 is an area where file names of video files recorded on the SD card 23 are displayed in a list. It is preferable to adjust the brightness of the vehicle interior based on an instruction from the user given by the interior enhancement button (first instruction input means) 134.

When the folder icon 102 is clicked and a folder containing video files is selected, the file names of the video files stored in that folder are displayed in the play list display area 141, as shown in FIG. , the file name of the video file is displayed in a clickable manner. Although the video recording start time is used as the file name, it goes without saying that the file name may be anything other than the recording start time.

FIG. 8 is a flowchart showing the processing procedure of the personal computer 50.

As described above, when the folder icon 102 is clicked and a desired folder is selected, the file name of the video file is displayed in the play list display area 141. When one of the file names displayed in the play list display area 141 is clicked using the mouse included in the input device 96, the file is selected (step 61). The corresponding video file is read from the SD card 23. The reproduction of the video data stored in the read video file is started, and the video is displayed in the video display area 111 as shown in FIG. 13 (step 62). In the header recording area of a video file, various header data are recorded in accordance with the recording time of the video data, and when the video at the recording time is displayed, the recorded header data is also read and read. Various superimposition processes and the like are performed on the video depending on the contents of the output header data. In addition, the map image display area 121 displays an image of a map near the shooting location of the video displayed in the video display area 111, and the vehicle status display area 132 records the video displayed in the video display area 111. The numerical value of the speed, acceleration (for example, acceleration in the longitudinal direction is expressed as an (for example, latitude, longitude), numerical values, status, etc. Further, the acceleration of the vehicle is displayed in a graph form in the acceleration graph display area 138.

FIG. 9 is a flowchart showing the reproduction processing procedure.

When a video file to be played is selected, the first frame of the video represented by the selected video file is displayed as a still image, as shown in FIG. In this state, when the user presses the playback button (triangular button pointing to the right) included in the operation buttons 133 (YES in step 63), video playback of the video represented by the selected video file starts ( Step 64). A video is displayed in the video display area 111.

FIG. 10 is a flowchart showing the brightness correction processing procedure.

When the user presses the indoor emphasis button 134 while the video is being played back in the video display area 111 or the video is being displayed as a still image in the video display area 111 (YES in step 71), the video display In-position brightness correction is performed on the video data representing the video displayed in area 111 (step 72), and the brightness adjustment amount of the video portion of the interior (inside the vehicle) is adjusted to match that of the video outside (outside the vehicle). The CPU 58 processes the video data so that the brightness is relatively larger than the brightness adjustment amount of the part. In general, indoors is darker than outdoors, but if you adjust the brightness so that both the indoor and outdoor video parts are uniformly bright, the bright outdoor video parts may be blown out. In this embodiment, the brightness adjustment amount for the indoor (vehicle interior) video portion is relatively larger than the brightness adjustment amount for the outdoor (outside the vehicle interior) video portion. It is possible to brighten the indoor video area to make it easier to see, while also preventing blown-out highlights in bright areas.

For brightness correction, the CPU 58 may perform digital gain processing to brighten the video data. At this time, the amount of gain that brightens the detected indoor video portion may be made larger than the gain amount that brightens the detected outdoor video portion.

When the normal display button 135 is pressed (YES in step 73), the process shifts to the process shown in FIG. 11, which will be described later.

14 and 15 are examples of videos displayed in the video display area 111, and both are examples of videos taken by the drive recorder 1 during the day.

FIG. 14 is an example of an image before the above-mentioned brightness correction, and FIG. 15 is an example of an image after the above-mentioned brightness correction.

Referring to FIG. 14, a video portion 112a on the left side of the video 112 is an outdoor video portion, and a video portion 112b on the right side of the video 112 is an indoor video portion. The outdoor image portion 112a is relatively bright, whereas the indoor image portion 112b is relatively dark. There is a difference in brightness between indoors and outdoors, making it difficult to see video 112.

Referring to FIG. 15, a video portion 112a on the left side of the video 112 is an outdoor video portion after brightness correction, and a video portion 112b on the right side of the video 112 is an indoor video portion after brightness correction. The outdoor image portion 112a is slightly brighter than before the brightness correction, and the indoor image portion 112b is also brighter than before the brightness correction. Since the amount of brightness adjustment for the indoor video portion 112b is relatively larger than the brightness adjustment amount for the outdoor video portion 112a, the brightness of the outdoor video portion 112a and the indoor video portion 112b are greater than before the brightness correction. The difference in brightness between the two images has become smaller, making it easier to see. In addition, by adjusting the brightness, the outdoor image area is prevented from becoming too bright, and bright areas are prevented from becoming overexposed.

16 and 17 are examples of videos displayed in the video display area 111, and both are examples of videos taken by the drive recorder 1 at night.

FIG. 16 is an example of an image before the above-mentioned brightness correction, and FIG. 17 is an example of an image after the above-mentioned brightness correction.

Referring to FIG. 16, a video portion 113a on the left side of the video 113 is an outdoor video portion, and a video portion 113b on the right side of the video 113 is an indoor video portion. The outdoor video portion 113a is a little bright, but it is dark because it is nighttime. On the other hand, the indoor video portion 113b is quite dark. Image 113 is overall quite dark and hard to see.

Referring to FIG. 17, a video portion 113a on the left side of video 113 is an outdoor video portion after brightness correction, and a video portion 113b on the right side of video 112 is an indoor video portion after brightness correction. The outdoor image portion 113a is a little brighter than before the brightness correction, and the indoor image portion 113b is also brighter than before the brightness correction, so that the situation inside the room can be seen a little better.

18 and 19 are examples of videos displayed in the video display area 111, and both are other examples of videos taken by the drive recorder 1 at night.

FIG. 18 is an example of an image before the above-mentioned brightness correction, and FIG. 19 is an example of an image after the above-mentioned brightness correction.

Referring to FIG. 18, a video portion 114a on the left side of the video 114 is an outdoor video portion, and a video portion 114b on the right side of the video 114 is an indoor video portion. The outdoor video portion 114a is a little bright, but it is dark because it is nighttime. In contrast, the indoor video portion 114b is quite dark. Image 114 is overall quite dark and hard to see.

Referring to FIG. 19, a video portion 114a on the left side of video 114 is an outdoor video portion after brightness correction, and a video portion 114b on the right side of video 114 is an indoor video portion after brightness correction. The outdoor image portion 114a is brighter than before the brightness correction, and the indoor image portion 114b is also brighter than before the brightness correction, making it relatively easy to understand the situation inside the room.

FIG. 11 is a flowchart showing the brightness correction stop processing procedure.

When the user presses the normal display button 135 (YES in step 81), the CPU 58 stops the brightness correction process (indoor brightness correction) described above (step 82).

When the indoor emphasis button 134 is pressed, the process shifts to the process shown in FIG. 10, and the brightness correction described above is performed.

In the above embodiment, the brightness of the outdoor area is adjusted, but the amount of adjustment of the brightness of the image part of the interior (inside the vehicle) is relative to the amount of adjustment of the brightness of the image part of the exterior (outside the vehicle). If you increase the brightness, you don't necessarily have to adjust the outdoor brightness.

Furthermore, in the above embodiment, the brightness adjustment amount for the indoor (vehicle interior) video portion is made relatively larger than the brightness adjustment amount for the outdoor (vehicle exterior) video portion. When the video contains both dark and bright areas within one frame, rather than outdoors, the brightness adjustment amount for the dark areas is relatively larger than the brightness adjustment amount for the bright areas. May be adjusted. In this way, it is not necessary to detect indoors and outdoors. Within one frame, it is preferable that the area has a brightness equal to or higher than a threshold value, and the dark area has a brightness less than the threshold value. A detection device (which is an example of a detection means) that detects a bright portion that is equal to or higher than a threshold value may be provided, and the detected portion may be defined as a bright portion, and the undetected portion may be defined as a dark portion. Alternatively, a detection device (which is an example of a detection means) for detecting a dark portion below a threshold value may be provided, and the detected portion may be defined as a dark portion, and the undetected portion may be defined as a bright portion. The brightness adjustment amount may be adjusted brightly by adjusting the gain of the video data. The brightness may be adjusted so that both bright and dark areas become brighter, such that the brightness adjustment amount for the dark area is relatively larger than the brightness adjustment amount for the bright area. The brightness may be adjusted so that only dark areas become bright. It is even better to provide an adjustment device (which is an example of adjustment means) for adjusting this threshold value. The adjusting device (which is an example of a means for adjusting) that adjusts the threshold value may be an adjusting device that adjusts the threshold value by inputting a user's instruction.

20 and 21 show conversion curves for adjusting the brightness of video data.

Figure 20 shows that when one frame contains a dark area and a bright area, the brightness is adjusted by making the brightness adjustment amount for the dark area relatively larger than the brightness adjustment amount for the bright area. This is an example of a brightness conversion curve that performs the following.

In FIG. 20, the horizontal axis indicates the brightness level (input level) before brightness conversion, with the left side being darker and the right side being brighter. The vertical axis indicates the brightness level (output level) after brightness conversion, with the lower side being darker and the upper side being brighter.

By performing brightness conversion based on the brightness conversion curve C1 shown in Figure 20, the brightness before the brightness conversion is darker than the brightness before the brightness conversion is brighter than the brightness after the brightness conversion. The rate at which the brightness becomes brighter increases. In brightness conversion based on the brightness conversion curve C1 shown in FIG. 20, if the brightness before conversion is extremely dark, the brightness after brightness conversion becomes brighter is not very large. In this way, when the brightness before conversion is extremely dark, the brightness after brightness conversion may be set not to increase in brightness. For example, assuming a first threshold and a second threshold that is larger than the first threshold, a brightness value that is greater than or equal to the first threshold and less than or equal to the second threshold is determined. The adjustment amount may be set to be brighter than the adjustment amount for brightness below the first threshold and brightness brighter than the first threshold.

When the indoor emphasis button 134 is pressed, brightness adjustment may be performed using the brightness conversion curve C1 shown in FIG. 20, or a button such as a dark area emphasis button may be used instead of the indoor emphasis button 134. The brightness may be adjusted using the brightness conversion curve C1 shown in FIG. 20 when the button is pressed.

FIG. 21 is an example of the brightness conversion curve C2 used when the brightness adjustment described above is not performed.

In FIG. 21 as well, the horizontal axis indicates the brightness before brightness conversion, with the left side being darker and the right side being brighter. The vertical axis shows the brightness after brightness conversion, with the lower side being darker and the upper side being brighter.

The brightness conversion curve C2 shown in Figure 21 has linearity, and by using the brightness conversion curve C2, brightness correction is performed at a constant rate regardless of the brightness before brightness conversion.

It is preferable that when the normal display button 135 is pressed, the brightness is adjusted using the brightness conversion curve C2 shown in FIG.

In the above, the reason why the brightness conversion curve C1 shown in FIG. When adjusting the brightness by making the brightness adjustment amount of the dark area relatively larger than the brightness adjustment amount, the indoor video part and the outdoor video part are detected within one frame. For the indoor video part, brightness correction is performed using the brightness conversion curve C1 shown in Figure 20, and for the detected outdoor video part, a brightness conversion curve with linearity, for example, shown in Figure 21, is used to correct the brightness. Brightness correction may be performed using the brightness conversion curve C2 shown in FIG.

In the above embodiment, the buttons 134 and 135 are used to instruct brightness correction and to stop brightness correction, but these buttons 134 and 135 are not provided in the playback window 100, as will be described later. A menu may be displayed on the playback window 100, and an instruction to correct brightness or an instruction to stop brightness correction may be selected from the displayed menu.

Drive recorders are increasingly using lenses that can record not only the front but also the front and rear, 240 degrees, the front as well as the inside of the car. If the image is corrected to match, the interior of the car or other interior will become dark. In nighttime images, it is easier to see inside the office than in the daytime in places with streetlights, but it becomes harder to see in darker places or when there are fewer streetlights. If you increase the brightness, the outside will be blown out, and it's a waste to increase the brightness when recording. With digital images, if the highlights are blown out, the data will be lost, but if the blown out highlights are blown out, the data will remain. Even in pitch-black images, the driver can be seen by correcting the brightness. If you correct the overall brightness to make the inside of the car visible, the bottleneck will be that the outside of the car will become too bright, but we will add a button to make it easier to see inside or outside the car. It is possible to create images that emphasize the outside of the vehicle or images that emphasize the outside of the vehicle. In the above-mentioned embodiment, only two types of brightness adjustment, ie, brightness adjustment or not, have been described, but it is also possible to perform brightness correction linearly. The brightness correction may be changed to an intermediate level.

As mentioned above, if you move the cursor over the displayed image and right-click, a menu will appear, and you can use that menu to perform interior priority correction (interior emphasis correction, car interior priority correction, vehicle interior priority correction). good.

Using the brightness conversion curve C1 shown in FIG. 20, the extent to which dark areas are brightened is increased, and bright areas are not corrected much so that they become brighter. The amount of brightness adjustment is changed between bright areas and dark areas. Brightness can be adjusted for both day and night use.

Although the brightness is corrected during playback, it is recommended that the brightness be corrected in the same way when playing back the video on the display 11 of the main body of the drive recorder 1. A menu may be displayed on the display 11 of the drive recorder 1, and a button for switching or setting brightness correction instructions and stopping may be selected from the menu, or a touch panel may be formed on the display 11, and the display Such a switching button or a button for setting may be provided. When reproducing video data while recording it on the drive recorder 1, the above-described brightness correction may be performed on the video data to be reproduced.

[Third example]
22 to 27 show a third embodiment, in which video in the playback window 100 is displayed separately from the playback window 100.

In the same way as described above, the display control device 51 displays a playback window 100 as shown in FIG. 23 on the display screen of the display device 52 of the personal computer 50. The user moves the cursor 162 onto the image displayed in the image display area 111 and right-clicks (YES in step 151). Then, the display control device 51 displays the menu 161 in a superimposed manner on the video displayed in the video display area 111 (step 152).

The menu 161 displays the following character strings: "Image status: normal rotation", "return to standard", "reverse", "enlarge/reduce", "in-vehicle priority correction", and "display in separate window".

In the video display area 111 of the playback window 100, it is possible to display not only a video taken in the forward direction of the vehicle, but also a so-called spherical video taken in all directions above, below, left and right with the drive recorder as the center. In such spherical images, it may be difficult to tell whether the image is in a normal or reversed state, so the "image state" indicates the state of the image. :Normal rotation" indicates that the image displayed in the image display area 111 is in a "normal rotation" state. Since spherical images capture images in all directions, up, down, left and right, with the drive recorder as the center, by moving the image displayed in the image display area 111, you can see the area in front of the vehicle in which the drive recorder is installed. In addition to images, it is possible to display images viewed in all directions, including backwards, left, right, upward, and downward. ``Return to standard'' is selected to change the image displayed in the image display area 111 to an image viewed from the standard direction (for example, from the front). When "Flip" is selected, character strings such as "Flip vertically", "Flip horizontally", "Flip front and back", etc. are displayed. When "Vertical Flip" is selected, the image is flipped vertically, when "Horizontal Flip" is selected, the image is flipped horizontally, and when "Front/Front Flip" is selected, the image changes from the front image to the rear image. Or the image is reversed from the rear image to the front image. "Enlarge/Reduce" is selected when enlarging or reducing the video. As described above, the "vehicle interior priority correction" is selected when adjusting the brightness by making the brightness adjustment amount that brightens the interior of the vehicle larger than the brightness adjustment amount that brightens the exterior. "Display in separate window" is selected when displaying the video displayed in the video display area 111 in a window separate from the playback window 100.

Returning to FIG. 22, when "display in separate window" is selected (YES at step 153), the CPU 58 generates a video display window separate from the playback window 100. As shown in FIG. 24, a video display window 160 appears on the display screen of the display device 52 in addition to the playback window 100 (step 154).

Referring to FIG. 24, the video that was displayed in the video display area 111 of the playback window 100 (which is an example of a window that includes an operation system for operating the displayed video) is displayed in the video display window 160. . The video display window 160 may be displayed on the entire display screen of the display device 52 so as to hide the playback window 100, or the video display window 160 may be displayed mainly on the left or right half of the display screen. Conversely, the playback window 100 may be displayed mainly on the right half or the left half of the display screen. Further, the video display window 160 may be displayed mainly in the upper half or the lower half of the display screen, and conversely, the playback window 100 may be displayed mainly in the lower half or the right half of the display screen. Further, the size of the video display window 160 and the size of the playback window 100 may be the same or may be different.

Furthermore, when the video display window 160 is displayed separately from the playback window 100, the video may be displayed in the video display area 111 of the playback window 100. However, it is not necessary to display the video in the video display area 111 of the playback window 100 at that time. When displaying the playback window 100 and the video display window 160 on the same display screen, it is preferable that no video be displayed in the video display area 111 of the playback window 100.

When the cursor 162 is moved over the video displayed in the video display window 160 and right-clicked, a menu 161 is displayed.

The menu 161 displayed on the video displayed in the video display window 160 is different from the menu 161 displayed on the video in the playback window 100, and the menu 161 that is displayed on the video in the video display window 160 is different from the menu 161 displayed on the video in the playback window 100. ” string is included. When the character string "Close window" is selected, the video display window 160 disappears, and the video displayed in the video display window 160 is displayed in the video display area 111 of the playback window 100.

The information displayed in the vehicle status display area 132 (which is an example of accompanying information) may be displayed on the image of the image display window 160. While viewing the video displayed on the video display window 160, it is possible to check the speed, acceleration, etc. of the vehicle equipped with the drive recorder that captured the video. When displaying the playback window 100 and the video display window 160 on the same display screen, when the vehicle status display area 132 is hidden by the video display window 160, the information displayed in the vehicle status display area 132 may be displayed. It is preferable to display it on the video displayed in the video display window 160.

25 to 27 show modified examples.

In FIGS. 22 to 24, the playback window 100 and the video display window 160 may be displayed on the same display screen or on different display screens, but the modified examples shown in FIGS. 25 to 27 Now display it on a different display screen. In this modification, an external display device other than the display device 52 is connected to the personal computer 50.

FIG. 25 is a flowchart showing a processing procedure for displaying the playback window 100 and the video display window 160 on a separate display screen.

As shown in FIG. 23, if you right-click the video displayed in the video display area 111 when the video display window 160 has not been generated (YES in step 171), a menu 161 will be displayed on the video ( Step 172). In this modified example, the character string "Display in separate display screen" appears in the menu 161 instead of "Display in separate window." When the user selects the character string "Display on separate display screen" that appears on the menu 161 (YES at step 173), the CPU 58 forms the video display window 160. Data representing the formed video display window 160 (data for displaying video on a separate display screen) is output to an external display device connected to the personal computer 50 (step 74).

FIG. 26 shows how the playback window 100 and the video display window 160 are displayed on different display screens.

A video display window 160 (which is an example of a first window) is displayed on a display screen 190 of an external display device different from the display device 52 of the personal computer 50, and a playback window 100 (which is an example of a first window) is displayed on the display screen 180 of the display device 52. (which is an example of the second window) is displayed.

Although no video is displayed in the video display area 111 of the playback window 100, it is preferable to display the video. When the video display window 160 is displayed on a display screen 190 that is different from the display screen 180 of the display device 52 of the personal computer 50, move the cursor 162 over the video display area 111 and right-click. Menu 161 appears. Menu 161 includes the character string "Close separate display screen." When the character string "Close separate display screen" is selected, the output of data to the external display device is stopped, and the video display window 160 displayed on the display screen 190 of the external display device disappears. If no video is displayed in the video display area 111 of the playback window 100, the video is displayed in the video display area 111.

FIG. 27 shows an example of how to use the playback window 100 and the video display window 160 when they are displayed on separate display screens.

For example, the video obtained by the drive recorder installed in the vehicle driven by the driver 211 is played back, and the above-mentioned playback window 100 is displayed on the display screen of the personal computer 200 of the administrator 201 of the driver 211. do. The personal computer 200 and a relatively large external display device 210 are connected, and data for displaying the above-mentioned video display window 160 is output to the external display device 210. A video display window 160 is displayed on the display screen of the external display device 210.

The administrator 201 or the like operates the playback window 100 displayed on the personal computer 200 at hand to control the video displayed on the display screen of the external display device 210. The driver 211 can receive instructions, warnings, etc. from the manager 201 while viewing the video displayed on the display screen of the external display device 210.

Further, in the above embodiment, the playback window 100 is displayed on the display screen of the personal computer 50, but the playback window 100 is displayed on the display 11 of the drive recorder 1, etc., instead of the display screen of the personal computer 50. You may.

For example, driving school cars are equipped with drive recorders, and when showing driving footage of students at the driving school during a lesson, it is difficult to see it on a small screen when showing it to everyone. By displaying this on the display screen of the external display device 210, it becomes easier to see. An administrator (for example, an instructor at a driving school), while operating the personal computer 200 at hand, displays the driving video of the trainee on the display screen of the external display device 210 and teaches the students at the driving school how to be careful when driving. Can be done. Similarly, the truck driver may be instructed by the operation manager.

In addition to the video, a map image displayed in the map image display area 121, information displayed in the vehicle status display area 132, etc. may be displayed in a separate window or a separate display screen. Other accompanying information may also be displayed in a separate window or display screen as necessary. To open a separate window or display screen, you may right-click to display a menu and select from the menu, or you may provide a button and operate the button to open a separate window or display screen.

It is preferable to operate the video using the playback window 100, which is a window of the operation unit, and not to operate the video using the video display window 160. The video display window 160 preferably does not include an operation system or operation buttons. The video display window 160 does not include an operation unit or an operation system, and it is preferable to display the video in such a window that cannot be operated. As described above, it is preferable that the video in the video display window 160 be controlled by operating the operation button 133 or the like in the playback window 100.

An operation playback window 100 may be displayed on the screen of the drive recorder, and a video display window 160 may be displayed on a separate monitor to display the video. The drive recorder 1 may perform operations that are performed using programs, software, etc. for personal computers.

[Fourth example]
28 to 34 show the fourth embodiment.

In the fourth embodiment, the video playback speed can be adjusted. For example, the personal computer 50 shown in FIG. 7 can be used as a system that can adjust the video playback speed, but the drive recorder 1 shown in FIG. 3 can also be used.

FIG. 28 is a flowchart showing an example of a video playback processing procedure.

When a video file to be played is specified, the video data stored in the specified video file is read by the CPU 58, and the playback window 100 displayed on the display screen of the display device 52 is opened as shown in FIG. A still image representing a portion of the video is displayed in the video display area 111 (step 221). The CPU 58 also reads vehicle speed pulse data accompanying the video data as additional information (step 221). When the user presses the playback button (triangular button facing right) included in the operation buttons 133 (YES at step 222), video playback starts. The CPU 58 determines whether the vehicle speed represented by the read vehicle speed pulse data is slow (step 223). For example, if the speed of the vehicle in which the drive recorder 1 that captured the video is installed is slower than the speed threshold, it is determined that the vehicle speed is slow, and if it is equal to or higher than the speed threshold, it is determined that the vehicle speed is not slow. be done. The speed threshold may be predetermined or specified by the user. Images of a stopped vehicle are also judged to be slow.

If it is determined that the speed of the vehicle is slow (YES in step 223), the reproduction speed of the video displayed in the reproduction window 100 is slowed down by the CPU 58 (step 224). The video playback speed may be slowed down if the video playback speed is made slower than the normal playback speed. The normal playback speed is the speed at which the shooting time during shooting and the playback time during playback are the same. For example, by setting the number of video frames per unit time to be the same during shooting and playback, the shooting time during shooting and the playback time during playback will be at the same speed. For example, if the rate of shooting is one frame every 1/30 seconds, the frame will be played back at a rate of one frame every 1/30 seconds without dropping any frames. If it is determined that the speed of the vehicle is not slow (NO in step 223, the playback speed of the video displayed in the playback window 100 is set to the normal speed by the CPU 58 (step 225). In this way, the playback speed of the video is is slowed down or made normal, and the video is played back (step 226) (this is an example of changing the playback speed of the captured video based on the speed of the vehicle).

FIG. 29 shows the relationship between vehicle speed and video playback speed.

In the video between time t0 and time t3, it is assumed that the speed of the vehicle is slow between time t1 and time t2. Further, it is assumed that the speed of the vehicle was not slow (normal speed or slow speed) between time t0 and time t1 and between time t2 and time t3. Then, the video playback speed between time t1 and time t2 is slowed down. The playback speed of the video between time t0 and time t1 and the video between time t2 and time t3 is set to the normal speed. It may be played at a faster speed. Further, a video when the speed is slow may be played back at the normal playback speed, and a video when the speed is not slow may be played back at a speed faster than the normal speed (for example, n times the normal playback speed).

FIG. 30 is a flowchart illustrating an example of a video playback processing procedure. In FIG. 30, the same processes as those shown in FIG. 28 are given the same reference numerals, and the description thereof will be omitted.

In the process shown in FIG. 28, when the vehicle speed is slow (YES in step 223), the video playback speed is slowed down, but in the process shown in FIG. 30, when the vehicle speed is slow (YES in step 223) ), the video playback speed is increased (for example, n times the normal playback speed) (step 224A). If the speed of the vehicle is not slow (NO in step 223), the video playback speed is made normal (or less than 1 times the playback speed) (step 225). When the vehicle is slow, such as when the vehicle is stopped, important scenes are not captured in the video, and scenes where the vehicle is slow can be fast-forwarded, reducing video playback time. For example, when viewing videos, a manager who manages the driving status of a driver can fast-forward through videos when the vehicle is slow and the information is not really needed, such as when the vehicle is stopped at a traffic light, thereby simplifying the viewing process.

In the embodiments described above, it is determined whether the vehicle is slow based on vehicle speed pulse data, but other information such as GPS information may be used to determine whether the vehicle is slow. Furthermore, in the above-described embodiment, a mode may be used in which the information before and after the occurrence of an event, which is obtained based on vehicle history information or the like, is played back at normal speed. Furthermore, since the images of cars parked are the same, they can be skipped over, and images of cars driving at high speeds can be played back slowly because they flow quickly. In particular, images obtained with a camera that shoots in the horizontal direction have a fast flow, so the images feel fast. For this purpose, it may be played back at a slow speed.

FIG. 31 is a flowchart showing the processing procedure of video analysis.

In this embodiment, a video analysis button (not shown) is formed in the playback window 100. However, without forming the video analysis button, for example, by right-clicking on the video display area 111, a menu will appear, and the video analysis process may be specified in that menu. Further, the video analysis shown in FIG. 31 may be performed before the video is played back in response to the pressing of the playback button.

When a video to be played is designated and the video analysis button is pressed (YES at step 231), video data representing the designated video is read (step 232). When video data is stored in the semiconductor memory of personal computer 50, it is read by SSD 53, and when video data is stored in SD card 23, it is read by SD card reader/writer 54.

The video represented by the read video data is analyzed by the CPU 58 (step 233). A portion of the video that includes other vehicles is detected (YES in step 234), and it is determined whether the own vehicle (the vehicle shooting the video) is slower than the other vehicles (YES in step 235). ). If it is late, data representing the late time period is recorded in association with the video data (step 236). For example, data representing a slow time zone is recorded in the header of a video file in which video data is stored. If the host vehicle is not slower than other vehicles (NO in step 235), the process in step 236 is skipped. The processes from steps 234 to 236 are repeated until the video analysis is completed (step 237).

Once you know the time period in the video where the vehicle is slow, you can slow down the video playback speed as shown in Figure 28 or Figure 29 when playing that video (Step 224 in Figure 28), or The video playback speed can be increased (step 224A in FIG. 29).

FIG. 32 is a flowchart showing another example of the video analysis processing procedure.

Similar to the process in FIG. 31, when a video to be played is specified and the video analysis button is pressed (YES at step 241), the video data is read (step 242) and the video is analyzed (step 243).

As a result of the video analysis, if the video portion shows only the front or rear of the vehicle (YES at step 244), identification data indicating that the video portion is a front or rear video is recorded (step 247). ). For example, identification data is recorded in the header of a video file in which video data is stored. As a result of the video analysis, if the video part does not show only the front or rear of the vehicle (NO in step 244), but only the video part in the lateral direction of the vehicle (YES in step 245), the video part shows only the lateral direction of the vehicle (YES in step 245). Identification data indicating that it is a video is recorded (step 248). At this time, for example, the identification data is recorded in the header of the video file in which the video data is stored. As a result of the video analysis, if the image portion is not only in the lateral direction of the vehicle (NO in step 245), then whether the proportion of the image in front or behind the vehicle (the proportion of the image in the front and rear of the vehicle) is greater than that in the lateral direction. (Step 246). If the proportion of the image in front or behind the vehicle is greater than that in the lateral direction (YES in step 246), identification data indicating that the image portion is a front or rear image is recorded (step 247). If the proportion of the lateral direction of the vehicle is greater than the proportion of the front or rear image (the proportion of the front and rear images of the vehicle) (NO in step 246), identification data indicating that the image portion is in the lateral direction is provided. recorded (step 248).

The processes from steps 244 to 249 are repeated until the video analysis is completed (step 248).

FIG. 33 is a flowchart illustrating an example of a reproduction processing procedure for a video on which the video analysis processing of FIG. 32 has been performed.

When a video to be played is specified, video data and video identification data are read (step 251). When the playback button included in the operation button 133 is pressed (YES in step 252), it is determined whether the image represented by the read video data is a front image, a rear image, or a horizontal image based on the read identification data. It is determined whether the image is a directional image (step 253).

If the video is a front video or a rear video, the video is played back at a faster than normal playback speed (step 254). When the video is from the front or the rear, there is often no discomfort even if the playback speed is slowed down. It is possible to shorten the playback time while playing back without feeling any discomfort. When the video is switched (YES in step 255), if the playback has not ended (NO in step 258), the process from step 253 is repeated again.

If the video is in the horizontal direction, the video is played back at a slower playback speed than usual (step 256). When viewing images in a horizontal direction, the flow of the images may seem fast, which may create a sense of discomfort. When playing horizontal images, the playback speed is slowed down, which often eliminates the sense of discomfort. When the video is switched (YES in step 257), if the playback has not ended (NO in step 258), the process from step 253 is repeated again.

Although the process shown in Figure 33 is performed on the video that has undergone the video analysis process shown in Figure 32, the process shown in Figure 33 is not necessarily performed on the video that has been subjected to the video analysis process shown in Figure 32; Processing similar to the processing shown in FIG. 33 may be performed on the video on which the video analysis processing shown in 32 has not been performed.

For example, if a first camera photographs the front or rear of the vehicle and a second camera photographs the side of the vehicle, the image photographed by the first camera will represent the front or rear of the vehicle, and the second camera will photograph the front or rear of the vehicle. It can be seen that the image taken by camera 2 represents the lateral direction of the vehicle. In such a case, the video taken by the first camera may be played back at a faster playback speed, and the video taken by the second camera may be played back at a slower playback speed. Data indicating whether the video was shot by the first camera or the second camera would be recorded in the header of the video file in which video data representing the video is stored.

Furthermore, when playing back images captured by a camera that can capture images in all directions, up, down, left, right, front, and back, it is possible to display the front or rear of the vehicle, or to display the lateral direction. In such videos, it is detected during playback whether the front or rear of the vehicle is displayed at a high rate, and when the front or rear of the vehicle is displayed at a high rate, the playback speed is increased and the vehicle is displayed at a higher rate. It is a good idea to slow down the playback speed when a large proportion of the horizontal direction is displayed.

FIG. 34 is a flowchart illustrating an example of a reproduction processing procedure.

In this embodiment, it is preferable that a normal playback command button is provided in the playback window 100, but a normal playback command can be issued by clicking on the video display area 111 to display a menu and giving a normal playback command from that menu. Even if there is no button, it is sufficient if a normal reproduction command can be given to the personal computer 50.

When the video is being played back, it is determined whether or not it is being played back at the normal playback speed (step 261). If the playback speed is faster than the normal playback speed or slower than the normal playback speed, it is determined that the playback is not at the normal playback speed. When the normal playback command is given to the personal computer 50 by pressing the normal playback button etc. when the playback is not being performed at the normal playback speed (YES at step 261), the normal playback speed is set. The video that is not being played starts to be played at the normal playback speed (step 263). Videos that are not being played back at normal playback speed can now be played back at normal playback speed.

When you are running, the image in front of you does not feel strange, but when you look at the image on the side, it becomes difficult to see, so you can slow down the display speed of the image. If you look at it from the front, it feels slow, so you can play it faster to shorten the playback time. When the vehicle is stopped, it plays normally, but since the video when driving at high speed is played, it plays slowly when the vehicle is driving fast, and when the vehicle is stopped, it automatically plays back at double speed. It may be played back. Normal playback is preferably playback during actual recording. For example, when waiting at a traffic light, things that would have required the user to press a fast-forward button will automatically become faster. The speed is preferably displayed in numbers on the PC viewer. The speed information contained in the drive recorder includes GPS information and vehicle speed pulses, so it is recommended to use these information. GPS information and vehicle speed pulses are included in the header of the video file in which video data is stored, so they can be used. The vehicle speed pulse is the source information of the vehicle's speedometer, and is output based on the signal output according to the number of rotations of the tires.

[Fifth example]
35 to 40 show the fifth embodiment. The fifth embodiment relates to the display form of the playback window.

FIG. 35 and FIG. 36 display map images in a large size.

FIG. 35 is a flowchart showing the processing procedure of the personal computer 50, and FIG. 36 is an example of the playback window 100.

When the display software for the playback window 100 is activated by the user, the process shown in FIG. 35 starts. Then, as shown in FIG. 12, a playback window 100 in which the video display area 111, map image display area 121, etc. are blank is displayed on the display screen of the display device 52 of the personal computer 50 (step 271). In the list display area 141, a tab is displayed that allows you to give instructions for reproducing the recorded video as well as historical information (vehicle position information, speed information, acceleration information, etc.) of the vehicle that photographed the video ( (Illustration is omitted in FIG. 36 etc.). By selecting that tab (which is an example of an enlargement command button) and selecting a history information file, an instruction to reproduce the history record data represented by the selected history information file is given to the personal computer 50. When an instruction to reproduce history record data is given to the personal computer 50 (YES in step 272), a command to replace the video display area 111 and the map image display area 121 is given to the display control device 51 of the personal computer 50. The area of the video display area 111 is replaced with the area of the map image display area 121 (step 273). As shown in FIG. 36, the map image and video are displayed by exchanging the video display area 111 and the map image display area 121 (this is an example of a two-stage area of a large area and a small area) (step 274). ). Since the map image is displayed in a large size, it is relatively easy to check the location where the image was taken and the position of the vehicle at the time of the image.

In the examples shown in FIGS. 35 and 36, the map image is enlarged, but the area for displaying not only the map image but also information regarding the vehicle obtained while driving the vehicle photographing the video may be enlarged. Information such as vehicle acceleration information, vehicle speed information, brake information, turn signal information, etc. may be displayed in a large size. For example, velocity graphs and acceleration graphs may be displayed in a larger size. It is preferable that the area in which information about the vehicle is displayed is switched from a small area to a large area in response to a command given from a button or the like that gives a command such as an area expansion command.

In the examples shown in FIGS. 35 and 36, the area of the video display area 111 and the area of the map image display area 121 are exchanged, but it is not necessary to exchange them. The size of the map image displayed when an instruction to play history record data is given is larger than the size of the map image displayed when no instruction to play history record data is given (this is an example of the default size). It would be better if the size of Rather than increasing the size in response to dragging, for example, an enlargement command button is provided in the playback window 100, and in response to an enlargement command button from the enlargement command button, a map image of a predetermined size related to the vehicle can be displayed. It is preferable to display the information in an area larger than the predetermined size area. Information regarding the vehicle, such as a map image of a predetermined size, may be displayed in an area larger than the size of the image, or may be displayed in an area smaller than the size of the image. Although it is not necessary to change the size of the video display area 111, it is particularly preferable to change the size of the video display area 111 as shown in FIG.

The history record data and video data are linked, and the video and vehicle information at the same shooting time are synchronized and displayed in the video display area 111. For example, a map image is displayed in the map image display area. Needless to say.

FIGS. 37 and 38 are three-dimensional displays of the acceleration of the vehicle in which the video was taken. The processes shown in FIGS. 37 and 38 may be performed in response to an instruction to reproduce video recording or an instruction to reproduce history record data.

Referring to FIG. 37, in the displayed playback window 100, an acceleration graph display area 142 is formed on the right side of a play list display area 141. In the acceleration graph display area 142, the acceleration of the vehicle that captured the video is displayed in three dimensions. In FIGS. 37 and 38, an X axis is formed in the depth direction, a Y axis is formed in the horizontal direction, and a Z axis is formed in the vertical direction. The X-axis is an axis that indicates acceleration in the front and rear directions of the vehicle, the Y-axis is an axis that indicates acceleration in the lateral direction of the vehicle, and the Z-axis is an axis that indicates acceleration in the vertical direction of the vehicle. Acceleration toward the front of the vehicle (positive acceleration) is graphed upward on the X-axis, and acceleration toward the rear of the vehicle (negative acceleration) is graphed downward on the X-axis. Acceleration to the right when facing the vehicle is graphed upward on the Y-axis, and acceleration to the left when facing the vehicle is graphed downward from the Y-axis. The upward acceleration of the vehicle is graphed to the left on the Z-axis, and the downward acceleration of the vehicle is graphed to the right on the Z-axis. Of course, if the acceleration is known, it is not necessary to specify it in this way.

When the acceleration graph display area 142 is formed, a velocity graph may be displayed in the acceleration graph display area 138.

FIG. 38 shows an enlarged view of the acceleration graph displayed in the acceleration graph display area 142 compared to FIG. 37.

Graph 281 represents the acceleration of the vehicle in the longitudinal direction, graph 282 represents the acceleration of the vehicle in the lateral direction, and graph 283 represents the acceleration of the vehicle in the vertical direction. In the acceleration graph display area 142, the acceleration of the vehicle (the vehicle that captured the video) for each time is displayed in three dimensions according to the time when the video was captured. The acceleration of the vehicle displayed in the acceleration graph display area 142 is preferably the acceleration of a part of the vehicle in the captured video, especially for a certain period of time before and after the video displayed in the video display area 111 (for example, from several seconds before and after). It is best to set the acceleration to about several minutes before and after). For example, if the current image displayed in the image display area 111 is an image at time t12, then the image of the vehicle during the period from time t11, which is before time t12, to time t13, which is after time t12. Preferably, a graph representing acceleration is displayed in acceleration graph display area 142. Data representing acceleration is recorded in a header attached to each frame of video represented by video data, and by reading the data, a graph representing acceleration is displayed. Additionally, if video data is recorded in the header of the video file that stores it, or in a file that stores information about the vehicle that is different from the video file, the graph representing the acceleration will be read from those files. It would be nice if it were displayed.

In Figures 37 and 38, graphs representing acceleration are shown in a coordinate space consisting of the X, Y, and Z axes, and the first axis (which is an example of the is expressed, the second axis (which is an example of the Y axis) represents the acceleration in the lateral direction (width direction) of the vehicle, and the third axis (which is an example of the Z axis) represents the acceleration in the vertical direction of the vehicle. , it is not necessary to use the X, Y, and Z axes. The first axis, the second axis, and the third axis may not be orthogonal to each other. It also changes in the longitudinal direction according to changes in the longitudinal acceleration of the vehicle, changes in the lateral direction according to changes in the lateral acceleration of the vehicle, and changes in the vertical direction according to changes in the vertical acceleration of the vehicle. A graph of changing acceleration may be displayed.

FIG. 39 shows a modification and is an example of the playback window 100.

The acceleration graph display area 142 in the playback window 100 shown in FIG. 37 is formed on the right side of the play list display area 141, but in the playback window 100 shown in FIG. is displayed superimposed on the . By doing this, it becomes relatively easy to check the acceleration while viewing the video.

The information superimposed on the video is not limited to the acceleration graph, but may be other information regarding the vehicle. This makes it easier to check other information about the vehicle while viewing the video. In particular, as described with reference to FIGS. 26 and 27, when displaying the playback window 100 and the video display window 160 on separate display screens, it is preferable to display information regarding the vehicle on the video. Furthermore, when displaying the playback window 100 and the video display window 160 on separate display screens, information about the vehicle is displayed on the video, and information about the vehicle displayed on the video is also displayed in the playback window 100. It's good to do that. When displaying the playback window 100 and the video display window 160 on separate display screens, the video with the acceleration graph display area 142 formed on the video is displayed on the video display window 160 as shown in FIG. , it is preferable to form an acceleration graph display area 142 in the playback window 100 displayed on another display screen as shown in FIG.

FIG. 40 shows another modification and is an example of the playback window 100.

An acceleration display area 143 is formed on the right side of a play list display area 141 formed in the playback window 100. An image of a car is displayed in the acceleration display area 143. In front of the image of the car are the letters "+X" with an arrow pointing forward; next to the image of the car are the letters "+Y" with an arrow pointing laterally; and above the image of the car are the letters "+Z" with an arrow pointing upwards. ' is displayed. The front or rear acceleration of the car is displayed in front of the car image, the lateral acceleration of the car is displayed in the lateral direction of the car image, and the vertical acceleration of the car is displayed above the car image. Ru. Since the vehicle's acceleration is displayed along with the exterior of the vehicle, it is relatively easy to understand in which direction the acceleration is being applied to the vehicle. The image of the car will be stored in the settings file of the program that displays the playback window 100.

Further, the direction of the arrow may be changed depending on the acceleration. For example, if acceleration is applied to the front of the car, the arrow points toward the front, and if acceleration is applied to the rear of the car, the arrow points toward the rear. Also, if acceleration is applied to the right side of the car, the arrow points toward the right, and if acceleration is applied to the left side of the car, the arrow points to the left. Furthermore, if acceleration is applied to the top of the car, the arrow points upward, and if acceleration is applied to the bottom of the car, the arrow points downward.

An image of a type of car may be displayed in the acceleration display area 143 depending on the type of vehicle that captured the video. For example, if the vehicle is a forklift and a video is captured by the forklift, an image of a forklift is displayed in the acceleration display area 143, and if the vehicle is a truck and a video is captured by the truck, the acceleration display area 143 displays an image of the forklift. It would be nice to see an image of the truck. It is preferable that an image of a vehicle corresponding to the type of video captured in this way is displayed in the acceleration display area 143. Furthermore, an image of a vehicle of a manufacturer corresponding to the photographed vehicle may be displayed in the acceleration display area 143. For example, when a video is taken in a vehicle of an automobile manufacturer of company A, that vehicle of company A is displayed in the acceleration display area 143, and when a video is taken in a vehicle of an automobile manufacturer of company B, the acceleration display area 143 In area 143, it is preferable that the company B's automobiles be displayed.

There is a concept called history. Until now, the spotlight has been on video recording, but when it comes to GPS information, I am trying to collect a lot of information on a different level. If it's just history, it's text information, so the amount of data is smaller than video. Information such as when and where the vehicle passed is also historical information. Although the map is displayed in a small size, it is better to display the map in a larger size in the case of the history recording screen as described above. When you load a history record, the map will be displayed larger. The recording area of the SD card is also divided into an event recording area, a constant recording area, and a history recording area. Continuous recording can only be recorded for about one hour, but historical recording can be recorded for 24 hours or 500 hours. The history records position information, speed information, acceleration information, brake information, and turn signal information. History records are different from event records and constant records. History recording simply means constant recording without video data. You can record up to 500 hours with the settings. History remains in video data such as constant recording and event recording. History information that contains images can be linked to display the images as well.

[Sixth Example]
FIG. 41 shows the sixth embodiment, and is a flowchart showing an example of the processing procedure of the personal computer 50.

When the playback software that displays the playback window 100 is activated, the process shown in FIG. 41 starts. Then, data representing the clock frequency of the CPU 58 of the personal computer 50 is read (step 291). Based on the read clock frequency of the CPU 58, it is determined whether frames will be dropped in the reproduction of the specified video (step 292). If it is determined that dropped frames will occur (YES in step 292), a message is displayed on the display screen of the display device 52 to the effect that frames may be dropped in the video because the processing capacity of the personal computer 50 is low (step 293). ). If frame drop does not occur (NO in step 292), the process in step 293 is skipped.

Thereafter, the playback window 100 is displayed on the display screen of the display device 52 (step 294), and the video is displayed in the video display area 111 (step 295). The user understands that the frame dropout is not due to a problem with the captured video or the device that takes the video, but rather due to the ability of the personal computer 50 that plays back the video. Although the playback window 100 is displayed after the display that a frame drop will occur, the playback window 100 may also be displayed together with the display that a frame drop will occur. In particular, it is preferable to display on the playback window 100 that a frame drop will occur.

In the example shown in FIG. 41, it is determined whether frame drop occurs based on the clock frequency of the CPU 58, but it is not limited to the clock frequency, but also the capacity of the RAM (random access memory) included in the personal computer 50, The capacity of VRAM (video random access memory), the reading speed of the SD card reader when video data is recorded on the SD card 23, the reading speed of the hard disk driver when video data is recorded on the hard disk, etc. It may also be possible to display that frame drop will occur based on the capability, performance, etc. of a system such as the personal computer 50. In addition, when the ability or performance of the personal computer 50 is low (for example, when the ability or performance is such that it cannot play back in the same situation as the one at the time of shooting), instead of dropping the video frame, it is possible to You can also reduce the number of videos or play them in slow motion. In that case as well, it is preferable to display (this is an example of notification) that slow playback will be performed due to the ability, performance, etc. of the personal computer 50. It is only necessary to notify that the device does not have the ability to process images.

Moving images or opening images in a separate window takes a lot of CPU power from your PC. Some PCs do not allow you to display video, such as playing in a separate window or changing the display range of the video at double speed. If you have a PC that cannot be moved, it may be a good idea to restrict it so that it can only be moved when it is paused. If you set up a separate window button and set the playback window 100 and the video window as separate windows, you can press the separate window button. You can't play it in a separate window if you don't have it. Please have a computer with these specs.''

[Seventh Example]
42 to 46 show a seventh embodiment, in which reproduction is performed from a predetermined reproduction position.

FIG. 42 is a flowchart showing the processing procedure of video analysis. The process shown in FIG. 42 starts when a video to be analyzed is designated and a video analysis command is given to the personal computer 50. For example, by displaying the playback window 100 in which a video analysis button is formed, specifying a video, and pressing the video analysis button, the process shown in FIG. 42 is started.

Video data representing the video to be analyzed is read (step 301), and the user inputs conditions for scenes to be found in the video (step 302). For example, a menu may be displayed by right-clicking on the video display area 111 of the playback window 100, and a desired scene may be selected from that menu, or a word representing the desired scene may be entered from that menu. You can. When the scene conditions are input, the video data is analyzed and a video portion that matches the input scene conditions is detected (step 303). The time of the video portion detected for each scene condition is stored in the header of the video file in which video data representing the analyzed video is stored (step 304).

FIG. 43 is a flowchart showing the processing procedure of the personal computer 50. When the software of the playback window 100 is activated, the processing shown in FIG. 43 starts.

A playback window 100 is displayed on the display screen of the display device 52 of the personal computer 50 (step 311). Data representing the time of the video portion matching the scene conditions found in the video analysis is read from the header of the video file (step 312). Furthermore, data representing history information associated with the video file is read out (step 313). A first mark is placed at a position indicating the time of the video portion that matches the scene condition (step 314), and a second mark is placed at a position indicating the time of the video portion based on the history information (step 315).

FIG. 44 is an example of the playback window 100.

In the playback window 100 shown in FIG. 44, a playback bar (seek bar) 300 is formed below the video display area 111. This playback bar 300 indicates the position of the video currently displayed in the video display area 111 among the entire video. On the reproduction bar 300, a circular circle mark 351, a first cross mark 361, and a second square mark 362 are displayed. A circle mark 351 indicates the position of the current video displayed in the video display area 111. The first mark 361 indicates the video position of the scene that matches the scene condition. For example, if "a scene in which a child is photographed" is specified as a scene condition, the video corresponding to the position of the first mark 361 will be a video in which a child is photographed. Multiple scene conditions may be specified. Although it is not necessary to change the mark depending on the type of scene condition, it is preferable to change the mark depending on the type of scene condition. The second mark 362 indicates the video position of the scene based on historical information (information regarding the vehicle). For example, if information about sudden braking is specified as history information, the image corresponding to the position of the second mark 362 will be an image where sudden braking was applied. You may specify multiple pieces of history information. Although it is not necessary to change the mark depending on the type of history information, it is preferable to change the mark depending on the type of history information.

Below the play bar 350, a first skip play button 341 and a second skip play button 342 are formed. The first skip playback button 341 displays the character string "skip 1". The character string “Skip 2” is displayed on the second skip playback button 342.

FIG. 45 is a flowchart showing the processing procedure of the first skip playback.

When the first skip playback button 341 is pressed (YES in step 321), the circle mark 351 is moved to the position of the first mark 361 closest to the circle mark 351 (step 322). If the video is being played, the video up to the position of the first mark 361 to which the circle mark 351 has moved is skipped, and the video is played back from the position of the first mark 361 to which the circle mark 351 has moved. If the video is not being played, a still image of the video corresponding to the position of the first mark 361 to which the circle mark 351 has moved is displayed in the video display area 111. When the first skip playback button 341 is pressed again, the circle mark 351 moves to the position of the next first mark 361.

By using the first skip playback button 341, the video can be viewed in a relatively short time while finding the video portion that matches the scene conditions.

FIG. 46 is a flowchart showing the processing procedure of the second skip playback.

When the second skip playback button 342 is pressed (YES in step 331), the circle mark 351 is moved to the position of the second mark 362 closest to the circle mark 351 (step 332). If the video is being played, the video is played back up to the second mark 362 to which the circle mark 351 has moved, and the video is played back from the second mark 362 to which the circle mark 351 has moved. If the video is not being played back, a still image of the video corresponding to the position of the second mark 362 to which the circle mark 351 has moved is displayed in the video display area 111. When the second skip playback button 342 is pressed again, the circle mark 351 moves to the next second mark 362 position.

By using the second skip playback button 342, the video can be viewed in a relatively short time while finding the video portion of the scene based on the history information. The first mark 361 and the second mark 362 need only have different display formats, and may have different shapes or colors.

When triggering an event search, you can either play back and check each event one by one, or you can remember the time and play back, but as mentioned above, ``At that time, there was a person in the picture.The traffic light was red.'' If you enter a scene condition such as "Where was that place? What time was it?", events will be found according to that input. Constant records and event records are recorded separately, but there is no relationship between them. When constantly reproducing records, the point at which an event record exists can be determined by looking at the date of the event record file. Displays a line indicating that there is an event at this location in the slide bar of the continuous recording file. You can fly directly to the position of such a line. It would be convenient if there was a button to jump directly to that line. Instead of pressing a button and flying directly to the location of the event, it would be better to analyze the image and fly directly to the location where the person was photographed earlier. It would be a good idea to display the locations of intersections, people crossing, and pictures of cute older ladies one after another. In addition to event recording, image analysis of constantly recorded video can be used to extract triggers such as traffic lights, people crossing, and road signs. You will be able to view them one after another from those locations. It is a good idea to change the color of the line for each type of trigger and repeat playback for each type. The playback command can be a button or a menu. It would be good if there was something called ``fly'' on the menu, and there were various kinds of things in it.

[Eighth Example]
47 to 49 show the eighth embodiment.

FIG. 47 is a flowchart showing the transmission processing procedure of the drive recorder 1.

The processing procedure shown in FIG. 47 is for restricting the transmission of video data from the drive recorder 1.

When an event or the like occurs as described above and a transmission command is generated by the CPU 20a of the drive recorder 1, it is checked whether the number of times video data has been transmitted from the drive recorder 1 has reached the limit (step 341). . The limited number of times is a limited number of times within a certain period of time, and may be any number of limited times such as hourly, daily, weekly, or monthly. If the limited number of times has not been reached (NO in step 341), it is checked whether the total amount of transmitted data has reached the limited data amount (step 342). The total amount of restricted data is also the total amount of transmitted data within a certain period of time, and may be any of the total amount of transmitted data on an hourly, daily, weekly, or monthly basis. If the total amount of data to be transmitted does not reach the data limit (NO in step 342), the video data for transmission is transmitted to a predetermined server, etc. using the LTE module 21 of the drive recorder 1, as described above. (Step 343).

When the transmission is completed, the number of transmissions is increased by one (step 344). Data representing the number of transmissions is stored in the RAM 20c of the drive recorder 1. The amount of data to be transmitted is also stored in the RAM 20c of the drive recorder 1 (step 345).

If the number of transmissions is the limited number of times (YES in step 341) or the total amount of data to be transmitted is the limited data amount (YES in step 342), a message indicating that data cannot be sent due to the transmission limit is displayed. The data shown is transmitted to the server etc. using the LTE module 21 of the drive recorder 1 (step 346). As a result, data that has not been received can be sent to a server or the like using Wi-Fi or the like without using the LTE module 21. Alternatively, the video data may be read from a memory card.

Communication using the LTE module 21 is expensive, but since the data to be transmitted is limited, the cost can be kept down. Further, if there is data that has not been received, the existence is notified, so the data can be received by a method other than the method using the expensive LTE module 21.

FIG. 48 is a flowchart showing the processing procedure for ranking display.

At the server, the amount of communication data is detected for each user (step 351). For example, if a SIM (Subscriber Identity Module) card is loaded in the LTE module 21 of the drive recorder 1, and communication based on LTE is performed using the SIM card, the contract company of the SIM card By accessing the server, the amount of communication data can be detected for each user. When the display command is given to the server (YES in step 352), the amount of communication data is displayed for each user on the display screen of the display device connected to the server (step 353).

FIG. 49 is an example of a graph showing the amount of communication data.

FIG. 49 shows the amount of communication data from user 1 to user 6. It can be seen that among the amount of communication data of users 1 to 6, the amount of communication data of user 4 is larger than that of other users. For example, if the amount of communication data of user 4 is compared with the amount of video data of user 4 sent to the server and they do not match, user 4 will not send video data from drive recorder 1, but , it is possible that the communication SIM card loaded in the drive recorder 1 was removed and used for personal purposes. Compare the amount of communication data of user 4 and the amount of video data of user 4 sent to the server, and if they match, if the driving is rough, many events will occur and the amount of video data sent will increase. . Therefore, it can be seen that user 4 is driving roughly. It can also be confirmed whether the amount of video data is increasing due to poor video data transmission settings in the drive recorder 1.

[Ninth Example]
50 to 52 show the ninth embodiment.

FIG. 50 is a flowchart showing the processing procedure of the drive recorder 1. For example, the process starts when the engine of the vehicle in which the drive recorder 1 is installed is turned on and the power of the drive recorder 1 is turned on.

A video data communication speed test is performed between the drive recorder 1 and the server (step 361). If the communication speed test is successful (YES in step 362), the resolution, frame rate, bit rate, etc. of the video data for transmission are determined in the CPU 20a of the drive recorder 1 based on the test results (step 363). . For example, if the communication speed is fast, the resolution, frame rate, bit rate, etc. are determined so that the video represented by the video data for transmission has relatively high quality; if the communication speed is slow, the video data for transmission The resolution, frame rate, bit rate, etc. are determined so that the video represented by the data has relatively low quality.

When a recording command is generated in the CPU 20a of the drive recorder 1 due to an event or the like (step 364: YES), video data for transmission is recorded at the determined resolution (step 365).

If the communication speed test is not successful (NO in step 362), it is checked whether the number of test failures has reached a predetermined number (step 366), and if it has not reached the predetermined number (step 366). NO), the communication speed test is retried (step 361). Needless to say, the number of failures in the communication speed test is counted in the controller 20 of the drive recorder 1. If necessary, the drive recorder 1 is provided with a counter. If the communication speed test has failed a predetermined number of times (YES in step 366), and a recording command is issued (YES in step 367), video data is recorded at a predetermined resolution, etc. (step 368).

51 and 52 show the processing procedure when the drive recorder 1 is equipped with a polarizing filter.

FIG. 51 is a flowchart showing the processing procedure of the drive recorder 1.

When a recording command is generated, the process shown in FIG. 51 starts. The CPU 20a of the drive recorder 1 confirms whether a polarizing filter is attached to the camera 17 of the drive recorder 1 (step 371). For example, if the drive recorder 1 is provided with a switch that indicates that a polarizing filter is attached, the attachment of the polarizing filter is detected by detecting the state of the switch. If the drive recorder 1 is provided with a sensor that indicates that a polarizing filter is attached, the sensor detects that the polarizing filter is attached. The CPU 20a analyzes (image recognition) the image represented by the image data taken by the camera 17 (for example, analyzes whether there is reflection on the window glass of the interior of the car or the polarizing filter on the dashboard, etc.). Attachment of a filter may also be detected.

The video data is recorded on the SD card 23 (step 372), and identification data indicating that the video data was recorded with a polarizing filter attached is also recorded in the header of the file in which the video data is stored. (step 374). The drive recorder 1 may be provided with a display device such as a light emitting diode, and if a polarizing filter is attached, the light emitting diode may be turned on.

If the drive recorder 1 is not equipped with a polarizing filter (NO in step 371), the video data is recorded on the SD card 23 without recording the above-mentioned identification data (step 374). However, it is preferable that identification data indicating that the video data is recorded without a polarizing filter attached is recorded in the header of the file in which the video data is stored.

FIG. 52 is a flowchart showing the reproduction processing procedure in the reproduction window 100.

The process shown in FIG. 52 starts when a file storing video data to be played is specified and a play command is given using a play button or the like.

The header of the file in which the video data is stored is searched, and it is checked whether identification data representing video data recorded with the polarizing filter attached to the drive recorder 1 is recorded in the header ( step 381).

Subsequently, the video data is played back and the video is displayed (step 382). During playback of video data, the user is informed (by display, audio output, etc.) that the video data was shot with a polarizing filter attached (step 383).

Even if the identification data is not recorded, the video data is played back and the video is displayed (step 384). When playing back video data, the user is not notified (by display, audio output, etc.) that the video data was shot with a polarizing filter attached. It is best to notify the user that the image was taken without a polarizing filter attached.

As the image quality of drive recorders continues to improve, even things that cannot actually be seen may be recorded. If you were a driver who was involved in an accident, you would want to say in court that the situation was different from what you actually saw, but if it were to be shown as evidence in court, it would be a clearer image that would be disadvantageous to the driver. There is a possibility that it will happen. By including information such as the presence or absence of a polarizing filter at that time, it becomes possible to claim that ``this is a video taken when a polarizing filter was attached, so it is different from what I saw.'' Become. serves as proof of such a claim.

[10th Example]
53 to 56 show the tenth embodiment.

FIG. 53 shows the processing procedure of the drive recorder 1.

The drive recorder 1 has a function of being able to talk (communicate) with a smartphone of a user other than the driver of the vehicle in which the drive recorder 1 is installed. Of course, it is also possible to talk (communicate) with the smartphone of the user of the vehicle driver. When the drive recorder 1 is connected to a predetermined communication device such as a smartphone held by a user outside (or inside) the vehicle (YES at step 391), the videophone recording mode changes to the drive recorder 1. It is checked whether it is set (step 392). Needless to say, the drive recorder 1 is provided with a mode switch for setting the videophone recording mode. If the videophone recording mode is set (YES at step 392), data representing the content of the call with the telephone device is recorded on the SD card 23. For example, a call recording area is defined in the SD card 23, and data representing the content of the call is recorded in the call recording area. The drive recorder 1 is provided with a microphone, and audio is input to the drive recorder 1 using the microphone.

The drive recorder 1 and the communication device may communicate (call) by receiving a call from the communication device, or the drive recorder 1 and the communication device may communicate by placing a call from the drive recorder 1 to the communication device. Good too. Even while driving, it becomes possible to talk on the phone using the drive recorder 1. By providing the drive recorder 1 with a camera capable of photographing the interior of the vehicle, the drive recorder 1 can be used as a videophone.

FIG. 53 is a flowchart showing the reproduction processing procedure by the reproduction window 100.

An SD card 23 on which a large number of files containing video data are recorded is loaded into a personal computer 50 for reproduction. The SD card stores not only files storing video data captured by the drive recorder 1, but also video data captured by surveillance cameras, etc. inside and outside the user's office, company, etc. It is assumed that many files are also recorded. In the headers of these files, data representing a user ID for identifying the user recorded in the video and the date of shooting are also recorded by image recognition or the like. Therefore, you can see who is in which file by shooting date.

For example, the user ID of the user who wants to check the date and trend of the video data to be played back is input, and one of the video files that stores the video data shot on that date is identified by the input user ID. Video data showing the user is found by the CPU 58 of the personal computer 50 (step 401). A plurality of pieces of video data that are found are continuously played back in chronological order (step 402).

The user's daily movements are all shown in a series of timelines, such as camera images when leaving the office, office, etc., camera images from drive recorder 1, and camera images when returning to the office, office, etc. You can see it above. It is possible for the user (employee) to easily understand, for example, whether or not the work situation was appropriate for one day.

It will be possible to seamlessly play back the video on the drive recorder and the video from the office. The drive recorder only shows what happened while I was in the car, but what was the condition of the driver before he got into the car, what was the condition of his face when he came to the office, and whether he had been drinking? Such. You will be able to understand the movements of the day. In addition to the drive recorder footage, it is also possible to obtain footage from when the driver came to work and when he/she reported on the way home, and can continuously play videos from the same day and with the same driver's ID. If you visit several locations and turn off the engine at each location, the files will be separated during the day, so you can connect them and play them. You can also connect videos taken by the office's surveillance cameras to replay the flow of the day. Video from the office, video from the first visit, video from the second visit, and so on can be played back in succession. For example, truck, bus, and taxi operation managers are required to perform a roll call, and by filming the roll call, companies and managers can confirm that they are checking the roll call in the morning and on their way home.

FIG. 55 is a flowchart showing the processing procedure of the drive recorder 1.

The process shown in FIG. 55 starts when a recording command is generated in the CPU 20a of the drive recorder 1. The process shown in FIG. 55 is performed in the drive recorder 1 which has a configuration in which two SD cards, a first SD card and a second SD card, can be loaded.

The video data photographed by the camera 17 is temporarily stored in the RAM 20c. The video data is read from the RAM 20c and processed by the CPU 20a (step 411). For example, processing such as cutting out a part of a video (particularly a celestial video) and converting coordinates is performed. The processed video data is recorded on the first SD card (step 412). Data representing the processing content is recorded in the header of the file in which the video data is stored (step 413). Further, unprocessed video data is read from the RAM 250c and recorded on the second SD card (step 414). The video data recorded on the second SD card is video data that represents video closer to raw video than processed video data, and is, for example, RAW video data, video data compressed based on H246, etc. .

Record unprocessed video data on a second SD card, read and process the second unprocessed video data from the second SD card, and display processed video data and processed video data. It is preferable that the data be recorded on the first SD card.

In addition, video data representing the front image of the camera's drive recorder (especially spherical drive recorder) is recorded on the first SD card (one SD card), and video data representing the rear image is recorded on the second SD card. It is preferable to have a configuration in which the information is recorded on the SD card (the other SD card). If the video represented by the video data recorded on the first SD card and the video represented by the video data recorded on the second SD card are at least partially overlapped at the same time, good.

FIG. 56 is a flowchart showing a procedure for reproducing video data recorded on two SD cards as described using FIG. 55.

The first SD card and the second SD card are loaded into a personal computer 50 for playback or the like.

Processed video data (first video data) recorded on the first SD card and data representing the content of the processing are read from the first SD card (step 421). Further, unprocessed video data (second video data) recorded on the second SD card is read from the second SD card (step 422).

A video represented by the second video data and a video represented by the first video data are displayed. At this time, a processing range indicating the processed portion of the unprocessed video is displayed (step 423). You can see which part of the entire video has been processed or displayed. In particular, it is preferable to synchronize two videos taken at the same time and display each video in two windows (display areas).

Instead of processing the video data in the drive recorder 1, the video data is processed in a personal computer, which is a playback device that plays back the video, and both the processed video and the unprocessed video (for example, RAW video) are processed. It may also be displayed.

In playback devices such as personal computers, the two videos are recorded on separate SD cards, so it is necessary to understand the correspondence between the files, but this can be done by looking at the time.

[11th Example]
57 to 59 show the eleventh embodiment.

FIG. 57 is a flowchart showing the processing procedure of the drive recorder 1.

In the processing procedure shown in FIG. 57, a share button is provided on the drive recorder 1. A signal indicating that the share button has been pressed is input to the controller 20 of the drive recorder 1. The share button does not necessarily have to be a physically existing button, but may be a software button displayed on the display screen of the display 11.

When the share button is pressed by the user (YES in step 431), video data representing images shot within a certain period before and after the share button was pressed, out of the video data that is constantly recorded, is displayed. , is sent to a predetermined cloud server (step 432). Video data sent to the cloud server is saved in a storage location specified by a predetermined URL (Uniform Resource Locator).

In addition, data indicating the location of the vehicle when the share button was pressed, the time when the share button was pressed, and the URL that specifies the storage location of the cloud server where constantly recorded video data is stored is stored. is recorded in the header of the file (step 433).

Not only when the share button is pressed, but also when an event occurs as described above. It may be a function to send video data (which may also be video data) to a cloud server.

FIG. 58 is a flowchart showing the reproduction processing procedure in the personal computer 50.

When the SD card 23 on which video data is recorded in the drive recorder 1 is loaded into the personal computer 50 and the playback button on the playback window 100 is pressed, the process shown in FIG. 58 starts.

A map showing the vicinity of the shooting location of the video being played back is displayed in the map image display area 121 of the playback window 100 (step 441). If the location represented by the map displayed in the map image display area 121 includes the location when the share button was pressed, an icon representing that location is displayed on the map (step 442). . Data indicating the location where the share button was pressed is recorded in the header of the file in which the video data is stored, so the icon is displayed on the map by reading from the header. In addition, the icon is clickable and has a URL link indicating the storage location of the video data. This URL is also recorded in the header of the file in which the video data is stored, so it is read from the header and linked to the icon.

When an icon displayed on the map is clicked by the user (YES in step 445), request data for the URL linked to the clicked icon is transmitted. Then, the video data stored in the storage location specified by the URL is transmitted to the personal computer 50 that is playing the map. The user can view the video near the location where the share icon is pressed.

For example, by publishing a map on which the above-mentioned icon is displayed on the Internet, a third party can view the map and click on the icon to view video of the area near the icon. You will be able to share the video you created.

FIG. 59 is a flowchart showing the processing procedure of a smartphone that can communicate with the drive recorder 1.

The smartphone is placed within a range where it can communicate with the drive recorder 1 installed in the vehicle via Wi-Fi, Bluetooth (registered trademark), or the like. For example, when the drive recorder 1 is installed in a car, a smartphone is placed inside the car. Smartphones also have the ability to connect to the Internet via wireless LAN (local area network) with smart speakers placed in homes, offices, offices, etc. that have a wireless LAN (local area network) environment.

For example, when a video data transmission command (trigger) is sent to a smartphone using a smart speaker, and the transmission command is received by the smartphone (YES in step 451), the received transmission command is driven by the smartphone. - Sent to recorder 1 (step 452). For example, if you say the command "Give me a video of Mr. A" to the smart speaker, that command will be sent to Mr. A's smartphone, and from the smartphone it will be sent to the drive recorder installed in the car that Mr. A is driving. 1, a video data transmission command is transmitted.

When the drive recorder 1 receives a transmission command from the smartphone, the video data recorded on the drive recorder 1 is transmitted from the drive recorder 1 to the smartphone in accordance with the command. The video data sent to the smartphone may be video data recorded immediately before the command was received, video data recorded within a certain period before or after the timing of receiving the command, or video data recorded on the day the command was received. It may also be recorded video data.

When the smartphone receives the video data transmitted from the drive recorder 1 (YES at step 453), the received video data is transmitted from the smartphone to the server (step 454).

In this way, the smartphone can be connected to the cloud, and the trigger can be pushed from the cloud to the smartphone and sent from the smartphone to the drive recorder connected to the smartphone via Wi-Fi or Bluetooth. It would be extremely convenient to be able to work in conjunction with IFTTT etc. For example, if Administrator X says to a smart speaker installed in the office, ``I would like to receive the video of Mr. A,'' a notification will be sent to Driver A's smartphone, and the smartphone will send a trigger to the dash recorder and record it on the dash recorder. Allow the user to set the trigger name. The viewer displays the set trigger name. For example, it may be displayed as ``Voice trigger for shooting from administrator

Smartphones are connected to the cloud. Since the smartphone is connected to the Internet, if the drive recorder does not have the ability to connect directly to LTE, the smartphone can connect to the Internet. You will be able to pair your smartphone and drive recorder and connect them via Bluetooth. The server will instruct the smartphone, and the smartphone will instruct the drive recorder. When you instruct a smart speaker installed in your office to send you a video of a certain driver, the smart speaker instructs your smartphone, which in turn instructs the drive recorder. The name of the trigger is important. Currently, trigger names are fixed; for example, there are only fixed triggers, such as acceleration triggers and switch press triggers, but it would be nice to be able to name the triggers. It would be good if this could be set such as a trigger when Mr. X gave an instruction to Mr. Y to shoot. It would be nice if the user could change the trigger name themselves in the viewer. To display the trigger name in the viewer, there is a green line on the slide bar for events, but when you hover your mouse over that green line, the trigger name will pop up there. If it's a yellow line, the trigger name should pop up, such as "smartphone trigger" or "smartphone trigger is a trigger from administrator A."

Also, since gamers are concerned about FPS (frames per second), it is a good idea to provide a PC viewer with a function that displays the recording frame rate and display frame rate. The viewer may be provided with a frame completion function. Although the recording frame rate is fixed, playback may not be able to reproduce the recording frame rate depending on the PC's capabilities. Both the recording frame rate and the playback (display) frame rate are displayed on the display screen. Displays something like a superimpose on the top left of the video being played. It is displayed at 30fps for recording and 25fps for playback, and the playback will change. It turns out that the video is not displayed smoothly, not because the specs of the drive recorder are bad, but because the specs of the PC are bad.

[12th Example]
60 to 62 show the twelfth embodiment.

FIG. 60 shows the processing procedure when playing back video data, and includes a flowchart showing the processing procedure of the playback device of the personal computer 50 and a flowchart showing the processing procedure of the robot that can communicate with the personal computer 50. It shows.

The robot is, for example, a humanoid robot with a head, arms, and legs attached to the body. The robot has communication functions with a personal computer 50 such as Wi-Fi. The operation of the robot is controlled by a computer built into the robot, and has the function of performing operations according to the content of communication with the personal computer 50. The robot is not limited to a humanoid robot, but may also be an animal-shaped robot such as a dog or cat.

When the SD card 23 on which video data is recorded is loaded into the personal computer 50 and the playback button is clicked after starting the playback program, the video data is played back (step 461). During reproduction of the video data, when an impact point such as a sudden brake is reproduced (YES in step 462), a command to perform an impact action is sent from the personal computer 50 to the robot (step 469).

When the command transmitted from the personal computer 50 is received by the robot (step 471: YES), the robot performs a falling motion in response to receiving the command (step 472). The robot gets up and warns the user about dangerous driving (step 473). For example, it outputs audible information indicating that driving is dangerous.

If the reproduction of the video data is not completed (NO in step 464, NO in step 474), the reproduction device repeats the processing from step 461, and the robot repeats the processing from step 471.

Send motion instructions from the PC viewer to the communication robot. For example, when playing back the part where the impact occurred, the communication robot behaves as if it would fall over. Then he gets up and makes a scolding motion telling him that he shouldn't drive in such a dangerous manner. It is extremely effective when placed in driving schools, etc. It would be upsetting to be scolded by an operator of a bus or transportation vehicle, but if it is scolded by a robot, it has no effect on human relationships. Normally, when a person called a traffic manager instructs a driver, the traffic manager gives the guidance, but instead, the robot objectively tells the driver. The contents of the trigger are associated with the contents of the robot's speech, and the robot speaks when the trigger is played. Remember the movements as well. It moves and adjusts the volume according to the magnitude of acceleration such as a G sensor. The correspondence relationship is recorded, and the robot instructs it when it is time to play it back.

In addition, it is preferable to give up on the use of forms in the personal computer 50 and provide a function for writing out presentation files. Business-use drive recorders have the function of producing paper reports for business use. Words of caution are written, such as, ``What score is your driving? You often turn suddenly.'' A form is sometimes output from the print function of a PC viewer. Instead of outputting a report, there is a presentation file, like a power point template, and a file that tells you how many points you scored for your driving will appear.

FIG. 61 is a flowchart showing the processing procedure of the virtual reality device.

A video data reproducing device such as a personal computer 50 and a virtual reality device are connected. The virtual reality device includes a VR controller and a head-mounted display, and a user wears the head-mounted display and operates the head-mounted display using the VR controller. When the user wears the head mounted display, the playback window 100 is virtually displayed (step 481).

When the dual screen display button formed on the VR controller of the virtual reality device is pressed (YES at step 482), the user wearing the head mounted display can see the first window and the second window. . The first window corresponds to the playback window 100 shown in FIG. 26 and displays operation buttons, and the second window corresponds to the video display window 160 shown in FIG. 26 and displays video. (Step 483).

When the unfolding display button formed on the VR controller of the virtual reality device is pressed (YES in step 484), the user wearing the head-mounted display will see an unfolded spherical image displayed ( step 485).

A Windows application window can be placed in three-dimensional space as a plane (board) or as part of a gentle cylindrical surface (or spherical surface). When you start the PC viewer (playback software that controls the playback window 100), the app screen is displayed as part of a plane (board) or a gentle cylindrical surface (or a gentle spherical surface) in three-dimensional space. can be placed. Therefore, when you press the dual-screen display button on the PC viewer, a display is created with the PC viewer operation section on one side and the playback video on the other side.

As mentioned above, there is a display button that expands (texture mapping) a spherical image over the entire interior of the sphere, starting from the location where the person is in the VR space. By pressing this button, you can view the video as if you were actually inside the car. The sphere may be displayed at the back of the Windows window, but the window may also be displayed in a state where all windows are hidden so that the user appears completely inside the car. It would be a good idea to be able to switch between the two companies using a button on the controller. The ball may also be displayed in front of the window. Change your position to the outside of the sphere (if you move forward, you will go outside the sphere and can operate the operation window etc. For example, it is a good idea to expand the image outside the sphere while you are outside the sphere. If you do this The image can also be viewed from outside the sphere.Instead of developing it into a perfect sphere, it may be mapped onto a curved surface that is set to correct distortions of optical systems such as lenses.Projecting the image It would be good to have a function to make the spherical surface semi-transparent.It would be good to be able to make it semi-transparent by pressing a button on the controller.The origin position and the radius of the sphere should be changed by moving the left and right sticks of the controller up and down, etc.Thumbnail Display, map display, acceleration graph, etc. should be arranged as independent panels in the three-dimensional space inside the sphere.Playback control functions such as play, stop, and next event should be configured so that they can be performed using a controller.

There is a VR headset, and when you start up the PC viewer, the VR screen is displayed like a board. Even if you play it back, the celestial sphere image will be played back on that board. When trying to display a two-screen display, it is best to create a configuration where it feels like the viewer's operation panel is on the left, and the panels of the video being played are on the right. A display button is provided that displays a spherical image inside the sphere, starting from where the person is in the VR space, and when pressed, the system enters VR mode. It is best to configure the display so that you feel like you are inside the celestial sphere. It is an image of sitting in the driver's seat. It feels like you are actually in the installation position.

It is best to configure the sphere to be displayed at the back of the Windows window. You can view the image of the ball while viewing the computer image (while operating the computer). Window operation The window can be hidden so that it appears in the video. Switch between both using the buttons on the control. Adjust the radius of the sphere using the stick on the controller. Adjust the origin position using the stick. The ball may be in front of the window. If you look behind you in that state, you will see a ball, but if the user moves and breaks the ball, you will see an image. If you look behind the ball in that state, you will see the ball, so if you paste the image on the back of the ball, you can see the image from outside the ball as well.

In reality, there is lens distortion, so the image develops into the shape of a rugby ball or egg. You can view images with lens distortion corrected. The window contains a map display, acceleration graph, and operation buttons, and these buttons and objects are assigned to the VR screen. Something like an operation panel is displayed as a panel in three-dimensional space.

FIG. 62 is a flowchart showing the processing procedure of the drive recorder 1, smartphone, and television device.

The process shown in FIG. 62 is for viewing video data recorded on the SD card 23 loaded in the drive recorder 1 on a television set located at home, office, office, etc.

A user at home, work, business, etc. uses a smartphone to send a playback command to the drive recorder 1 installed in the vehicle (step 501). When the engine of the vehicle is off, the drive recorder 1 is powered by a mobile battery or the like and receives a playback command.

When the playback command sent from the smartphone is received by the drive recorder 1 (YES in step 491), the video data recorded on the SD card 23 is read, and the read video data is transferred from the drive recorder 1 to the smartphone. (step 492).

When the video data transmitted from the drive recorder 1 is received by the smartphone (YES in step 502), the received video data is transmitted from the smartphone to the television device (step 503).

When the video data transmitted from the smartphone is received by the television device (YES in step 511), the video is played back and the video captured by the drive recorder 1 is displayed on the display screen of the television device. becomes (step 512). Television equipment has the ability to connect to smartphones via Wi-Fi, and uses Wi-Fi to receive and play video data sent from smartphones. Alternatively, the television device may be connected to a LAN in a home, office, office, etc., and connected to the Internet. Video data will be transmitted from the smartphone to the television device via the Internet.

When you say something like ``play pandas on a video playback site'' to the smart speaker, the TV connected to the Wi-Fi device turns on and the panda video plays on that TV. When you don't need to make detailed instructions, such as when you want a video to be played on your TV as BGV, it is very convenient to be able to give rough instructions via voice and have it played back on your TV. Many people do not play back the video on their dash record unless an accident occurs, and the video they take is rarely used. Furthermore, even though the drive recorder was not working properly before the accident occurred, if an accident occurs while using it without knowing this, the important accident video will not be recorded. Such a situation is possible.

To enable playback of drive record video with audio. You can operate the device using a smartphone, etc. in the car, but if the car is near your house and you can receive radio waves such as Wi-Fi, it is better to operate it from a smartphone, smart speaker, communication robot, special device, etc. at home. good. When you say to a voice recognition device (smartphone, smart speaker, communication robot, etc.), "Play the video from my dash record," it turns on the power to the dash record, and then broadcasts "my" video from my dash record via wireless communication such as Wi-Fi. Stream the video and play it on the TV (powered by battery if the engine is off). When you say, "Play the video from yesterday's drive recorder," search for yesterday's drive recorder video on the drive recorder or the storage device (smartphone or PC) that stores the data already transferred from the drive recorder, and if there is one, play it. For example, they said, ``We couldn't find any footage from yesterday's drive recorder.'' It is a good idea to play back the video from the closest drive recorder instead, such as "I will play the video from the drive recorder from the day before yesterday instead." If you say, "Play the video from the last time I went out," it will play the most recent video from the dash record. When playing back the drive recorder, it is preferable to play back the audio recorded along with the drive record video, but it is also possible to provide a function to play back other music or the like as BGM. If you say, ``Set the background music to ___'', the system will change to the drive recorder audio and play that song along with the video on music streaming services such as Spotify. It is best to perform digest playback, focusing on the parts where events occur. It would be good to be able to specify the playback time, such as ``Play the video on the drive recorder in about 5 minutes.'' At this time, for example, the videos are played back at a specified time. It is preferable that unplayed videos are played back with priority.

When a voice saying "Save this video" is recognized while the video on the drive recorder is "playing", the video file being played is transferred from the drive recorder to a storage device (smartphone, PC, etc.) via Wi-Fi etc. Forward. You can also upload it to video posting SNS and share it. When a voice saying "Save this video" is recognized while the video on the drive recorder is "recording", the record is transferred from the drive recorder to a storage device (smartphone, PC, etc.) via Wi-Fi, etc., a predetermined time before that point. The recorded video from the beginning and video data from that point until a predetermined time later are used as an audio instruction trigger to record the video on the drive recorder. When a voice saying "Send video" is recognized while the drive recorder is "recording" video, the recorded video from a predetermined time before that point and the video data from that point are sent to the server, etc. using LTE, etc. Send. You can also upload it to video posting SNS and share it. When the voice saying "Stop video transmission" is recognized, the transmission of the video to the server etc. is stopped. When a voice saying ``Send me the video up to this point'' is recognized while the camera recorder is ``recording'' video, the video that has been recorded from a predetermined time up to that point is uploaded to the server using LTE, etc. You can also upload it to video posting SNS and share it. When you say ``Start live streaming'' in front of the drive recorder, live distribution of the drive recorder video starts.

There are AI speakers, speakers connected to the cloud, and devices that connect smartphone images to a TV. On a smartphone, you have to specify each video one by one, but if you say ``play a panda video,'' the related video will continue to play automatically. The video on the drive recorder will not be played unless an accident occurs. In some cases, the drive recorder may not be working properly when an accident occurs. Playback video on a drive recorder based on voice instructions. In the car, use voice commands to operate the drive recorder on your smartphone. If your house and car are close to each other and the Wi-Fi signal from your home can reach your car, you can play the video from your car's dash recorder from home via Wi-Fi, and then watch the video from your dash recorder on the TV inside your house. Reproduce. Voice recognition technology allows you to stream images from your dash recorder to smartphones, smart speakers, robots, etc., which automatically turns on the power to a dash recorder near your house and broadcasts the image from your own dash recorder over Wi-Fi. Transfer and play on TV. Instead of giving a playback command to an AI speaker and playing a YouTube video, it gives a playback command to an AI speaker and plays pre-registered video on a smartphone, TV, etc. If the car's engine is off, it will run on battery power. The drive recorder itself only needs to function to receive instructions. When instructions come via Wi-Fi, the entire device is turned on and runs on battery power to transfer images. The cigarette socket burns out when the engine is turned off. The cigarette lighter socket connects to a mobile battery, and the mobile battery supplies power to the drive recorder. Up until now, I had to remove the SD card and put it into my computer to play it, but now I can easily view it.

If you ask me to play the video from yesterday's drive record, the video from yesterday's drive record will be played. You can specify the date of playback video. If it's not found, you can say ``Not found,'' or ``You can play the video from the day before instead of yesterday's video.'' If you say the video from when you just went out, you can play the ``latest drive record video.'' There is a designation function called "when". The drive recorder also records audio. Continuous recording and event recording also include audio. You can play the music as is by streaming, but if you want to do something about the BGM, the BGM will be played from a music streaming service instead of the audio from the drive recorder. You can watch driving record videos with music as BGM. The playback is a digest playback focusing on the part where the event occurs. There's no point in letting the long ones flow lazily. Skips about 30 seconds of the event and plays it back. To enable specifying the playback time of video on a drive recorder. Adjust by playing a digest of the event for about 5 minutes. Give priority to playing unplayed videos. Record that it has been played on your drive recorder, smartphone, etc. If you want to watch a video later while playing a video on your dash record, you can say ``Save this video,'' and the file will be transferred from your dash record to your smartphone via Wi-Fi.

If you are driving in the car while recording video on your dash recorder and say, ``Save this video to your smartphone,'' the dash recorder will transfer it to your smartphone if you have one in the car. If you say, "Save this video on your drive recorder," it will be recorded on your drive recorder as an event record. You can also say, "Start recording this video" or "Stop recording this video." It may be recorded in a separate area as voice instruction recording data. If you want to send this video while recording, it will be sent to the server using LTE. If you say "Stop sending video," it will stop sending the video to the server. If you want to send the video up to this point during recording, the constantly recorded data up to the previous set time will be sent to the server via LTE. When you say "Start live streaming" on Dorareko, the live streaming starts. Keep uploading it on stream.

[13th Example]
63 to 65(A), FIG. 65(B), FIG. 65(C), FIG. 65(D), and FIG. 65(E) show the thirteenth embodiment.

The 13th embodiment concerns thumbnail image display.

FIG. 63 is a flowchart showing the processing procedure for displaying thumbnail images in a playback device such as the personal computer 50. FIG. 64 is an example of displaying thumbnail images. FIGS. 65(A) to 65(E) are examples of display formats of thumbnail images.

This embodiment is suitable for displaying thumbnail images of circularly recorded images shot by a hemispherical camera, a spherical camera, etc., but can also be applied to displaying thumbnail images of other images. .

Referring to FIG. 64, SD card 23 on which video data representing a plurality of videos is recorded is loaded into personal computer 50, playback software is started, and playback window 100 is displayed on the display screen of display device 52. When displayed, the thumbnail images Ith1 to Ithn of the plurality of videos are displayed, for example, in the play list display area 141 of the playback window 100. When the cursor 531 is placed on one of the thumbnail images Ith1-Ithn of a plurality of videos and clicked (YES in step 521 of FIG. 63), the display format of the thumbnail image is displayed on the clicked thumbnail image. A pop-up menu 530 for selecting is displayed (FIG. 63, step 522).

Referring to Figure 64, the pop-up menu includes thumbnail image display menus for "Fisheye display," "Enlarged flat display," "Panorama display," "Ring display," and "Dome display." There is.

"Fisheye display", as shown in FIG. 65(A), displays a still image of a video shot using a fisheye lens as a thumbnail image. As shown in FIG. 65(B), the "enlarged flat display" is for displaying a part of interest to the user as a thumbnail image, enlarged more than other parts. As shown in FIG. 65(C), the "panoramic display" displays a horizontally long 360-degree image in the form of a thumbnail image. "Ring-type display" is to display thumbnail images in a ring-type manner, as shown in FIG. 65(D). The "dome-shaped display" is to display thumbnail images in a dome shape, as shown in FIG. 65(E).

The display method for one of the thumbnail images is specified using the pop-up menu 530 (step 523). If specified (YES in step 523), the clicked thumbnail image is displayed in the specified display method (step 524). Even if there are multiple thumbnail images, each thumbnail image can be displayed using a different display method.

In software that plays and manages video files, a thumbnail display is known in which the first image of a video is displayed in a reduced size in order to list the videos for purposes such as selecting a desired video. This kind of thumbnail display usually displays a still image at the beginning of a file or a specified frame in a reduced size. However, since the image of the half-celestial camera is recorded in a circular shape, it is better to have a function that allows you to select the display state of the thumbnail display, but it is not possible to change the display format of the thumbnail, and the display range cannot be changed. Common. On the other hand, in a certain smartphone app, you can switch between thumbnail displays by tapping the icon, change the display format of the thumbnail display, and move the display range of the video by flicking the thumbnail, but the thumbnail display method is , All are switched at once.

To this end, as described above, a function is provided to change the thumbnail display method and display range "for each individual thumbnail." Right-clicking on an individual thumbnail displays a pop-up menu that allows you to select how the clicked thumbnail is displayed. As the type of thumbnail display method, the above-mentioned method is preferable. As described above, display methods such as fisheye display, panoramic display, enlarged flat display, ring-shaped display, and dome-shaped display may be provided. Further, original, twin cut, panoramic cut, and angle cut may be provided. Left-click on an individual thumbnail to change the video display range. It is a good idea to change the display range. It is a good idea to provide menus for "rotate 180 degrees" and "rotate 90 degrees" in the right-click menu. In the case of two cameras, it is recommended to provide "camera switching". It has a function to save the changed thumbnail display method and display range.

As a general rule, it is best to configure the configuration so that the changes made are maintained. It has the ability to undo and redo changes to the thumbnail display method and display range. It has a function to match the display method and display range of thumbnails among thumbnails. If you left-click while holding down the shift key, a selection frame will be drawn around the thumbnail. Displayed by right-clicking on one of the thumbnails when there are multiple selection frames. Changes all thumbnails with selection frames in the display method selected in the pop-up menu. Also, when there are multiple selection frames, if you drag one thumbnail to the left, the status of all the thumbnails with selection frames in the range displayed by that thumbnail will be linked and changed in real time. Note that the selection frame is preferably released (selected state is released) when the left click on a portion other than the thumbnail is released.

It is preferable to provide a row for displaying thumbnails that are taken at the same time (for example, the same frame) but have different thumbnail display methods or display ranges. For example, with a 360 degree x 2 camera, a row of first thumbnails with a 360 degree front fisheye display and a second row of thumbnails with a 360 degree rearward fisheye display are created, so that thumbnails from the same time are in the same column. to draw. It is preferable to have a function that allows the number of thumbnail rows to be added arbitrarily. Adding thumbnails is best done by creating a space between the lines in the thumbnail display area, right-clicking on that space, and selecting "Insert Row" from the pop-up menu that appears. This pop-up menu may also include items such as ``Delete top line'' and ``Delete bottom line.'' It is recommended that rows or columns that exceed the display range of the screen can be scrolled using a scroll bar. For example, create 10 thumbnail rows with a 360 degree x 2 camera, and from the top: front fisheye display, rear fisheye display, panoramic display centered on the front, panoramic display centered on the back, panoramic display centered on the bottom. Enlarged flat display, enlarged flat display with the upper part stopped, ring-shaped display with the front as the most visible part, ring-shaped display with the back as the most visible part, dome-shaped display with the front as the most visible part, back It is best to display it like a dome display with the most visible part. These types of displays at the same time will be displayed in the same column. You can also scroll left and right to move the columns left and right, allowing you to see thumbnails of the entire page.

It is preferable to provide a function to play back the image of a thumbnail when a thumbnail is selected by left-double-clicking or the like, and it is particularly preferable that the playback starts in the display method and display range set as the thumbnail. In the pop-up menu that appears when you right-click in the playback window, there is an option to "Set this status as thumbnail." When this is selected, the current display method and video within the display range will be set as the thumbnail of that video. It would be good to have a setting function. Thumbnail display is a function that creates an area to display the original image corresponding to the right-clicked thumbnail, and clearly indicates the display area of the current thumbnail in that original image (surrounding it with a line, filling it with semi-transparent color, etc.). It is a good idea to provide In this way, it is possible to easily grasp which part of the original image is set as the thumbnail.

It would be good to have a function that clearly shows "the parts that connect images." For example, draw a red line on the stitched part. As the stitching accuracy increases, the joints become harder to see. You don't want to use seams in commercial photographs, etc., but if it becomes difficult to see, you may end up cropping across the seams, and it may become a problem if broken seams are discovered after delivery, such as after commercial printing. If you have such a function, you can deal with it.

Equipped with a "function that clearly shows the reflected parts of the main body." For example, it is preferable to provide a function of shading or semi-transparently filling a portion corresponding to the main body portion. An ``area that displays whether or not the image has been blurred'' is provided on the viewer screen. You may display an icon indicating that the video being played is a blurred image, or display an icon indicating that the image is a blurred image in the list of file names. You can. In connection with "wireless live streaming," it has a function to record along with live streaming, and stores information that lets you know which recorded video is being streamed, and when playing the recorded video on a viewer. It would be good to have a function to display whether or not it was live streamed. You can simply record what was live-streamed and what was not in separate files, and store information on whether it was live-streamed or not in the file properties, but if there is one thing in particular, It is a good idea to record the live streaming range and the non-live streaming range in the file so that they can be distinguished, and display the two areas on the timeline so that they can be distinguished.

It is preferable to record the video with information identifying it as a time-lapse photographed video, and to provide a function to display whether or not the video was time-lapse photographed when played back on a viewer. It would be good to have a function that handles a series of time-lapse images as one set. It is preferable to have a function to add and record information on the video from which device the recording start/recording end instruction/setting instruction came, and a function to display this information on the viewer. It would be good to have a function to operate the parts corresponding to buttons on the viewer using voice recognition, especially a function to change the display method and display range. It is also preferable to have a function to change the recording method and recording range using voice recognition.

The half-sphere camera records in a circle. Some allow you to select the state of thumbnails. Things like long thumbnails and things that you can switch between, and things that spin around when you flick them. Switching thumbnails will switch multiple thumbnails at once. Since this file cannot be used as a spherical or long thumbnail, it would be good to have a function that allows this. It would be a good idea to have a function where you can right-click on a thumbnail and a pop-up menu will appear, allowing you to choose the display method. There are display methods for thumbnails of forklift drive recorders, fisheye display, panoramic display, flat display, etc., and it would be good to be able to specify these display methods for each thumbnail. It would be good to have a function that allows you to change the way the image is displayed by left-clicking. It would be good to have a function that can be moved by dragging within the thumbnail. Add options such as 90 degree rotation and 180 degree rotation in the right-click menu. In the case of two cameras, camera switching is provided. It would be good to have a function to save the changed thumbnail display method and display range. It would be nice if the thumbnail state could be saved. After turning, save the turned state. This can be used when you only want to see the driver.

Add a button to undo changes to the thumbnail display method and display range. I tried the operation, but it resets to the initial state. I moved it a little, but I canceled it and returned it to its original state. It would be nice to have a function to align thumbnails. If you want to align all the thumbnails within the range of the driver, it would be difficult to align them one by one, so for example, if you hold down the control key and left-click, an outer frame will be created, and if you right-click, a pop-up menu will appear. A display method appears. Selecting this will change the display method all at once. If you drag the contents while there is a selection frame, everything should move. If you click on an unrelated area, the selection frame will be canceled. It is preferable to provide a row for displaying thumbnails that are considered to be the same time but have different thumbnail display methods or display ranges. I want to create a thumbnail row with a different perspective. Although there is only one thumbnail row for each page, it would be good to have a function to display thumbnails from different perspectives. For example, a 360-degree camera may display thumbnails from the front camera and thumbnails from the rear camera. It is better to add more rows, but the columns should be at the same time. It would be good if the number of thumbnail rows could be added as desired. It would be good to be able to view it from a variety of display methods and viewing angles. It is a good idea to leave a space between the rows of thumbnails and right-click in that space to insert and display a new row of thumbnails. Basically, when you insert a row, the copied line should be displayed. You may change them individually, or you may display a selection frame and change them all at once.

Select a thumbnail by double-clicking on it to play the thumbnail video. This is done using the display method and display range set for thumbnails when starting playback. If the file is displayed as a circular thumbnail, double-clicking it will play it in a circular format. If thumbnails are displayed in a long and narrow state, the video is played in the same way. The video being played is turned into a thumbnail. If you right-click in the playback window, a menu will appear that allows you to make thumbnails of the displayed video. The default should be the first frame of the file.

It would be good to have a function that shows where it is in the whole. Displays which part of the displayed video is displayed by the thumbnail.

With a full celestial sphere, it is necessary to connect the front image and the rear image, but since the connecting part is noticeable, it would be good to have a function that clearly draws a line where it is connected. When you press the button on the PC viewer, a line will be displayed. Reduces glare. Parts of the camera body, such as hands, are processed and erased, but it would be good to have a function that clearly indicates these parts, such as by shading them, so that they can be seen. In the case of a video with automatic face blurring, it is preferable to display on the PC viewer that the video has face blurring. You may also want to automatically blur faces on sites where you upload video from your drive recorder. A recording function is provided along with live streaming, and commands to start and end streaming are always recorded. You can see where in the recorded video you streamed. It may be displayed in pink in the live streaming section and slide bar. You can set time-lapse photography to, for example, take still images every hour or take 10 seconds. It is a good idea to always include a mark in the record to indicate that the photograph was taken as a time-lapse photograph. Videos shot in a time-lapse manner may be combined into one video.

Shooting starts based on instructions from some device. For example, when a door opens, an instruction comes. Record the device from which recording started and ended. Receive and record device ID, etc. Give each viewer button a name. Change settings by voice. Once set, the drive recorder will respond with the settings and begin recording with those settings.

Additionally, a SIM contract is required in order to upload video data shot from a drive recorder using a mobile phone network such as LTE to the cloud. Up until now, drive recorder video viewers have not had any functions related to fees or data capacity. Therefore, there was a problem in that the viewer could not tell how much communication was being done or whether the rate plan was appropriate.

Drive recorders for corporations can have functions that support a variety of communication rate plans, depending on how the corporation uses the vehicle.

Low-speed flat-rate unlimited capacity plans, nighttime special plans, uplink special plans, corporate share plans, etc. It would be nice to have a feature related to low-speed, flat-rate unlimited services that cost 300 yen to 500 yen per month. A function that supports SIMs that do not allow high-speed communication and have a speed of around 200 kbps. With 200 kbps, it is possible to send audio and still images (jpeg) at speeds faster than real time. Even at monaural 64 kbps, the sound quality is sufficient to determine whether or not the sound is an accident sound. Therefore, past data can be transmitted in a shorter time than real time. A function that calculates a bit rate that is shorter than the actual time by combining past minutes from the current (recent history, etc.) communication bit rate or SIM contract bit rate, compresses it at that bit rate, and sends it. . A function to send the past part as a compressed file and the current part by streaming (IP phone may also be used). A SIM that has a data limit of up to a certain amount of data for a certain period of time, such as 366MB for 3 days. It would be good to have a function to display the remaining transferable capacity on the drive recorder and viewer. As you near your data limit, it's a good idea to limit the types of information you send. For example, you may want to change the settings so that only audio data is sent without sending photo data. A low-speed dedicated SIM that allows burst transfer (burst function: outputs about several Mbps for the first 1-2 seconds). At the beginning of sending when an event occurs, several photos (or even just one photo) taken at different times related to the event are sent, and then audio data is sent. Photos are transferred in bursts.

Features compatible with night-time special plans. One company's ``night-only plan'' is limited to a maximum download speed of 200 kbps between 6 a.m. and 10 p.m., but during the night from 10 p.m. to 6 p.m., it allows for high-speed communication with a maximum download speed of 375 Mbps and a maximum upload speed of 50 Mbps, with 3GB/upload speed. Low monthly plan. It would be good to have a function that allows you to set times when high-speed communication is possible or times when communication is restricted, and to upload videos all at once during times when high-speed communication is possible. In particular, it would be good to have a function that automatically turns on the power to the drive recorder of a parked vehicle with the engine off during times when high-speed communication is possible, and starts transmitting video data.

Features compatible with upstream specialized plans. One company's ``uplink specialized plan'' keeps the downlink data communication speed as low as a maximum of 200kbit/s, but instead secures a wide uplink data communication speed of up to 50Mbit/s for sending data. The plan I did. 1.1GB is relatively low cost. Since it seems possible to secure the downlink IP phone bandwidth, it would be a good idea to have a function from the server that limits the downlink communication to instruction commands and IP phone communication. In other words, even if a drive recorder has a function of receiving video from another drive recorder, for example, it is preferable to limit that function.

Features compatible with corporate share plans. In particular, plans for corporations are expected to use data sharing plans that allow, for example, 50 GB to be shared among 20 lines. It would be good to have a function to obtain information from the management screen prepared by the SIM provider, etc. via scraping or API, count the SIM usage status on the drive recorder, obtain this information at the center, and display it on the viewer. It would be good to have a function to compare and display information. It is advisable to provide a function that displays on the viewer the total amount of packet usage and remaining usable amount of the drive recorder that is sharing packets. It would also be helpful to have a function that issues an alert when the remaining capacity is getting low, and a graph of predicted capacity usage. It would be good to have a function that displays the individual packet usage and remaining usable amount of packet sharing devices in a ranking format on the viewer. For example, by looking at the event status of a vehicle with a higher-ranked drive record, it is easy to verify whether the driver is driving poorly or if the drive record settings are incorrect. For companies that have set up multiple sharing groups, it would be a good idea to have a function that allows you to compare and display the usage status between the sharing groups. In this way, it can be used as a reference for optimizing the SIM allocation of each drive recorder to each group. It is also possible to use automatic analysis using AI to issue advice to move a specific SIM to another group. It is a good idea to display the file capacity in the viewer's display target selection field.

In the case of an SMS-compatible SIM, it is recommended to have a function that, when a trigger occurs, generates a character string with the importance of the trigger factor, such as information about the degree of maximum G, and sends it to the center phone number. It is preferable that the center (for example, a viewer) determines whether or not to send the transmission request instruction to the drive recorder based on the SMS data. It is preferable that the center determines information such as the type of data to be transmitted, such as video, and the transmission range to the drive recorder, and sends a transmission request instruction to the drive recorder.

In the case of a SIM that supports voice calls, the drive recorder should be equipped with a voice call function, and the center should be equipped with a function to transfer outside calls made to the center (sales office). The center should have a function that obtains the operating status of the drive recorder using LTE, etc., and prevents outside line transfers under set conditions, such as when the drive recorder is in a running state. It is a good idea to equip your drive recorder with a function that allows you to make a call to the center based on the situation of the drive recorder, such as comparing rates between IP phones and voice calls, the person you are calling, your past driving history, etc. It would be good to have a function that allows calls to be made by voice in the event of a strong shock, or by IP phone in the case of a weak shock, and a function to communicate with the center using VoIP and connect to an outside line by telephone.

When public wireless LAN and LTE are used together, a connection function is provided to the public wireless LAN, and LTE sends restricted data, while public wireless LAN sends unrestricted or loosely restricted video data. It would be good to have a function that allows you to upload from convenience stores' free Wi-Fi, etc.

There are many types of pricing plans for drive recorders for corporations. In the case of low speed and low rate 200k, only the audio is sent first. The server identifies whether the sound of the accident is included, and then sends a video transmission request to the drive recorder to see the accident. You may also send several still images. It is better to vary the bit rate. The past portion is sent as a compressed file, and the current portion is sent as a streaming file. Past events are made into files and sent little by little, and current events are sent as audio and video stream data. The image is that if an accident occurs, an IP phone will be connected. Audio comes into the center. For example, SIM has a limit on the amount of data sent within three days. Restrict users who use a large amount. For such SIMs, the usage amount is displayed on the display screen of the drive recorder. You can see the remaining transmission amount. The viewer can also send the capacity in real time and display it in a list. As you near your data limit, it's a good idea to limit the types of information you send.

Some low-speed SIMs are dedicated to burst transfers. The first 1-2 seconds will produce a few megabytes, but after that it will be limited to 200k. Video related to the event is sent first, and audio is sent later. Videos are sent during times when high-speed transmission is possible, such as on special night plans, and only audio is sent during restricted hours. Send a video from the drive recorder of a parked car at night. In the case of uplink special price plans, downlink is restricted. For example, when receiving video from another person's drive record, it is restricted. With the corporate share plan, it can be acquired and viewed on the viewer side. The usage status of the SIM may be acquired by the viewer.

It is recommended to compare and display the carrier usage amount with the actual value. It will be displayed when the limit is near. It would be a good idea to list the rankings of packet usage. Display usage by group. It is best to display the file capacity.

In the case of a SIM that supports SMS, a character string indicating the time and minute of the trigger occurrence is generated and sent to the drive recorder based on the SMS data. It's best to use something like an old-fashioned pager. Send only important data by instant SMS.

If you have a voice-enabled SIM, you can make calls using a regular phone number, so outside calls to the center will be forwarded to the phone number on your drive recorder. You can talk hands-free using a drive recorder. In the event of a strong impact, a voice call will be made, and in the event of a weak impact, an IP phone call will be made. Use a stable network if there is an emergency. Calls to the center are made via VOIP, packet communication, or IP telephone.

In the case of public wireless LAN, the drive recorder is equipped with a public wireless LAN connection function, and when connected via public wireless LAN, records are constantly updated.

The scope of the present invention is not limited to the configurations explicitly described in the specification, but also includes combinations of various aspects of the invention disclosed herein. Of the present invention, the structure for which a patent is sought has been specified in the attached claims, but currently, even if the structure is not specified in the claims, it is not disclosed in this specification. The applicant intends to make such a configuration the scope of a patent claim in the future.

The present invention is not limited to the configuration described in the embodiments described above. The components of each of the embodiments and modifications described above may be arbitrarily selected and combined. Also, any component of each embodiment or modification, any component described in the means for solving the invention, or a component that embodies any component described in the means for solving the invention. It may be configured in any combination. The applicant intends to obtain rights to these matters through amendments to the application or divisional applications.

In addition, the applicant intends to acquire rights to the entire design or partial design by filing a conversion application to a design application. Although the drawing depicts the entire device using solid lines, the drawing includes not only the overall design but also a partial design that claims some parts of the device. For example, it is a drawing that not only includes some parts of the device as a partial design, but also includes some parts of the device as a partial design regardless of the components. The part of the device may be a part of the device or a part of the device. We intend to obtain rights not only for the entire design, but also for partial designs in which any part of the solid line part of the drawing is a broken line part.

1: Drive recorder, 3: Windshield, 4: Room mirror, 5: Cigarette socket, 6: Power cable, 10: SD card slot, 11: Display, 12: Operation buttons, 13: Joint rail , 14: Memory card reader/writer, 15: Speaker, 16: GPS receiver, 17: Camera, 18: Acceleration sensor, 19: Communication circuit, 20: Controller, 20b: ROM, 20c: RAM, 20d: Timer , 21: LTE module, 23: SD card, 30: Recording settings window, 30a: Recording method selection frame, 30b: Recording resolution selection frame, 30c: Recording frame number selection frame, 30d: Event recording frame number selection frame, 30e: Event recording time selection frame, 30f: Transmission setting frame, 30g: First transmission resolution setting frame. 30h: First transmission frame number setting frame, 30i: Second transmission resolution setting frame, 30j: Second transmission frame number setting frame, 30k: Save instruction button, 50: Computer, 51: Display control device, 52: Display device, 54: Memory card reader/writer, 55: Input device, 56: Memory, 57: Communication circuit, 58: CPU, 96: Input device, 100: Playback window, 101: Menu bar, 102: Folder・Icon, 103: Camera icon, 104: Printer icon, 105: Reel icon, 106: FD icon, 107: Log icon, 108: Gear icon, 111: Video display area, 112: Video, 112a: Video part, 112b: video part, 113: video, 113a: video part, 113b: video part, 114: video, 114a: video part, 114b: video part, 121: map image display area, 131: vehicle information display area , 132: Vehicle status display area, 133: Operation button, 134: Interior emphasis button, 135: Normal display button, 136: Time display area, 138: Acceleration graph display area, 141: List display area, 160: Video display window , 161: Menu, 162: Cursor, 180: Display screen, 190: Display screen, 200: Computer, 201: Administrator, 210: External display device, 211: Driver, C1: Brightness conversion curve, C2: Brightness Conversion curve, t3: Time of event occurrence

Claims (17)

A system capable of displaying images taken in a vehicle,
A means for processing video data for the above display is provided, and the display is configured to slow down the playback speed of the video in the left or right direction relative to the normal playback speed of the video in the front or rear of the vehicle. Process video data for,
system. Processing the video data for the display so that an area for displaying information regarding the vehicle obtained while driving the vehicle in response to the enlargement command is larger than the size before the enlargement command was given. ，
The system according to claim 1. Processing the video data for the display so that an area for displaying information regarding the vehicle is larger than an area for displaying images shot by the vehicle;
A system according to claim 1 or 2. Processing the video data for the display so as to notify that the system does not have the ability to process the video data for the display in the system whose performance does not meet a predetermined condition;
A system according to any one of claims 1 to 3. processing the video data for the display so as to limit the number of times the video data representing the video shot in the vehicle is transmitted;
A system according to any one of claims 1 to 4. Performing a communication speed test before recording video data representing images taken in the vehicle, and processing the video data for display so as to transmit the video data under conditions obtained based on the communication speed test.
A system according to any one of claims 1 to 5. For the above-mentioned display to notify or record at least one of the existence of images that are different from the human-visible image when the image taken by the vehicle is different from the human-visible image. Process the video data of
A system according to any one of claims 1 to 6. processing the video data for the above display so as to continuously display a plurality of videos related to the movements of a specific person;
A system according to any one of claims 1 to 7. For displaying the video data representing the video shot in the vehicle so that the video data recorded on the first recording medium and the video data recorded on the second recording medium are different video data. Process the video data of
A system according to any one of claims 1 to 8. Processing the video data for display so that the video is captured in the vehicle based on a video capture command from outside the vehicle that captures the video;
A system according to any one of claims 1 to 9. processing the video data for display so as to display a recording frame rate and a display frame rate of the video shot in the vehicle;
A system according to any one of claims 1 to 10. Processing the video data for the display so as to give movement commands to the robot based on the playback of the video shot in the vehicle;
A system according to any one of claims 1 to 11. The video captured by the vehicle is a spherical video, and the video data for the display is processed so that the spherical video is displayed in the virtual reality space.
A system according to any one of claims 1 to 12. There are multiple videos taken in the vehicle, and the video data for the display is processed so that the display format of the thumbnail image is changed for each video.
A system according to any one of claims 1 to 13. acquiring information regarding the posture of the device held by the user's hand, and processing the video data for display so as to change the display range of the video data based on the information regarding the posture;
A system according to any one of claims 1 to 14. A method for controlling a system according to any one of claims 1 to 15. A computer readable program for controlling a computer of a system according to any one of claims 1 to 15.

Images