JP7450287B2 - Playback device, playback method, program thereof, recording device, recording device control method, etc. - Google Patents

Description

The present invention relates to, for example, a reproducing device, a reproducing method, a program thereof, a recording device, a method of controlling the recording device, and the like.

Vehicle-mounted video recording that captures images of the surrounding area of the vehicle using a camera installed in the vehicle, and records surrounding images and vehicle speed when the vehicle is impacted by an event such as sudden braking, sudden steering, or a collision. There are known playback devices that play back video captured by such a drive recorder. (Patent Document 1).

Patent No. 5966131

However, in the invention described in Patent Document 1, only captured images, such as images before and after the occurrence of an event, are played back.

An object of the present invention is, for example, to increase the possibility of displaying images that are more suitable for the user than in the past. The purpose of the invention of the present application is not limited to this, and we intend to acquire rights through divisional applications, amendments, etc. for structures that aim to obtain effects from the parts of the structure disclosed in this specification, drawings, etc. . For example, the present specification discloses a problem in which the phrase ``can be done'' is replaced with ``the problem is.'' Each of the problems is described as independent, and we intend to obtain rights for the structure to solve these problems independently through divisional applications, amendments, etc. Even if the problem is understood implicitly from the description, the applicant has the intention to claim a part of the structure described in the specification in an amendment or divisional application. .

(1) A playback device that plays back recorded video data, which processes the recorded video data based on acquired information and controls to display a video based on the processed video data. It's good to do.

In this way, the user can view a video that has been processed based on the acquired information, rather than the recorded video itself. By performing processing suitable for the user viewing the video, it is possible to obtain a video suitable for the user. Rather than simply processing recorded video, recorded video data is processed based on the acquired information, increasing the possibility of displaying video that is more suitable for the user viewing the video. For example, a playback device that plays back video data recorded in a vehicle, processes the recorded video data based on information acquired in the vehicle, and displays a video based on the processed video data. It is a good idea to carry out controls for this purpose. In particular, control is provided to process the recorded video data based on the video data recorded in the vehicle and information acquired in the vehicle at the same time, and to display a video based on the processed video data. It's good to do. The same period may be within a predetermined width of time, and the predetermined width may be, for example, several seconds, but it is particularly preferable that the same period is within a predetermined width of time.
The information to be acquired is preferably information that changes with the driving operations of the driver of the vehicle, and is particularly preferably information regarding the driver's own behavior. The information regarding the driver's own condition may be information that has a relationship between the driver's own condition and the video recorded in the vehicle. The relationship may be an objective relationship, but is particularly preferably a subjective relationship of the driver himself. As a subjective relationship, it is preferable to use the relationship that the driver thinks it looked like this, and for example, it is preferable to use the information necessary to infer that "it looked like this".
However, the information to be acquired may be information other than the speed of the car.

The recorded video data may be, for example, video data recorded by a recording device, or, for example, video data recorded on a memory card such as an SD (Secure Digital) card, a micro SD card, or an XQD (registered trademark) card. Data is fine. The acquired information may be, for example, information input by a user viewing the video. The user inputs information and the playback device obtains that information so that processing is performed that is appropriate for the user viewing the video, increasing the possibility that the video will be displayed that is suitable for the user viewing the video. . Further, the acquired information may be predetermined information. Information may be acquired by reading such predetermined information. For example, the processing only needs to change the video data before and after the processing.

(2) The processing may include processing to display a visual field range of the user on the video.

In this way, the range of the user's field of view in the video becomes easier to understand. Since the range of the user's field of vision is known, it becomes easier to see, for example, parts that the user cannot see and parts that the user can see. The user's visual field range may be fixed, may be determined according to the user's age, or may be determined by measurement for each user. In particular, the processing may be a process of displaying the user's field of view on the video based on the acquired information. In particular, processing may be performed to display the visual field range of the user on the video based on the range based on the information for adjusting the visual field inputted from the visual field range input means and the acquired information. Displaying the viewing range on an image means displaying it in a way that makes it possible to distinguish between the inside of the viewing range and the outside of the viewing range.For example, the viewing range may be displayed with a frame surrounding it, or the outside of the viewing range may be displayed with a frame. It may be displayed by filling it in or blurring it. The video may be, for example, a video recorded by a recording device using a wide-angle camera, or a video recorded by a recording device using a spherical camera.

(3) In response to the presence of an object to be noted in the image outside the user's field of view, a notification to that effect may be provided.

In this way, the user can easily understand that there is an object to be careful of in a part that the user cannot see. Whether or not there is an object to be noted in the video outside the user's visual field can be determined by, for example, detecting the object present in the video within each frame of the recorded video using image recognition, etc. In this case, it is preferable to determine that the object exists when the range of the detected object is outside the field of view, and that the object does not exist when it does not exist. In particular, in the object detection process, at least a part of the search for the object by image recognition within the visual field in the image of the frame may be omitted, and it is preferable not to do so. In the image of the frame, it is preferable that the search for the target object is performed by image recognition only in a range outside the visual field range. For example, if the video is captured by a recording device, it can be seen that there are objects that the user cannot see that need attention, so the user can move his or her head to pay attention to areas that cannot be seen while driving. It increases the possibility that you will be able to do it. Objects to be careful of may be, for example, pedestrians, bicycles, cars, motorcycles, etc. In particular, this may be notified by displaying the position in the video.

(4) The processing may be processing for displaying the user's line of sight on the video when the video data is recorded.

In this way, it becomes easier to understand where the user was looking when recording video data. In particular, the processing may be processing to display the user's line of sight at the time of recording the video data on the video based on the acquired information. In particular, it is preferable to display the user's line of sight in the video based on the user's line of sight input from an input means for inputting the user's line of sight at the time of recording video data and the information acquired in the previous period. For example, when recording video data, the movement of the user's eyes may be recorded, and the user's commands may be displayed on the video based on this recording. The line of sight is, for example, the direction in which the user is looking, and may also be the line of sight. Displaying the line of sight on the video may mean, for example, displaying the line of sight as points on the video in sequence, or displaying the locus of the line of sight on the video. For example, when recording video data, the user's line of sight may be determined by photographing the user with a photographing means and detecting the line of sight from the photographed image obtained by the photographing means, or by using a line of sight detection device such as eyeglasses for detecting the line of sight. The line of sight may be detected by a line of sight detection device such as glasses by having the user wear the eyeglasses.

(5) The processing may be a processing of moving the field of view in the direction in which the user is facing at the time of recording the video data and displaying it on the video.

In this way, the direction in which the user is facing when recording video data can be determined, and the viewing range in that direction can be easily determined. When recording video data, it becomes easier to see which part of the video was visible. In particular, it is preferable to acquire information indicating the direction detected by a detection means for detecting the direction in which the user is facing, and to move the visual field range based on the acquired information and display it on the video. The field of view can be moved by processing the frames so that the inside and outside of the field of view can be distinguished for each fixed frame. For example, the viewing range may be moved by relatively moving the viewing range, or the viewing range may be moved relatively by moving the image.

(6) For example, the processing may be processing for displaying on the video a range that the user has not seen during a certain period of time when video data is recorded.

In this way, it becomes easier to understand the range that the user has not viewed within a certain period of time. In particular, information obtained from a detection means that detects an area that the user has not seen during a certain period of time is acquired, and the area that the user has not seen during a certain period of time from the acquired information is displayed on the video. It is better to use processing. It is preferable to detect the user's line of sight and detect a range that the user has not looked at during a certain period of time from information on the detected user's line of sight and information on a certain period of time. For example, when recording video data, the user's line of sight may be determined by photographing the user with a photographing means and detecting the line of sight from the photographed image obtained by the photographing means, or by using a line of sight detection device such as eyeglasses for detecting the line of sight. The line of sight may be detected by a line of sight detection device such as glasses by having the user wear the eyeglasses. If the video data is recorded by a recording device, the certain period of time may be, for example, the period of time when the vehicle is stopped at an intersection, while turning right or left at an intersection, or while changing lanes. Displaying the unviewed range on the video may be done by, for example, filling out the unviewing range or blurring it.

(7) For example, the processing may be processing for displaying on the video whether or not the user's line of sight at the time of recording the video data matches the direction in which the user should look.

In this way, it becomes easy to tell whether the user is looking in the direction he or she should be looking or not. In particular, it acquires information from an input means that inputs information indicating whether the user's line of sight and the direction in which the user should view when recording video data matches, and records the video data based on the acquired information. It is preferable that processing is performed to display on the video whether the user's line of sight at the time matches the direction in which the user should look. In addition, information input from an input means for inputting information indicating the user's line of sight when recording video data and information input from an input means for inputting information the user should see, and based on the acquired information, Preferably, processing processing may be performed to display on the video whether the user's line of sight at the time of recording the video data matches the direction in which the user should look. Information obtained from a detection means that detects whether the user's line of sight at the time of recording video data matches the direction in which the user should look is acquired, and based on the acquired information, the user's line of sight at the time of recording the video data is detected. It is preferable that processing processing is performed to display on the video whether the line of sight of the user matches the direction in which the user should look. For example, when video data is recorded by a recording device, the direction in which the user should view is determined in advance according to the surrounding situation. The degree of coincidence between the user's line of sight and the direction in which the user should look may be calculated.

(8) For example, when the video data is recorded by a recording device, the direction in which the user should look is determined by whether the user's line of sight at the time of recording the video data matches the direction in which the user should look. It is preferable that it differs depending on the driving operation of the car at the time of recording the video data.

In this way, it becomes easy to understand that the direction in which the user looks differs depending on the driving of the car at the time of recording video data or the user's driving operation of the car. The direction in which the user looks differs depending on the driving of the car or the user's driving operations when recording video data, so it is difficult to determine whether the user is looking in the direction they should be looking or not depending on how the car is being driven. It becomes easier to tell whether you are looking in the direction you should be looking or not depending on the operation. In particular, it is preferable to determine the direction in which the user should look based on information from an input means that inputs information about the driving of the car at the time of recording video data or the operation of driving the car by the user. The direction you should look is, for example, when the car is driving straight, you should look almost straight ahead, and when changing lanes, you should look in the room mirror, the door mirror of the lane you are changing lanes, It is a good idea to look out the window on the side where you are changing lanes. In addition, when driving a car, when operating the steering wheel relatively sharply after operating the turn signal to turn right or left, look diagonally forward and diagonally behind the side on which you operated the steering wheel, and use the turn signal to change lanes. Later, when you turn the steering wheel relatively lightly, it is a good idea to look at the room mirror, the door mirror of the lane you are changing lanes from, or the window on the side of the lane you are changing lanes from.

(9) For example, the processing process acquires information indicating whether the order in which the user viewed the video data when recording the video data matches the order in which the user should view the data in advance, and displays the information on the video based on the acquired information. It is recommended that the processing be performed as follows.

In this way, it becomes easier to understand whether the order in which the user views the images matches the order in which they should be viewed in advance. In particular, based on information from a detection means that detects whether the order in which the user viewed the video data when recording the video data matches the order in which the user should view the video data in advance, It is preferable that information indicating whether the order matches the power order is displayed on the video. The order in which a user looked when recording video data can be determined, for example, by analyzing video data obtained by photographing a user driving a car and detecting the user's line of sight in chronological order. The order in which the images should be viewed can be determined in advance based on information input from an input means for inputting information indicating the order. For example, the recorded video data may be played back, and the instructor or the like may input the order in which the videos represented by the played video data should be viewed from the input means. The order in which you should view them in advance may change, for example, depending on the circumstances at the time of recording. For example, if the video data is from when a car is starting, for example, look ahead, look at the room mirror or rear, look at the right side door mirror or fender mirror, and look at the right window. This is an example of the order in which you should look at them. The degree of agreement between the order in which the user viewed the images and the order in which they should be viewed may be calculated in advance.

(10) For example, the processing may be processing for displaying parts of the video that the user should see.

In this way, it becomes easier for the user to understand from the video which part the user should look at. In particular, it may be a processing process that acquires information input from an input means that inputs information indicating the places the user should see, and displays the places the user should see on the video based on the most virtuous information. For example, if the video data is the image seen by the driver while driving, it will be easier for the driver to see what areas to look at, which is often useful for driving guidance. For example, processing may include marking the image area of the rear-view mirror if the area is to be viewed in the rear-view mirror, or marking the image area of the door mirror if the area is to be viewed in the door mirror. It's good to do that.

(11) For example, when recording video data, if the user is not looking at the part he or she should be looking at, it may be possible to display an image of that scene.

In this way, it becomes easier to understand what kind of scenes the user is not looking at what they should be looking at. In particular, when recording video data, if the user is not looking at the part he or she should be looking at, based on the information input from the input means that inputs information indicating that the user is not looking at the part he or she should be looking at, the screen for that scene may be displayed. It is good to display it. To determine whether the user is not looking at the place he or she should be looking at, line-of-sight information obtained from a detection device that detects the user's line of sight can be used. For example, if the video data represents images of the user driving, it will be easier to understand in what situations the user is driving and when they are not looking at what they should be looking at, which may be useful in improving driving skills. It increases. Furthermore, for example, if the video data represents a video of the user driving, it is possible to make an evaluation based on an image of a scene where the user is not looking at what he or she should be looking at.

(12) For example, the video may be displayed at a faster display speed for a video portion in which there is no change in the movement of the subject than in a display speed for a video portion in which there is a change in the movement of the object.

In this way, there is a high possibility that the entire video can be viewed quickly. In particular, it is possible to detect whether there is a video part with no change in the subject's movement or a video part with a change in the subject's movement. What is necessary is to use a detection means that does this. The detection means preferably detects a difference between images between frames, and if the difference is less than a certain value, there is no change in the movement of the subject, and if the difference is greater than a certain value, it is assumed that there is a change in the movement of the subject. For example, if the part of the video where there is no change in the subject's movement is not the relatively important part, but the part of the video where there is a change in the subject's movement is the relatively important part, the time will be shortened for the relatively unimportant part. You are more likely to be able to take a closer look at relatively important parts. Video parts where there is no change in the subject's movement not only means that there is no change in the subject's movement at all, but also include cases where there is almost no change in the subject's movement or where the change in the subject's movement is less than a certain level. In addition, in video parts where there are changes in the movement of the subject, it is not just that there is a change in the movement of the subject if there is even a slight movement of the subject, but also includes cases where the change in the movement of the subject is more than a certain level.

(13) For example, the video data is recorded by a recording device, and it is preferable to evaluate the driving of the car by the user for a video portion of the video where the movement of the subject changes.

In this way, it becomes easier to evaluate the user's car driving in the video portion where the movement of the subject changes. In particular, it is possible to detect whether there is a video part with no change in the subject's movement or a video part with a change in the subject's movement. What is necessary is to use a detection means that does this. The detection means preferably detects a difference between images between frames, and if the difference is less than a certain value, there is no change in the movement of the subject, and if the difference is greater than a certain value, it is assumed that there is a change in the movement of the subject. In particular, the evaluation of car driving is done by analyzing videos, and the closer the driving is to following predetermined traffic rules, the higher the evaluation, and the closer the driving is to not following predetermined traffic rules. It is better to use an evaluation method that gives a lower evaluation. Driving evaluation is more important in areas where the subject's movement changes as the subject makes a right turn, left turn, lane change, etc., than in a place where the subject's movement does not change, for example, when the subject is stopped at a traffic light. I can think. Driving evaluation can be done in such situations.

(14) For example, the video data may be recorded by a recording device, and the video may be displayed based on the video portion in which the customer is included.

In this way, it becomes easier to view a video based on the part of the video that includes the customer. It is preferable to determine whether a video portion includes a customer by analyzing the video data and detecting a person other than the driver inside the vehicle. For example, recording devices are installed in commercial vehicles such as buses and taxis, and when video data is recorded by such recording devices, images of customers in those commercial vehicles can be displayed.

(15) For example, a video portion related to the keyword may be found from the video based on a keyword representing the situation of the video portion.

This makes it easier to find video parts related to the keyword in the video. In particular, it is preferable to detect a keyword input from an input means for inputting a keyword representing the situation of a video portion, and to find a video portion related to the detected keyword by analyzing the video using an analysis means. The situation of the video portion may be, for example, an abstract situation such as bright or dark, or a situation where a specific subject is included, such as a bullet train or a red car.

(16) For example, the video data is recorded by a recording device, and based on a keyword representing the condition of the car at the time the video data was recorded, it is preferable to find a video part related to the keyword from the video. .

In this way, there is a high possibility that a video portion related to the keyword representing the state of the car at the time of recording can be found. It is preferable to detect a keyword input into an input means for inputting a keyword representing the state of the car at the time of recording the video data, and to find a video portion related to the detected keyword by analyzing the video using an analysis means. Keywords expressing the car situation may be, for example, the car has accelerated, the car has exceeded a certain speed, the turn signal has been turned on, etc.

(17) For example, the video data may be recorded by a recording device, and a warning may be issued when the user is not looking in the direction he or she should be looking while driving a car.

This makes it easier to warn the user that they are not looking in the direction they should be looking. In particular, it is preferable to issue a warning based on information input by an input means for inputting information indicating that the user is not looking in the desired direction. Such an input means displays the user's line of sight detected by a detection means that detects the user's line of sight while driving on the reproduced video, and the displayed line of sight indicates a predetermined direction in which the user should look. It is a good idea to enter information that was determined not to have been viewed. The warning may be, for example, adding a warning mark to the displayed image, adding a specific color to part or all of the displayed image, or outputting an alarm.

(18) For example, the video data is recorded by a recording device, and the processing process is performed to distinguish between a range visible to the user and a range not visible by the user while the user is driving a car. It is preferable to use processing to display it.

In this way, it becomes easier for the user to understand the range that is visible and the range that is not visible to the user. In particular, if the processing is to display the video by distinguishing between the range visible to the user and the range not visible by the user, based on information input from an input means for inputting information indicating the range visible by the user. good. For example, you can blur areas that are not visible to the user, fill them with a semi-transparent color, or fill them completely.

(19) For example, the processing may be performed on the recorded video data such that the display range of the video is changed depending on the orientation of the device that displays the video based on the video data after the processing.

In this way, it becomes easier to view the video according to the orientation of the device displaying the video. It is preferable to change the display range of the video based on information from a detection means that detects the orientation of the device that displays the video based on the processed video data. For example, it is a good idea to record the entire 360-degree range during recording, and then display the video corresponding to the orientation of the device displaying the video.

(20) For example, the video data is recorded by a recording device, and the processing is performed using the direction of travel of the car as a reference direction according to the orientation of the device that displays the video based on the video data after processing. It is advisable to process the recorded video data so as to change the display range of the video.

In this way, it is possible to change the display range of the image with the direction of travel of the car as the reference direction, increasing the possibility that the image can be viewed depending on the orientation of the device displaying the image. It is preferable to change the display range of the video based on information from a detection means that detects the orientation of the device that displays the video based on the processed video data. In addition to viewing images in the direction in which the car is traveling, it is also possible to view images in directions other than the direction in which the car is traveling.

(21) For example, information regarding the condition of the car at the time of recording the video data may be displayed.

This makes it easier to understand the condition of the car. The information regarding the vehicle situation may be, for example, the speed, acceleration, and position of the vehicle. In particular, it is preferable that the information is displayed on the display screen of the display device by the display control means based on information input from an input means for inputting information regarding the state of the vehicle at the time of recording the video data. The information regarding the vehicle situation may be displayed over the video, or may be displayed in a separate area from the area in which the video is displayed. For example, information regarding the vehicle status may be displayed in the top, bottom, left, or top area of the video.

(22) For example, video data is recorded by a recording device, a map image representing the range of movement of the car at the time of recording the video data is displayed, and the recording device displays a specified position in the displayed map image. It is preferable to display a video based on the video data recorded by.

In particular, the range of movement of the car at the time of recording video data is detected by the detection means, image data representing a map image representing the detected range of movement is acquired by the acquisition means, and the range of movement of the car is detected by the acquisition means. The display control means displays an image of a map on the display screen of the display device, the designation means specifies the position of one of the displayed map images, and the video data recorded at the designated position is It is preferable that the image analysis means finds out the image from among the video data, and the found image is displayed on the display screen of the display device by the display control means. This makes it easier to see the video at the designated location from the map image. Even if you don't know the time of the video you want to watch, if you know its location, you can watch the video at that location.

(23) For example, the video based on the video data recorded by the user's recording device at a specified position in the displayed map image, and a third party at the specified position in the displayed map image. It is preferable to display at least one of a video based on video data recorded by a recording device.

In this way, it becomes more likely that the user will be able to view not only videos based on video data recorded by the user's recording device, but also videos based on video data recorded by a third party's recording device. In particular, the transmitting means sends a command to request video data recorded by a third party's recording device at a specified position of the displayed map image from a server etc. where such video data is recorded. The receiving means receives the video data sent from the server or the like in response to the command, and the display control means displays the video based on the video data recorded by the third party's recording device on the display screen of the display device. It is good to display it. The video based on the video data recorded by the user's recording device and the video based on the video data recorded by the third party's recording device may both be displayed at the same time, or they may be displayed by switching between them. It's okay. The video based on the video data recorded by the third party's recording device may be from the same season as the video data was recorded by the user's recording device, or may be from a different season. Furthermore, the video based on the video data recorded by the third party's recording device may be of the same weather as the video data was recorded by the user's recording device, or may be of a different weather.

(24) For example, a name including the shooting location of the recorded video data may be displayed as a name for identifying the recorded video data.

In this way, a name that includes the shooting location can be used as a name for identifying video data. It becomes easier to find video data of shooting locations. In particular, a detection means detects a shooting location from data indicating a recording position of recorded video data, and a name including the detected shooting location is changed by a name changing means to a name that identifies the recorded video data. However, it is preferable that the changed name is displayed on the display screen of the display device by the display control means as a name for identifying the recorded video data. In particular, it is preferable that the file name for storing the recorded data be changed to a name that includes the shooting location of the recorded video data by the file name changing means. For example, the file name of a video file that stores video data may include the shooting location. The name of the shooting location itself may be used as a name for identifying the video data.

(25) For example, a plurality of pieces of video data may be recorded, and names for identifying video data of the same shooting location may be displayed together.

In this way, there is a high possibility that it will be easier to tell whether the video data was taken at the same location. In particular, it is preferable to control the display device by the display control means so that names identifying video data of the same shooting location are collectively displayed on the display screen, for example, consecutively. Shooting locations with many identical shooting locations can be considered to be everyday shooting locations such as commuting to work or school, and shooting locations with few identical shooting locations can be considered to be unusual shooting locations such as travel destinations. I can think. There is a high possibility that it will be easier to select video data of everyday shooting locations, video data of unusual shooting locations, etc.

(26) For example, it is recommended to give priority to displaying names that appear less frequently.

In this way, names that appear less frequently can be displayed preferentially, making it easier to select video data corresponding to the name. In particular, it is preferable to control the display device by the display control means so that names that appear less frequently are given priority and displayed on the display screen. Whether or not the appearance frequency is low may be determined, for example, when the appearance frequency is less than or equal to a predetermined threshold value. Priority display means, for example, when multiple identifying names are displayed in a vertical list format, it is best to display them at the top, and when they are displayed in order, it is best to display them in the earliest order. .

(27) For example, it is good to display an image of a map of the shooting location.

In this way, in addition to the video based on the video data, there is a high possibility that a map of the location where the video was taken can be viewed. In particular, it is preferable to use the search means to find a map image of the photographing place based on the data indicating the location of the photographing place, and to display the found map image on the display screen of the display device by the display control means.

(28) For example, the video data is recorded by a recording device, which displays an image of a map representing the range of movement of the car at the time of recording the video data, and displays information on the operation of the car at the time of recording the video data on the map. It is best to display it in the image.

In this way, it becomes easier to understand what kind of vehicle operation was performed at the location shown in the map image. By looking at information regarding vehicle operations displayed on a map image, it is possible to determine what kind of vehicle operations are likely to be required when driving through the location represented by the map image. Recognize. A search means finds an image of a map representing the travel range from the data showing the travel range of the car, and the display control means controls the display device so as to display the found map image on the display screen. It is preferable that information regarding the operation of the map is input from the input means, and the display control means controls the display device so as to display the input information on the map image. Car operations may include, for example, operating a turn signal, suddenly steering the steering wheel, or applying the brakes suddenly.

(29) For example, video data is recorded by a recording device, and in addition to displaying a video based on the video data after the processing, a comparison between a third party's driving and the user's driving is made at the location where the video data is recorded. It is a good idea to display information based on

This makes it easier to compare the third party's driving with the user's driving. Through this comparison, the user is more likely to be able to objectively evaluate his or her own driving skills compared to a third party's driving. In particular, data indicating the recording location of the video data is input from the input means, information indicating the state of driving of a third party at the recording location indicated by the input data is acquired by the acquisition means, and the acquired third party is A comparison means compares the user's driving with the driving of a third party using information about the user's driving, and the display control means controls the display device to display information based on the comparison on the display screen. good. Information based on the comparison between the third party's driving and the user's driving includes, for example, the ranking of the third party's driving and the user's driving, the average value of the third party's driving evaluation, and the user's driving evaluation. It is best to use a value etc.

(30) For example, video data is recorded by a user's recording device, and the video data is based on the video data after processing and the video data is recorded on the recording device according to the driving of a third party at the location where the video data is recorded. It is preferable to display an image based on the image data.

In this way, it becomes easier to compare and view the video of the user's driving and the video of a third party's driving. By observing a third party's driving, the user is more likely to be able to understand whether his or her own driving is good or bad. In particular, data indicating a recording location of video data is input from an input means, and video data recorded according to the driving of a third party at the recording location indicated by the input data is acquired by an acquisition means. The display device may be controlled by the display control means so as to display on the display screen a video based on video data recorded according to the third party's driving, and a video obtained according to the user's driving. The video data recording location is, for example, a location where the user records video data obtained by driving a car in which a recording device is installed.

(31) For example, information obtained through at least one of road-to-vehicle communication and vehicle-to-vehicle communication is recorded in association with video data, and road-to-vehicle communication is It is preferable to perform control to display information obtained by at least one of the vehicle-to-vehicle communication and the vehicle-to-vehicle communication.

In this way, information obtained through at least one of road-to-vehicle communication and vehicle-to-vehicle communication becomes easier to understand. In particular, it is preferable that the display device is controlled by the display control means so that the reading means reads the information obtained through at least one of the road-to-vehicle communication and the vehicle-to-vehicle communication, and displays the read information on the display screen. The information obtained through at least one of road-to-vehicle communication and vehicle-to-vehicle communication may be, for example, information obtained based on ITS (Intelligent Transport Systems). The information obtained through road-to-vehicle communication may be, for example, information transmitted from a roadside device installed on the road, and the information obtained through vehicle-to-vehicle communication may be communicated between cars. The term "displayed in relation to the video" means that the display only needs to be related to the video, and may be displayed on top of the video, in an area separate from the video, or when the video is displayed. It should be displayed occasionally.

(32) For example, the processing may be processing for correction to a memory color.

In this way, it becomes easier to obtain images based on memory colors. In particular, it is preferable to use a correction means for correcting the image to be stored. For example, there is a high possibility that the user will be able to obtain an image that is relatively similar to what he or she remembers. The memory color may be, for example, a color that the user remembers as an image or a color that the user remembers about a specific thing. To correct the memory color, for example, the brightness, hue, saturation, etc. may be changed through user operations while viewing the video.

(33) For example, the processing may be processing to blur at least a portion of the video based on the recorded video data.

In this way, there is a high possibility that at least a partially blurred image can be viewed. In particular, it is preferable to process the image using a blurring means that blurs the image. A designated range of the video may be blurred, or an area outside the designated range of the video may be blurred. It may be possible to switch between blurring within a specified range and blurring outside the specified range.

(34) For example, when outputting audio for the video based on the processed video data, face the display screen that displays the video, and when outputting audio to the viewer of the video based on the processed video data, It may further include a communication robot directed toward the viewer.

This increases the possibility of obtaining a communication robot that faces the display screen displaying the video when outputting audio for a video, and faces the viewer when outputting audio to a viewer of the video. In particular, the communication robot has an audio output means, a head rotation means that can move its head between the display screen and the viewer, and an analysis of the image displayed on the display screen. Judgment means for determining whether to output audio or to the viewer; It is preferable to include a head rotation control means for controlling the head rotation means so that the head can face the head. When outputting audio for a video, the viewer's attention is directed toward the display screen that displays the video, and when outputting audio to the viewer, the viewer's attention is directed toward the viewer, so the audio output from the communication robot are more likely to listen carefully. The communication robot is preferably one that can face the display screen and the viewer, and can output audio, for example.

(35) For example, control may be performed to change the manner in which the image is displayed based on the image data after processing, based on information regarding the road.

In this way, it becomes easy to change the display mode of the image based on the information regarding the road. In particular, a mode in which a video represented by video data is analyzed, information about roads included in the video is detected, and based on the detected information about roads, the video is displayed based on the processed video data. It is better to change the . The information regarding the road may be, for example, the width of the road, road signs, etc. For example, the appearance may be changed by displaying images from different camera angles, changing the brightness, hue, saturation, etc. of the image based on information about the road.

(36) For example, it may be a recording device that records the video data to be played back by the playback device.

In this way, it is more likely that the video data to be played back by the playback device can be recorded. For example, it may be a drive recorder that records video data to be played back by the playback device. The recording device may be installed in a four-wheeled vehicle such as a car, a two-wheeled vehicle such as a motorcycle, or a bicycle.

(37) For example, the recording device may perform control to record the video data with high image quality based on specified conditions.

This makes it easier to record video data with high image quality based on specified conditions. In particular, it is preferable that conditions are specified by the specifying means, the specified conditions are input by the input means, and the recording device is controlled by the recording control means so as to record video data with high image quality based on the input conditions. The specified conditions may be, for example, location, time, etc. High image quality may be achieved by, for example, increasing the frame rate above a certain threshold or setting a resolution higher than a certain resolution.

(38) For example, at least one of the playback device and the recording device is provided with a specification means for specifying conditions, and controls recording of the video data with high image quality based on the conditions specified by the specification means. Good.

In this way, it is more likely that video data can be recorded with high image quality based on the conditions specified by the specifying means. In particular, it is preferable that conditions are specified by the specifying means, the specified conditions are input by the input means, and the recording device is controlled by the recording control means so as to record video data with high image quality based on the input conditions. When the playback device is equipped with a specifying means, for example, data representing the conditions specified by the specifying means is recorded on a recording medium for recording video data, and such a recording medium is loaded into the recording device and the recording is started. It is preferable to record video data with high image quality in a recording device based on the specified conditions by reading data representing the specified conditions from the medium.

(39) For example, the condition may be either that the speed of the car in which the recording device is installed exceeds a certain level, or that the car in which the recording device is installed drives in a designated place. Good to have.

In this way, it becomes easier to record video data with high image quality when the speed of the vehicle exceeds a certain level or when the vehicle travels in a designated area. In particular, the speedometer of the car is photographed using a camera installed in the recording device, the photographed image is analyzed by the analysis means, and the speedometer is determined by the determination means when the speed of the car exceeds a certain level. It is preferable that high-quality recording is performed based on the determination that the Alternatively, the position where the car is traveling may be detected by the position detecting means, and high-quality recording may be performed when the determining means determines that the vehicle is traveling in a designated location.

(40) For example, when the video data is recorded by the recording device, information obtained through at least one of road-to-vehicle communication and vehicle-to-vehicle communication may be recorded in association with the video data.

This increases the possibility that information obtained through at least one of road-to-vehicle communication and vehicle-to-vehicle communication can be recorded in association with video data. In particular, it is preferable that the receiving means receives information transmitted through at least one of road-to-vehicle communication and vehicle-to-vehicle communication, and records the received information in association with video data obtained by photography. In particular, information transmitted through at least one of road-to-vehicle communication and vehicle-to-vehicle communication may be recorded in the header of a video file that stores video data obtained by shooting.

(41) For example, control may be performed to transmit data representing a first calculated value uniquely determined from the video data recorded by the recording device to the block chain system.

In this way, there is a high possibility that data representing the first calculated value uniquely determined from the video data can be sent to the block chain system. Data representing the first calculated value can now be recorded on the block chain. In particular, the transmitting means is configured to calculate a first calculated value that is uniquely determined from the video data recorded by the recording device, and to transmit data representing the calculated first calculated value to the block chain system. Good to control. The calculation means for calculating the first calculated value may be provided in the recording device or the reproducing device, and the transmitting means may also be provided in the recording device or in the reproducing device. may be provided. Block chains have a relatively low possibility of being tampered with, so by recording the data representing the first calculated value on the block chain, there is a possibility that the data representing the first calculated value has not been tampered with. It is found that the first calculated value is high, and there is a high possibility that it can be proven that the video data has not been tampered with using the data representing the first calculated value and the video data.

(42) For example, the video data recorded by the recording device is video data based on MPEG, and the first calculated value that is uniquely determined for each unit is set immediately before the P frame as the unit for dividing the video data. It is a good idea to control the data represented in the block chain to be recorded.

In this way, there is a high possibility that the block chain can record data that is divided immediately before the P frame and represents the first calculated value for each unit. A P frame cannot be reproduced as a single image without an I frame, so if you divide it just before the P frame and calculate the first calculated value for each unit, it will be more difficult to calculate the first calculated value. becomes more likely to occur. In particular, the first calculated value that is uniquely determined for each unit is calculated immediately before the P frame as a unit for dividing the video data, and the data representing the calculated first calculated value is used as a block chain system. It is preferable to control the transmitting means so as to transmit the information.

(43) For example, control may be performed to cause the block chain to record data representing the second calculated value uniquely determined from the event data.

In this way, there is a high possibility that data representing the second calculated value uniquely determined by the event data can be recorded on the block chain. This can make it difficult to falsify the second calculated value. In particular, when the calculation means calculates a second calculation value that is uniquely determined from the event data, and the transmission means is controlled to send data representing the calculated second calculation value to the block chain system. good. By comparing the second calculated value and the event data, it can be determined whether the event data has been tampered with, and since it is difficult to tamper with the second calculated value, it is also difficult to tamper with the event data.

(44) For example, data indicating whether the video data recorded in the recording device is for manual operation or automatic operation may be recorded in association with the video data.

This increases the possibility of knowing whether the video data is from manual driving or automatic driving. In particular, identification data indicating whether the video data is recorded on the recording device during manual operation or the video data recorded on the recording device during automatic operation is inputted from the input means, and the input identification data is associated with the video data. It is a good idea to record it. Sometimes it is advisable to record the identification data in the header area of the video file in which the video data is recorded. Manual driving, for example, is when a person operates the car's steering wheel, while automated driving is when a person operates the car's steering wheel using mechanical control. good.

(45) For example, the image quality of the video data recorded by the recording device may be changed between manual operation and automatic operation.

In this way, there is a high possibility that image data of different quality will be obtained between the video data during manual driving and the video data during automatic driving. In particular, when identification data indicating whether manual driving or automatic driving is in progress is inputted from the input means, the image quality of video data recorded when the identification data indicates manual driving and when automatic driving is in progress is determined. It is preferable that the image quality of the video data to be recorded is changed by the changing means. Video data may be recorded with high image quality during manual driving and low image quality during automatic driving, or vice versa. It is also possible to record video data by increasing the height.

(46) For example, it would be good to use a communication robot during autonomous driving.

In this way, there is a high possibility that the communication robot will be able to operate during autonomous driving. During automatic driving, for example, the user does not need to drive the vehicle, so he or she can communicate with the communication robot without any hindrance to driving. In particular, the input means of the recording device inputs identification data indicating that automatic operation is being performed, and the transmitting means of the recording device sends an operation command to the communication robot in response to input of such identification data into the input means. It is best to send it from

(47) For example, control may be performed to display a video based on recorded video data and change the display mode of the video in response to a determination that the possibility of an event occurring is high.

In this way, it becomes easier to change the display mode of the video in response to the determination that the possibility of an event occurring is high. In particular, the recording device installed in the vehicle is equipped with a display device, the video is displayed on the display screen of the display device, the video is analyzed by an analysis means while the vehicle is running, and as a result of the analysis, an event is detected in front of the vehicle. It is preferable that the display device is controlled by the display control means so as to change the mode of the image displayed on the display screen in response to the determination by the determination means that there is a high possibility of occurrence. For example, by recognizing that the display mode of the video has been changed, it is possible to know that an event is likely to occur, so that preparations can be made for that event. For example, if video data is captured by a drive recorder installed in a moving car, the video format may change, such as when an event occurs a few seconds later, such as when a pedestrian suddenly appears. , It can be seen that there is a possibility of a car flying out.

(48) For example, data representing the communication status of communication radio waves received by the communication device may be recorded.

This makes it easier to record data representing the communication status of communication radio waves. In particular, it is preferable to receive communication radio waves using a communication device that is capable of receiving communication radio waves, and record data representing the communication state of the communication radio waves. In particular, the data representing the communication status may be such as whether communication is in progress or communication is interrupted. For example, the communication radio waves may be radio waves based on LTE (Long Term Evolution), Wi-Fi, or Bluetooth (registered trademark). When displaying images, the display mode may be changed depending on the communication status.

(49) For example, the recording mode of the video data may be changed based on communication radio waves received by the communication device.

This makes it easier to change the recording mode of video data based on communication radio waves. In particular, it is preferable that the communication radio waves are received by a communication device that is capable of receiving communication radio waves, and the recording mode of the video data is changed based on the received communication radio waves. In particular, any recording mode may be used as long as the recording method changes. For example, the recording mode may be set to high image quality when communication radio waves are being received, and the recording mode may be set to low image quality when communication radio waves are not being received. It is also possible to do the opposite.

(50) For example, video data representing an image of a driver who has received a roll call and the video data obtained from the driver's driving may be recorded in association with each other.

In this way, there is a high possibility that the video data representing the video of the driver who received the roll call and the video data obtained from the driver's driving can be recorded in association with each other. In particular, video data representing an image of a driver who has received a roll call is input by an input means, and the input video data and video data obtained by photographing the driver's driving with a camera installed in a recording device. It is a good idea to associate and record this. In particular, it is recommended to store these video data in the same folder. For example, in conjunction with an image of a driver receiving a roll call, it becomes possible to display an image obtained from the driver's driving. For example, it will be possible to simultaneously display a video of a driver receiving a roll call and a video of that driver driving.

(51) For example, by associating the video data obtained by a recording device installed in a truck driven by a driver with the video data obtained by a recording device installed in a forklift operated by the driver. It is a good idea to control recording.

In this way, it becomes easier to associate and record the video data when the truck is being driven and the video data when the forklift is being driven. In particular, video data obtained by filming the truck driver's driving is sent to a server using a camera installed on a recording device installed on the truck, and video data obtained from the recording device installed on the forklift is sent to a server. It is preferable to send video data obtained by photographing the forklift driver's operation using a camera to the server, correlate the video data with each other, and record the video data on the server. In particular, a video folder is created for each driver on the server, and it is preferable to store video data when the driver drives a truck and video data when the driver drives a forklift in the same video file. Also, when driving a truck, the video data of the truck driving is recorded on a recording medium loaded in the recording device attached to the truck, and when driving a forklift, the recording medium is loaded into the recording device attached to the truck. Remove it from the recording device attached to the forklift, load it into the recording device attached to the forklift, and store the video data from driving the forklift in the same video file that stores the video data from driving the truck. It's okay. For example, it will be possible to compare video of a driver driving a truck with video of that driver driving a forklift.

(52) For example, control may be performed to display information that should be noted when driving, superimposed on the actual scenery seen from a moving car.

This increases the possibility of displaying information that should be noted when driving, superimposed on real scenery. When actually driving, you will be able to see information that you should be careful of while looking at the actual scenery. In particular, while viewing a real scene using an MR (Mixed Reality) device, it is preferable to display information that should be noted when driving on the MR device. In particular, for example, if a user is made to wear an MR device on glasses, a real scene is shown through the glasses, and information to be noted when driving is displayed on the surface of the glasses in a translucent form superimposed on the real scene. good.

(53) For example, based on the fact that the vehicle has traveled in the same place as the previous one, control may be performed to notify that fact.

In this way, it will be easier to tell whether the vehicle has been driven before. Since you can see where you have traveled before, you can consider new developments, such as whether to take the same route or a different route. In particular, the recording medium loaded into the recording device records data indicating the previous driving location, and it is possible to compare the previous driving location represented by the recorded data with the current driving location. If they match, the vehicle is notified that the vehicle is traveling in the same location as the previous location, through audio output from the audio output device and display on the display screen of the display device. A GPS (global positioning system) receiver provided in the recording device may be used to detect the location where the vehicle is currently traveling.

(54) For example, control may be performed to photograph the speedometer of a car and record the video data based on the photographed speedometer.

This increases the possibility of recording video data based on the photographed speedometer. In particular, in a car, a camera is installed in a position where the speedometer can be photographed by a recording device, the camera photographs the car's speedometer, the image obtained by photographing is analyzed by an analysis means, and the speedometer records the video data. It is preferable to start recording video data when the speed of recording conditions is indicated. For example, it is a good idea to record video data when a car's speedometer exceeds 80 km/h.

(55) For example, the recording device may be attached to an automatic guided vehicle, and control may be performed to record the video data based on detection of the approach of objects surrounding the automated guided vehicle.

In this way, it is more likely that video data can be recorded based on the detection of the approach of objects surrounding the automated guided vehicle. In particular, it is preferable to take a picture of the vicinity of the automated guided vehicle using a camera provided in the recording device, and when the detection means detects the approach of objects surrounding the automated guided vehicle, to record the video data obtained by taking the picture with the camera. For example, it will be possible to see when an automated guided vehicle comes into contact with surrounding objects.

(56) For example, the recording device may be attached to a forklift and perform control to record the video data based on at least one of the operation of the forklift and the presence of the driver on the forklift.

This makes it easier to record video data based on at least one of the operation of the forklift and the fact that the driver rides the forklift. In particular, it is preferable to start recording the video data in response to the detection means detecting at least one of the operation of the forklift and the fact that the driver got on the forklift. In particular, by photographing the inside of a forklift using a camera installed in a recording device and analyzing the video data obtained from the photography, it is possible to detect whether the forklift has moved or whether the driver has climbed onto the forklift. It is recommended to start recording video data accordingly. For example, the movement of the forklift may be defined as the movement of the forks. For example, recording of video data may be stopped when the driver gets off the forklift.

(57) Trigger information for event recording may be recorded in the header of the file storing the video data.

This increases the possibility that event recording trigger information can be recorded in the header of the file. For example, the possibility of developing various processes using the trigger information increases.

(58) For example, a playback method for playing back recorded video data, which processes the recorded video data based on acquired information and displays a video based on the processed video data. It is preferable that the feature is that control is performed.

This increases the possibility that the user will be able to view a video that has been processed based on the acquired information, rather than the recorded video itself. By performing processing suitable for the user viewing the video, it is possible to obtain a video suitable for the user. Rather than processing recorded video, recorded video data is processed based on the acquired information, increasing the possibility of displaying video that is more suitable for the user viewing the video.

(59) It is preferable to control a recording device to record video data to be reproduced in the reproduction method.

This increases the possibility that the video data to be reproduced can be recorded in the recording device.

(60) For example, a computer-readable program that plays back recorded video data, which processes the recorded video data based on acquired information, and produces a video based on the processed video data. It is preferable to use a computer-readable program that controls the display of the information.

In this way, the user can view a video that has been processed based on the acquired information, rather than the recorded video itself. By performing processing suitable for the user viewing the video, it is possible to obtain a video suitable for the user. Rather than processing recorded video, recorded video data is processed based on the acquired information, increasing the possibility of displaying video that is more suitable for the user viewing the video.

The inventions shown in (1) to (60) above can be combined arbitrarily. For example, at least a part of the structure of at least one invention described in (2) and subsequent ones may be added to all or part of the structure of the invention shown in (1). In particular, it is preferable to create an invention in which at least a part of the structure of at least one invention after (2) is added to the invention shown in (1). Further, arbitrary configurations may be extracted from the inventions shown in (1) to (60) and the extracted configurations may be combined. The applicant of this application intends to acquire rights to inventions containing these structures.

In this way, the user can view a video that has been processed based on the acquired information, rather than the recorded video itself. By performing processing suitable for the user viewing the video, it is possible to obtain a video suitable for the user. Rather than processing recorded video, recorded video data is processed based on the acquired information, increasing the possibility of displaying video that is more suitable for the user viewing the video.
The effects of the invention of the present application are not limited to these, but the effects obtained from the parts of the structure disclosed in the specification, drawings, etc. are also disclosed, and the rights to the structure that produces the effects have also been acquired through divisional applications, amendments, etc. have the intention to For example, in this specification, a portion where it is written as "can be done" is a statement that clearly indicates the effect to be achieved, and there are parts that show an effect even if there is no mention of "can be done". Further, even without such a description, there are effects that can be understood from the configuration.

It is a side view of a drive recorder. FIG. 2 is a block diagram showing the electrical configuration of a drive recorder. It shows the file structure of the video file. 3 is a flowchart showing a recording processing procedure. 3 is a flowchart illustrating a header data generation processing procedure. 3 is a flowchart showing a procedure for recording video data. 3 is a flowchart showing a data output processing procedure. 3 is a flowchart showing a data output processing procedure. This is an example of an image displayed on a display screen. FIG. 1 is a block diagram showing the electrical configuration of a personal computer. 3 is a flowchart showing a reproduction processing procedure. 3 is a flowchart showing a reproduction processing procedure. 3 is a flowchart showing a reproduction processing procedure. This is an example of a playback window. This is an example of a playback window. This is an example of a playback window. This is an example of a playback window. This is an example of a playback window. This is an example of a playback window. This is an example of a playback window. This is an example of a playback window. This is an example of a playback window. This is an example of a playback window. This is an example of a play list display area. This is an example of a play list display area. An I frame and a P frame are shown. 3 is a flowchart showing a recording processing procedure. It is a flowchart which shows the processing procedure of the drive recorder attached to the truck. It is a flowchart which shows the processing procedure of the drive recorder attached to the forklift. 3 is a flowchart showing a recording processing procedure. It is a flowchart which shows the processing procedure of an automatic guided vehicle. It is a flowchart which shows the processing procedure of driving evaluation. 3 is a flowchart showing a reproduction processing procedure. 3 is a flowchart showing a reproduction processing procedure. 3 is a flowchart showing a reproduction processing procedure. This is an example of a map image. 3 is a flowchart showing a reproduction processing procedure. This is an example of a playback window. This is an example of a playback window. This is an example of a personal computer and a communication robot. FIG. 2 is a block diagram showing the electrical configuration of a communication robot. 3 is a flowchart showing a processing procedure of a communication robot. It shows a communication robot placed inside a car.

(1) Recording Processing FIGS. 1 to 9 show recording processing using a drive recorder.

FIG. 1 shows an embodiment of the present invention, and is a side view of a drive recorder 1 (for example, an example of a recording device).

The drive recorder 1 includes a mounting section 2 and a camera section 4.

The attachment part 2 is fixed to the windshield (not shown) of the vehicle to which the drive recorder 1 is attached, and allows a memory card to be attached or removed. Furthermore, a display screen 3 is formed on a part of the outer circumferential surface of the mounting portion 2 to display the image being photographed. Furthermore, the mounting portion 2 includes a DC (direct current) jack (not shown) that is connected to the vehicle via a converter unit (not shown). Furthermore, the attachment part 2 is provided with predetermined buttons (not shown).

The camera unit 4 includes a front camera (not shown in FIG. 1) that mainly takes pictures of the front (the traveling direction of the vehicle to which the drive recorder 1 is attached) and a rear camera (not shown in FIG. 1) that takes pictures mainly of the rear. ) is built-in. The camera section 4 includes a front lens 5 for forming an image of a subject on the image forming plane of the front camera, and a rear lens 6 for forming an image of the subject on the image forming plane of the rear camera. The front camera can photograph the front of the vehicle, the left and right directions in front of the vehicle, and the vertical direction in front of the vehicle, and the rear camera can photograph the rear direction of the vehicle, the left and right directions behind the vehicle, and the vertical direction behind the vehicle. can. Therefore, the front camera and the rear camera can take pictures in all directions around the drive recorder 1.

The camera section 4 can move back and forth within a predetermined angle θ with respect to the mounting section 2, making it possible to take pictures at a relatively suitable angle regardless of the angle of the windshield. becomes.

FIG. 2 is a block diagram showing the electrical configuration of the drive recorder 1.

The entire operation of the drive recorder 1 is supervised by the controller 10.

The drive recorder 1 includes a memory card reader/writer 95. When the memory card 25 is installed in the drive recorder 1, the memory card reader/writer 95 reads data recorded on the memory card 25, writes data to the memory card 25, etc. An operating program, which will be described later, is recorded in the memory card 25, and the controller 10 reads the recorded operating program and installs it into the drive recorder 1. The controller 10 reads the operating program installed in the drive recorder 1 and controls each part of the drive recorder 1 according to the operating program. If the operating program for the drive recorder 1 is stored in a recording medium other than the memory card 25, it may be read from such a recording medium. In this case, the drive recorder will be provided with an interface such as a driver compatible with the recording medium. Further, it may be received by the communication circuit 17 and installed in the drive recorder 1.

The drive recorder 1 includes a front camera 11 and a rear camera 12, as described above. Video data captured by the front camera 11 and rear camera 12 is input to the controller 10. The drive recorder 1 also includes a display device having a display screen 3, and a display control device 13 controls the display on the display device 14. Furthermore, the drive recorder 1 includes a clock 15, a road-to-vehicle/vehicle-to-vehicle information receiving circuit 16, a communication circuit 17, and an acceleration sensor 18. Time data representing the time output from the clock 15 is output to the controller 10. The road-to-vehicle/vehicle-to-vehicle information receiving circuit 16 receives, for example, data representing surrounding conditions (an example of road-to-vehicle information) transmitted from a roadside device placed on a road based on an intelligent highway information system, and data transmitted from a vehicle. The controller 10 receives data representing own vehicle information (which is an example of vehicle-to-vehicle information), and outputs the received data to the controller 10. The road-to-vehicle/vehicle-to-vehicle information receiving circuit 16 uses, for example, a communication circuit based on DSRC (Dedicated Short Range Communications). The communication circuit 17 communicates with a communication robot when the communication robot is placed in the vehicle, for example. Output data from the communication circuit 17 is input to the controller 10. The acceleration sensor 18 detects the acceleration of the vehicle to which the drive recorder 1 is attached, and outputs the detected acceleration data to the controller 10.

Furthermore, the drive recorder 1 includes an input device 20 such as buttons, a memory 21 such as a RAM (Random Access Memory), and a GPS (Global Positioning System) receiver 22. The output signal from the input device 20, the output data from the memory 21, and the output data from the GPS receiver 22 are also input to the controller 10.

FIG. 3 shows the file structure of a video file that stores video data output from the front camera 11 and video data output from the rear camera 11.

The video file includes a header recording area 31 and a video recording area 32. The header recording area 31 is an area for recording header data, and the video recording area 32 is an area for recording video data.

FIG. 4 is a flowchart showing the recording processing procedure of the drive recorder 1.

The recording processing procedure shown in FIG. 4 is repeated, for example, every time one frame (or another frame) is captured until the recording processing is completed.

When the engine of the vehicle to which the drive recorder 1 is attached starts, the battery of the vehicle supplies power to the drive recorder. Then, the recording process shown in FIG. 4 starts. Additionally, a video file is generated on the memory card 25. When the recording process starts, video data output from the front camera 11 and rear camera 12 is input to the display control device 13 via the controller 10. The captured video is displayed on the display screen 3 of the display device 14 (see FIG. 9). The image displayed on the display screen 3 is either the image obtained by the front camera 11 or the image obtained by the rear camera 12 according to the program recorded on the memory card 25. However, both the images obtained by the front camera 11 and the images obtained by the rear camera 12 may be displayed. In addition, the image obtained by the front camera 11 or the image obtained by the rear camera 12 is displayed as a default display, and a switching button is provided on the input device 20, and the front camera 11 or the rear camera may be displayed.

First, header data is generated and the generated header data is temporarily recorded in the memory 21 (step 41), and then video data is temporarily recorded in the memory 21 (step 41). Step 42). Subsequently, various types of data are outputted (step 43), and data temporarily recorded in the memory 21 is written to the memory card 25 (step 44).

In FIG. 4, after the header data is generated and the generated header data is temporarily recorded in the memory 21 (step 41), the video data is temporarily recorded in the memory 21 (step 41). 42), after which output processing of various data, etc. is performed (step 43), but the order of these processings does not matter. Additionally, the header data and video data are temporarily recorded in the memory 21, the header data and video data recorded in the memory 21 are read, and the read header data and video data are written to the memory card 25. However, if possible, the data may be written directly to the memory card 25 without temporarily storing it in the memory 21. In addition, two or more desired processes among the header data generation and recording processing (step 41), video data recording processing (step 42), and data output processing (step 43) can be performed in parallel. You may also do so.

When the engine of the vehicle to which the drive recorder 1 is attached is stopped and the power supply to the drive recorder 1 is stopped, the recording processing procedure shown in FIG. 4 ends.

FIG. 5 is a flowchart showing the header data generation processing procedure (the processing procedure of step 41 in FIG. 4), and shows the processing procedure of the controller 10.

If the vehicle to which the drive recorder 1 is installed is capable of autonomous driving, the vehicle will record information from the vehicle to the drive recorder 1 during autonomous driving (during driving that is not based on the operation of the vehicle driver). Data to that effect is sent. For example, the drive recorder 1 is connected to the vehicle's in-vehicle network, and data output to the in-vehicle network indicating whether automatic driving or manual driving (during operation based on the operation of the vehicle driver) is detected. Accordingly, the controller 10 of the drive recorder 1 recognizes whether the vehicle is being driven automatically or manually. In addition, if the vehicle and the drive recorder 1 are capable of wireless communication and the vehicle transmits data indicating whether the vehicle is driving automatically or manually, the data is transmitted to the communication circuit 17 of the drive recorder 1. By receiving this information, the controller 10 of the drive recorder 1 may recognize whether the vehicle is being driven automatically or manually.

The controller 10 of the drive recorder 1 determines whether the vehicle is operating automatically or manually (step 51). During automatic driving, data indicating that the data recorded on the memory card 25 is video data taken during automatic driving is temporarily recorded in the memory 21 as header data (step 52). ). Additionally, during manual operation, data indicating that the data recorded on the memory card 25 is video data taken during manual operation is temporarily recorded in the memory 21 as header data ( Step 53). For example, data indicating whether video data photographed within a certain period of time is data during automatic operation or data during manual operation is temporarily recorded on the memory card 25. Of course, an automatic driving video folder is created to store video data during automatic driving, and a manual driving video folder is created to store video data during manual driving, and video data taken during automatic driving is stored in the automatic driving video folder. However, video data taken during manual operation may be stored in the manual operation video folder. Data indicating whether the time or video data stored in the automatic driving video folder or video data stored in the manual driving video folder is automatic driving video data or manual driving video data, and video data during automatic driving. Or it can be associated with video data during manual operation.

Next, the controller 10 confirms whether the road-to-vehicle/vehicle-to-vehicle information receiving circuit 16 has received the road-to-vehicle communication information or the vehicle-to-vehicle communication information (step 54). When road-to-vehicle communication information or vehicle-to-vehicle communication information is received (YES in step 54), data representing the received communication information is temporarily recorded in the memory 21 along with data representing the time when the communication information was received (step 55). ). Since the time when the communication information was received is known, it is also possible to know what kind of video was being shot at that time, and when playing back the shot video, road-to-vehicle communication information or vehicle-to-vehicle communication information can also be output at the same time. In this way, for example, road-to-vehicle communication information or field-of-vehicle communication information and video data can be associated based on time. If neither the road-to-vehicle communication information nor the vehicle-to-vehicle communication information is received (NO in step 54), the process in step 55 is skipped, and the received information is not temporarily recorded in the memory 21.

It is also checked whether communication radio waves have been received by the communication circuit 17 (step 56). When the communication radio wave is received by the communication circuit 17 (YES in step 56), data representing the communication state of the communication radio wave is temporarily recorded in the memory 21 (step 57). For example, when the communication circuit 17 receives radar waves emitted from speed control devices installed on the road such as Orbis, or radio waves used by emergency vehicles, etc., the data indicating the contents are stored in the memory 21. is temporarily recorded. If the communication radio wave is not received by the communication circuit 17 (NO in step 56), the process of step 57 is skipped, and data representing the communication state of the communication radio wave is not recorded in the memory 21.

Furthermore, the line of sight of the vehicle driver is detected by the controller 10 from the video data obtained by the rear camera 12 (step 58). The line-of-sight data representing the detected line-of-sight is also temporarily written to the memory 21 along with data representing time (step 59). You can see when the driver was looking and in what direction. The eye is detected in the image, and the line of sight can be determined based on the positional relationship between the pupil, iris, and cornea of the eye (the iris of the eye) and the conjunctiva (the white of the eye). Similarly, the video data obtained by the rear camera 12 is analyzed, the direction in which the driver of the vehicle is facing is also detected, and the data representing the detected direction is also temporarily stored in the memory 21 along with the data representing the shooting time. will be written to.

FIG. 6 is a flowchart showing the video data recording processing procedure (the processing procedure of step 42 in FIG. 4).

Also in the video data recording process, the controller 10 confirms whether automatic operation or manual operation is in progress (step 61). During automatic operation, relatively low-quality video data is recorded in the memory 21 (step 62). For example, pixel thinning is performed such that the number of pixels per frame represented by the video data output from the front camera 11 and the rear camera 12 is less than the first threshold value (reduction in resolution), or The front camera 11 and the rear camera 12 are controlled by the controller 10 so that the number of read frames per frame is less than the first number of frames (reduction in frame rate). As a result, video data with relatively low image quality is obtained. During manual operation, video data of relatively medium quality is recorded in the memory 21 (step 63). For example, pixel thinning is performed such that the number of pixels per frame represented by the video data output from the front camera 11 and rear camera 12 is greater than the first threshold, or the number of read frames per unit time is The front camera 11 and the rear camera 12 are controlled by the controller 10 so that the number of frames is greater than the first number of frames. This allows video data of relatively medium quality to be obtained. In this way, the image quality of video data can be adjusted by adjusting the resolution and frame rate. During automatic driving, relatively low-quality video data is recorded in the memory 21, and during manual driving, relatively medium-quality video data is recorded in the memory 21, but during automatic driving, video data with relatively low image quality is recorded in the memory 21. Video data with a relatively medium quality may be recorded in the memory 21, and video data with a relatively low quality may be recorded in the memory 21 during manual operation. It is sufficient that the image quality of video data recorded during automatic driving is different from the image quality of video data recorded during manual driving.

Next, the controller 10 confirms whether the specified condition has been met (step 64). For example, when the speed of the vehicle exceeds a certain level, when the vehicle is traveling in a designated place, or when the input device 20 of the drive recorder 1 is provided with a high-quality designation switch (which is an example of a designation means), is an example where the specified condition is that the high image quality designation switch is operated. If the drive recorder 1 is installed on the vehicle so that the speedometer of the vehicle can be photographed by the rear camera 12, whether the speed of the vehicle has exceeded a certain level can be determined by checking the image obtained by the rear camera 12. It can be detected by analyzing the distance traveled, and it can also be detected from the travel distance calculated based on the GPS receiver 22 and the time taken for the travel distance calculated based on the clock 15. Whether the vehicle is traveling in a designated location can be determined based on output data from the GPS receiver 22.

When the specified condition is met (YES in step 64), relatively high-quality video data is temporarily recorded in the memory 21 (step 65). For example, the front camera may be configured to stop pixel thinning of the video data output from the front camera 11 and rear camera 12, or to increase the number of read frames per unit time than the number of frames when the resolution is medium. Camera 11 and rear camera 12 are controlled by controller 10. This allows relatively high-quality video data to be obtained. If the specified condition is not met (NO in step 64), the process of step 65 is skipped, and the process of recording high-quality video data in the memory 21 is not performed.

Furthermore, it is also checked whether there has been a command to change the recording mode to the controller 10 (step 66). If there is a command to change the recording mode (YES in step 66), the recording mode is changed (step 67), and if there is no command to change the recording mode (NO in step 66), the recording mode is changed. will not be carried out. For example, as mentioned above, if the communication circuit 17 receives radar waves emitted from a speed control device installed on the road such as Orbis, or communication radio waves such as radio waves used by emergency vehicles, etc., the data will be recorded. It may be determined that a mode change command has been issued. In response to the instruction to change the recording mode, the recording mode can be changed so that, for example, high-quality video data is recorded in the memory 21.

In the video data recording process shown in Figure 6, after checking whether automatic operation or manual operation is in progress (step 61), it is checked whether the specified conditions have been met (step 64). , After checking whether the specified conditions have been met (step 64), checking whether automatic operation or manual operation is in progress (step 61) may be performed, or both of these checks (step 61, 64) may be performed in parallel.

FIG. 7 is a flowchart showing the data output processing procedure (the processing procedure of step 43 in FIG. 4).

Also in the data output processing procedure, the controller 10 confirms whether automatic operation or manual operation is in progress (step 71). When the vehicle is in automatic operation, an operation start command is sent from the communication circuit 17 to a communication robot (not shown) placed in the vehicle (step 72). During manual operation, no operation start command is sent from the communication circuit 17 to the communication robot placed in the vehicle. If the robot is in manual operation, a communication robot stop command may be sent. During automated driving, the driver is not driving manually, so even if the communication robot operates, there is no problem with driving. On the other hand, the driver can communicate with the communication robot, making the most of his time. Can be used. When the vehicle is being driven manually, the communication robot placed inside the vehicle does not operate, thereby preventing problems such as the driver being unable to concentrate on driving due to being spoken to by the communication robot. Furthermore, by sending an operation stop command to the communication robot during manual operation, it is possible to stop the operation of the communication robot if it is currently in operation.

Next, the controller 10 checks whether the event is likely to occur (step 73). For example, if the road you are driving on is narrow or there are a lot of people passing by, there is a high possibility that the driver will slam on the brakes and an event will occur. If there is a high possibility that the event will occur (YES in step 73), the display control device 13 is controlled by the controller 10 to change the display mode of the video displayed on the display screen 3 of the drive recorder 1 ( Step 74). For example, by displaying the words "Caution!" on the display screen, or by superimposing the entire image displayed on the display screen 3 or a part of the surrounding area of the image in translucent red, yellow, etc. , it is a good idea to change the display mode of the video. If the possibility of the event occurring is not high (NO in step 73), the display mode of the video displayed on the display screen 3 of the drive recorder 1 cannot be changed.

Further, when the vehicle is running (or even when the vehicle is stopped), the controller 10 also checks whether the location where the vehicle is traveling is the same as the location where it has previously traveled (step 75). In the memory card 25 installed in the drive recorder 1, GPS data indicating previously traveled locations is recorded in the header recording area of the video file. The controller 10 compares the GPS data recorded in the header recording area with the GPS data received by the GPS receiver 22 provided in the drive recorder 1 attached to the running vehicle. , you can check whether the location you are driving is the same as the location you have previously traveled. If the driver is traveling in the same location as before (YES in step 75), the drive recorder 1 notifies the driver that the vehicle is in the same location (step 76). If the vehicle is not traveling in the same location as before (NO in step 75), the drive recorder 1 will not notify that the vehicle is in the same location.

FIG. 9 is an example of the display screen 3 of the drive recorder 1.

On the display screen 3 of the drive recorder 1, an image captured by the front camera 11 is displayed. At the bottom of the display screen 3, a character string 7 is displayed that reads ``You are driving in the same place where you have traveled before.'' By looking at this character string 7, it can be seen that the driver is traveling in the same location as the driver was previously traveling. Drivers will be able to tell whether they are driving in a place they have the impression of having driven through a sense of déjà vu, or in a place they have actually driven before. Whether or not the location is the same as the previous location does not have to be an exact match, as long as it falls within a certain range.

The character string 7 indicates that the location is the same as the previous location, but instead of displaying the character string 7, a predetermined mark may be displayed or the display format of the display screen 3 may be changed. You may change it. When the driver sees such a mark or confirms a change in the display format, the driver knows that the vehicle is traveling in the same location as the vehicle previously traveled. In addition, if the drive recorder 1 is equipped with a speaker, the driver is notified of the location where he or she has previously traveled by outputting voice or sound from the speaker without displaying the character string 7, etc. You may. In any case, it is sufficient for the driver to know that he/she is driving in the same location as the previous location.

Returning to FIG. 8, when one frame of video data is temporarily recorded in the memory 21, the hash value of the recorded video data (an example of a first calculated value uniquely determined from the video data) ) is calculated by the controller 10 (step 77). Data representing the calculated hash value is transmitted to the block chain system by the communication circuit 17 (step 78). The block chain system records data representing the calculated hash value on the block chain (block transactions). Since block chains are difficult to tamper with, it will be possible to use hash values to confirm whether video data has been tampered with. The data representing the calculated hash value is also recorded on the memory card 25, and when checking whether the video data has been tampered with, the data representing the hash value sent to the block chain system is used. and the data representing the hash value recorded on the memory card 25. If the hash values match, the video data has not been tampered with, and if the hash values do not match, the video data has been tampered with. It is determined that the It goes without saying that the video data recorded on the memory card 25 and the data representing its hash value are recorded on the memory card 25 in association with each other. For example, data representing a hash value is recorded in the header recording area 31 of the video file together with data indicating from which time of the video data the hash value was calculated.

In this way, when the header data and video data are temporarily recorded in the memory 21, the recorded header data and video data are read out from the memory 21, and the header data is stored in the header recording area of the video file. is written, and the video data is written to the video recording area of the video file.

(2) Reproduction processing Figures 10 to 23 show reproduction processing using the reproduction viewer software.

FIG. 10 is a block diagram showing the electrical configuration of a tablet personal computer (which is an example of a playback device).

The entire operation of a tablet personal computer (hereinafter referred to as a personal computer) 80 is controlled by a CPU (Central Processing Unit) 80.

The personal computer 80 is provided with a display device 82. This display device 82 is
It is controlled by a display control device 81 which is controlled by a CPU 90. Further, the personal computer 80 is provided with an acceleration sensor 93, and an output signal from the acceleration sensor 93 is input to the CPU 90. Furthermore, the personal computer 80 includes an SSD (solid state drive) 94 that is accessed by the CPU 90, and a memory card that reads data recorded on the memory card 25 and writes data etc. to the memory card 25. Includes reader/writer 95.

Furthermore, the personal computer 80 includes an input device 96 such as a keyboard and a mouse, a memory 97 such as a RAM, and a communication circuit 98 that connects to a network such as the Internet.

An operating program, which will be described later, is received by the communication circuit 98 of the personal computer 80 via the Internet and installed on the personal computer 80. The CPU 90 controls each section by reading and executing the installed program. The operating program is stored in a recording medium such as the memory card 25, and the operating program may be read from such a recording medium and installed in the personal computer 80.

11 to 13 are flowcharts showing the reproduction processing procedure in the personal computer 80. 14 to 24 are examples of playback windows displayed on the display screen of the display device 92 of the personal computer 80.

The memory card 25 installed in the drive recorder 1 is removed from the drive recorder 1 and installed in the personal computer 80. When the operating program installed on the personal computer 80 is started by the user, the display device 92
A playback window 130 shown in FIG. 14 is displayed on the display screen.

FIG. 14 is an example of the playback window 130.

A menu bar 131 is formed at the top of the playback window 130, and a video display area 141 is formed below this menu bar 131. Furthermore, a map image display area 151, a vehicle information display area 161, and a play list display area 171 are formed below the video display area 141.

The menu bar 131 displays various items, and by selecting each item, a pull-down menu is displayed and detailed commands can be given. The menu bar 131 includes the text strings ``File,''``View,''``Play,''``Tool,'' and ``Information,'' a folder icon 132, a camera icon 133, a printer icon 134, and a reel icon. 135, an FD (flexible disk) icon 136, a log icon 137, and a gear icon 138 are displayed. By pulling down the strings ``File,''``Display,''``Playback,''``Tool,'' and ``Information,'' a pull-down menu corresponding to each item will appear, allowing detailed commands to be given. . A folder is specified by clicking the folder icon 132, a still image is converted by clicking the camera icon 133, printing is performed by clicking the printer icon 134, and a reel icon 135 is clicked. By clicking, video conversion is performed, by clicking the FD (flexible disk) icon 136, the data is backed up, by clicking the log icon 137, it is converted to log data, and by clicking the gear icon By clicking 138, the recording settings are changed. The video display area 141 is an area for displaying video recorded by the drive recorder 1 or the like. The map image display area 151 is an area that displays an image of a map near the shooting location of the video displayed in the video display area 141. The vehicle information etc. display area 161 is an area that displays information such as the speed of the vehicle at the time of video recording. The vehicle information display area 161 includes a vehicle status display area 162 that displays the status of the vehicle to which the drive recorder 1 that recorded the video displayed in the video display area 141 is installed, and a vehicle status display area 162 that displays the status of the vehicle, such as video playback, stop, etc. It includes an operation button 163 for giving operation commands, a time display area 166 for displaying the recording time of the video displayed in the video display area 141, an acceleration graph display area 168 for displaying a graph of the acceleration of the vehicle, etc. . The play list display area 171 is an area where file names of video files recorded on the memory card 25 are displayed in a list.

When the folder icon 132 is clicked and a folder containing video files is selected, the file names of the video files stored in that folder are displayed in the play list display area 171, as shown in FIG. , the file name of the video file is displayed in a clickable manner. Although the video recording start time is used as the file name, it goes without saying that the file name may be anything other than the recording start time.

When one of the file names displayed in the play list display area 171 is clicked using the mouse included in the input device 96, the corresponding video file is read out from the memory card 25. It will be done. Playback of the video data stored in the read video file starts (step 111 in FIG. 11). In the header recording area of a video file, various header data are recorded in accordance with the recording time of the video data, and when the video at the recording time is displayed, the recorded header data is also read and read. Various superimposition processes and the like are performed on the video depending on the contents of the output header data.

Playback of video data (The video data to be played may be one that was shot and recorded while the user viewing the video was driving, or one that was shot and recorded while a user other than the user viewing the video was driving) 15), the video represented by the video data is displayed in the video display area 141 as a moving image, as shown in FIG. Further, data representing GPS information recorded in the video file in which the video data is stored is transmitted via the Internet to a web site that transmits data representing the map image. Then, image data representing a map near the location where the video displayed in the video display area 141 was recorded is transmitted from that site to the personal computer 80. When image data representing a map is received by the personal computer 80, the map image is displayed in the map image display area 151. An arrow icon 152 representing the location displayed in the video display area 141 is also displayed on the map image. In addition, the vehicle status display area 162 of the vehicle information etc. display area 161 displays the speed of the vehicle when recording the video displayed in the video display area 141, the longitudinal acceleration (X) of the vehicle, and the left and right acceleration of the vehicle. The acceleration in the direction (Y), the acceleration in the vertical direction of the vehicle (Z), the latitude (N) indicating the vehicle position, and the longitude (E) indicating the vehicle position are displayed. The time display area 166 displays the recording time of the video displayed in the video display area 141, and the acceleration graph display area 168 displays an acceleration graph.

Video data is recorded in all directions, so it includes not only images of objects visible from the direction of travel of the vehicle, but also images of objects behind (opposite to the direction of travel) and to the left of the vehicle (in the direction of travel of the vehicle). It is also possible to display objects to the left (with reference to the direction of travel of the vehicle) or to the right of the vehicle (to the right with respect to the traveling direction of the vehicle). FIG. 16 shows a playback window 130 in which an image of a subject to the rear is displayed in the image display area 141, and FIG. 17 shows a playback window 130 in which an image of a subject to the right is displayed in the image display area 141. , FIG. 18 shows a playback window 130 in which a video of a subject to the left is displayed in a video display area 141. For example, if you drag the mouse to the right in the state shown in Figure 15, the image of the object in the forward direction will gradually switch to the image of the object in the right direction, and as shown in Figure 17, the image of the object in the right direction will change to the image of the object in the right direction. Displayed in display area 141. If you drag the mouse further to the right in the state shown in FIG. 17, the image will gradually change from the image of the object to the right, and the image of the object behind will be displayed in the image display area 141 as shown in FIG. If the user drags the mouse further to the right in the state shown in FIG. 16, the image gradually changes from the image of the object behind, and the image of the object to the left is displayed in the image display area 141, as shown in FIG. If the user drags the mouse further to the right in the state shown in FIG. 18, the image gradually changes from the image of the object in the left direction, and the image of the object in the advancing direction is displayed in the image display area 141, as shown in FIG. In this way, images of the subject in all directions can be displayed in the image display area 141. In the above example, the mouse is used to drag the mouse to the right, but even if the mouse is dragged to the left, the image of the object in the advancing direction as shown in FIG. The image gradually switches to the image of the object to the left as shown in Fig. 16, and then gradually changes to the image of the object to the rear as shown in Fig. 17. As shown in Figure 15, the image gradually switches to the image of the subject in the direction of travel.

In this embodiment, when a video is displayed in the video display area 141, the viewing range is superimposed on the video (step 111 in FIG. 11).

FIG. 19 is an example of the playback window 130 showing a state in which the user's visual field range is displayed.

In the video display area 141, a frame 143 indicating a viewing range is displayed by the CPU 90 superimposed on the video represented by the video data (this is an example of processing to be displayed on the video). The portion of the image outside the field of view 142 is blurred by the CPU 90 (hatching is shown to indicate blurring). The visual field range is predetermined for each user's age, and data representing the visual field range is recorded in the SSD 94 in advance. By specifying the user's age, data representing the visual field range corresponding to the specified age is read from the SSD 94, and the frame 143 indicating the visual field range and the area 142 outside the visual field range are blurred (obtained) as shown in FIG. This is an example of control for processing video data based on the processed information and displaying a video based on the processed video data). Of course, the visual field range of each user may be detected in advance, and data representing the user's unique visual field range may be recorded in the SSD 94 in advance. A frame 143 indicating the visual field range for each user will be superimposed on the video. Since the user can see the field of view, he or she can recognize what range is visible and what range is not visible.

The CPU 90 also determines whether there is an object (for example, an infant) that requires attention in the area 142 outside the visual field (step 113 in FIG. 11). If there is an object to be noted in the area 142 outside the field of view (YES in step 113 in FIG. 11), a warning (this is an example of notification to the user) is given to the user (step 114). For example, a frame or a mark may be placed around the area where an object to be noted is displayed. To determine whether there is an object to be careful of in the area 142 outside the visual field, the object to be careful is determined in advance, the CPU 90 performs video analysis, and the image of the object is detected in the area 142 outside the visual field. This may be done based on whether or not the CPU 90 detects that it is being displayed. The user understands that there are objects that need attention outside the field of view, and that driving while driving while substantially expanding the field of vision by shaking the head as necessary can help prevent accidents. can. If there is no object to be noted in the area 142 outside the visual field (NO in step 113 in FIG. 11), the process in step 114 is skipped and no warning is issued. If there is an object to be noted in the area 142 outside the viewing range, the user may be notified, so it does not necessarily have to be displayed in the area 142 outside the viewing range. For example, the user may be notified by outputting a sound.

Further, a mark 144 indicating the user's line of sight is superimposed on the video display area 141 by the CPU 90 (this is an example of a processing process for displaying the user's line of sight on the video) (step 115 in FIG. 11). Data indicating the user's line of sight is recorded in the header recording area 31 of the video file in association with the video recording time, so by reading the data indicating the user's line of sight from the header recording area 31, the data can be displayed. A mark representing the user's line of sight can be superimposed on the video displayed in the video display area 141 at the recording time of the video. Since the user's line of sight is superimposed on the video, it is possible to check whether the user is looking at objects that they should be looking at (for example, traffic lights, oncoming cars, pedestrians, etc.).

In the example shown in FIG. 19, a frame 143 of the visual field range and a mark 144 indicating the user's line of sight are displayed regardless of the user's line of sight. In other words, the viewing range frame 143 indicates the range that will be the viewing range when the user views the video displayed in the video display area 141. A viewing range frame 143 is displayed approximately in the center of the video display area 141 regardless of the user's line of sight while driving. For example, even if the user's line of sight is not at the center of the image but at the edge of the image displayed in the image display area 141, the field of view frame 143 is displayed approximately at the center of the image display area 141. be done. On the other hand, a frame 143 of the viewing range may be displayed in the video display area 141 centered on the user's actual line of sight. The user's field of view when actually driving can be seen. In this case as well, blurring processing or the like may be performed on the area 142 outside the viewing range. The area within the viewing range frame 143 is an example of the range that is visible to the user while the user is driving the vehicle, and the area 142 outside the viewing range is an example of the area that is not visible to the user while the user is driving the vehicle. This is an example of displaying these ranges separately.

If the direction in which the user is facing changes at the recording time of the video displayed in the video display area 142 (YES in step 116 in FIG. 11), the viewing range frame 143 is moved in the changed direction and superimposed on the video. (Figure 11 Step 117). Whether the direction in which the user is facing has changed can be determined based on data indicating the direction in which the user is facing, which is recorded in the header recording area of the video file. If the direction in which the user is facing does not change (NO in step 116 in FIG. 11), the field of view frame 143 is not moved.

FIG. 20 is an example in which the viewing range frame 143 is moved.

It is assumed that the direction in which the user is facing at the time of recording the video displayed in the video display area 141 changes from approximately the front in the direction of travel to the lower right direction. Then, the position of the viewing range frame 143 is moved to the position of the viewing range frame 143A. Further, the position of the mark 144 indicating the line of sight is moved to the position of the mark 144A indicating the line of sight. The user can recognize that the field of view has also changed due to the change in the direction in which the user is facing. When the direction in which the user is facing changes, the relative position of the image displayed in the image display area 141 and the frame 143 (143A) indicating the visual field range also changes. For this reason, the relative positions of the image and the frame 143 indicating the viewing range may be made to remain unchanged (for example, the frame 143 indicating the viewing range may be displayed approximately in the center of the image). For example, in the same way as changing the image displayed in the image display area 141 when dragged with a mouse, an image may be displayed in a direction in which the user is facing a different direction.

Next, it is determined whether the user's line of sight matches the direction in which the user should look (step 118 in FIG. 12). If they do not match (NO in step 118 in FIG. 12), a mark indicating the location to be viewed is displayed on the video display area 141 (step 119 in FIG. 12), and if they match (YES in step 118 in FIG. 12), A mark indicating a place to be viewed is not displayed on the video display area 141. However, if they match, the user may be notified of the match (by displaying on the video display area 141, outputting sound, etc.). Needless to say, the direction to be viewed differs depending on the driving at the time of recording (right turn, left turn, lane direction, etc.) or the user's driving operation (brake operation, turn signal lever operation, etc.).

FIG. 21 is an example in which marks are displayed to indicate the places to look at.

Even though the user should have looked at the lower right part of the video displayed in the video display area 141, if the user was not looking (for example, was looking straight ahead), then the lower right part of the video displayed in the video display area 141 A mark 144 is displayed in the video display area 141 to indicate that the location is the location that the user should see (this is an example of a processing process for displaying the location that the user should see on the video). If the displayed video is analyzed or event data indicating the event at the time of video recording (operation of turn signal lever, sudden braking, etc.) is recorded in the header recording area of the video file, that record is recorded. The CPU 90 determines where to read and view the event data being displayed. For example, if you operate the turn signal lever to change lanes from the left lane to the right lane, and you are not looking in the right side mirror, mark 145 will be displayed as the place to look in the right side mirror. . Furthermore, when it is necessary to confirm a predetermined place such as a right turn or a left turn, such a place may be displayed as a place to be seen if the user is not looking at such a place.

Furthermore, when the user is not looking at a part that should be viewed, the video display in the video display area 141 is temporarily stopped, and an image (still image) of that scene is displayed in the video display area 141. It's okay. In a video, a scene in which the user is not looking at the part they should be looking at ends in an instant, whereas a stopped image is displayed in the video display area 141, which can draw the user's attention more.

Furthermore, for the video portion of the video displayed in the video display area 141 where there is no (small) change in the movement of the subject (YES in step 120 in FIG. 12), the display speed is made relatively faster by the CPU 90 (FIG. 12 Step 121). For video parts where there is (large) change in the movement of the subject of the video (NO in step 120 in FIG. 12), the display speed is not relatively increased. Whether there is no (little) or significant (large) change in the movement of the subject can be determined by comparing the differences between video frames. Video parts where there is no (small) change in subject movement are not considered to be important scenes such as pedestrians running out or oncoming cars running out, so there is no problem even if the display speed is increased. Additionally, since the vehicle is stationary in the video portion where the vehicle's position does not change, it is thought that accidents are less likely to occur and are not important. For this reason, the display speed may be increased even for video portions where the position of the vehicle has not changed.

Further, the CPU 90 also determines whether the orientation of the display device 92 has been changed (step 122 in FIG. 12). If the orientation has been changed (YES in step 122 in FIG. 12), the display range of the video displayed in the video display area 141 is changed according to the orientation (step 123 in FIG. 12). If the orientation has not been changed (NO in step 122 in FIG. 12), the display range of the video is not changed in accordance with the changed orientation. The acceleration sensor 93 can detect whether the orientation of the display device 92 has been changed and in which direction. Assume that an image in the direction in which the vehicle is traveling (see FIG. 17) is displayed on the display screen of the display device 92 that was facing in a certain direction. When the display device 92, which was facing in a certain direction, is turned to the right, the image displayed in the image display area 141 is moved to the right in the same way as if it were dragged to the right (see Figure 17). is displayed in the video display area 141. Similarly, when the display device 92, which was facing in a certain direction, is turned to the left, the image displayed in the image display area 141 is dragged to the left (Fig. 18) is displayed in the video display area 141. Further, when the display device 92, which was facing in a certain direction, is turned backward, a rearward image (see FIG. 18) is displayed in the image display area 141. Changing the display range of the video using the direction of travel of the vehicle as the reference direction in this way is an example of processing of video data. It becomes possible to display pseudo VR (virtual reality).

Further, the CPU 90 determines whether information regarding the vehicle situation is recorded in the header recording area 31 of the video file at the recording time of the video displayed in the video display area 141 (step 124 in FIG. 12). If information regarding the vehicle status is recorded (YES in step 124 in FIG. 12), the information regarding the vehicle status is displayed in the video display area 141. If the information regarding the vehicle situation is not recorded (NO in step 124 in FIG. 12), the information regarding the vehicle situation is not displayed in the video display area 141.

FIG. 22 is an example in which information regarding the status of the vehicle is displayed in the video display area 141.

A speech bubble 146 in the first display mode is displayed in the video display area 141. This balloon 146 displays information regarding the state of the vehicle at the recording time of the video displayed in the video display area 141. In the example shown in FIG. 22, the character string "The brake was pressed!!" is displayed in the speech bubble 146, and the vehicle was braked (suddenly) at the recording time of the video displayed in the video display area 141. I can see that it was stepped on. Data representing information regarding the vehicle status is also recorded in the header recording area 31 of the video file in association with the time data, and is read from the header recording area 31. However, for example, the CPU 90 may estimate the situation that has occurred in the vehicle from a sudden change in acceleration and display it in the balloon 146 as information regarding the situation of the vehicle. For example, if the acceleration in the longitudinal direction of the vehicle suddenly changes in the negative direction at a certain recording time, it is considered that the brakes were suddenly pressed, so the speech bubble shown in Figure 22 is added to the video when such acceleration occurs. The same speech bubble as 146 may be displayed.

Furthermore, when the user specifies the position of any of the map images displayed in the map image display area 151 using the mouse (YES in step 126 of FIG. 13), the data representing the specified position is transferred to the server. data indicating the latitude and longitude of the location are transmitted from the server to the personal computer 80. When data indicating latitude and longitude are received by the personal computer 80, the scene of the video displayed in the video display area 141 that was shot at the position closest to the latitude and longitude is found, and the scene of the video displayed in the video display area 141 is found. The video from the scene is displayed in the video display area 141 (step 127 in FIG. 13). If no position in the map image is specified (NO in step 126 in FIG. 13), the process in step 127 is skipped, and no video is displayed by specifying a position in the map image. Furthermore, although the video of the specified position in the map image is displayed in the video display area 141, the video displayed in the video display area 141 is recorded by a drive recorder installed in the vehicle in which the user is riding. The video display area 141 may display either or both of the video recorded by the driver and the video recorded by a drive recorder installed in a third party's vehicle. In that case, the server stores data representing the video recorded by the drive recorder installed in the third party's vehicle, and the video data recording the location specified by the user is read out from the server. The data will be sent to the personal computer 80. If a large number of data representing images recorded by a drive recorder installed in a third party's vehicle are recorded on the server, the user is allowed to select the image to be displayed on the personal computer 80 from among those images. , the selected video may be displayed on the display screen of the display device 92.

Furthermore, the CPU 90 determines whether information on road-to-vehicle communication or vehicle-to-vehicle communication is recorded in the header recording area 31 of the video file at the recording time of the video displayed in the video display area 141 (step 128 in FIG. 13). ). If information on road-to-vehicle communication or vehicle-to-vehicle communication is recorded (YES in step 128 in FIG. 13), the information on the road-to-vehicle communication or vehicle-to-vehicle communication is displayed in the video display area 141. If the information on the road-to-vehicle communication and the vehicle-to-vehicle communication is not recorded (NO in step 128 in FIG. 13), the information on the road-to-vehicle communication or the vehicle-to-vehicle communication is not displayed in the video display area 141.

FIG. 23 is an example in which information on road-to-vehicle communication (the same applies to vehicle-to-vehicle communication) is displayed in the video display area 141.

In the video display area 141, a speech balloon 147 is displayed in a second display mode that is different from the first display mode of the speech balloon 146 described above (see FIG. 22). This balloon 147 displays information about the road-to-vehicle communication at the recording time of the video displayed in the video display area 141. In the example shown in FIG. 23, the text string "There is a vehicle running ahead and to the left" is displayed in the speech bubble 147, and there is a vehicle running ahead and to the left of the vehicle at the recording time of the video displayed in the video display area 141. It turns out that there was. Data representing information on road-to-vehicle communication and vehicle-to-vehicle communication is also recorded in the header recording area 31 of the video file in association with time data, and is read from the header recording area 31. Users who have obtained information on road-to-vehicle communications and vehicle-to-vehicle communications will be able to check what vehicle operations they have performed (for example, whether they have decelerated) based on that information when recording video.

In the playback processing procedure shown in Figure 11, there is a process to determine whether there is an object of attention outside the field of view (Step 113 in Figure 11), a process to determine whether the direction the user is facing has changed (Step 113 in Figure 11), a process to determine whether the user The process determines whether the line of sight of the user matches the direction in which the user should look (step 118 in FIG. 12), the process determines whether there is any change in the movement of the subject (step 120 in FIG. 12), and the direction of the display device 92 is changed. (Step 122 in Figure 12), Processing to determine whether there is information regarding the vehicle status (Step 124 in Figure 12), Processing to determine whether a position in the map image has been specified (Step 126 in Figure 13), Each process is performed in the order of determining whether information on vehicle-to-vehicle communication or road-to-vehicle communication is obtained (step 128 in Figure 13), but the order of these processes may be arbitrary, and each process may be performed in parallel. It may be done.

(3) Modification FIG. 24 shows a part of the play list display area 171 included in the playback window 130.

In the embodiment described above, the recording date and time of the video data is used for the file name of the video file displayed in the play list display area 171, but in this embodiment, the recording date and time of the video data is used instead of the recording date and time of the video data. The recording location of the video data (an example of a shooting location) is used as the file name of the video file.

Set the recording location of the video data as the file name of the video file as follows. For example, when a video file is generated in the drive recorder 1, the latitude and longitude of the location where the video file is generated is read by the GPS receiver 22, and the latitude and longitude data is transmitted from the communication circuit 17 to the server via the Internet. Send to. The server finds these latitude and longitude locations, receives data representing the found locations via the Internet, and uses the location obtained from the received data as the file name of the video file. However, the latitude and longitude data recorded in the header recording area 31 of the video file corresponding to the recorded date and time is sent to the server via the Internet during playback, and the data representing the location found is sent to the server. , and the location obtained from the received data may be used as the file name of the video file. In FIG. 24, the file names are displayed in the order in which they were recorded. The location of the recording is included in the file name, so you can see at a glance where the image was taken.

FIG. 25 also shows a part of the play list display area 171 included in the playback window 130.

In Figure 25, the play list is set so that the file names of video files that include locations at the time of recording that occur less frequently are displayed at the top (this is an example of displaying with priority). It is displayed in display area 171. File names of video files that include locations at the time of recording that occur frequently are displayed at the bottom (lower part). Recording locations with a high frequency of occurrence are likely to be recorded in daily life, such as records during commuting, whereas recording locations with a low frequency of occurrence are likely to be records of non-daily life, such as records of travel. It is thought that there are many records from daily life. To facilitate playback of video files recorded in daily life, file names of video files that include locations at the time of recording that occur less frequently are displayed at the top.

Furthermore, the same locations may be displayed together in the play list display area 171. It becomes easier to distinguish between video files from the same location and video files from different locations. Furthermore, the file name of the video file only needs to include the location at the time of recording, and the file name of the video file may include the recording date and time in addition to the location at the time of recording. Furthermore, an image of a map of the location corresponding to the video file name displayed in the play list display area 171 may be displayed in the video display area 141, for example. Since a map is displayed rather than just the name, it is easier to see which location the video file is located. For example, the file name of the video file including the location is transmitted from the communication circuit 98 to a map server (not shown) on the Internet, and the personal computer 80 receives the map data transmitted from the map server and displays the map in the video display area 141. Display the image. By selecting an image of the map displayed in the video display area 141, playback of the corresponding video file may be started.

FIG. 26 is an example of a frame structure based on MPEG (Moving Picture Experts Group).

In FIG. 26, those marked with "I" indicate I frames, and those marked with "P" indicate P frames. One I frame is followed by three P frames, and one I frame and three P frames constitute a GOP (Group Of Picture).

When calculating a hash value as shown in step 77 of FIG. 8, the controller 10 calculates a hash value for each unit by using the unit immediately before the P frame instead of the unit immediately before the I frame as a unit for dividing the video data. For example, a hash value is calculated in the controller 10 using a P frame, an I frame, and three P frames after the I frame as one unit. The hash value calculated in this way is sent to the block chain system.

In addition, instead of calculating the hash value from video data, when various events occur in the drive recorder (for example, sudden braking, side skidding, etc.), the hash value is calculated from the data representing the event. (which is an example of a second calculated value uniquely determined from event data) may be calculated in the controller 10, and the calculated hash value may be sent to the block chain system.

FIG. 27 is a flowchart showing an example of the recording processing procedure of the drive recorder 1.

In this embodiment, when a driver (for example, a professional driver) is called, the appearance of the driver is photographed and recorded. The recorded video data is stored in a video file, transmitted to the server for each driver, and stored in the server. The drive recorder 1 accesses the server and reads a video file in which video data during driver roll call is stored (step 181). The controller 10 of the drive recorder 1 records the video file at the time of roll call on the memory card 25 attached to the drive recorder 1 (step 182).

Next, the driver's name and date and time are used as the file name of a video file that stores video data during driving, and a video file is generated by the controller 10 (step 183). The driver's name may be stored in advance in the memory 21 of the drive recorder 1, or may be input from the input device 20. The engine of the vehicle to which the drive recorder 1 is installed starts, the recording of video data during driving begins, and the video data is stored in the video file as described above until the recording is completed (YES in step 185). Continue.

In this way, a video file storing video data during roll call and a video file storing video data during driving are recorded on the memory card 25. When such a memory card 25 is installed in the personal computer 80, the image during roll call and the image during driving can be displayed in the image display area 141 of the playback window 130. The video file storing the video data during roll call and the video file storing the video data during driving are associated by the driver's name (this is an example of association).

In this embodiment, in the drive recorder 1, a video file for roll call and a video file for driving are stored in the memory card 25, but during playback, the video file for roll call and the video file for driving are stored in the memory card 25. The video file may also be stored. For example, a memory card 25 storing a driving video file is installed in the personal computer 80, and the file name of the video file is displayed in the play list display area 172. When one of the video file names displayed in the play list display area 172 is selected, the video file name corresponding to the selected video file name is recorded in the header recording area 31. Data representing the name of the driver present is sent to the server that records the video file for the roll call, and the server sends the video file for the roll call to the personal computer 80. When the roll call video file is received by the personal computer 80, the roll call video represented by the driver's roll call video file and the driving time represented by the driver's driving video file are displayed. The video and are displayed in the video display area 141. The image during roll call and the image during driving are preferably displayed side by side in the image display area 141 at the same time, but this does not necessarily have to be the case.

28 and 29 are flowcharts showing an example of the recording processing procedure of the drive recorder 1. Figure 28 shows an example of the recording processing procedure when the drive recorder 1 is attached to a truck. It is a flowchart showing the recording processing procedure of the drive recorder 1 attached to the forklift when moving cargo.

Referring to FIG. 28, while the truck is driving, video data is recorded by the drive recorder 1 attached to the truck onto the memory card 25 attached to the drive recorder 1 (step 191). When recording of the video data to the memory card 25 is completed, the data representing the truck driver's name and the date and time are stored in the header recording area of the video file, and the video data is stored in the video recording area of the video file. The recorded video file is transmitted from the drive recorder 1 to the server (step 192). The memory 21 of the drive recorder 1 stores in advance data representing the driver's name. Further, the drive recorder sends a command for associating it with the video file from the forklift to the server (step 193). For example, the video file is transmitted from the same location as the last location (the truck's stopping location) represented by the video data stored in the video file transmitted from the truck's drive recorder 1, and the truck's stopping time. Within a few minutes, the drive recorder attached to the forklift sends a command to the server to associate it with the video file containing the video data that the drive recorder attached to the forklift started recording. Ru.

Referring to FIG. 29, when the drive recorder 1 attached to the forklift detects that the driver approaches the forklift, the front camera 11 and rear camera 12 included in the drive recorder 1 Shooting starts (step 201).

When the video captured by the front camera 11 or the rear camera 12 is analyzed and it is detected that the driver gets on the forklift and the forklift starts operating, or that the driver gets on the forklift (step 202 (YES), recording of video data to the memory card 25 attached to the drive recorder 1 attached to the forklift is started (step 203). When recording is completed, the video file is sent to the server (step 205).

On the server, if there are two video files whose recording end time and recording start time are almost the same, and where the location at the end of recording and the location at the time of recording start are almost the same, those video files will be As mentioned above, for example, the video file is treated as a video file of the same driver switching from a truck to a forklift. By transmitting a series of images as a video file to the personal computer 80, images of truck driving and forklift driving can be displayed in the video display area 141 of the playback window 130.

In the above-described embodiment, the truck is driven and then the forklift is driven, but the forklift and then the truck may be driven to obtain respective video files. Additionally, a series of video files can be obtained in which a forklift is used to load cargo onto a truck, the truck is then driven to a destination, and the forklift is used to unload the truck's cargo at the destination. It's okay.

FIG. 30 is a flowchart showing an example of the recording processing procedure of the drive recorder 1 attached to the vehicle.

The drive recorder 1 is attached to the upper part of the inside of the windshield of the vehicle at a position where the speedometer of the vehicle can be photographed. When the engine of the vehicle starts, the front camera 11 and rear camera 12 start photographing, and the rear camera 12 photographs the speedometer of the vehicle (step 211). The image of the speedometer of the vehicle is analyzed by the controller 10, and it is determined whether the speed of the vehicle is, for example, 30 km/h or more (this is an example of a speed above a certain level, but other speeds are also possible). (step 212). When the speed exceeds 30 km/h (YES in step 212), recording of video data to the memory card 25 starts (step 213). When the vehicle speed becomes less than 30 km/h (YES in step 214), recording of video data to the memory card 25 is stopped (step 215).

In this way, when the speed of the vehicle is 30 km/h or higher, the captured video data is recorded to the memory card 25, and when the vehicle speed is less than 30 km/h, the captured video data is recorded to the memory card 25. is stopped. When the vehicle is being driven at a speed greater than a certain speed, video data obtained through photography will be recorded on the memory card 25. In this embodiment, when the speed of the vehicle exceeds 30 km/h, the captured video data is recorded to the memory card 25, and when the vehicle speed becomes less than 30 km/h, the captured video data is recorded to the memory card 25. However, if the speed of the vehicle exceeds 30km/h, the captured video data will be recorded on the memory card 25, but even if the speed is less than 30km/h, the recording will continue. There is no need to stop recording the captured video data to the memory card 25, and the recorded video data is recorded to the memory card 25 regardless of the speed of the vehicle, and the speed is less than 30 km/h. In this case, recording of the photographed video data to the memory card 25 may be stopped.

FIG. 31 is a flowchart showing an example of a recording processing procedure of a drive recorder attached to an automatic guided vehicle (unmanned guided vehicle).

The drive recorder 1 described above is attached to an automatic guided vehicle. Automated guided vehicles are equipped with laser sensors that detect when surrounding objects approach the automated guided vehicle.

When the engine of the automatic guided vehicle is started (the power of the automatic guided vehicle is turned on), power supply to the drive recorder 1 starts. Then, the front camera 11 and rear camera 12 provided in the drive recorder 1 start photographing (step 221). When the laser sensor detects the approach of an object (this is an example of a peripheral object, and may be another automatic guided vehicle) to the automated guided vehicle (YES in step 222), the laser sensor attached to the drive recorder 1 Recording of video data to the memory card 25 begins (step 223). If the laser sensor does not detect the approach of a peripheral object to the automatic guided vehicle (step NO), recording of video data to the memory card 25 does not start. Even if a surrounding object accidentally comes into contact with the automated guided vehicle, the situation can be recorded. In this way, vehicles include not only vehicles in which a driver rides, but also vehicles in which a driver does not ride.

FIG. 32 is a flowchart illustrating an example of a processing procedure for driving evaluation.

A user drives a vehicle to which the drive recorder 1 is attached, removes the memory card 25 on which video data is recorded from the drive recorder 1, and attaches it to the personal computer 80. The playback window 130 shown in FIG. 14 is displayed on the display screen of the display device 92, and the video data recorded on the memory card 25 is played back (step 231). If there is movement in the subject of the video being played back (YES at step 232), the video is analyzed and a driving evaluation is performed from the video (step 233). For example, if the subject moves and the brakes are suddenly pressed, or if other vehicles such as pedestrians, bicycles, motorcycles, or cars approach too closely, the driving evaluation will be lowered. Acceleration data can tell whether the brakes have been applied suddenly, and whether pedestrians or other vehicles are getting too close can be determined by the relative size of the pedestrians or other vehicles in the video. If there is no movement in the subject of the video (NO at step 232), no driving evaluation is performed for that video portion. The processes of steps 232 and 233 are repeated until the video ends. Portions of the video where the subject does not move are considered to be when the vehicle is stationary, so there is little meaning in evaluating driving. Since driving in such places is not evaluated, there is a high possibility that unnecessary evaluations will not be made. Driving is evaluated when there is movement of the subject, and if there is no movement of the object, driving is not evaluated, but when there is movement of the vehicle, driving is evaluated, and when there is no movement of the vehicle, Driving evaluation may not be performed.

FIG. 33 is a flowchart illustrating an example of a procedure for reproducing video data recorded by the drive recorder 1.

In this embodiment, the drive recorder 1 is attached to a vehicle that transports passengers, such as a bus, taxi, hire car, or train, but it does not necessarily need to be attached to such a vehicle.

The memory card 25 installed in the drive recorder 1 is removed from the drive recorder 1 and installed in the personal computer 80. A playback window 130 shown in FIG. 14 is displayed on the display screen of the display device 14 of the personal computer 80, and the video data recorded on the memory card 25 is played back (step 241).

The video is analyzed by the CPU 90, and the CPU 90 determines whether or not the customer is included in the video (step 242). For example, if there is a person sitting in the vehicle seat other than the driver, it can be determined that there is a customer. If the customer is included in the video (YES at step 242), the video is displayed in the video display area 141 (step 243). If the customer is not included in the video (NO in step 242), the video display in the video display area 141 is stopped. The processes of steps 242 and 243 are repeated until the video ends (NO in step 244). In the above embodiment, when the customer is not included in the video, the video display in the video display area 141 is stopped.However, although the video is displayed, the display speed may be increased or the portion of the video that includes the customer may be You can also skip the video until the end.

FIG. 34 is a flowchart illustrating an example of a procedure for reproducing recorded video data. It is preferable that the data be recorded by the drive recorder 1, but it may be recorded by other recording devices. This playback process is to find a video of a desired scene from among the videos.

The memory card 25 on which video data is recorded is inserted into the personal computer 80, and the playback window 130 is displayed on the display screen of the display device 92. When the user clicks on the word "Tools" included in the menu bar 131 of the playback window 130, a pull-down menu containing the "Search" item appears. When the "Search" item is selected from the pull-down menu, a pop-up window for entering a keyword appears on the playback window 130. Into the pop-up window, the user inputs a keyword representing the situation of the video portion that the user wishes to find using the input device 96 (step 251). The CPU 90 analyzes the video data and finds videos of scenes related to the input keyword (step 258). The video of the found scene is displayed in the video display area 141 (step 254). The keyword input by the user to represent the situation of the video portion may be not only a clear one such as a video containing a bullet train, but also a vague one such as a bright video. A video portion corresponding to the ambiguous keyword is found. For example, if the image is bright, the parts of the image with relatively high brightness are found. Alternatively, a keyword representing the condition of the vehicle at the time of recording the video data may be input, and a video of that condition may be found. For example, if a keyword is used to describe the condition of a vehicle equipped with a drive recorder that recorded video data, such as speeding up, sudden acceleration, or turning on a turn signal, the system can find the video portion that corresponds to that keyword. can. Since the data representing the state of the vehicle is recorded together with the recording time in the header recording area 31 of the video file in which the video data is stored, the video portion can be found using this data. It is also possible to input a keyword to search for a video after a dynamic movement, such as finding a video after a red car passes a blue car, and find the corresponding video.

FIG. 35 shows a flowchart illustrating an example of a procedure for reproducing video data recorded by the tribe recorder 1.

The memory card 25 installed in the drive recorder 1 is removed from the drive recorder 1 and installed in the personal computer 80. A playback window 130 shown in FIG. 14 is displayed on the display screen of the display device 92 of the personal computer 80.

Data indicating the latitude and longitude indicating the moving position of the vehicle recorded in the header recording area 31 of the video file stored in the memory card 25 is read, and the moving range of the vehicle is determined by the CPU 90 of the personal computer 80. detected (step 261). A request command for map image data including the detected movement range is transmitted from the personal computer 80 to the server, and the map image data is transmitted from the server to the personal computer 80. When the map image data is received by the personal computer 80, the map image represented by the received image data is displayed in the video display area 141 (step 262). Furthermore, data representing the operation information of the vehicle at the time of recording the video data is read from the header recording area 31 of the video file, and is displayed on the map image displayed in the video display area 141. The operation status of is displayed (step 264).

FIG. 35 is an example of a map image displayed in the video display area 141.

In the map image, the movement status of the vehicle is indicated by a chain line 148. The dashed line 148 representing the movement status is obtained from data indicating the movement position of the vehicle recorded in the header recording area 31. Further, in the movement situation indicated by the chain line, the operating situation of the vehicle is displayed in a balloon 146. By looking at the map image, the user can relatively easily grasp the range of movement of the vehicle, as well as the location and operation of the vehicle during the movement.

FIG. 37 is a flowchart illustrating an example of a procedure for reproducing video data recorded by the drive recorder 1. In this embodiment, a video obtained by the user's driving and a video obtained by the third party's driving are displayed in the video display area 141.

The memory card 25 installed in the vehicle that the user was driving is removed, and the removed memory card 25 is installed in the personal computer 80. Video data obtained by the user driving the vehicle is read from the memory card 25 and played back (step 271). In addition, data indicating the recording location (vehicle movement range) of video data obtained by the user's driving (data indicating latitude and longitude, respectively) is read from the header recording area 31 of the video file, and the third party's driving The video files obtained are sent to a server where a large number of video files are stored. In the server, a video file storing video data obtained by driving in the same location as the user's driving location is found and transmitted to the personal computer 80. It is only necessary to find a video file of a third party that almost matches, if not completely matches, the direction and range of movement of the user. The found video file of the third party is received by the personal computer 80 (step 273), and the video obtained by the user's driving and the video obtained by the third party's driving are displayed in the video display area 141. (step 274).

FIG. 38 is an example of the playback window 130.

The video display area 141 is divided into two video display areas 141A and 141B on the left and right sides (not just left and right, but also top and bottom). In the image display area 141A on the left, images obtained by the user's driving (an example of images based on video data after processing) are displayed, and in the image display area 141B on the right, images found on the server are displayed. The video obtained by the third party driving represented by the file is displayed. In the video display area 141A, a balloon 146A is displayed that indicates information on the user's operation of the vehicle when the video displayed in the video display area 141A was recorded. Needless to say, the operation information displayed in the balloon 146A is obtained from the acceleration data etc. recorded in the header recording area 31 of the video file in which video data is stored. Similarly, a balloon 146B indicating vehicle operation information by a third party when the video displayed in the video display area 141B was recorded is also displayed in the video display area 141B. The reproduction control is performed by the CPU 90 so that the display of the images is synchronized so that the position indicated by the image displayed in the image display area 141A and the position indicated by the image displayed in the image display area 141B almost match. It is preferable that this is done.

You can compare and view the user's driving video displayed in the video display area 141A and the third party's driving video displayed in the video display area 141B, so you can compare your own driving with the third party's driving. You can improve your driving skills by comparing the In particular, since the balloons 146A and 146B allow you to see whether the operations match or differ, you can also see whether the operations match or differ from those of others.

Furthermore, the user's driving is compared with the driving of a third party, and information representing the evaluation of the user's driving (this is an example of information based on a comparison between the third party's driving and the user's driving) is displayed in the video display area. 141 may be displayed. For example, if your driving skills are evaluated to be worse than those of a third party, you can display warning information such as "Use the third party's driving as a reference to improve your driving skills." Alternatively, if the driver's driving skills are evaluated to be higher than that of a third party, information indicating the high evaluation may be displayed, such as "Your driving skills seem to be high." .

FIG. 39 is an example of the playback window 130.

Alphabetical characters indicating the locations and the order in which the driver of the vehicle should look when recording the video displayed in the video display area 141 are superimposed on the video. The example shown in FIG. 39 is, for example, a video when changing lanes, and it is preferable to view the parts indicated by alphabets in alphabetical order. When recording video data, the driver's line of sight is detected as described above, and the CPU 90 detects whether the driver's line of sight is moving in alphabetical order through the locations indicated by the letters. When the driver's line of sight moves in alphabetical order through the locations indicated by the alphabet, for example, the character string "This is ideal driving." is displayed in the video display area 141. The user can confirm that his or her driving skills are not bad. If the driver's line of sight is not moving at the location indicated by the alphabet, a character string such as "There are locations that have not been confirmed." is displayed in the video display area 141 to notify the user that there are locations that have not been confirmed. Let me know. Additionally, if the driver's line of sight does not move in alphabetical order, a string such as "The order of confirmation is not ideal." is displayed in the video display area 141 to notify the user that the order of confirmation is not correct. . Users can know whether their line of sight movements while driving are appropriate.

Furthermore, in the example shown in Figure 19, processing processing was performed to blur the area 142 outside the viewing range, but it is now possible to detect an area that the user has not seen within a certain period of time and blur the area that the user has not seen. You can. For example, the range that the user is not looking at may be detected from the start of the turn to the end of the turn, and the range that the user is not looking at may be blurred. Needless to say, the line of sight of the user, who is a driver, is detected when recording video data, and the range that the user is not looking at is detected using the data on the line of sight. For example, when an unviewed range is detected, the unviewed range is blurred or made semi-transparent in the same way as the area 142 outside the viewing range, as shown in FIG.

Further, when reproducing the video data recorded by the drive recorder 1, the CPU 90 may perform correction (this is an example of processing) so as to match the memory color of the driver. In order to correct to the memory color, the contrast, tone, and saturation of the obtained image are adjusted, but the degree of each adjustment may be determined in advance, or several preset adjustment values may be determined. It may also be possible to set it in advance and select it during playback.

Furthermore, a communication robot may be used together with the personal computer 80 when playing back images.

FIG. 40 is an example of a communication robot 280 placed near a personal computer 80.

The communication robot 280 includes a body 281 and a head 282. An eye camera is provided inside the eye 283 of the head 282, and a speaker is provided in the mouth 284. The head 282 is rotatable clockwise and counterclockwise relative to the body 281. A communication robot 280 is placed on the right side when facing the personal computer 80. A user sits in front of a personal computer 80. In this way, the positions of the personal computer 80, communication robot 280, and user are determined in advance. When displaying the playback window 130 on the display screen of the display device 92 of the personal computer 80 and outputting a voice comment about the video displayed in the video display area 141, the user turns toward the display device 92 and views the video. When outputting a voice comment to the user, the communication robot 280 is controlled so as to face the user.

FIG. 41 is a block diagram showing the electrical configuration of communication robot 280.

The communication robot 280 is connected to a CPU 291 that controls the overall operation, a memory 292 that stores data, etc., and a wireless communication circuit 294 that communicates with the personal computer 80, the drive recorder 1, etc. A wireless LAN (local area network) board 293 is included. The communication robot 280 also includes a head motor 296 that rotates the head 282, an eye camera 298, a speaker 300, and a microphone 301. The head motor 296 is controlled by a motor control board 295, and the image data taken by the eye camera 298 is taken into the communication robot 280 by the sensor board 297. Furthermore, audio output from speaker 300 and audio input from microphone 301 are controlled by audio input/output board 299.

FIG. 42 is a flowchart showing the processing procedure of communication robot 280. Needless to say, the operation program for communication robot 280 is stored in memory 292.

The above-described playback window 130 is displayed on the display screen of the display device 92 of the personal computer 80, and the video recorded by the drive recorder 1 is displayed in the video display area of the playback window 130. The personal computer 80 and the communication robot 280 communicate, and acceleration data is sent from the personal computer 80 to the communication robot 280 (data other than acceleration data and video file data). Data, event data, etc. recorded in the header recording area 31 may be transmitted from the personal computer 80 to the communication robot 280.Data that allows the communication robot 280 to advise the user etc. by viewing the video may be sent from the personal computer 80 to the communication robot 280).

Based on the acceleration data sent to the communication robot 280, the CPU 291 of the communication robot 280 detects sudden changes in acceleration in the longitudinal direction of the vehicle to which the drive recorder 1 that records the displayed image is attached. If it is determined that there was a brake failure (step 311), it is considered that the brakes of the vehicle were suddenly pressed. Then, as shown in FIG. 40, the head 282 of the communication robot 280, which was facing the user, is rotated 90 degrees clockwise (step 312), and is now facing the display device 92 of the personal computer 80. be exposed. The display screen of the display device 92 displays an image of the scene where the brakes are suddenly applied, and when the communication robot 280 turns toward the display device 92, an audio message is output from the speaker 300 indicating that the situation is a dangerous driving situation. (step 313). For example, a voice saying "Driving like this is dangerous." is output. The audio data is stored in memory 292.

The head 282 of the communication robot 280 is then moved 90 degrees counterclockwise (step 314) so that the communication robot 280 faces the user. When the communication robot 280 faces the user, a voice is output from the speaker 300 to warn the user not to drive in a dangerous manner, such as in the video scene displayed on the display screen of the display device 92 (step 315). ). For example, a voice message saying, ``Let's drive safely.'' is output. Since the warning comes from the communication robot 280, rather than a human like an instructor, the user obediently listens to the warning.

FIG. 43 shows the communication robot 280 described above placed on a dashboard 311 inside a car.

As described above, the drive recorder 1 is attached to the top of the windshield 310. When the drive recorder 1 and the communication robot 280 communicate, and as described above, when the drive recorder 1 recognizes that the vehicle is operating automatically, the drive recorder 1 causes the communication robot 280 to start operating. Command is sent. The communication robot 280 operates and starts talking with the driver of the vehicle, etc.

Furthermore, when playing back video data recorded on the drive recorder 1, information on the road on which the vehicle to which the drive recorder 1 is attached is running (for example, the road is narrow, the road is a dead end, etc.) information) from the video data, and the display of the video displayed in the video display area 141 may be changed depending on the analysis result. For example, a character string indicating detected road information may be displayed superimposed on the video, or some of the hue, brightness, and saturation of the entire or part of the video may be adjusted. Since the image displayed in the image display area 141 changes, it is possible to know that there is a change in the road condition.

Furthermore, trigger information for event recording may be recorded in the header recording area 31 of the video file. For example, information on turn signal operation, sudden brake operation, wiper operation, etc. is recorded as trigger information for event recording. Furthermore, the user may sit in the passenger seat while the drive recorder 1 is recording, and the glasses-type display device may be attached to the user's head. The display screen of the display device displays driving caution information, and through the display screen, the scenery in front of the vehicle being driven (an example of the actual scenery seen from a moving car) can be seen. Driving caution information will be displayed overlaid on the scenery in front of the vehicle. Needless to say, the road the vehicle will travel on is determined in advance, and caution information for that road is displayed on the display device. Alternatively, a third party other than the user (for example, a driving instructor) may ride in the vehicle and select the caution information to be displayed on the display device, and the selected caution information may be displayed on the user's display device. .

In the above-described embodiment, the drive recorder 1 is capable of recording in all directions such as the front, side, and rear of the vehicle, but it does not necessarily have to be able to record in all directions.

Furthermore, the recording in the drive recorder 1 may be performed in addition to normal constant recording and event recording. For example, for event recording, an acceleration sensor and a trigger switch are used to record the previous and subsequent events. Keep an event record separate from the regular record. You may also record the 30 seconds and 30 seconds before and after the acceleration sensor responds. For example, a first event occurs and a second event occurs, for example, a case may be recorded in which the car collides with a card rail the first time, and then the car is swung and collides with the guard rail a second time. If the recording times overlap, it may be treated as one event. If there is a gap between two events, they may be recorded as separate events, or if they do not overlap, they may be processed as two events.

Further, the drive recorder 1 described above may be used as a camera for constructing a line in a factory, monitoring the line, and adjusting the line in order to properly run the factory line. When assembling with machine tools, cameras are temporarily installed in areas where problems are likely to occur in order to take images of the assembly. When starting up a new line at a factory, it would be a good idea to run it for about three months and verify any defects. You can record before and after an alert is issued by a machine tool in a factory, or you can capture images before and after an alert is issued in response to a trigger such as a signal light attached to a machine tool, green, yellow, or red. good. Video data obtained from such a camera may be displayed in the playback window 130 described above. Images may be taken only at events, or images may be taken all the time.

Even with spherical drive recorders (drive recorders that shoot in all directions) and drive recorders with wide-angle cameras, the range that humans can see is not the same as the camera's imaging range, and the range that humans can see depends on the age of the person and individual differences. The range changes. During senior citizen training courses and driver's license renewals, narrow viewing angles can be dangerous and cause drivers to run over bumps, so record video of them driving using drive recorder 1 and show that they are running over bumps. It is a good idea to take a class while doing so. By displaying the visible range of the bow seat, the driver is not denied, but can be taught that if he cannot see it, he should look carefully and turn his head so that he can see it. For example, it may be a good idea to display a frame so that you can see the visible range during playback.

When recording the driving situation (scenery, landscape) using the drive recorder 1, if the driver's eye movements are recorded at the same time and the visual field is shifted, a vector indicating where the driver is facing within the visual field, a circle may be displayed. It is a good idea to record where your line of sight was when you were driving, and when displaying it in the playback window 130, shift the displayed range or frame in the direction you are facing. Since the drive recorder 1 is capable of capturing images in all directions, the front, sides, and rear of the vehicle are all recorded, so the direction in which the driver is facing can be determined by image recognition. It is a good idea to move the display frame accordingly. It is preferable to display in the playback window 130 how much you were checking not only the present but also the present, past, and future on the time axis, and how much you were looking straight at the moment but not looking at the past. For example, you can fill in areas that you are not looking at. You can see areas that you haven't seen at all. You can see how far they are shaking their head, how far they are looking, and how far they are looking. The drive recorder 1 and a line of sight detection sensor may be connected and the line of sight taken by the sensor may be displayed. See where users aren't looking. Imaging time varies depending on whether the vehicle is driving normally, at high speeds, or at an intersection. Things like driving time at driving school, before and after stopping at intersections, etc. The range to be viewed at each point in the driving school is predetermined, and it is judged how well it matches that range. It would be nice if there were a standard, such as a certain percentage that you need to see to pass. Even in normal driving, you can judge how much you look when you stop at an intersection, or how much you look depending on your speed, such as from 40 km/h to 80 km/h, or over 80 km/h. At high speeds, I don't look to the side, but I do look in the rearview mirror, and when I'm driving around town, I have to pay attention to the sides as well. When driving slowly at speeds of 20 km/h or less, you must look 360 degrees and check your mirrors and blind spots. The viewing range changes based on the turn signals, brakes, and steering angle. If you want to turn to the right, you have to look relatively accurately to the right. By using the playback window 130, it can be seen that, for example, confirmation is being made before turning on the turn signal or turning the steering wheel. Furthermore, it is a good idea to show in the playback window 130 that an elderly person actually drives in the driving school during a training session for the elderly, turns on a drive recorder, and later shows the driver driving in this manner. A simulator is fine, but elderly people won't be convinced unless they actually drive the car, so you can get them to understand using actual videos. This has two effects: it helps the person understand, and it makes it easier for the instructor to understand. It would be a good idea to provide information for instructors as well. The areas not seen can be considered to be information for instructors. If you are not looking at the mirror, you can have a point of focus in advance, such as a mirror, and check whether you are looking at that instrument (speedometer, etc.). You can record the driving situation when the driver wears the drunk glasses on the drive recorder 1 and show the driving situation.

There are more than 10 mirrors on a bus, so in addition to the tachograph, you can also use visual evaluation to see if the driver is looking at all of them. By displaying your line of sight, you can evaluate what percentage of your time you are looking at while driving. You can evaluate before and after using the turn signal, and where you are looking and where you are not looking. Buses and commercial vehicles always have a routine, and there is a manual to check at the bus stop and before departure, and forklifts also check front, back, left, and right, so this is a normal routine for buses, trains, trucks, and forklifts. Therefore, it is possible to evaluate whether the driver is driving. You can also see the movements of passengers.

If you keep a history of your viewing conditions and line of sight during playback, you can use the moment when you turn on your turn signal as a trigger, and analyze what happened before and after that to see if you are looking properly into blind spots. When driving, this is different from normal line of sight analysis because you have to look at the right place at the right time. In the video, the car is moving, the trigger, the turn signal, and the place to look before turning the steering wheel is determined. Things like conditions change over time. I may have seen it too late. It is important that it is appropriate in relation to time. There is something to see. Show the user what to look at and provide a model guide to ensure the user is looking at the bottom. If you are approaching an intersection, look here. If you are turning, look here. Types of triggers, what to trigger, speed, blinkers, location of intersections, red lights, road signs, if it's a crosswalk, are you checking left and right? Are you looking behind you when you put it in reverse? Are you looking ahead? Don't back up. Check the mirror. I only see the rear monitor. Look left and right. Even though I'm looking forward, perhaps because I'm looking at something like that, the back doesn't look right. Despite looking behind me, the drive is strange. The driver's line of sight can be analyzed and the driver's direction can be determined from the video captured. It may also be possible to detect whether the eyes are open and display information on whether the eyes are open during playback. It may also be possible to display thumbnails of scenes of bad driving, similar to a movie film. For example, there are certain things that need to be done during certain actions, such as not looking at the mirror when changing lanes, not operating the turn signal, accelerator, etc., so we can identify areas that do not match. It may also be displayed as a thumbnail.

When grading driving, stopping points may be excluded from scoring. Scores are given based on whether the vehicle moved smoothly, stopped, and whether there was sudden steering or braking. The evaluation target is whether the building is moving in the video, the car in front is not moving, and where and how much it is moving. When the vehicle is stationary, it is not subject to evaluation. Evaluate without using the parts that are stopped. You may also be able to skip over the part where the vehicle is stopped and skip to the next part, or fast-forward the video. The video may advance to the point where the light changes from red to green. This can save time for graders when viewing videos. In addition to showing images of the outside, the system also displays images inside when a passenger gets on the train, and only when a passenger gets on the train. If you play it only when there is a change in the number of passengers, it becomes a special feature when the passenger rides. This is because in the case of a taxi, it is enough to recognize the customer's face.

Also, when searching for a bright day, you can find bright images, and if there is a Shinkansen, you can find images of Shinkansen and pick up images of Shinkansen. In this way, it's a good idea to pick up things from the video that fit your conditions from a general range of things. When I have a week's footage and am told something ambiguous, I start looking for it. When you want to show something in a presentation, you can find out what happened in that area. You can also search for something like this with lights in a supermarket parking lot. It is best to use the key to find out whether it is a specific object, such as this situation, such time, the state of the car being recorded, whether it was speeding, accelerating suddenly, or using its turn signal. After the red car overtakes the blue car,
It is best to search for images after dynamic movement, and search for the desired scene after these consecutive scenes in the video.

It is also possible to display the actual image at the top and the recognized image at the bottom, or to distinguish the car type from the image, or to show it as an outline. Due to the ambiguity of such scenes, candidates for desired video scenes may be presented and selected. Each image is tagged to say what happened here, and how long it was there. If the video has been stopped for more than 5 minutes, the video may be stopped, or the software that plays the video may be equipped with a voice recognition function.

It would be good to display where the driver was looking during playback. When it is determined that the user is looking away, a mark may be placed on the playback window 130, or the video may be turned red the moment the user looks away. This is because images are taken with a spherical camera, and you can see where you are looking. If you look away, the alarm may go off. If you close your eyes, the alarm may go off. You can skip and play only that part, or you can play only that part. Cameras other than wearable cameras, such as 360-degree cameras, can tell you the direction the user is looking, and can display that direction. It could also be a wearable camera. Instead of a wearable camera, the camera may be placed on the user's head or attached to the user using something like a headband, like a wearable sensor. There may be a sensor that can tell where the driver is facing. Acceleration sensors (cameras) may be attached to ears, or sensors (cameras) may be attached to women's hair clips. The sensor (camera) may be clipped to the brim of the hat. You may be able to tell which direction the object is facing by combining these sensors with a spherical image. Can be analyzed using spherical images. It may be possible to show the opposite side and inform that there is a frame area on the back side that indicates the viewing range. It may be possible to notify the user that a frame is being displayed, but that it is not currently visible. You can also make the outer frame a green frame, or make the visible area look clear and the rest gray. The clear area may be expanded by checking. You can see at a glance what you are not looking at. You may clear the area during times when there is no head movement. The visible and invisible ranges may be displayed only before and after turning the steering wheel or before and after using the turn signal. You may want to show the angle the camera is facing on the map below. Both the angle of the line of sight (line of sight) and the angle of the vehicle may be displayed. The angle of the image itself may also be displayed. You may want to show how much you can see by looking at the direction of the car and your line of sight. The current angle of view may be displayed, or the angle of view of the currently displayed area may be displayed.

The drive recorder of the omnidirectional camera may be capable of displaying VR (virtual reality). The images from the spherical camera capture the entire 360 degrees, so you can get images similar to VR on a flat surface. At this time, the reference direction may be based on the front camera side, or may be based on the shooting direction, north side, etc. It is necessary to determine how far to map the direction with the reference direction north and the viewing angle range of VR. In the case of drive recorder 1, the forward direction is the front. It may also be a playback device in which the direction in which the main body is moved can be seen. This range is the range that you can see, but you can show that someone 10 years younger can see more. It may also have a function that allows comparison. You can see that what you can see now will not be as visible 10 or 30 years from now. The instructor may also have a function to switch the angle range for each age group. It may be switched according to age. VR display may be performed based on the direction that the driver can see. Software for playing back video recorded by the drive recorder 1, which can be displayed on a VR device, may be provided. With a VR device, acceleration and other information can be displayed as an overlay, or as an OSD (on-screen device). You can also display things like acceleration in 3D. Although the image itself cannot be viewed in 3D, it can be shown semi-transparently in front of the visible surface of the sphere. A panel may be placed in an area where it is not needed. You can tell just by moving your head. Where the line of sight is going, it should be in the foreground, and overlays should be in the foreground. It may become darker depending on how long you look at it, or it may become darker and lighter depending on how often you look at it. If you turn sideways, you can see the image on the side, and if you turn up, you can see the image above. It is best to detect the facing direction using an acceleration sensor, geomagnetic sensor, or gyro. When you are looking straight ahead, the camera is fixed, so you can see the direction in front of you. The center of the ball should be in front. The camera is always facing forward. Just give the image data to the framework.

From the administrator's point of view, there is no point in watching the video when there is no change in the video while waiting at a traffic light or when the vehicle is stopped. It's good to have it.

A function may be provided to display the trajectory of the vehicle on the map and jump to a specified point on the trajectory. To find out which time an image is at a certain point on the map, the time is written corresponding to the point on the map, and when you move the cursor over it, the time will appear, allowing you to find the image at that time. A street view may also be displayed. It may also be possible to provide a function to switch between Street View video and video recorded on a drive recorder. There are past and present street views, so you can see what the scenery looked like a year ago. You can see footage from three years ago of the run. You can see footage of the building before and after it was built. You can also display it in a separate window. Even though the map itself is old, it would be nice to be able to switch to it. If the shooting date is included, the desired season's video may be displayed, such as if you want to see an autumn video or a spring video. You can display summer and winter images while commuting to work in the same area. You may also choose to display the same image as the one you are currently running. You may display something like this on a rainy day.

In the play list display area 172 of the playback window 130, information indicating that this video is a video moved from this area to this area may be displayed. For example, information may be displayed indicating that the video travels from Okazaki City to Toyota City. It can be displayed as text or as a reduced map. It is much easier to understand and easier to use. If you have city/town/village and area data, it is possible to display from ○○ city to XX city. If you use the location API (Application Programming Interface) to send the latitude and longitude to a search server, you will receive information that the location is in XX city, so all you have to do is output that information. You can automatically set the destination of the video file. If the videos are lined up, if you commute by car every day, and you don't care about the commuting videos, but you only want to see the travel videos, it will be easier to choose from here if you categorize them as commuting. . In the case of the current XX city, it would be a good idea to categorize the places where the starting and ending points are the same, and put the rare ones at the top. This is because if you have only been to a place once, there is a high possibility that you were traveling.

In the header recording area 31 of the video file, GPS data is stored periodically, frame by frame, every second, every 0.5 seconds, etc. In order to name the video file as a specific place name, for example "Higashiokazaki", the coordinates can be sent to the API of the location search server, and the information returned from the server can be used to create the video file name. GPS data can be sent to a server that stores map images, and maps can be retrieved and displayed from that server. If the personal computer 80 has map data and place name information, there is no need to send it to an external server. Event information may also be displayed. Data containing only events can be displayed in the playback window 130, and information can be displayed indicating that when passing through this path, an event is occurring in this scene and in this scene. It may also be possible to display on the map where the driver suddenly braked, etc. If it is a commute to and from work, you may want to display all the events where the event occurred and what changed. It is also possible to display a route, and then display it on the map, indicating that an event has occurred after passing through this route. For example, if you press a certain road on the map, National Route 248, the events that occurred while passing through National Route 248 will be displayed on the map for a certain period of time, for example, five years. You can see that places are centrally dangerous. All the logs will appear, and when you click on them, you can see the logs, heart rate, or you can categorize them by category, such as bicycles. If a minor incident occurs, the accelerator pedal may be pressed a lot here, or other locations where accidents are more likely to occur may be identified and aggregated and displayed. You can display commuting video files by categorizing them, and when you click on them, they can all be mapped on a map.

Further, some kind of driving evaluation may be performed for each fixed section, for example from this intersection to this intersection, and the evaluation may be automatically saved on the server. When you playback, other people's evaluation data is also accumulated for each section, so you can see how your driving is evaluated compared to other people's driving, and whether it is rough or safe driving. It would be nice if the ratings were displayed in window 130 during playback. Under the video, a crying face may appear near the bottom of the video depending on the evaluation, or scores may be displayed, or other people's average scores may be displayed.

Furthermore, images may be stored on the server for each fixed section, and the same locations may be compared. For example, you can compare a certain road, National Route 1, from here to here. Driving scores are given based on the acceleration sensor, speed, sensor, gyro, and how you press the brakes, but on the other hand, does that score really mean a red point or a passing score for the particular road? . Naturally, on a winding road, you will turn the steering wheel many times, so even if it is said that the car meanderes too much, it is not an appropriate evaluation. This is because the road is crooked. Instead of an absolute evaluation, you can give a relative evaluation or give the results of what other people have driven. You can display the video of the number one driver, you are number 30, and this is the picture of great driving. There is a question of whether it is okay to display the first-place video, but it would be better to provide a function that allows this. You can display a message asking if you can provide this, or if you want others to see it, or you can add an upload button. The camera (drive recorder 1) itself may have a communication function and may upload or transmit score data. You can check this when installing the playback software on the personal computer 80, you can do it all at once during installation, you can choose to upload each video file one by one, or you can check it the moment you try to upload it. You may also have a permission button appear. Video files may be uploaded after checking, but scores may be uploaded without checking. It may be a rule that a user is not associated with information about the user's presence. You can also delete date data etc. when uploading. The day of the week and time may be included, but it is better to remove information that identifies individuals. It's fine if you don't know who the data belongs to, just the location and time without the car model or personal name.

There is a low possibility of an accident occurring on empty roads such as farm roads, so I will shoot (record) them, but I will lower the resolution and shoot (record) in high resolution because there are bumps in the streets. It would be good to be able to specify each location. It would be good to be able to specify the range from here to here as low-resolution recording, and this area as high-resolution recording. You will be able to shoot for a long time. The location is specified using the playback window 130. The user specifies in advance a range from here to here. It is a good idea to circle the area on the map and set the playback window 130 to shoot (record) at low resolution when the area is within this area. Insert the memory card 25 with recording settings into the drive recorder 1. Then, it will move accordingly. If the drive recorder 1 has a liquid crystal display device, the settings can also be made on the drive recorder 1. You can also adjust the resolution here when monitoring parking. The resolution may be changed even while the vehicle is running. Because absolutely nothing will happen here. All you have to do is take detailed pictures (record) of the event. If you have all the data on the cloud (server) side, you can tell whether an event will occur or not occur at this location, so you can adjust the resolution on the cloud side. Areas where the probability is low are recorded at low resolution (frame rate), and areas where the probability is high are recorded at high resolution. Since it is an event switching, it is a good idea to switch to high resolution when an event occurs. It may be possible to record in high resolution at the travel destination and in low resolution while on the way to the travel destination. There is no need to record when driving on a highway. You may also record only when an event occurs (event recording). For example, if you go to Kyoto, the area within Kyoto will be recorded in high resolution, but the area away from Kyoto will not be recorded. It may be recorded at one frame per second or at a low frame rate. You may also set in advance the location where you want to upload videos to the video posting site. For example, if you go to a circuit, you can record the circuit in high quality, but not record while on the way. You may specify an area to be recorded at a high frame rate (high image quality) instead of an area to be recorded at a low frame rate (low image quality). Parking monitoring may record at a low frame rate when the vehicle is parked.

For example, when you go to Kyoto to take high-resolution photos, it's a good idea to record the times you go back and forth. If it doesn't record, it's meaningless as a drive recorder, and even if I go into Kyoto and try to set it up, I can't set it up because I'm driving. For example, you may want to record only the vicinity of Kiyomizu Temple in high resolution, but you will forget to set it, so this can be prevented. It is also a good idea to record in high resolution while driving on mountain passes. Expressways may be recorded in high image quality, and general roads may be recorded in low image quality. You can record the Metropolitan Expressway in high quality and not record normal expressways, or you can record when the speed exceeds 100 km/h or when acceleration is applied at a corner. To set the area, simply set a slide switch on the drive recorder 1 to select high resolution or low resolution.

The 760MHz ITS service system, altitude road information, started as an experimental service around 2016, and it would be great if this information could be recorded together with the information from the drive recorder 1. Information includes, for example, other vehicles in the vicinity of the vehicle (location, direction of movement, acceleration, etc.), traffic lights (how many meters ahead, time until red, etc.), road signs (how many meters ahead, speed limit, etc.). etc.), pedestrians (location, direction of movement, etc.), emergency vehicle approach notification, sudden brake warning, traffic jam warning, failed signal vehicle warning, sign notification, motorcycle approach notification, signal timing notification, construction information notification, weather warning, etc. be. ITS includes road-to-vehicle communication and vehicle-to-vehicle communication. Road-to-vehicle communication is communication between the road and cars, and vehicle-to-vehicle communication is communication between cars. In road-to-vehicle communication, information about the vehicle and information obtained from sensors of surrounding pedestrians are communicated. Vehicle-to-vehicle communication involves communicating information such as which direction the vehicle is going and information about the current accelerator pedal. For example, if data between the road and the vehicle can be received and saved together with the video, for example, if it is an LED traffic light, it will be blinking, and the frame rate of the camera will be synchronized with the frame where the blinking disappears. Then, the light from the traffic light may not appear in the image at all, and depending on the situation, it may remain dark for a long time. Even if the light from the traffic light is not visible, the state of the traffic light can be recorded as data using ITS information. If the information that the signal is blue is known from the ITS information, the information can be displayed as the signal is blue at the end of the playback window 130. Since information is beginning to be provided using a dedicated frequency, it can be received by a receiver, recorded along with the video of the drive recorder, and displayed in the playback window 130.

It is preferable to record ITS service information with the drive recorder 1 and display the information. The drive recorder 1 records information on the memory card 25, such as a car coming from next door, a bicycle coming next, crossing at a red light, crossing at a yellow light, not pressing the brake even though the light is yellow. It would be nice if you could. It can also detect information about other cars in the vicinity of your car (location, direction of movement, acceleration), how many meters away there is a traffic light, how many meters away it takes for it to turn red, the type of road sign, and pedestrians. Therefore, it is recommended that the drive recorder 1 receives and records this information. When an emergency vehicle approaches, the information is sent from vehicle to vehicle in a bucket brigade when the vehicle in front brakes suddenly, so the drive recorder 1 can record such information on the memory card 25. Good. Also, since you can know in advance how many cars ahead brake suddenly, it would be nice to be able to record that information as well. In the case of road-to-vehicle traffic, the information is transmitted from antennas near intersections. When the driver enters an area within about 5 meters from the antenna, the drive recorder 1 automatically receives information. Vehicles that can receive information from the road can also transmit their own information. When receiving information on the road or between vehicles, such as how many kilometers per hour you are going, whether you are going to the left next time based on the car navigation system, or whether you have just stepped on the brake, the information is received using a drive recorder. The drive recorder can also receive information from vehicles adjacent to each other as it uses the same frequency. It is best to only receive calls from the drive recorder without making calls. The ITS device is a transmitter/receiver, and the drive recorder 1 records communication between a car equipped with a drive recorder and a different car. Just intercept it. You can also put software in your drive recorder and send it to other cars when it is received.

When there is a building at an intersection and a car approaches from the blind spot, one car communicates with the other car to see how the car behaves and whether it applies its brakes to prevent an accident. good for In addition, cameras are installed at intersections to provide information on the presence of pedestrians. The system is designed to remind people to be careful when turning, so if there is a hidden car, it will be easier for managers to manage it if they can see that they are stepping on the brakes. It would be good to record whether someone did not press the brake even though someone was there. Even though the car's display shows information that the traffic light in front of it is red and will not turn green for another 10 seconds, it is pointed out that if you do not reduce your speed, your fuel efficiency will be poor. good.

In the playback window 130, information about the car to which the drive recorder 1 is installed is displayed at the bottom. For example, speed, turn signal operation status, etc. In addition to such information, it is a good idea to send information obtained from the ITS system elsewhere.

For example, you can tell how many meters there is a car in front of you on the right, so if there is a traffic light in front of you that indicates a car is there, you can use an overlay to display a speech bubble the moment you see that traffic light. It would be good to display the time until the signal. May be displayed as text. It may be displayed as text or icons. A speech bubble is an example of an icon.

Drive recorder footage is used as evidence that objectively shows the circumstances of the accident, so it is desirable that it cannot be altered. It is preferable to be able to detect tampering if it occurs. In addition, the video recorded by the drive recorder 1 is not a record of the scenery that the driver remembers, but is objectively recorded on the spot by the camera (front camera) included in the drive recorder 1. 11, footage of the use of the rear camera 12) has been recorded. This is because the installation location of the drive recorder 1 is different from the position of the driver's eyes, and the recording system of the drive recorder 1 is different from the human visual system. The differences between the drive recorder imaging system and the human eye imaging system include, of course, the angle of view, sensitivity, head movement, eye movement, optical system, image sensor, and structure of the eye. There are differences, but in particular, the cognitive structure of the brain is different, so we obtain the image that the driver remembers, and then add a function to the drive recorder 1 that can produce an image as close to that as possible. It is advisable to educate the driver or correct distortions in the driver's perception based on the difference from the objective image recorded by the recorder 1.

It would be good if the video recorded by the drive recorder 1 could be processed with colors, etc. so that the video is what you thought it was. It would be a good idea to process the video so that it is objectively perceived, so that it can display an image that is closer to the impression the driver was feeling.

Regarding the detection of tampering, the hash value of the data recorded when recording the video of the drive recorder 1 is obtained, the hash value is stored in relation to the video, and the P2P (peer-to-peer) connected to the network is used. It is best to store it in a block chain on the peer) network. Until now, all video data was stored on a server by one person, but with a data distributed database, all kinds of people, many people connected to the Internet, can share the same data. Each server will have . Blocks posted on the Internet are attached to the existing block chain, and the chain is rapidly created. The same data for that chain is recorded on the database on all servers. The feature is that since everyone has the same data, tampering can be detected unless a majority of people tamper with it.

When creating data, when connecting blocks, the hash value of the previous block is calculated and it is always inserted into the next block. It is not possible to tamper with a single block; in order to tamper with that block, you must tamper with all subsequent blocks, making tampering extremely difficult. With Bitcoin, the amount of calculation is set to take about 10 minutes to calculate the value of a block chain, and it would be impossible in terms of time to recalculate all of the blocks. The entire flow of a contract can be recorded on a blockchain, such as a contract for export/import procedures, the person who carries out the transportation within the other country, the person who makes the payment within the destination, and the person who exchanges currency. You can record all kinds of contracts, such as people, orders, etc., in a chain. For example, what month and day did you pass through here? It is also possible to record all the information that the goods were loaded on the ship in the chain so that it can be verified later. It may be used in areas where there are conflicts of interest and where there is a mechanism that allows for verification. In addition, an advertiser buys an ad space and enters into a contract with the company to run a number of 15-second commercials in one month between, say, 7 p.m. and prime time 7 p.m. and 8 p.m. If it is a transit advertisement, there is no way to verify where it is posted, and if it is on the Internet, it is impossible to verify whether it really received that many impressions. Although it is not possible to verify whether or not the advertisement was made, blockchain may be used in cases where an audit is conducted between the person who posted the advertisement and the person who actually publishes the advertisement. For example, if you place an advertisement on a block chain, you will be able to continue to connect the advertisements in the block by posting them in the block. In this way, the person who asked for 1,000 entries and the person who has issued 800 entries now can be verified on the blockchain.

Video data can be recorded on a block chain so that it cannot be tampered with. For example, when data is stored in a video frame by frame, such as a video based on motion JPEG (Joint Photographic Experts Group), the hash value of that frame is calculated and sent as a block, and the block chain system uses it. If someone else connects it to a chain, it will be impossible to tamper with it. The hash value of the entire video file for a certain range of one minute may be calculated.

In video data based on MPEG, if it is separated just before an I frame, it will be easier to tamper with it, so it is better to separate it just before a P frame somewhere.

In a public type block chain, like Bitcoin, anyone can participate in the block chain and can see all the block data flowing in the block chain, but in a consortium type, private For example, a type of block chain is closed within a banking association, and the block chain can only be used between banks that are members of the banking association. For example, you can use a public block chain system. If you send not only the hash value, but also the location information, information about the current location of the drive recorder, to the blockchain system, information about who is where in that block, month, day, etc. is sent. For example, when the information that a drive recorder named drive recorder 901-1 is located here in Okazaki City, Aichi Prefecture is sent to the block chain system, the server that manages all block chains It will be broadcast simultaneously. By sending video data etc. to a block chain system, if there is a user who wants it, it can be sold to that user.

Advertising can also be distributed in the same way. For example, you may want to send an advertisement for a store to users in a specific location, Okazaki City, Aichi Prefecture, and send the advertisement pinpointed only to people who are directly in front of that store. Block chains require costs to maintain the chain, but when a transaction is made a small amount is deducted as a fee and distributed to the user who manages the server and the user who first added the block. Information on the currency in which the payment was made can also be recorded on the blockchain. For example, if you take a photo and ask for it for 100 yen, the account of the person who asked you to take the photo for 100 yen will be debited, the user of this drive recorder will receive about 80 yen, and 20 yen It is also possible to create a system in which the money is distributed to the person who maintains the server.

It would be a good idea to be able to enlarge or reduce the image using a remote control while the image is being played. Furthermore, for example, the frame may be moved using a cursor or the like, and the position of the moved frame may be stored in association with the video time. You can move the frame during playback, record the position of the moved frame, and later create a video stream by extracting only that part of the video based on the recorded frame position information. Good.

It would be good if the user could manually move the viewing range without having it automatically detected. For example, when a frame is moved using a joystick, mouse, keyboard, etc., it is a good idea to memorize the moved position so that the moved frame can be displayed during playback. This allows other people to see the range that a person can see, although the range of vision varies depending on the driver's level of concentration, age, and eye disease. For example, you can claim that you could see this range or that you were looking at this range. Further, it is preferable to enable correction to subjectively perceived brightness and color, and correction to memory color. In particular, it is preferable to display objective images and subjective images so that the user can compare the images captured by the camera with the images that the user thinks he or she has actually seen. By displaying two types of footage of the same scene, you can explain that this is how the camera actually recorded it during the trial, but this is how it looked to you. Regarding criminal liability, one important factor is the occurrence of an objective accident or result, and the other is intentionality. It is important to prove whether the person hit the object intentionally or negligently. Objective images cannot be tampered with, but subjective images allow you to see what you thought separately, and to show both together.

Furthermore, a hash value of the location information and event information of the vehicle may be calculated and sent to the block chain system. This makes it difficult to falsify event information such as sudden deceleration or acceleration.

Furthermore, in order to adjust the brightness, color, and memory color to be subjectively recognized, the drive recorder 1 is provided with a brightness adjustment slider, and by operating the slider, the brightness, color, and memory color can be adjusted. Alternatively, the playback window 130 may be provided with a brightness adjustment button, a color adjustment button, and a memory color adjustment button, and the brightness may be adjusted by operating the buttons.

Furthermore, in automated driving of a vehicle such as a car, what is recorded by the drive recorder 1 during manual operation when the driver is driving the vehicle, or what is recorded by the drive recorder 1 during automatic operation when the vehicle is automatically operated. It is preferable to record data for identifying whether the video is being played in the header recording area 31 of the video file as described above. In this case, preferably, the level of automatic driving is also recorded in the header recording area 31 of the video file. If identification data indicating whether automatic driving or manual driving is in progress, and data representing the level of automatic driving if automatic driving is transmitted during the CAN (Controller Area Network) of the vehicle, drive recorder 1 transmits CAN data. By receiving data communicated within the drive recorder (for example, connecting the drive recorder 1 to the CAN or receiving data communicated within the CAN at the communication circuit 17), the drive recorder 1 receives the video file. In the header recording area 31, identification data indicating whether automatic operation or manual operation is in progress, and data representing the level of automatic operation if automatic operation is in progress can be recorded. When playing video data, it is possible to display in the playback window 130 whether the video displayed in the video display area 141 is during automatic operation or manual operation. For example, the level of automated driving can also be displayed.

The video data may be recorded in the drive recorder 1 by switching the image quality between automatic driving and manual driving. It may be possible to record relatively high image quality in the case of automatic driving and relatively low image quality in the case of manual driving, or conversely, record in relatively low image quality in the case of automatic driving, and record in relatively low image quality in the case of manual driving. When driving, it may be recorded with relatively high image quality. Furthermore, the image quality may be changed depending on the level of automatic driving. The higher the level of automatic driving, the higher the image quality may be for recording, or the higher the level of automatic driving, the lower the image quality may be for recording.

Furthermore, if the drive recorder 1 includes multiple cameras (front camera 11, rear camera 12), you can switch the camera to take pictures depending on whether automatic driving or manual driving is in progress. good. For example, during automatic driving, video data obtained only from the front camera 11 may be recorded, and during manual driving, video data obtained only from the rear camera 12 may be recorded, or vice versa. . Further, during playback, only automatic driving videos may be played and manual driving videos may be skipped, or vice versa, manual driving videos may be played and automatic driving videos may be skipped. Further, video data recorded at one of the automatic driving levels may be played back, and video data recorded at other levels may be skipped.

Furthermore, when playing back video data, the communication robot may provide guidance on driving techniques for the played back video. When an instructor directly explains to a user, the user may become angry, but when a communication robot says, ``That was dangerous,'' the psychological burden on both the instructor and the user is reduced. The communication robot is placed near the instructor and the user who are looking at the display device 92 of the personal computer 80 to be played. For example, it is preferable that the communication robot is always placed on the right side of the display device 92 of the personal computer 80, and the seating positions of the instructor and the user are also determined in advance. It may be possible to set whether the communication robot is placed on the right side or the left side of the display device 92 of the personal computer 80 using the video data reproduction software (program). Of course, the communication robot may be equipped with a camera, and the camera may take pictures of the 360-degree surroundings of the communication robot to recognize the positions of the display device 92, the instructor, the user, etc. in the surroundings. A person wearing a uniform may be recognized as an instructor, and other persons may be recognized as a user.

Furthermore, the width of the road included in the video represented by the video obtained by shooting with the drive recorder 1 or the video data obtained by recording with the drive recorder 1 is detected, and the processing of the video is changed based on the road width. You may also do so. For example, when the road is narrow, images from left and right may be displayed, or the type of image during playback may be changed depending on the road condition (narrow, gravel, one-way, etc.). Furthermore, if the drive recorder 1 is at an intersection and is capable of displaying an image during recording, it may display the image of the front, or may similarly display the image of the front during playback. Further, for example, a right or left image may be displayed on the display screen 3 of the drive recorder 1 so that the right or left side of the road can be seen.

Furthermore, when recording video data with the drive recorder 1, the video from slightly ahead of the vehicle to which the drive recorder is installed is analyzed to determine if a dangerous situation is occurring, and to determine if a dangerous situation is occurring. You may want to switch the video a few seconds in advance to a camera where the danger is easier to see. The display mode may be changed so that the currently displayed image shows danger in two seconds. For example, if a car crosses in front of you two seconds later, the display is in celestial mode (a mode that displays the front, back, left, and right sides of the vehicle), but you may want to switch to a mode that displays only the front. As a result, a dangerous scene can be seen after 2 seconds. After that, you can return to the original celestial sphere mode. In this way, the display format may be automatically changed if something strange may occur. It is also possible to automatically analyze all video files in advance, detect abnormalities, and mark the videos to be played. Whether or not something is dangerous can be determined based on whether there are large changes in the image or changes in acceleration. The display can be changed without the user operating a switching button. For example, it is preferable if the switching can be performed several seconds, for example, 5 seconds, before an event such as danger is likely to occur.

Furthermore, communication data from Enstar and Car Security may be received by the drive recorder 1, and event recording may be performed using the received data as a trigger. The drive recorder 1 is equipped with a communication circuit based on LTE (Long Term Evolution), a communication circuit based on Wi-Fi, and a communication circuit based on Bluetooth (registered trademark), and records data representing the communication status of each. The state of the video to be displayed may be changed depending on the communication state. The display change process may be a change in the display form of the video displayed on the display screen 3 of the drive recorder 1, or may be a change in the display form of the video in the playback window 130 during playback. For example, when receiving a command signal to turn on or turn off car security, the system may automatically switch to a video that captures the left and right directions of the vehicle and around the doors. Also, when a Wi-Fi signal is received, it is assumed that the device has entered the area of the house, so the video can be stopped. If there are multiple communication means, it is a good idea to decide which video to display depending on each communication means. The display mode may be switched depending on the communication means or the content of the communication. When using Bluetooth (registered trademark) to unlock a car door or start the engine, the state in which images from around the vehicle are displayed together with images inside the vehicle is displayed in response to the reception of a Bluetooth signal. You may also switch to shooting, or you may shoot not only the area around the vehicle but also the inside of the vehicle. This allows you to know the condition inside the car at the moment the engine starts. When entering, it takes about 30 seconds to return to the original state, and when the doors are unlocked, it maintains the state in which both the surroundings of the vehicle and the inside of the vehicle are photographed until the doors are locked, and then switches to only the surroundings of the vehicle or only the front of the vehicle, and then returns. You can do it like this.

Furthermore, roll calls are always held for operation managers, buses, taxis, and logistics personnel. For example, an alcohol check is required to check whether you are feeling well today. A video of the roll call will be recorded. The video of the roll call and the video of the drive recorder may be viewed together in the playback window 130. For example, by specifying a video file of a roll call, it is possible to reproduce the video recorded on the drive recorder 1 of the driver who was called. For example, if there is a roll call and passengers board the vehicle in turn, the roll call video and the video on the drive recorder 1 can be associated based on their temporal relationship.

When a drive recorder 1 is attached to a forklift and a drive recorder 1 is also attached to a car such as a truck, the images recorded on any of the drive recorders 1 are sent to the server and the same It would be a good idea to be able to display images of the vehicles (trucks, forklifts) driven by drivers in a unified manner as a continuous day's movements. For example, if you drive a truck to a certain location, drive a forklift at a certain location, and then get in your car and drive to a delivery destination, it would be a good idea to be able to see a series of videos of that happening. By combining this with the roll call video, you can see the entire flow of the driver's day.

Furthermore, driving instruction may be given using mixed reality. For example, a user drives once and a video is recorded on the drive recorder. Thereafter, the video is played back in the playback window 130, and the driving skill is automatically evaluated. During this automatic evaluation, information on precautions is added to the video data. Next, the driver wears the MR glasses, drives or sits in the passenger seat, and displays the information on the MR glasses. The user can see information on points to be noted superimposed on the actual scenery. For example, information about where to look at traffic lights and where to look at stop lines is displayed on the lenses of MR glasses, and this information can be confirmed while looking at the actual scenery.

Alternatively, the driving history data may be recorded in the memory 21 of the drive recorder 1, and when the user comes near a place visited in the past, it may be possible to display where the user passed to go and when. By referring to driving history data, a display or audio output indicating that the vehicle has traveled previously may be displayed.

Furthermore, the rear camera 12 of the drive recorder 1 may take a picture of the vehicle's meter and recognize it as a trigger to start recording video data. Additionally, if the interior lamp of the vehicle is photographed and it is detected that the interior lamp is on, event recording may be started and data indicating that the interior lamp is on may be sent to the server. . For example, the server may send data to the user of the vehicle indicating that the interior lamp is on. The user can see that the room lamp is on.

Drive recorders (cameras) may be attached to automated guided vehicles in factories. Automated guided vehicles are equipped with LIDAR (Light Detection and Ranging) sensors that can detect surrounding objects by scanning at high speed with a laser. Event recording may be started as a trigger (recording for a certain period of time, for example, from several tens of seconds to several minutes, depending on the trigger being given) when the approach of a surrounding object is detected.

It is also possible to record an event using image recognition as a trigger the moment the fork of a forklift moves. Photographs may be taken using the front camera 11, rear camera 12, etc. of the drive recorder 1, and when the forklift moves, this may be used as a trigger to record. Additionally, event recording may be started by setting a trigger when a person wearing a helmet rides on a forklift, or conversely, when a person without a helmet rides on a forklift. In that case, you can start recording when someone gets on and stop recording when someone gets off. The start and end of recording and playback may be determined by a person getting on and off the vehicle. A characteristic of forklifts is that although the forklift is owned by a certain company, the people who drive it are not only employees of that company, but also people from the transportation company. Since the person driving the car changes, you can start recording when such a person gets on the car, start recording when someone comes, or record until 3 minutes after getting off and then stop. Forklift accidents have become a problem, but it will now be possible to find out who caused the damage.

Furthermore, when you take a photo, Exif information is attached to it, and text data is embedded within the JPEG data. The exposure and shutter speed taken at that time are embedded. Such information may be displayed in the playback window 130 using playback software. A still image file may be generated from a moving image and recorded as Exif information in the header recording area of the still image file. For example, it is preferable to record the information displayed in the playback window 130 as header recording area Exif information. For example, a series of changes in acceleration in a video may be historically recorded in the header recording area of a still image file. GPS information and speed, the maximum value of acceleration when moving within a certain range, the speed of the car, whether the brake was pressed, whether the turn signal was turned on, etc. may be recorded as Exif information in the header recording area. You can record all speeds within a certain range, or all values for the types of events that occurred within plus or minus 3 seconds before and after a certain event occurred, if the drive recorder has multiple cameras. Additionally, information indicating which camera's video was recorded, how many cameras were used to record the image, information indicating where on the vehicle the camera was attached, etc. may be recorded as Exif information in the header recording area. Furthermore, if a drive recorder is installed on an autonomous vehicle in a factory, it will be necessary to check the cause of the trigger, whether the error light was red or yellow, the condition of the forklift, the amount of rotation of the forklift, the gyro, and the battery. The status may be recorded as Exif information in the header recording area. Information obtained in AWS (Automatic Guided Vehicle), CWS (Collision Warning System), stopper, vehicle-to-vehicle communication, road-to-vehicle communication, etc. may also be recorded as Exif information in the header recording area. There is a person, there is no person, there is a driver, there is no driver, there is a person in the passenger seat, there is no person, there is a person in the back seat, there is no person sitting, commercial vehicle, different vehicle, taxi, bus, bus Information such as whether there is a person inside or not may be used as a trigger, and this information may also be recorded as Exif information in the header recording area. Furthermore, the information recorded as Exif information may be sent to the block chain system and recorded on the block chain as a transaction.

Furthermore, points may be awarded to the user in response to the use of playback software (program) that displays the playback window 130. For example, by connecting to the Internet and pressing a button called "Import", imported data may be uploaded and points may be added. If the drive recorder 1 is a communication type, once permission is granted, video data can be automatically transmitted.

The scope of the present invention is not limited to the configurations explicitly described in the specification, but also includes combinations of various aspects of the invention disclosed herein. Of the present invention, the structure for which a patent is sought has been specified in the attached claims, but currently, even if the structure is not specified in the claims, it is not disclosed in this specification. The applicant intends to make such a configuration the scope of a patent claim in the future.

The present invention is not limited to the configuration described in the embodiments described above. The components of each of the embodiments and modifications described above may be arbitrarily selected and combined. Also, any component of each embodiment or modification, any component described in the means for solving the invention, or a component that embodies any component described in the means for solving the invention. It may be configured in any combination. The applicant intends to obtain rights to these matters through amendments to the application or divisional applications.

The scope of the present invention is not limited to the configurations explicitly described in the specification, but also includes combinations of various aspects of the invention disclosed herein. Of the present invention, the structure for which a patent is sought has been specified in the attached claims, but currently, even if the structure is not specified in the claims, it is not disclosed in this specification. The applicant intends to make such a configuration the scope of a patent claim in the future.
The present invention is not limited to the configuration described in the embodiments described above. The components of each of the embodiments and modifications described above may be arbitrarily selected and combined. Also, any component of each embodiment or modification, any component described in the means for solving the invention, or a component that embodies any component described in the means for solving the invention. It may be configured in any combination. The applicant intends to obtain rights to these matters through amendments to the application or divisional applications.
In addition, the applicant intends to acquire rights to the entire design or partial design by filing a conversion application to a design application. Although the drawing depicts the entire device using solid lines, the drawing includes not only the overall design but also a partial design that claims some parts of the device. For example, it is a drawing that not only includes some parts of the device as a partial design, but also includes some parts of the device as a partial design regardless of the components. The part of the device may be a part of the device or a part of the device. We intend to obtain rights not only for the entire design, but also for partial designs in which any part of the solid line part of the drawing is a broken line part.

1: Drive recorder, 2: Mounting section, 3: Display screen, 4: Camera section, 5: Front lens, 6: Rear lens, 7: Character string, 10: Controller, 11: Front camera, 12: Rear camera, 13: Display control device, 14: Display device, 15: Clock, 16: Road/vehicle distance/vehicle distance information receiving circuit, 17: Communication circuit, 18: Acceleration sensor, 20: Input device, 21: Memory, 22 : GPS receiver, 25: memory card, 31: header recording area, 32: video recording area, 80: computer, 81: display control device, 82: display device, 90: CPU, 92: display device, 93: acceleration Sensor, 95: Memory card reader/writer, 96: Input device, 97: Memory, 98: Communication circuit, 130: Playback window, 131: Menu bar, 132: Folder icon, 133: Camera icon, 134 : Printer icon, 135: Reel icon, 136: FD icon, 137: Log icon, 138: Gear icon, 141: Image display area, 141A: Image display area, 141B: Image display area, 142: Viewing range Outside area, 143: Frame, 143A: Frame, 144: Mark, 144A: Mark, 145: Mark, 146: Balloon, 146A: Balloon, 146B: Balloon, 147: Balloon, 148: Chain line, 151: Map image display area , 152: Icon, 161: Vehicle information display area, 162: Vehicle status display area, 163: Operation button, 166: Time display area, 168: Acceleration graph display area, 171: Play/list display area, 172: Play/ List display area, 280: Communication robot, 281: Body, 282: Head, 283: Eyes, 284: Mouth, 291: CPU, 292: Memory, 293: Communication LAN board, 294: Wireless communication circuit, 295: Motor control board, 296: Head motor, 297: Sensor board, 298: Eye camera, 299: Audio input/output board, 300: Speaker, 301: Microphone, 310: Windshield, 311: Dashboard

Claims (5)

Equipped with a function that stores the captured video data in a different folder depending on the judgment result whether automatic operation or manual operation is in progress .
A drive recorder equipped with a function that changes the image quality of stored video data depending on the determination result of whether automatic driving or manual driving is in progress. The drive recorder according to claim 1 , further comprising a function of recording at a higher image quality than the image quality during the automatic driving and the manual driving when a specified condition is satisfied. The drive recorder according to claim 1 or 2, further comprising a function of switching a camera for recording video data according to a determination result as to whether automatic driving or manual driving is in progress. Equipped with a function that stores the captured video data in a different folder depending on the judgment result whether automatic operation or manual operation is in progress .
A drive recorder with a function that switches the camera that records video data depending on the determination result of whether automatic driving or manual driving is in progress. A program for causing a computer to implement the functions of the drive recorder according to any one of claims 1 to 4 .

Images