JP2022157267A - Video transmission device, video transmission method, program, and recording medium - Google Patents

Abstract

To provide a video transmission device and a video transmission method capable of transmitting a video of the outside of a vehicle to an operator of an external terminal in an appropriate manner according to the situation while reducing the amount of communication during communication.SOLUTION: A video transmission device that moves with a moving object includes a video acquisition unit that sequentially acquires videos around the moving object, a visibility state determination unit that sequentially acquires visibility information regarding the degree of opening of the visibility in front of the moving object, and a video transmission unit that transmits the video at a frame rate according to the degree of opening.SELECTED DRAWING: Figure 1

Description

The present invention relates to a video transmission device, a video transmission method, a program, and a recording medium.

2. Description of the Related Art There is a communication system that performs communication between an in-vehicle device mounted on a vehicle and an external terminal located outside the vehicle. For example, in Patent Document 1, a voice call is performed between a driver of a vehicle and an operator of a terminal outside the vehicle. A system is disclosed in which an image is transmitted to a terminal outside the vehicle and the image is displayed on the terminal outside the vehicle.

JP 2016-213791 A

In a system such as that disclosed in Patent Document 1, for example, in order to avoid an excessive amount of communication during video transmission, video may be transmitted to an external terminal in a mode that minimizes the amount of communication. Even in such a case, it is preferable not to make the operator of the external terminal who is watching the video feel uncomfortable.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned points, and transmits an image of the outside of the vehicle to the operator of the terminal outside the vehicle in an appropriate manner according to the situation while reducing the amount of communication during communication. It is an object of the present invention to provide a video transmission device and a video transmission method capable of

A video transmission device according to the present invention is a video transmission device that moves together with a moving object, and includes a video acquisition unit that sequentially acquires an image around the moving object, and sequentially acquires field-of-view information regarding the degree of opening of the field of view in front of the moving object. and a video transmission unit for transmitting video at a frame rate corresponding to the degree of opening.

A video transmission method according to the present invention is a video transmission method executed by a video transmission device that moves with a moving object, and includes a video acquisition step of sequentially acquiring images around the moving object, and a degree of opening of the field of view in front of the moving object. and a video transmission step of transmitting video at a frame rate corresponding to the degree of opening.

A program according to the present invention is a program to be executed by a computer, and includes an image acquisition step for sequentially acquiring an image around a moving body, and a visibility state determination step for sequentially acquiring view information regarding the degree of opening of the field of vision in front of the moving body. and a video transmission step of transmitting video at a frame rate corresponding to the degree of opening.

A recording medium according to the present invention is a recording medium on which the above program is recorded.

1 is a perspective view showing the configuration of a video distribution system according to Example 1; FIG. FIG. 2 is a diagram showing the configuration of the front seat portion of the vehicle according to the first embodiment; 2 is a diagram illustrating an example of a configuration of a terminal device T according to Example 1; FIG. 1 is a block diagram showing an example of a configuration of a video transmission device according to Embodiment 1; FIG. 2 is a diagram illustrating an example of a configuration of a terminal device T according to Example 1; FIG. 2 is a block diagram illustrating an example of a configuration of a server according to Example 1; FIG. 2 is a block diagram illustrating an example of a configuration of a terminal device according to Example 1; FIG. 4 is a flow chart showing a control routine of the video transmission device according to the first embodiment; 4 is a flow chart showing a control routine of the video transmission device according to the first embodiment;

Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. In the drawings, the same components are denoted by the same reference numerals, and redundant descriptions of the components are omitted.

FIG. 1 is a diagram showing the configuration of a video distribution system 100 according to the first embodiment. As shown in FIG. 1, in a video distribution system 100, an in-vehicle device 10, a terminal device T, and a server S mounted on a vehicle M as a moving object communicate via a network NW, for example, TCP/IP or UDP. Data can be exchanged between them using a communication protocol such as /IP.

The network NW is, for example, a mobile communication network such as LTE (Long Term Evolution), 4G (4th Generation) or 5G (5th Generation: fifth generation mobile communication system), wireless communication such as Wi-Fi (registered trademark), and It can be built with Internet communication, including wired communication.

FIG. 2 is a perspective view showing the front seat portion of the vehicle M on which the in-vehicle device 10 according to the first embodiment is mounted. The in-vehicle device 10 is connected to each of an exterior camera 12, an in-vehicle camera 13, a GPS receiver 14, a microphone 15, a touch panel display 16, and a speaker 17 installed in the vehicle M, and a control unit for controlling them. including. The in-vehicle device 10 is arranged, for example, in the central portion of the dashboard DB in the front seats of the vehicle M. As shown in FIG.

The exterior photographing camera 12 is an imaging device that photographs the situation outside the vehicle M. As shown in FIG. In this embodiment, the exterior camera 12 is a wide-angle camera that captures a wide area in front of the vehicle M through the windshield FG. In this embodiment, the vehicle exterior photographing camera 12 is arranged on the dashboard DB.

The in-vehicle photographing camera 13 is an imaging device that photographs the situation inside the vehicle M. As shown in FIG. In this embodiment, the in-vehicle photographing camera 13 is a camera that photographs the driver of the vehicle M. As shown in FIG. In this embodiment, the in-vehicle camera 13 is provided on the upper end of the windshield FG or on the ceiling near the upper end.

The GPS receiver 14 is a receiver that receives signals (GPS signals) from GPS (Global Positioning System) satellites. In this embodiment, the GPS receiver 14 is arranged on the dashboard DB.

The microphone 15 is a voice input device that receives sounds in the vehicle, such as voices uttered by the driver, and converts them into electrical signals. In this embodiment, the microphone 15 is arranged on the dashboard DB.

The touch panel display 16 is a display device in which a display that displays a screen based on the control of the in-vehicle device 10 and a touch panel that receives an input operation from a passenger (for example, a driver) of the vehicle M are combined. In this embodiment, the touch panel display 16 is arranged in the center of the dashboard DB.

The touch panel display 16 can display, for example, a car navigation image in which the current position of the vehicle M and the planned travel route are superimposed on a map. Also, operations related to car navigation functions, such as setting a destination via the touch panel display 16, may be possible.

The speaker 17 is an audio output device that outputs audio based on electrical signals transmitted from the outside. In this embodiment, the speaker 17 is provided on each of the two A-pillars AP.

In this embodiment, the driver of the vehicle M can make a voice call with the operator of the terminal device T via the microphone 15 and speaker 17 connected to the in-vehicle device 10 .

The positions of the in-vehicle device 10, the camera outside the vehicle 12, the camera inside the vehicle 13, the GPS receiver 14, the microphone 15, the touch panel display 16, and the speaker 17 in the front seat portion of the vehicle M are merely examples, and these position.

For example, the exterior photographing camera 12 may be installed anywhere as long as it can photograph the situation in front of the vehicle M, and may be installed on the upper end of the windshield FG or on the ceiling near the upper end.

In this embodiment, the in-vehicle device 10 sequentially transmits to the terminal device T via the server S image information indicating the image of the outside (front) of the vehicle M captured by the exterior camera 12 .

FIG. 3 is a diagram showing the configuration of the terminal device T and the display mode of video and images displayed on the terminal device T. As shown in FIG. FIG. 3 shows a case where the terminal device T displays a landscape image in which no object is approaching on the side of the road and the front view of the vehicle M is open. The terminal device T is a communication device including a frame F and a microphone 21, a speaker 22, and a touch panel display 23 housed in the frame F. FIG. In this embodiment, the terminal device T is a smart phone capable of communicating with others through communication.

The microphone 21 is a voice input device that receives voice uttered by the user of the terminal device T and converts it into an electric signal. The microphone 21 is provided on one end side of the terminal device T. As shown in FIG.

The speaker 22 is an audio output device that outputs audio based on electrical signals transmitted from the outside. The speaker 22 is provided on the other end side of the terminal device T. As shown in FIG.

The touch panel display 23 is a display device in which a display that displays a screen based on the control of the terminal device T and a touch panel that accepts an input operation from the user of the terminal device T are combined. The touch panel display 23 is provided in the center of the terminal device T. As shown in FIG.

In this embodiment, the operator of the terminal device T can make a voice call with the driver of the vehicle M in which the in-vehicle device 10 is mounted via the microphone 21 and the speaker 22 described above.

In the present embodiment, the terminal device T sequentially receives, via the server S, the image information indicating the image of the exterior of the vehicle M, which is sequentially transmitted from the in-vehicle device 10 of the vehicle M. FIG. Specifically, the terminal device T sequentially receives, via the server S, the image information indicating the image in front of the vehicle M captured by the external camera 12 .

As described above, in the present embodiment, the video distribution system 100 is configured so that the driver of the vehicle M in which the in-vehicle device 10 is mounted and the operator of the terminal device T can communicate with each other by voice. Further, in this embodiment, the image delivery system 100 is configured so that the image of the exterior of the vehicle M captured by the exterior camera 12 can be displayed on the touch panel display 23 of the terminal device T in real time. In other words, the video distribution system 100 is configured such that the in-vehicle device 10 can perform live streaming to the terminal device T. FIG.

In this way, while establishing voice communication between the in-vehicle device 10 and the terminal device T, the in-vehicle device 10 transmits to the terminal device T the image such as the image outside the vehicle M, the current position of the vehicle M, and the moving route. is called video communication in real time. In this embodiment, the in-vehicle device 10 and the terminal device T perform video communication via the server S. FIG.

By performing such video communication, the user of the terminal device T viewing the video transmitted from the in-vehicle device 10 feels as if the driver of the vehicle M and the vehicle M are riding together. be able to. In other words, video communication enables the user of the terminal device T to virtually ride in the vehicle M. A system configured by such video communication is called a virtual co-riding system.

The configuration of each of the in-vehicle device 10, the server S, and the terminal device T that constitute the video distribution system 100 will be described below with reference to FIGS. 4 to 7. FIG.

FIG. 4 is a block diagram showing the configuration of the in-vehicle device 10. As shown in FIG. The control unit 25 is a processing device including a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). In this embodiment, the control unit 25 functions as a computer, and the control unit 25 exhibits a video transmission function during video communication.

The encoding unit 26 is a part that encodes the video (also referred to as captured video) captured by the external camera 12 or the internal camera 13 based on a command from the control unit 25 . The encoding unit 26 has a CPU for video encoding, a so-called GPU, and encoding may be performed by the GPU.

The encoding unit 26 encodes the picked-up video in accordance with, for example, the MPEG-4 standard encoding method to generate encoded data as video data. For example, the encoding unit 26 may use the H.264 standard. H.264, Xvid, DivX, VP8, VP9, or other codec is used to generate encoded data from captured video.

The communication unit 27 is a communication device that transmits and receives data to and from an external device according to instructions from the control unit 25 . The communication unit 27 is, for example, a NIC (Network Interface Card) for connecting to the network NW.

The communication unit 27 is connected to the network NW described above, and transmits and receives various data between the in-vehicle device 10 and the terminal device T. FIG. In this embodiment, the control unit 25, the encoding unit, and the communication unit 27 may be, for example, a video transmission unit that transmits the encoded data of the captured video generated by the encoding unit 26 to the terminal device T via the server S.

The storage unit 28 is a storage device that stores and manages data required for processing by the control unit 25 . The storage unit 28 is, for example, a storage device such as a hard disk, flash memory, or SSD (Solid State Drive). The storage unit 28 stores, for example, the contact information of the terminal device T as the other party of the video communication. The storage unit 28 also stores, for example, map data to be displayed on the touch panel display 16 described above.

The functional blocks of the control unit 25 will be described below.

The video acquisition unit 31 is a part that acquires a captured video imaged by the above-described exterior camera 12 or interior camera 13 . The control unit 25 encodes the captured image acquired by the image acquisition unit 31 via the encoding unit 26 and transmits the encoded captured image to the server S via the communication unit 27 .

The image recognition unit 32 is a part that performs image recognition on the image representing the captured image acquired by the image acquisition unit 31 and acquires information indicating the recognition result of the object in the forward field of view of the vehicle M. For example, the image recognition unit 32 detects the contour of an object to be recognized from an image, separates the object from other objects, etc., and compares the object with an image as reference data to determine what the object is. to recognize The image as the reference data is stored in the storage unit 28, for example.

In this embodiment, the image recognition unit 32 recognizes an object that can affect the degree of opening of the field of view in front of the vehicle M in the above-described captured image. Specifically, the image recognition unit 32 recognizes an object present in front of the vehicle M along the side of the road on which the vehicle M travels.

More specifically, the image recognition unit 32 recognizes an object (hereinafter referred to as a continuous object ) are recognized. In addition, the image recognition unit 32 recognizes objects existing at intervals along the side of the road (hereinafter referred to as discontinuous objects), such as a plurality of guide poles for separating lanes, trees, and agricultural products. ) are recognized.

The visibility state determination unit 33 is a part that determines the degree of opening of the visibility in front of the vehicle M based on the image recognition result obtained by the image recognition unit 32 . That is, the visibility state determination unit 33 is a visibility information acquisition unit that acquires visibility information regarding the degree of opening of the field of vision in front of the vehicle M based on the image recognition result.

For example, the visibility state determination unit 33 acquires visibility information by determining the degree of visibility in front of the vehicle M based on the presence or absence of an object and the position of the object recognized by the image recognition unit 32 .

For example, when the visibility state determination unit 33 acquires an image recognition result indicating that the above-described continuous object is positioned along the side of the road on which the vehicle M travels, the degree of opening of the visibility in front of the vehicle M is small, that is, the field of view in front of the vehicle M is not open. In addition, when the visibility state determination unit 33 acquires an image recognition result indicating that nothing is positioned on the side of the road on which the vehicle M travels or that there are almost no continuous or discontinuous objects, is large, that is, the field of view in front of the vehicle M is determined to be open.

The control unit 25 changes the encoding mode of the captured image by the encoding unit 26 based on the degree of opening of the field of view in front of the vehicle M determined by the visibility state determination unit 33 .

Here, a mode of encoding performed by the encoding unit 26 in this embodiment when transmitting a captured image will be described. Hereinafter, the camera 12 for shooting outside the vehicle always shoots at a frame rate sufficiently higher than the frame rate of the video data after the encoding process described below, such as 27.5 fps. Description will be made on the assumption that encoded data is generated at variable frame rates by dropping.

The encoding unit 26 encodes the captured video in a plurality of different encoding modes based on instructions from the control unit 25 . Specifically, the encoding unit 26 has a normal mode as a first encoding mode and a high frame rate mode as a second encoding mode as operation modes during encoding.

The normal mode as the first encoding mode is an encoding mode that encodes captured video with a frame rate lower than that of the second encoding mode. In this embodiment, the normal mode is applied, for example, when the forward visibility of the vehicle M is open.

A high frame rate mode as a second encoding mode is an encoding mode in which a captured image is encoded in a mode in which the frame rate is higher than that in the first encoding mode. In this embodiment, the high frame rate mode is applied, for example, when the front view of the vehicle M is not open.

For example, when the front of the vehicle M is open, the encoding unit 26 encodes the captured video at a frame rate of 10 fps as a normal mode. Then, for example, when the vehicle M is traveling on a road that is not open in front of the vehicle M, the encoding unit 26 changes the encoding mode from the normal mode to the high frame rate mode, and images at a frame rate of 24 fps. Encode the video.

In this embodiment, when video communication is established between the in-vehicle device 10 and the terminal device T, the image of the exterior of the vehicle M transmitted from the in-vehicle device 10 is displayed along the road on which the vehicle M is traveling. The encoding mode is automatically changed according to the degree of opening of the field of view determined based on the presence or absence of the positioned object.

FIG. 5 is a diagram showing the terminal device T displaying an image in front of the vehicle M different from that in FIG. 3 in video communication between the in-vehicle device 10 and the terminal device T. FIG. In FIG. 5, the touch panel display 23 of the terminal device T displays an image of the front of the vehicle M, which is a road along which a concrete wall CW is provided along the side. That is, in FIG. 5, in the field of view in front of the vehicle M, there is the above-described continuous object along the side of the road on which the vehicle M is traveling.

In the encoding unit 26 of the in-vehicle device 10, for example, when all the images outside the vehicle M are encoded in a normal mode mode with a low frame rate and transmitted, the operator of the terminal device T that received the image, that is, the viewing A person may feel uncomfortable depending on the degree of openness of the field of vision in front of the vehicle M.

Specifically, for example, as shown in FIG. 5, when a continuous object such as a concrete wall CW is provided along the side of the road on which the vehicle M is traveling, the operator of the terminal device T When viewing video encoded in the normal mode mode with a low frame rate, the user may feel that the speed sensation visually obtained from the video is faster than the actual speed of the vehicle M, the so-called perceived speed. be.

In particular, when the height of an object located along the side of the road on which the vehicle M is traveling is higher than the line of sight of the camera, the operator of the terminal device T feels that the perceived speed is fast. Also, even if the height of the object is not higher than the line of sight of the camera, if the vehicle M is traveling on a narrow road such as a one-way road and the distance to the object is short, the terminal device T , the operator may feel that the bodily sensation speed is fast. There is a possibility that the operator of the terminal device T who is viewing the video may feel uncomfortable due to the speed of the sensible speed.

In addition, due to the difference between the speed of the vehicle M or the actual perceived speed of the driver of the vehicle M and the perceived speed of the operator of the terminal device T, the image There is a risk that the sense of sharedness of people will fade away. Moreover, for example, the operator of the terminal device T may be frightened.

According to this embodiment, the encoding unit 26 encodes video at a low frame rate in the normal mode in order to reduce the amount of communication during video communication, and at a high frame rate when the field of view in front of the vehicle M is not open. Encode video in frame rate mode. As a result, the in-vehicle device 10 can transmit the image of the exterior of the vehicle M to the terminal device T in an appropriate manner according to the situation while reducing the amount of communication during communication.

Therefore, the operator of the terminal device T that has received the image of the exterior of the vehicle M can view the image without discomfort. Therefore, the driver of the vehicle M and the operator of the terminal device T can share a sense of sharing of images, and information can be shared smoothly.

FIG. 6 is a block diagram showing the configuration of the server S. As shown in FIG. The server S has a function like a SIP server that establishes a voice call between the in-vehicle device 10 and the terminal device T and transfers data of the voice call during video communication.

The control unit 35 is a processing device including a CPU, ROM, and RAM. In this embodiment, the control unit 35 transfers the captured image transmitted from the in-vehicle device 10 to the terminal device T during video communication.

The communication unit 36 is a communication device that transmits and receives data to and from an external device according to instructions from the control unit 25 . The communication unit 36 is, for example, a NIC for connecting to the network NW. In this embodiment, the communication unit 36 may be a receiving unit that receives the encoded image of the outside of the vehicle M from the in-vehicle device 10 . Also, the communication unit 36 may be a transmission unit that transmits the encoded image of the outside of the vehicle M to the terminal device T. FIG.

FIG. 7 is a block diagram showing the configuration of the terminal device T. As shown in FIG. The control unit 37 is a processing device including a CPU, ROM, and RAM.

The communication unit 38 is a communication device that transmits and receives data to and from an external device according to instructions from the control unit 37 . The communication unit 38 is, for example, a NIC for connecting to the network NW. In this embodiment, the communication unit 38 may be a receiving unit that receives encoded images of the outside of the vehicle M transmitted from the server S. FIG.

The decoding unit 39 is a part that decodes, reproduces, and outputs the encoded image of the outside of the vehicle M received from the in-vehicle device 10 via the server S based on the command from the control unit 37 .

The decoding unit 39 decodes the encoded data by the codec used for encoding the encoded data, for example, the MPEG-4 standard encoding method, and reproduces and outputs the video. The reproduced video is displayed on the touch panel display 23 by the control unit 37 .

In this way, the terminal device T receives and decodes the video (encoding data) outside the vehicle M transmitted from the in-vehicle device 10 in the video communication, thereby displaying the video on the touch panel display 23. be able to. In addition, the terminal device T can display video with different frame rates according to the degree of opening of the field of view in front of the vehicle M by using a plurality of different encoding modes during decoding of the in-vehicle device 10 described above.

A specific operation of the in-vehicle device 10 in this embodiment will be described below.

FIG. 8 is a flowchart showing a video distribution routine RT1 executed by the control unit 25 of the in-vehicle device 10. As shown in FIG. The control unit 25 starts the video distribution routine RT1 using, for example, establishment of a connection for video communication (hereinafter referred to as video connection) between the in-vehicle device 10 and the terminal device T via the server S as a start trigger. to start.

Specifically, for example, when the touch panel display 16 of the in-vehicle device 10 is turned on by the driver of the vehicle M, a video call acceptance screen is displayed on the touch panel display 16 . A video connection is established between the in-vehicle device 10 and the terminal device T by selecting the terminal device T as the destination of the video connection on the screen. Thereby, the control unit 25 starts the video distribution routine RT1.

The control unit 25 judges whether video communication has been started by establishing a video connection (step S101). When the control unit 25 determines that the video communication has not started (step S101: NO), it ends the video distribution routine RT1.

When the control unit 25 determines that video communication has started (step S101: YES), the control unit 25 acquires a captured image, encodes the image, and starts a video distribution operation for transmitting the encoded image to the terminal device T via the server S ( step S102).

Specifically, in step S102, the image acquisition unit 31 of the control unit 25 acquires the imaged image of the exterior of the vehicle M from the exterior camera 12, and the image recognition unit 32 of the control unit 25 indicates the imaged image. An image is processed to recognize objects within the image. Then, the visibility state determination unit 33 acquires the recognition result of the object in the image showing the captured video by the image recognition unit 32, that is, the visibility information. The control unit 25 instructs the encoding unit 26 and the communication unit 27 to transmit the encoded data obtained by encoding the captured image to the terminal device T via the server S.

After step S102, the control unit 25 determines whether or not the video communication has ended (step S103). When the control unit 25 determines that the video communication has ended (step S103: YES), it ends the video distribution operation (step S104). In other words, when the control unit 25 determines that the video communication with the external device 70 has ended, it ends acquisition of the captured image, image recognition, encoding and transmission of the captured image.

If the control unit 25 determines that the video communication has not ended (step S103: NO), it repeats step S103.

The control unit 25 of the in-vehicle device 10 encodes the video outside the vehicle M and transmits it to the terminal device via the server S as long as the video communication between the in-vehicle device 10 and the terminal device T continues according to the above-described video distribution routine RT1. Continue sending to T.

FIG. 9 is a flow chart showing a frame rate control routine RT2 executed by the controller 25 of the in-vehicle device 10. As shown in FIG. The control unit 25 starts the frame rate control routine RT2 using, for example, the start of the above-described video distribution operation in the in-vehicle device 10 by video communication as a start trigger.

First, the control unit 25 determines whether or not the field of view in front of the vehicle M is open based on the presence or absence of the continuous object (or discontinuous object) as the recognition result of the image in front of the vehicle M ( step S201). When the controller 25 determines that the front field of view of the vehicle M is open (step S201: YES), the frame rate control routine RT2 is terminated.

When the control unit 25 determines that the field of view in front of the vehicle M is not open (step S201: NO), it changes the encoding mode of the encoding unit 26 to the high frame rate mode (step S202). Specifically, the control unit 25 changes the encoding mode of the encoding unit 26 to a high frame rate mode, which is a mode in which the frame rate of the picked-up video during encoding is higher than in the normal mode.

After step S202, the control unit 25 determines whether or not the field of vision ahead of the vehicle M has opened (step S203). When the controller 25 determines that the field of vision in front of the vehicle M is wide (step S203: YES), it changes the encoding mode of the encoder 26 to the normal mode (step S204).

If the control unit 25 determines that the field of view in front of the vehicle M is not open (step S203: NO), it repeats step S203.

In the frame rate control routine RT2, the control unit 25 controls the encoding during encoding by the encoding unit 26 when changing from the normal mode to the high frame rate mode and when changing from the high frame rate mode to the normal mode. A mode change signal may be transmitted to the terminal device T to notify that the mode has been changed.

According to the present embodiment, as described above, during the video communication between the in-vehicle device 10 and the terminal device T, the image of the outside of the vehicle M transmitted from the in-vehicle device 10 has a wide field of view in front of the vehicle M. When it is determined that there is no such encoding, the encoding mode is automatically changed.

Therefore, according to the present embodiment, the in-vehicle device 10 can transmit an image of the outside of the vehicle M in an appropriate manner according to the situation while reducing the amount of communication during communication. Therefore, according to the present embodiment, it is possible to smoothly share information between the driver of the vehicle M and the operator of the terminal device T while reducing the amount of communication during communication.

In this embodiment, the visibility state determination unit 33 of the control unit 25 determines the degree of opening of the visibility based on the presence or absence of a continuous object (or discontinuous object) based on image recognition as visibility information. The method for determining the opening degree of is not limited to this.

For example, the vehicle M may be equipped with a distance sensor such as LiDAR (Light Detection And Ranging) or a millimeter wave radar. may be measured. Further, the control unit 25 of the in-vehicle device 10 may function as a distance information acquiring unit that acquires the distance between the vehicle M and an object positioned from the side to the front of the vehicle M measured by the distance sensor.

As a result, the visibility state determination unit 33 acquires, as visibility information, image data representing an object positioned from the side to the front of the vehicle M represented by the point cloud data based on the distance described above. (or a discontinuous object) determines the degree of opening of the field of view. For example, if there is a continuous object (or a discontinuous object) in the forward field of view of the vehicle M, the field of view determination unit 33 determines that the field of view is not open.

Therefore, the control unit 25 uses the distance sensor to determine the visibility in front of the vehicle M based on the distance between the vehicle M and a continuous object (or discontinuous object) positioned along the side of the road on which the vehicle M travels. The degree of opening can be determined, and the mode of encoding can be changed according to the degree of opening of the field of view.

As another method for determining the degree of opening of the field of view, the current position of the vehicle M and map information may be used to determine the degree of opening of the field of view.

For example, the control unit 25 may function as a position information acquisition unit that acquires the current position of the vehicle M via the GPS receiver 14, and the map data stored in the storage unit 28 includes areas on the map. Attribute information may be provided for each. Specifically, the map data may include information indicating attributes such as mountains, dense residential areas, coastal areas, industrial areas, rural areas, etc. as the environment of the surrounding area for each link or node. . The visibility state determination unit 33 acquires the current position of the vehicle M on the map and the attribute information described above as visibility information.

As a result, the visibility state determination unit 33 determines, based on the current position of the vehicle M on the map and the attribute information described above, when the attribute information of the area including the current position of the vehicle M is specific attribute information. If the attribute information indicates that there may be a continuous object (or a discontinuous object) on the side of the road such as a residential area, it is determined that the field of view is not open.

Therefore, the control unit 25 determines the degree of opening of the field of view in front of the vehicle M based on the attribute information of the area including the current position of the vehicle M on the map, and changes the encoding mode according to the degree of opening of the field of view. can be made

Also, the degree of opening of the field of view is obtained by inputting video data of the front of the vehicle M captured by the camera 12 outside the vehicle to an AI having a learning model for determining the degree of opening of the field of view from the video data, and obtaining its output. may be determined by

In the first embodiment, the control unit 25 of the in-vehicle device 10 changes the encoding mode of the encoding unit 26 when the field of view in front of the vehicle M is not open, that is, changes the frame rate in two steps. Although an example has been described, this aspect may be modified in three or more steps. That is, there may be three or more encoding modes for changing the video frame rate.

Contrary to the above description, the field of view in front of the vehicle M may be widened in the normal mode depending on the shooting environment, shooting target, type or specification of the shooting equipment in the vehicle M, or depending on the type or specification of the terminal device T. In this case, the operator of the terminal device T may feel that the perceived speed is faster than the speed of the vehicle M or the actual perceived speed of the driver of the vehicle M. In such a case, the encoding modes described above are reversed.

For example, the control unit 25 of the in-vehicle device 10 performs encoding in the normal mode when the field of view in front of the vehicle M is not open, and when the field of view in front of the vehicle M is open, the frame rate is higher than in the normal mode. You may change to the high frame rate mode which is the aspect which raises a rate.

In this embodiment, the resolution of the video during encoding by the encoding unit 26 may be changed according to the encoding mode described above. For example, the control unit 25 may reduce the resolution of the video when encoding in the high frame rate mode when the field of view in front of the vehicle M is not open. As a result, for example, it is possible to suppress an increase in the amount of communication due to an increase in the video frame rate due to a change from the normal mode to the high frame rate mode.

In the present embodiment, the video communication between the in-vehicle device 10 and the terminal device T via the server S has been described. ) directly, such as by communication.

In this embodiment, the case where the in-vehicle device 10 is connected to the touch panel display 16 has been described, but the present invention is not limited to this. For example, the in-vehicle device 10 may communicate with a smart phone carried by the driver of the vehicle M and display a screen related to video communication on the smart phone display instead of the touch panel display 16 .

Further, the in-vehicle device 10 may be configured not to display the screen presented to the driver of the vehicle M. FIG. For example, the in-vehicle device 10 may have a configuration like a drive recorder, and may be a device that is integrated with the exterior photographing camera 12, for example. Specifically, the in-vehicle device 10 may be, for example, a device in which hardware that performs the above-described functions of the in-vehicle device 10 is built in the housing of the camera 12 outside the vehicle. In this case, the in-vehicle device 10 may not perform various display outputs as described above.

Furthermore, the video transmission device of the present application may have a configuration in which a terminal device (for example, FIG. 4) having the same configuration as the in-vehicle device 10 in the present embodiment, the camera outside the vehicle 12, and the touch panel display 16 are integrated. good.

In the present embodiment, a case has been described in which an image outside the vehicle M is transmitted from the in-vehicle device 10 to the terminal device T. You may switch to the image|video inside the vehicle M image|photographed by the in-vehicle imaging|photography camera 13. FIG. When the video of the inside of the vehicle M is being transmitted, the user of the terminal device T can communicate with the driver of the vehicle M while viewing the inside of the vehicle M, for example.

Note that the switching operation for switching between the image of the vehicle exterior camera 12 and the vehicle interior camera 13 to be transmitted to the terminal device T may be performed in the in-vehicle device 10 . Also, the switching operation may be performed remotely by the user of the terminal device T operating the terminal device T. FIG.

In this embodiment, the control unit 37 of the terminal device T receives the image of the exterior of the vehicle M transmitted from the in-vehicle device 10 via the server S, but in addition to the image, the current location of the vehicle M and A map image showing the planned travel route, the name of the driver of the vehicle M, the traveling speed of the vehicle M, and the like may be received.

For example, the control unit 37 of the terminal device T causes the touch panel display 23 to display the map image received from the in-vehicle device 10, the name of the driver of the vehicle M, and the traveling speed of the vehicle M on an image outside the vehicle M. Alternatively, the image may be displayed in a display area different from the image outside the vehicle M. At this time, the map image, the name of the driver of the vehicle M, and the traveling speed of the vehicle M may be freely switched between display and non-display by the operator of the terminal device T. FIG.

In the present embodiment, an example in which the in-vehicle device 10 is mounted on the vehicle M has been described, but the in-vehicle device 10 may be mounted on other mobile objects such as bicycles and motorcycles. Alternatively, a person may hold the in-vehicle device 10 and, for example, perform video communication while walking to distribute images.

Although the case where the terminal device T is a smart phone has been described in this embodiment, it is not limited to this. For example, the terminal device T may be a tablet terminal capable of video communication with the in-vehicle device 10 .

In this embodiment, the case where the control unit 25 of the in-vehicle device 10 starts the video distribution operation after the video communication between the in-vehicle device 10 and the terminal device T is established has been described as an example. However, the control unit 25 may distribute moving images in a form such as YouTube (registered trademark) or niconico live broadcasting (registered trademark). That is, the video distribution operation may be started even if communication with the operator's terminal of the terminal device T is not established. Specifically, uploading of video data to the server S by the in-vehicle device 10 may be started even if communication between the in-vehicle device 10 and the terminal of the operator of the terminal device T is not established.

For example, the video distribution operation by the in-vehicle device 10 may be started without establishing a communication connection with the terminal device T after communication between the in-vehicle device 10 and the server S is established. In this case, an unspecified or permitted specific terminal device T connects to the server S to receive the video distributed from the in-vehicle device 10, and the operator of the terminal device T can view the video. becomes possible.

A series of processes in the control unit 25 of the in-vehicle device 10 described in the first embodiment may be a program executed by a computer. The program may be recorded on a computer-readable recording medium. The type of recording medium is not particularly limited, and may be, for example, an optical disk, hard disk, or semiconductor memory such as flash memory or SSD. Further, the program may be downloaded and installed in the in-vehicle device 10, the server S, and the terminal device T via communication.

The control routine shown in the above-described first embodiment is merely an example, and can be appropriately selected and changed according to the application, use conditions, and the like.

100 information processing system M vehicle 10 in-vehicle device S server T terminal device 12 exterior camera 13 interior camera 14 GPS receivers 15, 21 microphones 16, 23 touch panel displays 17, 22 speakers 25, 35, 37 control unit 26 encoding unit 27 , 36, 38 communication unit 28 storage unit 31 video acquisition unit 32 image recognition unit 33 visibility state determination unit 39 decoding unit

Claims (11)

A video transmission device that moves with a moving object,
an image acquisition unit that sequentially acquires an image around the moving object;
a visibility state determination unit that sequentially acquires visibility information regarding the degree of opening of the visibility in front of the moving object;
and a video transmission unit that transmits the video at a frame rate corresponding to the degree of opening. 2. The video transmission device according to claim 1, wherein said degree of opening is determined based on the presence or absence of a continuous object positioned continuously along the side of the road on which said mobile body travels. The degree of opening is determined based on whether or not a plurality of objects exist in the forward field of view at intervals along the sides of the road on which the moving body travels. 3. The video transmission device according to claim 1 or 2, wherein 2. The opening degree according to claim 1, characterized in that the degree of opening is determined based on the distance between the moving body and a continuous object continuously positioned along the side of the road on which the moving body travels. Video transmission device. 2. The video transmission device according to claim 1, wherein said degree of opening is determined based on attribute information of an area including the position of said moving object on a map. 6. The video transmission device according to claim 1, wherein the video transmission unit transmits the video with a high frame rate when the degree of opening is small. The video transmission unit has a first encoding mode as an operation mode for encoding the video and a second encoding mode for encoding the video in a manner in which the frame rate is higher than that in the first encoding mode. has
7. The video transmitting apparatus according to claim 1, wherein said encoding section operates in said second encoding mode when said field of view is not open. 8. The video transmitting apparatus according to claim 7, wherein the resolution of said video in said second encoding mode is smaller than the resolution of said video in said first encoding mode. A video transmission method executed by a video transmission device that moves with a mobile body,
an image acquisition step of sequentially acquiring an image around the moving object;
a visibility state determination step of sequentially acquiring information about the degree of opening of the field of vision in front of the moving body;
and a video transmission step of transmitting the video at a frame rate corresponding to the degree of opening. A program to be executed by a computer,
an image acquisition step of sequentially acquiring an image around the moving object;
a visibility state determination step of sequentially acquiring information about the degree of opening of the field of vision in front of the moving object;
and a video transmission step of transmitting the video at a frame rate corresponding to the degree of opening. A recording medium storing the program according to claim 10 .

Images