JP2022144246A - Information processing device - Google Patents

Abstract

To make it possible to stably transmit video data from vehicles.SOLUTION: An information processing device is mounted on a vehicle and transmits video data by wireless communication. The information processing device includes a control unit that lowers resolution of video data to be transmitted and its transmission rate, when at least one of a determination that a radio wave condition around the vehicle is poor and a determination that a processing load on the information processing device exceeds a threshold value is made.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to an information processing device.

Conventionally, a camera that captures the surroundings of a vehicle, an information processing unit that identifies a person and a communication terminal in the captured image, and a communication unit that transmits the captured image to the communication terminal. There is an in-vehicle video system that realizes a video transfer system by specifying a terminal and transmitting a captured image to the specified communication terminal via a base station device and a network (for example, Patent Document 1).

JP 2016-009431 A

An object of the present disclosure is to provide an information processing device that enables stable transmission of video data from a vehicle.

One aspect of the present disclosure is an information processing device that is mounted on a vehicle and transmits video data by wireless communication. It is an information processing apparatus including a control unit that reduces the resolution of video data to be transmitted and the transmission rate thereof when at least one of the above is determined to be exceeded.

Aspects of the present disclosure may include at least one of an information processing method, an information processing system, a program, and a recording medium recording the program, which have the same characteristics as the information processing apparatus.

According to the present disclosure, it is possible to stably transmit video data from a vehicle.

FIG. 1 is a schematic diagram of an information processing system according to an embodiment. FIG. 2 is a diagram showing a configuration example of a terminal. FIG. 3 is a flowchart illustrating an example of processing performed by a terminal.

An information processing apparatus according to an embodiment is an information processing apparatus that is mounted in a vehicle and that transmits video data by wireless communication. The information processing device determines that the radio wave condition around the vehicle is poor and determines that the processing load of the information processing device exceeds a threshold, and if at least one of the determination is made, the video data to be transmitted is transmitted. resolution and its transmission rate.

According to the information processing device, when at least one of the judgment that the radio wave condition is poor and the judgment that the processing load exceeds the threshold value is made, the resolution of the video data to be transmitted and the transmission rate of the video data are reduced. . In this way, by reducing the amount of video data transmitted from the information processing device per unit time, the bit error rate on the receiving side rises and retransmission processing becomes more difficult even when the radio wave condition is poor. A suitable throughput can be ensured by suppressing the increase. That is, video data can be stably transmitted. Therefore, on the receiving side, although the image quality is degraded, it is possible to view the reproduced video in which interruption of reproduction is suppressed.

Also, when the load of the information processing device exceeds the threshold, the resolution and transmission rate of the video data to be transmitted are reduced, so that processing for generating video data, encoding and modulating the video data for transmission are performed. The amount of processing such as processing is reduced. Therefore, the load on the information processing device is reduced, heat generation due to overloading of devices such as processors that perform processing in the information processing device is avoided, and performance degradation due to heat generation is avoided, thereby stably transmitting video data. be able to.

As a method of lowering the transmission rate, there is a method of changing the encoding method and modulation method of video data so as to lower the transmission rate. Alternatively, it is conceivable to temporarily store (buffer) video data in a storage device and control the reading rate from the buffer (the amount of video data read per time) so as to reduce the transmission rate. However, the method of lowering the transmission rate is not limited to the above.

An information processing apparatus according to an embodiment will be described below with reference to the drawings. The configuration of the embodiment is an example, and the present disclosure is not limited to the configuration of the embodiment.

<Configuration of information processing system>
FIG. 1 is a schematic diagram of an information processing system according to an embodiment. In FIG. 1 , the information processing system includes a network 1 , a server 2 , a terminal 3 mounted on a vehicle 10 and a base station 4 connected to the network 1 . Terminal 3 is an example of an “information processing device”.

The network 1 is formed by connecting a public communication network such as the Internet and a wireless communication network, for example. The wireless communication network conforms to or complies with wireless communication standards available to the terminals 3 . Examples of wireless communication standards include 4G (Long Term Evolution (LTE)), 5G, wireless LAN (Local Area Network: including Wi-Fi), and BLE, but other standards may also be used.

The base station 4 is a radio base station capable of radio communication with the terminal 3 . Although one base station 4 is shown in FIG. 1, a plurality of base stations are installed according to the communication area that provides the terminal 3 with an environment in which wireless communication is possible.

The terminal 3 is a terminal (in-vehicle terminal) mounted in the vehicle 10 . The terminal 3 may be fixed (installed) to the vehicle 10 or detachable from the vehicle 10 . The vehicle 10 is an automobile (passenger car, truck, bus, etc.), but may be a motorcycle. The terminal 3 transmits video data captured by the camera 61 mounted on the vehicle 10 to the server 2 via the base station 4 and the network 1 . As used herein, the term "video" includes not only moving images but also still images.

In this embodiment, as an example, the terminal 3 supports 5G, and the network 1 provides the terminal 3 with a 5G line. However, even if the terminal 3 is capable of communicating in other than 5G (for example, 4G (LTE)), it may support two or more wireless communication standards (for example, 4G and 5G). The network 1 provides the terminal 3 with a line conforming to the wireless communication standard supported by the terminal 3 .

The server 2 provides various information to the terminal 3 accessing via the network 1 . The server 2 also receives video data transmitted from the terminal 3 via the network 1 and analyzes the video.

<Device configuration>
FIG. 2 is a diagram showing a configuration example of the terminal 3. As shown in FIG. The terminal 3 includes a processor 31 connected to the bus B, a storage device 32, a communication interface (communication IF) 33, an input device 34, a display 35, a video encoding circuit 36a, an audio encoding circuit 36b, a video decoding circuit 37a, an audio A decoding circuit 37b and a speaker 38 are included. An antenna 33 a is connected to the communication IF 33 . The video encoding circuit 36 a is connected to a camera 61 mounted on the vehicle 10 , and the audio encoding circuit 36 b is connected to a microphone 62 mounted on the vehicle 10 .

The storage device 32 includes a main storage device and an auxiliary storage device. The main memory is used as a storage area for programs and data, a development area for programs, a work area for programs, a buffer area for communication data, and the like. The main memory is composed of a RAM (Random Access Memory) or a combination of a RAM and a ROM (Read Only Memory). Auxiliary storage is used as a storage area for data and programs. Non-volatile storage media such as hard disks, Solid State Drives (SSDs), flash memories, and EEPROMs (Electrically Erasable Programmable Read-Only Memory) can be used as auxiliary storage devices.

The communication IF 33 is a wireless communication circuit that conforms to or complies with the wireless communication standard supported by the terminal 3, and communicates wirelessly with the base station 4 via the antenna 33a according to the protocol according to the wireless communication standard. part). The communication IF 33 converts a digital signal (baseband signal) generated by modulating data to be transmitted into a radio signal, or converts a radio signal received from the antenna 33a into a digital signal.

The input device 34 is a key, button, pointing device, touch panel, or the like, and is used to input information. The display 35 is, for example, a liquid crystal display or an organic EL display, and displays information and data. The speaker 38 is used to emit sound based on the audio signal.

The video signal captured by the camera 61 is input to the video encoding circuit 36a. The video encoding circuit 36a converts the video signal into video encoded data ( video file). The video encoding circuit 36a can compress and encode video signals using any one of a plurality of encoding methods with different resolutions. The encoding format used by the video encoding circuit 36a is specified by the processor 31, for example.

An audio signal picked up by a microphone 62 is input to the audio encoding circuit 36b. ).

Under the control of the processor 31, the video encoded data and audio encoded data are directly transmitted from the communication IF 33, or are temporarily stored (accumulated) in the storage device 32 and then transmitted from the communication IF 33. . The video encoded data and the audio encoded data are linked and managed so that they are played back in synchronization on the same time axis. However, video data may not be associated with audio data (video data without audio).

In this embodiment, the video data to be transmitted is the video data of the video imaged by the camera 61, but is not limited to this. That is, video data acquired by a device other than the camera 61 and stored in the storage device 32 may be included in the transmission target.

The video decoding circuit 37a decodes the encoded video data read from the storage device 32 or the encoded video data received by the communication IF 33 to generate a video signal. The video signal is connected to the display 35, and the display 35 displays video based on the video signal.

The audio decoding circuit 37b decodes the encoded audio data read from the storage device 32 or the encoded audio data received by the communication IF 33 to generate an audio signal. The audio signal is connected to speaker 38, and speaker 38 emits sound based on the audio signal.

The processor 31 is, for example, a CPU (Central Processing Unit), and the storage device 32
Various processes are performed by executing various programs stored in the . For example, the processor 31 controls the operations of the video encoding circuit 36a, the audio encoding circuit 36b, and the communication IF 33, encodes the video captured by the camera 61 and the audio captured by the microphone 62, and transmits the It controls transmission from the communication IF 33 to the destination. At this time, the processor 31 performs channel coding and modulation processing (orthogonal modulation) on the video data and audio data to be transmitted, and performs processing for converting the video data and audio data into baseband signals. The baseband signal is input to the communication IF 33, converted into a radio signal having a predetermined radio frequency, and then radiated from the antenna 33a.

The processor 31 can also perform processing of demodulating and decoding the baseband signal input from the communication IF 33 and converting the baseband signal into data. The processor 31 also writes the video data and audio data obtained by demodulation and decoding into the storage device 32 and sends them to the video decoding circuit 37a and the audio decoding circuit 37b.

Further, the processor 31 changes the resolution of the video data according to the radio wave conditions around the terminal 3 (vehicle 10 ) and the processing load of the terminal 3 . That is, the processor 31 performs determination processing related to the radio wave condition and the processing load, and if it is determined that at least one of the radio wave condition is poor and the processing load exceeds the threshold, the resolution of the video data and the transmission rate of the video data process to reduce

As the processor 31, a plurality of CPUs or a multi-core CPU may be applied. At least part of the processing performed by the CPU may be performed by a processor other than the CPU, such as a DSP (Digital Signal Processor) or GPU (Graphical Processing Unit). In addition, at least part of the processing performed by the CPU is a dedicated or general-purpose integrated circuit (hardware) such as ASIC (Application Specific Integrated Circuit) and FPGA (Field Programmable Gate Array), or a combination of a processor and an integrated circuit may be executed. The combination is called, for example, a microcontroller (MCU), SoC (System-on-a-chip), system LSI, or chipset.

<Operation example>
FIG. 3 is a flow chart showing a processing example of the processor 31 of the terminal 3. As shown in FIG. The processing shown in FIG. 3 is the video data to be transmitted from the communication IF 33, that is, generated by the video encoding circuit 36a, converted into a baseband signal by the processor 31, converted into a radio signal by the communication IF 33, and transmitted from the antenna 33a. This is processing (resolution control processing) for controlling the resolution of the image data emitted (transmitted) from the camera 61 . The trigger for starting the resolution control process may be the operation of the input device 34 (operation to start the process), the turn-on of the camera 61, or other factors.

In step S001, the processor 31 determines whether or not the radio wave condition around the terminal 3 (vehicle 10) is poor. For example, the terminal 3 measures the electric field intensity of radio waves received by the antenna 33a and determines whether the electric field intensity is below a predetermined threshold. If it is determined that the electric field strength is below the threshold, it is determined that the radio wave condition is poor. If the radio wave condition is determined to be poor, the process proceeds to step S004; otherwise, the process proceeds to step S002.

In determining whether the radio wave condition is bad, the S/N ratio of the received radio wave, the signal to interference noise ratio (SINR) of the received radio wave, or the bit error rate (BER) of the data obtained by demodulating and decoding the received radio wave ) and the like may be used. In addition, the above shows a method of measuring the radio wave condition in the downlink (the receiving direction of the terminal 3), and in this embodiment, the radio wave condition in the uplink direction (the transmitting direction of the terminal 3) is estimated from the radio wave condition of the downlink. . As for the radio wave condition of the uplink, a method in which the terminal 3 is notified of the radio wave condition (reception power, etc.) of the radio signal from the terminal 3 from the base station 4 is conceivable, and such a method may be used.

In step S002, the processor 31 determines whether or not the value indicating the processing load of the terminal 3 is equal to or greater than a threshold (exceeds the threshold). That is, the processor 31 refers to the usage rate of the CPUs that constitute the processor 31 as a value indicating the processing load, and if the usage rate is equal to or greater than a predetermined threshold, the processing load is equal to or greater than the threshold (exceeding the threshold). there is). If the processing load is determined to be equal to or greater than the threshold, the process proceeds to step S005; otherwise, the process proceeds to step S003. The determination of the processing load may be performed using information indicating the processing load other than the CPU usage rate. Also, the determination of the processing load may be performed on a part other than the processor 31 as a target.

In step S003, processor 31 sets the initial resolution. That is, the processor 31 sets the encoding format used by the video encoding circuit 36a to the encoding format for compressing and encoding the video signal at the initial resolution. However, if the encoding format corresponding to the initial resolution has already been set, nothing is done.

In step S004, the processor 31 performs processing for setting the transmission rate of video data to be transmitted (for example, transmitted to the server 2) to the transmission rate applied at the initial resolution. At this time, the channel coding scheme and modulation scheme are changed as necessary. Furthermore, in order for the base station 4 to operate synchronously, the base station 4 is notified of information indicating a change in resolution, a change in transmission rate, or a change in encoding method and modulation method, etc., as necessary. be done. In step S004, if the transmission rate has already been set to the transmission rate corresponding to the initial resolution, nothing is done.

At step S005, the processor 31 sets the low resolution. That is, the processor 31 changes the encoding format used by the video encoding circuit 36a to a encoding format that compresses and encodes the video signal at a low resolution in which the data size of one frame (one screen) is smaller than the initial resolution. set. However, if the encoding format corresponding to the low resolution has already been set, nothing is done.

In step S006, the processor 31 performs a process of lowering the transmission rate of the video data to be transmitted (for example, transmitted to the server 2) to the transmission rate (corresponding to the low resolution) applied at the time of low resolution. conduct. At this time, the channel coding scheme and modulation scheme are changed as necessary. Further, if necessary, the base station 4 is notified of information indicating a change in resolution, a change in transmission rate, or a change in encoding method and modulation method. In step S004, if the transmission rate has already been set to the transmission rate corresponding to the initial resolution, nothing is done.

Also, as a method of lowering the transmission rate of video data, the following method (referred to as a buffering method) may be applied. In the initial resolution video data transmission, the video data generated by the video encoding circuit 36a is passed to the processor 31 for modulation and the like without being stored in the storage device 22. FIG. On the other hand, when the buffering method is applied, the video data generated by the video encoding circuit 36 a is temporarily stored (buffered) in the storage device 32 . The processor 31 performs readout control to read buffered video data from the storage device 32 at a readout rate (amount read from the storage device 22 per unit time) that matches the transmission rate applied when the resolution is low. The video data read out from the storage device 32 is subjected to transmission line coding and modulation. When performing the buffering method, channel coding and modulation schemes may be changed as needed. Such a buffering method may also be performed for the initial resolution, so that the readout rate for the low resolution is lower than the readout rate for the initial resolution.

In step S007, the processor 31 performs processing for notifying the user of the terminal 3 of information indicating the change in resolution and transmission rate. For example, the processor 31 causes the display 35 to display information indicating the change in resolution and transmission rate, or causes the speaker 38 to emit sound indicating the change. This step S007 is optional.

In step S<b>008 , the processor 31 determines whether or not the resolution control process is to be terminated by an end operation by the input device 34 or the like. If it is determined that the resolution control process should end, the resolution control process ends; otherwise, the process returns to step S001.

According to the resolution control process, when it is determined that the radio wave condition is poor or the amount of processing added is equal to or greater than the threshold, the video data is transmitted at a low resolution and a corresponding transmission rate (low transmission rate). Otherwise, video data is transmitted at an initial resolution higher than the low resolution and at a transmission rate higher than the low transmission rate (high transmission rate).

Note that the order of steps S001 and S002 may be reversed. Also, although the resolution and transmission rate are two levels in this embodiment, they may be adjusted in three or more levels. Also, one of the determination of poor radio wave conditions (step S001) and the determination of excess processing load threshold (step S002) may be omitted. Also, the resolution and transmission rate may be lowered when it is determined that the radio wave condition is bad and the processing load exceeds the threshold.

<Action and effect of the embodiment>
According to the embodiment, in the terminal 3 that is mounted on the vehicle 10 and transmits video data by wireless communication, the processor 31 as a control unit determines that the radio wave condition around the vehicle 10 (terminal 3) is poor, or When it is determined that the processing load of the terminal 3 is equal to or greater than the threshold, the resolution and transmission rate of the video data to be transmitted are lowered. Therefore, video data can be stably transmitted.

That is, the reduction in resolution reduces the amount of video data, and the reduction in transmission rate reduces the transmission amount of video data per unit time and the amount of processing associated with the transmission. By reducing the amount of video data transmitted, the bit error rate at the receiving end can be suppressed to maintain throughput even when radio conditions are poor. It can be reproduced favorably.

In addition, since the transmission rate decreases as the resolution decreases, the amount of processing required for modulating the video data is reduced, so the load on devices such as the processor 31 is reduced. By suppressing the heat generation of devices such as the processor 31 by reducing the load, it is possible to suppress performance deterioration due to heat generation. Therefore, video data can be stably transmitted.

<Others>
The above-described embodiment is merely an example, and the present disclosure can be modified as appropriate without departing from the scope of the present disclosure. Also, the processing described as being performed by one device may be shared and performed by a plurality of devices. Alternatively, processes described as being performed by different devices may be performed by one device. In a computer system, it is possible to flexibly change the hardware configuration (server configuration) to implement each function.

The present disclosure can also be implemented by supplying a computer program implementing the functions described in the above embodiments to a computer, and reading and executing the program by one or more processors of the computer. Such a computer program may be provided to the computer by a non-transitory computer-readable storage medium connectable to the system bus of the computer, or may be provided to the computer via a network. A non-transitory computer-readable storage medium can be any type of disk such as, for example, a magnetic disk (floppy disk, hard disk drive (HDD), etc.), an optical disk (CD-ROM, DVD disk, Blu-ray disk, etc.). be. Non-transitory computer readable media also include read only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, optical cards, suitable for storing electronic instructions. Including any type of media.

REFERENCE SIGNS LIST 1 network 2 server 3 terminal 4 base station 31 processor 32 storage device 33 communication interface 34 input device 35 display 36a Video encoding circuit 36b Audio encoding circuit 37a Video decoding circuit 37b Audio decoding circuit 38 Speaker

Claims (1)

An information processing device mounted on a vehicle for transmitting video data by wireless communication,
When at least one of a determination that the radio wave condition around the vehicle is poor and a determination that the processing load of the information processing device exceeds a threshold is made, the resolution and resolution of the video data to be transmitted An information processing device including a control unit that reduces its transmission rate.

Images