JP7058757B2 - Information processing equipment and information processing method - Google Patents

Description

The present invention relates to an information processing device and an information processing method for presenting vehicle information to an occupant.

The technology to present vehicle information to occupants directly or in cooperation with the cloud has been realized. Vehicle information presented to occupants is roughly divided into stream data, which is continuous data, and non-stream data, which is discrete data. A device or cloud installed in the vehicle uses at least one of stream data and non-stream data to generate the necessary information and present it to the occupants. Conventionally, a technique for processing stream data has been disclosed (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2013-63737

Stream data and non-stream data may be linked in order to present useful information to the occupants. In general, stream data and non-stream data have different characteristics, so stream data and non-stream data may be processed separately. When a device installed in a vehicle links stream data and non-stream data that have been separately processed, the processing for linking becomes complicated, such as the need to synchronize the stream data and non-stream data. There's a problem. On the other hand, when the cloud links stream data and non-stream data that have been separately processed, there is a problem that the processing for linking is complicated. Since Patent Document 1 does not mention the process of linking stream data and non-stream data, there is a problem that the process for linking as described above becomes complicated.

As described above, conventionally, there has been a problem that the processing for linking stream data and non-stream data becomes complicated and the load of data processing becomes large.

The present invention has been made to solve such a problem, and an object of the present invention is to provide an information processing apparatus and an information processing method capable of reducing the load of data processing.

In order to solve the above problems, the information processing apparatus according to the present invention is based on a data acquisition unit that acquires stream data that is continuous data and non-stream data that is discrete data, and a predetermined conversion definition. , The non-stream data that should be linked with the stream data among the non-stream data acquired by the data acquisition unit is converted into simulated stream data that simulates the stream data, and the non-stream data that is not linked with the stream data is converted into simulated stream data. A stream processing unit that performs predetermined stream processing on the data conversion unit that does not perform, the stream data acquired by the data acquisition unit, and the simulated stream data converted by the data conversion unit, and a non-stream acquired by the data acquisition unit. It is equipped with a non-stream processing unit that performs predetermined non-stream processing for non-stream data that is not linked with stream data among the data, and the stream data exists in the video data, audio data, music data, and the periphery of the vehicle. The non-stream data includes at least one of remote sensing data indicating the presence or absence of an object, vehicle condition data indicating the condition of the vehicle, and occupant condition data indicating the physical condition of the occupant of the vehicle, and the non-stream data is the occupant of the vehicle. It includes at least one of the line-of-sight data indicating the movement of the line of sight, the operation state data indicating the operation state of the occupant, and the time data indicating the current time .

According to the present invention, the information processing apparatus simulates a simulated stream of non-stream data to be linked with stream data among the non-stream data acquired by the data acquisition unit based on a predetermined conversion definition. Since it is provided with a data conversion unit that converts non-stream data that is converted into data and does not link with stream data into simulated stream data, it is possible to reduce the load of data processing.

Objectives, features, embodiments, and advantages of the present invention will be made more apparent with the following detailed description and accompanying drawings.

It is a block diagram which shows an example of the structure of the information processing apparatus by embodiment of this invention. It is a block diagram which shows an example of the structure of the information processing apparatus by embodiment of this invention. It is a figure which shows an example of the conversion definition by embodiment of this invention. It is a figure which shows an example of the distribution definition by embodiment of this invention. It is a figure which shows an example of the process flow definition by embodiment of this invention. It is a block diagram which shows an example of the hardware composition of the information processing apparatus by embodiment of this invention. It is a flowchart which shows an example of the operation of the information processing apparatus by embodiment of this invention. It is a flowchart which shows an example of the operation of the data conversion part by embodiment of this invention. It is a figure for demonstrating the operation of the data conversion part by embodiment of this invention. It is a figure for demonstrating the operation of the data conversion part by embodiment of this invention. It is a figure for demonstrating the operation of the data conversion part by embodiment of this invention. It is a figure for demonstrating the operation of the data conversion part by embodiment of this invention. It is a flowchart which shows an example of the operation of the data distribution part by embodiment of this invention.

Embodiments of the present invention will be described below with reference to the drawings.

<Embodiment>
<Structure>
FIG. 1 is a block diagram showing an example of the configuration of the information processing apparatus 1 according to the present embodiment. Note that FIG. 1 shows the minimum necessary configuration for configuring the information processing apparatus according to the present embodiment.

As shown in FIG. 1, the information processing apparatus 1 includes a data acquisition unit 2, a data conversion unit 3, a stream processing unit 4, and a non-stream processing unit 5. The data acquisition unit 2 acquires stream data which is continuous data and non-stream data which is discrete data.

The data conversion unit 3 converts the non-stream data to be linked with the stream data among the non-stream data acquired by the data acquisition unit 2 into simulated stream data simulating the stream data based on a predetermined conversion definition. do. Further, the data conversion unit 3 does not perform conversion processing on the stream data and the non-stream data that is not linked with the stream data.

In the stream processing unit 4, the data acquisition unit 2 performs predetermined stream processing on the stream data and the simulated stream data converted by the data conversion unit 3. The non-stream processing unit 5 performs predetermined non-stream processing on the non-stream data that is not linked with the stream data among the non-stream data acquired by the data acquisition unit 2.

Next, another configuration of the information processing apparatus including the information processing apparatus 1 shown in FIG. 1 will be described.

FIG. 2 is a block diagram showing an example of the configuration of the information processing apparatus 6 according to another configuration.

As shown in FIG. 2, the information processing apparatus 6 includes a data acquisition unit 2, a data conversion unit 3, a data distribution unit 7, a stream processing unit 4, a non-stream processing unit 5, and a data utilization unit 8. , A conversion definition 9 and a distribution definition 10. It is assumed that the information processing device 6 is mounted on the vehicle. For example, the information processing device 6 may be included in the navigation device mounted on the vehicle, or the information processing device 6 may be mounted on the vehicle alone.

The data acquisition unit 2 acquires stream data and non-stream data from the camera 11, the sensor 12, the microphone 13, the touch panel 14, and the ECU (Electronic Control Unit) 15. The stream data includes, for example, video data taken by the camera 11, audio data acquired from the microphone 13, music data, remote sensing data acquired from LIDAR (Light Detection and Ranging) or radar, and a vehicle indicating the state of the vehicle. Examples include state data and occupant state data indicating the physical state of the occupants of the vehicle. The remote sensing data is data indicating the presence or absence of an object existing in the vicinity of the vehicle, and includes the distance between the vehicle and the object and the direction in which the object exists with respect to the vehicle. Vehicle condition data includes vehicle speed, acceleration, steering angle, and the like. The occupant status data includes the pulse value of the occupant of the vehicle and the like. The ECU 15 acquires data from a sensor that detects the state of the vehicle.

The non-stream data includes line-of-sight data indicating the movement of the line of sight of the occupant of the vehicle, operation state data indicating the operation state of the occupant, time data, and a position indicating the current position of the vehicle acquired from GPS (Global Positioning System). Data etc. can be mentioned. The non-stream data may be data acquired at regular intervals or may be acquired irregularly. Examples of the data acquired irregularly include acceleration data detected during sudden braking of the vehicle. The ECU 15 detects the time data and acquires the position data from GPS or the like.

The camera 11 is a camera provided in the vehicle, and examples thereof include a camera for photographing the front of the vehicle, a camera for photographing the inside of the vehicle, and the like. When the camera 11 is a camera that photographs the inside of the vehicle, it is possible to detect the movement of the line of sight of the occupant. The sensor 12 is a sensor provided in the vehicle and includes, for example, lidar and radar. Further, the sensor 12 can detect, for example, the physical condition of the occupant, the movement of the occupant's line of sight, and the like. When detecting the movement of the line of sight of the occupant, the camera 11 and the sensor 12 may be combined.

The microphone 13 can detect the operation state by the voice of the occupant. The touch panel 14 can detect the operation state by the touch operation of the occupant. The ECU 15 can collect vehicle state data, time data, and position data. The ECU 15 may collect the information detected by the sensor 12.

The data conversion unit 3 converts the non-stream data to be linked with the stream data among the non-stream data acquired by the data acquisition unit 2 into simulated stream data simulating the stream data based on the predetermined conversion definition 9. Convert. Further, the data conversion unit 3 does not perform conversion processing on the stream data and the non-stream data that is not linked with the stream data. Examples of the non-stream data to be linked with the stream data include acceleration data which is non-stream data linked with the video data which is the stream data.

Here, the conversion definition 9 will be described. FIG. 3 is a diagram showing an example of the conversion definition 9. It is assumed that the conversion definition 9 is stored in a storage unit (not shown).

As shown in FIG. 3, the conversion definition 9 is composed of a “data name”, a “type”, and a “conversion definition”. The "data name" indicates the name of various data acquired by the data acquisition unit 2. "Type" indicates whether the data is used for stream processing or non-stream processing. "Conversion definition" indicates a conversion method when converting non-stream data to stream data.

For example, the line-of-sight data, which is non-stream data, is data used for non-stream processing. Therefore, the line-of-sight data is not converted into simulated stream data by the data conversion unit 3.

The operation state data, which is non-stream data, is data used for stream processing. Therefore, the operation state data is converted into simulated stream data by the data conversion unit 3. For example, the data conversion unit 3 converts the operation state data into simulated stream data by "complementing with 0".

Video data, which is stream data, is data used for stream processing. Therefore, the video data is not converted into simulated stream data by the data conversion unit 3.

Acceleration data, which is non-stream data, is data used for stream processing. Therefore, the acceleration data is converted into simulated stream data by the data conversion unit 3. For example, the data conversion unit 3 converts the acceleration data into simulated stream data by "complementing with 0".

In the above, the case where the data conversion unit 3 "complements with 0" for the non-stream data has been described, but the present invention is not limited to this. For example, the data conversion unit 3 may complement the approximate value of the non-stream data acquired by the data acquisition unit 2, or may complement any other value.

The data conversion unit 3 converts the non-stream into a simulated stream so as to be time-synchronized with the stream data to be linked. It is assumed that the sampling rate at the time of conversion is included in the conversion definition 9.

The conversion definition 9 can be arbitrarily changed via, for example, the data utilization unit 8 and the HMI 17.

Returning to the description of FIG. 2, the data distribution unit 7 transfers the stream data and the simulated stream data to the stream processing unit 4 and the non-stream data to the non-stream processing unit 5 based on the predetermined distribution definition 10. Sort.

Here, the distribution definition 10 will be described. FIG. 4 is a diagram showing an example of the distribution definition 10. It is assumed that the distribution definition 10 is stored in a storage unit (not shown).

As shown in FIG. 4, the distribution definition 10 is composed of a “data name” and a “type”. The "data name" indicates the name of various data acquired by the data acquisition unit 2. "Type" indicates whether the data is used for stream processing or non-stream processing.

For example, the line-of-sight data is data used for non-stream processing. Therefore, the data distribution unit 7 distributes the line-of-sight data to the non-stream processing unit 5.

The operation state data, video data, and acceleration data are data used for stream processing. Therefore, the data distribution unit 7 distributes the operation state data, the video data, and the acceleration data to the stream processing unit 4.

The "operation state data + video data + acceleration data" in FIG. 4 indicates that the operation state data, the video data, and the acceleration data are linked and stream-processed by the stream processing unit 4. The operation state data and the acceleration data are data converted into simulated stream data by the data conversion unit 3. That is, the operation state data and the acceleration data are data to be linked with the video data.

The distribution definition 10 can be arbitrarily changed via, for example, the data utilization unit 8 and the HMI 17.

Returning to the description of FIG. 2, the stream processing unit 4 performs predetermined stream processing on the stream data and the simulated stream data distributed by the data distribution unit 7. Specifically, the stream processing unit 4 performs stream processing on the stream data and the simulated stream data based on a predetermined processing flow definition. It is assumed that the stream processing unit 4 holds the processing flow definition.

Here, the processing flow definition will be described. FIG. 5 is a diagram showing an example of a processing flow definition, and shows an example of a processing flow definition for executing a service application that realizes a drive recorder service. A service application is software that realizes a service that presents vehicle information to occupants directly or in cooperation with the cloud. The drive recorder service is one of the services presented to the occupants.

As shown in FIG. 5, the processing flow definition is configured by ordering the processing functions of "forward camera", "acceleration", "synchronization", "sudden braking", "video extraction", and "transmission". There is. Each processing function is composed of software.

The "front camera" is a processing function for acquiring an image from a camera that has taken a picture of the front of the vehicle. "Acceleration" is a processing function for acquiring acceleration data from an acceleration sensor provided in a vehicle. That is, the "forward camera" and "acceleration" are processing functions for collecting data sources such as video and acceleration data. In addition to the "front camera" and "acceleration", the processing functions for collecting data sources include, for example, the "in-vehicle camera", which is a processing function for acquiring images from a camera that has taken a picture of the inside of the vehicle, and the line of sight of the driver. "Line-of-sight coordinates" which is a processing function to detect, "LIDAR" which is a processing function to detect pedestrians and objects existing around the vehicle using laser light, and walking existing around the vehicle using millimeter waves. Examples include "millimeter wave" which is a processing function for detecting a person and an object.

"Synchronization" is a processing function that synchronizes multiple data sources. In the example of FIG. 5, the image acquired from the front camera and the acceleration data acquired from the acceleration sensor are synchronized.

"Sudden braking" is a processing function for detecting sudden braking of a vehicle based on acceleration data, and is a trigger for transmitting the result of data processing based on the processing flow definition. In addition to "sudden braking", the processing function that serves as the trigger includes, for example, "line-of-sight attention", which is a processing function for detecting that the driver's line of sight is in a predetermined direction, and sudden acceleration of the vehicle. Examples include "rapid acceleration", which is a processing function for detecting data sources, and "constant cycle", which is a processing function for transmitting a data source at a fixed cycle.

"Movie extraction" is a processing function for extracting a moving image within a certain period of time from a video taken by a camera, and generates information to be transmitted as a result of data processing based on a processing flow definition. In addition to "video extraction", the processing functions that generate information to be transmitted as a result of data processing include, for example, "image extraction" that extracts at least one image from a plurality of images, and "map generation" that generates a map. , And "risk information" that generates information about driving risks.

In this way, the stream processing unit 4 performs a process of cutting out a part of the stream data and the simulated stream data when a predetermined event occurs according to the processing flow definition. Then, the stream processing unit 4 sends the cut out data to the data utilization unit 8 and discards other data. Specifically, the stream processing unit 4 cuts out a part of the stream data and the simulated stream data in a predetermined time point or a predetermined time range. Predetermined time points include, for example, a time point when an event occurs and a time point before or after the time when the event occurs. The predetermined time range includes, for example, a few seconds before and after the time when the event occurs, a few seconds until the time when the event occurs, a few seconds from the time when the event occurs, and the like.

When the stream data is video data, the stream processing unit 4 may perform a process of cutting out a predetermined area of the video data. For example, the stream processing unit 4 may perform a process of cutting out an image region corresponding to a predetermined range based on the driver's line of sight based on the driver's line of sight data detected by the sensor 12. .. In this case, it is assumed that the line-of-sight data, which is non-stream data, is converted into simulated stream data by the data conversion unit 3 and synchronized with the video data, which is stream data.

The processing flow definition can be arbitrarily changed via, for example, the data utilization unit 8 and the HMI 17.

Returning to the description of FIG. 2, the non-stream processing unit 5 performs predetermined non-stream processing on the non-stream data distributed by the data distribution unit 7.

The data utilization unit 8 uses the first data stream-processed by the stream processing unit 4 and the second data non-stream-processed by the non-stream processing unit 5.

Specifically, the data utilization unit 8 transmits the first data stream-processed by the stream processing unit 4 and the second data non-stream-processed by the non-stream processing unit 5 to the cloud 16 as they are. The cloud 16 uses the first data and the second data acquired from the data utilization unit 8 to generate information to be presented to the occupants of the vehicle and presents the information to the occupants of the vehicle. In this case, the information generated by the cloud 16 may be presented to the occupant via the HMI 17.

Further, the data utilization unit 8 generates information to be presented to the occupants of the vehicle by using the first data stream-processed by the stream processing unit 4 and the second data non-stream processed by the non-stream processing unit 5. , The information is presented to the occupants via the HMI (Human Machine Interface) 17.

The information processing device 6 can realize an AI (Artificial Intelligence) assistant in cooperation with the cloud 16. In this case, the data utilization unit 8 transmits the data necessary for executing the AI assistant to the cloud 16. Examples of the data transmitted by the data utilization unit 8 to the cloud 16 include data in which voice data and position data are combined, that is, first data that has been stream-processed by the stream processing unit 4. In this case, it is assumed that the position data, which is non-stream data, is converted into simulated stream data by the data conversion unit 3 and synchronized with the audio data, which is stream data. The cloud 16 performs processing necessary for executing the AI assistant using the data acquired from the data utilization unit 8, and transmits the result of the processing to the information processing apparatus 6.

FIG. 6 is a block diagram showing an example of the hardware configuration of the information processing apparatus 6.

Each function of the data acquisition unit 2, the data conversion unit 3, the data distribution unit 7, the stream processing unit 4, the non-stream processing unit 5, and the data utilization unit 8 in the information processing apparatus 6 is realized by a processing circuit. That is, the information processing apparatus 6 acquires stream data and non-stream data, converts the non-stream data to be linked with the stream data into simulated stream data, and transfers the stream data and simulated stream data to the stream processing unit, and the non-stream data. Is distributed to the non-stream processing unit 5, stream processing is performed on the stream data and the simulated stream data, non-stream processing is performed on the non-stream data, and the stream-processed first data and the non-stream-processed first data are performed. It is provided with a processing circuit for using 2 data. The processing circuit is a processor 18 (also referred to as a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor)) that executes a program stored in the memory 19.

The functions of the data acquisition unit 2, the data conversion unit 3, the data distribution unit 7, the stream processing unit 4, the non-stream processing unit 5, and the data utilization unit 8 in the information processing apparatus 6 are software, firmware, or software and firmware. It is realized by the combination with. The software or firmware is described as a program and stored in the memory 19. The processing circuit realizes the functions of each part by reading and executing the program stored in the memory 19. That is, the information processing apparatus 6 has a step of acquiring stream data and non-stream data, a step of converting non-stream data to be linked with stream data into simulated stream data, and sending stream data and simulated stream data to a stream processing unit. A step of distributing stream data to the non-stream processing unit 5, a step of performing stream processing on stream data and simulated stream data, a step of performing non-stream processing on non-stream data, and a step of performing non-stream processing on the stream-processed first data and non-stream data. A memory 19 for storing a program in which a step using the stream-processed second data is to be executed as a result is provided. Further, these programs cause a computer to execute the procedures or methods of the data acquisition unit 2, the data conversion unit 3, the data distribution unit 7, the stream processing unit 4, the non-stream processing unit 5, and the data utilization unit 8. It can be said that there is. Here, the memory is, for example, non-volatile or volatile such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EPROM (Electrically Erasable Programmable Read Only Memory). It may be a sex semiconductor memory, a magnetic disk, a flexible disk, an optical disk, a compact disk, a DVD, or the like, or any storage medium that will be used in the future.

<Operation>
FIG. 7 is a flowchart showing an example of the operation of the information processing apparatus 6.

In step S11, the data acquisition unit 2 acquires stream data and non-stream data from the camera 11, the sensor 12, the microphone 13, the touch panel 14, and the ECU 15.

In step S12, the data conversion unit 3 determines whether or not there is non-stream data to be linked with the stream data among the non-stream data acquired by the data acquisition unit 2 based on the conversion definition 9. If there is non-stream data to be linked with the stream data, the process proceeds to step S13. If there is no non-stream data to be linked with the stream data, the process proceeds to step S14.

In step S13, the data conversion unit 3 converts the non-stream data into simulated stream data based on the conversion definition 9.

In step S14, the data distribution unit 7 distributes stream data and simulated stream data to the stream processing unit 4 and non-stream data to the non-stream processing unit 5 based on the distribution definition 10.

In step S15, the stream processing unit 4 performs predetermined stream processing on the stream data and the simulated stream data distributed by the data distribution unit 7.

In step S16, the non-stream processing unit 5 performs predetermined non-stream processing on the non-stream data distributed by the data distribution unit 7.

Note that FIG. 7 shows a case where the non-stream processing is performed after the stream processing is performed, but the present invention is not limited to this, and the stream processing may be performed after the non-stream processing is performed.

FIG. 8 is a flowchart showing an example of the operation of the data conversion unit 3, and shows the details of step S13 in FIG. 7. Hereinafter, the case where the data conversion unit 3 converts the position data, which is the non-stream data shown in FIG. 9 acquired by the data acquisition unit 2, into the simulated stream data will be described. In FIG. 9, the “time” indicates the date when the position data was acquired, and for example, “201806221151933” represents “June 22, 2018 15:19:33”.

In step S21, the data conversion unit 3 prepares a data string having a period based on the sampling rate included in the conversion definition 9. Specifically, a data string as shown in FIG. 10 is prepared. The sampling rate is set to be synchronized in time with the stream data linked with the position data.

In step S22, the data conversion unit 3 writes non-stream data to the data string based on the time information. Specifically, as shown in FIG. 11, the data conversion unit 3 writes the position data in the data string based on the time information.

In step S23, the data conversion unit 3 complements the data in a place not written in the data string. Specifically, as shown in FIG. 12, the data conversion unit 3 complements the approximate value of each position data written in step S22. In addition, in FIG. 12, the data conversion unit 3 may complement 0 in the portion not written in the data string.

In step S24, the data conversion unit 3 uses the data string supplemented in step S23 as simulated stream data, and synchronizes the simulated stream data with the linked stream data.

FIG. 13 is a flowchart showing an example of the operation of the data distribution unit 7, and shows the details of step S14 in FIG. 7.

In step S31, the data distribution unit 7 determines whether to distribute each of the stream data, the non-stream data, and the simulated stream data to the stream processing unit 4 or the non-stream processing unit 5 based on the distribution definition 10. do.

In step S32, the data distribution unit 7 distributes the stream data and the simulated stream data to the stream processing unit 4 and the non-stream data to the non-stream processing unit 5.

<Effect>
Conventionally, when the stream data and the non-stream data are linked by a device provided in the vehicle or the cloud, there is a problem that the process for linking becomes complicated and the load of data processing is increased. On the other hand, according to the present embodiment, the data conversion unit 3 acquires stream data and non-stream data in real time from the data acquisition unit 2, and there is almost no time lag between the acquired data. Therefore, the data conversion unit 3 can convert the non-stream data into the simulated stream data based on the conversion definition 9, and can easily perform the synchronization processing between the stream data and the simulated stream data. As a result, it is possible to reduce the load of data processing as compared with the conventional case.

According to the present embodiment, the data conversion unit 3 does not convert all the non-stream data into the simulated stream, but converts the necessary non-stream data into the simulated stream data based on the conversion definition 9. Therefore, the load of data processing can be reduced as compared with the case of converting all non-stream data into a simulated stream.

According to the present embodiment, the data distribution unit 7 distributes stream data and simulated stream data to the stream processing unit 4 and non-stream data to the non-stream processing unit 5 based on the distribution definition 10. Therefore, for non-stream data that does not require stream data processing, non-stream data processing can be performed when the non-stream data is required, and the overall data processing load can be reduced.

In general, the hardware configuration installed in a vehicle differs depending on the vehicle type, and the data that can be acquired from the hardware also differs. Therefore, the service application developer considers the hardware configuration installed in each vehicle and considers the service application. Had to develop. On the other hand, according to the present embodiment, since the conversion definition 9 may be changed according to the change in the hardware configuration, the burden on the developer of the service application can be reduced.

In the present invention, the embodiments can be appropriately modified or omitted within the scope of the invention.

Although the present invention has been described in detail, the above description is exemplary in all embodiments and the invention is not limited thereto. It is understood that innumerable variations not illustrated can be assumed without departing from the scope of the present invention.

1 Information processing device, 2 Data acquisition unit, 3 Data conversion unit, 4 Stream processing unit, 5 Non-stream processing unit, 6 Information processing device, 7 Data distribution unit, 8 Data utilization unit, 9 Conversion definition, 10 Distribution definition , 11 cameras, 12 sensors, 13 microphones, 14 touch panels, 15 ECUs, 16 clouds, 17 HMIs, 18 processors, 19 memories.

Claims (12)

A data acquisition unit that acquires stream data that is continuous data and non-stream data that is discrete data,
Based on a predetermined conversion definition, the non-stream data to be linked with the stream data among the non-stream data acquired by the data acquisition unit is converted into simulated stream data simulating the stream data, and the above-mentioned A data conversion unit that does not convert the non-stream data that is not linked to the stream data into the simulated stream data ,
A stream processing unit that performs predetermined stream processing on the stream data acquired by the data acquisition unit and the simulated stream data converted by the data conversion unit.
A non-stream processing unit that performs predetermined non-stream processing on the non-stream data that is not linked to the stream data among the non-stream data acquired by the data acquisition unit.
Equipped with
The stream data includes video data, audio data, music data, remote sensing data indicating the presence or absence of an object existing around the vehicle, vehicle state data indicating the state of the vehicle, and the physical state of the occupant of the vehicle. Contains at least one of the occupant status data shown
The non-stream data is an information processing apparatus including at least one of line-of-sight data indicating the movement of the line-of-sight of a vehicle occupant, operation state data indicating the operation state of the occupant, and time data indicating the current time. .. A data distribution unit that distributes the stream data and the simulated stream data to the stream processing unit and the non-stream data that is not linked to the stream data to the non-stream processing unit based on a predetermined distribution definition. The information processing apparatus according to claim 1, further comprising. The information processing apparatus according to claim 1, wherein the data conversion unit converts the non-stream data into the simulated stream data so as to be time-synchronized with the stream data to be linked. The information processing apparatus according to claim 1, wherein the stream processing unit cuts out a part of the stream data and the simulated stream data when a predetermined event occurs. The information processing apparatus according to claim 4, wherein the stream processing unit cuts out a part of the stream data and the simulated stream data at a predetermined time point or in a predetermined time range. The stream data includes video data and includes video data.
The information processing apparatus according to claim 4, wherein the stream processing unit cuts out a predetermined area of the video data. The information processing apparatus according to claim 1, wherein the stream processing unit executes the stream processing according to a predetermined processing flow definition. The information processing apparatus according to claim 1, wherein the conversion definition can be changed. The information processing apparatus according to claim 2, wherein the distribution definition can be changed. The information processing apparatus according to claim 7 , wherein the processing flow definition can be changed. A data utilization unit that uses the first data that the stream processing unit has performed the stream processing and the second data that the non-stream processing unit has performed the non-stream processing is further provided.
The information processing apparatus according to claim 1, wherein the data utilization unit transmits the first data and the second data to an external cloud. Get stream data that is continuous data and non-stream data that is discrete data,
Based on a predetermined conversion definition, the non-stream data to be linked with the stream data among the acquired non-stream data is converted into simulated stream data simulating the stream data and not linked with the stream data. Without converting the non-stream data to the simulated stream data,
A predetermined stream process is performed on the acquired stream data and the converted simulated stream data.
Of the acquired non-stream data, the non-stream data that is not linked to the stream data is subjected to predetermined non-stream processing .
The stream data includes video data, audio data, music data, remote sensing data indicating the presence or absence of an object existing around the vehicle, vehicle state data indicating the state of the vehicle, and the physical state of the occupant of the vehicle. Contains at least one of the occupant status data shown
The non-stream data includes at least one of line-of-sight data indicating the movement of the line-of-sight of a vehicle occupant, operation state data indicating the operation state of the occupant, and time data indicating the current time. ..

Images