EP4080359A1 - Signal processing device and vehicle display device having same - Google Patents
Signal processing device and vehicle display device having same Download PDFInfo
- Publication number
- EP4080359A1 EP4080359A1 EP21878751.3A EP21878751A EP4080359A1 EP 4080359 A1 EP4080359 A1 EP 4080359A1 EP 21878751 A EP21878751 A EP 21878751A EP 4080359 A1 EP4080359 A1 EP 4080359A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- virtual machine
- data
- shared memory
- display
- signal processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 claims abstract description 62
- 230000015654 memory Effects 0.000 claims description 188
- 239000000872 buffer Substances 0.000 claims description 100
- 230000004044 response Effects 0.000 claims description 9
- 239000002131 composite material Substances 0.000 claims description 6
- 238000004891 communication Methods 0.000 description 63
- 230000001360 synchronised effect Effects 0.000 description 28
- 230000005540 biological transmission Effects 0.000 description 11
- 238000009877 rendering Methods 0.000 description 10
- 238000001514 detection method Methods 0.000 description 6
- 230000005236 sound signal Effects 0.000 description 3
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 239000000446 fuel Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
- B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
- B60R16/023—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for transmission of signals between vehicle parts or subsystems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45579—I/O management, e.g. providing access to device drivers or storage
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45583—Memory management, e.g. access or allocation
Definitions
- the present disclosure relates to a signal processing device and a display apparatus for vehicle including the same, and more particularly to a signal processing device capable of dividing and processing data using a plurality of virtual machines for a plurality of displays in a vehicle and a display apparatus for vehicle including the same.
- a vehicle is an apparatus that a driver moves in a desired direction.
- a representative example of the vehicle is a car.
- a display apparatus for vehicle is located in the vehicle for convenience of users who use the vehicle.
- a display is disposed in a cluster in order to display various kinds of information.
- various displays such as an audio video navigation (AVN) display, are located in the vehicle, in addition to the cluster.
- APN audio video navigation
- An object of the present disclosure is to provide a signal processing device capable of dividing and processing data using a plurality of virtual machines for a plurality of displays in a vehicle and a display apparatus for vehicle including the same.
- Another object of the present disclosure is to provide a signal processing device capable of performing high-speed data communication between a plurality of virtual machines and a display apparatus for vehicle including the same.
- a further object of the present disclosure is to provide a signal processing device capable of performing high-speed data communication even though a plurality of virtual machines is driven by different operating systems and a display apparatus for vehicle including the same.
- a signal processing device including a processor configured to perform signal processing for a plurality of display located in a vehicle, wherein the processor is configured to execute first to third virtual machines on a hypervisor in the processor, the second virtual machine is operated for a first display among the plurality of display, the third virtual machine is operated for a second display among the plurality of display, and the first virtual machine in the processor is configured to share at least some of data with the second virtual machine and the third virtual machine for processing of divided data.
- the first virtual machine in the processor may be configured to write some of the data in a first shared memory to be transmitted to the second virtual machine and write some other of the data in the first shared memory to be transmitted to the third virtual machine, and the second virtual machine and the third virtual machine may be configured to process the received data and write the processed data in a second shared memory.
- the first virtual machine in the processor may be configured to process some other of the data and write the processed data in the second shared memory.
- the processor may further execute a fourth virtual machine, the first virtual machine in the processor may be configured to write some other of the data in the first shared memory, and the fourth virtual machine may be configured to process received data and write the processed data in the second shared memory.
- the first virtual machine in the processor may create command queues for distributed processing of the data in the second virtual machine and the third virtual machine.
- the first virtual machine in the processor may create one command queue.
- command queue may include at least one of virtual machine index information, task information, or shared memory information.
- the first virtual machine in the processor may create command queues corresponding to the number of virtual machines for distributed processing of the data.
- the first virtual machine in the processor may be configured to write some of the image data in the first shared memory to be transmitted to the second virtual machine and write some other of the image data in the first shared memory to be transmitted to the third virtual machine, and the second virtual machine and the third virtual machine may be configured to detect objects from the received image data and write the image data from which the objects have been detected in the second shared memory.
- the processor may further execute a fourth virtual machine, the first virtual machine in the processor may be configured to write some other of the image data in the first shared memory, and the third virtual machine may detect an object from the received image data and write the image data from which the object has been detected in the second shared memory.
- the first virtual machine in the processor may be configured to write some of the image data in the first shared memory to be transmitted to the second virtual machine and write some other of the image data in the first shared memory to be transmitted to the third virtual machine
- the second virtual machine may decrypt some of the image data and write the decrypted image data in the second shared memory
- the third virtual machine may image-process the received image data and write the imageprocessed image data in the second shared memory.
- the first virtual machine in the processor may include an input and output server interface and a security manager
- each of the second virtual machine and the third virtual machine may include an input and output client interface
- the input and output server interface may transmit information regarding the first shared memory comprising key data to the input and output client interface after allocation of the first shared memory, and the input and output client interface may access the first shared memory based on the key data.
- the input and output server interface may receive information regarding a first buffer in the first shared memory, the first buffer being empty, may write first data in the first buffer in the first shared memory, and may transmit buffer information of the first buffer to the input and output client interfaces in the second virtual machine and the third virtual machine.
- a reference count of the first buffer may be changed in a first direction based on writing of the first data in the first buffer, and in response to copying of the first data from the first buffer being completed, the reference count of the first buffer may be changed in a second direction, the second direction being opposite the first direction.
- the processor may further execute a legacy virtual machine configured to receive and process Ethernet data
- the first virtual machine may receive, process, and output at least one of vehicle sensor data, position information data, camera image data, audio data, or touch input data.
- the first virtual machine may receive and process wheel speed sensor data of the vehicle, and may transmit the processed wheel speed sensor data to at least one of the second virtual machine or the third virtual machine.
- a display apparatus for vehicle including a first display, a second display, and a signal processing device including a processor configured to perform signal processing for the first display and the second display, wherein the processor is configured to execute first to third virtual machines on a hypervisor in the processor, the second virtual machine is operated for the first display, the third virtual machine is operated for the second display, and the first virtual machine in the processor is configured to share at least some of data with the second virtual machine and the third virtual machine for processing of divided data.
- the signal processing device may be configured to divide and process image data using the plurality of virtual machines, combine the divided and processed image data, and display the composite image data through the plurality of displays.
- a signal processing device includes a processor configured to perform signal processing for a plurality of display located in a vehicle, wherein the processor is configured to execute first to third virtual machines on a hypervisor in the processor, the second virtual machine is operated for a first display among the plurality of display, the third virtual machine is operated for a second display among the plurality of display, and the first virtual machine in the processor is configured to share at least some of data with the second virtual machine and the third virtual machine for processing of divided data. Consequently, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process data.
- high-speed data communication may be performed between the plurality of virtual machines. Furthermore, high-speed data communication may be performed even though the plurality of virtual machines is driven by different operating systems.
- the first virtual machine in the processor may be configured to write some of the data in a first shared memory to be transmitted to the second virtual machine and write some other of the data in the first shared memory to be transmitted to the third virtual machine, and the second virtual machine and the third virtual machine may be configured to process the received data and write the processed data in a second shared memory. Consequently, the plurality of virtual machines may divide and process data.
- the first virtual machine in the processor may be configured to process some other of the data and write the processed data in the second shared memory. That is, the first virtual machine may perform data processing in addition to the second virtual machine and the third virtual machine. Consequently, the plurality of virtual machines may divide and process data.
- the processor may further execute a fourth virtual machine
- the first virtual machine in the processor may be configured to write some other of the data in the first shared memory
- the fourth virtual machine may be configured to process received data and write the processed data in the second shared memory. Consequently, the plurality of virtual machines may divide and process data.
- the first virtual machine in the processor may create command queues for distributed processing of the data in the second virtual machine and the third virtual machine. Consequently, the plurality of virtual machines may divide and process data.
- the first virtual machine in the processor may create one command queue. Consequently, the same data may be synchronized and shared.
- the command queue may include at least one of virtual machine index information, task information, or shared memory information. Consequently, access to the shared memory through the command queue is possible.
- the first virtual machine in the processor may create command queues corresponding to the number of virtual machines for distributed processing of the data. Consequently, each virtual machine-specific command may be transmitted.
- the first virtual machine in the processor may be configured to write some of the image data in the first shared memory to be transmitted to the second virtual machine and write some other of the image data in the first shared memory to be transmitted to the third virtual machine
- the second virtual machine and the third virtual machine may be configured to detect objects from the received image data and write the image data from which the objects have been detected in the second shared memory.
- the plurality of virtual machines for the plurality of displays in the vehicle may divide and process data at the time of object detection. Consequently, rapid data processing is possible.
- the processor may further execute a fourth virtual machine
- the first virtual machine in the processor may be configured to write some other of the image data in the first shared memory
- the third virtual machine may detect an object from the received image data and write the image data from which the object has been detected in the second shared memory.
- the first virtual machine in the processor may be configured to write some of the image data in the first shared memory to be transmitted to the second virtual machine and write some other of the image data in the first shared memory to be transmitted to the third virtual machine
- the second virtual machine may decrypt some of the image data and write the decrypted image data in the second shared memory
- the third virtual machine may image-process the received image data and write the imageprocessed image data in the second shared memory. Consequently, the decrypted image data may be rapidly processed.
- the first virtual machine in the processor may include an input and output server interface and a security manager
- each of the second virtual machine and the third virtual machine may include an input and output client interface. Consequently, high-speed data communication between the plurality of virtual machines may be performed using the input and output server interface and the input and output client interface.
- the input and output server interface may transmit information regarding the first shared memory comprising key data to the input and output client interface after allocation of the first shared memory, and the input and output client interface may access the first shared memory based on the key data. Consequently, 1:N data communication using the shared memory, not 1:1 data communication, may be performed between the virtual machines.
- the input and output server interface may receive information regarding a first buffer in the first shared memory, the first buffer being empty, may write first data in the first buffer in the first shared memory, and may transmit buffer information of the first buffer to the input and output client interfaces in the second virtual machine and the third virtual machine. Consequently, 1:N data communication using the shared memory, not 1:1 data communication, may be performed between the virtual machines.
- a reference count of the first buffer may be changed in a first direction based on writing of the first data in the first buffer, and in response to copying of the first data from the first buffer being completed, the reference count of the first buffer may be changed in a second direction, the second direction being opposite the first direction. After completion of copying, therefore, new data may be written in the first buffer.
- the processor may further execute a legacy virtual machine configured to receive and process Ethernet data
- the first virtual machine may receive, process, and output at least one of vehicle sensor data, position information data, camera image data, audio data, or touch input data. Consequently, data processed only by the legacy virtual machine and data processed by the first virtual machine may be distinguished from each other, whereby data processing may be efficiently performed.
- the first virtual machine may be configured to process most of the data, whereby 1:N data sharing may be achieved.
- the first virtual machine may receive and process wheel speed sensor data of the vehicle, and may transmit the processed wheel speed sensor data to at least one of the second virtual machine or the third virtual machine. Consequently, at least one virtual machine may share the wheel speed sensor data of the vehicle.
- a display apparatus for vehicle includes a first display, a second display, and a signal processing device including a processor configured to perform signal processing for the first display and the second display, wherein the processor is configured to execute first to third virtual machines on a hypervisor in the processor, the second virtual machine is operated for the first display, the third virtual machine is operated for the second display, and the first virtual machine in the processor is configured to share at least some of data with the second virtual machine and the third virtual machine for processing of divided data. Consequently, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process image data.
- the signal processing device may be configured to divide and process image data using the plurality of virtual machines, combine the divided and processed image data, and display the composite image data through the plurality of displays. Consequently, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process image data.
- suffixes “module” and “unit” are given only in consideration of ease in preparation of the specification, and do not have or serve different meanings. Accordingly, the suffixes “module” and “unit” may be used interchangeably.
- FIG. 1A is a view showing an example of the exterior and interior of a vehicle.
- the vehicle 200 is moved by a plurality of wheels 103FR, 103FL, 103RL,... rotated by a power source and a steering wheel 150 configured to adjust an advancing direction of the vehicle 200.
- the vehicle 200 may be provided with a camera 195 configured to acquire an image of the front of the vehicle.
- the vehicle 200 may be further provided therein with a plurality of displays 180a and 180b configured to display images and information.
- a cluster display 180a and an audio video navigation (AVN) display 180b are illustrated as the plurality of displays 180a and 180b.
- APN audio video navigation
- HUD head up display
- the audio video navigation (AVN) display 180b may also be called a center information display.
- the embodiment of the present disclosure proposes a scheme for a display apparatus 100 for vehicle including a plurality of displays 180a and 180b to divide data processing. This will be described with reference to FIG. 12 and subsequent figures.
- the vehicle 200 described in this specification may be a concept including all of a vehicle having an engine as a power source, a hybrid vehicle having an engine and an electric motor as a power source, and an electric vehicle having an electric motor as a power source.
- FIG. 1B is a view showing another example of the interior of the vehicle.
- a cluster display 180a an audio video navigation (AVN) display 180b, rear seat entertainment displays 180c and 180d, and a rear-view mirror display (not shown) may be located in the vehicle.
- APN audio video navigation
- rear seat entertainment displays 180c and 180d rear seat entertainment displays
- rear-view mirror display (not shown) may be located in the vehicle.
- the embodiment of the present disclosure proposes a scheme for a display apparatus 100 for vehicle including a plurality of displays 180a to 180d to divide data processing. This will be described with reference to FIG. 12 and subsequent figures.
- FIG. 2 is a view showing the external appearance of a display apparatus for vehicle according to an embodiment of the present disclosure.
- the display apparatus 100 for vehicle may include a plurality of displays 180a and 180b and a signal processing device 170 configured to perform signal processing in order to display images and information on the plurality of displays 180a and 180b.
- the first display 180a which is one of the plurality of displays 180a and 180b, may be a cluster display 180a configured to display a driving state and operation information
- the second display 180b may be an audio video navigation (AVN) display 180b configured to display vehicle driving information, a navigation map, various kinds of entertainment information, or an image.
- APN audio video navigation
- the signal processing device 170 may have a processor 175 provided therein, and first to third virtual machines 520 to 540 may be executed by a hypervisor 505 in the processor 175.
- the second virtual machine 530 may be operated for the first display 180a, and the third virtual machine 540 may be operated for the second display 180b.
- the first virtual machine 520 in the processor 175 shares at least some of data with the second virtual machine 530 and the third virtual machine 540 for processing of divided data. Consequently, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process data.
- the first virtual machine 520 in the processor 175 may receive and process wheel speed sensor data of the vehicle, and may transmit the processed wheel speed sensor data to at least one of the second virtual machine 530 or the third virtual machine 540. Consequently, at least one virtual machine may share the wheel speed sensor data of the vehicle.
- the first virtual machine 520 in the processor 175 writes some of data in a first shared memory 508a to be transmitted to the second virtual machine 530, and write some other of data in the first shared memory 508a to be transmitted to the third virtual machine 540.
- the second virtual machine 530 and the third virtual machine 540 are configured to process the received data, and write the processed data in a second shared memory 508b. Consequently, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process data.
- the first virtual machine 520 in the processor 175 may be configured to set a shared memory 508 based on the hypervisor 505 for transmission of the same data to the second virtual machine 530 and the third virtual machine 540. Consequently, the first display 180a and the second display 180b in the vehicle may display the same information or the same images in a synchronized state.
- the display apparatus 100 for vehicle may further include a rear seat entertainment (RSE) display 180c configured to display driving state information, simple navigation information, various kinds of entertainment information, or an image.
- RSE rear seat entertainment
- the signal processing device 170 may further execute a fourth virtual machine 550 (see FIG. 12 ), in addition to the first to third virtual machines 520 to 540, on the hypervisor 505 in the processor 175 to control the RSE display 180c.
- a fourth virtual machine 550 see FIG. 12
- some of the plurality of displays 180a to 180c may be operated based on a Linux Operating System (OS), and others may be operated based on a Web Operating System (OS).
- OS Linux Operating System
- OS Web Operating System
- the signal processing device 170 may divide and process data for the displays 180a to 180c configured to be operated under various operating systems.
- the signal processing device 170 may be configured to display the same information or the same image in a synchronized state even if the displays 180a to 180c are operated under various operating systems.
- FIG. 3 illustrates an example of an internal block diagram of the display apparatus for vehicle according to the embodiment of the present disclosure.
- the display apparatus 100 for vehicle may include an input device 110, a transceiver 120, an interface 130, a memory 140, a signal processing device 170, a plurality of displays 180a to 180c, an audio output device 185, and a power supply 190.
- the input device 110 may include a physical button or pad for button input or touch input.
- the input device 110 may include a microphone (not shown) for user voice input.
- the transceiver 120 may wirelessly exchange data with a mobile terminal 800 or a server 900.
- the transceiver 120 may wirelessly exchange data with a mobile terminal of a vehicle driver.
- Any of various data communication schemes such as Bluetooth, Wi-Fi, WIFI Direct, and APIX, may be used as a wireless data communication scheme.
- the transceiver 120 may receive weather information and road traffic situation information, such as transport protocol expert group (TPEG) information, from the mobile terminal 800 or the server 900. To this end, the transceiver 120 may include a mobile communication module (not shown).
- TPEG transport protocol expert group
- the interface 130 may receive sensor information from an electronic control unit (ECU) 770 or a sensor device 760, and may transmit the received information to the signal processing device 170.
- ECU electronice control unit
- the sensor information may include at least one of vehicle direction information, vehicle position information (global positioning system (GPS) information), vehicle angle information, vehicle velocity information, vehicle acceleration information, vehicle inclination information, vehicle forward/backward movement information, battery information, fuel information, tire information, vehicle lamp information, in-vehicle temperature information, and in-vehicle humidity information.
- vehicle position information global positioning system (GPS) information
- GPS global positioning system
- the sensor information may be acquired from a heading sensor, a yaw sensor, a gyro sensor, a position sensor, a vehicle forward/backward movement sensor, a wheel sensor, a vehicle velocity sensor, a car body inclination sensor, a battery sensor, a fuel sensor, a tire sensor, a steering-wheel-rotation-based steering sensor, an in-vehicle temperature sensor, or an in-vehicle humidity sensor.
- the position module may include a GPS module configured to receive GPS information.
- the interface 130 may receive front-of-vehicle image data, side-of-vehicle image data, rear-of-vehicle image data, and obstacle-around-vehicle distance information from a camera 195 or lidar (not shown), and may transmit the received information to the signal processing device 170.
- the memory 140 may store various data necessary for overall operation of the display apparatus 100 for vehicle, such as programs for processing or control of the signal processing device 170.
- the memory 140 may store data about the hypervisor and first to third virtual machines executed by the hypervisor in the processor 175.
- the audio output device 185 may convert an electrical signal from the signal processing device 170 into an audio signal, and may output the audio signal. To this end, the audio output device 185 may include a speaker.
- the power supply 190 may supply power necessary to operate components under control of the signal processing device 170.
- the power supply 190 may receive power from a battery in the vehicle.
- the signal processing device 170 may control overall operation of each device in the display apparatus 100 for vehicle.
- the signal processing device 170 may include a processor 175 configured to perform signal processing for the displays 180a and 180b.
- the processor 175 may execute the first to third virtual machines 520 to 540 on the hypervisor 505 (see FIG. 5 ) in the processor 175.
- the first virtual machine 520 may be called a server virtual machine
- the second and third virtual machines 530 and 540 may be called guest virtual machines.
- the second virtual machine 530 may be operated for the first display 180a, and the third virtual machine 540 may be operated for the second display 180b.
- the first virtual machine 520 in the processor 175 may receive, process, and output at least one of vehicle sensor data, position information data, camera image data, audio data, or touch input data. Data processed only by a legacy virtual machine and data processed by the first virtual machine 520 may be distinguished from each other, whereby data processing may be efficiently performed. In particular, the first virtual machine 520 may be configured to process most of the data, whereby 1:N data sharing may be achieved.
- the first virtual machine 520 may directly receive and process CAN communication data, audio data, radio data, USB data, and wireless communication data for the second and third virtual machines 530 and 540.
- the first virtual machine 520 may transmit the processed data to the second and third virtual machines 530 and 540.
- the first virtual machine 520 among the first to third virtual machines 520 to 540, may receive communication data and external input data, and may perform signal processing, whereby load in signal processing by the other virtual machines may be reduced and 1:N data communication may be achieved, and therefore synchronization at the time of data sharing may be achieved.
- the first virtual machine 520 in the processor 175 writes some of data in the first shared memory 508a to be transmitted to the second virtual machine 530, and write some other of data in the first shared memory 508a to be transmitted to the third virtual machine 540.
- the second virtual machine 530 and the third virtual machine 540 are configured to process the received data, and write the processed data in the second shared memory 508b. Consequently, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process data.
- data may be any one of image data, audio data, navigation data, and voice recognition data.
- the first virtual machine 520 may be configured to process some other of data, and write the processed data in the second shared memory 508b. That is, the first virtual machine 520 may perform data processing in addition to the second virtual machine 530 and the third virtual machine 540.
- the first virtual machine 520 may write some other of data in the first shared memory 508a, and the fourth virtual machine 550 may be configured to process the received data and write the processed data in the second shared memory 508b.
- the first virtual machine 520 may create command queues for distributed processing of data in the second virtual machine 530 and the third virtual machine 540. Consequently, the plurality of virtual machines may divide and process data.
- the first virtual machine 520 in the processor 175 may create one command queue. Consequently, the same data may be synchronized and shared.
- the first virtual machine 520 may create command queues corresponding to the number of virtual machines for distributed processing of data.
- the first virtual machine 520 may be configured to transmit at least some of data to at least one of the second virtual machine 530 or the third virtual machine 540 for distributed processing of data.
- the first virtual machine 520 may allocate the first shared memory 508a for transmitting at least some of data to at least one of the second virtual machine 530 or the third virtual machine 540, and image data processed by the second virtual machine 530 or the third virtual machine 540 may be written in the second shared memory 508b.
- the first virtual machine 520 may be configured to write data in the shared memory 508, whereby the second virtual machine 530 and the third virtual machine 540 share the same data.
- the first virtual machine 520 may be configured to write radio data or wireless communication data in the shared memory 508, whereby the second virtual machine 530 and the third virtual machine 540 share the same data. Consequently, 1:N data sharing may be achieved.
- the first virtual machine 520 may be configured to process most of the data, whereby 1:N data sharing may be achieved.
- the first virtual machine 520 in the processor 175 may be configured to set the shared memory 508 based on the hypervisor 505 in order to transmit the same data to the second virtual machine 530 and the third virtual machine 540.
- the first virtual machine 520 in the processor 175 may transmit the same data to the second virtual machine 530 and the third virtual machine 540 in a synchronized state using the shared memory 508 based on the hypervisor 505. Consequently, the plurality of displays 180a and 180b in the vehicle may display the same images in a synchronized state.
- the signal processing device 170 may be configured to process various signals, such as an audio signal, an image signal, and a data signal.
- the signal processing device 170 may be implemented in the form of a system on chip (SOC).
- FIG. 4 is a view showing a system driven in a signal processing device related to the present disclosure.
- FIG. 4 is a view illustrating that virtual machines are used for the cluster display 180a and the AVN display 180b.
- the system 400 driven in the signal processing device of FIG. 4 illustrates that a cluster virtual machine 430 and an AVN virtual machine 440 are executed through a hypervisor 405 in the processor 175.
- system 400 driven in the signal processing device of FIG. 4 illustrates that a legacy virtual machine 410 is also executed on the hypervisor 405 in the processor 175.
- the legacy virtual machine 410 may include an interface 412 for data communication with the memory 140 and an interface 413 for Ethernet communication.
- the cluster virtual machine 430 may include an interface 431 for CAN communication, an interface 432 for communication with the interface 412 of the legacy virtual machine 410, and an interface 433 for communication with the interface 413 of the legacy virtual machine 410.
- the AVN virtual machine 440 may include an interface 441 for input and output of audio data, radio data, USB data, and wireless communication data, an interface 442 for communication with the interface 412 of the legacy virtual machine 410, and an interface 443 for communication with the interface 413 of the legacy virtual machine 410.
- the cluster virtual machine 430 and the AVN virtual machine 440 must include the interfaces 431 and 432 and the interfaces 441 and 442, respectively, for memory data and Ethernet communication data input and output in the legacy virtual machine 410.
- the present disclosure proposes a scheme for improving the system of FIG. 4 . That is, unlike FIG. 4 , virtual machines are classified into a server virtual machine and guest virtual machines such that various memory data and communication data are input and output not in the guest virtual machines but in the server virtual machine. This will be described with reference to FIG. 5 and subsequent figures.
- FIG. 5 is a view showing an example of a system driven in a signal processing device according to an embodiment of the present disclosure.
- the system 500 of FIG. 5 illustrates that the first virtual machine 520, which is a server virtual machine, the second virtual machine 530, which is a guest virtual machine, and the third virtual machine 540, which is a guest virtual machine, are executed on the hypervisor 505 in the processor 175 of the signal processing device 170.
- the second virtual machine 530 may be a virtual machine for the cluster display 180a
- the third virtual machine 540 may be a virtual machine for the AVN display 180b.
- the second virtual machine 530 and the third virtual machine 540 may be operated for image rendering of the cluster display 180a and the AVN display 180b, respectively.
- system 50 driven in the signal processing device 170 of FIG. 5 illustrates that a legacy virtual machine 510 is also executed on the hypervisor 505 in the processor 175.
- the legacy virtual machine 510 may include an interface 511 for data communication with the memory 140 and Ethernet communication.
- the legacy virtual machine 510 may further include a virtio-backend interface 512 for data communication with the second and third virtual machines 530 and 540.
- the first virtual machine 520 may include an interface 521 for input and output of audio data, radio data, USB data, and wireless communication data and an input and output server interface 522 for data communication with the guest virtual machines.
- the first virtual machine 520 which is a server virtual machine, may provide inputs/outputs (I/O) difficult to virtualize with standard virtualization technology (VirtIO) to a plurality of guest virtual machines, such as the second and third virtual machines 530 and 540.
- I/O inputs/outputs
- VirtualIO standard virtualization technology
- the first virtual machine 520 which is a server virtual machine, may control radio data and audio data at a supervisor level, and may provide the data to a plurality of guest virtual machines, such as the second and third virtual machines 530 and 540.
- the first virtual machine 520 which is a server virtual machine, may be configured to process vehicle data, sensor data, and surroundings-of-vehicle information, and may provide the processed data or information to a plurality of guest virtual machines, such as the second and third virtual machines 530 and 540.
- the first virtual machine 520 may provide supervisory services, such as processing of vehicle data and audio routing management.
- the second virtual machine 530 may include an input and output client interface 532 for data communication with the first virtual machine 520 and APIs 533 configured to control the input and output client interface 532.
- the second virtual machine 530 may include a virtio-backend interface for data communication with the legacy virtual machine 510.
- the second virtual machine 530 may receive memory data by communication with the memory 140 and Ethernet data by Ethernet communication from the virtio-backend interface 512 of the legacy virtual machine 510 through the virtio-backend interface.
- the third virtual machine 540 may include an input and output client interface 542 for data communication with the first virtual machine 520 and APIs 543 configured to control the input and output client interface 542.
- the third virtual machine 540 may include a virtio-backend interface for data communication with the legacy virtual machine 510.
- the third virtual machine 540 may receive memory data by communication with the memory 140 and Ethernet data by Ethernet communication from the virtio-backend interface 512 of the legacy virtual machine 510 through the virtio-backend interface.
- the legacy virtual machine 510 may be provided in the first virtual machine 520, unlike FIG. 5 .
- CAN communication data are input and output only in the first virtual machine 520, but may be provided to a plurality of guest virtual machines, such as the second and third virtual machines 530 and 540, through data processing in the first virtual machine 520. Consequently, 1:N data communication by processing of the first virtual machine 520 may be achieved.
- audio data, radio data, USB data, and wireless communication data are input and output only in the first virtual machine 520, but may be provided to a plurality of guest virtual machines, such as the second and third virtual machines 530 and 540, through data processing in the first virtual machine 520. Consequently, 1:N data communication by processing of the first virtual machine 520 may be achieved.
- the second and third virtual machines 530 and 540 may be operated based on different operating systems.
- the second virtual machine 530 may be operated based on a Linux OS
- the third virtual machine 540 may be operated based on a Web OS.
- the shared memory 508 based on the hypervisor 505 is set for data sharing, even though the second and third virtual machines 530 and 540 are operated based on different operating systems. Even though the second and third virtual machines 530 and 540 are operated based on different operating systems, therefore, the same data or the same images may be shared in a synchronized state. Eventually, the plurality of displays 180a and 180b may display the same data or the same images in a synchronized state.
- FIG. 6 is a view referred to in the description of operation of the system driven in the signal processing device according to the embodiment of the present disclosure
- FIGS. 7A to 18 are views referred to in the description of FIG. 5 or 6 .
- the processor 175 in the signal processing device 170 executes the first to third virtual machines 520 to 540 on the hypervisor 505 in the processor 175, and the first virtual machine 520 in the processor 175 is configured to set the shared memory 508 based on the hypervisor 505 for transmission of the same data to the second and third virtual machines 530 and 540.
- the same image data may be illustrated as the same data. Consequently, the plurality of displays 180a and 180b in the vehicle may display the same images in a synchronized state.
- the processor 175 in the signal processing device 170 executes the first to third virtual machines 520 to 540 on the hypervisor 505 in the processor 175, and the first virtual machine 520 in the processor 175 may transmit the same data to the second and third virtual machines 530 and 540 in a synchronized state using the shared memory 508 based on the hypervisor 505.
- CAN communication data, audio data, radio data, USB data, wireless communication data, position information data, or touch data may be illustrated as the same data. Consequently, the plurality of displays 180a and 180b in the vehicle may display the same data in a synchronized state.
- the legacy virtual machine 510 may transmit memory data from the memory 140 and Ethernet data by Ethernet communication to the second and third virtual machines 530 and 540 in a synchronized state using the shared memory 508 based on the hypervisor 505. That is, it is possible to perform 1:N data communication with respect to the memory data or the Ethernet data, whereby it is possible to transmit the same data in a synchronized state.
- FIG. 7A illustrates that three virtual machines 420, 430, and 440 are operated in a system 400b.
- the first virtual machine 420 which is a Linux-based virtual machine, may include an input and output server interface 422 for data transmission
- the second virtual machine 430 and the third virtual machine 440 may include input and output client interfaces 432 and 442 for data communication with the input and output server interface 422, respectively.
- the first virtual machine 420 must set a first shared memory 408a in a hypervisor 405 in order to transmit first data to the second virtual machine 430, and the first virtual machine 420 must set a second shared memory 408b different from the first shared memory 408a in the hypervisor 405 in order to transmit the same first data to the third virtual machine 440.
- FIG. 7B illustrates that, by the system 400b of FIG. 7A , the second virtual machine 430 displays image data received through the first shared memory 408a on the first display 180a, and the third virtual machine 440 displays image data received through the second shared memory 408b on the second display 180b.
- FIG. 7B illustrates that an image 705a displayed on the first display 180a and an image 705b displayed on the second display 180b are not synchronized with each other and that the image 705b displayed on the second display 180b corresponds to a more previous frame than the image 705a displayed on the first display 180a.
- the present disclosure proposes a scheme for assigning a single shared memory at the time of transmission of the same data. Consequently, 1:N data communication is performed, whereby synchronized data transmission is achieved.
- FIG. 8 illustrates that the first to third virtual machines 520 to 540 are executed on the hypervisor 505 in the processor 175 of the system 500 according to the present disclosure and that the first virtual machine 520 in the processor 175 is configured to set the shared memory 508 based on the hypervisor 505 in order to transmit the same data to the second virtual machine 530 and the third virtual machine 540.
- the first display 180a and the second display 180b in the vehicle may display the same images in a synchronized state.
- high-speed data communication may be performed between the plurality of virtual machines. Furthermore, high-speed data communication may be performed even though the plurality of virtual machines is driven by different operating systems.
- the first virtual machine 520 in the processor 175 may not allocate memories corresponding in number to the virtual machines but may use a single shared memory 508, not memory allocation when transmitting the data processed by the first virtual machine 520 to another virtual machine. Consequently, 1:N data communication using the shared memory 508, not 1:1 data communication, may be performed between the virtual machines.
- the first virtual machine 520 in the processor 175 may include an input and output server interface 522 and a security manager 526.
- the second virtual machine 530 and the third virtual machine 540 may include input and output client interfaces 532 and 542, respectively. Consequently, high-speed data communication between the plurality of virtual machines may be performed using the input and output server interface 522 and the input and output client interfaces 532 and 542.
- the input and output server interface 522 in the first virtual machine 520 may receive requests for transmission of the same data from the input and output client interfaces 532 and 542 in the second virtual machine 530 and the third virtual machine 540, and may transmit shared data to the shared memory 508 through the security manager 526 based thereon.
- FIG. 9A is a view illustrating transmission of shared data in more detail.
- the input and output server interface 522 in the first virtual machine 520 transmits a request for allocation of the shared memory 508 to the security manager 526 (S1).
- the security manager 526 may allocate the shared memory 508 using the hypervisor 505 (S2), and write shared data in the shared memory 508.
- the input and output client interfaces 532 and 542 may transmit a request for connection to the input and output server interface 522 after allocation of the shared memory 508 (S3).
- the input and output server interface 522 transmits information regarding shared memory 508 including key data to the input and output client interfaces 532 and 542 after allocation of the shared memory 508 (S4).
- the key data may be private key data for data access.
- the first virtual machine 520 in the processor 175 may transmit information regarding the shared memory 508 to the second virtual machine 530 and the third virtual machine 540 after setting of the shared memory 508.
- the input and output server interface 522 in the first virtual machine 520 is configured to create a command or a command queue for event processing, other than data, to control distributed processing between the virtual machines (S5).
- the figure illustrates that a command queue is created in a command queue buffer 504 in the hypervisor 505 under control of the input and output server interface 522.
- the present disclosure is not limited thereto, and the command queue may be created in the first virtual machine 520, not the hypervisor 505, under control of the input and output server interface 522.
- the input and output client interfaces 532 and 542 access the command queue buffer 504 to receive the created command queue or information regarding the command queue (S6).
- the created command queues may be the same.
- the input and output client interfaces 532 and 542 may access the shared memory 508 based on the received key data (S5), and may copy the shared data from the shared memory 508 (S7).
- the input and output client interfaces 532 and 542 may access the shared memory 508 based on the same command queues and the same key data (S5), and may copy the shared data from the shared memory 508.
- the second virtual machine 530 and the third virtual machine 540 may access the shared memory 508, and may eventually share the shared data.
- the second virtual machine 530 and the third virtual machine 540 may share the image data, and eventually the plurality of displays 180a and 180b in the vehicle may display the same shared images in a synchronized state.
- FIG. 9B illustrates that, by the system 500b of FIG. 9A , the second virtual machine 530 displays image data received through the shared memory 508 on the first display 180a, and the third virtual machine 540 displays image data received through the shared memory 508 on the second display 180b.
- FIG. 9B illustrates that an image 905 displayed on the first display 180a and an image 905 displayed on the second display 180b are synchronized, whereby the same images are displayed.
- image data processed by the first virtual machine 520 in the processor 175 are transmitted to the second virtual machine 530 and the third virtual machine 540 through the shared memory 508, and the first image 905 displayed on the first display 180a and the second image 905 displayed on the second display 180b based on the image data may be the same. Consequently, the plurality of displays 180a and 180b in the vehicle may display the same images in a synchronized state. In addition, high-speed data communication between the plurality of virtual machines may be performed.
- FIG. 10 is a view showing the output server interface 522 of FIG. 8 in detail.
- a plurality of buffers 507a, 507b, and 507c may be set in the shared memory 508.
- the input and output client interfaces 532 and 542 in the second virtual machine 530 and the third virtual machine 540 may include consumers 533 and 543, respectively.
- the input and output server interface 522 in the first virtual machine 520 may include a producer 1010 configured to create a synchronization object for graphical synchronization, a recvQueue 1020 configured to manage, particularly receive, a queue, a WorkThread 1030 configured to manage a queue and to control operation of the queue, and a sendQueue 1040 configured to manage, particularly transmit, a queue.
- a producer 1010 configured to create a synchronization object for graphical synchronization
- a recvQueue 1020 configured to manage, particularly receive, a queue
- a WorkThread 1030 configured to manage a queue and to control operation of the queue
- a sendQueue 1040 configured to manage, particularly transmit, a queue.
- the producer 1010 receives information regarding the index of an empty index, among the plurality of buffers 507a, 507b, and 507c in the shared memory 508, from the recvQueue 1020 (Sa1). For example, in the case in which the first buffer 507a, among the plurality of buffers 507a, 507b, and 507c, is empty, information regarding the first buffer 507a is received (Sa1).
- the producer 1010 may receive information regarding the first buffer 507a having a reference count refcnt of 0 from the recvQueue 1020.
- the producer 1010 creates a synchronization object for graphical synchronization in order to perform writing in the first buffer 507a (Sa2).
- the producer 1010 writes data about the created synchronization object in the first buffer 507a in the shared memory 508 (Sa3).
- the producer 1010 queues information regarding the first buffer 507a, i.e. a buffer index, to the sendQueue 1040 (Sa4).
- the WorkThread 1030 senses and receives or reads the data (Sa5) .
- the WorkThread 1030 receives the information.
- the WorkThread 1030 waits until a frame about the created synchronization object is completely drawn (Sa6).
- the WorkThread 1030 increases the reference count refcnt of the buffer corresponding to the first buffer 507a by the number of consumers or the number of input and output client interfaces (Sa7).
- the reference count refcnt of the buffer corresponding to the first buffer 507a is increased from 0 to 2.
- the WorkThread 1030 transmits the buffer index to the consumers 533 and 543 (Sa8). For example, information corresponding to the first buffer 507a is transmitted (Sa8).
- the consumers 533 and 543 access the first buffer 507a in the shared memory 508 using the received buffer index to copy data (Sa9).
- the consumers 533 and 543 return the buffer index to the producer 1010 or the WorkThread 1030 after completion of data copying (Sa10).
- the producer 1010 reduces the reference count refcnt of the first buffer 507a by 1 based on information or a buffer index received after completion of data copying for each of the consumers 533 and 543.
- the reference count refcnt of the first buffer 507a is reduced from 2 to 1.
- the reference count refcnt of the first buffer 507a is reduced from 1 to 0.
- the buffer may be used by the producer 1010.
- first frame data may be shared using the first buffer 507a
- second frame data may be shared using the second buffer 507b
- third frame data may be shared using the third buffer 507c
- fourth frame data may be shared using the first buffer 507a again.
- the input and output server interface 522 may receive information regarding the empty first buffer 507a in the shared memory 508, may write the first data in the first buffer 507a in the shared memory 508, and may transmit buffer information of the first buffer 507a to the input and output client interfaces 532 and 542 in the second virtual machine 530 and the third virtual machine 540.
- the reference count of the first buffer 507a may be changed in a first direction (e.g. the reference count being increased) based on writing of the first data in the first buffer 507a, and in the case in which copying of the first data from the first buffer 507a is completed, the reference count of the first buffer 507a may be changed in a second direction, which is opposite the first direction, (e.g. the reference count being decreased).
- the consumers 533 and 543 in the input and output client interfaces 532 and 542 in the second virtual machine 530 and the third virtual machine 540 may change the reference count of the first buffer 507a in the first direction (e.g. the reference count being increased) based on writing of the first data in the first buffer 507a.
- the producer 1010 in the input and output server interface 522 in the first virtual machine 520 may change the reference count of the first buffer 507a in the second direction, which is opposite the first direction, (e.g. the reference count being decreased). After completion of copying, therefore, new data may be written in the first buffer 507a.
- the first virtual machine 520 may write first frame data to third frame data in the first buffer 507a to the third buffer 507c, respectively, and the input and output client interfaces 532 and 542 in the second virtual machine 530 and the third virtual machine 540 may sequentially copy the first frame data to the third frame data from the first buffer 507a to the third buffer 507c.
- the second virtual machine 530 may copy the second frame data from the second buffer 507b. Consequently, synchronization between the second virtual machine 530 and the third virtual machine 540 may be performed at the time of data sharing.
- FIG. 11A is a view illustrating that various drivers DRa, DRb, and DRc are provided in the first virtual machine 520.
- the first virtual machine 520 may include a position information driver DRa configured to process position information, a touch driver DRb configured to process touch input, and a camera driver DRc configured to process an image from the camera.
- a position information driver DRa configured to process position information
- a touch driver DRb configured to process touch input
- a camera driver DRc configured to process an image from the camera.
- the first virtual machine 520 may set the shared memory based on the hypervisor 505 for each of the position information driver DRa, the touch driver DRb, and the camera driver DRc.
- the input and output server interface 522 may set a first shared memory 508a for transmission of image data from the camera driver DRc, and may set a second shared memory 508b for transmission of position information from the position information driver DRa.
- key data of the first shared memory 508a and key data of the second shared memory 508b may be transmitted to the second virtual machine 530 and the third virtual machine 540, and the second virtual machine 530 and the third virtual machine 540 may access the first shared memory 508a and the second shared memory 508b based on the key data of the first shared memory 508a and the key data of the second shared memory 508b.
- FIG. 11B is a view showing a table including key data information.
- the security manager 526 may create key data information for sharing, and may create and register virtual machine information, information for each piece of equipment, allocated memory address information, buffer index information, and the created key data information in the form of a table.
- the input and output client interfaces 532 and 542 in the second virtual machine 530 and the third virtual machine 540 may be connected to the security manager 526, may request and receive key data with reference to the table in the security manager 526, and may access a corresponding one of the shared memories using the received key data.
- the virtual machine information in one case and the virtual machine information in the other case are the same to each other, but the information for each piece of equipment, the allocated memory address information, the buffer index information, and the created key data information in one case and the information for each piece of equipment, the allocated memory address information, the buffer index information, and the created key data information in the other case are different from each other.
- FIG. 12 is a view referred to in describing data division in the signal processing device according to the embodiment of the present disclosure
- FIGS. 13 to 14B are views referred to in describing the operation of FIG. 12 .
- FIG. 12 illustrates that, in order to signal-process image data 1010 acquired through a camera 915, the image data are divided and processed.
- the first virtual machine 520 in the processor 175 may write some 1010b of the image data 1010 in the first buffer in the first shared memory 508a and write some other 1010c of the image data 1010 in the second buffer in the first shared memory 508a.
- the second virtual machine 530 and the third virtual machine 540 may detect objects from the received image data 1010b and 1010c, and may be configured to write the image data from which the objects have been detected in the second shared memory 508b.
- the second virtual machine 530 may write data from which an object BG1 has been detected in the first buffer in the second shared memory 508b
- the third virtual machine 540 may write data from which an object BG2 has been detected in the second buffer in the second shared memory 508b.
- the first virtual machine 520 in the processor 175 may write some 1010b of the image data 1010 in the first buffer in the first shared memory 508a, may write some other 1010c of the image data 1010 in the second buffer in the first shared memory 508a, and write some other 1010d of the image data 1010 in the third buffer in the first shared memory 508a.
- the second virtual machine 530 may write data from which the object BG1 has been detected in the first buffer in the second shared memory 508b
- the third virtual machine 540 may write data from which the object BG2 has been detected in the second buffer in the second shared memory 508b
- the fourth virtual machine 550 may write data from which an object BG3 has been detected in the third buffer in the second shared memory 508b.
- the plurality of virtual machines for the plurality of displays in the vehicle may divide and process data at the time of object detection. Consequently, rapid data processing is possible.
- the image data divided and processed in the signal processing device 170 may be combined, and a composite image 1015 may be displayed on the display. Consequently, the rapidly signal-processed image data may be displayed through the display in the vehicle.
- the figure illustrates that the composite image 1015 is displayed on the RSE display 180c.
- the present disclosure is not limited thereto, and the image may be displayed on any one of the cluster display 180a and the audio video navigation (AVN) display 180b.
- APN audio video navigation
- the first virtual machine 520 transmits image data from the camera driver DRc to the input and output server interface 522 (Sb1).
- the input and output server interface 522 writes the image data in the first shared memory 508a (Sb2) .
- the input and output server interface 522 may create a command or a command queue for event processing in addition to allocation of the shared memory for data transmission (Sb3).
- the first virtual machine 520 in the processor 175 may create command queues 504a and 504b for distributed processing of data in the second virtual machine 530 and the third virtual machine 540.
- the first virtual machine 520 in the processor 175 may create command queues 504a, 504b, and 504c for distributed processing of data in the second virtual machine 530, the third virtual machine 540, and the fourth virtual machine 550.
- the first virtual machine 520 in the processor 175 may create command queues corresponding to the number of virtual machines for distributed processing of data.
- three command queues 504a, 504b, and 504c may be created.
- the first virtual machine 520 in the processor 175 may create one command queue.
- command queue may include at least one of virtual machine index information, task information, or shared memory information.
- the virtual machine index information may indicate information regarding virtual machines that will divide and process data
- the task information may indicate details of a command to be executed at the time of divided processing of data
- the shared memory information may indicate position information of a memory for shared data or buffer information.
- the task information may be classified as object detection, image display, DRM decryption, and userspecific sound detection.
- the input and output server interface 522 may transmit information regarding the first shared memory 508a including key data to input and output client interfaces 532, 542, and 552.
- the input and output client interfaces 532, 542, and 552 in the second to fourth virtual machines 530, 540, and 550 may copy data in the first shared memory 508a based on the command queues 504a, 504b, and 504c, respectively (Sb4).
- the input and output client interfaces 532, 542, and 552 in the second to fourth virtual machines may copy some data 1010b, 1010c, and 1010d in FIG. 12 , respectively.
- signal processing modules DDa, DDb, and DDc in the second to fourth virtual machines 530, 540, and 550 perform signal processing (Sb5).
- the signal processing modules DDa, DDb, and DDc in the second to fourth virtual machines 530, 540, and 550 may detect different objects.
- the input and output client interfaces 532b, 542b, and 552b in the second to fourth virtual machines 530, 540, and 550 perform writing in the second shared memory 508b (Sb6).
- a completion message may be transmitted to the input and output server interface 522 (Sb7).
- the input and output server interface 522 may transmit a third command queue 504c including a command for image display to the fourth virtual machine 550 (Sb8).
- the fourth virtual machine 550 may receive the third command queue 504c, and may be configured to display write the image data written in the second shared memory 508b through the RSE display 180c (Sb9) .
- FIG. 14A is similar to FIG. 12 except that a fifth virtual machine 560 for HUD display is further executed.
- the image data are divided and processed.
- the first virtual machine 520 in the processor 175 may write some 1010b of the image data 1010 in the first buffer in the first shared memory 508a, may write some other 1010c of the image data 1010 in the second buffer in the first shared memory 508a, and write some other 1010d of the image data 1010 in the third buffer in the first shared memory 508a.
- the input and output client interfaces 532, 542, and 552 in the second to fourth virtual machines 530, 540, and 550 may receive some image data, may perform signal processing, and write the signal-processed image data in the second shared memory 508b.
- the image data from which an object has been detected may be written in the second shared memory 508b.
- the first virtual machine 520 may transmit a command queue 504d for image display to the fifth virtual machine 560, and the fifth virtual machine 560 may copy the image date written in the second shared memory 508b and may be configured to display a composite image 1015 on a HUD display 180f. Consequently, image display based on division of signal processing may be performed.
- FIG. 14B is a view showing that data division is performed for decryption, not object detection, unlike FIG. 14A .
- the first virtual machine 520 in the processor 175 may write encrypted image data 1032 in the first buffer in the first shared memory 508a, and write unencrypted image data 1034 in the second buffer in the first shared memory 508a.
- the first command queue 504a and the second command queue 504b are used for data transmission to the second virtual machine 530 and the third virtual machine 540.
- the second virtual machine 530 decrypts the partial encrypted area and write the decrypted data in the second buffer in the second shared memory 508b.
- the third virtual machine 540 signal-processes the unencrypted image data 1034 and write the signal-processed data in the first buffer in the second shared memory 508b.
- the fourth virtual machine 550 may be configured to copy the image data written in the second shared memory 508b and display decrypted image 1036 on the HUD display 180f based on the third command queue 504c. Consequently, image display based on division of signal processing may be performed.
- FIG. 15A illustrates that the same images 1210a and 1210b are displayed respectively on the first display 180a and the second display 180b in the vehicle.
- FIG. 15B illustrates that the same images 1210a, 1210b, 1210c, and 1210d are displayed respectively on the first display 180a, the second display 180b, the third display 180c, and the fourth display 180d in the vehicle.
- the third virtual machine 540 is operated for rendering of the second display 180b
- the fourth virtual machine (not shown) is operated for rendering of the third display 180c
- a fifth virtual machine (not shown) is operated for rendering of the fourth display 180d
- 1:N data communication by the first virtual machine 520 is performed, whereby synchronized images are displayed, as shown in the figure.
- FIG. 15C illustrates that a display 180e is formed on a rear-view mirror in the vehicle in order to display an image 1210e.
- the same images 1210a, 1210b, and 1210e may be displayed respectively on the first display 180a, the second display 180b, and the rear-view mirror display 180e in the vehicle.
- the second virtual machine 530 is operated for rendering of the first display 180a
- the third virtual machine 540 is operated for rendering of the second display 180b
- the fourth virtual machine (not shown) is operated for rendering of the rear-view mirror display 180e
- 1:N data communication by the first virtual machine 520 is performed, whereby synchronized images are displayed, as shown in the figure.
- the same images 1210a, 1210b, 1210c, 1210d, and 1210e may be displayed respectively on the first display 180a, the second display 180b, the third display 180c, the fourth display 180d, and the rear-view mirror display 180e in the vehicle.
- FIG. 16 illustrates that a HUD display 180f is disposed in the vehicle.
- the display apparatus 100 for vehicle may include a first display 180a corresponding to an instrument cluster, a second display 180b corresponding to an AVN, and a HUD display 180f.
- the signal processing device 170 may be configured to display an image created by combining first image data processed by the first virtual machine 520, second image data processed by the second virtual machine 530, and third image data processed by the third virtual machine 540 on the third display 180c.
- the figure illustrates that the first virtual machine 520 processes rear-view mirror image data Imga and side mirror image data Imgb, the second virtual machine 530 processes front camera image data Imgc, the third virtual machine 540 processes navigation image data Imgd, and an image created by combining the four kinds of image data is displayed on the HUD display 180f, which is a third display.
- the figure illustrates that an image 1300 corresponding to the front camera image data Imgc is a background, and images 1310, 1320, and 1330 corresponding respectively to the navigation image data Imgd, the side mirror image data Imgb, and the rear-view mirror image data Imga are displayed together. As a result, synchronized images may be displayed together. Consequently, it is possible for a user to check all images around the vehicle through the HUD display 180f.
- FIG. 17 illustrates that portions Oba and OBb of an image displayed on the second display 180b are processed and displayed on the third display 180c and the fourth display 180d, which are RSE displays, in a synchronized state.
- the figure illustrates that a partial image OBad is displayed on the third display 180c and another partial image OBbd is displayed on the fourth display 180d in a synchronized state.
- FIG. 18 illustrates that an image 1510b displayed on the second display 180b is processed and displayed on the third display 180c and the fourth display 180d, which are RSE displays, in a synchronized state.
- the figure illustrates that the same image 1510c is displayed on the third display 180c and the same image 1510d is displayed on the fourth display 180d in a synchronized state.
- the first virtual machine 520 in the signal processing device 170 may receive first touch input on the first display 180a through the second virtual machine 530 and may transmit image data processed based on the first touch input to the second virtual machine 530 and the third virtual machine 540 through the shared memory 508, and the first display 180a and the second display 180b may display the same images corresponding to the first touch input. Consequently, the plurality of displays 180a and 180b in the vehicle may display the same images in a synchronized state.
Abstract
Description
- The present disclosure relates to a signal processing device and a display apparatus for vehicle including the same, and more particularly to a signal processing device capable of dividing and processing data using a plurality of virtual machines for a plurality of displays in a vehicle and a display apparatus for vehicle including the same.
- A vehicle is an apparatus that a driver moves in a desired direction. A representative example of the vehicle is a car.
- Meanwhile, a display apparatus for vehicle is located in the vehicle for convenience of users who use the vehicle.
- For example, a display is disposed in a cluster in order to display various kinds of information. Meanwhile, in order to display vehicle driving information, various displays, such as an audio video navigation (AVN) display, are located in the vehicle, in addition to the cluster.
- In the case in which the number of displays in the display apparatus for vehicle is increased, however, signal processing for the displays is complicated.
- Particularly, in the case in which the same images are displayed on a plurality of displays, synchronization is important.
- An object of the present disclosure is to provide a signal processing device capable of dividing and processing data using a plurality of virtual machines for a plurality of displays in a vehicle and a display apparatus for vehicle including the same.
- Another object of the present disclosure is to provide a signal processing device capable of performing high-speed data communication between a plurality of virtual machines and a display apparatus for vehicle including the same.
- A further object of the present disclosure is to provide a signal processing device capable of performing high-speed data communication even though a plurality of virtual machines is driven by different operating systems and a display apparatus for vehicle including the same.
- In accordance with an aspect of the present disclosure, the above and other objects can be accomplished by the provision of a signal processing device including a processor configured to perform signal processing for a plurality of display located in a vehicle, wherein the processor is configured to execute first to third virtual machines on a hypervisor in the processor, the second virtual machine is operated for a first display among the plurality of display, the third virtual machine is operated for a second display among the plurality of display, and the first virtual machine in the processor is configured to share at least some of data with the second virtual machine and the third virtual machine for processing of divided data.
- Meanwhile, the first virtual machine in the processor may be configured to write some of the data in a first shared memory to be transmitted to the second virtual machine and write some other of the data in the first shared memory to be transmitted to the third virtual machine, and the second virtual machine and the third virtual machine may be configured to process the received data and write the processed data in a second shared memory.
- Meanwhile, the first virtual machine in the processor may be configured to process some other of the data and write the processed data in the second shared memory.
- Meanwhile, the processor may further execute a fourth virtual machine, the first virtual machine in the processor may be configured to write some other of the data in the first shared memory, and the fourth virtual machine may be configured to process received data and write the processed data in the second shared memory.
- Meanwhile, the first virtual machine in the processor may create command queues for distributed processing of the data in the second virtual machine and the third virtual machine.
- Meanwhile, when the second virtual machine and the third virtual machine share the same data, the first virtual machine in the processor may create one command queue.
- Meanwhile, the command queue may include at least one of virtual machine index information, task information, or shared memory information.
- Meanwhile, the first virtual machine in the processor may create command queues corresponding to the number of virtual machines for distributed processing of the data.
- Meanwhile, in response to the data being image data, the first virtual machine in the processor may be configured to write some of the image data in the first shared memory to be transmitted to the second virtual machine and write some other of the image data in the first shared memory to be transmitted to the third virtual machine, and the second virtual machine and the third virtual machine may be configured to detect objects from the received image data and write the image data from which the objects have been detected in the second shared memory.
- Meanwhile, the processor may further execute a fourth virtual machine, the first virtual machine in the processor may be configured to write some other of the image data in the first shared memory, and the third virtual machine may detect an object from the received image data and write the image data from which the object has been detected in the second shared memory.
- Meanwhile, in response to the data being image data, the first virtual machine in the processor may be configured to write some of the image data in the first shared memory to be transmitted to the second virtual machine and write some other of the image data in the first shared memory to be transmitted to the third virtual machine, the second virtual machine may decrypt some of the image data and write the decrypted image data in the second shared memory, and the third virtual machine may image-process the received image data and write the imageprocessed image data in the second shared memory.
- Meanwhile, the first virtual machine in the processor may include an input and output server interface and a security manager, and each of the second virtual machine and the third virtual machine may include an input and output client interface.
- Meanwhile, the input and output server interface may transmit information regarding the first shared memory comprising key data to the input and output client interface after allocation of the first shared memory, and the input and output client interface may access the first shared memory based on the key data.
- Meanwhile, the input and output server interface may receive information regarding a first buffer in the first shared memory, the first buffer being empty, may write first data in the first buffer in the first shared memory, and may transmit buffer information of the first buffer to the input and output client interfaces in the second virtual machine and the third virtual machine.
- Meanwhile, a reference count of the first buffer may be changed in a first direction based on writing of the first data in the first buffer, and in response to copying of the first data from the first buffer being completed, the reference count of the first buffer may be changed in a second direction, the second direction being opposite the first direction.
- Meanwhile, the processor may further execute a legacy virtual machine configured to receive and process Ethernet data, and the first virtual machine may receive, process, and output at least one of vehicle sensor data, position information data, camera image data, audio data, or touch input data.
- Meanwhile, the first virtual machine may receive and process wheel speed sensor data of the vehicle, and may transmit the processed wheel speed sensor data to at least one of the second virtual machine or the third virtual machine.
- In accordance with another aspect of the present disclosure, there is provided a display apparatus for vehicle, the display apparatus including a first display, a second display, and a signal processing device including a processor configured to perform signal processing for the first display and the second display, wherein the processor is configured to execute first to third virtual machines on a hypervisor in the processor, the second virtual machine is operated for the first display, the third virtual machine is operated for the second display, and the first virtual machine in the processor is configured to share at least some of data with the second virtual machine and the third virtual machine for processing of divided data.
- Meanwhile, the signal processing device may be configured to divide and process image data using the plurality of virtual machines, combine the divided and processed image data, and display the composite image data through the plurality of displays.
- A signal processing device according to an embodiment of the present disclosure includes a processor configured to perform signal processing for a plurality of display located in a vehicle, wherein the processor is configured to execute first to third virtual machines on a hypervisor in the processor, the second virtual machine is operated for a first display among the plurality of display, the third virtual machine is operated for a second display among the plurality of display, and the first virtual machine in the processor is configured to share at least some of data with the second virtual machine and the third virtual machine for processing of divided data. Consequently, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process data.
- Meanwhile, high-speed data communication may be performed between the plurality of virtual machines. Furthermore, high-speed data communication may be performed even though the plurality of virtual machines is driven by different operating systems.
- Meanwhile, the first virtual machine in the processor may be configured to write some of the data in a first shared memory to be transmitted to the second virtual machine and write some other of the data in the first shared memory to be transmitted to the third virtual machine, and the second virtual machine and the third virtual machine may be configured to process the received data and write the processed data in a second shared memory. Consequently, the plurality of virtual machines may divide and process data.
- Meanwhile, the first virtual machine in the processor may be configured to process some other of the data and write the processed data in the second shared memory. That is, the first virtual machine may perform data processing in addition to the second virtual machine and the third virtual machine. Consequently, the plurality of virtual machines may divide and process data.
- Meanwhile, the processor may further execute a fourth virtual machine, the first virtual machine in the processor may be configured to write some other of the data in the first shared memory, and the fourth virtual machine may be configured to process received data and write the processed data in the second shared memory. Consequently, the plurality of virtual machines may divide and process data.
- Meanwhile, the first virtual machine in the processor may create command queues for distributed processing of the data in the second virtual machine and the third virtual machine. Consequently, the plurality of virtual machines may divide and process data.
- Meanwhile, when the second virtual machine and the third virtual machine share the same data, the first virtual machine in the processor may create one command queue. Consequently, the same data may be synchronized and shared.
- Meanwhile, the command queue may include at least one of virtual machine index information, task information, or shared memory information. Consequently, access to the shared memory through the command queue is possible.
- Meanwhile, the first virtual machine in the processor may create command queues corresponding to the number of virtual machines for distributed processing of the data. Consequently, each virtual machine-specific command may be transmitted.
- Meanwhile, in response to the data being image data, the first virtual machine in the processor may be configured to write some of the image data in the first shared memory to be transmitted to the second virtual machine and write some other of the image data in the first shared memory to be transmitted to the third virtual machine, and the second virtual machine and the third virtual machine may be configured to detect objects from the received image data and write the image data from which the objects have been detected in the second shared memory. As a result, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process data at the time of object detection. Consequently, rapid data processing is possible.
- Meanwhile, the processor may further execute a fourth virtual machine, the first virtual machine in the processor may be configured to write some other of the image data in the first shared memory, and the third virtual machine may detect an object from the received image data and write the image data from which the object has been detected in the second shared memory. As a result, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process data at the time of object detection. Consequently, rapid data processing is possible.
- Meanwhile, in response to the data being image data, the first virtual machine in the processor may be configured to write some of the image data in the first shared memory to be transmitted to the second virtual machine and write some other of the image data in the first shared memory to be transmitted to the third virtual machine, the second virtual machine may decrypt some of the image data and write the decrypted image data in the second shared memory, and the third virtual machine may image-process the received image data and write the imageprocessed image data in the second shared memory. Consequently, the decrypted image data may be rapidly processed.
- Meanwhile, the first virtual machine in the processor may include an input and output server interface and a security manager, and each of the second virtual machine and the third virtual machine may include an input and output client interface. Consequently, high-speed data communication between the plurality of virtual machines may be performed using the input and output server interface and the input and output client interface.
- Meanwhile, the input and output server interface may transmit information regarding the first shared memory comprising key data to the input and output client interface after allocation of the first shared memory, and the input and output client interface may access the first shared memory based on the key data. Consequently, 1:N data communication using the shared memory, not 1:1 data communication, may be performed between the virtual machines.
- Meanwhile, the input and output server interface may receive information regarding a first buffer in the first shared memory, the first buffer being empty, may write first data in the first buffer in the first shared memory, and may transmit buffer information of the first buffer to the input and output client interfaces in the second virtual machine and the third virtual machine. Consequently, 1:N data communication using the shared memory, not 1:1 data communication, may be performed between the virtual machines.
- Meanwhile, a reference count of the first buffer may be changed in a first direction based on writing of the first data in the first buffer, and in response to copying of the first data from the first buffer being completed, the reference count of the first buffer may be changed in a second direction, the second direction being opposite the first direction. After completion of copying, therefore, new data may be written in the first buffer.
- Meanwhile, the processor may further execute a legacy virtual machine configured to receive and process Ethernet data, and the first virtual machine may receive, process, and output at least one of vehicle sensor data, position information data, camera image data, audio data, or touch input data. Consequently, data processed only by the legacy virtual machine and data processed by the first virtual machine may be distinguished from each other, whereby data processing may be efficiently performed.
- In particular, the first virtual machine may be configured to process most of the data, whereby 1:N data sharing may be achieved.
- Meanwhile, the first virtual machine may receive and process wheel speed sensor data of the vehicle, and may transmit the processed wheel speed sensor data to at least one of the second virtual machine or the third virtual machine. Consequently, at least one virtual machine may share the wheel speed sensor data of the vehicle.
- Meanwhile, a display apparatus for vehicle according to an embodiment of the present disclosure includes a first display, a second display, and a signal processing device including a processor configured to perform signal processing for the first display and the second display, wherein the processor is configured to execute first to third virtual machines on a hypervisor in the processor, the second virtual machine is operated for the first display, the third virtual machine is operated for the second display, and the first virtual machine in the processor is configured to share at least some of data with the second virtual machine and the third virtual machine for processing of divided data. Consequently, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process image data. Meanwhile, the signal processing device may be configured to divide and process image data using the plurality of virtual machines, combine the divided and processed image data, and display the composite image data through the plurality of displays. Consequently, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process image data.
-
- FIG. 1A is a view showing an example of the exterior and interior of a vehicle;
-
FIG. 1B is a view showing another example of the interior of the vehicle; -
FIG. 2 is a view showing the external appearance of a display apparatus for vehicle according to an embodiment of the present disclosure; -
FIG. 3 illustrates an example of an internal block diagram of the display apparatus for vehicle ofFIG. 2 ; -
FIG. 4 is a view showing a system driven in a signal processing device related to the present disclosure; -
FIG. 5 is a view showing an example of a system driven in a signal processing device according to an embodiment of the present disclosure; -
FIG. 6 is a view referred to in the description of operation of the system driven in the signal processing device according to the embodiment of the present disclosure; and -
FIGS. 7A to 18 are views referred to in the description ofFIG. 5 or6 . - Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.
- With respect to constituent elements used in the following description, suffixes "module" and "unit" are given only in consideration of ease in preparation of the specification, and do not have or serve different meanings. Accordingly, the suffixes "module" and "unit" may be used interchangeably.
- FIG. 1A is a view showing an example of the exterior and interior of a vehicle.
- Referring to the figure, the
vehicle 200 is moved by a plurality of wheels 103FR, 103FL, 103RL,... rotated by a power source and asteering wheel 150 configured to adjust an advancing direction of thevehicle 200. - Meanwhile, the
vehicle 200 may be provided with acamera 195 configured to acquire an image of the front of the vehicle. - Meanwhile, the
vehicle 200 may be further provided therein with a plurality ofdisplays - In FIG. 1A, a
cluster display 180a and an audio video navigation (AVN)display 180b are illustrated as the plurality ofdisplays - Meanwhile, the audio video navigation (AVN)
display 180b may also be called a center information display. - The embodiment of the present disclosure proposes a scheme for a
display apparatus 100 for vehicle including a plurality ofdisplays FIG. 12 and subsequent figures. - Meanwhile, the
vehicle 200 described in this specification may be a concept including all of a vehicle having an engine as a power source, a hybrid vehicle having an engine and an electric motor as a power source, and an electric vehicle having an electric motor as a power source. -
FIG. 1B is a view showing another example of the interior of the vehicle. - Referring to the figure, a
cluster display 180a, an audio video navigation (AVN)display 180b, rear seat entertainment displays 180c and 180d, and a rear-view mirror display (not shown) may be located in the vehicle. - The embodiment of the present disclosure proposes a scheme for a
display apparatus 100 for vehicle including a plurality ofdisplays 180a to 180d to divide data processing. This will be described with reference toFIG. 12 and subsequent figures. -
FIG. 2 is a view showing the external appearance of a display apparatus for vehicle according to an embodiment of the present disclosure. - The
display apparatus 100 for vehicle according to the embodiment of the present disclosure may include a plurality ofdisplays signal processing device 170 configured to perform signal processing in order to display images and information on the plurality ofdisplays - The
first display 180a, which is one of the plurality ofdisplays cluster display 180a configured to display a driving state and operation information, and thesecond display 180b may be an audio video navigation (AVN)display 180b configured to display vehicle driving information, a navigation map, various kinds of entertainment information, or an image. - The
signal processing device 170 may have aprocessor 175 provided therein, and first to thirdvirtual machines 520 to 540 may be executed by ahypervisor 505 in theprocessor 175. - The second
virtual machine 530 may be operated for thefirst display 180a, and the thirdvirtual machine 540 may be operated for thesecond display 180b. - Meanwhile, the first
virtual machine 520 in theprocessor 175 shares at least some of data with the secondvirtual machine 530 and the thirdvirtual machine 540 for processing of divided data. Consequently, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process data. - Meanwhile, the first
virtual machine 520 in theprocessor 175 may receive and process wheel speed sensor data of the vehicle, and may transmit the processed wheel speed sensor data to at least one of the secondvirtual machine 530 or the thirdvirtual machine 540. Consequently, at least one virtual machine may share the wheel speed sensor data of the vehicle. - Meanwhile, the first
virtual machine 520 in theprocessor 175 writes some of data in a first sharedmemory 508a to be transmitted to the secondvirtual machine 530, and write some other of data in the first sharedmemory 508a to be transmitted to the thirdvirtual machine 540. The secondvirtual machine 530 and the thirdvirtual machine 540 are configured to process the received data, and write the processed data in a second sharedmemory 508b. Consequently, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process data. - Meanwhile, the first
virtual machine 520 in theprocessor 175 may be configured to set a sharedmemory 508 based on thehypervisor 505 for transmission of the same data to the secondvirtual machine 530 and the thirdvirtual machine 540. Consequently, thefirst display 180a and thesecond display 180b in the vehicle may display the same information or the same images in a synchronized state. - Meanwhile, the
display apparatus 100 for vehicle according to the embodiment of the present disclosure may further include a rear seat entertainment (RSE)display 180c configured to display driving state information, simple navigation information, various kinds of entertainment information, or an image. - The
signal processing device 170 may further execute a fourth virtual machine 550 (seeFIG. 12 ), in addition to the first to thirdvirtual machines 520 to 540, on thehypervisor 505 in theprocessor 175 to control theRSE display 180c. - Consequently, it is possible to control
various displays 180a to 180c using a singlesignal processing device 170. - Meanwhile, some of the plurality of
displays 180a to 180c may be operated based on a Linux Operating System (OS), and others may be operated based on a Web Operating System (OS). - The
signal processing device 170 according to the embodiment of the present disclosure may divide and process data for thedisplays 180a to 180c configured to be operated under various operating systems. - Meanwhile, the
signal processing device 170 according to the embodiment of the present disclosure may be configured to display the same information or the same image in a synchronized state even if thedisplays 180a to 180c are operated under various operating systems. -
FIG. 3 illustrates an example of an internal block diagram of the display apparatus for vehicle according to the embodiment of the present disclosure. - Referring to the figure, the
display apparatus 100 for vehicle according to the embodiment of the present disclosure may include aninput device 110, atransceiver 120, aninterface 130, amemory 140, asignal processing device 170, a plurality ofdisplays 180a to 180c, anaudio output device 185, and apower supply 190. - The
input device 110 may include a physical button or pad for button input or touch input. - Meanwhile, the
input device 110 may include a microphone (not shown) for user voice input. - The
transceiver 120 may wirelessly exchange data with amobile terminal 800 or aserver 900. - In particular, the
transceiver 120 may wirelessly exchange data with a mobile terminal of a vehicle driver. Any of various data communication schemes, such as Bluetooth, Wi-Fi, WIFI Direct, and APIX, may be used as a wireless data communication scheme. - The
transceiver 120 may receive weather information and road traffic situation information, such as transport protocol expert group (TPEG) information, from themobile terminal 800 or theserver 900. To this end, thetransceiver 120 may include a mobile communication module (not shown). - The
interface 130 may receive sensor information from an electronic control unit (ECU) 770 or asensor device 760, and may transmit the received information to thesignal processing device 170. - Here, the sensor information may include at least one of vehicle direction information, vehicle position information (global positioning system (GPS) information), vehicle angle information, vehicle velocity information, vehicle acceleration information, vehicle inclination information, vehicle forward/backward movement information, battery information, fuel information, tire information, vehicle lamp information, in-vehicle temperature information, and in-vehicle humidity information.
- The sensor information may be acquired from a heading sensor, a yaw sensor, a gyro sensor, a position sensor, a vehicle forward/backward movement sensor, a wheel sensor, a vehicle velocity sensor, a car body inclination sensor, a battery sensor, a fuel sensor, a tire sensor, a steering-wheel-rotation-based steering sensor, an in-vehicle temperature sensor, or an in-vehicle humidity sensor. Meanwhile, the position module may include a GPS module configured to receive GPS information.
- Meanwhile, the
interface 130 may receive front-of-vehicle image data, side-of-vehicle image data, rear-of-vehicle image data, and obstacle-around-vehicle distance information from acamera 195 or lidar (not shown), and may transmit the received information to thesignal processing device 170. - The
memory 140 may store various data necessary for overall operation of thedisplay apparatus 100 for vehicle, such as programs for processing or control of thesignal processing device 170. - For example, the
memory 140 may store data about the hypervisor and first to third virtual machines executed by the hypervisor in theprocessor 175. - The
audio output device 185 may convert an electrical signal from thesignal processing device 170 into an audio signal, and may output the audio signal. To this end, theaudio output device 185 may include a speaker. - The
power supply 190 may supply power necessary to operate components under control of thesignal processing device 170. In particular, thepower supply 190 may receive power from a battery in the vehicle. - The
signal processing device 170 may control overall operation of each device in thedisplay apparatus 100 for vehicle. - For example, the
signal processing device 170 may include aprocessor 175 configured to perform signal processing for thedisplays - The
processor 175 may execute the first to thirdvirtual machines 520 to 540 on the hypervisor 505 (seeFIG. 5 ) in theprocessor 175. - Among the first to third
virtual machines 520 to 540 (seeFIG. 5 ), the firstvirtual machine 520 may be called a server virtual machine, and the second and thirdvirtual machines - The second
virtual machine 530 may be operated for thefirst display 180a, and the thirdvirtual machine 540 may be operated for thesecond display 180b. - For example, the first
virtual machine 520 in theprocessor 175 may receive, process, and output at least one of vehicle sensor data, position information data, camera image data, audio data, or touch input data. Data processed only by a legacy virtual machine and data processed by the firstvirtual machine 520 may be distinguished from each other, whereby data processing may be efficiently performed. In particular, the firstvirtual machine 520 may be configured to process most of the data, whereby 1:N data sharing may be achieved. - As another example, the first
virtual machine 520 may directly receive and process CAN communication data, audio data, radio data, USB data, and wireless communication data for the second and thirdvirtual machines - The first
virtual machine 520 may transmit the processed data to the second and thirdvirtual machines - Consequently, only the first
virtual machine 520, among the first to thirdvirtual machines 520 to 540, may receive communication data and external input data, and may perform signal processing, whereby load in signal processing by the other virtual machines may be reduced and 1:N data communication may be achieved, and therefore synchronization at the time of data sharing may be achieved. - Meanwhile, the first
virtual machine 520 in theprocessor 175 writes some of data in the first sharedmemory 508a to be transmitted to the secondvirtual machine 530, and write some other of data in the first sharedmemory 508a to be transmitted to the thirdvirtual machine 540. The secondvirtual machine 530 and the thirdvirtual machine 540 are configured to process the received data, and write the processed data in the second sharedmemory 508b. Consequently, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process data. - At this time, data may be any one of image data, audio data, navigation data, and voice recognition data.
- Meanwhile, the first
virtual machine 520 may be configured to process some other of data, and write the processed data in the second sharedmemory 508b. That is, the firstvirtual machine 520 may perform data processing in addition to the secondvirtual machine 530 and the thirdvirtual machine 540. - Meanwhile, when a fourth
virtual machine 550 configured to be operated for thethird display 180c is executed in theprocessor 175, the firstvirtual machine 520 may write some other of data in the first sharedmemory 508a, and the fourthvirtual machine 550 may be configured to process the received data and write the processed data in the second sharedmemory 508b. - Meanwhile, the first
virtual machine 520 may create command queues for distributed processing of data in the secondvirtual machine 530 and the thirdvirtual machine 540. Consequently, the plurality of virtual machines may divide and process data. - Meanwhile, when the second
virtual machine 530 and the thirdvirtual machine 540 share the same data, the firstvirtual machine 520 in theprocessor 175 may create one command queue. Consequently, the same data may be synchronized and shared. - Meanwhile, the first
virtual machine 520 may create command queues corresponding to the number of virtual machines for distributed processing of data. - Meanwhile, the first
virtual machine 520 may be configured to transmit at least some of data to at least one of the secondvirtual machine 530 or the thirdvirtual machine 540 for distributed processing of data. - For example, the first
virtual machine 520 may allocate the first sharedmemory 508a for transmitting at least some of data to at least one of the secondvirtual machine 530 or the thirdvirtual machine 540, and image data processed by the secondvirtual machine 530 or the thirdvirtual machine 540 may be written in the second sharedmemory 508b. - Meanwhile, the first
virtual machine 520 may be configured to write data in the sharedmemory 508, whereby the secondvirtual machine 530 and the thirdvirtual machine 540 share the same data. - For example, the first
virtual machine 520 may be configured to write radio data or wireless communication data in the sharedmemory 508, whereby the secondvirtual machine 530 and the thirdvirtual machine 540 share the same data. Consequently, 1:N data sharing may be achieved. - Eventually, the first
virtual machine 520 may be configured to process most of the data, whereby 1:N data sharing may be achieved. - Meanwhile, the first
virtual machine 520 in theprocessor 175 may be configured to set the sharedmemory 508 based on thehypervisor 505 in order to transmit the same data to the secondvirtual machine 530 and the thirdvirtual machine 540. - That is, the first
virtual machine 520 in theprocessor 175 may transmit the same data to the secondvirtual machine 530 and the thirdvirtual machine 540 in a synchronized state using the sharedmemory 508 based on thehypervisor 505. Consequently, the plurality ofdisplays - Meanwhile, the
signal processing device 170 may be configured to process various signals, such as an audio signal, an image signal, and a data signal. To this end, thesignal processing device 170 may be implemented in the form of a system on chip (SOC). -
FIG. 4 is a view showing a system driven in a signal processing device related to the present disclosure. - Referring to the figure,
FIG. 4 is a view illustrating that virtual machines are used for thecluster display 180a and theAVN display 180b. - The
system 400 driven in the signal processing device ofFIG. 4 illustrates that a clustervirtual machine 430 and an AVNvirtual machine 440 are executed through ahypervisor 405 in theprocessor 175. - Meanwhile, the
system 400 driven in the signal processing device ofFIG. 4 illustrates that a legacyvirtual machine 410 is also executed on thehypervisor 405 in theprocessor 175. - The legacy
virtual machine 410 may include aninterface 412 for data communication with thememory 140 and aninterface 413 for Ethernet communication. - Meanwhile, the cluster
virtual machine 430 may include aninterface 431 for CAN communication, aninterface 432 for communication with theinterface 412 of the legacyvirtual machine 410, and aninterface 433 for communication with theinterface 413 of the legacyvirtual machine 410. - Meanwhile, the AVN
virtual machine 440 may include aninterface 441 for input and output of audio data, radio data, USB data, and wireless communication data, aninterface 442 for communication with theinterface 412 of the legacyvirtual machine 410, and aninterface 443 for communication with theinterface 413 of the legacyvirtual machine 410. - In the
system 400, there is a disadvantage in that CAN communication data are input and output only in the clustervirtual machine 430, whereby the CAN communication data cannot be utilized in the AVNvirtual machine 440. - Also, in the
system 400 ofFIG. 4 , there is a disadvantage in that audio data, radio data, USB data, and wireless communication data are input and output only in the AVNvirtual machine 440, whereby these data cannot be utilized in the clustervirtual machine 430. - Meanwhile, there is a disadvantage in that the cluster
virtual machine 430 and the AVNvirtual machine 440 must include theinterfaces interfaces virtual machine 410. - Therefore, the present disclosure proposes a scheme for improving the system of
FIG. 4 . That is, unlikeFIG. 4 , virtual machines are classified into a server virtual machine and guest virtual machines such that various memory data and communication data are input and output not in the guest virtual machines but in the server virtual machine. This will be described with reference toFIG. 5 and subsequent figures. -
FIG. 5 is a view showing an example of a system driven in a signal processing device according to an embodiment of the present disclosure. - Referring to the figure, the
system 500 ofFIG. 5 illustrates that the firstvirtual machine 520, which is a server virtual machine, the secondvirtual machine 530, which is a guest virtual machine, and the thirdvirtual machine 540, which is a guest virtual machine, are executed on thehypervisor 505 in theprocessor 175 of thesignal processing device 170. - The second
virtual machine 530 may be a virtual machine for thecluster display 180a, and the thirdvirtual machine 540 may be a virtual machine for theAVN display 180b. - That is, the second
virtual machine 530 and the thirdvirtual machine 540 may be operated for image rendering of thecluster display 180a and theAVN display 180b, respectively. - Meanwhile, the system 50 driven in the
signal processing device 170 ofFIG. 5 illustrates that a legacyvirtual machine 510 is also executed on thehypervisor 505 in theprocessor 175. - The legacy
virtual machine 510 may include aninterface 511 for data communication with thememory 140 and Ethernet communication. - Meanwhile, the legacy
virtual machine 510 may further include a virtio-backend interface 512 for data communication with the second and thirdvirtual machines - The first
virtual machine 520 may include aninterface 521 for input and output of audio data, radio data, USB data, and wireless communication data and an input andoutput server interface 522 for data communication with the guest virtual machines. - That is, the first
virtual machine 520, which is a server virtual machine, may provide inputs/outputs (I/O) difficult to virtualize with standard virtualization technology (VirtIO) to a plurality of guest virtual machines, such as the second and thirdvirtual machines - Meanwhile, the first
virtual machine 520, which is a server virtual machine, may control radio data and audio data at a supervisor level, and may provide the data to a plurality of guest virtual machines, such as the second and thirdvirtual machines - Meanwhile, the first
virtual machine 520, which is a server virtual machine, may be configured to process vehicle data, sensor data, and surroundings-of-vehicle information, and may provide the processed data or information to a plurality of guest virtual machines, such as the second and thirdvirtual machines - Meanwhile, the first
virtual machine 520 may provide supervisory services, such as processing of vehicle data and audio routing management. - Next, the second
virtual machine 530 may include an input andoutput client interface 532 for data communication with the firstvirtual machine 520 andAPIs 533 configured to control the input andoutput client interface 532. - In addition, the second
virtual machine 530 may include a virtio-backend interface for data communication with the legacyvirtual machine 510. - The second
virtual machine 530 may receive memory data by communication with thememory 140 and Ethernet data by Ethernet communication from the virtio-backend interface 512 of the legacyvirtual machine 510 through the virtio-backend interface. - Next, the third
virtual machine 540 may include an input andoutput client interface 542 for data communication with the firstvirtual machine 520 andAPIs 543 configured to control the input andoutput client interface 542. - In addition, the third
virtual machine 540 may include a virtio-backend interface for data communication with the legacyvirtual machine 510. - The third
virtual machine 540 may receive memory data by communication with thememory 140 and Ethernet data by Ethernet communication from the virtio-backend interface 512 of the legacyvirtual machine 510 through the virtio-backend interface. - Meanwhile, the legacy
virtual machine 510 may be provided in the firstvirtual machine 520, unlikeFIG. 5 . - In the
system 500, CAN communication data are input and output only in the firstvirtual machine 520, but may be provided to a plurality of guest virtual machines, such as the second and thirdvirtual machines virtual machine 520. Consequently, 1:N data communication by processing of the firstvirtual machine 520 may be achieved. - Also, in the
system 500 ofFIG. 5 , audio data, radio data, USB data, and wireless communication data are input and output only in the firstvirtual machine 520, but may be provided to a plurality of guest virtual machines, such as the second and thirdvirtual machines virtual machine 520. Consequently, 1:N data communication by processing of the firstvirtual machine 520 may be achieved. - Meanwhile, in the
system 500 ofFIG. 5 , the second and thirdvirtual machines - For example, the second
virtual machine 530 may be operated based on a Linux OS, and the thirdvirtual machine 540 may be operated based on a Web OS. - In the first
virtual machine 520, the sharedmemory 508 based on thehypervisor 505 is set for data sharing, even though the second and thirdvirtual machines virtual machines displays -
FIG. 6 is a view referred to in the description of operation of the system driven in the signal processing device according to the embodiment of the present disclosure, andFIGS. 7A to 18 are views referred to in the description ofFIG. 5 or6 . - First, in the
system 500 ofFIG. 6 , theprocessor 175 in thesignal processing device 170 executes the first to thirdvirtual machines 520 to 540 on thehypervisor 505 in theprocessor 175, and the firstvirtual machine 520 in theprocessor 175 is configured to set the sharedmemory 508 based on thehypervisor 505 for transmission of the same data to the second and thirdvirtual machines - For example, the same image data may be illustrated as the same data. Consequently, the plurality of
displays - Meanwhile, in the
system 500 ofFIG. 6 , theprocessor 175 in thesignal processing device 170 executes the first to thirdvirtual machines 520 to 540 on thehypervisor 505 in theprocessor 175, and the firstvirtual machine 520 in theprocessor 175 may transmit the same data to the second and thirdvirtual machines memory 508 based on thehypervisor 505. - For example, CAN communication data, audio data, radio data, USB data, wireless communication data, position information data, or touch data may be illustrated as the same data. Consequently, the plurality of
displays - Meanwhile, although not shown in
FIG. 6 , the legacyvirtual machine 510 may transmit memory data from thememory 140 and Ethernet data by Ethernet communication to the second and thirdvirtual machines memory 508 based on thehypervisor 505. That is, it is possible to perform 1:N data communication with respect to the memory data or the Ethernet data, whereby it is possible to transmit the same data in a synchronized state. -
FIG. 7A illustrates that threevirtual machines system 400b. - Referring to the figure, the first
virtual machine 420, which is a Linux-based virtual machine, may include an input andoutput server interface 422 for data transmission, and the secondvirtual machine 430 and the thirdvirtual machine 440 may include input and output client interfaces 432 and 442 for data communication with the input andoutput server interface 422, respectively. - For example, the first
virtual machine 420 must set a first sharedmemory 408a in ahypervisor 405 in order to transmit first data to the secondvirtual machine 430, and the firstvirtual machine 420 must set a second sharedmemory 408b different from the first sharedmemory 408a in thehypervisor 405 in order to transmit the same first data to the thirdvirtual machine 440. - In the case in which the same first data are transmitted, there is a disadvantage in that resources are wasted and synchronization is not easy in the case in which an additional shared memory is used, as shown in
FIG. 7A . -
FIG. 7B illustrates that, by thesystem 400b ofFIG. 7A , the secondvirtual machine 430 displays image data received through the first sharedmemory 408a on thefirst display 180a, and the thirdvirtual machine 440 displays image data received through the second sharedmemory 408b on thesecond display 180b. -
FIG. 7B illustrates that animage 705a displayed on thefirst display 180a and animage 705b displayed on thesecond display 180b are not synchronized with each other and that theimage 705b displayed on thesecond display 180b corresponds to a more previous frame than theimage 705a displayed on thefirst display 180a. - When the first
virtual machine 420 transits the same image data, as described above, synchronization cannot be performed at the time of display of the image, as shown inFIG. 7B , in the case in which an additional shared memory is used, as shown inFIG. 7A . - In order to solve this problem, the present disclosure proposes a scheme for assigning a single shared memory at the time of transmission of the same data. Consequently, 1:N data communication is performed, whereby synchronized data transmission is achieved.
-
FIG. 8 illustrates that the first to thirdvirtual machines 520 to 540 are executed on thehypervisor 505 in theprocessor 175 of thesystem 500 according to the present disclosure and that the firstvirtual machine 520 in theprocessor 175 is configured to set the sharedmemory 508 based on thehypervisor 505 in order to transmit the same data to the secondvirtual machine 530 and the thirdvirtual machine 540. - Consequently, the
first display 180a and thesecond display 180b in the vehicle may display the same images in a synchronized state. - Meanwhile, high-speed data communication may be performed between the plurality of virtual machines. Furthermore, high-speed data communication may be performed even though the plurality of virtual machines is driven by different operating systems.
- Meanwhile, the first
virtual machine 520 in theprocessor 175 may not allocate memories corresponding in number to the virtual machines but may use a single sharedmemory 508, not memory allocation when transmitting the data processed by the firstvirtual machine 520 to another virtual machine. Consequently, 1:N data communication using the sharedmemory 508, not 1:1 data communication, may be performed between the virtual machines. - Meanwhile, the first
virtual machine 520 in theprocessor 175 may include an input andoutput server interface 522 and asecurity manager 526. - Meanwhile, the second
virtual machine 530 and the thirdvirtual machine 540 may include input and output client interfaces 532 and 542, respectively. Consequently, high-speed data communication between the plurality of virtual machines may be performed using the input andoutput server interface 522 and the input and output client interfaces 532 and 542. - The input and
output server interface 522 in the firstvirtual machine 520 may receive requests for transmission of the same data from the input and output client interfaces 532 and 542 in the secondvirtual machine 530 and the thirdvirtual machine 540, and may transmit shared data to the sharedmemory 508 through thesecurity manager 526 based thereon. -
FIG. 9A is a view illustrating transmission of shared data in more detail. - Referring to the figure, in order to transmit shared data, the input and
output server interface 522 in the firstvirtual machine 520 transmits a request for allocation of the sharedmemory 508 to the security manager 526 (S1). - Subsequently, the
security manager 526 may allocate the sharedmemory 508 using the hypervisor 505 (S2), and write shared data in the sharedmemory 508. - Meanwhile, the input and output client interfaces 532 and 542 may transmit a request for connection to the input and
output server interface 522 after allocation of the shared memory 508 (S3). - Meanwhile, the input and
output server interface 522 transmits information regarding sharedmemory 508 including key data to the input and output client interfaces 532 and 542 after allocation of the shared memory 508 (S4). At this time, the key data may be private key data for data access. - Meanwhile, the first
virtual machine 520 in theprocessor 175 may transmit information regarding the sharedmemory 508 to the secondvirtual machine 530 and the thirdvirtual machine 540 after setting of the sharedmemory 508. - Subsequently, the input and
output server interface 522 in the firstvirtual machine 520 is configured to create a command or a command queue for event processing, other than data, to control distributed processing between the virtual machines (S5). - The figure illustrates that a command queue is created in a command queue buffer 504 in the
hypervisor 505 under control of the input andoutput server interface 522. However, the present disclosure is not limited thereto, and the command queue may be created in the firstvirtual machine 520, not thehypervisor 505, under control of the input andoutput server interface 522. - Subsequently, the input and output client interfaces 532 and 542 access the command queue buffer 504 to receive the created command queue or information regarding the command queue (S6).
- For example, when the commands transmitted to the input and output client interfaces 532 and 542 are the same, the created command queues may be the same.
- As another example, when the commands transmitted to the input and output client interfaces 532 and 542 are different from each other, different command queues may be transmitted to the input and output client interfaces 532 and 542.
- Subsequently, the input and output client interfaces 532 and 542 may access the shared
memory 508 based on the received key data (S5), and may copy the shared data from the shared memory 508 (S7). - Particularly, when the input and output client interfaces 532 and 542 receive the same shared data, the input and output client interfaces 532 and 542 may access the shared
memory 508 based on the same command queues and the same key data (S5), and may copy the shared data from the sharedmemory 508. - Consequently, the second
virtual machine 530 and the thirdvirtual machine 540 may access the sharedmemory 508, and may eventually share the shared data. - For example, in the case in which the shared data are image data, the second
virtual machine 530 and the thirdvirtual machine 540 may share the image data, and eventually the plurality ofdisplays -
FIG. 9B illustrates that, by thesystem 500b ofFIG. 9A , the secondvirtual machine 530 displays image data received through the sharedmemory 508 on thefirst display 180a, and the thirdvirtual machine 540 displays image data received through the sharedmemory 508 on thesecond display 180b. -
FIG. 9B illustrates that animage 905 displayed on thefirst display 180a and animage 905 displayed on thesecond display 180b are synchronized, whereby the same images are displayed. - That is, image data processed by the first
virtual machine 520 in theprocessor 175 are transmitted to the secondvirtual machine 530 and the thirdvirtual machine 540 through the sharedmemory 508, and thefirst image 905 displayed on thefirst display 180a and thesecond image 905 displayed on thesecond display 180b based on the image data may be the same. Consequently, the plurality ofdisplays -
FIG. 10 is a view showing theoutput server interface 522 ofFIG. 8 in detail. - Referring to the figure, a plurality of
buffers memory 508. - Meanwhile, the input and output client interfaces 532 and 542 in the second
virtual machine 530 and the thirdvirtual machine 540 may includeconsumers - Meanwhile, the input and
output server interface 522 in the firstvirtual machine 520 may include aproducer 1010 configured to create a synchronization object for graphical synchronization, arecvQueue 1020 configured to manage, particularly receive, a queue, aWorkThread 1030 configured to manage a queue and to control operation of the queue, and asendQueue 1040 configured to manage, particularly transmit, a queue. - First, the
producer 1010 receives information regarding the index of an empty index, among the plurality ofbuffers memory 508, from the recvQueue 1020 (Sa1). For example, in the case in which thefirst buffer 507a, among the plurality ofbuffers first buffer 507a is received (Sa1). - In particular, the
producer 1010 may receive information regarding thefirst buffer 507a having a reference count refcnt of 0 from therecvQueue 1020. - Next, the
producer 1010 creates a synchronization object for graphical synchronization in order to perform writing in thefirst buffer 507a (Sa2). - Next, the
producer 1010 writes data about the created synchronization object in thefirst buffer 507a in the shared memory 508 (Sa3). - Next, the
producer 1010 queues information regarding thefirst buffer 507a, i.e. a buffer index, to the sendQueue 1040 (Sa4). - Next, in the case in which data are input to the
sendQueue 1040, which is periodically monitored, theWorkThread 1030 senses and receives or reads the data (Sa5) . - For example, in the case in which information regarding the
first buffer 507a is input to thesendQueue 1040, which is monitored, theWorkThread 1030 receives the information. - Meanwhile, the
WorkThread 1030 waits until a frame about the created synchronization object is completely drawn (Sa6). - Next, the
WorkThread 1030 increases the reference count refcnt of the buffer corresponding to thefirst buffer 507a by the number of consumers or the number of input and output client interfaces (Sa7). - For example, since the number of
consumers first buffer 507a is increased from 0 to 2. - Next, the
WorkThread 1030 transmits the buffer index to theconsumers 533 and 543 (Sa8). For example, information corresponding to thefirst buffer 507a is transmitted (Sa8). - Next, the
consumers first buffer 507a in the sharedmemory 508 using the received buffer index to copy data (Sa9). - Next, the
consumers producer 1010 or theWorkThread 1030 after completion of data copying (Sa10). - Next, the
producer 1010 reduces the reference count refcnt of thefirst buffer 507a by 1 based on information or a buffer index received after completion of data copying for each of theconsumers - For example, in the case in which the
first consumer 533 completes data copying, the reference count refcnt of thefirst buffer 507a is reduced from 2 to 1. - Subsequently, in the case in which the
second consumer 543 completes data copying, the reference count refcnt of thefirst buffer 507a is reduced from 1 to 0. - Meanwhile, in the case in which the reference count refcnt of the
first buffer 507a is 0, the buffer may be used by theproducer 1010. - Similarly, first frame data may be shared using the
first buffer 507a, then second frame data may be shared using thesecond buffer 507b, then third frame data may be shared using thethird buffer 507c, and then fourth frame data may be shared using thefirst buffer 507a again. - That is, the input and
output server interface 522 may receive information regarding the emptyfirst buffer 507a in the sharedmemory 508, may write the first data in thefirst buffer 507a in the sharedmemory 508, and may transmit buffer information of thefirst buffer 507a to the input and output client interfaces 532 and 542 in the secondvirtual machine 530 and the thirdvirtual machine 540. - In particular, the reference count of the
first buffer 507a may be changed in a first direction (e.g. the reference count being increased) based on writing of the first data in thefirst buffer 507a, and in the case in which copying of the first data from thefirst buffer 507a is completed, the reference count of thefirst buffer 507a may be changed in a second direction, which is opposite the first direction, (e.g. the reference count being decreased). - For example, the
consumers virtual machine 530 and the thirdvirtual machine 540 may change the reference count of thefirst buffer 507a in the first direction (e.g. the reference count being increased) based on writing of the first data in thefirst buffer 507a. - Meanwhile, in the case in which copying of the first data from the
first buffer 507a is completed, theproducer 1010 in the input andoutput server interface 522 in the firstvirtual machine 520 may change the reference count of thefirst buffer 507a in the second direction, which is opposite the first direction, (e.g. the reference count being decreased). After completion of copying, therefore, new data may be written in thefirst buffer 507a. - Meanwhile, the first
virtual machine 520 may write first frame data to third frame data in thefirst buffer 507a to thethird buffer 507c, respectively, and the input and output client interfaces 532 and 542 in the secondvirtual machine 530 and the thirdvirtual machine 540 may sequentially copy the first frame data to the third frame data from thefirst buffer 507a to thethird buffer 507c. - Meanwhile, after the input and
output client interface 542 in the thirdvirtual machine 540 copying the first frame data from thefirst buffer 507a is completed, the secondvirtual machine 530 may copy the second frame data from thesecond buffer 507b. Consequently, synchronization between the secondvirtual machine 530 and the thirdvirtual machine 540 may be performed at the time of data sharing. -
FIG. 11A is a view illustrating that various drivers DRa, DRb, and DRc are provided in the firstvirtual machine 520. - Referring to the figure, the first
virtual machine 520 may include a position information driver DRa configured to process position information, a touch driver DRb configured to process touch input, and a camera driver DRc configured to process an image from the camera. - Consequently, the first
virtual machine 520 may set the shared memory based on thehypervisor 505 for each of the position information driver DRa, the touch driver DRb, and the camera driver DRc. - Meanwhile, the input and
output server interface 522 may set a first sharedmemory 508a for transmission of image data from the camera driver DRc, and may set a second sharedmemory 508b for transmission of position information from the position information driver DRa. - Meanwhile, key data of the first shared
memory 508a and key data of the second sharedmemory 508b may be transmitted to the secondvirtual machine 530 and the thirdvirtual machine 540, and the secondvirtual machine 530 and the thirdvirtual machine 540 may access the first sharedmemory 508a and the second sharedmemory 508b based on the key data of the first sharedmemory 508a and the key data of the second sharedmemory 508b. - In the figure, the case in which data from the position information driver DRa and data from the camera driver DRc are shared is illustrated, and therefore two shared
memories hypervisor 505. - Since a shared memory is set for each of different kinds of shared data, as described above, it is possible to prevent data confusion at the time of data sharing and to perform high-speed data communication between the plurality of virtual machines.
-
FIG. 11B is a view showing a table including key data information. - Referring to the figure, when different kinds of shared data are shared, the
security manager 526 may create key data information for sharing, and may create and register virtual machine information, information for each piece of equipment, allocated memory address information, buffer index information, and the created key data information in the form of a table. - Meanwhile, the input and output client interfaces 532 and 542 in the second
virtual machine 530 and the thirdvirtual machine 540 may be connected to thesecurity manager 526, may request and receive key data with reference to the table in thesecurity manager 526, and may access a corresponding one of the shared memories using the received key data. - Meanwhile, in the case in which data from the camera driver DRc are shared through the first shared
memory 508a and in the case in which data from the position information driver DRa are shared through the second sharedmemory 508b, as shown inFIG. 11B , the virtual machine information in one case and the virtual machine information in the other case are the same to each other, but the information for each piece of equipment, the allocated memory address information, the buffer index information, and the created key data information in one case and the information for each piece of equipment, the allocated memory address information, the buffer index information, and the created key data information in the other case are different from each other. - Consequently, it is possible to prevent data confusion at the time of sharing of different kinds of data and to perform high-speed data communication between the plurality of virtual machines.
-
FIG. 12 is a view referred to in describing data division in the signal processing device according to the embodiment of the present disclosure, andFIGS. 13 to 14B are views referred to in describing the operation ofFIG. 12 . - First,
FIG. 12 illustrates that, in order to signal-process image data 1010 acquired through a camera 915, the image data are divided and processed. - First, when data are
image data 1010, the firstvirtual machine 520 in theprocessor 175 may write some 1010b of theimage data 1010 in the first buffer in the first sharedmemory 508a and write some other 1010c of theimage data 1010 in the second buffer in the first sharedmemory 508a. - The second
virtual machine 530 and the thirdvirtual machine 540 may detect objects from the receivedimage data memory 508b. - Specifically, the second
virtual machine 530 may write data from which an object BG1 has been detected in the first buffer in the second sharedmemory 508b, and the thirdvirtual machine 540 may write data from which an object BG2 has been detected in the second buffer in the second sharedmemory 508b. - Meanwhile, when the fourth
virtual machine 550 for theRSE display 180c is executed in addition to the secondvirtual machine 530 and the thirdvirtual machine 540, as shown in the figure, the firstvirtual machine 520 in theprocessor 175 may write some 1010b of theimage data 1010 in the first buffer in the first sharedmemory 508a, may write some other 1010c of theimage data 1010 in the second buffer in the first sharedmemory 508a, and write some other 1010d of theimage data 1010 in the third buffer in the first sharedmemory 508a. - The second
virtual machine 530 may write data from which the object BG1 has been detected in the first buffer in the second sharedmemory 508b, the thirdvirtual machine 540 may write data from which the object BG2 has been detected in the second buffer in the second sharedmemory 508b, and the fourthvirtual machine 550 may write data from which an object BG3 has been detected in the third buffer in the second sharedmemory 508b. - As a result, the plurality of virtual machines for the plurality of displays in the vehicle may divide and process data at the time of object detection. Consequently, rapid data processing is possible.
- Meanwhile, the image data divided and processed in the
signal processing device 170 may be combined, and acomposite image 1015 may be displayed on the display. Consequently, the rapidly signal-processed image data may be displayed through the display in the vehicle. - The figure illustrates that the
composite image 1015 is displayed on theRSE display 180c. However, the present disclosure is not limited thereto, and the image may be displayed on any one of thecluster display 180a and the audio video navigation (AVN)display 180b. - Referring to
FIG. 13 , for divided processing of the image data ofFIG. 12 , the firstvirtual machine 520 transmits image data from the camera driver DRc to the input and output server interface 522 (Sb1). - Subsequently, the input and
output server interface 522 writes the image data in the first sharedmemory 508a (Sb2) . - Meanwhile, the input and
output server interface 522 may create a command or a command queue for event processing in addition to allocation of the shared memory for data transmission (Sb3). - For example, the first
virtual machine 520 in theprocessor 175 may createcommand queues virtual machine 530 and the thirdvirtual machine 540. - As another example, as shown in
FIG. 13 , the firstvirtual machine 520 in theprocessor 175 may createcommand queues virtual machine 530, the thirdvirtual machine 540, and the fourthvirtual machine 550. - Meanwhile, the first
virtual machine 520 in theprocessor 175 may create command queues corresponding to the number of virtual machines for distributed processing of data. - That is, when the number of virtual machines for distributed processing of data is three, i.e. the second
virtual machine 530, the thirdvirtual machine 540, and the fourthvirtual machine 550, as shown inFIG. 13 , threecommand queues - Meanwhile, when the second
virtual machine 530 and the thirdvirtual machine 540 share the same data, as shown inFIG. 9A , the firstvirtual machine 520 in theprocessor 175 may create one command queue. - Meanwhile, the command queue may include at least one of virtual machine index information, task information, or shared memory information.
- The virtual machine index information may indicate information regarding virtual machines that will divide and process data, the task information may indicate details of a command to be executed at the time of divided processing of data, and the shared memory information may indicate position information of a memory for shared data or buffer information.
- Meanwhile, the task information may be classified as object detection, image display, DRM decryption, and userspecific sound detection.
- Meanwhile, after allocation of the first shared
memory 508a, the input andoutput server interface 522 may transmit information regarding the first sharedmemory 508a including key data to input and output client interfaces 532, 542, and 552. - Meanwhile, the input and output client interfaces 532, 542, and 552 in the second to fourth
virtual machines memory 508a based on thecommand queues - That is, the input and output client interfaces 532, 542, and 552 in the second to fourth virtual machines may copy some
data FIG. 12 , respectively. - Subsequently, signal processing modules DDa, DDb, and DDc in the second to fourth
virtual machines - For example, the signal processing modules DDa, DDb, and DDc in the second to fourth
virtual machines - Subsequently, the input and output client interfaces 532b, 542b, and 552b in the second to fourth
virtual machines memory 508b (Sb6). - Subsequently, when writing of data processed by the second to fourth
virtual machines memory 508b is completed, a completion message may be transmitted to the input and output server interface 522 (Sb7). - The input and
output server interface 522 may transmit athird command queue 504c including a command for image display to the fourth virtual machine 550 (Sb8). - Subsequently, the fourth
virtual machine 550 may receive thethird command queue 504c, and may be configured to display write the image data written in the second sharedmemory 508b through theRSE display 180c (Sb9) . -
FIG. 14A is similar toFIG. 12 except that a fifthvirtual machine 560 for HUD display is further executed. - Referring to the figure, in order to signal-process the image data acquired through the camera 915, the image data are divided and processed.
- When data are
image data 1010, the firstvirtual machine 520 in theprocessor 175 may write some 1010b of theimage data 1010 in the first buffer in the first sharedmemory 508a, may write some other 1010c of theimage data 1010 in the second buffer in the first sharedmemory 508a, and write some other 1010d of theimage data 1010 in the third buffer in the first sharedmemory 508a. - The input and output client interfaces 532, 542, and 552 in the second to fourth
virtual machines memory 508b. - In particular, the image data from which an object has been detected may be written in the second shared
memory 508b. - Subsequently, the first
virtual machine 520 may transmit acommand queue 504d for image display to the fifthvirtual machine 560, and the fifthvirtual machine 560 may copy the image date written in the second sharedmemory 508b and may be configured to display acomposite image 1015 on a HUD display 180f. Consequently, image display based on division of signal processing may be performed. -
FIG. 14B is a view showing that data division is performed for decryption, not object detection, unlikeFIG. 14A . - Referring to the figure, when a partial encrypted area is included in
image data 1030, the firstvirtual machine 520 in theprocessor 175 may writeencrypted image data 1032 in the first buffer in the first sharedmemory 508a, and writeunencrypted image data 1034 in the second buffer in the first sharedmemory 508a. - The
first command queue 504a and thesecond command queue 504b are used for data transmission to the secondvirtual machine 530 and the thirdvirtual machine 540. - Based on the
first command queue 504a, the secondvirtual machine 530 decrypts the partial encrypted area and write the decrypted data in the second buffer in the second sharedmemory 508b. - Meanwhile, based on the
second command queue 504b, the thirdvirtual machine 540 signal-processes theunencrypted image data 1034 and write the signal-processed data in the first buffer in the second sharedmemory 508b. - Meanwhile, when the first
virtual machine 520 outputs thethird command queue 504c for image display, the fourthvirtual machine 550 may be configured to copy the image data written in the second sharedmemory 508b and display decryptedimage 1036 on the HUD display 180f based on thethird command queue 504c. Consequently, image display based on division of signal processing may be performed. -
FIG. 15A illustrates that thesame images first display 180a and thesecond display 180b in the vehicle. - For example, in the case in which the second
virtual machine 530 is operated for rendering of thefirst display 180a and the thirdvirtual machine 540 is operated for rendering of thesecond display 180b, 1:N data communication by the firstvirtual machine 520 is performed, whereby synchronized images are displayed, as shown in the figure. -
FIG. 15B illustrates that thesame images first display 180a, thesecond display 180b, thethird display 180c, and thefourth display 180d in the vehicle. - For example, in the case in which the second
virtual machine 530 is operated for rendering of thefirst display 180a, the thirdvirtual machine 540 is operated for rendering of thesecond display 180b, the fourth virtual machine (not shown) is operated for rendering of thethird display 180c, and a fifth virtual machine (not shown) is operated for rendering of thefourth display 180d, 1:N data communication by the firstvirtual machine 520 is performed, whereby synchronized images are displayed, as shown in the figure. -
FIG. 15C illustrates that adisplay 180e is formed on a rear-view mirror in the vehicle in order to display animage 1210e. - For example, in the case in which
FIGS. 15A and15C are linked to each other, thesame images first display 180a, thesecond display 180b, and the rear-view mirror display 180e in the vehicle. - Meanwhile, in the case in which the second
virtual machine 530 is operated for rendering of thefirst display 180a, the thirdvirtual machine 540 is operated for rendering of thesecond display 180b, and the fourth virtual machine (not shown) is operated for rendering of the rear-view mirror display 180e, 1:N data communication by the firstvirtual machine 520 is performed, whereby synchronized images are displayed, as shown in the figure. - As another example, in the case in which
FIGS. 15B and15C are linked to each other, thesame images first display 180a, thesecond display 180b, thethird display 180c, thefourth display 180d, and the rear-view mirror display 180e in the vehicle. -
FIG. 16 illustrates that a HUD display 180f is disposed in the vehicle. - Referring to the figure, the
display apparatus 100 for vehicle may include afirst display 180a corresponding to an instrument cluster, asecond display 180b corresponding to an AVN, and a HUD display 180f. - The
signal processing device 170 may be configured to display an image created by combining first image data processed by the firstvirtual machine 520, second image data processed by the secondvirtual machine 530, and third image data processed by the thirdvirtual machine 540 on thethird display 180c. - The figure illustrates that the first
virtual machine 520 processes rear-view mirror image data Imga and side mirror image data Imgb, the secondvirtual machine 530 processes front camera image data Imgc, the thirdvirtual machine 540 processes navigation image data Imgd, and an image created by combining the four kinds of image data is displayed on the HUD display 180f, which is a third display. - The figure illustrates that an
image 1300 corresponding to the front camera image data Imgc is a background, andimages -
FIG. 17 illustrates that portions Oba and OBb of an image displayed on thesecond display 180b are processed and displayed on thethird display 180c and thefourth display 180d, which are RSE displays, in a synchronized state. - The figure illustrates that a partial image OBad is displayed on the
third display 180c and another partial image OBbd is displayed on thefourth display 180d in a synchronized state. -
FIG. 18 illustrates that animage 1510b displayed on thesecond display 180b is processed and displayed on thethird display 180c and thefourth display 180d, which are RSE displays, in a synchronized state. - The figure illustrates that the
same image 1510c is displayed on thethird display 180c and thesame image 1510d is displayed on thefourth display 180d in a synchronized state. - Meanwhile, in the
display apparatus 100 for vehicle according to the embodiment of the present disclosure, the firstvirtual machine 520 in thesignal processing device 170 may receive first touch input on thefirst display 180a through the secondvirtual machine 530 and may transmit image data processed based on the first touch input to the secondvirtual machine 530 and the thirdvirtual machine 540 through the sharedmemory 508, and thefirst display 180a and thesecond display 180b may display the same images corresponding to the first touch input. Consequently, the plurality ofdisplays - It will be apparent that, although the preferred embodiments have been shown and described above, the present disclosure is not limited to the above-described specific embodiments, and various modifications and variations can be made by those skilled in the art without departing from the gist of the appended claims. Thus, it is intended that the modifications and variations should not be understood independently of the technical spirit or prospect of the present disclosure.
Claims (19)
- A signal processing device comprising a processor configured to perform signal processing for a plurality of display located in a vehicle, whereinthe processor is configured to execute first to third virtual machines on a hypervisor in the processor,the second virtual machine is operated for a first display among the plurality of display,the third virtual machine is operated for a second display among the plurality of display, andthe first virtual machine in the processor is configured to share at least some of data with the second virtual machine and the third virtual machine for processing of divided data.
- The signal processing device of claim 1, whereinthe first virtual machine in the processor is configured to write some of the data in a first shared memory to be transmitted to the second virtual machine and write some other of the data in the first shared memory to be transmitted to the third virtual machine, andthe second virtual machine and the third virtual machine are configured to process the received data and write the processed data in a second shared memory.
- The signal processing device of claim 1, wherein the first virtual machine in the processor is configured to process some other of the data and write the processed data in the second shared memory.
- The signal processing device of claim 1, whereinthe processor further executes a fourth virtual machine,the first virtual machine in the processor is configured to write some other of the data in the first shared memory, andthe fourth virtual machine is configured to process received data and write the processed data in the second shared memory.
- The signal processing device of claim 1, wherein the first virtual machine in the processor is configured to create command queues for distributed processing of the data in the second virtual machine and the third virtual machine.
- The signal processing device of claim 5, wherein, when the second virtual machine and the third virtual machine share the same data, the first virtual machine in the processor is configured to create one command queue.
- The signal processing device of claim 5, wherein the command queue comprises at least one of virtual machine index information, task information, or shared memory information.
- The signal processing device of claim 1, wherein the first virtual machine in the processor is configured to create command queues corresponding to the number of virtual machines for distributed processing of the data.
- The signal processing device of claim 1, whereinin response to the data being image data, the first virtual machine in the processor is configured to write some of the image data in the first shared memory to be transmitted to the second virtual machine and write some other of the image data in the first shared memory to be transmitted to the third virtual machine, andthe second virtual machine and the third virtual machine are configured to detect objects from the received image data and write the image data from which the objects have been detected in the second shared memory.
- The signal processing device of claim 9, whereinthe processor further executes a fourth virtual machine,the first virtual machine in the processor is configured to write some other of the image data in the first shared memory, andthe third virtual machine is configured to detect an object from the received image data and write the image data from which the object has been detected in the second shared memory.
- The signal processing device of claim 1, whereinin response to the data being image data, the first virtual machine in the processor is configured to write some of the image data in the first shared memory to be transmitted to the second virtual machine and write some other of the image data in the first shared memory to be transmitted to the third virtual machine,the second virtual machine is configured to decrypt some of the image data and write the decrypted image data in the second shared memory, andthe third virtual machine is configured to process the received image data and write the image-processed image data in the second shared memory.
- The signal processing device of claim 1, whereinthe first virtual machine in the processor comprises an input and output server interface and a security manager, andeach of the second virtual machine and the third virtual machine comprises an input and output client interface.
- The signal processing device of claim 12, whereinthe input and output server interface transmits information regarding the first shared memory comprising key data to the input and output client interface after allocation of the first shared memory, andthe input and output client interface accesses the first shared memory based on the key data.
- The signal processing device of claim 12, wherein the input and output server interface receives information regarding a first buffer in the first shared memory, the first buffer being empty, writes first data in the first buffer in the first shared memory, and transmits buffer information of the first buffer to the input and output client interfaces in the second virtual machine and the third virtual machine.
- The signal processing device of claim 14, whereina reference count of the first buffer is changed in a first direction based on writing of the first data in the first buffer, andin response to copying of the first data from the first buffer being completed, the reference count of the first buffer is changed in a second direction, the second direction being opposite the first direction.
- The signal processing device of claim 1, whereinthe processor further executes a legacy virtual machine configured to receive and process Ethernet data, andthe first virtual machine receives, processes, and outputs at least one of vehicle sensor data, position information data, camera image data, audio data, or touch input data.
- The signal processing device of claim 1, wherein the first virtual machine receives and processes wheel speed sensor data of the vehicle, and transmits the processed wheel speed sensor data to at least one of the second virtual machine or the third virtual machine.
- A display apparatus for vehicle, the display apparatus comprising:a first display;a second display; anda signal processing device comprising a processor configured to perform signal processing for the first display and the second display,wherein the signal processing device comprises a signal processing device of any one of claim 1 to 17.
- The display apparatus of claim 18, wherein the signal processing device is configured to divide and process image data using the plurality of virtual machines, combine the divided and processed image data, and display the composite image data through the plurality of displays.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR20210026453 | 2021-02-26 | ||
KR20210053004 | 2021-04-23 | ||
PCT/KR2021/009405 WO2022181899A1 (en) | 2021-02-26 | 2021-07-21 | Signal processing device and vehicle display device having same |
Publications (2)
Publication Number | Publication Date |
---|---|
EP4080359A1 true EP4080359A1 (en) | 2022-10-26 |
EP4080359A4 EP4080359A4 (en) | 2024-04-10 |
Family
ID=83048267
Family Applications (8)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP21878751.3A Pending EP4080359A4 (en) | 2021-02-26 | 2021-07-21 | Signal processing device and vehicle display device having same |
EP21928193.8A Pending EP4307112A1 (en) | 2021-02-26 | 2021-07-21 | Signal processing device, and vehicle display device including same |
EP21928198.7A Pending EP4300299A1 (en) | 2021-02-26 | 2021-08-13 | Vehicular display device |
EP21928197.9A Pending EP4300298A1 (en) | 2021-02-26 | 2021-08-13 | Vehicular display device |
EP21878752.1A Pending EP4080360A4 (en) | 2021-02-26 | 2021-10-22 | Signal processing device, and vehicle display device comprising same |
EP21928252.2A Pending EP4300300A1 (en) | 2021-02-26 | 2021-12-28 | Signal processing device and display apparatus for vehicle comprising same |
EP21928253.0A Pending EP4300301A1 (en) | 2021-02-26 | 2021-12-28 | Signal processing device and vehicle display device having same |
EP22715510.8A Pending EP4075272A4 (en) | 2021-02-26 | 2022-02-22 | Signal processing device, and vehicle display device comprising same |
Family Applications After (7)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP21928193.8A Pending EP4307112A1 (en) | 2021-02-26 | 2021-07-21 | Signal processing device, and vehicle display device including same |
EP21928198.7A Pending EP4300299A1 (en) | 2021-02-26 | 2021-08-13 | Vehicular display device |
EP21928197.9A Pending EP4300298A1 (en) | 2021-02-26 | 2021-08-13 | Vehicular display device |
EP21878752.1A Pending EP4080360A4 (en) | 2021-02-26 | 2021-10-22 | Signal processing device, and vehicle display device comprising same |
EP21928252.2A Pending EP4300300A1 (en) | 2021-02-26 | 2021-12-28 | Signal processing device and display apparatus for vehicle comprising same |
EP21928253.0A Pending EP4300301A1 (en) | 2021-02-26 | 2021-12-28 | Signal processing device and vehicle display device having same |
EP22715510.8A Pending EP4075272A4 (en) | 2021-02-26 | 2022-02-22 | Signal processing device, and vehicle display device comprising same |
Country Status (4)
Country | Link |
---|---|
US (2) | US20230168913A1 (en) |
EP (8) | EP4080359A4 (en) |
KR (3) | KR20230148341A (en) |
WO (8) | WO2022181899A1 (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2024071944A1 (en) * | 2022-09-27 | 2024-04-04 | 엘지전자 주식회사 | Signal processing device and vehicle display device including same |
WO2024071476A1 (en) * | 2022-09-29 | 2024-04-04 | 엘지전자 주식회사 | Signal processing device, and display device for vehicle having same |
Family Cites Families (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3617371B2 (en) * | 1999-05-07 | 2005-02-02 | セイコーエプソン株式会社 | Projector and information storage medium |
US8200796B1 (en) * | 2005-05-05 | 2012-06-12 | Digital Display Innovations, Llc | Graphics display system for multiple remote terminals |
US7650603B2 (en) * | 2005-07-08 | 2010-01-19 | Microsoft Corporation | Resource management for virtualization of graphics adapters |
KR101620058B1 (en) * | 2009-11-23 | 2016-05-24 | 삼성전자주식회사 | Apparatus for switching screen between virtual machines and method thereof |
KR101087479B1 (en) * | 2010-01-29 | 2011-11-25 | 주식회사 팬택 | Multi display device and method for controlling the same |
JP5774032B2 (en) * | 2011-02-09 | 2015-09-02 | パナソニック インテレクチュアル プロパティ コーポレーション オブアメリカPanasonic Intellectual Property Corporation of America | Virtual computer display device, virtual computer display method, virtual computer display program, recording medium, and integrated circuit |
US9152373B2 (en) * | 2011-04-12 | 2015-10-06 | Apple Inc. | Gesture visualization and sharing between electronic devices and remote displays |
KR101323858B1 (en) * | 2011-06-22 | 2013-11-21 | 한국과학기술원 | Apparatus and method for controlling memory access in virtualized system |
US9514507B2 (en) * | 2011-11-29 | 2016-12-06 | Citrix Systems, Inc. | Methods and systems for maintaining state in a virtual machine when disconnected from graphics hardware |
CN103856547B (en) * | 2012-12-07 | 2017-07-07 | 华为技术有限公司 | The mapping method of multi-dummy machine, system and client device |
JP6032176B2 (en) * | 2012-12-19 | 2016-11-24 | 株式会社デンソー | Display control device for vehicle |
EP3092566B1 (en) * | 2014-01-06 | 2019-07-17 | Johnson Controls Technology Company | Vehicle with multiple user interface operating domains |
KR20150142298A (en) | 2014-06-11 | 2015-12-22 | 현대자동차주식회사 | Vehicle, control method of vehicle and vehicle driving sound control apparatus |
KR101639797B1 (en) * | 2015-10-16 | 2016-07-14 | 주식회사 구버넷 | Network interface apparatus and method for processing virtual machine packets |
KR102490614B1 (en) * | 2018-03-16 | 2023-01-19 | 현대모비스 주식회사 | Apparatus and method for controlling display of control device using wirtual operating system |
US10957107B2 (en) * | 2019-01-09 | 2021-03-23 | Vmware, Inc. | Snapping, virtual inking, and accessibility in augmented reality |
DE102019203377B3 (en) * | 2019-03-13 | 2020-08-13 | Continental Automotive Gmbh | Vehicle system, vehicle and method for operating such a vehicle system |
EP3722947A1 (en) * | 2019-04-12 | 2020-10-14 | Aptiv Technologies Limited | Distributed system for displaying a content |
JP7259571B2 (en) * | 2019-06-11 | 2023-04-18 | 株式会社デンソー | VEHICLE CONTROL DEVICE, VEHICLE DISPLAY SYSTEM, AND VEHICLE DISPLAY CONTROL METHOD |
-
2021
- 2021-07-21 WO PCT/KR2021/009405 patent/WO2022181899A1/en unknown
- 2021-07-21 EP EP21878751.3A patent/EP4080359A4/en active Pending
- 2021-07-21 KR KR1020237031947A patent/KR20230148341A/en unknown
- 2021-07-21 US US17/755,040 patent/US20230168913A1/en active Pending
- 2021-07-21 EP EP21928193.8A patent/EP4307112A1/en active Pending
- 2021-07-21 WO PCT/KR2021/009402 patent/WO2022181898A1/en active Application Filing
- 2021-08-13 WO PCT/KR2021/010773 patent/WO2022181904A1/en active Application Filing
- 2021-08-13 WO PCT/KR2021/010772 patent/WO2022181903A1/en active Application Filing
- 2021-08-13 EP EP21928198.7A patent/EP4300299A1/en active Pending
- 2021-08-13 EP EP21928197.9A patent/EP4300298A1/en active Pending
- 2021-10-22 WO PCT/KR2021/014950 patent/WO2022181922A1/en unknown
- 2021-10-22 US US17/755,046 patent/US20230168914A1/en active Pending
- 2021-10-22 EP EP21878752.1A patent/EP4080360A4/en active Pending
- 2021-10-22 KR KR1020237031948A patent/KR20230150826A/en unknown
- 2021-12-28 EP EP21928252.2A patent/EP4300300A1/en active Pending
- 2021-12-28 WO PCT/KR2021/020056 patent/WO2022181960A1/en active Application Filing
- 2021-12-28 WO PCT/KR2021/020076 patent/WO2022181961A1/en active Application Filing
- 2021-12-28 EP EP21928253.0A patent/EP4300301A1/en active Pending
-
2022
- 2022-02-22 KR KR1020237031949A patent/KR20230150318A/en unknown
- 2022-02-22 WO PCT/KR2022/002595 patent/WO2022182103A1/en active Application Filing
- 2022-02-22 EP EP22715510.8A patent/EP4075272A4/en active Pending
Also Published As
Publication number | Publication date |
---|---|
EP4300301A1 (en) | 2024-01-03 |
EP4300298A1 (en) | 2024-01-03 |
KR20230150826A (en) | 2023-10-31 |
WO2022181960A1 (en) | 2022-09-01 |
EP4307112A1 (en) | 2024-01-17 |
EP4080359A4 (en) | 2024-04-10 |
EP4080360A1 (en) | 2022-10-26 |
WO2022181903A1 (en) | 2022-09-01 |
WO2022181899A1 (en) | 2022-09-01 |
EP4080360A4 (en) | 2024-03-06 |
WO2022181922A1 (en) | 2022-09-01 |
WO2022181904A1 (en) | 2022-09-01 |
KR20230150318A (en) | 2023-10-30 |
EP4300300A1 (en) | 2024-01-03 |
US20230168913A1 (en) | 2023-06-01 |
WO2022181961A1 (en) | 2022-09-01 |
EP4300299A1 (en) | 2024-01-03 |
US20230168914A1 (en) | 2023-06-01 |
EP4075272A1 (en) | 2022-10-19 |
EP4075272A4 (en) | 2023-11-29 |
WO2022181898A1 (en) | 2022-09-01 |
KR20230148341A (en) | 2023-10-24 |
WO2022182103A1 (en) | 2022-09-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP4080359A1 (en) | Signal processing device and vehicle display device having same | |
CN104884309B (en) | Vehicle display control unit | |
JP2006518474A (en) | Image display system for displaying different images on separate display devices | |
JP2020044945A (en) | Display control device | |
EP4300917A1 (en) | Communication device for vehicle and display device for vehicle, having same | |
US20240143360A1 (en) | Signal processing device and display apparatus for vehicles including the same | |
US7328102B2 (en) | Navigation intervention apparatus, intervention method and vehicle navigation system | |
US20240140334A1 (en) | Vehicle communication device and display apparatus for vehicle including the same | |
CN114435270A (en) | Vehicle integrated computer | |
US20240143375A1 (en) | Signal processing device and display apparatus for vehicle including the same | |
EP4329317A1 (en) | Signal processing device and vehicle display device having same | |
US20240143374A1 (en) | Signal processing device and display apparatus for vehicle including the same | |
EP4071608A1 (en) | Signal processing device and vehicle display apparatus including the same | |
KR20220139790A (en) | Signal processing device, and display apparatus for vehicle including the same | |
US20230409370A1 (en) | Signal processing device and display apparatus for vehicles including the same | |
US20240037999A1 (en) | Vehicle system | |
JP2018200200A (en) | On-vehicle system, method for operating on-vehicle system, and program | |
EP4300302A1 (en) | Display device for vehicle | |
US20240038000A1 (en) | Vehicle system | |
JP7367728B2 (en) | vehicle system | |
WO2013145580A1 (en) | Information display device | |
CN117687384A (en) | Intelligent cabin system, vehicle control system and method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20220521 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
A4 | Supplementary search report drawn up and despatched |
Effective date: 20240311 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: B60R 16/023 20060101ALI20240304BHEP Ipc: G06F 15/78 20060101ALI20240304BHEP Ipc: G06F 9/451 20180101ALI20240304BHEP Ipc: G06F 9/455 20180101AFI20240304BHEP |