WO2020196933A1

WO2020196933A1 - Electronic device for vehicle and operation method of electronic device for vehicle

Info

Publication number: WO2020196933A1
Application number: PCT/KR2019/003369
Authority: WO
Inventors: 박현상; 강태권; 신동헌; 윤용철
Original assignee: 엘지전자 주식회사
Priority date: 2019-03-22
Filing date: 2019-03-22
Publication date: 2020-10-01

Abstract

The present invention relates to an electronic device for a vehicle, the electronic device including a processor that: acquires image data outside the vehicle; detects at least one object on the basis of the image data; acquires point of interest (POI) data around the vehicle; and synchronizes the POI data with the image data on the basis of a timestamp of the image data.

Description

Vehicle electronic device and operation method of vehicle electronic device

The present invention relates to an electronic device for a vehicle and a method of operating the electronic device for a vehicle.

A vehicle is a device that moves in a direction desired by a boarding user. A typical example is a car. Autonomous vehicle refers to a vehicle that can be driven automatically without human driving operation. Since the user does not need to drive in the autonomous vehicle, the autonomous vehicle can provide a variety of information to the user. Among various types of information, research is needed on a system that provides information that users need and can use.

In order to solve the above problems, an object of the present invention is to provide an electronic device for a vehicle that provides location-based POI information to a user.

Another object of the present invention is to provide an electronic device for a vehicle that provides location-based POI information to a user.

The problems of the present invention are not limited to the problems mentioned above, and other problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, the electronic device for a vehicle according to an embodiment of the present invention acquires image data outside the vehicle, detects at least one object based on the image data, and detects a point of interest (POI) around the vehicle. ) A processor for obtaining data and synchronizing POI data with the image data based on a timestamp of the image data.

According to an embodiment of the present invention, the processor matches the POI data with the object.

According to an embodiment of the present invention, it further comprises a memory; wherein the processor receives the image data from a camera provided in the vehicle, and receives the POI data from an external server using a communication device provided in the vehicle. Then, the image data and the POI data are separated and stored in the memory.

According to an embodiment of the present invention, when a first signal based on a first user input is received, the processor generates a control signal for reproducing an image based on image data synchronized with the POI data.

According to an embodiment of the present invention, the processor generates a control signal for reproducing an image based on image data synchronized from a first point in time to a point in time at which the first signal is received, and the first point in time is the first signal It is defined as the time point before the preset time from the time of reception.

According to an embodiment of the present invention, the processor generates data in which the image data and the POI data are synchronized in a segment unit of the preset time length.

According to an embodiment of the present invention, when the first signal is received while the first segment is being generated, the processor is based on a portion of a previously generated first segment and a second segment generated before the first segment. Generates a control signal for reproducing an image.

According to an embodiment of the present invention, when the second signal based on the second user input is received while the image is being reproduced, the processor generates a control signal for outputting detailed information on at least one POI. Generate.

According to an embodiment of the present invention, the processor generates a control signal for displaying a POI list based on the POI data according to the reproduced image.

According to an embodiment of the present invention, the processor generates a control signal for displaying the graphic object of the image in a state matched with an actual object corresponding to the graphic object.

A method of operating an electronic device for a vehicle according to an embodiment of the present invention may include: obtaining, by at least one processor, image data outside the vehicle; Detecting, by at least one processor, at least one object based on the image data; At least one processor, obtaining POI (Point of Interest) data around the vehicle; And synchronizing, by at least one processor, POI data with the image data based on a timestamp of the image data.

According to an embodiment of the present invention, the synchronizing step includes, by at least one processor, matching the POI data with the object.

According to an embodiment of the present invention, the obtaining of the image data may include: receiving, by at least one processor, the image data from a camera provided in a vehicle; And storing, by at least one processor, the image data in a memory, wherein the obtaining of the POI data comprises: at least one processor, by using a communication device provided in the vehicle, from an external server. Receiving POI data; And storing the POI data separately from the image data in the memory by at least one processor.

According to an embodiment of the present invention, when a first signal based on a first user input is received by at least one processor, generating a control signal for reproducing an image based on the image data synchronized with the POI data; It includes more.

According to an embodiment of the present invention, the generating may include: generating, by at least one processor, a control signal for reproducing an image based on image data synchronized from a first point in time to a point in time at which the first signal is received; And the first time point is defined as a time point before a preset time from the time point at which the first signal is received.

According to an embodiment of the present invention, the generating may include generating, by at least one processor, data in which the image data and the POI data are synchronized in units of segments of the preset time length; Include.

According to an embodiment of the present invention, the generating of the synchronized data includes, by at least one processor, when the first signal is received while the first segment is being generated, the first segment and the first And generating a control signal for reproducing an image based on a part of the second segment generated before the segment.

According to an embodiment of the present invention, when a second signal based on a second user input is received by at least one processor while the image is being played back, control for outputting detailed information on at least one POI Generating a signal; further includes.

According to an embodiment of the present invention, at least one processor generates a control signal for displaying a POI list based on the POI data according to the reproduced image.

According to an embodiment of the present invention, the generating step includes, by at least one processor, generating a control signal for displaying the graphic object of the image in a state matched with an actual object.

Details of other embodiments are included in the detailed description and drawings.

According to the present invention, there are one or more of the following effects.

First, there is an effect that a user can immediately use the corresponding POI information by providing nearby POI information based on the current location of the vehicle.

Second, POI data is stored separately from image data and synchronized according to the situation to be provided to the user, thereby having an advantageous effect on processing load management.

Third, there is an effect of increasing user convenience.

The effects of the present invention are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a view showing the exterior of a vehicle according to an embodiment of the present invention.

2 is a control block diagram of a vehicle according to an embodiment of the present invention.

3 is a diagram referenced to describe an entire system according to an embodiment of the present invention.

4 is a control block diagram of an electronic device for a vehicle according to an embodiment of the present invention.

5 is a flow chart of an electronic device for a vehicle according to an embodiment of the present invention.

6 to 11C are views referenced to explain the operation of the electronic device for a vehicle according to an embodiment of the present invention.

Hereinafter, exemplary embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but identical or similar elements are denoted by the same reference numerals regardless of reference numerals, and redundant descriptions thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used interchangeably in consideration of only the ease of preparation of the specification, and do not have meanings or roles that are distinguished from each other. In addition, in describing the embodiments disclosed in the present specification, when it is determined that a detailed description of related known technologies may obscure the subject matter of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed in the present specification is not limited by the accompanying drawings, and all modifications included in the spirit and scope of the present invention , It should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various elements, but the elements are not limited by the terms. These terms are used only for the purpose of distinguishing one component from another component.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In the present application, terms such as "comprises" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance.

Referring to FIG. 1, a vehicle 10 according to an embodiment of the present invention is defined as a means of transport running on a road or track. The vehicle 10 is a concept including a car, a train, and a motorcycle. The vehicle 10 may be a concept including both an internal combustion engine vehicle including an engine as a power source, a hybrid vehicle including an engine and an electric motor as a power source, and an electric vehicle including an electric motor as a power source. The vehicle 10 may be a shared vehicle. The vehicle 10 may be an autonomous vehicle.

The electronic device 100 may be included in the vehicle 10. The electronic device 100 may provide Point of Interest (POI) information to a user in augmented reality (AR).

Referring to FIG. 2, the vehicle 10 includes an electronic device 100 for a vehicle, a user interface device 200, an object detection device 210, a communication device 220, a driving operation device 230, and a main ECU 240. ), a vehicle driving device 250, a driving system 260, a sensing unit 270, and a location data generating device 280.

The vehicle electronic device 100 may provide Point of Interest (POI) information to a user in augmented reality. The vehicle electronic device 100 may synchronize image data and POI data, and reproduce an image based on the synchronized data. POI information may be included in the reproduced image. The vehicle electronic device 100 may interact with a user. The vehicle electronic device 100 may receive a signal based on a user input and may control an output according to the received signal.

The user interface device 200 is a device for communicating with the vehicle 10 and a user. The user interface device 200 may receive a user input and provide information generated in the vehicle 10 to the user. The vehicle 10 may implement a user interface (UI) or a user experience (UX) through the user interface device 200.

The object detection device 210 may detect an object outside the vehicle 10. The object detection device 210 may include at least one of a camera, a radar, a lidar, an ultrasonic sensor, and an infrared sensor. The object detection device 210 may provide data on an object generated based on a sensing signal generated by a sensor to at least one electronic device included in the vehicle.

The communication device 220 may exchange signals with devices located outside the vehicle 10. The communication device 220 may exchange signals with at least one of an infrastructure (eg, a server, a broadcasting station) and another vehicle. The communication device 220 may include at least one of a transmission antenna, a reception antenna, a radio frequency (RF) circuit capable of implementing various communication protocols, and an RF element to perform communication.

The driving operation device 230 is a device that receives a user input for driving. In the case of the manual mode, the vehicle 10 may be driven based on a signal provided by the driving operation device 230. The driving operation device 230 may include a steering input device (eg, a steering wheel), an acceleration input device (eg, an accelerator pedal), and a brake input device (eg, a brake pedal).

The main ECU 240 may control the overall operation of at least one electronic device provided in the vehicle 10.

The vehicle drive device 250 is a device that electrically controls driving of various devices in the vehicle 10. The vehicle driving apparatus 250 may include a power train driving unit, a chassis driving unit, a door/window driving unit, a safety device driving unit, a lamp driving unit, and an air conditioning driving unit. The power train driving unit may include a power source driving unit and a transmission driving unit. The chassis driving unit may include a steering driving unit, a brake driving unit, and a suspension driving unit.

Meanwhile, the safety device driving unit may include a safety belt driving unit for controlling the safety belt.

The ADAS 260 may control a movement of the vehicle 10 or generate a signal for outputting information to a user based on data on an object received by the object detection apparatus 210. The ADAS 260 may provide the generated signal to at least one of the user interface device 200, the main ECU 240, and the vehicle driving device 250.

ADAS 260 includes an adaptive cruise control system (ACC), an automatic emergency braking system (AEB), a forward collision warning system (FCW), and a lane maintenance assistance system (LKA: Lane Keeping Assist), Lane Change Assist (LCA), Target Following Assist (TFA), Blind Spot Detection (BSD), Adaptive High Beam Control System (HBA: High) Beam Assist), Auto Parking System (APS), PD collision warning system, Traffic Sign Recognition (TSR), Traffic Sign Assist (TSA), At least one of a night vision system (NV: Night Vision), a driver status monitoring system (DSM), and a traffic jam assistance system (TJA) may be implemented.

The sensing unit 270 may sense the state of the vehicle. The sensing unit 270 includes an inertial navigation unit (IMU) sensor, a collision sensor, a wheel sensor, a speed sensor, a tilt sensor, a weight detection sensor, a heading sensor, a position module, and a vehicle. At least one of forward/reverse sensor, battery sensor, fuel sensor, tire sensor, steering sensor by steering wheel rotation, vehicle interior temperature sensor, vehicle interior humidity sensor, ultrasonic sensor, illuminance sensor, accelerator pedal position sensor, and brake pedal position sensor It may include. Meanwhile, the inertial navigation unit (IMU) sensor may include one or more of an acceleration sensor, a gyro sensor, and a magnetic sensor.

The sensing unit 270 may generate state data of the vehicle based on a signal generated by at least one sensor. The sensing unit 270 includes vehicle attitude information, vehicle motion information, vehicle yaw information, vehicle roll information, vehicle pitch information, vehicle collision information, vehicle direction information, vehicle angle information, and vehicle speed. Information, vehicle acceleration information, vehicle tilt information, vehicle forward/reverse information, battery information, fuel information, tire information, vehicle ramp information, vehicle internal temperature information, vehicle internal humidity information, steering wheel rotation angle, vehicle exterior illuminance, accelerator pedal It is possible to acquire a sensing signal for the pressure applied to the brake pedal and the pressure applied to the brake pedal.

In addition, the sensing unit 270 includes an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor (AFS), an intake air temperature sensor (ATS), a water temperature sensor (WTS), and a throttle position sensor. (TPS), a TDC sensor, a crank angle sensor (CAS), and the like may be further included.

The sensing unit 270 may generate vehicle state information based on the sensing data. The vehicle status information may be information generated based on data sensed by various sensors provided inside the vehicle.

For example, the vehicle status information includes vehicle attitude information, vehicle speed information, vehicle tilt information, vehicle weight information, vehicle direction information, vehicle battery information, vehicle fuel information, vehicle tire pressure information, It may include vehicle steering information, vehicle interior temperature information, vehicle interior humidity information, pedal position information, vehicle engine temperature information, and the like.

Meanwhile, the sensing unit may include a tension sensor. The tension sensor may generate a sensing signal based on a tension state of the seat belt.

The location data generating device 280 may generate location data of the vehicle 10. The location data generating apparatus 280 may include at least one of a Global Positioning System (GPS) and a Differential Global Positioning System (DGPS). The location data generating apparatus 280 may generate location data of the vehicle 10 based on a signal generated by at least one of GPS and DGPS. According to an embodiment, the location data generating apparatus 280 may correct the location data based on at least one of an IMU (Inertial Measurement Unit) of the sensing unit 270 and a camera of the object detection apparatus 210.

Vehicle 10 may include an internal communication system 50. A plurality of electronic devices included in the vehicle 10 may exchange signals through the internal communication system 50. The signal may contain data. The internal communication system 50 may use at least one communication protocol (eg, CAN, LIN, FlexRay, MOST, Ethernet).

3 to 4, the system may include a vehicle 10, a mobile terminal 20, and an external device 30. The vehicle 10 may exchange signals with the mobile terminal 20 and the external device 30 through at least one communication network.

The vehicle 10 is as described with reference to FIGS. 1 and 2. The vehicle 10 may include an electronic device 100 for a vehicle.

The vehicle electronic device 100 may exchange signals with the mobile terminal 20 and the external device 30 through the communication device 220. Depending on the embodiment, the vehicle electronic device 100 may be connected to the communication device 220 via an Ethernet switch hub. The processor 170 may exchange signals with the mobile terminal 20 through the interface unit 180 and the communication device 220. The processor 170 may receive data from the mobile terminal 20 and transmit the data to the mobile terminal 20. The processor 170 may exchange signals with the external device 30 through the interface unit 180 and the communication device 220. The processor 170 may receive data from the external device 30 and transmit the data to the external device 30.

The vehicle electronic device 100 may include a memory 140, a processor 170, an interface unit 180, and a power supply unit 190.

The memory 140 is electrically connected to the processor 170. The memory 140 may store basic data for a unit, control data for controlling the operation of the unit, and input/output data. The memory 140 may store data processed by the processor 170. In terms of hardware, the memory 140 may be configured with at least one of ROM, RAM, EPROM, flash drive, and hard drive. The memory 140 may store various data for overall operation of the electronic device 100, such as a program for processing or controlling the processor 170. The memory 140 may be implemented integrally with the processor 170. Depending on the embodiment, the memory 140 may be classified as a sub-element of the processor 170.

The memory 140 may store image data received from a camera provided in the vehicle 10. The memory 140 may store POI data received from the external server 34 using the communication device 220 provided in the vehicle 10. The memory 140 may separately store image data and POI data.

The interface unit 180 may exchange signals with at least one electronic device provided in the vehicle 10 by wire or wirelessly. The interface unit 280 includes an object detection device 210, a communication device 220, a driving operation device 230, a main ECU 140, a vehicle driving device 250, an ADAS 260, and a sensing unit 170. And it is possible to exchange a signal with at least one of the location data generating device 280 wired or wirelessly. The interface unit 280 may be configured with at least one of a communication module, a terminal, a pin, a cable, a port, a circuit, an element, and a device.

The interface unit 180 may receive location data of the vehicle 10 from the location data generating device 280. The interface unit 180 may receive driving speed data from the sensing unit 270. The interface unit 180 may receive object data around the vehicle from the object detection device 210.

The power supply unit 190 may supply power to the electronic device 100. The power supply unit 190 may receive power from a power source (eg, a battery) included in the vehicle 10 and supply power to each unit of the electronic device 100.

The processor 170 may be electrically connected to the memory, the interface unit 180, and the power supply unit 190 to exchange signals. The processor 170 includes application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, and controllers. It may be implemented using at least one of (controllers), micro-controllers, microprocessors, and electrical units for performing other functions.

The processor 170 may include a POI application 110. The POI application may be installed on the processor 170. The POI application 110 may perform a POI-related operation described below.

The processor 170 may acquire image data outside the vehicle. The processor 170 may receive image data from a camera provided in the vehicle 10. The camera may be classified as a sub-element of the object detection device 210. The camera may acquire an external image of the vehicle 10. The camera may acquire front, rear, left and right images of the vehicle 10. The camera may transmit the acquired image data to the processor 170.

The processor 170 may detect at least one object based on the image data. Here, the object may be a building, a store, a landmark, a tourist destination, a restaurant, a historical site, or the like that is a target of POI.

The processor 170 may acquire Point of Interest (POI) data around the vehicle. The processor 170 may receive POI data from the external server 34 using the communication device 220 provided in the vehicle 10. The processor 170 may transmit location information of the vehicle 10 to the external server 34. The external server 34 may determine a POI located within a distance from the location of the vehicle 10 and provide data on the POI to the processor 170.

The processor 170 may separate and store image data and POI data in the memory 140. POI data has a smaller amount of data than image data, and thus memory 140-based management is possible. When a signal based on a user input is received, since POI data and image data are separated, operations such as POI navigation or POI selection are possible. In addition, there is an advantage in that it is possible to remove the limit on the number of POI target objects contained in one screen. In addition, the operation of putting the POI data as meta data in the image data is a processing load.

The processor 170 may synchronize POI data with image data based on a timestamp of the image data. The processor 170 may synchronize the video data and the POI data based on a time stamp (eg, Video PTS) at the time of video playback. The processor 170 may synchronize the image data and the POI data at all times while the vehicle 10 is running, and store them in the memory 140. The processor 170 may match the POI data with an object detected based on the image data.

The processor 170 may receive a first signal based on a first user input. For example, the processor 170 may receive a signal generated by a user's manipulation of the touch module 211 through a USB hub. When a first signal based on a first user input is received, the processor 170 may generate a control signal for reproducing an image based on image data synchronized with POI data. The processor 170 may transmit a control signal to the display 214.

The processor 170 may generate a control signal for reproducing an image based on the synchronized image data from the first point in time to the first signal reception point. Here, the first time point may be defined as a time point before a preset time from the first signal reception time point. For example, the processor 170 may reproduce an image based on the synchronized image data 30 seconds before a time when a first user input is made. The amount of the video to be played may be 30 seconds.

The processor 170 may generate data in which image data and POI data are synchronized in units of segments having a preset length of time. The processor 170 may generate synchronization data in units of segments in real time and store them in the memory 140. The processor 170 may delete the stored synchronization data in units of segments after a preset time. When the first signal is received while the first segment is being generated, the processor 170 controls to play back an image based on a part of a previously generated first segment and a second segment generated before the first segment Can generate signals. In this case, the total sum of the playback times may be the preset length of time. For example, the total sum of playback times may be 30 seconds.

The processor 170 may receive a second signal based on a second user input while an image is being played back. For example, the processor 170 may receive a signal generated by a user's manipulation of the touch module 211 through a USB hub. The processor 170 may generate a control signal for outputting detailed information on at least one POI when a second signal based on a second user input is received while an image is being reproduced. The processor 170 may receive detailed information on the POI from the external server 34 using the communication device 220.

The processor 170 may generate a control signal for displaying a POI list based on the POI data according to a reproduced image. The processor 170 may provide a control signal to the display 214. For example, the processor 170 may generate a control signal for displaying a POI list according to a reproduced image based on a relative positional relationship between the vehicle 10 and the POI.

The processor 170 may display POI information contained in the image as augmented reality. The processor 170 may generate a control signal for displaying a graphic object of an image in a state matched with an actual object corresponding to the graphic object. The processor 170 may provide a control signal to the display 214.

The processor 170 includes a signage client 121, a media player 122, a POI detector 123, a voice agent 123, and a fleet client 125 can do.

The signage client 121 may communicate with at least one external server in order to obtain signage content. The media player 122 can play content. The POI detector 123 may communicate with the POI server based on GPS information to exchange signals, information, and data with the POI server. The POI detector may detect an object based on the image data. The voice agent 124 may generate a control signal based on a user's voice command. The fleet client 125 is connected to the control server and serves to transmit a control command to at least one electronic device (eg, object detection device, main ECU, driving system) included in the vehicle, By transmitting vehicle sensor data to the server, central control and control are possible.

Meanwhile, the touch module 211, the camera 212, the microphone 213, and the display 214 may be classified as sub-components of the user interface device 200.

The electronic device 100 may include at least one printed circuit board (PCB). The electronic device 110, the memory, the interface unit 180, the power supply unit, and the processor 170 may be electrically connected to a printed circuit board.

The mobile terminal 20 can exchange signals with the vehicle 10. For example, the mobile terminal 20 can exchange signals with the vehicle 10 through a short-range communication method (eg, Bluetooth).

The external device 30 can be exchanged with the vehicle 10 using at least one communication network. The external device 30 includes a fleet management server 31, a signage server 32, a big data server 33, and a point of interest (POI) server ( 34) may be included.

The fleet management server 31 may control a vehicle and set a driving route through communication with the fleet client 125, and perform a vehicle providing service in connection with a reservation server. The signage server 32 , Through communication with the signage client 121, it is possible to provide signage content. The big data server 33 may extract and provide big data based on the provided context information. The POI server 34 provides POI information registered based on GPS information. For example, the POI server may provide shopping street information, representative product information, representative advertisement information, and link information.

Referring to FIG. 5, the processor 170 may acquire image data outside the vehicle (S510). In the step of obtaining image data (S510), the at least one processor 170 receives image data from a camera provided in the vehicle 10, and the at least one processor 170 stores the image data in a memory ( 140).

The processor 170 may detect at least one object based on the image data (S515).

The processor 170 may transmit the vehicle location data to the external server 34 (S520). The external server 34 may select a POI around the vehicle and provide POI data based on the location data of the vehicle. The processor 170 may acquire POI data around the vehicle (S530). In the step of obtaining POI data (S530), the at least one processor 170 receives POI data from the external server 34 using the communication device 220 provided in the vehicle 10, and at least One processor 170 may include storing the POI data in the memory 140 by separating from the image data.

The processor 170 may synchronize the POI data with the image data based on the time stamp of the image data (S540). The synchronizing step (S540) may include matching, by at least one processor, the POI data with the object detected in step S515.

When a first signal based on a first user input is received (S550), the processor 170 may generate a control signal for reproducing an image based on image data synchronized with POI data (S560). The generating step (S560) includes generating, by the at least one processor 170, a control signal for reproducing an image based on the image data synchronized from a first point in time to a first signal reception point (step S561). can do. Here, the first time point may be defined as a time point before a preset time from the time when the first signal is received. The generating step (S561) may include generating, by the at least one processor 170, data in which the image data and the POI data are synchronized in a segment unit of a preset time length (step S562). have. In the step of generating synchronized data (S562), when the first signal is received while the first segment is being generated, by the at least one processor 170, the generated first segment and the generated before the first segment It may include generating a control signal for reproducing an image based on a part of the second segment.

Meanwhile, the generating step (S560) may include generating, by at least one processor, a control signal for displaying a graphic object of an image in a state matched with an actual object corresponding to the graphic object.

The processor 170 may generate a control signal for displaying a POI list based on the POI data according to a reproduced image (S570).

The processor 170 may generate a control signal for outputting detailed information on at least one POI when a second signal based on a second user input is received (S580) while an image is being played back. (S590).

Referring to FIG. 6, the processor 170 may acquire image data 610. The processor 170 may receive image data 610 from a camera that photographs the exterior of the vehicle. The processor 170 may detect at least one object based on the image data 610.

The processor 170 may obtain the POI data 620. The processor 170 may transmit location data of the vehicle 10 to the external server 34 using the communication device 220. The processor 170 may receive POI data 620 based on location data of the vehicle 10 from the external server 34 using the communication device 220.

The processor 170 may separate the image data 610 and the POI data 620 and store them in the memory 140.

The processor 170 may synchronize the POI data 620 with the image data 610. The processor 170 may synchronize the POI data 620 with the detected object.

Referring to FIG. 7, the processor 170 may generate data in which image data and POI data are synchronized in segment units of a preset time length (eg, 30 seconds). As illustrated in FIG. 7, the processor 170 may generate first segment data 710, second segment data 720, and third segment data 730. The processor 170 generates the second segment data 720 after generating the third segment data 730 according to the passage of time, and the first segment after generating the second segment data 720 Data 710 may be generated.

While generating the first segment data 710, when a first signal based on a first user input is received, the processor 170 may include a portion of the first segment data 710 and the second segment data 720. The video can be played based on a part. In this case, the image may be reproduced for the predetermined length of time (eg, 30 seconds). The processor 170 includes the first segment data 710 from the first segment data 710 generation start point to the first signal reception point and the second segment data 720 at a preset time length. An image may be reproduced based on a part of the second segment data 720 that is excluded by a partial time length of the 710. Although divided into a first segment 710 and a second segment 720, since the first segment 710 is continuous to the second segment 720, the image can be continuously reproduced.

Referring to FIG. 8, the external server 34 may receive vehicle location data from the vehicle 10. The external server 34 may select

POIs

810 and 820 located within a preset distance based on the location of the vehicle 10 from a plurality of POIs. The external server 34 may generate POI data corresponding to the POI. The external server 34 may provide POI data to the vehicle electronic device 100.

Referring to FIG. 9, the POI detector 123 may request image data from the video player 910 (S910). The video player 910 may be software or hardware that processes an image acquired from a camera. The POI detector 123 may receive image data and a time stamp value from the video player 910 (S920).

The POI detector 123 may search for a POI list matching the time stamp from the POI record file 620 of the memory 140 (S930).

The POI detector 123 may reproduce an image based on the image data synchronized with the POI (S940). When a specific POI is selected according to a user input (S950), the processor 170 may request detailed information on the corresponding POI from the POI server 34. The processor 170 may receive detailed information on the POI from the POI server 34 (S970). The processor 170 may display detailed information on the POI (S980).

10A to 10B, a display 214 may be disposed in an area 1010 of a window of the vehicle 10. The display 214 may be implemented as a transparent display. The transparent display can display a predetermined screen while having a predetermined transparency. Transparent display, in order to have transparency, transparent display is transparent TFEL (Thin Film Elecroluminescent), transparent OLED (Organic Light-Emitting Diode), transparent LCD (Liquid Crystal Display), transmissive transparent display, transparent LED (Light Emitting Diode) display It may include at least one of. The transparency of the transparent display can be adjusted.

The processor 170 may provide a control signal for reproducing an image including POI information on the display 214. When the vehicle 10 is close to the POI point, the processor 170 may provide a control signal for displaying POI information on the display 214.

The display 214 may implement a touch screen by forming a mutual layer of the touch input unit and the layer. A user may interact with an electronic device for a vehicle through a touch screen. The processor 170 may be operated according to a signal generated based on a user input through a touch screen. For example, the user may perform a double tap input by tapping the touch screen twice in order to receive POI information. The double tap input may be an example of the above-described first user input. The touch screen may convert a user's double tap input into an electrical signal. The electrical signal based on the double tap input may be the above-described first signal. The processor 170 may receive the first signal and perform a predetermined operation according to the first signal. For example, the user may perform a touch input on the first POI displayed on the touch screen in order to receive detailed information of the POI. The touch input for the first POI may be an example of the aforementioned second user input. The touch screen may convert a user's touch input into an electrical signal. The electrical signal based on the touch input may be the second signal described above. The processor 170 may receive the second signal and perform a predetermined operation according to the second signal.

11A to 11C, the processor 170 may play a video 1110 on a region of the display 214. The reproduced image may be an image in which POI information is combined with an image outside the vehicle 10. The processor 170 may display a POI list 1120 together with an image reproduced in one area of the display 214. The processor 170 may match and display a POI graphic object in a reproduced image with a POI list. For example, the processor 170 may highlight the first graphic object 1111 corresponding to the first POI closest to the vehicle 10. In this case, the processor 170 may highlight the first POI item 1121 matching the first graphic object 1111 in the POI list.

As the vehicle 10 moves, when the nearest POI is changed from the first POI to the second POI, the processor 170 converts the highlighted graphic object from the first graphic object 1111 to a second graphic object ( 1112). In this case, the processor 170 may change the first POI item 1121 to be highlighted in the POI list to the second POI item 1122. The second POI item 1122 may be an item matching the second graphic object.

When the first POI is selected by the user, the processor 170 may output detailed information 1150 on the first POI. In a state in which detailed information 1150 is displayed, when a user input (touch input for 1151 or 1152) for receiving additional information is received, the processor 170 may output additional information. For example, the processor 170 may output a description of a product or an advertisement video.

Meanwhile, the processor 170 may display the route 1130 of the vehicle 10 and the traffic condition information 1140 on one area of the display 213.

The present invention described above can be implemented as a computer-readable code in a medium on which a program is recorded. The computer-readable medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is also a carrier wave (eg, transmission over the Internet). In addition, the computer may include a processor or a control unit. Therefore, the detailed description above should not be construed as restrictive in all respects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

[Explanation of code]

10: vehicle

100: vehicle electronic device

Claims

Acquiring image data outside the vehicle, detecting at least one object based on the image data,

Acquire Point of Interest (POI) data around the vehicle,

And a processor for synchronizing POI data with the image data based on a timestamp of the image data.
The method of claim 1,

The processor,

An electronic device for a vehicle that matches the POI data with the object.
The method of claim 1,

And a memory;

The processor,

Receiving the image data from a camera provided in the vehicle,

Using a communication device provided in the vehicle, receiving the POI data from an external server,

The electronic device for a vehicle separately storing the image data and the POI data in the memory.
The method of claim 1,

The processor,

When a first signal based on a first user input is received, a control signal for reproducing an image based on image data synchronized with the POI data is generated.
The method of claim 4,

The processor,

Generating a control signal for reproducing an image based on the image data synchronized from a first point in time to a point in time at which the first signal is received,

The first time point is defined as a time point before a preset time from the time point when the first signal is received.
The method of claim 5,

The processor,

An electronic device for a vehicle that generates data in which the image data and the POI data are synchronized in a segment unit of the preset time length.
The method of claim 6,

The processor,

When the first signal is received while the first segment is being generated, a vehicle electronics that generates a control signal for reproducing an image based on a portion of a previously generated first segment and a second segment generated before the first segment Device.
The method of claim 4,

The processor,

In a state in which the image is being reproduced, when a second signal based on a second user input is received, a control signal for outputting detailed information on at least one POI is generated.
The method of claim 4,

The processor,

A vehicle electronic device that generates a control signal for displaying a POI list based on the POI data according to the reproduced image.
The method of claim 4,

The processor,

An electronic device for a vehicle that generates a control signal for displaying the graphic object of the image in a state matched with an actual object corresponding to the graphic object.
Obtaining, by at least one processor, image data outside the vehicle;

Detecting, by at least one processor, at least one object based on the image data;

At least one processor, obtaining POI (Point of Interest) data around the vehicle; And

And synchronizing, by at least one processor, POI data with the image data based on a timestamp of the image data.
The method of claim 11,

The synchronizing step,

And a step of matching the POI data with the object by at least one processor.
The method of claim 11,

The step of obtaining the image data,

Receiving, by at least one processor, the image data from a camera provided in the vehicle; And

Including, at least one processor, storing the image data in a memory,

The step of obtaining the POI data,

Receiving, by at least one processor, the POI data from an external server using a communication device provided in the vehicle; And

And storing the POI data separately from the image data in the memory by at least one processor.
The method of claim 11,

Generating, by at least one processor, a control signal for reproducing an image based on the image data synchronized with the POI data, when a first signal based on a first user input is received; How it works.
The method of claim 14,

The generating step,

Generating, by at least one processor, a control signal for reproducing an image based on the image data synchronized from a first time point to a time point at which the first signal is received,

The first time point is defined as a time point before a preset time from the time point when the first signal is received.
The method of claim 15,

The generating step,

And generating, by at least one processor, data in which the image data and the POI data are synchronized in units of segments of the preset time length.
The method of claim 16,

The step of generating the synchronized data,

At least one processor, when the first signal is received while the first segment is being generated, a control for reproducing an image based on a portion of a previously generated first segment and a second segment generated before the first segment Generating a signal; operating method of a vehicle electronic device comprising a.
The method of claim 14,

Generating, by at least one processor, a control signal for outputting detailed information on at least one POI when a second signal based on a second user input is received while the image is being played back; further Operating method of a vehicle electronic device including.
The method of claim 14,

Generating, by at least one processor, a control signal for displaying a POI list based on the POI data according to the reproduced image.
The method of claim 14,

The generating step,

Generating, by at least one processor, a control signal for displaying the graphic object of the image in a state matched with an actual object.