KR20210095757A - Vehicle for performing autonomous driving using a plurality of sensors and operating method thereof - Google Patents

Abstract

Various embodiments of the present invention relate to a vehicle for performing autonomous driving by using a plurality of sensors having different viewing angles and an operating method thereof. The operating method of an electronic device included in a vehicle comprises: an operation of using at least one front sensor to determine a driving state of a first front vehicle positioned in a first area; an operation of using at least one side sensor to acquire information on at least one object positioned in a second area if the driving state of the first front vehicle corresponds to a designated driving state; and an operation of controlling driving of the vehicle based on the information on the at least one object and driving state information of the first front vehicle. At least a portion of the second area can include an area which is not overlapped with the first area. One or more among an autonomous vehicle, a user terminal, and a server of the present invention can be linked to an artificial intelligence module, a drone (unmanned aerial vehicle (UAV)), a robot, an augmented reality (AR) device, a virtual reality (VR) device, and devices associated with 5G services.

Description

A vehicle performing autonomous driving using a plurality of sensors and an operating method thereof

Various embodiments of the present disclosure relate to a vehicle that performs autonomous driving using a plurality of sensors and an operating method thereof.

The autonomous driving vehicle refers to a vehicle having a function that can drive itself without a user's manipulation. The autonomous vehicle transmits information obtained from at least one sensor mounted on the vehicle to a server, and receives information on conditions of surrounding vehicles and roads from the server, thereby performing automatic driving without user manipulation.

The autonomous vehicle may detect an object around the vehicle by using a sensor facing the front area of the vehicle. For example, the autonomous vehicle may acquire an image of a front area of the vehicle through a front camera mounted on the vehicle, and analyze the obtained image to detect an object located in the front area of the vehicle.

On the road, there may be a blind spot in which an object cannot be detected using the front sensor of the vehicle. For example, due to the shape of the road on which the vehicle is located, or surrounding obstacles (eg, other vehicles, buildings, objects), a blind spot that cannot be monitored at least temporarily through a front sensor while driving in the vehicle may occur. If other vehicles and/or pedestrians are approaching on the vehicle's driving path in the vehicle's blind spot, the vehicle may not be able to detect other vehicles and/or pedestrians located in the vehicle's blind spot, which may result in the vehicle and other vehicles and/or pedestrians Pedestrian collisions may occur.

Accordingly, various embodiments of the present disclosure disclose a vehicle that performs autonomous driving using a plurality of sensors having different viewing angles and an operating method thereof.

Various embodiments of the present disclosure disclose a vehicle that performs autonomous driving using at least one sensor facing a front area of the vehicle and at least one sensor facing a lateral area of the vehicle, and a method of operating the same.

The technical problems to be achieved in this document are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. will be able

According to various embodiments of the present disclosure, a method of operating an electronic device included in a vehicle includes: determining a driving state of a first front vehicle located in a first area using at least one front sensor; When the driving state of the vehicle corresponds to the specified driving state, an operation of acquiring information on at least one object located in the second area using at least one lateral sensor, and the information on the at least one object and the and controlling the driving of the vehicle based on driving state information of the first front vehicle, wherein at least a portion of the second region may include a region that does not overlap the first region.

According to various embodiments of the present disclosure, an electronic device included in a vehicle includes at least one front sensor, at least one side sensor, and a processor, and the processor includes: At least one object located in the second region is determined by determining the driving state of the first front vehicle located in the first region, and when the driving state of the first forward vehicle corresponds to the specified driving state, using at least one lateral sensor obtains information on the at least one object and controls the driving of the vehicle based on the information on the at least one object and the driving state information of the first front vehicle, wherein at least a partial region of the second region includes the first It may include a region that does not overlap the region.

The autonomous vehicle according to various embodiments of the present disclosure minimizes a blind spot that cannot be monitored by using at least one front sensor and at least one side sensor during autonomous driving, so that other vehicles and/or Collision with pedestrians can be prevented.

The autonomous driving vehicle according to various embodiments of the present disclosure monitors a blind spot that cannot be monitored by a front sensor with a side sensor to determine an intention for a current driving state of another vehicle located in a front area of the vehicle, and It is possible to predict the next driving state of the vehicle, and based on this, autonomous driving of the vehicle can be efficiently performed.

1 shows an example of basic operations of an autonomous vehicle and a 5G network in a 5G communication system.
2 shows an example of an application operation of an autonomous vehicle and a 5G network in a 5G communication system.
3 to 6 show an example of an autonomous driving vehicle operation using 5G communication.
7A illustrates sensors included in a headlamp of a vehicle according to various embodiments of the present disclosure;
7B illustrates a field of view (FOV) of sensors in a headlamp in accordance with various embodiments.
8 is a control block diagram of a vehicle according to various embodiments of the present disclosure;
9 is a block diagram of an electronic device included in a vehicle according to various embodiments of the present disclosure;
10A and 10B are exemplary views of detecting an object using front cameras and side cameras in a vehicle according to various embodiments of the present disclosure;
11 is a flowchart illustrating autonomous driving using a plurality of sensors in a vehicle according to various embodiments of the present disclosure;
12 is a flowchart illustrating autonomous driving of a host vehicle based on a driving state of a front vehicle in a vehicle according to various embodiments of the present disclosure;
13A is an exemplary diagram of changing a driving path of a host vehicle based on a driving state of a front vehicle positioned on a driving path in a vehicle according to various embodiments of the present disclosure;
13B is an exemplary diagram of controlling the driving of the own vehicle based on the driving state of the front vehicle positioned on the driving path in the vehicle according to various embodiments of the present disclosure;
14 is a flowchart of detecting a surrounding object based on a collision risk of the surrounding object in a vehicle according to various embodiments of the present disclosure;
15 is an exemplary diagram of determining a collision risk of a surrounding object in a vehicle according to various embodiments of the present disclosure;
16 is an exemplary diagram of changing a collision risk based on a change in a position of a surrounding object in a vehicle according to various embodiments of the present disclosure;
17 is a flowchart of controlling driving of the own vehicle based on the type and/or collision risk of another vehicle in a driving path in a vehicle according to various embodiments of the present disclosure;
18 is an exemplary diagram of controlling the driving of the own vehicle based on the type and/or collision risk of another vehicle in the driving path in the vehicle according to various embodiments of the present disclosure;
19 is a flowchart of obtaining an intention for a driving state of a vehicle in front by using a side camera in a vehicle according to various embodiments of the present disclosure;
20 is an exemplary diagram of analyzing an intention for a driving state of a front vehicle by using a side camera in a vehicle according to various embodiments of the present disclosure;
21 is a flowchart of controlling a setting of a lidar based on information on a surrounding object in a vehicle according to various embodiments of the present disclosure;
22 is an exemplary diagram of obtaining information on a surrounding object using a left lidar and a right lidar in a vehicle according to various embodiments of the present disclosure;
23 is an exemplary diagram of controlling an output intensity of each laser point of a lidar based on distances of surrounding objects in a vehicle according to various embodiments of the present disclosure;
24 is an exemplary diagram of controlling a laser irradiation period of a lidar based on a distance from a surrounding object in a vehicle according to various embodiments of the present disclosure;
24 is an exemplary diagram of determining a laser irradiation point of a lidar for a surrounding object in a vehicle according to various embodiments of the present disclosure;
26 is a flowchart of switching a use mode of a camera and a lidar in a vehicle according to various embodiments of the present disclosure;

Advantages and features of the present disclosure, and methods of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, but will be implemented in various different forms, and only the present embodiments allow the disclosure of the present disclosure to be complete, and are common in the art to which the present disclosure pertains. It is provided to fully inform those with knowledge of the scope of the disclosure, which is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

When one component is referred to as “connected to” or “coupled to” with another component, it means that it is directly connected or coupled to another component or intervening another component. including all cases. On the other hand, when one component is referred to as “directly connected to” or “directly coupled to” with another component, it indicates that another component is not interposed therebetween. “and/or” includes each and every combination of one or more of the recited items.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present disclosure. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” refers to the presence of one or more other components, steps, operations and/or elements mentioned. or addition is not excluded.

Although the first, second, etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another.

Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present disclosure. Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which this disclosure belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly defined in particular.

The term 'unit' or 'module' used in this embodiment means software or hardware components such as FPGA or ASIC, and 'unit' or 'module' performs certain roles. However, 'part' or 'module' is not meant to be limited to software or hardware. A 'unit' or 'module' may be configured to reside on an addressable storage medium or may be configured to reproduce one or more processors. Thus, as an example, 'part' or 'module' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, may include procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Components and functionality provided in 'units' or 'modules' may be combined into a smaller number of components and 'units' or 'modules' or additional components and 'units' or 'modules' can be further separated.

The steps of a method or algorithm described in connection with some embodiments of the present disclosure may be directly implemented in hardware executed by a processor, a software module, or a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of recording medium known in the art. An exemplary recording medium is coupled to the processor, the processor capable of reading information from, and writing information to, the storage medium. Alternatively, the recording medium may be integral with the processor. The processor and recording medium may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal.

1 shows an example of basic operations of an autonomous vehicle and a 5G network in a 5G communication system.

The autonomous vehicle transmits specific information transmission to the 5G network (S1).

The specific information may include autonomous driving-related information.

The autonomous driving-related information may be information directly related to driving control of a vehicle. For example, the autonomous driving-related information may include one or more of object data indicating objects around the vehicle, map data, vehicle state data, vehicle location data, and driving plan data. .

The autonomous driving-related information may further include service information required for autonomous driving. For example, the specific information may include information about the destination and the vehicle's stability level input through the user terminal. Then, the 5G network may determine whether to remotely control the vehicle (S2).

Here, the 5G network may include a server or module for performing remote control related to autonomous driving.

In addition, the 5G network may transmit information (or signals) related to remote control to the autonomous vehicle (S3).

As described above, the information related to the remote control may be a signal directly applied to the autonomous driving vehicle, and further may further include service information required for autonomous driving. In an embodiment of the present disclosure, the autonomous vehicle may provide services related to autonomous driving by receiving service information such as insurance for each section selected on a driving route and information on dangerous sections through a server connected to the 5G network.

Hereinafter, in FIGS. 2 to 6 , in order to provide an insurance service applicable to each section in the autonomous driving process according to an embodiment of the present disclosure, an essential process for 5G communication between an autonomous driving vehicle and a 5G network (eg, a vehicle and a The initial access procedure between 5G networks, etc.) will be briefly described.

2 shows an example of an application operation of an autonomous vehicle and a 5G network in a 5G communication system.

The autonomous vehicle performs an initial access procedure with the 5G network (S20).

The initial access procedure includes a cell search for acquiring a downlink (DL) operation, a process of acquiring system information, and the like.

Then, the autonomous vehicle performs a random access procedure with the 5G network (S21).

The random access process may include a preamble transmission, random access response reception process, etc. for acquiring uplink (UL) synchronization or UL data transmission.

Then, the 5G network transmits a UL grant for scheduling transmission of specific information to the autonomous vehicle (S22).

The UL grant reception includes a process of receiving time/frequency resource scheduling for transmission of UL data to a 5G network.

Then, the autonomous vehicle transmits specific information to the 5G network based on the UL grant (S23).

Then, the 5G network determines whether to remotely control the vehicle (S24).

Then, the autonomous vehicle receives a DL grant through a physical downlink control channel to receive a response to specific information from the 5G network (S25).

Then, the 5G network transmits information (or signals) related to remote control to the autonomous vehicle based on the DL grant (S26).

Meanwhile, in FIG. 3 , an example in which an initial access process and/or a random access process and a downlink grant reception process of an autonomous vehicle and 5G communication are combined through the processes S20 to S26 has been exemplarily described, but various embodiments of the present disclosure are not limited thereto.

For example, an initial access process and/or a random access process may be performed through processes S20, S22, S23, S24, and S24. Also, for example, an initial access process and/or a random access process may be performed through processes S21, S22, S23, S24, and S26. In addition, a process in which an AI operation and a downlink grant reception process are combined may be performed through S23, S24, S25, and S26.

In addition, in FIG. 2 , the autonomous vehicle operation is exemplarily described through S20 to S26, and various embodiments of the present disclosure are not limited thereto.

For example, in the autonomous vehicle operation, S20, S21, S22, and S25 may be selectively combined with S23 and S26. Also, for example, the autonomous driving vehicle operation may include S21, S22, S23, and S26. Also, for example, the autonomous driving vehicle operation may include S20, S21, S23, and S26. Also, for example, the autonomous driving vehicle operation may be configured by S22, S23, S25, and S26.

3 to 6 show an example of an autonomous driving vehicle operation using 5G communication.

Referring first to FIG. 3 , an autonomous driving vehicle including an autonomous driving module performs an initial access procedure with a 5G network based on a synchronization signal block (SSB) to obtain DL synchronization and system information ( S30 ).

Then, the autonomous vehicle performs a random access procedure with the 5G network for UL synchronization acquisition and/or UL transmission (S31).

Then, the autonomous vehicle receives a UL grant to the 5G network to transmit specific information (S32).

Then, the autonomous vehicle transmits specific information to the 5G network based on the UL grant (S33).

Then, the autonomous vehicle receives a DL grant for receiving a response to specific information from the 5G network (S34).

Then, the autonomous vehicle receives information (or signals) related to remote control from the 5G network based on the DL grant (S35).

A beam management (BM) process may be added to S30, and a beam failure recovery process related to PRACH (physical random access channel) transmission may be added to S31, and a UL grant is included in S32. A QCL relationship may be added in relation to the beam reception direction of the PDCCH, and the addition of a QCL relationship is added in relation to the beam transmission direction of a physical uplink control channel (PUCCH)/physical uplink shared channel (PUSCH) including specific information in S33. can be In addition, a QCL relationship may be added in relation to the beam reception direction of the PDCCH including the DL grant in S34.

Referring to FIG. 4 , the autonomous vehicle performs an initial access procedure with the 5G network based on the SSB to acquire DL synchronization and system information ( S40 ).

Then, the autonomous vehicle performs a random access procedure with the 5G network for UL synchronization acquisition and/or UL transmission (S41).

Then, the autonomous vehicle transmits specific information to the 5G network based on a configured grant (S42). Instead of the process of performing the UL grant from the 5G network, the process of granting a configured grant will be described in more detail in the following paragraphs.

Then, the autonomous vehicle receives information (or signals) related to remote control from the 5G network based on the set grant (S43).

Referring to FIG. 5 , the autonomous vehicle performs an initial access procedure with the 5G network based on the SSB to obtain DL synchronization and system information ( S50 ).

Then, the autonomous vehicle performs a random access procedure with the 5G network for UL synchronization acquisition and/or UL transmission (S51).

Then, the autonomous vehicle receives a DownlinkPreemption IE from the 5G network (S52).

Then, the autonomous vehicle receives DCI format 2_1 including a preemption indication from the 5G network based on the DownlinkPreemption IE (S53).

And, the autonomous vehicle does not perform (or expect or assume) the reception of eMBB data in the resource (PRB and/or OFDM symbol) indicated by the pre-emption indication (S54).

Then, the autonomous vehicle receives a UL grant to the 5G network to transmit specific information (S55).

Then, the autonomous vehicle transmits specific information to the 5G network based on the UL grant (S56).

Then, the autonomous vehicle receives a DL grant for receiving a response to specific information from the 5G network (S57).

Then, the autonomous vehicle receives information (or signals) related to remote control from the 5G network based on the DL grant (S58).

Referring to FIG. 6 , the autonomous vehicle performs an initial access procedure with the 5G network based on the SSB to obtain DL synchronization and system information ( S60 ).

Then, the autonomous vehicle performs a random access procedure with the 5G network for UL synchronization acquisition and/or UL transmission (S61).

Then, the autonomous vehicle receives a UL grant to the 5G network to transmit specific information (S62).

The UL grant includes information on the number of repetitions for the transmission of the specific information, and the specific information is repeatedly transmitted based on the information on the number of repetitions (S63).

And, the autonomous vehicle transmits specific information to the 5G network based on the UL grant.

In addition, repeated transmission of specific information may be performed through frequency hopping, transmission of the first specific information may be transmitted in a first frequency resource, and transmission of the second specific information may be transmitted in a second frequency resource.

The specific information may be transmitted through a narrowband of 6RB (Resource Block) or 1RB (Resource Block).

Then, the autonomous vehicle receives a DL grant for receiving a response to specific information from the 5G network (S64).

Then, the autonomous vehicle receives information (or signal) related to remote control from the 5G network based on the DL grant (S65).

The above salpin 5G communication technology may be applied in combination with the methods proposed in the present specification to be described later, or may be supplemented to specify or clarify the technical characteristics of the methods proposed in the present specification.

The vehicle described herein is connected to an external server through a communication network, and can move along a preset route without driver intervention using autonomous driving technology. The vehicle of the present disclosure may be implemented as an internal combustion engine vehicle having an engine as a power source, a hybrid vehicle having an engine and an electric motor as a power source, an electric vehicle having an electric motor as a power source, and the like.

In the following embodiments, a user may be interpreted as a driver, a passenger, or an owner of a user terminal. The user terminal may be a mobile terminal, for example, a smart phone, which is portable and capable of executing a phone call and various applications, but is not limited thereto. For example, the user terminal may be interpreted as a mobile terminal, a personal computer (PC), a notebook computer, or an autonomous vehicle system.

In an autonomous vehicle, the type and frequency of accidents can vary greatly depending on the ability to sense surrounding risk factors in real time. The route to the destination may include sections with different risk levels due to various causes, such as weather, terrain characteristics, and traffic congestion. The present disclosure guides the insurance required for each section when the user inputs a destination, and updates the insurance guide through real-time monitoring of the risk section.

At least one of an autonomous vehicle, a user terminal, and a server of the present disclosure, an artificial intelligence module, a drone (Unmanned Aerial Vehicle, UAV), a robot, an augmented reality (AR) device, a virtual reality, VR) and devices related to 5G services, etc.

For example, the autonomous driving vehicle may operate in connection with at least one artificial intelligence module or robot included in the vehicle.

For example, the vehicle may interact with at least one robot. The robot may be an Autonomous Mobile Robot (AMR) capable of driving by itself. The mobile robot is free to move because it can move by itself, and is provided with a plurality of sensors for avoiding obstacles while driving, so that it can run while avoiding obstacles. The mobile robot may be a flying robot (eg, a drone) having a flying device. The mobile robot may be a wheel-type robot having at least one wheel and moving through rotation of the wheel. The mobile robot may be a legged robot having at least one leg and moving using the leg.

The robot can function as a device that complements the convenience of vehicle users. For example, the robot may perform a function of moving a load loaded in a vehicle to a final destination of a user. For example, the robot may perform a function of guiding a user who got off the vehicle to a final destination. For example, the robot may perform a function of transporting a user who got out of a vehicle to a final destination.

At least one electronic device included in the vehicle may communicate with the robot through the communication device.

At least one electronic device included in the vehicle may provide the robot with data processed by the at least one electronic device included in the vehicle. For example, the at least one electronic device included in the vehicle may include at least one of object data indicating objects around the vehicle, map data, vehicle state data, vehicle location data, and driving plan data. Either one can be provided to the robot.

At least one electronic device included in the vehicle may receive, from the robot, data processed by the robot. At least one electronic device included in the vehicle may receive at least one of sensing data generated by the robot, object data, robot state data, robot position data, and movement plan data of the robot.

At least one electronic device included in the vehicle may generate a control signal based on data received from the robot. For example, the at least one electronic device included in the vehicle compares the information on the object generated by the object detection device with the information on the object generated by the robot, and generates a control signal based on the comparison result. can At least one electronic device included in the vehicle may generate a control signal to prevent interference between the movement path of the vehicle and the movement path of the robot.

At least one electronic device included in the vehicle may include a software module or a hardware module (hereinafter, referred to as an artificial intelligence module) for implementing artificial intelligence (AI). At least one electronic device included in the vehicle may input acquired data to an artificial intelligence module and use data output from the artificial intelligence module.

The artificial intelligence module may perform machine learning on input data using at least one artificial neural network (ANN). The artificial intelligence module may output driving plan data through machine learning on input data.

At least one electronic device included in the vehicle may generate a control signal based on data output from the artificial intelligence module.

According to an embodiment, at least one electronic device included in a vehicle may receive data processed by artificial intelligence from an external device through a communication device. At least one electronic device included in the vehicle may generate a control signal based on data processed by artificial intelligence.

In various embodiments of the present disclosure below, a head lamp among lamps of a vehicle will be described as an example, but various embodiments of the present disclosure will not be limited to the head lamp. For example, various embodiments of the present disclosure may be equally applicable to a rear lamp, and/or a side lamp of a vehicle.

7A illustrates sensors included in a headlamp of a vehicle according to various embodiments of the present disclosure;

Referring to FIG. 7A , each of the vehicle headlamps 701 and 703 according to various embodiments includes at least two cameras 711 , 713 , 731 , 733 , at least one lidar 721 , 723 , and at least one light source 741 and 743 .

According to various embodiments, the first headlamp 701 may include a first front camera 711 , a first side camera 731 , a first lidar 721 , and at least one first light source 741 . It may include at least one. The first headlamp 701 may be a headlamp mounted on the right front side of the vehicle, the first front camera 711 is disposed to sense the front of the vehicle, and the first side camera 731 is, It may be arranged to sense at least a part of the front and a right direction. The first lidar 721 may be disposed to sense at least a portion of the front of the vehicle and a right direction. For example, the first front camera 711 may be disposed on the left side of the interior space area of the first headlamp 701 for sensing the front of the vehicle. The first side camera 731 may be disposed on the right side of the interior space of the first headlamp 701 for sensing at least a part of the front of the vehicle and the right direction. The first lidar 721 may be disposed on the right side of the interior space of the first headlamp 701 for sensing at least a part of the front of the vehicle and the right direction. In FIG. 7A , the first side camera 731 is disposed under the first lidar 721 , but this is only an example, and various embodiments of the present disclosure are not limited thereto. For example, the first side camera 731 may be disposed in any one direction of the upper, lower, left, or right side of the first lidar 721 . According to an embodiment, the first side camera 731 and the first lidar 721 may be disposed to be spaced apart from each other, or may be disposed to directly contact each other without being spaced apart. The at least one first light source 741 may be disposed in a central area of the inner space area of the first head lamp 701 . This is only an example, and various embodiments of the present disclosure are not limited thereto. For example, the at least one first light source 741 may be disposed in a left area adjacent to the first front camera 711 among the internal spatial areas of the first headlamp 701 or to the first side camera 731 . It may be disposed in an adjacent right area. The at least one first light source 741 may be plural. For example, the first head lamp 701 may include a plurality of light sources.

According to various embodiments, the second headlamp 703 may include at least one of a second front camera 713 , a second side camera 733 , and a second lidar 723 . The second headlamp 703 may be a headlamp mounted on the left front side of the vehicle, the second front camera 713 is disposed to sense the front of the vehicle, and the second side camera 733 is the It may be arranged to sense at least a part of the front and a left direction. The second lidar 723 may be disposed to sense at least a portion of the front of the vehicle and a right direction. For example, the second front camera 713 may be disposed on the right side of the inner space area of the second head lamp 703 for sensing the front of the vehicle. The second side camera 733 may be disposed in a left area of the interior space area of the second headlamp 703 for sensing at least a part of the front of the vehicle and the left direction. The second lidar 723 may be disposed in at least a portion of the front of the vehicle and in a right area of the inner space area of the second headlamp 703 for sensing in the left direction. In FIG. 7A , the second side camera 733 is disposed under the second lidar 723 , but this is only an example, and various embodiments of the present disclosure are not limited thereto. For example, the second side camera 733 may be disposed in any one direction of the upper, lower, left, or right side of the second lidar 723 . According to an embodiment, the second side camera 733 and the second lidar 723 may be disposed to be spaced apart from each other, or may be disposed to directly contact each other without being spaced apart. The at least one second light source 743 may be disposed in a central area of the inner space area of the second head lamp 703 . This is only an example, and various embodiments of the present disclosure are not limited thereto. For example, the at least one second light source 743 may be disposed in a left area adjacent to the second front camera 713 among the internal spatial areas of the second head lamp 703 , or to the second side camera 733 . It may be disposed in an adjacent right area. The at least one second light source 743 may be plural. For example, the second head lamp 703 may include a plurality of light sources.

7B illustrates a field of view (FOV) of sensors in a headlamp in accordance with various embodiments. The headlamps mounted on the vehicle of FIG. 7B may be the headlamps 701 and 703 illustrated in FIG. 7A .

Referring to FIG. 7B , the vehicle uses the first front camera 711 included in the first headlamp 701 and the second front camera 713 included in the second headlamp 703 to the object positioned in front of the vehicle. can be sensed. A field of view of each of the first front camera 711 and the second front camera 713 may be, for example, about 40 degrees.

The vehicle senses an object located in the front and/or side by using the first lateral camera 731 included in the first headlamp 701 and the second lateral camera 733 included in the second headlamp 703 . can do. A field of view of each of the first side camera 731 and the second side camera 733 may be, for example, about 140 degrees.

The vehicle senses an object located in the front and/or side by using the first lidar 721 included in the first headlamp 701 and the second lidar 723 included in the second headlamp 703 . can do. A field of view of each of the first lidar 721 and the second lidar 723 may be, for example, about 120 degrees.

The above-described viewing angles are merely examples, and various embodiments of the present disclosure are not limited thereto. For example, the viewing angles of the sensors 711 , 731 , 721 , 723 , 731 , and 733 included in the first head lamp 701 and the second head lamp 703 may be set to different angles by a designer. there is.

In addition, in the above-described embodiments of FIGS. 7A and 7B and to be described later, it is assumed that two headlamps 701 and 703 located in the front of the vehicle are described, but various embodiments of the present disclosure are not limited thereto. For example, various embodiments of the present disclosure may be applied in the same manner to each of the lamps located on the side and/or rear of the vehicle.

Hereinafter, for convenience of description, the first front camera 711 illustrated in FIGS. 7A and 7B is referred to as a right front camera, the second front camera 713 is referred to as a left front camera, and the first side The camera 731 may be referred to as a right side camera, and the second side camera 733 may be referred to as a left side camera.

8 is a control block diagram of a vehicle according to an embodiment of the present disclosure;

Referring to FIG. 8 , the vehicle includes a user interface device 800 , an object detection device 810 , a communication device 820 , a driving manipulation device 830 , a main ECU 840 , a driving control device 850 , and an autonomous vehicle. It may include a driving device 860 , a sensing unit 870 , and a location data generating device 880 . The object detecting device 810 , the communication device 820 , the driving manipulation device 830 , the main ECU 840 , the driving control device 850 , the autonomous driving device 860 , the sensing unit 870 , and the location data generating device 880 may be implemented as electronic devices that each generate electrical signals and exchange electrical signals with each other.

The user interface device 800 is a device for communication between a vehicle and a user. The user interface device 800 may receive a user input and provide information generated in the vehicle to the user. The vehicle may implement a user interface (UI) or a user experience (UX) through the user interface device 800 . The user interface device 800 may include an input device, an output device, and a user monitoring device.

The object detecting apparatus 810 may generate information about an object outside the vehicle. The information about the object may include at least one of information on the existence of the object, location information of the object, distance information between the vehicle and the object, and relative speed information between the vehicle and the object. The object detecting apparatus 810 may detect an object outside the vehicle. The object detecting apparatus 810 may include at least one sensor capable of detecting an object outside the vehicle. The object detecting apparatus 810 may include at least one of a camera, a radar, a lidar, an ultrasonic sensor, and an infrared sensor. The object detecting apparatus 810 may provide data on an object generated based on a sensing signal generated by a sensor to at least one electronic device included in the vehicle.

The camera may generate information about an object outside the vehicle by using the image. The camera may include at least one lens, at least one image sensor, and at least one image signal processor. The image signal processor may be electrically connected to the image sensor, process a signal received from the image sensor, and generate data about the object based on the processed signal. The camera may be at least one of a mono camera, a stereo camera, and an AVM (Around View Monitoring) camera. The camera may obtain position information of the object, distance information from the object, or relative speed information with the object by using various image processing algorithms. For example, the camera may acquire distance information and relative velocity information from an object based on a change in the size of the object over time from the acquired image. For example, the camera may acquire distance information and relative speed information with respect to an object through a pinhole model, road surface profiling, or the like. For example, the camera may acquire distance information and relative velocity information from an object based on disparity information in a stereo image obtained from the stereo camera.

The camera may be mounted at a position where a field of view (FOV) can be secured in the vehicle in order to photograph the outside of the vehicle. The camera may be disposed adjacent to the front windshield in the interior of the vehicle to acquire an image of the front of the vehicle. The camera may be placed around the front bumper or radiator grill. The camera may be disposed adjacent to the rear glass in the interior of the vehicle to acquire an image of the rear of the vehicle. The camera may be placed around the rear bumper, trunk or tailgate. The camera may be disposed adjacent to at least one of the side windows in the interior of the vehicle in order to acquire an image of the side of the vehicle. Alternatively, the camera may be disposed around a side mirror, a fender or a door.

The radar may generate information about an object outside the vehicle using radio waves. The radar may include an electromagnetic wave transmitter, an electromagnetic wave receiver, and at least one processor. The at least one processor may be electrically connected to the electromagnetic wave transmitter and the electromagnetic wave receiver, process the received signal, and generate data for the object based on the processed signal. The radar may be implemented in a pulse radar method or a continuous wave radar method in terms of a radio wave emission principle. The radar may be implemented as a frequency modulated continuous wave (FMCW) method or a frequency shift keying (FSK) method according to a signal waveform among continuous wave radar methods. The radar detects an object based on an electromagnetic wave, a time of flight (TOF) method or a phase-shift method, and detects the position of the detected object, the distance to the detected object, and the relative speed. can The radar may be placed at a suitable location outside of the vehicle to detect objects located in front, rear or side of the vehicle.

The lidar may generate information about an object outside the vehicle by using the laser light. The lidar may include a light transmitter, a light receiver, and at least one processor. It may be electrically connected to the light transmitter and the light receiver, process a received signal, and generate data for an object based on the processed signal. The lidar may be implemented in a time of flight (TOF) method or a phase-shift method. Lidar can be implemented as driven or non-driven. When implemented as a driving type, the lidar is rotated by a motor and can detect objects around the vehicle. When implemented as a non-driven type, the lidar may detect an object located within a predetermined range with respect to the vehicle by light steering. Vehicle 100 may include a plurality of non-driven lidar. LiDAR detects an object based on a time of flight (TOF) method or a phase-shift method with a laser light medium, and calculates the position of the detected object, the distance to the detected object, and the relative speed. can be detected. The lidar may be placed at a suitable location outside of the vehicle to detect an object located in front, rear or side of the vehicle.

The communication apparatus 820 may exchange signals with a device located outside the vehicle. The communication device 820 may exchange signals with at least one of an infrastructure (eg, a server, a broadcasting station), another vehicle, and a terminal. The communication device 820 may include at least one of a transmission antenna, a reception antenna, a radio frequency (RF) circuit capable of implementing various communication protocols, and an RF element to perform communication.

For example, the communication device may exchange signals with an external device based on V2X technology. For example, the V2X technology may include LTE-based sidelink communication and/or NR-based sidelink communication. In addition, the communication device can exchange information with objects such as other vehicles, mobile devices, and roads based on the 5G network. The contents related to V2X will be described later.

For example, the communication device is based on IEEE 80211p PHY/MAC layer technology and IEEE 1609 Network/Transport layer technology-based Dedicated Short Range Communications (DSRC) technology or WAVE (Wireless Access in Vehicular Environment), SAEJ2735, SAE J2945 standards. Signals can be exchanged with external devices. DSRC (or WAVE standard) technology is a communication standard prepared to provide an Intelligent Transport System (ITS) service through short-distance dedicated communication between in-vehicle devices or between roadside devices and in-vehicle devices. The DSRC technology may use a frequency of the 59 GHz band and may be a communication method having a data transmission rate of 3 Mbps to 27 Mbps. The IEEE 80211p technology may be combined with the IEEE 1609 technology to support DSRC technology (or WAVE standard).

The communication apparatus of the present disclosure may exchange a signal with an external device using only one of the V2X technology or the DSRC technology. Alternatively, the communication apparatus of the present disclosure may exchange a signal with an external device by hybridizing the V2X technology and the DSRC technology. The V2X standard was created through IEEE (IEEE 80211p, IEEE 1609), a standardization organization in the electrical and electronic field, and SAE (SAE J2735, SAE J2945, etc.), a group of automotive engineers, and standardizes the physical layer, SW stack, and application layer, respectively. in charge In particular, with respect to the message standard, SAE established standards for defining the message standard for V2X communication.

The driving operation device 830 is a device that receives a user input for driving. In the manual mode, the vehicle may be driven based on a signal provided by the driving manipulation device 830 . The driving manipulation device 830 may include a steering input device (eg, a steering wheel), an accelerator input device (eg, an accelerator pedal), and a brake input device (eg, a brake pedal).

The main ECU 840 may control the overall operation of at least one electronic device included in the vehicle.

The drive control device 850 is a device that electrically controls various vehicle drive devices in the vehicle. The drive control device 850 may include a power train drive control device, a chassis drive control device, a door/window drive control device, a safety device drive control device, a lamp drive control device, and an air conditioning drive control device. The power train drive control device may include a power source drive control device and a transmission drive control device. The chassis drive control device may include a steering drive control device, a brake drive control device, and a suspension drive control device. Meanwhile, the safety device drive control device may include a safety belt drive control device for seat belt control.

The drive control device 850 includes at least one electronic control device (eg, a control ECU (Electronic Control Unit)).

The driving control device 850 may control the vehicle driving device based on a signal received from the autonomous driving device 860 . For example, the drive control device 850 may control a power train, a steering device, and a brake device based on a signal received from the autonomous driving device 860 .

The autonomous driving device 860 may generate a path for autonomous driving based on the obtained data. The autonomous driving device 860 may generate a driving plan for driving along the generated path. The autonomous driving device 860 may generate a signal for controlling the movement of the vehicle according to the driving plan. The autonomous driving device 860 may provide the generated signal to the driving control device 850 .

The autonomous driving device 860 may implement at least one Advanced Drive Assistance System (ADAS) function. ADAS includes Adaptive Cruise Control (ACC), Autonomous Emergency Braking (AEB), Forward Collision Warning (FCW), and Lane Keeping Assist (LKA). ), Lane Change Assist (LCA), Target Following Assist (TFA), Blind Spot Detection (BSD), Adaptive Headlight System (AHS) , Auto Parking System (APS), Pedestrian Collision Warning System (PD Collision Warning System), Traffic Sign Recognition (TSR), Traffic Sign Assist (TSA), Night Vision System At least one of a Night Vision (NV), a Driver Status Monitoring (DSM), and a Traffic Jam Assist (TJA) may be implemented.

The autonomous driving device 860 may perform a switching operation from the autonomous driving mode to the manual driving mode or a switching operation from the manual driving mode to the autonomous driving mode. For example, the autonomous driving device 860 may change the mode of the vehicle from the autonomous driving mode to the manual driving mode or to switch from the manual driving mode to the autonomous driving mode based on the signal received from the user interface device 800 . can

The sensing unit 870 may sense the state of the vehicle. The sensing unit 870 may include an inertial measurement unit (IMU) sensor, a collision sensor, a wheel sensor, a speed sensor, an inclination sensor, a weight sensor, a heading sensor, a position module, and a vehicle. It may include at least one of a forward/reverse sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor, a temperature sensor, a humidity sensor, an ultrasonic sensor, an illuminance sensor, and a pedal position sensor. Meanwhile, an inertial measurement unit (IMU) sensor may include at least one of an acceleration sensor, a gyro sensor, and a magnetic sensor.

The sensing unit 870 may generate state data of the vehicle based on a signal generated by at least one sensor. The vehicle state data may be information generated based on data sensed by various sensors provided inside the vehicle. The sensing unit 870 may include vehicle attitude data, vehicle motion data, vehicle yaw data, vehicle roll data, vehicle pitch data, vehicle collision data, vehicle direction data, vehicle angle data, and vehicle speed. data, vehicle acceleration data, vehicle inclination data, vehicle forward/reverse data, vehicle weight data, battery data, fuel data, tire pressure data, vehicle interior temperature data, vehicle interior humidity data, steering wheel rotation angle data, vehicle exterior illumination Data, pressure data applied to the accelerator pedal, pressure data applied to the brake pedal, and the like may be generated.

The location data generating apparatus 880 may generate location data of the vehicle. The location data generating apparatus 880 may include at least one of a Global Positioning System (GPS) and a Differential Global Positioning System (DGPS). The location data generating apparatus 880 may generate location data of the vehicle based on a signal generated by at least one of GPS and DGPS. According to an embodiment, the location data generating apparatus 880 may correct the location data based on at least one of an Inertial Measurement Unit (IMU) of the sensing unit 870 and a camera of the object detecting apparatus 810 . The location data generating device 880 may be referred to as a Global Navigation Satellite System (GNSS).

The vehicle may include an internal communication system 855 . A plurality of electronic devices included in the vehicle may exchange signals through the internal communication system 855 . Signals may contain data. The internal communication system 855 may use at least one communication protocol (eg, CAN, LIN, FlexRay, MOST, Ethernet).

9 is a block diagram of an electronic device included in a vehicle according to various embodiments of the present disclosure; The electronic device of FIG. 9 may be at least a part of the electronic device included in the vehicle of FIGS. 7A, 7B, and 8 . The configuration of the electronic device 900 illustrated in FIG. 9 is an example, and each component may be composed of one chip, component, or electronic circuit, or a combination of chips, components, or electronic circuit. According to another embodiment, some of the components shown in FIG. 9 may be divided into a plurality of components and configured as different chips or components or electronic circuits, and some components may be combined to form one chip, component, or It may consist of an electronic circuit. According to another embodiment, some of the components shown in FIG. 9 may be omitted or other components not shown in FIG. 9 may be added. According to an embodiment, some of the components shown in FIG. 9 are not included in the electronic device 900 but are included in the vehicle, and thus are electrically connected to at least one other component included in the electronic device 900 . can be connected to According to an embodiment, at least some of the components shown in FIG. 9 may be devices included in the vehicle of FIG. 8 . Hereinafter, at least some operations of the components of FIG. 9 may be described with reference to FIGS. 10A and 10B . 10A and 10B are exemplary views of detecting an object using front cameras and side cameras in a vehicle according to various embodiments of the present disclosure;

Referring to FIG. 9 , an electronic device 900 according to various embodiments may include a processor 910 , a sensor module 920 , a light source 930 , and a memory 940 .

The processor 910 may perform overall control operations necessary for autonomous driving of the vehicle. The processor 910 may include the autonomous driving device 833 of FIG. 8 .

According to various embodiments, the processor 910 detects surrounding objects using a plurality of sensors, and performs autonomous driving of a vehicle including the electronic device 900 (hereinafter referred to as 'own vehicle') based on the detection result. can be controlled According to an embodiment, the processor 910 may detect the surroundings using at least one front sensor and at least one side sensor, and control autonomous driving of the own vehicle using the detection result. The at least one front sensor may include, for example, at least one of a left front camera 713 and a right front camera 711 . The at least one lateral sensor may include, for example, at least one of a left lateral camera 733 , a right lateral camera 731 , a first lidar 721 , and a second lidar 723 . According to an embodiment, the processor 910 is configured to perform a left lateral camera 733, a right lateral camera 731, and a first lateral camera based on at least one of time information, brightness information of an area to be detected, and driving environment information. At least one lateral sensor among the IDA 721 and the second lidar 723 may be selected, and an object in the lateral region may be detected using the selected lateral sensor. For example, the processor 910 may determine whether it is daytime or nighttime based on the current time information. When the current time information corresponds to the daytime, the processor 910 may detect an object in the lateral region using at least one of the left lateral camera 733 and the right lateral camera 731 . In this case, the first lidar 721 or the second lidar 723 may be in an inactivated state. When the current time information corresponds to nighttime, the processor 910 is configured to include at least one of the left side camera 733 and the right side camera 731, and the first lidar 721, or the second lidar ( 723), an object in the lateral region may be detected using at least one lidar. In the case of night, the use of the camera and lidar at the same time is because it is difficult to accurately detect an object with only the camera due to the brightness in the lateral direction. As another example, when the current time information corresponds to the daytime, but the brightness of the area to be detected is less than or equal to the threshold brightness, the processor 910 may include at least one of the left side camera 733 and the right side camera 731, and An object in the lateral region may be detected using at least one lidar of the first lidar 721 and the second lidar 723 . As another example, if the current time information corresponds to the daytime information, but the driving environment information indicates that the vehicle is driving adjacent to a specific feature (eg, a guard rail) on a highway, It is possible to detect a specific feature in the lateral area using the radar (eg, the first lidar 721 and the second lidar 723 ), and perform autonomous driving based on the detection result. In this case, a side camera (eg, a left side camera 733 or a right side camera 731 ) in a direction adjacent to a specific feature may be deactivated. According to one embodiment, when using at least one side camera (731, 733) and at least one lidar (721, 723) at the same time, the processor 910 uses at least one side camera (731, 733) Information on the detected first object may be compared with information on the first object detected using at least one lidar 721 and 723 . The processor 910 may improve detection accuracy of the first object by irradiating light onto an area in which the first object exists using at least one light source when the information of the first object is different than the specified level as a result of the comparison. can

According to various embodiments, the processor 910 uses a side sensor to detect an object in a blind spot that cannot be detected by a front sensor according to a shape of a road, a structure around the road, or a driving operation of a vehicle while driving. additionally available. The processor 910 may monitor the blind spot by using a side sensor and obtain information on at least one object located in the blind spot. The processor 910 determines a collision risk (or collision level) indicating a possibility that a collision with the own vehicle and at least one object will occur based on information about at least one object located in the blind spot, and A collision with at least one object may be prevented by performing autonomous driving based on the autonomous driving method. For example, as shown in FIG. 10A , when the processor 910 turns right at an intersection, the left front camera image 1002 and the right front camera image 1003 are displayed due to the shape of the road and the driving operation of the own vehicle. The area of the blind spot that cannot be detected through the camera may be monitored through the left lateral camera image 1001 and the right lateral camera image 1004 . The left lateral camera image 1001 and the right lateral camera image 1004 may vary for each viewpoint 1011 , 1012 , and 1013 according to the driving operation of the own vehicle. The processor 910 includes at least one other vehicle 1033 , 1035 , 1043 located in the blind spot area 1031 , 1041 through the left lateral camera image 1001 and the right lateral camera image 1004 , and at least one of pedestrians 1045 and 1047 can be detected. The processor 910 performs autonomous driving so that the own vehicle does not collide with at least one other vehicle 1033 , 1035 , 1043 located in the blind spot area 1031 , 1041 , and at least one pedestrian 1045 , 1047 . can As another example, as shown in FIG. 10B , when driving straight, the processor 910 is an area of a blind spot that cannot be detected through the left front camera image 1002 and the right front camera image 1003 due to the road shape 1051) can be monitored through the right side camera image 1004. The processor 1004 may detect the presence of at least one other vehicle through the right side camera image 1004 , and perform autonomous driving of the own vehicle based on the detection result.

According to various embodiments, the processor 910 may determine the driving state of the first front vehicle located in the first area using at least one front sensor. The at least one front sensor may include at least one of a front camera having a viewing angle toward the front area of the vehicle, or a front lidar that irradiates a laser to the front area of the vehicle. According to an embodiment, the processor 910 obtains information on the first front vehicle located in the front region of the vehicle by analyzing the image obtained from at least one front camera, and based on the obtained information, The driving condition can be determined. The first front vehicle may include at least one of a front vehicle positioned on a travel path of the own vehicle and a front vehicle adjacent to a travel path of the own vehicle. The driving state may include at least one of a high-speed driving state, a normal driving state, a low-speed driving state, and a stop state (or a stationary state). The information of the first front vehicle includes whether the wheels of the first front vehicle rotate, wheel rotation speed, wheel rotation angle, wheel size, movement speed, movement direction, rear lamp (or rear lamp) information, rearview mirror state information, vehicle body height, It may include at least one of a vehicle body length, a vehicle type, and a distance to the own vehicle. The rear lamp information may include information indicating whether a light source is being irradiated from at least one of a brake lamp and a turn signal lamp. The rearview mirror state information may include information indicating whether the rearview mirror is in a folded state or an unfolded state.

According to various embodiments, the processor 910 may control a function for autonomous driving of the vehicle in which the electronic device 900 is mounted based on the driving state of the first front vehicle. According to an embodiment, the processor 910 may determine whether to additionally use at least one lateral sensor for autonomous driving based on whether the driving state of the first front vehicle corresponds to a specified state. For example, when the driving state of the first front vehicle does not correspond to the specified state, the processor 910 may control autonomous driving of the own vehicle based on information obtained using at least one front sensor. When the driving state of the first front vehicle corresponds to the specified state, the processor 910 may control autonomous driving of the own vehicle based on information obtained using at least one front sensor and at least one side sensor. The designated state may include at least one of a stopped state and a low-speed driving state below a threshold speed.

According to an embodiment, when the driving state of the first front vehicle corresponds to the specified state, the processor 910 may obtain information on at least one object located in the second area using at least one lateral sensor. there is. The at least one lateral sensor may include at least one of a lateral camera having a viewing angle toward a partial region and the lateral region of the front region of the vehicle, or a lateral lidar irradiating a laser to a partial region and lateral region of the front region of the vehicle can The second area may include at least a partial area of the first area detectable by the at least one front sensor and at least some area of the area of the blind spot undetectable by the at least one front sensor. At least some of the areas of the blind spot are areas that do not overlap the first area, and may include areas in the lateral direction of the vehicle. According to an embodiment, the processor 910 may acquire information on an object located in the second area by analyzing an image obtained from at least one side camera. The at least one object located in the second area may include at least one of at least one other vehicle, a pedestrian, or a structure. The at least one object located in the second area may include at least one of an object located in a front area and/or a side area of the first front vehicle. For example, the processor 910 uses at least one of a left side camera and a right side camera to determine the wheel rotation speed, wheel rotation angle, movement speed, movement direction, and rear lamp of at least one other vehicle located in the second area. Information indicating at least one of information, rearview mirror state information, vehicle location, vehicle body height, vehicle body length, wheel size, vehicle type, and distance to the own vehicle may be acquired. As another example, the processor 910 obtains information indicating the position of the pedestrian located in the second area, the size of the pedestrian, the movement speed of the pedestrian, or whether the pedestrian is moving by using at least one of the left lateral camera and the right lateral camera. can do. As another example, the processor 910 may obtain information indicating at least one of a type, shape, location, characteristic, or size of a structure located in the second area using at least one of a left side camera and a right side camera. there is. For example, the structure may include at least one of a traffic light, a street tree, a street light, a building, a bridge, or a tunnel.

According to embodiments, the processor 910 may control the location of the vehicle to be changed in order to obtain information on at least one object located in the second area. For example, the processor 910 may change a lane, move a position of a vehicle to the left and/or right within a lane, or move according to a driving route to change the position. For example, when the information on the second front vehicle located in the front region of the first front vehicle is not obtained through the at least one side sensor, the processor 910 transmits the own vehicle at the first position through the at least one side sensor. It may be moved to a second position where information on the second front vehicle can be obtained.

According to an embodiment, the processor 910 may control driving of the own vehicle based on driving state information of the first front vehicle and information on at least one object. For example, the processor 910 may analyze the intention or cause of the driving state of the first front vehicle based on the information on the at least one object. For example, when the first front vehicle is in a stopped state, the processor 910 may determine the stopping intention and/or cause of the first front vehicle based on information about an object obtained through at least one side sensor. The processor 910 may predict the next driving state of the first front vehicle based on the intention for the driving state of the first front vehicle. The processor 910 may determine whether to change the driving path of the own vehicle and/or whether to stop the vehicle based on the next driving state of the first front vehicle. For example, when it is predicted that the stopped state of the first front vehicle will continue for a specified threshold time or longer, the processor 910 may change the driving path of the own vehicle and drive the vehicle in the changed driving path. As another example, when it is predicted that the first front vehicle will stop and travel within a threshold time, the processor 910 causes the own vehicle to stop while the first front vehicle is in a stopped state while maintaining the own vehicle's travel route, When the first front vehicle is in the driving state, it is possible to control the vehicle to travel according to a driving route based on the driving of the first front vehicle.

According to various embodiments, the processor 910 may control at least one lateral sensor based on the collision risk between the own vehicle and objects (eg, a pedestrian, a vehicle, and/or a structure) around the own vehicle. For example, the processor 910 may detect at least one other vehicle in the vicinity of the own vehicle and determine the risk of collision between the own vehicle and the at least one other vehicle. The collision risk is based on at least one of the distance between the own vehicle and the object, the moving direction of the own vehicle and the object, the speed of the own vehicle, the speed of the object, the relative speed of the own vehicle and the object, or the relative moving direction between the own vehicle and the object. can be decided. The collision risk may be expressed in stages or numerically. For example, the collision risk may be expressed as one of high, medium, and low, or one of the first to Nth stages. As another example, the collision risk may be expressed as a numerical value representing a probability. The expression of the collision risk is merely exemplary, and various embodiments of the present disclosure are not limited thereto. According to an embodiment, the processor 910 increases the frame rate of the at least one side camera or scans the at least one side lidar when there is a surrounding object having a collision risk exceeding the specified first risk level. The resolution can be increased. For example, the processor 910 increases the frame rate of the at least one side camera from about 30 Hz to about 40 Hz, or increases the scanning resolution of the at least one side camera by changing the laser irradiation range angle for each point from about 0.1 deg to about 0.2 deg. can be increased to According to an embodiment, when there is no surrounding object having a collision risk exceeding the specified second risk level, the processor 910 may determine that an object capable of colliding at least temporarily does not exist in the own vehicle. The processor 910 may minimize power consumption by controlling at least one lidar to be deactivated at least temporarily when there is no surrounding object having a collision risk exceeding the second risk level. For example, the processor 910 may control to stop supplying power to at least one lidar, or control the at least one lidar to operate in at least one of a low power mode and a sleep mode.

According to various embodiments, the processor 910 may change a setting for an operation of at least one sensor based on information on at least one object located in the second area. According to an embodiment, the processor 910 may be configured to select one of a laser irradiation range, an irradiation period (or a transmission period, an output period) of the lidar, or a laser output intensity based on information on at least one object located in the second area. At least one can be changed. For example, the laser irradiation range of the lidar may be set based on the size and/or position of at least one object located in the second area, and the laser period and/or laser output according to the distance information of the at least one object You can set the intensity.

According to various embodiments, the processor 910 may operate a plurality of sensors based on the data transmission capacity inside the vehicle. The data transmission capacity inside the vehicle may be the maximum data capacity that can be processed in the network inside the vehicle.

According to one embodiment, the processor 910 is configured such that the amount of data obtained from the plurality of lidars during a specified time period is less than or equal to the maximum data capacity that can be processed in the network inside the vehicle, the laser irradiation period of the plurality of lidars; And/or it is possible to adjust the laser irradiation range. For example, the processor 910 may control the left lidar and the right lidar to acquire data at different time points by setting the irradiation period of the left lidar and the right lidar to be different from each other. Accordingly, the amount of data obtained at one time from a plurality of lidars is less than or equal to the maximum data capacity that can be processed in the network inside the vehicle, thereby preventing a bottleneck from occurring in the communication network between components inside the vehicle. . As another example, the processor 910 may adjust the laser irradiation range of at least one of the left lidar and the right lidar from all laser points to some laser points. For example, the processor 910 sets the laser irradiation range of the left lidar to correspond to the first horizontal range and/or the first vertical range, and sets the laser irradiation range of the right lidar to the second horizontal range, and/or the second 2 It can be set to correspond to the vertical range. The first horizontal range and the second horizontal range may be different from or the same as each other. The first vertical extent and the second vertical extent may be different from or the same as each other. The first horizontal range and/or the first vertical range may be determined and/or changed based on at least one of a location, size, or direction of movement of an object to be detected (or an object to be monitored) in each lidar. there is.

According to an embodiment, the processor 910 obtains size and/or shape information of an object expressed in a 3D coordinate form by irradiating a laser using all laser points of at least one lidar upon initial detection of the object. can be obtained The processor 910 may determine at least one representative laser point to detect the object based on the size and/or shape information of the object, and detect the corresponding object using only the determined at least one representative laser point. The representative laser point may group each laser point using 3D coordinates and determine at least one laser point for each group as the representative laser point. In addition, the processor 910 may adjust the laser output intensity of the lidar based on the distance between the own vehicle and the object. For example, the processor 910 sets the laser output intensity for the representative laser point to be small in proportion to the distance when the distance between the own vehicle and the object gradually gets closer, and when the distance between the own vehicle and the object gradually increases, the representative laser The laser power intensity for the point can be set to be large in proportion to the distance.

According to various embodiments, the processor 910 may be configured to operate in any one of a mode using at least one camera and at least one lidar together, or a mode using only at least one lidar, based on the characteristics of the road section being driven. can operate as For example, the processor 910 may determine whether a road section in which the own vehicle is traveling satisfies a specified condition (eg, presence of a guard rail) based on at least one of map data or lidar scanning information. For example, when the road section in which the own vehicle is driving is a highway section in which a guard rail exists, the processor 910 operates in a mode using only lidar, and obtains information of surrounding terrain features using at least one lidar. can When the section in which the own vehicle is traveling is a general road section in which a guard rail does not exist, the processor 910 sets a mode in which a camera and a lidar are used together, and detects an object using at least one camera and at least one lidar can do. According to an embodiment, during the mode using only the lidar, at least one camera may be switched to an inactive state. The deactivation state of the at least one camera may include at least one of a state in which power is not supplied to the camera, a state in a sleep mode, and a state in a low power mode.

The sensor module 920 may include at least one front sensor for detecting a front area of the vehicle, and at least one side sensor for detecting a side area of the vehicle. Each of the front sensor and the side sensor may include at least one of a camera and a lidar. According to various embodiments, the sensor module 920 may include a camera module 922 and a lidar module 924 . According to an embodiment, the sensor module 920 may include at least one of the object detecting apparatus 810 and the sensing unit 870 of FIG. 8 .

The camera module 922 may acquire at least one image under the control of the processor 910 using at least one camera, and detect an object using the acquired image. According to an embodiment, the camera module 922 may include a left lateral camera 733 , a left lateral camera 713 , a right lateral camera 731 , or a right lateral camera 731 , as shown in FIGS. 7A and/or 7B . At least one of the cameras 711 may be included.

According to various embodiments, at least one camera included in the camera module 922 acquires an image of at least some of the detection regions under the control of the processor 910, and analyzes the acquired image to obtain an object ( For example, other vehicles or pedestrians) can be created. The camera module 922, for example, analyzes the image through an image signal processor to determine the radius of the wheel of another vehicle, the number of revolutions per second of the wheel, the angle of the wheel, the size of the wheel, the state of the rear lamp, the state of the rearview mirror, the vehicle type, Alternatively, data indicating at least one of a distance to the host vehicle may be generated. According to an embodiment, the image signal processor may be configured as at least a part of the processor 910 or as a separate processor separate from the processor 910 . According to an embodiment, the camera module 922 may control the active state of at least one camera according to the control of the processor 910 . For example, the camera module 922 may change at least one of the left lateral camera 733 and the right lateral camera 731 from an inactive state to an active state. The inactive state of the camera may include at least one of a state in which power supply to the camera is stopped, a state in which the camera is in a sleep mode, or a state in which the camera operates in a low power mode. The active state of the camera may include a state in which at least one image is acquired and processed by the camera.

The lidar module 924 may detect an object by emitting a laser beam under the control of the processor 910 . According to an embodiment, the lidar module 924 may include at least one of a first lidar 721 and a second lidar 723 as shown in FIGS. 7A and/or 7B . there is. According to one embodiment, the lidar module 924 emits a laser beam in a direction requested by the processor 930 when the vehicle is driving at night, and based on the laser beam reflected and received by the object, data can be generated. According to one embodiment, the lidar module 924 is the first lidar 721 or the second lidar 723 according to the control of the at least one laser irradiation period, laser irradiation range, laser At least one of the output strengths may be set and/or changed.

The light source 930 may emit light under the control of the processor 910 . The light source 930 may emit light to a designated area under the control of the processor 930 in order to improve the object detection accuracy of the camera.

The memory 940 may store various data and/or programs necessary for the operation of the processor 910 . According to an embodiment, the memory 940 may store high-precision map data. According to an embodiment, the memory 940 may store vehicle type information according to the wheel size of the vehicle, the vehicle body height, or the vehicle body length.

11 is a flowchart illustrating autonomous driving using a plurality of sensors in a vehicle according to various embodiments of the present disclosure; In the following embodiment, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Here, the vehicle may include the electronic device 900 of FIG. 9 .

Referring to FIG. 11 , in operation 1101 , the vehicle may determine the driving state of the first front vehicle in the first area using at least one front sensor. According to embodiments, the processor 910 and/or the sensor module 920 included in the electronic device 900 of the vehicle may include a left front camera (second front camera 713 ) or a right front camera (second front camera). Information on the first front vehicle located in the first area may be obtained using at least one of the first front cameras 711), and a driving state of the first front vehicle may be determined based on the obtained information. The first front vehicle may include at least one of a front vehicle positioned on a travel path of the own vehicle and a front vehicle adjacent to a travel path of the own vehicle. For example, while the vehicle performs a right turn operation at an intersection, the first front vehicle may include at least one other vehicle positioned in a right turn direction of the vehicle. The vehicle uses at least one of the left front camera 713 or the right front camera 711 to display an image including at least a portion of a rear surface of the first front vehicle and/or at least a portion of a side surface of the first front vehicle. can be obtained In the acquired image, the vehicle includes information on whether the wheels of the first front vehicle rotate, wheel rotation speed, wheel rotation angle, wheel size, rear lamp information, rearview mirror state information, body height, body length, body color, and distance to the own vehicle in the acquired image. At least one of them can be obtained. According to an embodiment, the vehicle may determine the driving state of the first front vehicle based on at least one of whether the wheels of the first front vehicle rotate, wheel rotation speed, rear lamp information, or rearview mirror state information. The rear lamp information may include information indicating a blinking state of at least one of a brake lamp, a turn signal lamp, and an emergency lamp. The driving state may include at least one of a high-speed driving state, a normal driving state, a low-speed driving state, and a stop state. For example, when the wheels of the first front vehicle are not rotating or when the brake lamp of the rear lamp is turned on, the processor 910 may determine the driving state of the first front vehicle as the stopped state. As another example, when the wheel rotation speed of the first front vehicle corresponds to the specified first rotation speed range, the processor 910 may determine the driving state of the first front vehicle as the low-speed driving state. As another example, when the wheel rotation speed of the first front vehicle corresponds to the specified second rotation speed range, the processor 910 determines the driving state of the first front vehicle as a normal driving state, and the wheels of the first front vehicle When the rotation speed corresponds to the specified third rotation speed range, the driving state of the first front vehicle may be determined as the high-speed driving state. The above-described method of determining the driving state is merely an example, and various embodiments of the present disclosure are not limited thereto. For example, the vehicle according to various embodiments of the present disclosure may determine the driving state of the first front vehicle by using various pieces of information acquired through the front sensor.

In operation 1103 , the vehicle may determine whether the driving state of the first front vehicle corresponds to the specified driving state. The designated driving state may be a stopped state. For example, the processor 910 may determine whether the driving state of the first front vehicle is a stopped state.

When the driving state of the first front vehicle corresponds to the specified driving state, in operation 1105 , the vehicle may acquire object information of the second area using a side sensor. For example, when the first front vehicle is in a stopped state, the processor 910 may obtain information on at least one object located in the second area using at least one side sensor. According to an embodiment, the at least one lateral sensor may include at least one of a left lateral camera 733 and a right lateral camera 731 . According to an embodiment, the at least one lateral sensor may be operated in a deactivated state during operations 1101 and 1103 , and may be activated by the processor 910 and/or the camera module 922 . According to an embodiment, the at least one lateral sensor may be in an activated state during operations 1101 and 1103 . The second area includes at least a partial area of the first area detectable by the left front camera 713 and the right front camera 711 , and an area of a blind spot that cannot be detected by the left front camera 713 and the right front camera 711 . may include at least a partial region of the For example, the processor 910 and/or the camera module 922 may use at least one of the left front camera 713 and the right front camera 711 to determine the first front vehicle located in the second area, at least Information about one other vehicle, pedestrian, or structure may be obtained. For example, the processor 910 , and/or the camera module 922 may use at least one of the left front camera 713 and the right front camera 711 to perform at least a part of the first front vehicle, Information on at least one other vehicle, pedestrian, or traffic light located in the front area and/or the side area may be acquired. The information on at least a part of the first front vehicle may include at least one of rearview mirror state information located on a side surface of the first front vehicle and rear lamp information located on the rear side of the first front vehicle. Information about other vehicles includes: wheel rotation speed, wheel rotation angle, movement speed, movement direction, rear lamp information, rearview mirror status information, vehicle position, vehicle body height, body length, wheel size, vehicle type, or distance from the own vehicle It may include information indicating at least one of The information about the pedestrian may include information indicating at least one of a location of the pedestrian, a size of the pedestrian, a moving speed of the pedestrian, or whether the pedestrian moves. The information on the traffic light may include information indicating at least one of the type of the traffic light and information on the blinking state of the traffic light.

In operation 1107, the vehicle may control the driving of the vehicle based on the driving state of the first front vehicle and the object information. For example, the processor 910 may analyze an intention and/or a cause of the driving state of the first front vehicle based on information on at least one object. For example, when the first front vehicle is in a stopped state, the processor 910 may determine that the first front vehicle is another vehicle located in a front area of the first front vehicle based on information about an object obtained through at least one side sensor. Whether the vehicle is stopped by the parking/stop of It may be determined whether the vehicle has stopped for boarding of a pedestrian detected in the lateral area of the first forward vehicle. According to an embodiment, the processor 910 predicts the next driving state of the first front vehicle based on the intention for the driving state of the first front vehicle, and determines the driving path of the own vehicle based on the predicted next driving state. can be maintained or changed. For example, when it is determined that the first front vehicle is being stopped by the pedestrian, the processor 910 may predict the next driving state of the first front vehicle as the driving state and maintain the driving path of the own vehicle. As another example, when it is determined that the first front vehicle is being stopped by another vehicle parked/stopped in the front area, the processor 910 predicts the next driving state of the first front vehicle as the stopped state, and the driving of the own vehicle You can change the path. In the above description, the intention and/or the cause of the driving state of the first front vehicle will not be limited to the listed examples.

In FIG. 10 described above, it has been described that the designated driving state is the stopped state as an example, but various embodiments of the present disclosure are not limited thereto. For example, the designated driving state may include a stopped state and/or a low-speed driving state.

12 is a flowchart illustrating autonomous driving of a host vehicle based on a driving state of a front vehicle in a vehicle according to various embodiments of the present disclosure; At least some operations of FIG. 12 may be detailed operations of operations 1103 , 1105 , and 1107 of FIG. 11 . In the following embodiment, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Here, the vehicle may include the electronic device 900 of FIG. 9 . Hereinafter, at least some operations of FIG. 12 will be described with reference to FIGS. 13A and 13B . 13A is an exemplary view of changing a driving path of the own vehicle based on a driving state of a front vehicle located on the driving path in a vehicle according to various embodiments of the present disclosure, and FIG. It is an exemplary diagram of controlling the driving of the own vehicle based on the driving state of the located front vehicle.

Referring to FIG. 12 , in operation 1201 , the vehicle may detect a stop of the first front vehicle. For example, as described in operation 1101 of FIG. 11 , the processor 910 of the vehicle may determine the driving state of the first front vehicle and detect that the determined driving state is the stopped state. The first front vehicle may be another vehicle located on the driving path of the own vehicle. For example, as shown in FIG. 13A , the vehicle 1301 may detect a stop of the first front vehicle 1321 located on the driving path L1 1311 of the own vehicle. As another example, as shown in FIG. 13B , the vehicle 1351 may detect a stop of the first front vehicle 1362 located on the straight travel path of the own vehicle.

In operation 1202 , the vehicle may determine whether the stop of the first front vehicle is a temporary stop. According to an exemplary embodiment, the processor 910 of the vehicle temporarily detects the first front vehicle based on information of the first front vehicle obtained based on at least one of the at least one front camera and the at least one side camera. You can decide whether to stop or not. For example, the processor 910 determines whether the first front vehicle is stopped based on whether the wheels of the first front vehicle are rotated, and rear lamp information (eg, brake light on/off state, emergency light blinking state information, direction It may be determined whether the first front vehicle is temporarily stopped or is stopped due to parking based on at least one of the blinking state information of the indicator lamp) or the rearview mirror state information. For example, when the brake light of the first front vehicle is on and the rearview mirror is unfolded, the processor 910 may determine the stopped state of the first front vehicle as a temporary stop. As another example, when the emergency light of the first front vehicle is on and the brake light is off, the processor 910 may determine the stopped state of the first front vehicle as a stop due to an emergency rather than a temporary stop. As another example, when the brake light of the first front vehicle is on and the turn indicator light is blinking, the processor 910 may determine the stopped state of the first front vehicle as a lane change or a temporary stop due to a direction change. As another example, when all of the emergency light, the brake light, and the turn indicator light of the first front vehicle are in the off state, the processor 910 may determine the stopping state of the first front vehicle as a parking stop rather than a temporary stop.

In case of a temporary stop, the vehicle may detect at least one object located in a front area of the first front vehicle in operation 1203 . According to an exemplary embodiment, the processor 910 and/or the camera module 920 of the vehicle are configured to use at least one of the left lateral camera 733 and the right lateral camera 731 in a front area of the first front vehicle. At least one positioned object may be detected. The at least one object located in the area in front of the first front vehicle may include at least one of at least one other vehicle, a pedestrian, or a structure (eg, a traffic light). For example, as shown in FIG. 13A , the vehicle 1301 is located in the front area of the first front vehicle 1321 located on the right turn path L1 1311 of the own vehicle using the right side camera 731 . The second front vehicle 1322 may be detected. As another example, as shown in FIG. 13B , the vehicle 1351 uses the left lateral camera 733 to drive forward vehicles ( 1363, 1365) can be detected.

In operation 1205, the vehicle may predict the stopping time of the first front vehicle based on the detection result of at least one object located in the front area. According to an embodiment, the processor 910 and/or the camera module 920 analyzes the image obtained from the left side camera 733 and/or the right side camera 731 , and the front of the first front vehicle The state of at least one object located in the area may be determined. The processor 910 and/or the camera module 920 may predict the stop time of the first front vehicle based on the determined state of the at least one object. When the at least one object includes another vehicle, the state of the at least one object may include a driving state of the other vehicle (eg, a stopped state, a low-speed driving state, a normal driving state, or a high-speed driving state), a rear lamp of the other vehicle. information or rearview mirror status information. When the at least one object includes a pedestrian, the at least one object state may include at least one of a location of the pedestrian, whether the pedestrian moves, or a movement speed of the pedestrian. When the at least one object is a traffic light, the state of the at least one object may include at least one of a type of a traffic light (eg, a pedestrian traffic light, a vehicle traffic light), or color information of a signal emitting light from the traffic light. According to an embodiment, in the processor 910 and/or the camera module 920 , a state of another vehicle included in the at least one object state is a stopped state, rear lamp information indicates a brake lamp is on, and the rearview mirror is unfolded. When the state is indicated, it may be determined that the first front vehicle is stopped due to the temporary stop of another vehicle, and the stopping time of the first front vehicle may be determined as a value smaller than a specified threshold time. According to an embodiment, in the processor 910 and/or the camera module 920 , a state of another vehicle included in at least one object state is a stopped state, rear lamp information is a state in which a brake lamp is turned off, and the rearview mirror is folded. When the state is indicated, it may be determined that the first front vehicle is temporarily stopped due to parking of another vehicle, and the stopping time of the first front vehicle may be determined as a value smaller than a specified threshold time. According to an embodiment, when the other vehicle included in the at least one object state is a vehicle of a specified type (eg, a heavy equipment vehicle or a fire engine), the processor 910 and/or the camera module 920 may perform the first front It may be determined that the vehicle is temporarily stopped due to construction or an accident, and the stopping time of the first front vehicle may be determined as a value greater than a specified threshold time. According to an exemplary embodiment, the processor 910 and/or the camera module 920 determines that when the pedestrian is moving at speed A in the front area of the first front vehicle, the first front vehicle is stopped by the pedestrian crossing the road. , and the predicted stopping time may be set to a value smaller than the threshold time. In this case, the predicted stopping time may be determined based on the moving speed of the pedestrian, the current location of the pedestrian, or the width of the road. According to an embodiment, the processor 910 and/or the camera module 920 determines the predicted stop time as a threshold time when a vehicle traffic light in the front region of the first vehicle in front is emitting a red color instructing to stop the vehicle. It can be set to a smaller value. The above-described stop time prediction method is only an example for helping understanding of the present disclosure, and various embodiments of the present disclosure will not be limited thereto.

In operation 1207 , the vehicle may determine whether the predicted stop time is greater than a threshold time. For example, the processor 910 of the vehicle may determine whether the first front vehicle is expected to stop for more than the threshold time by comparing a preset threshold time with the stop time predicted in operation 1205 .

If the predicted stop time is greater than the threshold time, the vehicle may determine whether a change of the driving route is possible in operation 1209 . For example, when the first front vehicle is expected to stop for more than a threshold time, the processor 910 of the vehicle may determine whether a change of the driving route is possible. According to one embodiment, the processor 910 is based on at least one of the current location, the destination, the surrounding road conditions obtained through the front cameras 711 and 713 and the side cameras 731 and 733, or map data. , it is possible to determine whether it is possible to change the driving route.

When it is possible to change the driving path or when the vehicle is not temporarily stopped, the vehicle may change the driving path of the own vehicle in operation 1211 . For example, the processor 910 of the vehicle may change the driving path of the vehicle to L2 1312 or L3 1313 as shown in FIG. 13A . The processor 910 predicts the stop time of the second front vehicle 1322 , and when the predicted stop time of the second front vehicle 1322 is less than the threshold time, the processor 910 sets the driving path of the own vehicle as the path L2 1312 . can be changed When the predicted stopping time of the second front vehicle 1322 is greater than the threshold time, the processor 910 may change the driving path of the own vehicle to the path of the L3 1313 . Here, the paths L2 1312 and L3 1313 are exemplary, and various embodiments of the present disclosure are not limited thereto. For example, the driving path of the host vehicle may be changed to another path different from the L2 1312 and the L3 1313 .

When the predicted stopping time is less than or equal to the threshold time, or when it is impossible to change the driving route, the vehicle may control the vehicle to drive after stopping for the predicted stopping time in operation 1213 . For example, as shown in FIG. 13B , the processor 910 of the vehicle may operate the vehicle if it is impossible to change the driving route due to the shape of the road and/or other vehicles 1361 to 1368 in the vicinity of the own vehicle. In step 1213 , it is possible to control the vehicle to drive after stopping for the predicted stopping time.

14 is a flowchart of detecting a surrounding object based on a collision risk of the surrounding object in a vehicle according to various embodiments of the present disclosure; At least some operations of FIG. 14 may be detailed operations of operations 1105 and 1107 of FIG. 11 . In the following embodiment, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Here, the vehicle may include the electronic device 900 of FIG. 9 . According to an embodiment, at least some operations of FIG. 14 may be performed when at least some operations of FIG. 12 are performed. For example, at least some operations of FIG. 12 and at least some operations of FIG. 14 may be performed in parallel. Hereinafter, at least some operations of FIG. 14 will be described with reference to FIGS. 15 and 16 . 15 is an exemplary diagram of determining a collision risk of a surrounding object in a vehicle according to various embodiments of the present disclosure .

Referring to FIG. 14 , in operation 1401 , the vehicle may determine whether an object having a risk of collision exists. For example, the processor 910 of the vehicle may include at least one front sensor (eg, a left front camera 713, a right front camera 711), and at least one side sensor (eg, a left side camera 733); It may be determined whether an object having a risk of collision is detected based on the right side camera 731 , the first lidar 721 , and the second lidar 723 ). For example, when a moving object does not exist within a threshold distance from the own vehicle, the vehicle may determine that an object with a risk of collision does not exist. The vehicle may determine that an object with a risk of collision exists when a moving object exists within a threshold distance from the own vehicle. The movable object may include at least one of at least one other vehicle or a pedestrian. According to an embodiment, the distance to the object located in the lateral region of the vehicle may be obtained based on detection results of at least two sensors for the object. For example, the distance to the object located in the side area of the vehicle may be obtained based on a detection result using at least one side camera and the lidar. As another example, it may be obtained based on a detection result using at least one front camera and at least one side camera. As another example, it may be obtained based on a detection result using the left front camera and the right front camera. For example, the distance to an object located in an area where the viewing angle of the left front camera 713 and the viewing angle of the right front camera 711 of the vehicle overlaps is an offset between the center of the left front camera 713 and the center of the right front camera 711 . can be obtained based on The offset may mean a distance between the center of the left front camera 713 and the center of the right front camera 711 .

If there is an object having a risk of collision, the vehicle may determine a degree of risk of collision in operation 1403 . According to an embodiment, the collision risk may be determined based on at least one of a distance between the own vehicle and the object, the speed of the own vehicle, the speed of the object, the traveling path (or movement direction) of the own vehicle, and the movement direction of the object. The collision risk may be expressed, for example, in stages or numerically. For example, as shown in FIG. 15 , the processor 910 determines a collision monitoring boundary 1511 and a collision danger boundary 1513 of the own vehicle 1501 , and an object ( 1503), the collision risk of each object may be determined based on the collision monitoring boundary 1521 and the collision danger boundary 1523 . According to an embodiment, the collision monitoring boundary lines 1511 and 1521 and the collision risk boundary lines 1513 and 1523 of the vehicle and/or object may be determined based on the speed of the vehicle and/or the object. For example, when the speed of the vehicle is Nkph, the collision monitoring boundary line of the vehicle may be determined as a circle having a radius of 0.66*Nm from the vehicle, and the collision risk boundary line of the vehicle may be determined as a circle having a radius of 0.33*Nm from the vehicle. Here, the numerical values of 0.66 and 0.33 are values obtained through experiments, and may be changed by a designer and/or a business operator.

According to an embodiment, the processor 910 may further determine at least one of a collision monitoring range of the own vehicle, a collision risk range of an object, and a collision monitoring range of an object. The processor 910 may determine a region corresponding to the viewing angle of at least one sensor of the vehicle among regions within the collision monitoring boundary line 1511 of the own vehicle 1501 as a collision monitoring rage. The processor 910 operates the object based on the moving direction of the object, the azimuth (θ1) of the object with respect to the moving direction of the vehicle, and the angle (θ2) between the line of sight connecting the headlamp of the vehicle and the object. At least one of a collision risk range of , or a collision monitoring range of an object may be determined. For example, the collision monitoring range of the object may be determined as an area corresponding to a plane perpendicular to the moving direction of the object, among areas within the collision monitoring boundary of the object. For example, the collision monitoring range of the object may be determined as an area between a -90 degree direction and a +90 degree direction based on the moving direction of the object, among areas within the collision monitoring boundary of the object. The collision risk range of the object may be determined as at least a partial area of the collision monitoring range of the object. For example, a first angle (ratio*θ2) that is a product of a ratio determined according to the relative distance between a vehicle and an object and an angle (θ2) between the vehicle and the object based on the moving direction of the object among the areas within the collision monitoring boundary of the object ) may be determined as an area between a direction (heading-ratio*θ2) minus a direction (heading-ratio*θ2) plus a first angle (heading+ratio*θ2). The ratio may be determined based on the relative distance between the vehicle and the object and the amount of change in the relative distance.

According to an embodiment, the vehicle 1501 includes the boundary lines 1511 and 1513 of the own vehicle 1501 and/or the collision monitoring range and the boundary lines 1521 and 1523 of the object 1503, the collision monitoring range, and/or the Alternatively, the collision risk may be determined based on whether the collision risk ranges overlap or contact each other. When the collision monitoring boundary line 1511 of the own vehicle 1501 and the collision monitoring boundary line 1521 of the object 1503 do not contact or overlap, the processor 910 may determine the collision risk as the lowest level. The processor 910 is configured such that the collision monitoring boundary line 1511 of the own vehicle 1501 and the collision monitoring boundary line 1521 of the object 1503 are in contact, and the collision monitoring range of the own vehicle 1501 is in contact with or overlaps the collision monitoring range of the object In this case, the collision risk may be determined to be low. When the collision monitoring range of the own vehicle 1501 and the collision risk range of the object 1503 contact or overlap, the processor 910 may determine the collision risk as Medium. When the collision risk boundary line 1513 of the own vehicle 1501 and the collision danger boundary line 1523 of the object 1503 contact or overlap, the processor 910 may determine the collision risk to be high. According to an embodiment, when the collision risk between the own vehicle 1501 and the object 1503 corresponds to any one of high, medium, and low, the processor 910 may list a collision object to continuously monitor the object 1503 . can be added to The collision object list may include a collision risk of at least one object, and a tracking number. The tracking number may be a number assigned by the vehicle's processor 910 to identify the at least one object. In the above description, the method of determining the risk of collision is only for helping understanding, and various embodiments of the present disclosure are not limited thereto.

In operation 1405 , the vehicle may determine a sensor operation mode based on the risk of collision. According to an embodiment, the processor 910 of the vehicle may determine the sensor mode as the normal mode when the collision risk of at least one surrounding object is lower than the reference collision risk (eg, medium). According to an embodiment, the processor 910 of the vehicle may determine the sensor mode as the monitoring mode when the collision risk of at least one surrounding object is equal to or higher than the reference collision risk (eg, medium). The monitoring mode may detect at least one surrounding object more frequently and more precisely than the normal mode. For example, when the sensor operation mode is the normal mode, the frame rate of the at least one side camera (731, 733) is set to 30 Hz, when the sensor operation mode is the monitoring mode, the at least one side camera (731, 733) ) may be set to 40 Hz. As another example, when the sensor operation mode is the normal mode, the scanning resolution of at least one lidar 721 and 723 for detecting the lateral region is set to 0.1deg/1point, and when the sensor operation mode is the monitoring mode, at least one The scanning resolution of the lidar of can be set to 0.2deg/1point.

In operation 1407 , the vehicle may control the operation of the at least one side sensor according to the determined sensor operation mode. For example, when at least one object whose collision risk is determined to be medium or high exists in the vicinity of the own vehicle, the processor 910 may be configured to operate at least one side camera and/or at least one lidar in a monitoring mode. can be controlled

In operation 1409 , the vehicle may detect a possible collision object using a motion-controlled lateral sensor. For example, the processor 910 may detect at least one object located around the vehicle using at least one side camera and/or at least one lidar operating in a monitoring mode. The object around the vehicle may include at least one object whose collision risk is determined to be medium or high. Additionally, the objects around the vehicle may further include at least one object having the lowest or lowest collision risk. In this case, the collision risk of at least one object may be changed based on the object detection result. According to an embodiment, the processor 910 generates a route alarm or controls an autonomous emergency braking (AEB) system when the collision risk of the first object is changed from medium to high as a result of object detection. can stop

If there is no object with a risk of collision, the vehicle may determine the sensor operation mode as a normal mode in operation 1411 . For example, when the moving object does not exist within a threshold distance from the own vehicle, the processor 10 may determine that an object with a risk of collision does not exist, and determine the sensor operation mode as a normal mode.

In operation 1413 , the vehicle may control the operation of the at least one lateral sensor according to the normal mode. According to an embodiment, the processor 910 may control at least one side camera and/or at least one lidar to operate in a normal mode when there is no object with a risk of collision in the vicinity of the own vehicle. . According to an embodiment, when there is no object having a risk of collision in the vicinity of the own vehicle, the processor 910 may deactivate at least one lidar 721 and 723 . For example, when there is no object having a risk of collision near the own vehicle, the processor 910 may operate in a mode of detecting a surrounding object using only at least one front camera and at least one side camera.

14 and 15, the vehicle periodically determines the collision risk of the surrounding object and controls at least one sensor based on the collision risk, while preventing collision with the surrounding object. there is. For example, as shown in FIG. 16 , the vehicle 1501 at the first time point 1611 lowers the collision risk of the surrounding first object 1503 and the second object 1505 respectively (low), and Mid. For example, in the vehicle 1501 , the collision monitoring range of the vehicle 1501 and the collision monitoring range of the first object 1503 overlap 1601 , but the collision monitoring range of the vehicle 1501 and the first object 1503 collide Recognizing that the danger ranges do not overlap, the collision risk of the first object 1503 may be determined to be low. The vehicle 1501 recognizes that the collision monitoring range of the vehicle 1501 and the collision risk range of the second object 1505 overlap 1603, and determines the collision risk of the second object 1505 as Mid. there is. The vehicle 1501 adds the first object 1503 and the second object 1505 to the collision object list, and the operation mode of the first lidar 721 detects the right side in which the second object 1505 is detected. can be switched from normal mode to monitoring mode. At a second time point 1612 , the vehicle 1501 may determine the collision risk of the first object 1503 as the lowest and change the collision risk of the second object 1505 to high. For example, in the vehicle 1501 , the collision monitoring range of the vehicle 1501 and the collision monitoring range and the collision risk range of the first object 1503 do not overlap, but are within the detectable area 1621 by the right lateral sensor, so that the first The collision risk of the object 1503 may be determined to be the lowest. Since the vehicle 1501 is in contact with the collision risk boundary line 1513 of the vehicle 1501 and the collision danger boundary line 1527 of the second object 1505, it is determined that the collision risk of the second object 1505 is high. can At the third time point 1613 , the vehicle 1501 deletes the first object 1503 in the vicinity from the collision object list, maintains the collision risk of the second object 1505 at high, and provides a warning alarm and stops. action can be performed. For example, the vehicle 1501 may recognize that the first object 1503 is out of the detectable area 1621 by the right lateral sensor, and delete the first object 1503 from the collision object list. Since the vehicle 1501 overlaps the collision risk boundary line 1513 of the vehicle 1501 and the collision risk range of the second object 1505 1626 , the collision risk of the second object 1505 is maintained as high. While doing so, a route alarm can be generated or the vehicle can be stopped by controlling the automatic emergency braking system (AEB).

17 is a flowchart of controlling the driving of the own vehicle based on the type and/or collision risk of another vehicle in the driving path in the vehicle according to various embodiments of the present disclosure; At least some operations of FIG. 17 may be detailed operations of operation 1211 of FIG. 12 . In the following embodiment, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Here, the vehicle may include the electronic device 900 of FIG. 9 . Hereinafter, at least some operations of FIG. 17 will be described with reference to FIG. 18 . 18 is an exemplary diagram of controlling the driving of the own vehicle based on the type and/or collision risk of another vehicle in the driving path in the vehicle according to various embodiments of the present disclosure;

Referring to FIG. 17 , in operation 1701 , the vehicle may determine whether another vehicle in the avoidance path is detected. The avoidance path may include the driving path changed in operation 1211 of FIG. 12 . For example, the processor 910 of the vehicle may detect whether at least one other vehicle exists on the driving path changed in operation 1211 of FIG. 12 using at least one sensor. For example, the processor 910 of the vehicle determines whether there is at least one other vehicle having a path overlapping with at least some of the driving path L2 or L3 changed in operation 1211 of FIG. 12 using at least one sensor. can detect

If no other vehicle in the avoidance route is detected, the vehicle may travel on the avoidance route in operation 1711 .

When another vehicle in the avoidance path is detected, the vehicle may determine whether the type of the other vehicle is the designated first type in operation 1703 . For example, as shown in FIG. 18 , when the third vehicle 1823 expected to pass through the avoidance path L2 or at least a part of the path L3 is detected, based on the information of the third vehicle 1823 It may be determined whether the vehicle type of the third vehicle 1823 is the designated first type. For example, the processor 910 of the vehicle obtains wheel size, body height, or body length information of the third vehicle 1823 using the right side camera 731 and/or the first lidar 721 , The type of the third vehicle 1823 may be determined based on the obtained information. According to an embodiment, the designated first type may be a type indicating at least one of a large vehicle, a truck, a heavy equipment vehicle, a fire engine, and an ambulance.

When the other vehicle type is the designated first type, the vehicle may travel to an avoidance path after stopping in operation 1713 . For example, when the type of the third vehicle 1823 as shown in FIG. 18 is a large car, the processor 910 stops until the third vehicle 1823 passes, and then the avoidance path L2 1812, Alternatively, it may travel to L3 (1813).

If the type of the other vehicle is not the designated first type, the vehicle may determine a collision risk for the other vehicle in operation 1705 . For example, when the type of the third vehicle 1823 as shown in FIG. 18 is a small car or a medium-sized car, the processor 910 may determine the risk of collision of the third vehicle 1823 with respect to the own vehicle 1851 . there is. The collision risk may be determined in the same manner as described with reference to FIG. 14 .

In operation 1707 , the vehicle may determine whether a risk of collision for another vehicle is greater than a specified threshold risk. The specified threshold risk may be low in collision risk. This is merely an example, and various embodiments of the present disclosure are not limited thereto.

When the risk of collision with respect to another vehicle is greater than the specified threshold risk, the vehicle may travel to an avoidance path after stopping in operation 1709 . For example, the processor 910 may determine that, as shown in FIG. 18 , when the collision risk of the third vehicle 1823 is high or medium, the third vehicle 1823 selects an avoidance path. The host vehicle 1851 is controlled to stop until it passes, and after the third vehicle 1823 passes the avoidance route, it may travel on the avoidance route L2 1812 or L3 1813 .

If the risk of collision for another vehicle is less than or equal to the specified threshold risk, the vehicle may travel at a low speed on the avoidance path in operation 1715 . For example, the processor 910 sets the speed of the vehicle below the threshold speed when the collision risk of the third vehicle 1823 is low or the lowest, as shown in FIG. 18 . It may travel on the avoidance path L2 ( 1812 ) or L3 ( 1813 ).

19 is a flowchart of obtaining an intention for a driving state of a vehicle in front by using a side camera in a vehicle according to various embodiments of the present disclosure; At least some operations of FIG. 19 may be detailed operations of operation 1107 of FIG. 11 or detailed operations of operation 1203 of FIG. 12 . In the following embodiment, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Here, the vehicle may include the electronic device 900 of FIG. 9 . Hereinafter, at least some operations of FIG. 19 will be described with reference to FIG. 20 . 20 is an exemplary diagram of analyzing an intention for a driving state of a front vehicle by using a side camera in a vehicle according to various embodiments of the present disclosure;

Referring to FIG. 19 , in operation 1901 , the vehicle may detect a front area of the first front vehicle using the front camera. The first front vehicle may be the first front vehicle of FIGS. 11 and/or 12 . According to an embodiment, the driving state of the first front vehicle may be a stopped state.

In operation 1903 , the vehicle may determine whether an intention for the driving state of the first front vehicle is obtained. The processor 910 of the vehicle may analyze an image obtained through the front camera to determine whether information indicating a cause and/or an intention for the driving state of the first front vehicle is obtained. According to an embodiment, the information indicating the cause and/or intention of the driving state of the first front vehicle may include rear lamp information of the second front vehicle located in a front region of the first front vehicle, information on whether wheels are rotated, or It may include rearview mirror status information. According to an embodiment, the information indicating the cause and/or intention of the driving state of the first front vehicle includes at least one of information about a pedestrian located in a front area of the first front vehicle, or information about a structure can do. For example, as shown in FIG. 19 , the processor 910 of the vehicle uses the left front camera 2002 and the right front camera image 2003 at a first time point 2011 to display the first front vehicle in a stopped state ( 2022) and the second front vehicle 2021 are detected, rear lamp information of the second front vehicle 2021 that can determine the stopping intention of the first front vehicle 2022 and the second front vehicle 2021 is acquired You can decide whether or not

If the intention for the driving state of the first front vehicle is not obtained, the vehicle may detect a front area of the first front vehicle using the side camera in operation 1905 . For example, as shown in FIG. 19 , the processor 910 of the vehicle transmits the image of the second front vehicle 2021 through the left front camera 2002 and the right front camera image 2003 at a first time point 2011 . When the rear lamp information for identifying the stopping intention is not obtained, the left front camera image 2001 and the right side camera image in addition to the left front camera 2002 and the right front camera image 2003 at the second time point 2011 The image 2004 may be additionally analyzed to detect a second front vehicle located in an area in front of the first front vehicle.

In operation 1907 , the vehicle may determine whether an intention for the driving state of the first front vehicle is obtained through the side camera. For example, the processor 910 of the vehicle analyzes an image obtained through at least one of the left side camera and the right side camera, so that information indicating the cause and/or intention of the driving state of the first front vehicle is provided. It can be determined whether or not

When the intention for the driving state of the first front vehicle is not obtained through the side camera, the vehicle may move the location of the vehicle in operation 1909, return to operation 1905, and perform the following operations again. For example, as shown in FIG. 19 , the processor 910 of the vehicle performs the second front vehicle 2021 through the left side camera image 2001 and the right side camera image 2004 at the second time point 2011 . When rear lamp information that can determine the stopping intention of the vehicle is not obtained, the processor 910 of the vehicle may move the location of the vehicle. The location movement of the vehicle may include at least one of changing a lane, changing a location within a lane, or changing a location through driving according to a driving route. As shown in FIG. 19 , the processor 910 of the vehicle acquires images 2001 and 2004 in which the area of the blind spot is captured through the left lateral camera and the right lateral camera at a third time point 2011 after the position is moved. By doing so, the stopping intention of the second front vehicle 2021 can be grasped. For example, the processor 910 may analyze the right side camera image 2004 acquired at the third time point, and recognize that the brake lamp in the rear lamp is on and there is a pedestrian. The processor 910 may determine that the cause and/or intention of the second front vehicle 2021 is to wait for the pedestrian to board. According to another embodiment, when the brake lamp in the rear lamp is turned off and the turn indicator is turned on in the right side camera image 2004 acquired at the third time point, the processor 910 stops the second front vehicle 2021 The cause, and/or intent, may be determined as a lane change.

21 is a flowchart of controlling a setting of a lidar based on information on a surrounding object in a vehicle according to various embodiments of the present disclosure; At least some operations of FIG. 21 may be detailed operations of operation 1105 of FIG. 11 . In the following embodiment, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Here, the vehicle may include the electronic device 900 of FIG. 9 . Hereinafter, at least some operations of FIG. 21 will be described with reference to FIGS. 22 to 25 . 22 is an exemplary diagram of obtaining information on a surrounding object using a left lidar and a right lidar in a vehicle according to various embodiments of the present disclosure, and FIG. This is an example of controlling the output intensity of each laser point of the lidar. 24 is an exemplary diagram of controlling a laser irradiation period of a lidar based on a distance from a surrounding object in a vehicle according to various embodiments, and FIG. It is an exemplary diagram for determining the laser irradiation point.

Referring to FIG. 21 , in operation 2101, the vehicle may irradiate a laser based on designated setting information of the lidar. According to various embodiments, the processor 910 and/or the sensor module 920 of the vehicle activates at least one lidar, and controls the activated at least one lidar to irradiate a laser according to specified setting information. can do. The setting information may include at least one of a laser output intensity, a laser irradiation range, or a laser irradiation period. The laser irradiation range may include at least one of a horizontal irradiation range and a vertical irradiation range. According to an embodiment, the laser output intensity of at least one lidar may be set to a maximum value, and the laser irradiation range may be set to all points among laser point clouds. According to an embodiment, the at least one laser irradiation period may be set as the first period. According to an embodiment, the processor 910 and/or the sensor module 920 may activate or deactivate at least one lidar based on time information and/or ambient brightness information. According to an embodiment, the processor 910 and/or the sensor module 920 may activate or deactivate at least one lidar among a plurality of lidars based on driving information (eg, rotation information) of the vehicle. can be controlled For example, when the vehicle turns left and/or turns right, the processor 910 and/or the sensor module 920 activates both the first lidar 711 and the second lidar 713 to designate It can be controlled to irradiate the laser according to the setting information. As another example, the processor 910 and/or the sensor module 920 activates any one of the first lidar 711 and the second lidar 713 based on lane information when the vehicle travels straight. It can be controlled to irradiate the laser according to the specified setting information. This is illustrative, and various embodiments of the present disclosure are not limited thereto.

In operation 2103, the vehicle may receive the reflected laser to obtain information about the object. For example, the processor 910 and/or the sensor module 920 of the vehicle receives a laser reflected by the object after being irradiated from at least one lidar, and information about the object based on the received laser can be obtained. The information about the object may include a distance between the vehicle and the object, the position of the object, the size of the object, or information on whether the object is moved. According to an embodiment, the processor 910 and/or the sensor module 920 periodically performs a laser transmission/reception operation of receiving a reflected laser after irradiating a laser using at least one lidar according to a laser irradiation cycle. can be done The processor 910 and/or the sensor module 920 may acquire information on whether an object moves or not based on a result of periodic transmission and reception of the laser.

In operation 2105, the vehicle may change setting information of the lidar based on the information on the object. According to an embodiment, the processor 910 of the vehicle, and/or the sensor module 920, based on the information on the object of the vehicle, the laser output intensity of at least one lidar for the object, the laser irradiation range, or At least one of the laser irradiation periods may be changed. For example, as shown in FIG. 22 , the vehicle 2201 detects pedestrian groups 1 2221 and 2 2222 in the right lateral direction through the first lidar 711 and 2212, and the second radar It is possible to detect another vehicle 1 2202 in the left lateral direction through the IDAs 713 and 2211 . The vehicle 2201 detects that the distance between the vehicle 2201 and the pedestrian group 2 2222 is greater than the distance between the vehicle 2201 and the other vehicle 1 2202, and detects the pedestrian group 2 2222. The laser output intensity of the first lidar (711, 2212) may be set to be greater than the laser output intensity of the second lidar (713, 2211). At this time, since the other vehicle 2202 moves at a faster speed than the pedestrian groups 1 2221 and 2 2222, the laser irradiation period of the second lidar 713 and 2211 is changed to that of the first lidar 711 and 2212. It can be set shorter than the laser irradiation period.

According to an embodiment, the processor 910 of the vehicle, and/or the sensor module 920 allocates different points of the lidar to each detected object, and provides a different laser output intensity for each detected object, and/or Alternatively, the laser irradiation cycle may be set. For example, as shown in FIG. 23 , vehicle 2301 allocates first laser points to another vehicle 1 2311 , allocates second laser points to another vehicle 2 2312 , and the first laser The laser output intensity of the points may be set to be greater than the output intensity of the second laser points. The vehicle 2301 detects another vehicle 1 2311 at a long distance by irradiating a laser 2331 using the first laser points, and irradiating a laser using the second laser points, thereby irradiating a laser beam using the second laser points. ) can be detected.

According to an embodiment, the processor 910 and/or the sensor module 920 of the vehicle may change the laser irradiation period of at least one lidar according to a change in the distance between the own vehicle and the object. For example, as shown in FIG. 24 , when the distance from the vehicle 2401 to the other vehicle 2402 is equal to or greater than the threshold distance, the laser irradiation period of the second lidars 713 and 2411 is set to 100 msec, When the distance to the other vehicle 2402 is within the threshold distance due to the right turn operation of the vehicle 2401 and the straight driving of the other vehicle 2402, the laser irradiation period of the second lidars 713 and 2411 is set to 10 msec. can According to an embodiment, the processor 910 and/or the sensor module 920 of the vehicle may control the irradiation period of the lidar to be inversely proportional to the distance from the object located in the corresponding direction.

According to an embodiment, the processor 910 and/or the sensor module 920 of the vehicle selects some laser points irradiating a laser to a portion corresponding to the characteristic of the object based on the object detection result, and selects the selected portion Only laser points can detect the object. When the detected object is a vehicle, a portion corresponding to the characteristic of the object may include, for example, at least one of a wheel, a body, and a windshield of the vehicle. When the detected object is a pedestrian, a portion corresponding to the characteristic of the object may include, for example, at least one of a head, an arm, a leg, and a torso of the pedestrian.

According to an embodiment, the processor 910 and/or the sensor module 920 of the vehicle classifies each laser point into a plurality of groups based on the laser point cloud obtained by the object detection result, and each group The object may be detected by selecting at least one representative laser point and controlling the laser to be irradiated using only the representative laser points selected for each group. The processor 910 and/or the sensor module 920 of the vehicle may save energy by controlling the laser not to be irradiated to unselected laser points. For example, as shown in Fig. 25, the vehicle divides the laser points obtained as a result of pedestrian detection into a plurality of groups 2501 to 2509 based on the characteristics of the pedestrian, sets a representative point of each group, and , it is possible to detect a pedestrian using only the set representative points.

According to an embodiment, the processor 910 and/or the sensor module 920 may change the setting information of the plurality of sensors by further considering the data transmission capacity inside the vehicle in addition to the detection result of the object. For example, the processor 910 and/or the sensor module 920 may determine that the amount of object detection result data obtained from a plurality of lidars during a specified time period is less than or equal to the maximum data capacity that can be processed in the network inside the vehicle. to adjust the laser irradiation period, and/or the laser irradiation range of the plurality of lidar.

A vehicle according to various embodiments of the present disclosure may save energy by changing the laser setting information as described above.

26 is a flowchart of switching a use mode of a camera and a lidar in a vehicle according to various embodiments of the present disclosure; In the following embodiment, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Here, the vehicle may include the electronic device 900 of FIG. 9 .

Referring to FIG. 26 , in operation 2601 , the vehicle may determine whether the vehicle is driving in a section satisfying a specified condition. For example, the processor 910 and/or the sensor module 920 of the vehicle may determine whether the vehicle enters a highway section based on map data stored in the memory 940 and current location information. When entering a highway section, the processor 910 and/or the sensor module 920 uses at least one camera and/or at least one lidar to move the vehicle to a specific terrain (eg, a guard rail). You can determine whether you are driving adjacently. The processor 910 and/or the sensor module 920 of the vehicle may determine that the vehicle is traveling in a section satisfying a specified condition when the vehicle is traveling adjacent to a specific feature within a highway.

When the vehicle is traveling in a section that satisfies the specified condition, the vehicle may switch to the lidar use mode in operation 2603 . For example, the processor 910 and/or the sensor module 920 of the vehicle may deactivate at least one camera when the vehicle is driving adjacent to a specific feature within a highway, thereby using only the lidar to access the object. can be switched to the detection mode. The processor 910 and/or the sensor module 920 of the vehicle may deactivate only the cameras in a direction adjacent to a specific feature, and may deactivate all cameras. This is because, when the vehicle is driving adjacent to a specific feature, at least one of the location, size, and shape of the specific feature can be determined using only the side lidar. Accordingly, in the embodiment of the present disclosure, when the vehicle is traveling in a section that satisfies a specified condition, energy can be saved by inactivating the camera and detecting the object using only the lidar.

In operation 2605 , the vehicle may acquire information of a surrounding topographical feature using lidar. For example, the processor 910 and/or the sensor module 920 of the vehicle irradiate a laser using a lidar in a direction in which the guardrail is located, thereby determining at least one of the position, size, and shape of the guardrail. information can be obtained. According to an embodiment, the processor 910 and/or the sensor module 920 of the vehicle may change the laser irradiation range based on information on a specific feature. For example, the processor 910 of the vehicle and/or the sensor module 920 extracts the coordinate values of the height of the guardrail, the boundary of the guardrail, or the pole of the guardrail using the laser point cloud of the lidar. In addition, it is possible to detect a fixed part of a guard rail that is constantly repeated (eg, a guard rail column, a junction between a guard rail and a road). The processor 910 and/or the sensor module 920 of the vehicle may change the laser irradiation range so that the laser is irradiated only to a fixed portion of the detected guard rail. The processor 910 and/or the sensor module 920 may save energy by turning off power of laser points that are not irradiated with a laser or switching to a sleep mode. According to an embodiment, the processor 910 and/or the sensor module 920 of the vehicle may use at least one other lidar to control an object (eg, another vehicle, pedestrian, or other terrain) located in the opposite direction of a specific terrain. features) can be detected.

In operation 2607, the vehicle may perform autonomous driving based on the acquired information of the surrounding terrain features. For example, the processor 910 and/or the sensor module 920 of the vehicle may analyze the shape of a road based on the information of the guard rail and perform autonomous driving according to the analyzed road shape. As another example, the processor 910 and/or the sensor module 920 of the vehicle may perform autonomous driving adjacent to the guard rail based on location information of the guard rail. According to an embodiment, when the completion of driving of a section satisfying a specified condition is predicted, the processor 910 of the vehicle and/or the sensor module 920 activates at least one camera again and performs operation 2601. can For example, the processor 910 and/or the sensor module 920 of the vehicle predicts the end of the guardrail section in advance based on information obtained using the lidar, and the vehicle does not have a guardrail. At least one camera can be re-activated before entering the section. The activated at least one camera may detect a lane ahead and surrounding objects.

When the vehicle is not traveling in a section that satisfies a specified condition, the vehicle may detect an object using a lidar and a camera in operation 2609 . For example, the processor 910 and/or the sensor module 920 of the vehicle may at least Objects can be detected using both a single camera and lidar.

In operation 2611 , the vehicle may determine whether an object having a risk of collision exists. Whether an object with a risk of collision exists may be determined by the operation described in operation 1401 of FIG. 14 .

If there is no object with a risk of collision, the vehicle may control the operation of the lidar in operation 2613 . According to an embodiment, the processor 910 and/or the sensor module 920 of the vehicle may deactivate at least one lidar. The deactivation of the lidar may include at least one of stopping power supply to the lidar, a sleep mode of the lidar, and a low power mode of the lidar. According to an embodiment, the processor 910 of the vehicle and/or the sensor module 920 may control at least one of a laser irradiation period of at least one lidar, a laser output intensity, and a density of a laser irradiation point. For example, the processor 910 and/or the sensor module 920 of the vehicle may reduce the density of the laser irradiation points by widening the horizontal and/or vertical spacing between the laser irradiation points, thereby saving energy. can do. According to an embodiment, the processor 910 and/or the sensor module 920 of the vehicle may re-activate the deactivated lidar of the own vehicle when at least one other vehicle entering the movement path of the vehicle is detected. there is. According to an embodiment, when at least one other vehicle entering the movement path of the own vehicle is detected, the processor 910 and/or the sensor module 920 of the vehicle detects a laser irradiation period of the own vehicle's lidar, a laser At least one of the output intensity and the density of the laser irradiation point may be changed to be the same as the setting information of the initial operation time of the lidar. For example, the processor 910 and/or the sensor module 920 of the vehicle may set the setting information of the lidar as in operation 2101 of FIG. 21 .

If there is an object having a risk of collision, the vehicle may proceed to operation 1201 of FIG. 12 .

In the above description, when there is no object with the possibility of collision, at least one lidar is controlled to be deactivated (or changed to an off state).

According to various embodiments of the present disclosure, based on whether the information on the expected driving route obtained based on the map data and the surrounding environment information obtained through the vehicle's camera are the same, You can control the active state. For example, the vehicle may control to activate at least one inactivated lidar when the information on the expected driving route obtained based on the map data and the surrounding environment information obtained through the vehicle's camera are different from each other. As another example, when the information on the expected driving route obtained based on the map data and the surrounding environment information obtained through the vehicle's camera are the same, the vehicle may control at least one activated lidar to be deactivated.

According to various embodiments of the present disclosure, the vehicle may control the active state of at least one lidar based on whether a specific driving operation is performed. For example, when driving using a detour is detected by changing a route on a highway, the vehicle may activate at least one lidar. The activated lidar may detect a surrounding object by irradiating a laser based on the maximum output intensity, the shortest period, and/or all irradiation ranges. When the driving of the detour route is completed, the vehicle may determine whether to deactivate at least one lidar using the methods described above.

According to various embodiments of the present disclosure, based on at least one of a distance to at least one other vehicle, a speed difference between the vehicle and the other vehicle, or a driving path between the vehicle and the other vehicle, the vehicle You can control the active state.

According to various embodiments of the present disclosure, a method of operating an electronic device 900 included in a vehicle includes an operation of determining a driving state of a first front vehicle located in a first area using at least one front sensor; When the driving state of the vehicle corresponds to the specified driving state, an operation of acquiring information on at least one object located in the second area using at least one lateral sensor, and the information on the at least one object and the and controlling the driving of the vehicle based on the driving state information of the first front vehicle, wherein at least a portion of the second region may include a region that does not overlap the first region.

According to an embodiment, the designated driving state may include at least one of a stopped state and a state of traveling at a speed less than or equal to a threshold speed.

According to an embodiment, the front sensor may include a camera that has a viewing angle covering the first area and is disposed in front of the vehicle.

According to an embodiment, the side sensor may include at least one of a camera and a lidar that has a viewing angle covering the second area and is disposed on the side of the vehicle.

According to an embodiment, the information on the at least one object may include information on at least one of at least one other vehicle, a pedestrian, or a structure.

According to an embodiment, the information on the at least one other vehicle may include: wheel rotation speed, wheel rotation angle, wheel size, movement speed, movement direction, rear lamp state, rearview mirror state, vehicle body height, vehicle body length, vehicle type, Or it may include at least one of the distance to the own vehicle.

According to an embodiment, the information on the at least one pedestrian may include at least one of a pedestrian location, a pedestrian size, a pedestrian movement speed, or whether the pedestrian moves.

According to an embodiment, the information on the at least one structure may include at least one of a location of a structure, a size of a structure, a type of a traffic light, or a signal state of a traffic light.

According to an embodiment, the operation of controlling the driving of the vehicle may include predicting the driving motion of the first front vehicle based on the information on the at least one object, and the predicted driving motion of the first front vehicle. It may include an operation of controlling the driving of the vehicle based on the driving operation.

According to an embodiment, predicting the driving motion of the first front vehicle may include predicting the stopping time of the first front vehicle based on information on the at least one object.

According to an embodiment, the operation of controlling the driving of the vehicle based on the predicted driving motion of the first front vehicle may include performing any one of stopping or changing a driving route based on the predicted stopping time. It can include actions.

According to an embodiment, the operation of performing any one of stopping or changing the driving route may include, when the predicted stopping time is shorter than a specified threshold time, maintaining the driving route of the vehicle while maintaining the driving route of the vehicle during the predicted stopping time. controlling the vehicle to stop, and when the predicted stopping time is longer than the specified threshold time, changing the driving route of the vehicle based on the positions of the first front vehicle and the at least one object can

According to an embodiment, the operation of determining the collision risk for at least one of the first front vehicle or the at least one object, and the operation of determining the detection mode of the at least one side sensor based on the collision risk level may further include.

According to an embodiment, based on the detection mode of the side sensor, at least one of a frame rate of at least one side camera and a scanning resolution of at least one side lidar may be determined.

According to an embodiment, when the determined collision risk is lower than a threshold risk, the operation of deactivating the at least one lidar may be further included.

According to an embodiment, the obtaining of information on the at least one object located in the second area includes using at least one side camera to obtain information about the at least one object located in the second area while being located in the second area in the front area of the first front vehicle. An operation of detecting the located second front vehicle, an operation of determining whether driving-related information of the second forward vehicle is obtained using the at least one side camera, and an operation of determining whether driving-related information of the second forward vehicle is not obtained In this case, the method may include an operation of moving so that the position of the vehicle is changed, and an operation of acquiring driving-related information of the second front vehicle using the at least one side camera at the changed position.

According to an embodiment, the acquiring information on the at least one object located in the second area may include, based on at least one of location, size, and distance information of the at least one object, the at least one side It may include an operation of controlling at least one of a shooting range of the camera and a sampling operation.

According to an embodiment, the obtaining of information on the at least one object located in the second area may include, based on at least one of location, size, and distance information of the at least one object, the at least one It may include the operation of controlling at least one of the laser irradiation range, the laser irradiation period, and the laser output intensity of the da.

According to an embodiment, the at least one lateral sensor includes at least one of a left lidar and a right lidar, and a laser irradiation period of at least one of the left lidar and the right lidar is, may be determined based on the data transmission capacity of

According to one embodiment, the laser irradiation period of the left lidar and the laser irradiation period of the right lidar, the left lidar and the right lidar may be determined to irradiate the laser at different times.

Claims (20)

A method of operating an electronic device included in a vehicle, the method comprising:
determining a driving state of the first front vehicle located in the first area using at least one front sensor;
obtaining information on at least one object located in a second area using at least one side sensor when the driving state of the first front vehicle corresponds to a specified driving state; and
and controlling the driving of the vehicle based on the information on the at least one object and the driving state information of the first front vehicle,
At least a partial region of the second region includes a region that does not overlap the first region.
According to claim 1,
The specified driving state includes at least one of a stopped state and a state of traveling at a speed less than or equal to a threshold speed.
According to claim 1,
The front sensor includes a camera disposed in front of the vehicle and having a viewing angle covering the first area,
The lateral sensor includes at least one of a camera and a lidar disposed on a side of the vehicle and having a viewing angle covering the second area.
According to claim 1,
The information on the at least one object includes information on at least one of at least one other vehicle, a pedestrian, or a structure,
The information on the at least one other vehicle may include one of a wheel rotation speed, a wheel rotation angle, a wheel size, a moving speed, a moving direction, a rear lamp state, a rearview mirror state, a body height, a body length, a vehicle type, or a distance to the own vehicle. contains at least one,
The information on the at least one pedestrian includes at least one of a pedestrian location, a pedestrian size, a pedestrian movement speed, or whether a pedestrian is moving,
The information on the at least one structure includes at least one of a location of a structure, a size of a structure, a type of a traffic light, or a signal state of a traffic light.
According to claim 1,
The operation of controlling the driving of the vehicle,
predicting a driving motion of the first front vehicle based on the information on the at least one object; and
and controlling the driving of the vehicle based on the predicted driving motion of the first forward vehicle.
6. The method of claim 5,
The operation of predicting the driving motion of the first front vehicle includes:
and predicting a stop time of the first front vehicle based on the information on the at least one object.
7. The method of claim 6,
The operation of controlling the driving of the vehicle based on the predicted driving motion of the first forward vehicle includes:
and performing any one of stopping or changing a driving route based on the predicted stopping time.
8. The method of claim 7,
The operation of performing any one of the stopping or changing the driving route includes:
when the predicted stop time is shorter than a specified threshold time, controlling the vehicle to stop during the predicted stop time while maintaining the driving route of the vehicle; and
and changing the driving route of the vehicle based on the positions of the first front vehicle and the at least one object when the predicted stopping time is longer than the specified threshold time.
According to claim 1,
determining a collision risk for at least one of the first front vehicle or the at least one object;
Based on the collision risk, the method further comprises determining a detection mode of the at least one side sensor,
At least one of a frame rate of at least one side camera or a scanning resolution of at least one side lidar is determined based on a detection mode of the side sensor.
10. The method of claim 9,
When the determined collision risk is lower than a threshold risk, the method further comprising the operation of deactivating the at least one lidar.
According to claim 1,
The operation of obtaining information on at least one object located in the second area includes:
detecting a second front vehicle located in the second area and located in a front area of the first front vehicle using at least one side camera;
determining whether driving-related information of the second front vehicle is obtained using the at least one side camera;
moving to change the location of the vehicle when driving-related information of the second front vehicle is not obtained;
and acquiring driving-related information of the second front vehicle using the at least one side camera at the changed position.
According to claim 1,
The operation of obtaining information on at least one object located in the second area includes:
and controlling at least one of a shooting range of the at least one side camera and a sampling operation based on at least one of location, size, and distance information of the at least one object.
According to claim 1,
The operation of obtaining information on at least one object located in the second area includes:
A method comprising controlling at least one of a laser irradiation range, a laser irradiation period, and a laser output intensity of the at least one lidar based on at least one of the position, size, and distance information of the at least one object .
According to claim 1,
The at least one lateral sensor includes at least one of a left lidar and a right lidar,
The laser irradiation period of at least one of the left lidar and the right lidar is determined based on a data transmission capacity inside the vehicle.
15. The method of claim 14,
The laser irradiation period of the left lidar and the laser irradiation period of the right lidar are determined so that the left lidar and the right lidar are irradiated with lasers at different times.
An electronic device included in a vehicle, comprising:
at least one front sensor;
at least one lateral sensor; and
A processor comprising:
determining the driving state of the first front vehicle located in the first area using the at least one front sensor,
When the driving state of the first front vehicle corresponds to the specified driving state, using at least one side sensor to obtain information on at least one object located in the second area,
based on the information on the at least one object and the driving state information of the first front vehicle, to control the driving of the vehicle,
At least a partial area of the second area includes an area that does not overlap the first area.
17. The method of claim 16,
The designated driving state includes at least one of a stopped state and a state in which the vehicle travels at a speed less than or equal to a threshold speed.
17. The method of claim 16,
The at least one front sensor includes at least one camera disposed in front of the vehicle and having a viewing angle covering the first area,
The at least one side sensor includes at least one of at least one camera and at least one lidar that has a viewing angle covering the second area and is disposed on a side of the vehicle.
17. The method of claim 16,
The information on the at least one object includes information on at least one of at least one other vehicle, a pedestrian, or a structure,
The information on the at least one other vehicle may include one of a wheel rotation speed, a wheel rotation angle, a wheel size, a moving speed, a moving direction, a rear lamp state, a rearview mirror state, a body height, a body length, a vehicle type, or a distance to the own vehicle. contains at least one,
The information on the at least one pedestrian includes at least one of a pedestrian location, a pedestrian size, a pedestrian movement speed, or whether a pedestrian is moving,
The information on the at least one structure includes at least one of a location of a structure, a size of a structure, a type of a traffic light, and a signal state of a traffic light.
17. The method of claim 16,
The processor predicts a driving motion of the first front vehicle based on the information on the at least one object,
An electronic device that controls the driving of the vehicle based on the predicted driving motion of the first forward vehicle.

Images